10-07-2012, 02:55 PM
Probability and Random Variables
Probability and Random Variables.pdf (Size: 2.21 MB / Downloads: 374)
2.1 Introduction
At the start of Sec. 1.1.2, we had indicated that one of the possible ways
of classifying the signals is: deterministic or random. By random we mean
unpredictable; that is, in the case of a random signal, we cannot with certainty
predict its future value, even if the entire past history of the signal is known. If the
signal is of the deterministic type, no such uncertainty exists.
Consider the signal x (t ) = A cos(2πf1 t + θ). If A, θ and f1 are known,
then (we are assuming them to be constants) we know the value of x (t ) for all t .
( A, θ and f1 can be calculated by observing the signal over a short period of
time).
Now, assume that x (t ) is the output of an oscillator with very poor
frequency stability and calibration. Though, it was set to produce a sinusoid of
frequency f = f1, frequency actually put out maybe f1
' where f (f f ) 1 1 1
' ∈ ± Δ .
Even this value may not remain constant and could vary with time. Then,
observing the output of such a source over a long period of time would not be of
much use in predicting the future values. We say that the source output varies in
a random manner.
Another example of a random signal is the voltage at the terminals of a
receiving antenna of a radio communication scheme. Even if the transmitted
Principles of Communication Prof. V. Venkata Rao
Indian Institute of Technology Madras
2.2
(radio) signal is from a highly stable source, the voltage at the terminals of a
receiving antenna varies in an unpredictable fashion. This is because the
conditions of propagation of the radio waves are not under our control.
But randomness is the essence of communication. Communication
theory involves the assumption that the transmitter is connected to a source,
whose output, the receiver is not able to predict with certainty. If the students
know ahead of time what is the teacher (source + transmitter) is going to say
(and what jokes he is going to crack), then there is no need for the students (the
receivers) to attend the class!
Although less obvious, it is also true that there is no communication
problem unless the transmitted signal is disturbed during propagation or
reception by unwanted (random) signals, usually termed as noise and
interference. (We shall take up the statistical characterization of noise in
Chapter 3.)
However, quite a few random signals, though their exact behavior is
unpredictable, do exhibit statistical regularity. Consider again the reception of
radio signals propagating through the atmosphere. Though it would be difficult to
know the exact value of the voltage at the terminals of the receiving antenna at
any given instant, we do find that the average values of the antenna output over
two successive one minute intervals do not differ significantly. If the conditions of
propagation do not change very much, it would be true of any two averages (over
one minute) even if they are well spaced out in time. Consider even a simpler
experiment, namely, that of tossing an unbiased coin (by a person without any
magical powers). It is true that we do not know in advance whether the outcome
on a particular toss would be a head or tail (otherwise, we stop tossing the coin
at the start of a cricket match!). But, we know for sure that in a long sequence of
tosses, about half of the outcomes would be heads (If this does not happen, we
suspect either the coin or tosser (or both!)).
Principles of Communication Prof. V. Venkata Rao
Indian Institute of Technology Madras
2.3
Statistical regularity of averages is an experimentally verifiable
phenomenon in many cases involving random quantities. Hence, we are tempted
to develop mathematical tools for the analysis and quantitative characterization
of random signals. To be able to analyze random signals, we need to understand
random variables. The resulting mathematical topics are: probability theory,
random variables and random (stochastic) processes. In this chapter, we shall
develop the probabilistic characterization of random variables. In chapter 3, we
shall extend these concepts to the characterization of random processes.
2.2 Basics of Probability
We shall introduce some of the basic concepts of probability theory by
defining some terminology relating to random experiments (i.e., experiments
whose outcomes are not predictable).
2.2.1. Terminology
Def. 2.1: Outcome
The end result of an experiment. For example, if the experiment consists
of throwing a die, the outcome would be anyone of the six faces, F1,..,F6