Download Conditional Probability

Probability
• All possible outcomes of a chance experiment
form a sample space
• A subset of the sample space is called an event
• Each event is assigned a probability, such that
(i) Pr[E] ≥ 0
(ii) Pr[Ω]=1
(iii) Pr[E1 UE2] = Pr[E1] +Pr[E2] if E1∩E2 = ø
• Union bound
Pr[E1 UE2] = Pr[E1] + Pr[E2] - Pr[E1∩E2]
≤ Pr[E1] + Pr[E2]
Conditional Probability
• Probability of an even (E) conditioned on the
occurrence of another event (M)
• Bayes rule
Pr[E | M] = Pr[ E∩M] / Pr[M]
• Chain rule
Pr[E1∩E2 ∩…∩En] = Pr[E1 | E2 ∩ …∩En] ×
Pr[E2 | E3 ∩…∩En] … × Pr[En-1 | En] Pr[En]
• Total Probability
Pr[E | M] = Pr[E | A1]P[A1] + … + Pr[E | An]P[An],
Ai ∩Aj= ø, A1UA2U…An =M
• Independence
Pr[E1|E2] = Pr[E1]
1
Random Variable
• Real function of events X(E)
• Cumulative distribution function (CDF)
FX(x) = Pr[X ≤ x]
• Probability density function (PDF) for continuous
random variable
fX(x) =dFX(x) / dx
• Probability mass function (PMF) for discrete
random variable
pX(x) = Pr[X=x]
• Mean and variation
Two random variables
• Joint CDF
FXY(x,y) = Pr[X ≤ x, Y≤ y]
• Joint PDF
fXY(x,y) =d2FXY(x,y) / dxdy
• Conditional PDF
fX|Y(x|y) = fXY(x,y) / fY(y)
• Independence
fXY(x,y) = fX(x) fY(y)
• Covariance
– CXY = 0 Æ uncorrelated
2
Gaussian Random Variable
• A continuous random variable
with the pdf
– Gaussian random variable is completely specified by
its mean and variance
– cdf
– Q-function
–
mean
is a Gaussian random variable with
and variance
Jointly Gaussian Random Variables
• Two random variables
joint pdf
and
with the
(correlation coefficient)
– When
,
and
are uncorrelated and
– Uncorrelated Gaussian random variables are also
independent
3
Jointly Gaussian Random Variables
–
(a and b are not both zero) is a
Gaussian random variable with mean
and variance
– Given any 2£2 invertible matrix , the two
random variables and defined as
are jointly Gaussian.
Complex Gaussian Random Variable
• Circular complex Gaussian random variable
and are independent Gaussian random variables of
the same variance
– A complex random variable is just a compact
representation of two real random variables
• Two complex Gaussian random variables are
independent if all the four real and imaginary
parts are independent.
4
Rayleigh Random Variable
• For circular complex Gaussian random
variable
When
and
are both zero mean
–
is a Rayleigh random variable
–
is a uniform random variable
–
and
are independent
Ricean Random variable
When
–
and/or
are non-zero mean
is a Ricean random variable
is the 0th order Bessel function of the 1st kind
–
–
is not uniform
and are not independent
5
Gaussian Random Vector
• n random variables collected in a vector
and with a joint pdf
– Each is a Gaussian random variable with
mean
and variance
–
gives the covariance of and
– When is a diagonal matrix,
are
uncorrelated and thus independent
Gaussian Random Vector
• Given any non-zero n£1 vector ,
is a
Gaussian random variable with mean
and variance
• Given any invertible n£n matrix , the
vector
is a Gaussian random vector
with mean vector
and variance
6
Gaussian Random Process
• A random process N(t) such that for any k>0
and any k epochs, t1, t2, …, tk, the random
variables N(t1), N(t2), …, N(tk) are jointly
Gaussian
• If the E[N(t)] and E[N(t+τ)N(t)] are not functions
of t, the process N(t) is called stationary.
• R(τ) = E[N(t)N(t+τ)] is called the correlation
function of the stationary process N(t).
• Φ(f)= FT{R(τ)} is called the power spectral
density (PSD) of N(t).
• If R(τ) = σ2 δ(τ), N(t) is called a white Gaussian
process.
Circular Complex Gaussian Process
• Two jointly Gaussian random processes
X(t) and Y(t), both zero-mean and with the
same correlation function R(τ), and
independent, can be written in a complexvalued random process
Z(t) = X(t) + jY(t)
E[Z(t)] = 0
RZ(τ) = 1/2E[Z*(t)Z(t+τ)] = R(τ)
7
Gaussian Random Process
• A Gaussian process N(t) (zero mean and
correlation function R(τ)) inputted to a
linear time-invariant system h(t), the output
is also a Gaussian process with zero
mean and correlation function
– When the input process is white,
Gaussian Random Process
• A Gaussian process N(t) (zero mean and
correlation function R(τ)) and a set of
signals Φ1(t), …, Φn(t) of the L2 space, the
inner products
are jointly Gaussian and all zero mean
– If N(t) is white
– If, furthermore, Φ1(t), …, Φn(t) are
orthonormal, Zj’s are independent and
identically distributed.
8
Passband Gaussian Process
• If the PSD of a Gaussian process satisfies the
passband condition
Φ(f) = 0 for |f| < fc – B and |f|>fc+B
• Complex envelop of a passband Gaussian
process
• If N(t) is white, zero-mean
– N_I(t) and N_Q(t) are independent, both zero mean
and white, so
is circular complex Gaussian
9