Download Conditional Expectation for discrete random variables

Lecture 28
Agenda
1. Conditional Expectation for discrete random variables
2. Joint Distribution of Continuous Random Variables
Conditional Expectation for discrete random
variables
Let X and Y be two discrete random variables. For X = x we know that
instead of using {pY (y) : y ∈ Range(Y )} it’s better to use {pY |X=x (y) : y ∈
Range(Y )}. We defined this conditional distribution as,
pY |X=x (y) =
P (X = x, Y = y)
pX,Y (x, y)
=
pX (x)
P (X = x)
for x ∈ Range(X) and y ∈ Range(Y ). Note that this definition is conceptually nothing new, because we have already learned the definition of
conditional probability. Now we define conditional expectation of Y given
X = x.
Definition 1. For two discrete random variables X and Y , for some x ∈
Range(X), we define the conditional expectation of Y given X = x as,
X
E(Y |X = x) =
y × pY |X=x (y)
y∈Range(Y )
Please note that E(X|Y = y) can be defined similarly.
Example
The joint distribution of X and Y is given in the following table
Y=-1
Y=2.5
Y=3
Y=4.7
X=0 X=1 X=3
0.11 0.03 0.00
0.03 0.09 0.16
0.15 0.15 0.06
0.04 0.16 0.02
1
Let’s find E(Y |X = 1).
P (X = 1) = 0.03 + 0.09 + 0.15 + 0.16 = 0.43
Hence,
E(Y |X = 1) = −1 ×
0.03
0.09
0.15
0.16
+ 2.5 ×
+3×
+ 4.7 ×
= 3.2488
0.43
0.43
0.43
0.43
Joint Distribution of Continuous Random Variables
We previously studied the joint probability mass function of two discrete
random variables, which described their joint behaviour. Today, we repeat
the same procedure for continuous random variables.
Let’s recollect that for a continuous random variable X, there has to be
a density function fX such that,
• fX (x) ≥ 0
∀x ∈ R
• ∀a < b, P (a < X < b) =
Rb
a
fX (x)dx
Suppose we have two continuous random variables X and Y . The joint
probability behaviour of X and Y is described by the “joint probability
density function” fX,Y of X and Y which has the following properties
• fX,Y (x, y) ≥ 0
∀(x, y) ∈ R2
• P (a ≤ X ≤ b, c ≤ Y ≤ d) =
c < d.
RbRd
a
c
fX,Y (x, y)dxdy for all a < b and
It follows that the “joint probability distribution function” FX,Y is
defined as,
Z a Z b
FX,Y (a, b) = P (X ≤ a, Y ≤ b) =
fX,Y (x, y)dxdy
−∞
−∞
Example
A certain process for producing an industrial chemical yeilds a product that
contains two main types of impurities. Let X denote the proportion of impurities of Type I and Y denote the proportion of impurities of Type II.
2
Suppose that the joint density of X and Y can be modelled as,
fX,Y (x, y) = 2(1 − x)
= 0
if 0 ≤ x ≤ 1, 0 ≤ y ≤ 1
otherwise
Compute P (0 ≤ X ≤ 0.5, 0.4 ≤ Y ≤ 0.7).
Z 0.5 Z 0.7
2(1 − x)dydx
P (0 ≤ X ≤ 0.5, 0.4 ≤ Y ≤ 0.7) =
0
0.4
Z 0.5
(0.7 − 0.4) × 2(1 − x)dx
=
0
Z 0.5
= 0.3 ×
2(1 − x)dx
0
0.5
= 0.3 × −(1 − x)2 0
= 0.3 × −(1 − 0.5)2 + (1 − 0)2
= 0.3 × [1 − 0.25]
= 0.3 × 0.75
= 0.225
Lemma 1. If X and Y are continuous random variables, with joint pdf fX,Y
then the individual or marginal pdf ’s fX and fY are given by,
Z ∞
fX (x) =
fX,Y (x, y)dy
−∞
Z
∞
fY (y) =
fX,Y (x, y)dx
−∞
For the industrial production example considered previously, we compute
3
fX and fY .
Z
∞
fX,Y (x, y)dy
fX (x) =
Z−∞
∞
2(1 − x)1{0≤x≤1,0≤y≤1} dy
=
−∞
Z 1
2(1 − x)1{0≤x≤1} dy
=
0
= 2(1 − x)1{0≤x≤1}
= 2(1 − x)
if 0 ≤ x ≤ 1
= 0
otherwise
Z
∞
fY (y) =
fX,Y (x, y)dx
Z−∞
∞
2(1 − x)1{0≤x≤1,0≤y≤1} dx
=
−∞
Z 1
=
2(1 − x)1{0≤y≤1} dx
1
1{0≤y≤1} −(1 − x)2 0
1{0≤y≤1} × 1
1
if 0 ≤ y ≤ 1
0
otherwise
0
=
=
=
=
Please note that by using the indicator function 1{...} we are considering
the various cases at once and making the calculations easier.
Suppose we are interseted in the behaviour of the random variable Y
given X = x. To develop a framework for expressing this behaviour we
need the notion of “conditional probability density function” (or “conditional
pdf” in short). But before we go on let’s note one thing. When we defined
P (A|B) = P P(A∩B)
, we inherently assumed that P (B) > 0. But if we want to
(B)
calculate P (Y ≤ 0.5|X = 0.3), we can’t assume P (X = 0.3) > 0 because X
is continuous. So we need another strategy.
Definition 2. Let X and Y be continuous random variables with joint pdf
fX,Y and marginal pdf ’s fX and fY . Then for any x such that fX (x) > 0,
4
we define the conditional pdf of Y given X = x as,
fY |X=x (y) =
fX,Y (x, y)
fX (x)
for all y ∈ R.
Note that when fX (x) = 0 we don’t define the above quantity. Now we
can calculate
Z 0.5
fY |X=0.3 (y)dy
P (Y ≤ 0.5|X = 0.3) =
−∞
Homework::
1. For homework from Lecture 27, find E(Y |X = 1) for Problem
1 and E(X|Y = −1) for Problem 2.
2. 5.27,5.31,5.32,5.34
5