Download Chapter 8 Joint Distributions § 8.1 Bivariate Distributions Joint

_________________________________________________________________________________
Chapter 8 Joint Distributions
§ 8.1 Bivariate Distributions
Joint Probability Functions
• Definition: Let X and Y be two discrete random
variables defined on the same sample space. Then
p(x,y) = P(X=x,Y=y)
is called the joint probability function of X and Y.
* If the sets of all possible values of X and Y are called A
and B respectively, then
pX(x) =
pY(y) =

p(x,y)
y B

p(x,y)
x A
• Definition: The above two functions are called the
marginal probability functions of random variables X
and Y, respectively.
 see Example 8.1
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
* Example 8.2: Roll a balanced die and let the outcome be
X. Then toss a fair coin X times and let Y denote the
number of tails.
Solution:
p(x,y) = P(X=x,Y=y) = (1/6)C(x,y)(0.5)x
pX(x) and pY(y) can be obtained from the table in text.
Joint Probability Density Functions
• Definition: Two continuous random variables X and Y
defined on the same sample space have a continuous
joint probability density function (jpdf) if there exists a
nonnegative function of two variables f(x,y) on RxR, such
that for any region in the xy-plane that can be formed
from rectangles by a countable number of set operations,
P{(X,Y) R} = fX,Y(x,y)
dx dy.
R
The function is f(x,y) called the joint probability density
function of X and Y .


*
- - fX,Y(x,y)dx dy = 1
*
P(X=a,Y=b) =
*
P(a <X  b, c <Y  d) = a c fX,Y(x,y)
a b
a b fX,Y(x,y) dx dy = 0
b d
__________________________________________________________
dx dy
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
*

fX(x) = - fX,Y(x,y) dy

fY(y) = - fX,Y(x,y) dx
and
• Definition: The above two functions are called the
marginal probability density functions of X and Y,
respectively.
• Definition: The joint (cumulative) probability
distribution function of X and Y is given by
FX,Y(t,u) = P(X  t, Y  u)
and the marginal cdf are given by
FX(t) = FX,Y(t, )
t
FY(u) = FX,Y(,u)
u
- - fX,Y(x,y)dx dy
• FX,Y(t,u) =
• fX,Y(x,y)
and
= 2FX,Y(x,y)/ xy
x
• FX(x) = FX,Y(x,) = - fX(t)dt
y
• FY(y) =FX,Y(,y) = - fY(t)dt
• fX(x) = FX'(x) and fY(y) = FY'(y)
* Example 8.4: With the following joint pdf
f(x,y) = xy2
0xy1
(a) What is ?
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
(b) Find the marginal pdf.



We have - - fX,Y(x,y)dx dy = 1, so
Solution:

- - fX,Y(x,y)dx dy
= (/10) = 1
1
1
= 0 (x fX,Y(x,y)dy ) dx
and = 10.

fX(x) = - fX,Y(x,y)dy

fY(y) = - fX,Y(x,y)dx
1
=
x xy2dy = (10/3)x(1-x3)
=
0 xy2dx = 5y4
y
* see Examples 8.4 and 8.5
Example 8.6
(Plot geometric model)
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
• Definition: Let S be a subset of the plane with area
A(S). A point is said to be randomly selected from S if for
any subset R of S with area A(R), the probability that R
contains the point is A(R)/A(S).
* Example 8.7: If a men and his fiancée are scheduled to
meet between 11:30 and 12:00. Suppose that they arrive at
random times. What is the probability that they meet
within 10 minutes of arrival?
Solution: Let X and Y be the times from 11:30 that these
two men arrive, respectively. Then 0  X, Y  30. So the
probability is P(|X-Y| 10).
Exercise 22 (p. 329)
*
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
• Theorem 8.2: Let f(x,y) be the joint pdf of random
variables X and Y. If h is a function of two variables from
R2 to R, then Z = h(X,Y) is a random variable with


E(Z) = - - h(x,y) f(x,y)dx dy
Provided the integral is absolutely convergent.
Corollary: For random variables X and Y,
E(X + Y) = E(X) + E(Y)
* Example 8.8: Let X and Y have
f(x,y) = (3/2) (x2+y2)
=0
if 0 < X,Y < 1
otherwise
Find E(X2+Y2).
Solution:


E(X2+Y2) = - - (x2+y2)f (x,y) dxdy
1 1
= 0 0 (3/2) (x2+y2)2 dxdy
1 1
= (3/2) 0 0 (x4+2x2y2+y4) dxdy
= 14/15
Q: d. r. v. version of Theorem 8.2?
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
§ 8.2 Independent Random Variables
• Theorem 8.3: Let X and Y be two random variables
defined on the same sample space. If F is the joint
probability distribution function (JCDF) of X and Y, then
X and Y are independent if and only if for all real
numbers t and u,
F(t,u) = FX(t)FY(u)
• Theorem 8.4: Let X and Y be two discrete random
variables defined on the same sample space. If p is the
joint probability function of X and Y, then X and Y are
independent if and only if for all real numbers x and y,
p(x,y) = pX(x)pY(y)
* X and Y are independent discrete random variables if
knowing the value of one of them does not change the
probability function of the other.
P(X=x|Y=y) = P(X=x) and P(Y=y|X=x) = P(Y=y)
• Theorem 8.5: Let X and Y be jointly continuous
random variables with joint probability density function
f(x,y). Then X and Y are independent if and only if f(x,y)
is the product of their marginal densities fX(x) and fY(y),
i.e.
f(x,y) = fX(x) fY(y)
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
* Example 8.12: There are two stores whose weekly
profits in thousand dollars are independent identically
distributed (i.i.d.) random variables given by
f(x) = x/4
=0
if 1 < x < 3,
otherwise
What is the probability that one store makes at least $500
more than the other next week?
Solution: Let theses two random variables be X and Y.
Since X and Y are independent, so their joint pdf is given
by the product of their respective marginal pdf, i.e.
f(x,y) = xy/16
=0
1 < x,y < 3
otherwise
We can integrate over the gray area where X>Y+0.5 and
multiply the probability by two to get the answer.
3
x-0.5
P = 2 1.5 (1
3
(xy/16) dy)dx = (1/8)1.5 [xy2/2]1
x-0.5
dx
3
= (1/16)1.5 (x3 - x2 - 0.75x)dx = 0.54
Fig. 8.4
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
* Examples 8.14 (Buffon’s Needle Problem).
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
• Theorem 8.5: Let X and Y be independent random
variables and f:R->R and g:R->R be real-valued
functions, then f(X) and g(Y) are also independent
random variables.
• Theorem 8.6: Let X and Y be independent random
variables. Then for all real-valued function g:R->R and
h:R->R,
E[g(X)h(Y)] = E[g(X)]E[h(Y)],
where we assume that E[g(X)] and E[h(Y)] are finite.
Proof:
The joint pdf is separable. For detail see text.
* If X and Y are independent, then E(XY) = E(X)E(Y).
Note the converse of this property is not true.
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
§ 8.3 Conditional Distributions
• Given two discrete random variables X and Y, the
conditional probability function of X given that Y=y is
defined as
pX|Y(x|y) = P(X=x|Y=y) = P(X=x,Y=y)/P(Y=y)
= p(x,y)/pY(y).
* Note that pX|Y is itself a probability function with all
possible values as those of pX. When X and Y are
independent, we have pX|Y
= pX.
• Given two discrete random variables X and Y. The
conditional distribution function of X given that Y=y is
defined as
FX|Y(x|y) = P(Xx|Y=y) =
=
P(X=t,Y=y)/P(Y=y)
tx
pX|Y(t,y) .
tx
• Example 8.17: Let the number of men and women
entering a post office in a certain interval be two
independent Poisson random variables with parameters
 and  , respectively. Find the conditional probability
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
function of the number of men given the total number of
persons.
Solution: Let N, M, K be the total number of men, women,
and persons entering the post office. Note that K = M+N
and M, N are independent. So we have K is also Poisson
with parameter +.
pN|K(n|k) = P(N=n)P(M=k-n)/P(K=k)
= [e-t(t)n/n!][e-t(t)k-n/(k-n)!]/[e-(+)t(()t)k/k!]
= C(k,n)[()]n[()]k-n
Note that this is just binomial distribution with
parameter k and /(+).
• Given two continuous random variables X and Y, the
conditional probability density function (pdf) of X given
Y is defined as
fX|Y(x|y) = f(x,y)/fY(y),

provided fY(y) > 0. Note that - fX|Y(x|y)dx = 1 and
hence it is itself a pdf. If X and Y are independent, then
fX|Y = fX .
• The conditional probability distribution function (cdf)
of X given Y is defined as
x
FX|Y(x|y) = P(Xx|Y=y) = - f(t,y)/fY(y) dt.
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
So, d[FX|Y(x|y)]/dx = fX|Y(x|y).
* Example: Select a point Y from (0,1). Then select a point
X from (0,Y). Find the pdf of X.
Solution:
fX|Y(x|y) = 1/y when 0<x<y and fX|Y(x|y) =
0 otherwise.


fX(x) = - fX,Y(x,y) dy = - fX|Y(x|y)fY(y)dy
1
x (1/y)dy
=
fX(x)
= -lnx.
= -lnx
=0
0<x<1
otherwise
• The conditional expectation of X given Y is defined as

E(X|Y=y) =
=
for all x

- xfX|Y(x|y)dx
*
E(h(X)|Y=y) =
=
* 2X|Y=y
xP(X=x|Y=y)
=

for discrete r.v.
for continuous r.v.
h(x)P(X=x|Y=y)
for all x

- h(x)fX|Y(x|y)dx
for discrete r.v.
for continuous r.v.

- [x-E(X|Y=y)]2fX|Y(x|y)dx
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
* Example: Let X and Y be continuous random variables
with joint probability density function
f(x,y) = e-y
=0
if y>0, 0<x<1
otherwise
Find E(X|Y=2).
Solution: E(X|Y=2) =

= - xf(x,2)/fY(2) dx
fY(2) =
1
0 f(x,2)dx
E(X|Y=2) =

- xfX|Y(x|2)dx
1
0 xe-2/fY(2)dx
=
1
= 0 e-2dx
= e-2. Hence
1
0 xe-2/e-2dx = 1/2.
Example 8.20 (The Box Problem: to Switch or Not to
Switch, p.347)
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
§ 8.4 Transformation of Two Random Variables
• Theorem 8.8: Let X and Y be continuous random
variables with joint probability density function f(x,y). Let
h1 and h2 be two real-valued functions of two variables, U
= h1 (X,Y) and V = h2 (X,Y) to a set Q in the uv plane. If
(a) the system of two equations of two unknowns
h1(x,y) = u, h2(x,y) = v.
has a unique solution for x and y in terms of u and v, i.e.,
x = w1(u,v) and y = w2(u,v) and
(b) the function w1 and w2 have continuous partial
derivatives, and the Jacobian of the transformation
x = w1(u,v) and y = w2(u,v) is nonzero at all points (u,v);
that is, the following 2 x 2 determinant is everywhere
nonzero:
J = |w1/u w2/v|
|w1/u w2/v|
= w1/u w2/v - w1/v w2/u
 0, for all (u,v)  Q,
then the random variables U and V are jointly continuous
with the joint probability density function g(u,v) given by
g(u,v) = f(w1(u,v), w2(u,v)) |J|
=0
(u,v)  Q,
otherwise.
Proof: This theorem is the result of the change of
variables theorem in double integrals. See text.
* see Example 8.36
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___
_________________________________________________________________________________
* Example 8.27 Box-Muller Theorem (Generation of
Normal r.v.)
Let X and Y be two independent uniform random
variables over (0,1); show that the random variables
U = cos(2X)(-2lnY)0.5 and V = sin(2X)(-2lnY)0.5
are independent standard normal random variables.
Solution: We have
u2 + v2 = -2lny, y = w2 = e -(u2+v2)/2.
cos(2x) = u/(u2+v2)0.5, sin(2x) = v/(u2+v2) 0.5
x = w1 = (1/2cos-1(u/(u2+v2) 0.5) .
The first condition is satisfied. Next the Jacobian is
w1/u w2/v - w1/v w2/u
= {(-v)/[2(u2+v2)]}{ (-v) e -(u2+v2)/2} {u/[2(u2+v2)]}{ (-u) e -(u2+v2)/2}
= (1/2) e -(u2+v2)/2  0.
We have the joint pdf of U and V is given by
g(u,v) = f (x,y)|J| = f (x,y) (1/2) e -(u2+v2)/2
= (1/2)e -(u2+v2)/2.
Integrating over v and u, respectively, one obtains
gU(u) = (1/2)0.5e -u2/2 and gV(v) = (1/2)0.5e -v2/2 .
So U and V are independent standard normal random
variables.
__________________________________________________________
© Shi-Chung Chang, Tzi-Dar Chiueh
___