Download Y - WordPress.com

Probability and Statistical
Inference (9th Edition)
Chapter 4
Bivariate Distributions
November 4, 2015
1
Joint Probability Mass Function
 Let X and Y be two discrete random variables defined on
the same outcome set. The probability that X=x and Y=y
is denoted by PX,Y(x,y)= P(X=x,Y=y) and is called the
joint probability mass function (joint pmf) of X and Y
 PX,Y(x,y) satisfies the the following 3 properties:
(1) 0  PX ,Y x, y   1
(2)
P  x, y   1

 
x , y S
X ,Y
(3) Pr ob X , Y   A 
P x, y , where A is a subset of S.

 
x , y A
X ,Y
2
Joint Probability Mass Function
 Example: Roll a pair of unbiased dice. For each of
the 36 possible outcomes, let X denote the smaller
number and Y denote the larger number
 The joint pmf of X and Y is:
1 / 36 1  x  y  6
PX ,Y x, y   
2 / 36 1  x  y  6
3
Joint Probability Mass Function
 Note that we can always create a common
outcome set for any two or more random
variables. For example, let X and Y
correspond to the outcomes of the first and
second tosses of a coin, respectively. Then,
the outcome set of X is {head up, tail up} and
the outcome set of Y is also {head up, tail up}.
The common outcome set of X and Y is
{(head up, head up),(head up, tail up),(tail up,
head up),(tail up, tail up)}
4
Joint Probability Mass Function
 Another Example: Assume that we toss a dice
once. Let random variable X correspond to
whether the outcome is less than or equal to 2,
and random variable Y correspond to whether
the outcome is an even number. Then, the joint
pmf of X and Y is shown on the next page
5
Joint Probability Mass Function
Y
PXY(1,1)=1/6
1
PXY(0,1)=1/3
PXY(1,0)=1/6
PXY(0,0)=1/3
X
0
1
Outcome
1
2
3
4
5
6
X
Y
1
0
1
1
0
0
0
1
0
0
0
1
6
Marginal Probability Mass Function
 Let PXY(x,y) be the joint pmf of discrete random variables
X and Y. Then
PX x   Pr ob X  x    Pr obX  x, Y  yi 
yj
  PXY ( x, yi ) 
yj
is called the marginal pmf of X
 Similarly,
PY  y    PX ,Y xi , y 
xi
is called the marginal pmf of Y
7
Independent Random Variables
 Two discrete random variables X and Y are said
to be independent if and only if
PX ,Y x, y   PX x PY  y .
 Otherwise, X and Y are said to be dependent
8
Uncorrelated Random Variables
 Let X and Y be two random variables. Then, E[(XµX)(Y-µY)] is called the covariance of X and Y
(denoted by Cov(X,Y))
 Covariance is a measure of how much two random
variables change together
 A positive value of Cov(X,Y) indicates that Y tends to
increase as X increases
 Two discrete random variables X and Y are said to
be uncorrelated if Cov(X,Y)=0
 Otherwise, X and Y are said to be correlated
9
Independent Implies Uncorrelated
 Cov(X,Y) = E[(X-µX)(Y-µY)]
= E[XY- µYX- µXY+ µXµY]
= E[XY]- µYE[X]- µXE[Y]+E[µXµY]
= E[XY]- µXµY
 If X and Y are independent, then
E[ XY ]   xyPXY ( x, y )
x
y
  xyPX ( x) PY ( y )
x
y
  xPX ( x) yPY ( y )
x
y
 E[ X ]E[Y ]
 Therefore, if X and Y are independent, then Cov(X,Y)=0
10
 The converse statement is not true (example later)
Correlation Coefficient
 Correlation coefficient of X and Y:
 Insights: If X and Y are above or below their respective
means simultaneously, then ρXY > 0. If X is above µX
whenever Y is below µY, and X is below µX whenever Y is
above µY, then ρXY < 0
11
Addition of Two Random Variables
 Let X and Y be two random variables. Then,
E[X+Y]=E[X]+E[Y]
 Note that the above equation holds even if X and Y
are dependent
 Proof of the discrete case:
E[ X  Y ]   PXY ( x, y )( x  y )
x
y
  x  PXY ( x, y )   y  PXY ( x, y )
x
y
x
y
  x  PXY ( x, y )   y  PXY ( x, y )
x
y
y
x
  xPX ( x)   yPY ( y )  E[ X ]  E[Y ]
x
y
12
Addition of Two Random Variables
 On the other hand,
Var[ X  Y ]
 E[(( X  Y )  (  x   y )) 2 ]
 E[( X  Y ) 2  (  x   y ) 2  2( X  Y )(  x   y )]
 E[( X  Y ) 2 ]  (  x   y ) 2  2(  x   y ) 2
 E[ X ]2  E[Y ]2  2 E[ XY ]   x   y  2 x  y
2
2
 ( E[ X ]   x )  ( E[Y ]   y )  2( E[ XY ]   x  y )
2
2
2
2
 Var[ X ]  Var[Y ]  2( E[ XY ]  E[ X ]E[Y ]) #
13
Addition of Two Random Variables
 Note that if X and Y are independent, then
E[XY]=E[X]E[Y]
 Therefore, if X and Y are independent, then
Var[X+Y]=Var[X]+Var[Y]
14
Examples of Correlated Random
Variables
 Assume that a supermarket collected the
following statistics of customers’ purchasing
behavior:
Purchasing
Wine
Not Purchasing
Wine
Male
45
255
Female
70
630
Male
Female
Purchasing
Juice
60
210
Not Purchasing
Juice
240
490
15
Examples of Correlated Random
Variables
 Let random variable M correspond to whether a
customer is male, random variable W
correspond to whether a customer purchases
wine, random variable J correspond to whether
a customer purchases juice
16
Examples of Correlated Random
Variables
 The joint pmf of M and W is
W
PMW (0,1) = 0.07
PMW (1,1) = 0.045
M
PMW (0,0) = 0.63
PMW (1,0) = 0.255
Cov(M,W)= E[MW] - E[M]E[W]
= 0.045 – 0.3*0.115
= 0.0105 > 0
M and W are positively correlated (outcome M=1
makes it more likely that W=1)
17
Examples of Correlated Random
Variables
 The joint pmf of M and J is
W
PMJ (0,1) = 0.21
PMJ (1,1) = 0.06
M
PMJ (0,0) = 0.49
PMJ (1,0) = 0.24
Cov(M,J)= E[MJ] - E[M]E[J]
= 0.06 – 0.3*0.27
= -0.021 < 0
M and J are negatively correlated
18
Example of Uncorrelated Random
Variables
 Assume X and Y have the following joint
pmf:
PXY(0,1)= PXY(1,0)= PXY(2,1)= 1/3
 We can derive the following marginal pmfs:
PX 0   PXY (0, y )  1 / 3 ; PX 1   PXY (1, y )  1 / 3
y
y
PX 2   PXY (2, y )  1 / 3 ; PY 0   PXY ( x,0)  1 / 3
x
PY 1   PXY ( x,1)  2 / 3
x
x
19
Example of Uncorrelated Random
Variables
 Since PXY(0,1) = 1/3, and
PX(0) x PY(1) = 1/3 x 2/3 = 2/9,
X and Y are not independent
 However,
Cov(X,Y) = E[XY] – E[X]E[Y] =
[2/9 x 1 + 2/9 x 2] – [1 x 2/3] = 0.
Thus, X and Y are uncorrelated
 Thus, uncorrelated does not imply independence
20
Conditional Distributions
 Let X and Y be two discrete random variables. The
conditional probability mass function (pmf) of X,
given that Y=y, is defined by
PXY x, y 
PX Y x y  
, provided that y  Space of Y.
PY  y 
 Similarly, if X and Y are continuous random variables,
then the conditional probability density function (pdf)
of X, given that Y=y, is defined by
f XY x, y 
f X Y x y  
.
fY  y 
21
Conditional Distributions
 Assume that X and Y are two discrete random
variables. Then,
PXY x, y 
a  PX Y x y  
 0, provided that y  Space of Y.
PY  y 
P  x, y 

 x, y  
P y

 1.
PXY
b   PX Y x y   
PY  y 
x
x
XY
x
PY  y 
Y
PY  y 
 Similarly, for two continuous random variables X and
Y, we have
f XY x, y 
a  f X Y x y  
 0, provided that y  Space of Y.
fY  y 
b  S
X
f X Y x y   1.
22
Conditional Distributions
 The conditional mean of X, given that Y=y, is
defined by
X Y  Ex y    xPX Y x y .
x
 The conditional variance of X, given that Y=y, is
defined by


 X Y   x   X Y 2 PX Y x y .
x
23
Example 1
 Let X and Y have the joint pmf
It can be easily shown that
Then, the conditional pmf of X, given that Y=y, is
24
Example 1 (Cont.)
 Similarly, the conditional pmf of Y, given that X=x, is
25
Example 2
 3 blue balls (labeled A, B, C) and 2 red balls (labeled
D, E) are in a bag
 Randomly taking a ball out of the bag, what is the
probability of getting a blue ball? (Ans: 3/5)
 What is the probability of getting A? (Ans: 1/5)
 What is the probability of getting A, given that the
ball we get is a blue ball? (Ans: 1/3)
X = label of the ball we get
Y = color of the ball we get
P(X=A | Y=blue)
= P(X=A, Y=blue) / P(Y=blue)
= (1/5) / (3/5) = 1/3
26
Bivariate Normal Distribution
 The joint pdf of bivariate normal
 The joint pdf of multivariate normal
where in the case of bivariate
and | | denotes the determinant of a matrix
27
Bivariate Normal Distribution
0.4
0.4
0.3
0.3
Probability Density
Probability Density
 Graphic representations of bivariate (2D) normal
0.2
0.1
0
0.2
0.1
0
2
3
2
3
2
0
1
2
0
1
0
-1
-2
x2
0
-3
-1
-2
-2
x1
x2
28
-2
-3
x1
3
3
2
2
1
1
x2
x2
Bivariate Normal Distribution
0
0
-1
-1
-2
-2
-3
-3
-2
-1
0
x1
1
2
3
-3
-3
-2
-1
0
x1
1
2
3
29
0.4
0.4
0.3
0.3
Probability Density
Probability Density
Bivariate Normal Distribution
0.2
0.1
0
0.2
0.1
0
2
3
2
3
2
0
1
2
0
1
0
-1
-2
x2
0
-3
-1
-2
-2
x1
x2
-2
-3
x1
30
3
3
2
2
1
1
x2
x2
Bivariate Normal Distribution
0
0
-1
-1
-2
-2
-3
-3
-2
-1
0
x1
1
2
3
-3
-3
-2
-1
0
x1
1
2
3
31
Bivariate Normal Distribution
0.4
2
0.3
1
0.2
x2
Probability Density
3
0
0.1
-1
0
2
3
-2
2
0
1
0
-1
-2
x2
-2
-3
x1
-3
-3
-2
-1
0
x1
1
2
3
32
Example
 Let us assume that in a certain population of college
students, the respective grade point average
(GPA)—say X and Y—in high school and the first
year in college have an approximate bivariate
normal distribution with parameters
 Then, for example,
where
33
Example (Cont.)
 The conditional pdf of Y, given that X=x, is normal, with
mean
and variance
34
Example (Cont.)
 Since the conditional pdf of Y, given that X=3.2, is
normal with mean
and standard deviation
we have
35
Correlations and Independence for
Normal Random Variables
 In general, random variables may be
uncorrelated but statistically dependent (i.e.,
uncorrelated does not imply independence)
 But if a random vector has a multivariate normal
distribution, then any two or more of its
components that are uncorrelated are
independent (i.e., uncorrelated does imply
independence in this case)
36
Correlations and Independence for
Normal Random Variables
 The fact that two random variables X and Y
both have a normal distribution does not
imply that the pair (X, Y) has a joint normal
distribution.
 Example: Suppose X has a normal
distribution with expected value 0 and
variance 1. Let
where c is a positive number
 X and Y are not jointly normally distributed,
even though they are separately normally
distributed
37
Correlations and Independence for
Normal Random Variables
 If X and Y are normally distributed and
independent, this implies they are "jointly
normally distributed", i.e., the pair (X, Y)
must have multivariate normal distribution
 However, a pair of jointly normally distributed
variables need not be independent (would
only be so if uncorrelated)
38