Download Probability and Stochastic Processes

Random Variables
Discrete: Bernoulli, Binomial, Geometric, Poisson
Continuous: Uniform, Exponential, Gamma, Normal
Expectation & Variance, Joint Distributions, Moment
Generating Functions, Limit Theorems
Chapter 2
1
Definition of random variable
A random variable is a function that assigns a number to
each outcome in a sample space.
• If the set of all possible values of a random variable X is
countable, then X is discrete. The distribution of X is
described by a probability mass function:
p  a   P s  S : X  s   a  P  X  a
• Otherwise, X is a continuous random variable if there is a
nonnegative function f(x), defined for all real numbers x,
such that for any set B,
P s  S : X  s   B  P  X  B   f  x dx
B
f(x) is called the probability density function of X.
Chapter 2
2
pmf’s and cdf’s
• The probability mass function (pmf) for a discrete
random variable is positive for at most a countable
number of values of X: x1, x2, …, and  p  xi   1
i 1
• The cumulative distribution function (cdf) for any
random variable X is F  x   P  X  x
F(x) is a nondecreasing function with
lim F  x   0 and lim F  x   1
x
x 
• For a discrete random variable X, F  a    p  xi 
Chapter 2
xi  a
3
Bernoulli Random Variable
• An experiment has two possible outcomes, called “success”
and “failure”: sometimes called a Bernoulli trial
• The probability of success is p
• X = 1 if success occurs, X = 0 if failure occurs
Then p(0) = P{X = 0} = 1 – p and p(1) = P{X = 1} = p
X is a Bernoulli random variable with parameter p.
Chapter 2
4
Binomial Random Variable
• A sequence of n independent Bernoulli trials are performed,
where the probability of success on each trial is p
• X is the number of successes
Then for i = 0, 1, …, n,
 n i
n i
p  i   P  X  i    p 1  p 
i
where
 n
n!
 
 i  i ! n  i  !
X is a binomial random variable with parameters n and p.
Chapter 2
5
Geometric Random Variable
• A sequence of independent Bernoulli trials is performed
with p = P(success)
• X is the number of trials until (including) the first success.
Then X may equal 1, 2, … and
p  i   P  X  i  1  p 
i 1
p, i  1, 2,...

1
X is named after the geometric series: If r  1, then  r 

1 r
i 1
Use this to verify that  p  i   1
i
i 1
Chapter 2
6
Poisson Random Variable
X is a Poisson random variable with parameter l > 0 if
e l l i
p  i   P  X  i 
, i  0,1,...
i!


note: i0 p  i   1 follows from el  i0 l i i !
X can represent the number of “rare events” that occur
during an interval of specified length
A Poisson random variable can also approximate a
binomial random variable with large n and small p if l =
np: split the interval into n subintervals, and label the
occurrence of an event during a subinterval as “success”.
Chapter 2
7
Continuous random variables
A probability density function (pdf) must satisfy:
f  x  0

 f  x  dx  1

b
P a  X  b   f  x  dx
(note P  X  a  0)
a
The cdf is:
F  a   P  X  a  
a

dF  x 
f  x  dx, so f  x  
dx

 
P a   X  a     f  a  means that f(a) measures how
2
2

likely X is to be near a.
Chapter 2
8
Uniform random variable
X is uniformly distributed over an interval (a, b) if its pdf is
 1
, a xb
all we know about

f  x  b  a
X is that it takes a
0, otherwise
value between a and b
Then its cdf is:
0, x  a
xa

F  x  
,a xb
b  a
1, x  b
Chapter 2
9
Exponential random variable
X has an exponential distribution with parameter l > 0 if its
pdf is
l e l x , x  0
f  x  
0, otherwise
Then its cdf is:
0, x  0
F  x  
l x
1

e
,x0

This distribution has very special characteristics that we will
use often!
Chapter 2
10
Gamma random variable
X has an gamma distribution with parameters l > 0 and a > 0
if its pdf is
 l e l x  l x a 1
, x0

f  x  
 a 

0, otherwise

It gets its name from the gamma function  a    e x xa 1dx
0
If a is an integer, then  a   a 1!
Chapter 2
11
Normal random variable
X has a normal distribution with parameters m and s2 if its pdf
is
1  x  m 2 2s 2
f  x 
e
,   x  
2
This is the classic “bell-shaped” distribution widely used in
statistics. It has the useful characteristic that a linear
function Y = aX+b is normally distributed with parameters
amb and (as2 . In particular, Z = (X – m)/s has the
standard normal distribution with parameters 0 and 1.
Chapter 2
12
Expectation
Expected value (mean) of a random variable is
  i xi p  xi , discrete

EX    
xf x dx, continuous

-  
Also called first moment – like moment of inertia of the
probability distribution
If the experiment is repeated and random variable observed
many times, it represents the long run average value of the
r.v.
Chapter 2
13
Expectations of Discrete Random
Variables
•
•
•
•
Bernoulli: E[X] = 1(p) + 0(1-p) = p
Binomial: E[X] = np
Geometric: E[X] = 1/p (by a trick, see text)
Poisson: E[X] = l : the parameter is the expected or
average number of “rare events” per interval; the random
variable is the number of events in a particular interval
chosen at random
Chapter 2
14
Expectations of Continuous Random
Variables
•
•
•
•
Uniform: E[X] = (a + b)/2
Exponential: E[X] = 1/l
Gamma: E[X] = ab
Normal: E[X] = m : the first parameter is the expected
value: note that its density is symmetric about x = m:
1  x  m 2
f  x 
e
2
2s 2
,   x  
Chapter 2
15
Expectation of a function of a r.v.
• First way: If X is a r.v., then Y = g(X) is a r.v.. Find the
distribution of Y, then find E Y    i yi p  yi 
• Second way: If X is a random variable, then for any realvalued function g,
  i g  xi  p  xi , X discrete

E  g  X     
g x f x dx, X continuous

-    
If g(X) is a linear function of X:
E  aX  b  aE  X   b
Chapter 2
16
Higher-order moments
The nth moment of X is E[Xn]:
  xi n p  xi , discrete
i

n
E  X    
n
x
f  x  dx, continuous

-
The variance is
2

Var  X   E  X  E  X  


It is sometimes easier to calculate as
Var  X   E  X    E  X 
2
Chapter 2
2
17
Variances of Discrete Random Variables
• Bernoulli: E[X2] = 1(p) + 0(1-p) = p; Var(X) = p – p2 = p(1p)
• Binomial: Var(X) = np(1-p)
• Geometric: Var(X) = 1/p2 (similar trick as for E[X])
• Poisson: Var(X) = l : the parameter is also the variance of
the number of “rare events” per interval!
Chapter 2
18
Variances of Continuous Random
Variables
•
•
•
•
Uniform: Var(X) = (b - a)2/2
Exponential: Var(X) = 1/l
Gamma: Var(X) = ab2
Normal: Var(X) = s 2: the second parameter is the variance
Chapter 2
19
Jointly Distributed Random Variables
See text pages 46-47 for definitions of joint cdf, pmf, pdf,
marginal distributions.
Main results that we will use:
E  X  Y   E  X   E Y 
E  aX  bY   aE  X   bE Y 
E  a1 X 1  a2 X 2  ...  an X n   a1E  X 1   ...  an E  X n 
especially useful with indicator r.v.’s: IA = 1 if A occurs, 0
otherwise
Chapter 2
20
Independent Random Variables
X and Y are independent if P  X  a, Y  b  P  X  a P Y  b
This implies that: p  x, y   p X  x  pY  y  (discrete)
f  x, y   f X  x  fY  y  (continuous)
Also, if X and Y are independent, then for any functions h
and g, E  g  X  h Y    E  g  X   E  h Y  
Chapter 2
21
Covariance
The covariance of X and Y is:
Cov  X , Y   E  X  E  X  Y  E Y   E  XY   E  X  E Y 
If X and Y are independent then Cov(X,Y) = 0.
Properties: Cov  X , X   Var  X 
Cov  X , Y   Var Y , X 
Cov  cX , Y   cCov  X , Y 
Cov  X , Y  Z   Cov  X , Y   Cov  X , Z 
Chapter 2
22
Variance of a sum of r.v.’s
n
n
 n
 n
 n

Var   X i   Cov   X i ,  X j    Var  X i   2 Cov  X i , X j 
j=1
i=1 j<i
 i=1 
 i=1
 i=1
If X1, X2, …, Xn are independent, then
 n
 n
Var   X i    Var  X i 
 i=1  i=1
Chapter 2
23
Moment generating function
The moment generating function of a r.v. X is
  i etxi p  xi , X discrete

tX
  t   E e    
tx
e
f  x  dx, X continuous


 -
n
d
 t 
Its name comes from the fact that
n



E
X
n

dt t 0
Also, if X and Y are independent, then X Y  t   X  t Y  t 
And, there is a one-to-one correspondence between the m.g.f.
and the distribution function of a r.v. – this helps to identify
distributions with the reproductive property
Chapter 2
24