Download Document

EMIS 7300
SYSTEMS ANALYSIS METHODS
Spring 2006
Dr. John Lipp
Copyright © 2002 - 2005 Dr. John Lipp
Today’s Topics
• Continuous Random Variables
–
–
–
–
–
–
–
–
–
–
–
Continuous Random Variables.
Continuous PDF and CDF.
Stochastic Expectation.
Mixed Random Variables.
Uniform Random Variables.
Gaussian Random Variables.
Central Limit Theorem.
Chi-Square Random Variables.
Exponential Random Variables
Rayleigh Random Variables.
Cauchy Random Variables.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-2
Continuous Random Variables
• The elementary outcomes are mapped to real numbers, i.e.,
x = X(E), where x is a real number and E is an outcome in a
sample space S.
E
S
x = X(E)
-3
EMIS 7300 Spring 2006
-2
-1
0
1
Copyright  2002 - 2005 Dr. John Lipp
x
2
3
S3P2-3
Continuous Random Variables (cont.)
• The number of possible x values is infinite.
• Question: can the mean calculation

   xi f X ( xi )
i 1
be performed when the xi are real numbers?
• Or put differently, can the real numbers be made into a list
{x1, x2, x3, …, x} which contains all possible real numbers?
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-4
Continuous Random Variables (cont.)
• Proof by contradiction.
– Consider just the real numbers between 0 and 1.
– Write them as decimals, e.g., 0.598248766…
– Assume that the (infinite) list of numbers shown below
contains all the real numbers between 0 and 1.
{0.887871659879013691591325…,
0.698173286190238298539754…,
0.213498077356359723983947…,
0.162390283987279837459734…,
0.903743761618759869834982…,
…}
– Find a real number between 0 and 1 not in the list!
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-5
Continuous RV - Cumulative Distribution Function
• For a discrete random variable, P(X = xi) = fX[xi].
• For a (pure) continuous random variable P(X = x) = 0!
S
E
-3
-2
-1
0
1
2
3
• Therefore the definition of PDF and CDF needs to be
redefined for continuous random variables.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-6
Continuous RV - Cumulative Distribution Function
• Start by keeping the cumulative distribution function (CDF)
as P(X  x) = FX(x).
S
E
X(E)  x
-3
EMIS 7300 Spring 2006
-2
-1
0
x
1
Copyright  2002 - 2005 Dr. John Lipp
2
3
S3P2-7
Continuous RV - Cumulative Distribution Function (cont.)
• The CDF for a continuous RV has some basic properties:
FX(+) = P(X  +) = 1
FX(x) may contain
jump discontinuities
1
FX(x) is right-continuous
x
-3
-2
-1
0
1
2
3
FX(-) = P(X  -) = 0
CDFs are monotone increasing,
that is, FX(x)  FX(y) when x  y
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-8
Continuous RV - Probability Distribution Function
• Use the Fundamental Theorem of Calculus,
a
FX (a )  FX (b)   f X ( x)dx
b
where
d FX ( x)
f X ( x) 
dx
• The function fX(x) is the probability density function (PDF) for
a continuous random variable. Note
FX ( x) 
x
 f X ( y)dy

• The PDF really has no meaning outside of an integral.
Specifically, P(X = x)  fX(x). [Exception: the special case of
jump discontinuities in the CDF.].
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-9
Continuous RV - Probability Distribution Function (cont.)
• The PDF has some basic properties:
– fX(x)  0 (because FX(x) is monotone increasing).
– fX(x) may contain Dirac delta functions (from the jump
discontinuities in FX(x)).
FX(x)


fX(x)
 a ( x)dx  a 

– The area under fX(x) is unity,

 f X ( x)dx  FX ()  FX ()  1  0  1

EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-10
Continuous RV - Probability Distribution Function (cont.)
• Expectation becomes an integral of the PDF:

Eg ( X )   g ( x) f X ( x)dx

• The mean of a continuous random variable is

 x  EX    x f X ( x)dx

• The variance of a continuous random variable is

 x2  E  X   x 2     x   x 2 f X ( x)dx

or

 x2  E  X 2   x2   x 2 f X ( x)dx   x2

EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-11
The Mean, Mode, Median, and Variance
• The mode is the value of x at the maximum (peak) of fX(x).
– In unusual situations, their can be multiple modes.
• The median is the value of x at which it is equally probable
for X to have a value above or below the median, that is,
median


median
 f X ( x)dx   f X ( x)dx  FX (median)  0.5
• The mode, median, or mean are not necessarily the same
values!
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-12
Mixed Distributions
• The CDF can be separated as the sum of two functions: a
continuous function and a sum of step functions.
1
FXc(x)
x
-3
-2
1
0
-1
2
3
1
FXd(x)
x
-3
EMIS 7300 Spring 2006
-2
-1
0
1
Copyright  2002 - 2005 Dr. John Lipp
2
3
S3P2-13
Mixed Distributions (cont.)
• The continuous portion represents a “pure” continuous
random variable and the step portion represents a “pure”
discrete random variable.
• The latter’s continuous PDF is a sum of Dirac delta functions,
M
f X d ( x)   P( X  xi ) ( x  xi )
i 1
1
fXd(x) = a(x - x1)
x
-3
EMIS 7300 Spring 2006
-2
-1
0
1
Copyright  2002 - 2005 Dr. John Lipp
2
3
S3P2-14
Mixed Distributions (cont.)
• Recall the definition of discrete PDF, fX[xi] = P(X = xi). Then,
Md
f X d ( x)   f X d [ xi ] ( x  xi )
i 1
• Using the sifting property of the Dirac delta function yields
the expected result for the discrete portion of a continuous
CDF,
FX d ( x) 
x
x Md
f X [ yi ] ( y  xi )dy
 f X ( y )dy   
i 1
d

Md x


d
Md
f X [ xi ]u ( x  xi )
 f X [ yi ] ( y  xi )dy  
i 1
i 1 
d
d
where u(x) is the unit step function.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-15
Time to Gather some Data
• Aim at the bull’s eye using the dart gun’s sights.
• Measure the X and Y displacements from the bull’s eye.
Y
X
• Collect 15 data sample pairs for your dart gun.
• Plot the data on the provided probability paper.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-16
Distribution: Uniform
• A most simple distribution. The name comes from its PDF.
1
1
b-a
x
a
-3
-2
0
-1
1
b
2
3
• Vital statistics (mean, variance, median, NO mode):
x
ab
dx 
 median
2
aba
b
  E{ X }  
2
2
x
a

b
(
b

a
)


 2  E{ X 2 }   2  
dx  
 
12
 2 
aba
b
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
2
S3P2-17
Distribution: Gaussian
• The Gaussian distribution is also known as the normal
distribution and is often written in shorthand as N(,  2).
• PDF:
f X ( x) 
1
2
2
e
 x   2 2 2
x
1
x


FX ( x)  
  1  erfc


2

2




• CDF:
No closed form for integral of fx(x)!
(See Appendix A, Table II, for values.)
• Mean is  and variance is  2.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-18
Distribution: Gaussian (cont.)
• The PDF for  = 1 and  = 1.5
0.35
0.3
0.25
95.5%
0.2
0.15
99.7%
68.3%
0.1
0.05
0
-4
-3
EMIS 7300 Spring 2006
-2
-1
0
1
2
Copyright  2002 - 2005 Dr. John Lipp
3
4
5
6
S3P2-19
Distribution: Gaussian (cont.)
• The sum of any number of normally distributed data values
(and constants), from independent experiments or otherwise,
and most any linear operation on normally distributed data, is
also normally distributed.
• The sample mean of normally distributed data {x1, x2, …, xn}
is normally distributed:
– Sum the data, then
– Divide by a scalar, n, a linear operation.
• Neither the sample variance nor the sample standard deviation
of normally distributed data {x1, x2, …, xn} are normally
distributed:
– Both involve the sum of squared normal data.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-20
Distribution: Gaussian (cont.)
• Let X be N(x, x2). Define Y = mX + b. Then Y is Gaussian
with mean and variance
y = E{Y} = E{mX + b} = mx + b
y2 = E{Y 2}- y2 = E{(mX + b)2} – (mx + b)2
= E{m 2 X 2 + 2mbX + b 2} – (mx + b)2
= m 2(x2 + x2) + 2mbx + b 2 – (mx + b)2
= m2x2
• Thus, the PDF of Y is fY ( y ) 
EMIS 7300 Spring 2006
1
2 m 
Copyright  2002 - 2005 Dr. John Lipp
2
e
 y m x b 2 2  m 2
S3P2-21
Distribution: Gaussian (cont.)
• A normal random variable N(0,1) is known as the standard
normal random variable. It is often denoted as Z.
X  x
• If X ~ N(x, x2) then a standard RV is Z 
.
x
0.4
0.35
0.3
X
f (x)
0.25
0.2
0.15
0.1
0.05
0
-5
EMIS 7300 Spring 2006
-4
-3
-2
-1
0
x
1
2
Copyright  2002 - 2005 Dr. John Lipp
3
4
5
S3P2-22
Central Limit Theorem
• Let Y be the sum (or average) of a “very large” number of
mutually independent random variables Xi with identical PDFs
and finite means and variances.
• Standardize Y:
N
1 N
X i  N x
Xi  x

Y  y 
Z
 i1
 N i1
y
N x
x N
• Z tends towards a standard normal random variable.
• In general, the sum of even a small number of independent
identically distributed (i.i.d.) random variables tends towards
a Gaussian distribution.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-23
Central Limit Theorem (cont.)
• Consider the sum of IID (independent identical distributed)
uniform random variables ranged between -1 and +1
N
U N   U (1,1)
i 1
• Use the following theorem:
If X and Y are independent random variables, the PDF of
Z = X + Y is the convolution of the PDFs of X and Y,
f Z ( z )  f X ( x)  fY ( y ) 
EMIS 7300 Spring 2006
z
 f X (u ) fY ( z  u )du

Copyright  2002 - 2005 Dr. John Lipp
S3P2-24
Central Limit Theorem (cont.)
0.4
0.4
U
U
2
0.6
1
0.6
0.2
0
-4
0.2
-2
0.6
0
z
2
0
-4
4
-2
0
z
2
4
-2
0
z
2
4
0.6
U
U
4
0.4
3
0.4
0.2
0
-4
0.2
-2
EMIS 7300 Spring 2006
0
z
2
4
0
-4
Copyright  2002 - 2005 Dr. John Lipp
S3P2-25
Distribution: Chi-Square
• Let X be a zero mean (x = 0) Gaussian random variable with
variance x2.
• Then Y = X 2 has a chi-square distribution (with one degree of
freedom).
• PDF:
fY ( y ) 

1
2 y
2
x
e
y
2 x2
,
y0
• The mean is x2 and the variance is 2x4.
• The median is  0.4550 x2. (Found numerically.)
• Mode = 0.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-26
Distribution: Chi-Square (cont.)
1.4
=1
=2
1.2
1
Y
f (y)
0.8
0.6
0.4
0.2
0
EMIS 7300 Spring 2006
0
1
2
3
4
5
y
6
7
Copyright  2002 - 2005 Dr. John Lipp
8
9
10
S3P2-27
Distribution: Exponential
• Let X1 and X2 be zero mean (x = 0) independent Gaussian
random variables with equal variances x2.
• Then Y = X12 + X22 has an exponential distribution with
parameter  = 2 x2.
y
Note: The textbook has
1 
• PDF:
fY ( y )  e
, y0
a different “definition.”

EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-28
Distribution: Exponential (cont.)
• CDF:
FY ( y )  1  e

y

, y0
• The mean is , the variance is 2 2, the median is - ln(2),
and the mode = 0.
• Let U be a random variable uniformly distributed between
0 and 1. Then Y = -  ln(U) is exponentially distributed.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-29
Distribution: Chi-Square with n-degrees of freedom
• Let X1, X2, …, Xn be zero mean (x = 0) independent Gaussian
random variables with equal variances x2.
• Then Y = X12 + X22 + … + Xn2 has a chi-square distribution
with n-degrees of freedom.
1
( n 2 )1 ( y / 2 x2 )
• PDF:
fY ( y )  n / 2 n
y
e
,y0
2  x (n / 2)
3
n = 10
n = 25
n = 100
2.5
1.5

f 2(2)
2
1
0.5
0
0
EMIS 7300 Spring 2006
0.5
1
1.5

2
2.5
3
2
Copyright  2002 - 2005 Dr. John Lipp
S3P2-30
Distribution: Chi-Square with n-degrees of freedom (cont.)
• The gamma function (r) is a “continuous” generalization of
the factorial (needed to count continuous things)
r is an integer 
 (r  1)!
(r )   x e dx  1/ 2

 (r  3 / 2)! r mulitple of 1 2
0

r 1  x
• The mean is nx2 and the variance is 2nx4.
• For n = 2, the chi-square distribution is the exponential with
parameter  = 2x2:
1
1 ( y / 2 x2 )
( 2 2 ) 1  ( y / 2 x2 )
fY ( y )  2 / 2 2
y
e

e
2
2  x (2 / 2)
2 x
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-31
Distribution: Chi-Square with n-degrees of freedom (cont.)
• The chi-square distribution is the distribution for the sample
variance:
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-32
Distribution: Rayleigh
• Let X1 and X2 be zero mean (x = 0) independent Gaussian
random variables with equal variances x2.
• Then Y  X 12  X 22 has a Rayleigh distribution ( = x2).
• The PDF :
fY ( y ) 
y

e
y2

2
, y0
0.35
=4
0.3
0.2
Y
f (y)
0.25
0.15
0.1
0.05
0
0
EMIS 7300 Spring 2006
1
2
3
4
5
y
6
Copyright  2002 - 2005 Dr. John Lipp
7
8
9
10
S3P2-33
Distribution: Rayleigh (cont.)
FY ( y )  1  e
• CDF:
1
y2

2
, y0
=4
0.8
Y
F (y)
0.6
0.4
0.2
0
0
1
2
3
4
5
y
6
7
8
9
10
• The mean is  2 , the variance (2 - /2), the mode  , the
median 2 ln(2).
• The Rayleigh distribution has important applications in miss
distance calculations, radar and other coherent signal systems.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-34
Distribution: Rayleigh (cont.)
• A ballistic shell is required to hit within 1 meter of a target.
• After several trials it is observed that the downrange and
crossrange miss distances are independent and Gaussian with
a standard deviation of 0.8 meters.
• Thus, the radial miss distance, the RSS (root sum squares) of
the downrange and crossrange, is Rayleigh distributed with
 = 0.64.
• The Ph is then the CDF of 1,
1
y
Ph  P(Y  1)  FY (1)  
e
0.64
0
EMIS 7300 Spring 2006
y2

1.28
dy  1  e
Copyright  2002 - 2005 Dr. John Lipp
1

1.28
 0.54
S3P2-35
Distribution: Cauchy
• Let X1 and X2 be zero mean (x = 0) independent Gaussian
random variables with equal variances x2.
• Then Y = X1 / X2 is Cauchy distributed with parameter  = 1.
 1
fY ( y ) 
  2  y2
• PDF:
0.35
0.3
0.2
Y
f (y)
0.25
0.15
0.1
0.05
0
-10
-8
EMIS 7300 Spring 2006
-6
-4
-2
0
y
2
Copyright  2002 - 2005 Dr. John Lipp
4
6
8
10
S3P2-36
Distribution: Cauchy (cont.)
• The Cauchy distribution has no mean or variance! Why?
• The mode and median are both zero.
EMIS 7300 Spring 2006
Copyright  2002 - 2005 Dr. John Lipp
S3P2-37
Homework
• Mandatory (answers in the back of the book):
4-1
(part a correct answer = 0.3679)
4-13
4-29
4-41
4-47
4-79
4-101
• Mandatory: Subtract the X and Y
sample means from your dart gun
data and compute the radial miss
distance and angle errors. Using
probability paper, attempt to
determine the appropriate PDF of
the radial and angle errors.
EMIS 7300 Spring 2006
R

Ri  ( xi  x ) 2  ( yi  y ) 2
Copyright  2002 - 2005 Dr. John Lipp
 yi  y 

 i  tan 
 xi  x 
1
S3P2-38