Download Continuous Random Variable

Physical Fluctuomatics
3rd Random variable, probability distribution and
probability density function
Kazuyuki Tanaka
Graduate School of Information Sciences, Tohoku University
kazu@smapip.is.tohoku.ac.jp
http://www.smapip.is.tohoku.ac.jp/~kazu/
Physics Fluctuomatics (Tohoku
University)
1
Probability
a. Event and Probability
b. Joint Probability and Conditional
Probability
c. Bayes Formula, Prior Probability and
Posterior Probability
d. Discrete Random Variable and
Probability Distribution
e. Continuous Random Variable and
Probability Density Function
f. Average, Variance and Covariance
g. Uniform Distribution
h. Gauss Distribution
Physics Fluctuomatics (Tohoku
University)
Last Talk
Present Talk
2
Probability and Random Variable
We introduce a one to one mapping X(A) from every
events A to a mutual different real number. The
mapping X(A) is referred to as Random Variable of A.
The random variable X(A) is often denoted by just the
notation X.
Probability of the event X=x that the random variable X
takes a real number x is denoted by Pr{X=x}. Here, x is
referred to as the state of the random variable X．The set
of all the possible states is referred to as State Space.
If events X=x and X=x’ are exclusive of each other, the
states x and x’ are excusive of each other.
Physics Fluctuomatics (Tohoku
University)
3
Discrete Random Variable and
Continuous Random Variable
Discrete Random Variable:
Random Variable in Discrete State Space
Example:{x1,x2,…,xM}
Continuous Random Variable:
Random Variable in Continuous State Space
Example：(−∞,+∞)
Physics Fluctuomatics (Tohoku
University)
4
Discrete Random Variable and
Probability Distribution
Let us suppose that the sample W is expressed by
Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj
are exclusive of each other.
We introduce a one to one mapping X:Ai xi
(i=1,2,…,M).
If all the probabilities for the events X=x1, X=x2,…, X=xM are
expressed in terms of a function P(x) as follows:
PrX  x  Px x  x1 , x2 ,, xM 
Random Variable
State Variable
State
the function P(x) and the variable x is referred to as
Probability Distribution and State Variable, respectively.
Physics Fluctuomatics (Tohoku
University)
5
Discrete Random Variable and
Probability Distribution
Probability distributions have the following properties:
0  Pxi   1 i  1,2,, M 
M
 Px   1
i 1
i
Normalization Condition
Physics Fluctuomatics (Tohoku
University)
6
Average and Variance
Average of Random Variable X : μ
M
  E X    xi Pxi 
i 1
Variance of Random Variable X: σ2
M
  V X    xi    Pxi 
2
2
i 1
：Standard Deviation
Physics Fluctuomatics (Tohoku
University)
7
Discrete Random Variable and
Joint Probability Distribution
If the joint probability
Pr{(X=x)∩(Y=y)}= Pr{X=x,Y=y}
is expressed in terms of a function P(x,y) as
follows:
PrX  x, Y  y  Px, y 
P(x,y) is referred to as Joint Probability Distribution.
X
 
Y 
Probability Vector
 x
 
 y
Physics Fluctuomatics (Tohoku
University)
State Vector
8
Discrete Random Variable and
Marginal Probability Distribution
Let us suppose that the sample W is expressed by
Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj are
exclusive of each other.
We introduce a one to one mapping X:Ai xi (i=1,2,…,M).
M
PY  y    Pxi , y 
Marginal
Probability
Distribution
i 1
PY  y    P x, y  Simplified Notation
x
Summation over all the possible events in which every pair of events are exclusive of each other.
 P( x, y)  1
x
Normalization Condition
y
Physics Fluctuomatics (Tohoku
University)
9
Discrete Random Variable and
Marginal Probability
Marginal Probability of High Dimensional Probability Distribution
Marginal Probability Distribution
PY  y    Px, y, z, u 
x
z
u
X
Y
Z
U
Marginalize
Physics Fluctuomatics (Tohoku
University)
10
Independency of
Discrete Random Variable
If random variables X and Y are independent of each others,
Px, y   P1 xP2  y 
Joint Probability
Distribution of Random
Variables X and Y
Marginal Probability
Distribution of
Random Varuiable Y
 P1 ( x)   P2 ( y)  1
x
y
Probability Distrubution of
Random Variable Y
Probability Distribution of
Random Variable X
PY  y    Px, y   P2  y 
x
Physics Fluctuomatics (Tohoku
University)
11
Covariance of
Discrete Random Variables
Covariance of Random Variables X and Y
CovX , Y    xi   X y j  Y Pxi , y j 
M
N
i 1 j 1
 X  E[ X ]   xi Pxi , y j  Y  E[Y ]   yi Pxi , y j 
M
M
N
i 1 j 1
i 1 j 1
Cov[ X , X ]  V [ X ]
Covariance Matrix
N
Cov[Y , Y ]  V [Y ]
Cov[ X , Y ] 
 V[ X ]

R  
V[Y ] 
 Cov[Y , X ]
Physics Fluctuomatics (Tohoku
University)
12
Example of Probability Distribution
exp ax 
x  1
P( x) 
2 cosh a 
EX   tanh a
E[X]
VX   1  tanh a 
2
Physics Fluctuomatics (Tohoku
University)
0
a
13
Example of Joint Probability Distributions
exp axy
x  1, y  1
P( x, y) 
4 cosh a 
EX   0 VX   1
Cov[ X , Y ]  EXY 
 tanh a 
Physics Fluctuomatics (Tohoku
University)
Cov[X,Y]
0
a
14
Example of Conditional Probability Distribution
Conditional
Probability of Binary
Symmetric Channel
1 x , y
P( y x)  p
x  1, y  1
exp axy

1  p  
2 cosh a 
x,y
1 1 p 

a  ln 
2  p 
Physics Fluctuomatics (Tohoku
University)
15
Continuous Random Variable and
Probability Density Function
For a random variable X defined in the state space
(−∞,+∞), the probability that the state x is in the
interval (a,b) in expressed as
Pra  X  b  Pr   X  b Pr   X  a
F  x   Pr   X  x
Distribution Function
Pra  X  b  F b  F a     x dx
b
a
Probability Density Function
dF  x 
 x  
dx
Physics Fluctuomatics (Tohoku
University)
16
Continuous Random Variable and
Probability Density Function
 x  0    x  



 xdx  1
Normalization Condition
Physics Fluctuomatics (Tohoku
University)
17
Average and Variance of
Continuous Random Variable
Average of Random Variable X
  EX    x x dx


Variance of Random Variable X
  V X   
2


x     x dx

Physics Fluctuomatics (Tohoku
University)
2
18
Continuous Random variables and
Joint Probability Density Function
For random variables X and Y defined in the state
と probability
Y の状態空間
において状
space確率変数
(−∞,+∞),Xthe
that(−∞,+∞)
the state
vector (x,y)
態 xregion
と y が区間
(a,b)×(c,d)
にある確率
is in the
(a,b)(c,d)
is expressed
as
Pra  X  b   c  Y  d 

d
c

b
a
 x, y dxdy
Joint Probability Density Function
 
   x, y dxdy 1
Normalization Condition
 
Physics Fluctuomatics (Tohoku
University)
19
Continuous Random Variables and
Marginal Probability Density Function

Y  y     x, y dx

Marginal Probability Density Function of
Random Variable Y
Physics Fluctuomatics (Tohoku
University)
20
Independency of
Continuous Random Variables
Random variables X and Y are independent
of each other.
 x, y   1 x2  y 
Joint Probability
Density Function
of X and Y

  1 ( x)dx  1

   2 ( y)dy  1
Probability Density Function of Y
Probability Density Function of X

Marginal
Probability Density
Function Y
Y  y     x, y dx   2  y 

Physics Fluctuomatics (Tohoku
University)
21
Covariance of
Continuous Random Variables
Covariance of Random Variables X and Y
CovX , Y   
 





x


y



x
,
y
dxdy
X
Y
 
 X  E[ X ]  
Y
 

 E[Y ]   
 
 
 
Cov[ X , X ]  V [ X ]
Covariance Matrix
x x, y dxdy
y x, y dxdy
Cov[Y , Y ]  V [Y ]
Cov[ X , Y ] 
 V[ X ]

R  
V[Y ] 
 Cov[Y , X ]
Physics Fluctuomatics (Tohoku
University)
22
Uniform Distribution U(a,b)
Probability Density Function of Uniform Distribution
b  a 
 x   
 0
1
a  x  b 
   x  a, b  x  
ab
E X  
2
2

b  a
V X  
12
p(x)
(b-a)-1
Physics Fluctuomatics (Tohoku
University)
0 a
b
x
23
Gauss Distribution N(μ,σ2)
Probability Density Function of Gauss Distribution
with average μ and variance σ2
(  0)
 1
2
 x  
exp   2 x        x  
 2
 p(x)
2 2
1
E X   
VX   
The average and the variance are derived
by means of Gauss Integral Formula
2
0
μ
x
 1 2
 exp   2  d  2

Physics Fluctuomatics (Tohoku
University)
24
Multi-Dimensional Gauss Distribution
For a positive definite real symmetric matrix C, twoDimensional Gaussian Distribution is defined by
 1
x  X 
1 
 
  x, y  
exp   x   X , y  Y C 
 y  Y  
2 2 det C  2
1
   x  ,    y  
The covariance matrix is given in terms of the matrix C as follows:
Cov[ X , Y ] 
 V[ X ]

  C
V[Y ] 
 Cov[Y , X ]
by using the following d -dimentional Gauss integral formula
 

 1  T 1   
d
       exp   2 
C  d 

Physics Fluctuomatics (Tohoku
University)
2 
det C
25
Law of Large Numbers
Let us suppose that random variables X1,X2,...,Xn are
identical and mutual independent random variables with
average . Then we have
1
Yn  ( X 1  X 2    X n )   (n  )
n
Central Limit Theorem
We consider a sequence of independent, identical distributed
random variables, {X1,X2,...,Xn}, with average  and variance 2.
Then the distribution of the random variable
Yn 
1
( X1  X 2    X n )
n
tends to the Gauss distribution with average  and variance
2/n as n+.
Physics Fluctuomatics (Tohoku
University)
26
Summary
a. Event and Probability
b. Joint Probability and Conditional
Probability
c. Bayes Formula, Prior Probability and
Posterior Probability
d. Discrete Random Variable and
Probability Distribution
e. Continuous Random Variable and
Probability Density Function
f. Average, Variance and Covariance
g. Uniform Distribution
h. Gauss Distribution
Physics Fluctuomatics (Tohoku
University)
Last Talk
Present Talk
27
Practice 3-1
Let us suppose that a random variable X takes binary
values 1 and the probability distribution is given by
exp ax 
x  1
P( x) 
2 cosh a 
Derive the expression of average E[X] and variance V[X]
and draw their graphs by using your personal computer.
Physics Fluctuomatics (Tohoku
University)
28
Practice 3-2
Let us suppose that random variables X and Y take
binary values 1 and the joint probability distribution is
given by
exp axy
x  1, y  1
P( x, y) 
4 cosh a 
Derive the expressions of Marginal Probability
Destribution of X, P(X), and the covariance of X and
Y, Cov[X,Y].
Physics Fluctuomatics (Tohoku
University)
29
Practice 3-3
Let us suppose that random variables X and Y take
binary values 1 and the conditional probability
distribution is given by
1 x , y
P( y x)  p
1  p 
 x,y
Show that it is rewritten as
exp axy
P( y x) 
2 cosh a 
Hint
p  exp ln p
 x, y
1 1 p 

a  ln 
2  p 
1
 1  xy
2
x  1, y  1
cosh(c) is an even function for any real number c.
Physics Fluctuomatics (Tohoku
University)
30
Practice 3-4
Prove the Gauss integral formula:


0

 1 2
exp    d 
2
 2 
Hint
R
R
 1 2
 1 2
 1 2
exp    d  2 lim  exp    d
 exp   2  d  Rlim
   R
R   0
 2 
 2 

 2 lim
R  
R
R
0
0
 
 1 2 1 2
exp      d d  2 lim
R 
2 
 2
Physics Fluctuomatics (Tohoku
University)
 1 2 
r 1 exp   2 r dr
31
Practice 3-5
Let us suppose that a continuous random variable X takes any
real number and its probability density function is given by
 1
2
p x  
exp   2 x    
 2

2 2
1
   x  
Prove that the average E[X] and the variance V[X] are given by
E X   
VX   
2
Draw the graphs of p(x) for μ=0, σ=10, 20, 40 by using
your personal computer.
Physics Fluctuomatics (Tohoku
University)
32
Practice 3-6
Make a program for generating random numbers of
uniform distribution U(0,1). Draw histgrams for N
generated random numbers for N=10, 20, 50, 100 and
1000.
1
x
rand()
randmax
In the C language, you can use the function rand()
that generate one of values 0,1,2,…,randmax,
randomly. Here, randmax is the maximum value
of outputs of rand().
Physics Fluctuomatics (Tohoku
University)
33
Practice 3-7
Make a program that generates random numbers of Gauss
distribution with average  and variance σ2. Draw
histgrams for N generated random numbers for N=10, 20,
50, 100 and 1000.
Hint:
For n random numbers x1,x2,…,xn generated by any probability
distribution, (x1+x2+…+xn )/n tends to the Gauss distribution with
average m and variance σ2/n for sufficient large n. [Central Limit
Theorem]
First we have to generate twelve uniform random numbers x1,x2,…,x12
in the interval [0,1].
Gauss random
number with average
1
2
12
0 and variaince 1
  x  x   x  6
σξ+μ generate Gauss random numbers with average μ and variance σ2
Physics Fluctuomatics (Tohoku
University)
34
Practice 3-8
For any positive integer d and d d positive definite
real symmetric matrix C, prove the following ddimensional Gauss integral formulas:



 1   T 1  
       exp   2    C   
 
Hint:


 
d 

2 d det C
By using eigenvalues λi and their corresponding

eigenvectors ui (i=1,2,…,d) of the matrix C, we have
 1 0

 0 2
C  U 0 0

 
0 0

0

0
3




0

0

0
0 U 1

 
d 
 

U  u1 , u1 ,, ud 
Physics Fluctuomatics (Tohoku
University)
35
Practice 3-9
We consider continuous random variables X1,X2,…,,Xd.
The joint probability density function is given by

p x  
 1   T 1   
exp   x    C x   
d

2  det C  2
1


 x 1 


 
   x2 
d 


x




,



  


 
x 


 d



Prove that the average vector is  and the covariance
matrix is C.
Physics Fluctuomatics (Tohoku
University)
36