Download Lecture 8: Continuous random variables, expectation and variance

Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Lecture 8: Continuous random variables,
expectation and variance
Lejla Batina
Institute for Computing and Information Sciences – Digital Security
Radboud University Nijmegen
Version: spring 2012
Lejla Batina
Version: spring 2012
Wiskunde 1
1 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Outline
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Lejla Batina
Version: spring 2012
Wiskunde 1
2 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
This course: still to happen
• Lectures:
• Normal lectures this week
• Next week’s schedule:
• Tuesday, 19-06, 13:45-14:30, last lecture + info about exams
• Tuesday, 19-06, 14:45-15:30, “responsiecollege”
• Thursday, 21-06, 8:45-10:30 (returning last 2 homeworks, see
below)
• Homeworks:
• Homework 8: as usual, due Monday 18-06
• Homework 9 (bonus): posted on Wednesday 13-06, due
Wednesday 20-06 (12:00!)
• Examination:
• tentamen: Wednesday, 27-06, 8:30-10:30, HG.00.304
• hertentamen: Thursday, 09-08, 13:30-15:30, HG.00.068
Lejla Batina
Version: spring 2012
Wiskunde 1
3 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Random variable
Definition
Let S be a sample space and A is an event from S. A random
variable is a real function defined on a S, f : S → R.
A random variable that takes on a finite or a countably infinite
number of values is called a discrete random variable, otherwise we
have a non-discrete or continuous random variable.
Lejla Batina
Version: spring 2012
Wiskunde 1
4 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Probability distribution
Definition (Discrete probability distributions)
Let X be a discrete random variable obtaining values x1 , x2 , . . . , to
which we assign probabilities P(X = xk ) = f (xk ), k = 1, 2, . . ..
The probability function (or probability distribution) is given by:
P(X = x) = f (x); for x = xk we get f (xk ) and for x 6= xk
f (x) = 0.
f (x) is a probability function if:
1
2
f (x) ≥ 0.
P
x f (x) = 1.
Lejla Batina
Version: spring 2012
Wiskunde 1
5 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Distribution functions
Definition
For a random variable X , the (cumulative) distribution function is
defined by: F (x) = P(X ≤ x) for x ∈ R.
The distribution function F (x) has the following properties:
• F (x) ≤ F (y ) if x ≤ y .
• limx→−∞ F (x) = 0 and limx→+∞ F (x) = 1.
• limh→0+ F (x + h) = F (x), ∀x, i.e. F (x) is continuous from the
right.
Lejla Batina
Version: spring 2012
Wiskunde 1
6 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Distribution functions for discrete random variables
Let X be a discrete random variable X , which takes on the values
x1 , x2 , . . . , xn . Then: P
P
F (x) = P(X ≤ x) = u≤x f (u) = xi ≤x pi , in more detail

0, − ∞ < x < x1 ,





f (x1 ), x1 ≤ x < x2 ,


f (x1 ) + f (x2 ), x2 ≤ x < x3 ,
F (x) =







Lejla Batina
...
f (x1 ) + f (x2 ) + . . . + f (xn ), xn ≤ x < ∞.
Version: spring 2012
Wiskunde 1
7 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Example
Example
A coin is tossed twice, then a sample space is
S = {HH, HT , TH, TT }, and X - number of heads.
X
0
1
2
f (x) 1/4 1/2 1/4
The distribution function F (x):

0, − ∞ < x < 0,



 1/4, 0 ≤ x < 1,
F (x) =

3/4, 1 ≤ x < 2,



1, 2 ≤ x < ∞.
Lejla Batina
Version: spring 2012
Wiskunde 1
8 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Distribution functions for non-discrete random variables
Definition (Distribution function for continuous random
variables)
A non-discrete random variable is continuous if its distribution function F
can be represented as:R
x
F (x) = P(X ≤ x) = −∞ f (u)du for −∞ < x < ∞, where the function
f has the following properties:
• f (x) ≥ 0,
R∞
• −∞ f (x)dx = 1.
f is called the probability density function of the random variable X .
Ra
It is evident that f (x) = dFdx(x) and F (a) := P(X ≤ a) = −∞ f (x)dx.
R x2
P(x1 < X ≤ x2 ) = F (x2 ) − F (x1 ) = x1 f (x)dx, so the area under f (x)
in the interval (x1 , x2 ) represents the probability that the random variable
X lies in the interval (x1 , x2 ) as above.
Lejla Batina
Version: spring 2012
Wiskunde 1
9 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Some properties
Theorem
1
F (+∞) = 1 and F (−∞) = 0
Proof: F (+∞) = P(X ≤ +∞) = P(S) = 1.
2
F is a non-decreasing function of x so:
x1 < x2 ⇒ F (x1 ) ≤ F (x2 ).
3
If F (x0 ) = 0 ⇒ F (x) = 0, ∀x ≤ x0 .
4
P(X > x) = 1 − F (x)
5
F (x) is continuous from the right so F (x + ) = F (x).
6
P(x1 < X ≤ x2 ) = F (x2 ) − F (x1 ).
Lejla Batina
Version: spring 2012
Wiskunde 1
10 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Uniform distribution
X is said to be uniformly distributed in (a, b), −∞ < a < b < ∞,
if its density
 function is:
1

, a ≤ x ≤ b,
b−a
f (x) =

0, otherwise
The distribution
function of X is given by:

0,
x < a,


 x −a
, a ≤ x ≤ b,
F (x) =

b−a


1, x > b
Rx
Rx 1
1
F (x) = a f (x)dx = a b−a
dx = b−a
x|xa =
R∞
Rb 1
Also, −∞ f (x)dx = a b−a dx = 1.
Lejla Batina
Version: spring 2012
x−a
b−a .
Wiskunde 1
11 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Normal (Gaussian) distribution
It is one of the most commonly used probability distribution for
applications.
When an experiment is repeated numerous times then the random
variable representing the average or mean tends to have a normal
distribution as the number of experiments becomes large.
This fact is also known as the central limit theorem and it is very
important for many statistical techniques.
Many physical values follow this distribution e.g. heights, weights.
Also often used in social sciences and for grades, errors, etc.
Lejla Batina
Version: spring 2012
Wiskunde 1
12 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Normal distribution:definition
Definition
We say that X is a normal or Gaussian random variable with
parameters µ and σ (and we write X ∼ N(µ, σ)) if its density
function is given by:
f (x) = σ√12Π e −
deviation.
(x−µ)2
2σ 2
, where µ and σ are the mean and standard
The distribution function is then given by:
(v −µ)2
R∞
F (x) = P(X ≤ x) = σ√12Π −∞ e − 2σ2 dv
The integral cannot be computed exactly, so we use tables of
cumulative probabilities for a special normal distribution to
calculate the probabilities.
Lejla Batina
Version: spring 2012
Wiskunde 1
13 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
The normal distribution:
Radboud University Nijmegen
source Wikipedia
The parameter µ determines the location of (the axe of symmetry
of) the distribution while σ determines the width of the curve.
Lejla Batina
Version: spring 2012
Wiskunde 1
14 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Standardizing normal distribution
N(0, 1) is often called the standard normal distribution.
Theorem
If X is a normal random variable with mean µ and standard
deviation σ, then Z = X σ−µ is a standard normal random variable.
Lejla Batina
Version: spring 2012
Wiskunde 1
15 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Mathematical Expectation
Definition
For a discrete random variable X having the possible values
x1 , x2 , . . . , xn the expectation of X is defined as:
E (X
P) = x1 · P(X = x1 ) + x2 · P(X = x2 ) + . . . + xn · P(X = xn ) =
= nj=1 xj · P(X = xj ).
As a special case, if all the probabilities are equal, we get:
n
,
E (X ) = x1 +x2 +...+x
n
which is called the arithmetic mean of x1 , x2 , . . . , xn .
For a continuous random variable X having density function f (x),
the expectation
of X is defined as:
R∞
E (X ) = −∞ xf (x)dx provided that the integral converges.
Lejla Batina
Version: spring 2012
Wiskunde 1
16 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
History 1/2
The problem of points, also called the problem of division of the
stakes:
Consider a game of chance with two players who have equal
chances of winning each round. The players contribute equally to a
prize pot, and agree in advance that the first player to have won a
certain number of rounds will collect the entire prize.
What happens if the game is interrupted by external circumstances
before either player has won? How does one then divide the pot
fairly?
Lejla Batina
Version: spring 2012
Wiskunde 1
17 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
History 2/2
It is expected that the division should depend somehow on the
number of rounds won by each player, such that a player who is
close to winning gets a larger part of the pot. But the problem is
not merely one of calculation; it also includes the explanation on a
“fair” division.
Lejla Batina
Version: spring 2012
Wiskunde 1
18 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Examples
Example
In a lottery there are 200 prizes of 5 Euros, 20 of 25 Euros, 5 of 100
Euros. Assuming that 10000 tickets will be issued and sold, what is a fair
price to pay for a ticket.
X - ran. var. denoting the amount of money to be won by a ticket
X
5
25
100
0
P(X = x) 0.02 0.002 0.0005 0.9775
P(X = 5) = 10200
000 = 0.02
P(X = 25) = 1020000 = 0.002
P(X = 100) = 10 5000 = 0.0005.
E (X ) = 5 · 0.02 + 25 · 0.002 + 100 · 0.0005 + 0 = 0.2, so a ticket should
cost 20 cents.
Lejla Batina
Version: spring 2012
Wiskunde 1
19 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Example (Cauchy distribution)
Let X be a random variable with the following density function:
f (x) = x 2C+1 , −∞ < x < ∞.
• Find the value of C .
• Find the probability that X 2 lies between 31 and 1.
R∞
The condition is −∞ f (x)dx = 1. Then we have:
R∞
R ∞ Cdx
R B dx
f
(x)dx
=
=
C
lim
B→∞
2
−∞
−∞ x +1
−B x 2 +1 =
π
π
B
= C limB→∞ arctan |−B = C [ 2 − (− 2 )] = C π ⇒ C = π1 .
When
1
3
P( 13
x2
=
1
π
√
3
3
or
3√ ≤ x
√3
− 3
3
3 ) + P( 3 ≤
≤ x 2 ≤ 1 ⇒ −1 ≤ x ≤ −
≤ √ ≤ 1) = P(−1 ≤ x ≤
R − 33 dx
R1
+ π1 √3 x 2dx+1 = 61 .
−1
x 2 +1
Lejla Batina
√
≤ 1. Then
x ≤ 1) =
3
Version: spring 2012
Wiskunde 1
20 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Example
A continuous random variable X has probability density function
given by:
(
2e −2x , x > 0,
f (x) =
0, x ≤ x0,
Find E (X ).
E (X ) =
R∞
−∞ xf (x)dx
=
R∞
0
x2e −2x dx = 2
R∞
0
xe −2x dx = 2I |∞
0 .
R
R
I = xe −2x dx =
[u = x, du = dx, v = e −2x dx = − 21 e −2x ] =
R
= − x2 e −2x + 12 e −2x dx = − x2 e −2x − 14 e −2x
1
1
1
E (X ) = 2[− x2 e −2x − 14 e −2x ]|∞
0 = 2[0 − 4 · 0 + 0 + 4 ] = 2 .
Lejla Batina
Version: spring 2012
Wiskunde 1
21 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Some properties
For X and Y random variables, we have:
• E (X + Y ) = E (X ) + E (Y )
• E (αX ) = αE (X ), α ∈ R
• If X and Y are independent random variables, then:
E (XY ) = E (X ) · E (Y ).
Lejla Batina
Version: spring 2012
Wiskunde 1
22 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Variance and standard deviation
Definition (Variance)
Let X be a random variable withPmean µ. Then the value:
σ 2 = Var (X ) = E [(X − µ)2 ] = nj=1 (xj − µ)2 f (xj ) represents the
average square deviation of X around its mean. This value is called
the variance of the random variable X .
In the special case where all the probabilities are equal, we have:
2
2 +...(x −µ)2 ]
n
.
σ 2 = E [(x1 −µ) +(x2 −µ)
n
For a Rcontinuous variable X with a density function f (x):
∞
σ 2 = −∞ (x − µ)2 f (x)dx.
p
The value σ = E [(X − µ)2 ] is called the standard deviation of X .
The variance is a measure of the dispersion or scatter of the values
(of the random variable considered) around the mean µ.
Lejla Batina
Version: spring 2012
Wiskunde 1
23 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Theorems on Variance
Theorem
1
Var (X ) = 0 ⇔ X = C , C ∈ R.
2
Var (αX ) = α2 Var (X )
3
Var (X + C ) = Var (X )
4
Var (X ) = E (X 2 ) − E (X )2
5
If X and Y are independent random variables then:
Var (X + Y ) = VarX + VarY
Proof of 4.
Var (X ) = E [(X − µ)2 ] = E [X − 2µX + µ2 ] =
E [X 2 ] − 2µE [X ] + µ2 = E [X 2 ] − 2µ2 + µ2 = E [X 2 ] − E [X ]2 .
Lejla Batina
Version: spring 2012
Wiskunde 1
24 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Expectation and variance of the uniform distribution
Example
µ=
R∞
−∞ xf (x)dx
E (X 2 ) =
R∞
−∞ x
=
Rb
xdx
a b−a
2 f (x)dx
=
=
Rb
a
1 x2 b
b−a 2 |a
dx
x 2 b−a
=
=
a+b
2 .
1
3 b
3(b−a) x |a
=
b 3 −a3
3(b−a)
=
a2 +ab+b 2
.
3
Then it follows:
Var (X ) = E (X 2 ) − E (X )2 =
1
(a − b)2 .
= 12
Lejla Batina
Version: spring 2012
a2 +ab+b 2
3
− 14 (a2 + 2ab + b 2 ) =
Wiskunde 1
25 / 26
Continuous Random Variables and Probability Distributions
Examples of Distributions
Expectation and Variance
Radboud University Nijmegen
Example
Let Z =
X −µ
σ .
Find the expectation and variance of Z .
E (Z ) = E ( X σ−µ ) = σ1 [E (X − µ)] = σ1 [E (X ) − µ] =
= σ1 [E (X ) − E (X )] = 0, since E (X ) = µ.
Var (Z ) = Var ( X σ−µ ) =
1
E [(X
σ2
− µ)2 ] = 1, since
E [(X − µ)2 ] = σ 2 .
Lejla Batina
Version: spring 2012
Wiskunde 1
26 / 26