Download lect_05.pps

Lecture (5)
Introduction to Probability
Theory
and
Applications
Experiments
Experiment: a process that generates welldefined outcomes.
Experiment
Roll a dice
outcomes
1,2,3,4,5,6
Sample space: all possible outcomes.
For the “Roll a dice” example: S={1,2,3,4,5,6}.
Sample point: any particle outcome.
In the example {1}.
Events
An event is a collection of sample points (or,
an event is a subset of a sample space)
(1) Rolling two dices: (a) the sum of the
numbers that come up is odd, (b) the
numbers that come up are 3 and 6, etc.
(2) Tossing two coins: at least one of them
is head (it is a collection of the following
sample points (H,T), (T,H) and (H,H)).
Coin Tossing
Sometimes the experiment consists of
several steps. In such a case, a tree
diagram is handy. Consider the experiment
of tossing two coins. S={(H,H), (H,T), (T,H),
(T,T)}
Step 1
Step 2
First Coin Toss
Second Coin Toss
Sample Point
(H,H)
Head
Head
Tail
Tail
(H,T)
Head
(T,H)
Tail
(T,T)
T
Complement of an event: Given an event A,
the complement of A is defined to be the
event of all sample points that are not in A.
The complement of A is denoted by Ac
Venn Diagram
Sample
space S
Ac
Event A
Rolling two dices (Example)
7
event Ac
6
event A
5
4
3
2
1
0
0
1
2
3
4
5
6
7
Event A – at least one dice shows the number 1
Thus, event Ac – none of the dices shows the
number 1
Union of two events
the union of A and B is the event containing
all sample points belonging to A or B or
both. The union is denoted by
A B
Sample
space S
Event A
Event B
Rolling two dices (Example)
7
6
event A
event B
5
4
3
2
1
0
0
1
2
3
4
5
6
7
Event A – at least one dice shows the number 1
Event B – the sum of the numbers is at most 4
Intersection of two events
Given two events A and B, the intersection of A and
B is the event containing the sample points
belonging to both A and B. The intersection is
denoted by
A B
Sample
space S
Event A
Event B
A B
Rolling two dices (Example)
7
event B
6
event A
5
4
3
A B
2
1
0
0
1
2
3
4
5
6
7
Event A – at least one dice shows the number 1
Event B – the sum of the numbers is at most 4
The intersection (1,1), (1,2), (1,3), (2,1) and (3,1)
Mutually exclusive events: two events are
said to be mutually exclusive if the events
has no sample points in common.
Sample
space S
Event A
Definitions
Event B
Events
Rolling two dices
7
event B
6
event A
5
4
3
2
1
0
0
1
2
3
4
5
6
7
Event A – at least one dice shows the number 1
Event B – the sum of the numbers is bigger than
10
Probability
Probability is a number expressing the likelihood
that a specific event will occur.
Let Ei be a specific event (for example, the
sum of the numbers on two dices is 2.)
Let Pr( Ei ) be the probability of this event.
Axioms of probability:
1. 0  Pr( Ei )  1
2.
 Pr( E )  1
i
all possiblei
for all i
Assessing Probability
The probability of occurrence of a given event
can be assessed as the ratio of the number of
occurrences to the total number of possible
occurrences and non-occurrences of the event.
Occurrence of an event is called “ Success”
Non-occurrence of the event is called “Failure”
The number of successes in N trials = n
The relative frequency of successes = n/N
n
n
p (X )  lim 
N
N  N
Assessing Probability (Cont.)
n
n
p (X )  lim 
N
N  N
Fair Coin:
Head (H) or Tail (T)
p (H )  p (T )  0.5
nH
nT
p (H )  lim
 0.5, p (T )  lim
 0.5
N  N
N  N
Long Run Behavior of Coin Tossing
0.6
0.5
0.4
10
Figure 4.1.1
1
2
10
10
Number of tosses
3
10
4
Proportion of heads versus number of tosses
for John Kerrich's coin tossing experiment.
From Chance Encounters by C.J. Wild and G.A.F. Seber, © John Wiley & Sons, 2000.
Probability ( Ex.)
Coin: Sample Space {H, T}
Do the following experiments with one coin:
n=5
n=10
n=20
n=50
n=100
Calculate p(H), p(T).
Draw the convergence curve between n and p(H), p(T).
Basic Probability Laws
Pr( S )  1
Thus,
where S is the set of all the sample points
Pr( A)  Pr( A )  1
c
Example
A “at
is the
event
least one
Tossing
two(tossing
coins: Atwo
is coins):
the event
least
one“at
of them
is
of them
head”,
and
Ac “none
is the event
head”,
andisthus
Ac is
thethus
event
of the“none
coins of
is the
coins is head”
head”
1
c
3
and Pr A c  {(T , T )}  1
Pr  A  {( H , T ), (T , H ), ( H , H )} 3 and
Pr  A  {( H , T ), (T , H ), ( H , H )}  4
4
3 31 1
c
c
Thus,
AAPr Pr
A A   1 1
Thus,Pr Pr
4 44 4
  


Pr A  {(T , T )}  4
4
Basic Probability Laws
Pr( A  B)  Pr( A)  Pr( B)  Pr( A  B)
Let’s start with two mutually exclusive events.
In this case, Pr( A  B )  0 and
Pr( A  B)  Pr( A)  Pr( B)
Rolling two dices: Event A – at least one dice shows the
number 1, and event B – the sum of the numbers is bigger than 10
Basic Probability Laws (Example Solution)
Rolling two dices: Event A – at least one dice shows the
number 1, and event B – the sum of the numbers is bigger than 10
11
Pr( A) 
36
Dice #1
Dice #2
Dice #1
Dice #2
3
14
and Pr( B) 
thus Pr( A  B) 
36
36
Basic Probability Laws
Pr(A  B )  Pr(A or B )
= Pr( A )  Pr( B )  Pr( A  B )
Why do we need to subtract Pr( A  B) when the two
events are not mutually exclusive? Because otherwise
we would double-count it—both Pr( A) and Pr(B)
include it.
Rolling two dices: Event A – at least one dice shows the
number 1, and event B – the sum of the numbers is at most 4
Basic Probability Laws (Example Solution)
Rolling two dices: Event A – at least one dice shows the
number 1, and event B – the sum of the numbers is at most 4
11
6
5
Pr( A) 
, Pr( B ) 
and Pr(A  B) 
36
36
36
12
Pr( A  B) 
36
Dice #1
Dice #2
Conditional Probability
One of the most important concepts. It is denoted as
Pr( A | B) which means “the probability of event A given
the condition that event B has occurred.”
Rolling two dices: Event A – at least one dice shows the
number 1, and event B – the sum of the numbers is at most 4.
What is the probability of event A?
Pr( A) 
And what is the probability of event A
if we know that event B has occurred?
Pr( A | B) 
Pr( A  B)  Pr( A)  Pr( B)
How did you calculate Pr( A | B) ?
Conditional Probability (Cont.)
Pr( A  B)
Pr( A | B) 
Pr( B)
Event B
Event A
A B
Event B has occurred:
Event A and B with
probability Pr( A  B)
Event B with
probability Pr(B)
General Multiplication Law
Pr(A  B )  Pr(A and B )  Pr(A , B )
 Pr( A | B ) Pr( B )
Multiplication Rule for Independent Events
Independent events are events in which the
occurrence of the events will not affect the
probability of the occurrence of any of the other
events.
The multiplication Rule for Independent
Events (Example)
Example:
Picking a color from a set of crayons, then tossing a
die. Separately, each of these events is a simple
event and the selection of a color does not affect
the tossing of a die.
If the set of crayons consists only of red, yellow,
1
and blue, the probability of picking red is 3 . The
1
probability of tossing a die and rolling a 5 is . But
6
the probability of picking red and rolling a 5 is given
by:
Pr(red  5)  Pr(red)  Pr(5)
1 1 1
  
3 6 18
The multiplication Rule for Independent
Events (Example) Cont.
This can be illustrated using a “tree” diagram.
Since there are three choices for the color and six
choices for the die, there are eighteen different
results. Out of these, only one gives a combination
of red and 5. Therefore, the probability of picking
a red crayon and rolling a 5 is given by:
Pr(red  5)  Pr(red)  Pr(5)
1 1 1
  
3 6 18
The multiplication Rule for Independent
Events
The general formula states:
Pr(A  B )  Pr(A | B ) Pr(B )
The multiplication rule for independent events can
be stated as:
Pr(A  B )  Pr(A )  Pr(B )
This rule can be extended for more than two
independent events:
Pr(A  B  C ,etc .)  Pr(A )  Pr(B )  Pr(C ),etc .
Multiplication Rule for Dependent Events
The occurrence of one event affects the probability
of the occurrence of other events.
An example of dependent events:
Picking a card from a standard deck then picking
another card from the remaining cards in the deck.
For instance, what is the probability of picking two
kings from a standard deck of cards? The 4 1
probability of the first card being a king is 52  13 .
However, the probability of the second card depends
on whether or not the first card was a king.
Multiplication Rule for Dependent Events
(Example) Cont.
If the first card was a king then the probability of
the second card being a king is 3  1 .
51 17
If the first card was not a king, the probability of
4
the second card being a king is .
51
Therefore, the selection of the first card affects
the probability of the second card.
Multiplication Rule for Dependent Events
(Example) Cont.
The multiplication rule that include dependent events
reads:
Pr(A  B )  Pr(A | B ) Pr(B )
Pr(A  B )  Pr(B | A ) Pr(A )
Example 2
In a group of 25 people 16 of them are married and 9 are single.
What is the probability that if two people are randomly selected
from the group, they are both married?
If A represents the first person chosen is married and
B represents the second person chosen is married then:
Here, P (B|A) is now the event of picking another married person
from the remaining 15 married persons. The probability for the
selection made in B is affected by the selection in A.
16 15 2
Pr( A  B ) 


25 24 5
Estimating Probability form Histogram
36
27
1.3
17.3
9
8
1.3
Probability that Q is 10,000 to 15, 000 = 17.3%
Prob that Q < 20,000 = 1.3 + 17.3 + 36 = 54.6%
Estimating Probability form Cumulative
Histogram
F(x1) - F(x2)
Continuous
Discrete
F(x1) = P(x < x1)
Prob that Q < 20,000 = 1.3 + 17.3 + 36 = 54.6%
Prob that Q = 20,000, = 80.6-54.6 = 27%
Bivariate Distributions
• The bivariate (or joint) distribution is used when
the relationship between two random variables is
studied.
• The probability that X assumes the value x, and Y
assumes the value y is denoted
p(x,y) = P(X=x and Y = y)
Bivariate Distributions
The joint probabilit y function
satisfies the following conditions :
1. 0  p(x, y)  1
2.
  p(x, y)  1
all x all y
Bivariate Distributions
• Example
– X and Y are two variables. Let X and Y denote
the yearly runoff and rainfall (inches)
respectively.
– The bivariate probability distribution is
presented next.
Bivariate Distributions
Example
continued
0.42
p(x,y)
Y (in)
0
5
10
0.21
X (in)
5
.42
.06
.02
0
.12
.21
.07
0.12
0.06
0.06
0.07
0.02
0.01
Y
X=0
y=0
0.03
y=5
y=10
X=5
X=10
X
10
.06
.03
.01
Marginal Probabilities
• Example- continued
– Sum across rows and down columns
p(0,0)
p(0,5)
p(0,10)
Y
0
5
10
p(x)
0
.12
.21
.07
.40
X
5
.42
.06
.02
.50
The marginal probability P(X=0)
10
.06
.03
.01
.10
p(y)
.60
.30
.10
1.00
P(Y=5), the
marginal
probability.
Conditional Probability
p (X  x and Y  y )
p (X  x |Y  y ) 
p (Y  y )
Example - continued
X
Y
0
5
10
p(x)
P (X  0 and Y  5) .21

 .7
P (Y  5)
.30
P (X  5 and Y  5) .06
P (X  5 |Y  5) 

 .2
P (Y  5)
.30
P (X  10 and Y  5) .03
P (X  10 |Y  5) 

 .1
P (Y  5)
.30
0
.12
.21
.07
.40
5
.42
.06
.02
.50
P (X  0 |Y  5) 
The sum is
equal to 1.0
10
.06
.03
.01
.10
p(y)
.60
.30
.10
1.00
Counting Techniques: Permutations
Permutations:
It is an arbitrary ordering of a number of different objects
using all of them.
If we want to permute n different objects in r arrangements:
The number of permutations of n different objects is the
number of different arrangements in which r (r<n) of these
objects can be placed, with attention given to the order of
the items in each arrangement.
n!
permutation: n Pr  n(n  1)(n  2)...(n  r  1) 
(n  r )!
If n=r,
n!
permutation: n Pn  n(n  1)(n  2)...(n  n  1)   n !
0!
The number of permutations of n different objects of groups
of which n_i are alike, is
n!
permutation=
n1 !n2 !n3 !...
Example 1
Example: with objects a, b, and c how many permutations can
we do in 2 arrangements?:
ab, ac, ba, bc,
ca,
cb
First place: we have 3 choices.
Second place we have two choice.
Altogether =3!/(3-2)!=6 permutation of three objects in two
arrangements.
Example 2
Example: with objects a, b, and c we can produce six
permutations:
abc, acb, bac, bca,
cab,
cba
First place: we have 3 choices.
Second place we two choices.
Third place we one choice.
Altogether 3*2*1 = 3! =6 permutation of three objects.
For n objects we get n*(n-1)*(n-2)………2*1=n! permutations
Example 3
The number of permutations of n objects consisting of groups
of which n_i are alike, is
n!
permutation=
, n  n1  n2  n3 ...
n1 !n2 !n3 !...
The number of permutations of letters in the word
“STATISTICS” =50400.
n(number of letters)  10
n1 (number of S's)  3
n2 (number of T's)  3
n3 (number of I's)  2
n4 (number of A's)  1
n5 (number of C's)  1
10!
 50400
3!3!2!1!1!
Counting Techniques: Combinations
Combinations:
The number of combinations of n different objects r at a
time (n>r) is the number of different selection that can be
made of r objects out of n, without giving attention to the
order of arrangement within each selection.
 n  n Pr
n!
combination: n Cr    

r ! (n  r )!r !
r
 n 
n!
n!
n Pn  r


 n Cr
n Cn  r  

 n  r  (n  r )! r !(n  r )! (n  r )!r !
Stirling formula
n!
n n
2n .n e
Example 1
Example: with objects a, b, and c we can produce six
permutations:
abc, acb, bac, bca,
cab,
3!=3*2*1=6
But three combinations:
abc=cba=acb=bca=bac=cab
3!/3!.0!=3*2*1/3*2*1=1
cba
Example 2
Example:
Country A: 7 watersheds, and Country B: 4 Watersheds.
Five watersheds have to be selected for research 3 from A
and 2 from B. How many different ways this choice can be
made?
Solution:
This is a problem of combinations because the order of the
watersheds is not important.
A: n= 7, r= 3
B: n=4, r=2
7! 4!
combinations: 7C 3 . 4C 2 
.
 210
4!3! 2!2!
7! 4!
permutations: 7 P3 . 4 P2  .  25200
4! 2!