Download CONDITIONAL PROBABILITY, INDEPENDENCE AND BAYES

Math/Stat 352
Lecture 4
Section 2.3
Conditional Probability and
Independence
CONDITIONAL PROBABILITY and INDEPENDENCE
In many experiments we have partial information about the outcome,
when we use this info the sample space becomes smaller.
EXAMPLE. Roll a die. Events: A: score is odd={1, 3, 5}. B: score is 2.
C: score is 3
P(B)=1/6. Now, suppose we know A occurred. Then P(B given A)=0.
P(C)=1/6. Suppose A occurred, what is P(C)? The new sample space is
A ={1, 3, 5}. Then P(C given A)=1/3.
So, the conditional probability of C given A is the probability of C
relative to the probability of A. Formally:
P(C and A)
Probability of C given A = P(C |A)= ---------------- .
P(A)
Here, event A stands for the partial information, A is “the condition”.
Example:
Testing for disease: You are walking down the street and notice that
Dept. of Public Health is giving a free medical test for a certain
disease. The test is 99% reliable in the following sense: If a
person has the disease, there is a probability of 0.99 that the test
will give a positive response. If the person does not have the
disease, there is a probability of 0.02 that the test will give a
positive response. State the probabilities given in the problem as
conditional probabilities.
Solution.
Truth
Test result
Pos
Neg
Sick
correct
Healthy wrong
wrong
correct
Statistical Independence
Two events A and B are independent if the occurrence of one does
not affect the chances of the occurrence of the other.
EXAMPLES.

Toss 2 coins.
A: event that 1st comes up H;
independent events
B: event that 2nd comes up T.

Draw two cards from a deck without replacement.
A: event that 1st comes out red;
NOT independent events
B: event that second comes up red.
Statistical independence, contd.

Formally, events A and B are independent if and only if (iff )
P(A|B) = P(A) or P(B|A)=P(B).

Independence is a symmetric relation. If A is independent of B,
then B is independent of A.

Multiplication Rule. Events A and B are independent iff
P(A and B) = P(A) x P(B).

SUMMARY: For independent events A and B
P(A|B)=P(A and B)/P(B)=P(A)
Statistical independence, contd.

Example. Roll two fair dice. Find the probability that the score on
the first die will be odd and the score on the second die will be 5.
Solution.
Generalization of independence to many events
Events A1, A2, …, An are independent if the occurrence of any of them
does not change the probability of occurrence of any other.
Formally:
P ( A1 ∩ A2 ∩ A3 An) = P ( A1) P ( A2) ×  × P ( An)
Or , for any subset Aj1, Aj2, …, Ajk
P( Ai | Aj1 ∩ Aj 2 ∩  ∩ Ajk ) = P( Ai ) if
P( Aj1 ∩ Aj 2 ∩  ∩ Ajk ) ≠ 0.
Example
Of the microprocessors manufactured by a certain process, 20%
are defective. Five microprocessors are chosen at random.
Assume they function independently. What is the probability that
they all work?
8
Notes on independence
NOTE 1: If A and B are independent, then A and (not B), (not A)
and B, and (not A) and (not B) are also independent. Thus:
P(A and not B) =
P(not A and B)=
P(not A and not B)=
NOTE 2. If A and B and C are independent, then
P(A and B and C) =P(A) x P(B) x P(C),
that is Multiplication Rule extents to any number of events.
Summary of the Multiplication Rule
Multiplication Rule:
Always: P(A and B)=P(A|B)P(B)=P(B|A)P(A)
If A and B are independent: P(A|B)=P(A) and P(B|A)=P(B), so
If A and B are independent: P(A and P)= P(A)P(B).
Examples

Example. Sample two cards from a deck of 52 without
replacement. Find the probability that the first card is red and
the second is black.

Solution.

Example. Do the same problem, but sample with replacement.

Solution.
TOTAL PROBABILITY FORMULA
If A1, A2, …, An are mutually exclusive and cover the sample space, that is
A1 ∪ A2 ∪  ∪ An = S , then
for any event B we have
P ( B ) = P ( A1 ∩ B) + P( A2 ∩ B) +  + P( An ∩ B) =
P( B | A1) P( A1) + P( B | A2) P( A2) +  + P( B | An) P( An).
Note: Events A1, A2, …, An are sometimes called “exhaustive”, or a
partition of S.
Example.
Customers who buy a certain car have a choice of three engine sizes:
small, medium and large. Of all cars sold, 45% have the small engine,
35% have medium engine, and the remaining 20% have large engine.
Of cars with the small engine, 10% fail the emission test within 2 years
of purchase, while 12% of those with medium and 15% of those with
large engine fail the emission test within 2 years of purchase.
a.
What is the probability that a randomly chosen car will fail the
emission test within 2 years of purchase?
b.
Given a car that failed emission test within 2 years of purchase, what
is the probability that it was a car with medium engine?
Bayes’ Rule
Let A1, A2, …, An are a partition of the sample space S, such that P(Ai)>0
for all i. Let B be any event such that P(B) >0. Then
P ( Ai | B ) =
P ( B | Ai ) P ( Ai )
n
∑ P( B | Ak ) P( Ak )
k =1
In particular, for any two events A and B with nonzero probabilities, we
have
P( B | A) P( A)
P( A | B) =
.
P( B)
Examples
Diagnostic Test contd. Suppose the probability that a person has the disease is
0.005. If a person tests positive, what is the probability that the person has the
disease?
Flaws in cans. In a process that manufactures aluminum cans, the probability that
a can has a flaw on its side is 0.02, the probability that a can has a flaw on the
top is 0.03, and the probability that a can has a flaw on both the side and the
top is 0.01. What is the probability that a can will have a flaw on the side, given
that it has a flaw on the top?
Deliveries. A delivery company offers express and standard delivery services.
Seventy-five percent of parcels are sent by standard delivery, and 25% are sent
express. Of those sent standard, 80% arrive the next day, and of those sent
express, 95% arrive the next day. A record of a delivery is selected at random.
a. What is the probability that the parcel was shipped express and arrived the
next day?
b. What is the probability that it arrived the next day?
c. Given that the package arrived the next day, what is the probability that it was
sent express?
Example.
A lot of 10 components contains 3 that are defective. Two components
are drawn at random and tested. Let A be the event that the first
component drawn is defective, and let B be the event that the second
component drawn is defective. Find the following probabilities:
a.
b.
c.
d.
e.
f.
P(A)
P(B|A)
P(A and B)
P(Ac and B)
P(B)
Are A and B independent?
Example.
The probability that I will go to San Francisco the next
weekend is 0.5. The probability that my Friend Kate will go to
San Francisco on the same weekend is 0.3. We make our
travel plans independently. What are the following
probabilities:
A. Both of us will be in SF next weekend;
B. None of us will be in SF next weekend
C. At least one of us will be in SF next weekend
D. I will be there, but Kate will not.

Solution.
