Download Lecture 4

PHP2510: Principles of Biostatistics & Data Analysis
Lecture IV: Discrete probability distributions
PHP 2510 – Lec IV
1
Random Variable
A random variable is a variable that takes random values. In other
words, it is essentially a random number.
Example. Toss a fair coin 3 times. The sample space of all
possible sequences is
Ω = {hhh, hht, hth, thh, htt, tht, tth, ttt}
Examples of random variables:
X
=
number of heads
Y
=
number of consecutive heads
Z
=
whether there is any heads (1 if Yes, 0 if Not)
PHP 2510 – Lec IV
{0, 1, 2, 3}
{0, 1, 2, 3}
{0,1}
2
Random Variable
• Discrete random variable takes on a finite (or countable)
number of distinct values, such as the number of illnesses in a
year.
• Continuous random variables take on values along a
continuum, such as time until an event, or height of a randomly
selected person.
Our focus today is on discrete random variables
PHP 2510 – Lec IV
3
Probability mass function
A probability mass function (PMF) describes the probability
of each value of a random variable.
Example. Let X be the number of heads in three tosses of a fair
coin. The PMF of X is
P (X = 0)
= 1/8
{T T T }
P (X = 1)
= 3/8
{HT T, T HT, T T H}
P (X = 2)
= 3/8
{HHT, T HH, HT H}
P (X = 3)
= 1/8
{HHH}
Denominator: sampling (H,T) with replacement three times: 23 = 8
PHP 2510 – Lec IV
4
Probability mass function
Example. Let Y be the number of consecutive heads in three
tosses of a fair coin.
P (Y = 0) =
1/8
{T T T }
P (Y = 1) =
4/8
{HT T, T HT, T T H, HT H}
P (Y = 2) =
2/8
{HHT, T HH}
P (Y = 3) =
1/8
{HHH}
Example. Let Z = 1 if 3 heads are tossed, and Z = 0 otherwise.
The PMF of Z is
PHP 2510 – Lec IV
P (Z = 0) =
7/8
P (Z = 1) =
1/8
{HHH}
5
.
Cummulative Distribution Function
The probability mass function (PMF) is usually denoted by
p(x) = P (X = x). For a discrete variable having outcomes
x1 , x2 , . . ., the PMF sums to one:
X
p(xi ) = 1
i
The cumulative distribution function (CDF) is defined as
F (x) = P (X ≤ x).
PHP 2510 – Lec IV
6
Cummulative Distribution Function
For the previous examples: Let X denote the number of heads in
three tosses of a coin.
p(x)
F (x)
0
1/8
1/8
1
3/8
4/8
2
3/8
7/8
3
1/8
1
0
PHP 2510 – Lec IV
1
x
2
3
0.000
CDF
0.500
PMF
0.125
0.625
1.000
x
0
1
x
2
3
7
Bernoulli Distribution
Bernoulli Distribution:
The random variable X takes value 1 or 0.
• Ω = {0, 1}
• P (X = 1) = p
• P (X = 0) = 1 − p
Probability
Notice that X is random but p is not.
1−p
p
0.0
0.2
0.4
0.6
0.8
1.0
X
PHP 2510 – Lec IV
8
Bernoulli Distribution
0.6
p
0.4
1−p
0.0
0.2
Probability
0.8
1.0
Example 1: Flip a fair coin. Let 1 represent H, 0 represent T.
0.0
0.2
0.4
0.6
0.8
1.0
X
PHP 2510 – Lec IV
9
Bernoulli Distribution
0.4
0.6
1−p
p
0.0
0.2
Probability
0.8
1.0
Example 2: A class with 100 students include 75 females and 25
males. Randomly select one from the class, let 1 represent male
and 0 represent female. P (X = 1) = .25, P (X = 0) = .75
0.0
0.2
0.4
0.6
0.8
1.0
X
PHP 2510 – Lec IV
10
Bernoulli Distribution
0.4
0.6
1−p
0.2
Probability
0.8
1.0
Example 3. Prevalence of smoking in US adults is 20.8%.
Randomly choose one adult from this country, consider the random
variable X of the smoking status of this person.
P (X = 1) = .208, P (X = 0) = .792
0.0
p
0.0
0.2
0.4
0.6
0.8
1.0
X
PHP 2510 – Lec IV
11
Bernoulli Distribution
Example: The prevalence of HIV infection is 11%. Let X be the
HIV status of a randomly chosen person. X = 1 if HIV+; X = 0 if
HIV-. Then, X has a Bernoulli distribution.
p(X = 1) = 0.11,
PHP 2510 – Lec IV
p(X = 0) = 0.89.
12
Bernoulli trials
In all the above examples, the experiments have an outcome that is
random and can only take either of two possible outcomes
“success” or “failure”.
These experiments are called “Bernoulli trials”. The distribution of
the random variable is a ”Bernoulli distribution”.
If we perform two trials and the outcome of either trial does not
affect the other, these are independent trials.
PHP 2510 – Lec IV
13
Bernoulli trials
Example of independent trials:
• Flip a coin n times (independent Bernoulli trials)
• Roll a dice n times (independent trials)
• Randomly choose a
he/she is smoker or
Randomly choose a
he/she is smoker or
PHP 2510 – Lec IV
person at Providence and observe whether
not (independent Bernoulli trials);
person at Boston and observe whether
not (independent Bernoulli trials)
14
Bernoulli trials
Consider three independent trials of “randomly sample one person
from a population with HIV infection prevalence 11%.” Let 1
indicate HIV+ and 0 for HIV-.
P (X = 1) = 0.11 = p, P (X = 0) = 0.89 = 1 − p
What is the probability of three HIV+?
P (+ + +) = P (+)P (+)P (+) = (.11)(.11)(.11) = (.11)3 = pk
What is the probability that only the second one is HIV-?
P (+−+ ) = P (X1 =1)P (X2 =0)P (X3 =1) = .11×.89×.11 = p(1−p)p = p2 (1−p)
What is the probability that “two out of the three is HIV+”?
P (++− ∪ +−+ ∪
−++
) = P (++− ) + P (+−+ ) + P (−++ )
= pp(1 − p) + p(1 − p)p + (1 − p)pp
=
PHP 2510 – Lec IV
3p2 (1 − p)
15
Binomial Distribution
Now consider an experiment that involves n independent Bernoulli
trials, each with the same success probability of success p. Let X
be the random variable of “total number of successes among the n
independent and identical Bernoulli trials”.
What is the distribution of X?
1. Consider the number of success 0.
P (X = 0) = P (F F . . . F ) = (1 − p)(1 − p) . . . (1 − p) = (1 − p)n
2. For all successes,
P (X = n) = P (SS . . . S) = pp . . . p = pn
3. What about other k for 0 < k < n?
We need to find out how many different ways we can allocate the k
n
successes into the n trials: the combination number k .
PHP 2510 – Lec IV
16
Binomial Distribution
For n = 3
k (X1 X2 X3 )
P (X1 X2 X3 )
Combination
n
k
P(X=k)
0
000
(1 − p)3
1
(1 − p)3
1
001
p1 (1 − p)2
3
3p(1 − p)2
p2 (1 − p)1
3
3p2 (1 − p)
p3
1
p3
010
100
2
011
101
110
3
111
PHP 2510 – Lec IV
17
Binomial Distribution
PHP 2510 – Lec IV
18
Binomial Distribution
Combination
n
k
P(X=k)
k
(X1 X2 X3 X4 )
P (X1 X2 X3 X4 )
0
0000
(1 − p)4
1
1
0001
p1 (1 − p)3
4
4p(1 − p)3
p2 (1 − p)2
6
6p2 (1 − p)2
p3 (1 − p)
4
4p3 (1 − p)
p4
1
p4
(1 − p)4
0100
0010
1000
2
0011
0101
0110
1001
1010
1100
3
0111
1011
1101
1110
4
1111
PHP 2510 – Lec IV
19
Binomial Distribution
In general, if we have n independent and identical Bernoulli trials
with success probability p, the number of successes has probability
n k
P (X = k) =
p (1 − p)n−k , k = 0, 1, 2, . . . , n
k
n
where k is the number of k−combinations from a size n set.
n!
n
=
k
k!(n − k)!
We call the distribution for random variable X Binomial
Distribution and often write X ∼ Binom(n, p).
PHP 2510 – Lec IV
20
Binomial Distribution
Review Counting Techniques:
1. Permuataion: the number of possible arrangement of n objects is
n! = n(n − 1)(n − 2) . . . 1
(By definition 0!=1)
2. From n objects, select r at a time, the number of ordered arrangements is
n!
= n(n − 1) . . . (n − r + 1)
(n − r)!
3. Combination: from n objects, select r at a time, the number of unique
combinations of size r (WITHOUT regard to order) is “n choose r”
Notice that
PHP 2510 – Lec IV
n
k
=
n
n−k
,
n
0
n
n!
=
k
(n − r)!r!
=
n
n
=1
21
Binomial Distribution
0.4
0.6
N= 5 , success prob.= 0.5
0.0
0.2
Probability
0.8
1.0
Probability Mass Function of Binomial Distribution
0
1
2
3
4
5
X
PHP 2510 – Lec IV
22
0.4
0.6
N= 5 , success prob.= 0.2
0.0
0.2
Probability
0.8
1.0
Binomial Distribution
0
1
2
3
4
5
X
PHP 2510 – Lec IV
23
Binomial Distribution
1.0
P rob(X = 5) =
P rob(X ≥ 10) =
X
× .510 = .0916
P (X = k) = .15
0.8
k=10
0.4
0.6
N= 15 , success prob.= 0.5
0.0
0.2
Probability
15
.55
5
15
0
5
10
15
X
PHP 2510 – Lec IV
24
Binomial Distribution
Binomial Examples:
• Z = number of 100 patients in a clinical trial who have cancer
remission following an experimental treatment, suppose these
patients are independent of each other and with the same risk
for remission.
• W = number of the 3 transferred embryos that implant in a
woman’s uterus following in-vitro fertilization, suppose each
embryo is independent of another and with the same
probability of implanting.
• 29% of Americans are smokers. Suppose you select 5 people at
random from the population. Let X denote the number of
smokers in the sample.
PHP 2510 – Lec IV
25
Binomial Distribution
Binomial Example:
For a diagnostic test with sensitivity .9 and specificity .95, what is
the probability of getting 8 positive result from tests on 10 people
with the disease?
10 8
10
p (1 − p)2 =
( )8 ( )2 = .194
8
2
What is the probability of observing at least 1 positive from 10
people without the disease?
P (k ≥ 1) = 1 − P (k = 0) = 1 −
PHP 2510 – Lec IV
10 0
p (1 − p)10 = .4
0
26
Binomial Distribution
Binomial Example:
Suppose the mortality rate for a disease is .10 and 10 people in a
community catch the disease. What is the probability that
• None of them survive?
P (X = 10) =
10
.110 .90 = 10−10
10
• Exactly 5 will die from the disease?
10
P (X = 5) =
.15 .95 = .149
5
• At least 3 will die from the disease?
P (X ≥ 3)
=
=
=
PHP 2510 – Lec IV
1 − P (X < 3) = 1 − {P (X = 0) + P (X = 1) + P (X = 2)}
o
n 10 2 8
10 1 9
10 0 10
.1 .9
.1 .9 +
.1 .9 +
1−
2
1
0
1 − (.3487 + .3874 + .1937) = .0702
27
Poisson Distribution
Poisson Distribution
Described occurrences of events that are distributed randomly over
space or time.
• Number of seizures per week for an epileptic individual
• Number of hospitalizations per month for a chronically ill
individual
• Cases of a rare disease in a fixed period
• Number of forrest fire in a year
• yearly suicide counts
A common feature: All these are counts taking on values of 0, 1, 2,
. . .. That is, the number of occurrences X takes non-negative
integer values 0, 1, 2, . . . (what is the sample space Ω?)
PHP 2510 – Lec IV
28
Poisson Distribution
Assumptions:
• The occurrence of events over an interval or time are
independent
• In theory, an infinite number of occurrences is possible (X is
not bounded from above)
• Not more than one event can happen at exactly the same time
or same location
Then X follows a Poisson distribution with probability masses:
e−λ λk
P (X = k) =
k!
where λ is the average number of events.
Recall from your calculus class, that
∞
X
λk
k=0
k!
PHP 2510 – Lec IV
= eλ =⇒
∞
X
k=0
P (X = k) = e−λ
∞
X
λk
k=0
k!
= e−λ eλ = 1
29
Poisson Distribution
Example of Poisson distribution: In World War II, German flying
bombs hit London. Were they able to target important sites, or
were they dropping bombs randomly?
The British made a grid of the central 24km × 24km of London into
1km × 1km blocks and counted the number of bombs in each block.
• If bombs were not targeted to strategic locations but were
randomly dropped, Poisson distribution
• If bombs were targeting certain areas more, we expect to see
deviation from the Poisson distribution.
PHP 2510 – Lec IV
30
Poisson Distribution
Summary
a
number of hits k
Number of blocks with k hits
Poisson prediction
0
1
2
3
4
≥5
229
211
93
35
7
1
227.5
211.3
98.1
30.4
7.1
1.5
Total number of blocks: 24 × 24 = 576
Total number of bombs: 535
Average number of bombs per block: 535/576
Poisson prediction computed by
576 × P (X = k) where X ∼ P oisson(535/576)
a For
more details, read http://www.dur.ac.uk/stat.web/bomb.htm
PHP 2510 – Lec IV
31
Poisson Distribution
Poisson distribution example:
Suppose the average number of fatal travel accidents in RI is 3 a
year. What is the probability of having a fatal accident free year?
What is the probability of having 5 fatal accidents a year? What is
the probability of having not more than 3 fatal accidents in a year?
e−3 30
= .05
P (X = 0) =
0!
e−3 35
P (X = 5) =
= .10
5!
P (X ≤ 3) =
3
X
P (X = k) = .65
k=0
PHP 2510 – Lec IV
32
Poisson Distribution
0.4
0.8
Poisson distribution probability mass function:
0.2
0.0
0.0
0.4
λ=.3
λ=1
10
15
20
0
5
10
15
20
15
20
0.20
5
0.20
0
0.00
0.00
0.10
λ=10
0.10
λ=5
0
5
PHP 2510 – Lec IV
10
15
20
0
5
10
33
Poisson Distribution
Poisson distribution and rate: If the average occurrence for a
given period (area,...) is µ, then the distribution of number of
events for period t is P oisson(µt)
For example
• If the yearly incidence rate of a rare disease is 10 per million,
for a city of population 700,000, the number of new cases per
year has distribution P oisson(7).
• The number of new cases for a state of population 5 million per
month is P oisson(10 × 5/12) = P oisson(4.17)
PHP 2510 – Lec IV
34
.
Poisson Distribution
Let X denote the number of TB cases in a single month at a clinic
in South Africa. Suppose X follows a Poisson distribution with rate
parameter λ = 3, which means on average, there are 3 cases per
month.
• What is the probability of observing zero cases in a month?
0
3
= .0498
P (X = 0) = e−3
0!
• What is the probability of observing 2 cases ?
−3 3
P (X = 2) = e
PHP 2510 – Lec IV
2
2!
= .2240
35
.
Poisson Distribution
• The clinic observes 7 cases in a single month. Does this suggest
an increase of the rate?
P (X ≤ 6)
= P (X = 0) + P (X = 1) + · · · + P (X = 6) = .9665
P (X > 6)
= 1 − .9665 = .0335
How can you use this information to answer the question?
PHP 2510 – Lec IV
36
Poisson Distribution
Review: Conditional probability
For a Poisson distribution with mean 4, what is the probability
that exactly 3 events happen?
e−4 43
= .195
P (X = 3) =
3!
What is the probability that 3 events happen, given at least one
event happens?
P (X = 3|X ≥ 1)
=
=
PHP 2510 – Lec IV
P (X = 3&X ≥ 1)
P (X = 3)
=
P (X ≥ 1)
1 − P (X = 0)
.195
= .199
1 − e−4
37
Poisson Distribution
Poisson as approximation for Binomial Distribution
Poisson distribution is useful in describing events happening
randomly over space or time. Another use of Poisson distribution is
to approximate Binomial distribution.
When n is large and p is very small, the Binomial distribution
Binom(n, p) can be approximated by P ois(np). Notice that both
distributions have the average (expected) number of events np.
PHP 2510 – Lec IV
38
Poisson Distribution
Example: Binomial and Poisson
Suppose the incidence of Down’s syndrome in 40-year-old mother is
1/100. Out of 25 babies born to 40-year-old mothers, what is the
frequency of babies with Down’s syndrome?
• Binomial distribution with n = 25, p = .01
• Poisson approximation λ = np = .25
PHP 2510 – Lec IV
39
1e−02
1e−05
px (1 − p)n−x
.7788
1
.1947
.1964
2
.0243
.0238
>2
.0022
.0020
Binomial
Poisson approx
.7778
1e−08
0
1e−11
λ /x!
1e−14
e
n
x
0.4
−λ x
Binomial
0.2
Poisson
0.6
P (X = x)
0.0
x
0.8
Poisson Distribution
0
2
4
6
8
10
0
2
4
6
8
10
Approximation is even better when n is larger.
PHP 2510 – Lec IV
40
Geometric Distribution
Other Common Discrete Probability Distributions:
Geometric distribution: Distribution of the random variable X
for the number of independent identical Bernoulli trials needed to
get the first success
P (X = k) = (1 − p)k−1 p
Recall that for Binomial distribution, there are fixed number of
trials, and the random variable Y is the number of successes.
PHP 2510 – Lec IV
41
Geometric Distribution
Example. Probability of contracting HIV in a single sexual
encounter is 1 in 500. Let X denote the encounter during which a
person gets infected for the first time. Assume each encounter is
independent and carries the same risk.
The mass function is
P (X = k)
=
499
500
k−1 1
500
Example. What is the probability of contracting HIV within the
first 3 encounters?
PHP 2510 – Lec IV
P (X = 1)
= · · · = .002
P (X = 2)
= · · · = .001996
P (X = 3)
= · · · = .001992
P (X ≤ 3)
= · · · = .006
42
.
Hypergeometric distribution
Hypergeometric distribution : In a finite population of size n,
r are successes. If we randomly draw m from this population, the
number of successes is a random variable X.
P (X = k) =
r
k
n−r
m−k
n
m
,
where k ≥ 0, k ≥ r − (n − m), k ≤ m and k ≤ r
PHP 2510 – Lec IV
43
Hypergeometric distribution
Consider a bag of n balls and r of these are red, the rest are black.
If we randomly and blindly pick m balls out of the bag, what is the
probability of getting k red balls (successes)?
drawn
not drawn
total
Red(Success)
k
r-k
r
Black(Failure)
m-k
(n-r)-(m-k)
n-r
Total
m
n-m
n
We know k can not exceed the total number of balls drawn or total
number of red balls, thus k ≤ r and k ≤ m =⇒ k ≤ min(n, m).
Also, since there are only n − r black balls, if we draw more than
n − r balls out, then at least m − (n − r) of them are red. So
k ≥ m − (n − r). Another way to see this is to notice that after we
take r balls out, the number of black balls left in the bag has to be
nonnegative, thus (n − r) − (m − k) ≥ 0 =⇒ k ≥ m − (n − r)
PHP 2510 – Lec IV
44
Hypergeometric distribution
Now that we find out the sample space of X, what are the
probability masses for each k?
n
There are total m ways of drawing m balls out of the bag.
Among those, how many satisfy k red balls (and thus m − k black
n−r
r
balls)? There are k ways we can draw the k red balls and m−k
black balls. Thus the proportion of qualifying draws is
n−r r
P (X = k) =
PHP 2510 – Lec IV
k
m−k
n
m
45
Hypergeometric distribution
Example: Suppose we have a class of 10 boys and 12 girls. There
are only 14 tickets to the school football game, if we randomly
distribute the tickets to the students, what is the probability that 7
boys will receive tickets?
Consider the hypergeometric distribution n=22, r=10, m=14, k=7
12 10
P (X = 7) =
7
14−7
22
14
= .297
What is the sample space of X?
What is the probability that 5 girls will get tickets, if 12 tickets are
randomly distributed to the students?
PHP 2510 – Lec IV
46
Hypergeometric distribution
Hypergeometric distribution (n,r,m) converges to binomial
distribution (m,p=r/n) as n → ∞ and r/n → p.
So for a large population (n large), if r of them are successes, we
compute the probability of success in a single random draw
p = r/n, and approximate the hypergeometric distribution by
binominal Binom(m, p)
PHP 2510 – Lec IV
47