Download Statistical Distributions

Statistical Distributions
1
Discrete Distributions
1. The uniform distribution. A random variable (r.v.) X has a uniform distribution on the
n-element set A = {x1 , x2 , . . . , xn } if P (X = x) = 1/n whenever x is in the set A. The
following applies to the special case A = {1, 2, . . . , n}.
Support: x = 1, 2, . . . , n.
Parameters: n.
Probability function (p.f.): f (x|n) = 1/n, x = 1, 2, . . . , n.
Moment generating function (m.g.f.): M (t) =
et (1−etn )
n(1−et ) ,
t 6= 0, and M (t) = 1 for t = 0.
V ar(X) = (n2 − 1)/12.
Mean and variance: E(X) = (n + 1)/2,
2. The Bernoulli distribution. A random variable X has a Bernoulli distribution with parameter p (0 ≤ p ≤ 1) if it takes only values 0 and 1 with probabilities 1 − p and p, respectively.
Notation: Bernoulli(p).
Support: x = 0, 1.
Parameters: 0 ≤ p ≤ 1.
Probability function (p.f.): f (x|p) = px (1 − p)1−x for x = 0, 1.
Moment generating function (m.g.f.): M (t) = pet + (1 − p).
Mean and variance: E(X) = p,
V ar(X) = p(1 − p).
3. The binomial distribution. If X1 , . . . , Xn are independent, identically distributed (i.i.d.)
Bernoulli r.v.’s with parameter p, then X = X1 + · · · + Xn has a binomial distribution with
parameters n and p. If a lot contains N items, of which A are defective, and if a sample of
size n items is chosen with replacement from the lot, then the number of defective items
in the sample, X, has a binomial distribution with parameters n and p = A/N .
Notation: Bin(n, p).
Support: x = 0, . . . , n.
Parameters: 0 ≤ p ≤ 1 and n ∈ {1, 2, . . .}.
P.f.: f (x|n, p) =
n!
x
x!(n−x)! p (1
− p)n−x .
M.g.f.: M (t) = [pet + (1 − p)]n .
Mean and variance: E(X) = np,
V ar(X) = np(1 − p).
4. The hypergeometric distribution. If a lot contains N items, of which A are defective
(and B = N − A are non-defective), and if a sample of size n items is chosen without
replacement from the lot, then the number of defective items in the sample, X, has a
hypergeometric distribution with parameters A, B, and n.
Notation: Hyp(A, B, n).
1
Support: max{0, n − B} ≤ x ≤ min{n, A}.
Parameters: A, B, n ∈ {1, 2, . . .}.
P.f.: f (x|A, B, n) =
(A+B)!
A!
B!
x!(A−x)! (n−x)!(B−n+x)! / n!(A+B−n)! .
Mean and variance: E(X) = np,
V ar(X) = np(1 − p) A+B−n
A+B−1 , where p =
A
A+B .
5. The geometric distribution. Consider a sequence of Bernoulli trials in which the outcome
of any trial is either 1 (success) or 0 (failure) and the probability of success on any trial is p.
Let X denote the number of trials needed so that the success occurs for the first time at the
last trial. Then, X is said to have a geometric distribution with parameter p.
Notation: Geo(p).
Support: positive integers.
Parameters: 0 ≤ p ≤ 1.
P.f.: f (x|p) = p(1 − p)x−1 , x = 1, 2, 3, . . ..
M.g.f.: M (t) =
pet
1−(1−p)et ,
t < ln(1/(1 − p)).
Mean and variance: E(X) = 1p ,
V ar(X) =
1−p
p2 .
Note. An alternative definition of geometric distribution: number of failures, Y , that occur
before the first success is obtained (this is the definition adopted in the text). Since X −1 = Y ,
the formulas for Y are easily derived from those for X:
Support: non-negative integers.
Parameters: 0 ≤ p ≤ 1.
P.f.: f (x|p) = p(1 − p)x , x = 0, 1, 2, . . ..
M.g.f.: M (t) =
p
1−(1−p)et ,
t < ln(1/(1 − p)).
Mean and variance: E(X) =
1−p
p ,
V ar(X) =
1−p
p2 .
6. The negative binomial distribution. Consider a sequence of Bernoulli trials in which
the outcome of any trial is either 1 (success) or 0 (failure) and the probability of success
on any trial is p. Let X denote the number of trials needed so that the success occurs for
the rth time at the last trial. Then, X is said to have a negative binomial distribution with
parameters r and p. If X1 , . . . , Xr are i.i.d. r.v.’s and each has a Geo(p) distribution, then
X = X1 + · · · + Xr has a negative binomial distribution with parameters r and p.
Notation: N B(r, p).
Support: x ∈ {r, r + 1, r + 2, . . .}.
Parameters: r ∈ {1, 2, 3, . . .} and 0 ≤ p ≤ 1.
P.f.: f (x|r, p) =
M.g.f.: M (t) =
(x−1)!
r
(r−1)!(x−r)! p (1
h
ir
pet
1−(1−p)et
− p)x−r .
, t < ln(1/(1 − p)).
Mean and variance: E(X) = r/p,
V ar(X) =
2
r(1−p)
.
p2
Note. An alternative definition of negative binomial distribution: number of failures, Y ,
that occur before the rth success is obtained (this definition is adotped in our text). Since
X − r = Y , the formulas for Y are easily derived from those for X:
Support: non-negative integers.
Parameters: r ∈ {1, 2, 3, . . .} and 0 ≤ p ≤ 1.
P.f.: f (x|r, p) =
M.g.f.: M (t) =
(r+x−1)! r
(r−1)!x! p (1
h
ir
p
1−(1−p)et
− p)x , x = 0, 1, 2, . . ..
, t < ln(1/(1 − p)).
Mean and variance: E(X) =
r(1−p)
p ,
V ar(X) =
r(1−p)
p2 .
7. The Poisson distribution.
Notation: P oisson(λ).
Support: x ∈ {0, 1, 2, . . .}.
Parameters: λ > 0.
e−λ λx
x! .
t
= eλ(e −1) .
P.f.: f (x|λ) =
M.g.f.: M (t)
Mean and variance: E(X) = λ,
V ar(X) = λ.
8. The multinomial distribution. This is a multivariate distribution where each of n independent trials can result in one of r types of outcomes, and on each trial the probabilities of
the r outcomes are p1 , p2 , . . . , pr . The variable here is a vector (X1 , . . . , Xr ), where Xi is the
number of outcomes of type i
Support: xi ∈ {0, 1, 2, . . . n}, i = 1, 2, . . . r, x1 + · · · + xr = n.
Parameters: n, r - positive integers, n > r > 1; p1 , p2 , . . . , pr - positive numbers that add
up to one.
P.f.: f (x1 , . . . , xr ) =
x1
n!
x1 !...xr ! p1
. . . pxr r .
M.g.f.:
Mean and variance: E(Xi ) = npi ,
2
V ar(Xi ) = npi (1 − pi ).
Continuous distributions
1. The uniform distribution.
Notation: U (a, b).
Support: x ∈ [a, b].
Parameters: −∞ < a < b < ∞.
Probability density function (p.d.f ): f (x|a, b) =
M.g.f.: M (t) =
1
b−a .
ebt −eat
(b−a)t .
Mean and variance: E(X) =
a+b
2 ,
V ar(X) =
3
(b−a)2
12 .
Mode:
Median:
a+b
2 .
2. The exponential distribution.
Notation: Exp(β).
Support: x ≥ 0.
Parameters: β > 0.
P.d.f: f (x|β) = βe−βx .
M.g.f.: M (t) =
β
β−t ,
t < β.
V ar(X) = 1/β 2 .
Mean and variance: E(X) = 1/β,
Mode: 0.
Median: ln 2/β
R
3. The gamma distribution. For any α > 0, the value Γ(α) denotes the integral 0∞ xα−1 e−x dx,
and is called the gamma function. Some properties of the gamma function include: Γ(α+1) =
αΓ(α), Γ(n) = (n − 1)! for any integer n ≥ 1, and Γ(1/2) = π 1/2 .
Notation: G(α, β).
Support: x ≥ 0.
Parameters: α, β > 0.
P.d.f: f (x|α, β) =
M.g.f.: M (t) =
β α α−1 −βx
e
.
Γ(α) x
α
β
β−t
, t < β.
V ar(X) = α/β 2 .
Mean and variance: E(X) = α/β,
Mode: (α − 1)/β if α > 1 and 0 for 0 < α ≤ 1.
Notes. First, note that Exp(β) = G(1, β). Next, it follows from the formula of gamma m.g.f.,
that if Xi ∼ G(αi , β) are independent, then X = X1 +· · ·+Xn has a gamma G(α1 +· · ·+αn , β)
distribution. In particular, if Xi ∼ Exp(β) are independent, then X = X1 + · · · + Xn ∼
G(n, β).
4. The chi-square distribution with n degrees of freedom. Special case of gamma distribution G(α, β) with α = n/2, β = 1/2, n = 1, 2, 3, . . ..
Notation: χ2n .
Support: x ≥ 0.
Parameters: n - a positive integer called the degrees of freedom.
P.d.f: f (x|n) =
M.g.f.: M (t) =
1
1
xn/2−1 e− 2 x .
Γ(n/2)2n/2
1
1−2t
n/2
, t < 1/2.
Mean and variance: E(X) = n,
V ar(X) = 2n.
Mode: n − 2 if n ≥ 2 and 0 for n = 1.
4
Notes. The chi-square distribution arises in connection with random samples from the normal
distribution: If Z1 , . . . Zn are i.i.d. standard normal variables, then the quantity
X = Z12 + · · · + Zn2
has the χ2n distribution. Consequently, if X1 , . . . Xn are i.i.d. normal N (µ, σ) variables, then
the quantity
n
1 X
X= 2
(Xi − µ)2
σ i=1
has the χ2n distribution. It is also true that under the above conditions the quantity
X=
n
1 X
(Xi − X)2
σ 2 i=1
has the χ2n−1 distribution, where X is the sample mean of the Xi ’s.
5. The t distribution (Student’s distribution) with n degrees of freedom.
Notation: tn .
Support: −∞ < x < ∞.
Parameters: n - a positive integer called the degrees of freedom (d.f.)
P.d.f: f (x|n) =
Γ[(n+1)/2]
√
(1
nπΓ(n/2)
+ x2 /n)−(n+1)/2 .
M.g.f.:
Mean and variance: E(X) = 0 (for n > 1),
V ar(X) = n/(n − 2) (for n > 2).
Mode: 0.
Median: 0
Notes. The t distribution is related to the normal and chi-square distributions, and arises in
connection with random samples from the normal distribution.
If Z is standard normal and
p
U is chi-square with n d.f., then the quantity X = Z/ U/n has the t distribution with n d.f.
Consequently, if X1 , . . . , Xn are i.i.d. normal N (µ, σ) variables, then the quantity
X=
X −µ
√ ,
S/ n
where X is the sample mean of the Xi ’s and
S2 =
n
1 X
(Xi − X)2
n − 1 i=1
is the sample variance of the Xi ’s, has the tn−1 distribution.
6. The F distribution with m and n degrees of freedom.
Notation: Fm,n .
Support: x ≥ 0.
5
Parameters: m and n - positive integers called the degrees of freedom (d.f.)
P.d.f: f (x|m, n) =
Γ[(m+n)/2]
m/2 xm/2−1 (1
Γ(m/2)Γ(n/2) (m/n)
Mean and variance: E(X) =
n
n−2
+ mx/n)−(m+n)/2 .
(for n > 2),
V ar(X) =
2n2 (m+n−2)
m(n−2)2 (n−4)
(for n > 4).
Notes. The F distribution is related to chi-square distribution, and arises in connection with
two independent random samples from normal distributions. If U and V are independent
chi-square random variables with m and n degrees of freedom, respectively, then the quantity
X=
U/m
V /n
has the F distribution with m and n d.f. Consequently, if X1 , . . . , Xm are i.i.d. normal
N (µ1 , σ) variables, and if Y1 , . . . , Yn are i.i.d. normal N (µ2 , σ) variables, then the quantity
X = S12 /S22 has the Fm−1,n−1 distribution, where
S12 =
m
n
1 X
1 X
(Xi − X)2 and S22 =
(Yi − Y )2
m − 1 i=1
n − 1 i=1
are the sample variances of the Xi ’s and the Yi ’s.
7. The Beta distribution.
Notation: Beta(α, β).
Support: 0 ≤ x ≤ 1.
Parameters: α, β > 0.
P.d.f: f (x|α, β) =
Γ(α+β) α−1
(1
Γ(α)Γ(β) x
Mean and variance: E(X) =
− x)β−1 .
α
α+β ,
V ar(X) =
αβ
.
(α+β)2 (α+β+1)
Note. Special case: Beta(1, 1) = U (0, 1).
8. The normal distribution.
Notation: N (µ, σ).
Support: −∞ < x < ∞.
Parameters: −∞ < µ < ∞ and σ > 0.
P.d.f: f (x|µ, σ) =
√ 1 e−
2πσ
µt+σ 2 t2 /2
M.g.f.: M (t) = e
(x−µ)2
2σ 2
.
.
Mean and variance: E(X) = µ,
V ar(X) = σ 2 .
Mode: µ
Median: µ
9. The lognormal distribution.
Notation: LN (µ, σ).
6
Support: x ≥ 0.
Parameters: −∞ < µ < ∞ and σ > 0.
P.d.f: f (x|µ, σ) =
√ 1
e−
2πσx
(ln x−µ)2
2σ 2
.
Mean and variance: E(X) = eµ+σ
Mode: eµ−σ
2 /2
,
2
2
V ar(X) = (eσ − 1)e2µ+σ .
2 /2
Median: eµ
Note. If Y is normal N (µ, σ) then X = eY is LN (µ, σ).
10. The Pareto distribution.
Notation: P (k, α).
Support: x ≥ k.
Parameters: k > 0 and α > 0.
P.d.f: f (x|k, α) = αk α x−(α+1) .
Mean and variance: E(X) =
αk
α−1
(for α > 1), V ar(X) =
αk2
(α−2)(α−1)2
(for α > 2).
Mode: k
Median: 21/α k
11. The Laplace (double exponential) distribution.
Notation: L(µ, σ).
Support: −∞ < x < ∞.
Parameters: −∞ < µ < ∞ and σ > 0.
|x−µ|
1 − σ
.
2σ e
µt
e
, −1/σ
1−t2 σ 2
P.d.f: f (x|µ, σ) =
M.g.f.: M (t) =
< t < 1/σ.
Mean and variance: E(X) = µ,
V ar(X) = 2σ 2 .
Mode: µ
Median: µ
12. The Cauchy distribution.
Notation:
Support: −∞ < x < ∞.
Parameters: −∞ < µ < ∞ and σ > 0.
h
P.d.f: f (x|µ, σ) = (πσ)−1 1 + {(x − θ)/σ)}2
i−1
.
Mean and variance: Do not exist.
Mode: µ
Median: µ
Note. The standard Cauchy distribution corresponds to µ = 0 and σ = 1.
7
13. The Weibull distribution.
Notation: W (α, β).
Support: x ≥ 0.
Parameters: α, β > 0.
P.d.f: f (x|α, β) =
β
α
x β−1 −(x/α)β
e
.
α
Mean and variance: E(X) = αΓ(1 + 1/β), V ar(X) = α2 [Γ(1 + 2/β) − {Γ(1 + 1/β)}2 ].
Mode: α
β−1 1/β
β
if β > 1 and 0 for 0 < β ≤ 1.
Median: α(ln 2)1/β
Notes. If X is standard exponential, then W = αX 1/β has the W (α, β) distribution.
14. The bivariate normal distribution. Joint continuous distribution of X and Y .
Notation: N (µ1 , µ2 , σ1 , σ2 , ρ).
Support: −∞ < x, y < ∞.
Parameters: −∞ < µ1 , µ2 < ∞, σ1 , σ2 > 0, −1 < ρ < 1.
P.d.f:
f (x, y) =
1
p
2πσ1 σ2 1 − ρ2
−
e
1
2(1−ρ2 )
h
(x−µ1 )2
(y−µ2 )2
+
σ2
σ2
1
2
−
2ρ(x−µ1 )(y−µ2 )
σ1 σ2
i
.
Mean and variance:
E(X) = µ1 , E(Y ) = µ2 , V ar(X) = σ12 , V ar(Y ) = σ22 , Cov(X, Y ) = σ1 σ2 ρ
.
8