Download No Slide Title

1A.1
Appendix 1A
Review
Some Basic
Statistical Concepts
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.2
Random Variable
random variable:
A variable whose value is unknown until it is observed.
The value of a random variable results from an experiment.
The term random variable implies the existence of some
known or unknown probability distribution defined over
the set of all possible values of that variable.
In contrast, an arbitrary variable does not have a
probability distribution associated with its values.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.3
Controlled experiment values
of explanatory variables are chosen
with great care in accordance with
an appropriate experimental design.
Uncontrolled experiment values
of explanatory variables consist of
nonexperimental observations over
which the analyst has no control.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.4
Discrete Random Variable
discrete random variable:
A discrete random variable can take only a finite
number of values, that can be counted by using
the positive integers.
Example: Prize money from the following
lottery is a discrete random variable:
first prize: $1,000
second prize: $50
third prize: $5.75
since it has only four (a finite number)
(count: 1,2,3,4) of possible outcomes:
$0.00; $5.75; $50.00; $1,000.00
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.5
Continuous Random Variable
continuous random variable:
A continuous random variable can take
any real value (not just whole numbers)
in at least one interval on the real line.
Examples:
Gross national product (GNP)
money supply
interest rates
price of eggs
household income
expenditure on clothing
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Dummy Variable
1A.6
A discrete random variable that is restricted
to two possible values (usually 0 and 1) is
called a dummy variable (also, binary or
indicator variable).
Dummy variables account for qualitative differences:
gender (0=male, 1=female),
race (0=white, 1=nonwhite),
citizenship (0=U.S., 1=not U.S.),
income class (0=poor, 1=rich).
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.7
A list of all of the possible values taken
by a discrete random variable along with
their chances of occurring is called a probability
function or probability density function (pdf).
die
one dot
two dots
three dots
four dots
five dots
six dots
x
1
2
3
4
5
6
f(x)
1/6
1/6
1/6
1/6
1/6
1/6
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.8
A discrete random variable X
has pdf, f(x), which is the probability
that X takes on the value x.
f(x) = P(X=xi)
Therefore,
0 < f(x) < 1
If X takes on the n values: x1, x2, . . . , xn,
then f(x1) + f(x2)+. . .+f(xn) = 1.
For example in a throw of one dice:
(1/6) + (1/6) + (1/6) + (1/6) + (1/6) + (1/6)=1
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.9
In a throw of two dice, the discrete random variable X,
x = 2 3 4 5 6 7 8
9 10 11 12
f(x) =(1/36)(2/36)(3/36)(4/36)(5/36)(6/36)(5/36)( 4/36)(3/36)(2/36)(1/36)
the pdf f(x) can be shown by the presented by height:
f(x)
0 2 3 4
5 6 7
8 9 10 11 12
X
number, X, the possible outcomes of two dice
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.10
A continuous random variable uses
area under a curve rather than the
height, f(x), to represent probability:
f(x)
red area
0.1324
green area
0.8676
.
.
$34,000
$55,000
X
per capita income, X, in the United States
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.11
Since a continuous random variable has an
uncountably infinite number of values,
the probability of one occurring is zero.
P[X=a] = P[a<X<a]=0
Probability is represented by area.
Height alone has no area.
An interval for X is needed to get
an area under the curve.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.12
The area under a curve is the integral of
the equation that generates the curve:
P[a<X<b]=
a
b
f(x) dx
For continuous random variables it is the
integral of f(x), and not f(x) itself, which
defines the area and, therefore, the probability.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.13
Rules of Summation
n
Rule 1:
Rule 2:
xi = x1 + x2 + . . . + xn

i=1
n
n
i=1
i=1
 kxi = k  xi
n
Rule 3:
n
n
i=1
i=1
xi + yi =  xi +  yi

i=1
Note that summation is a linear operator
which means it operates term by term.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.14
Rules of Summation (continued)
n
Rule 4:
Rule 5:
n
n
i=1
i=1
axi + byi = a  xi + b  yi

i=1
x
n
= n  xi =
i=1
1
x1 + x2 + . . . + xn
n
The definition of x as given in Rule 5 implies
the following important fact:
n
xi x) = 0

i=1
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.15
Rules of Summation (continued)
n
Rule 6:
f(xi) = f(x1) + f(x2) + . . . + f(xn)

i=1
Notation:
n m
Rule 7:
n
x f(xi) = i f(xi) = i =1 f(xi)
n
[ f(xi,y1) + f(xi,y2)+. . .+ f(xi,ym)]
  f(xi,yj) = i 
=1
i=1 j=1
The order of summation does not matter :
n m
m n
f(xi,yj)
  f(xi,yj) =j =

1 i=1
i=1 j=1
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.16
The Mean of a Random Variable
The mean or arithmetic average of a
random variable is its mathematical
expectation or expected value, E(X).
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Expected Value
1A.17
There are two entirely different, but mathematically
equivalent, ways of determining the expected value:
1. Empirically:
The expected value of a random variable, X,
is the average value of the random variable in an
infinite number of repetitions of the experiment.
In other words, draw an infinite number of samples,
and average the values of X that you get.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Expected Value
1A.18
2. Analytically:
The expected value of a discrete random
variable, X, is determined by weighting all
the possible values of X by the corresponding
probability density function values, f(x), and
summing them up.
In other words:
E(X) = x1f(x1) + x2f(x2) + . . . + xnf(xn)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.19
Empirical vs. Analytical
As sample size goes to infinity, the
empirical and analytical methods
will produce the same value.
In the empirical case when the
sample goes to infinity the values
of X occur with a frequency
equal to the corresponding f(x)
in the analytical expression.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.20
Empirical (sample) mean:
n
x =  xi/n
i=1
where n is the number of sample observations.
Analytical mean:
n
E(X) =  xi f(xi)
i=1
where n is the number of possible values of xi.
Notice how the meaning of n changes.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.21
The expected value of X:
n
E (X) =
xi f(xi)
i=1
The expected value of X-squared:
2
E (X ) =
n
xi
2
i=1
f(xi)
It is important to notice that f(xi) does not change!
The expected value of X-cubed:
n
E (X )= i=1xi f(xi)
3
3
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.22
E(X) = 0 (.1) + 1 (.3) + 2 (.3) + 3 (.2) + 4 (.1)
= 1.9
2
2
2
2
2
2
E(X )= 0 (.1) + 1 (.3) + 2 (.3) + 3 (.2) + 4 (.1)
= 0 + .3 + 1.2 + 1.8 + 1.6
= 4.9
3
3
3
3
3
3
E( X ) = 0 (.1) + 1 (.3) + 2 (.3) + 3 (.2) +4 (.1)
= 0 + .3 + 2.4 + 5.4 + 6.4
= 14.5
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.23
n
E[g(X)] =

g(xi)
i=1
f(xi)
g(X) = g1(X) + g2(X)
n
E[g(X)] =

g1(xi) + g2(xi)] f(xi)
i=1
n
E[g(X)] =
n

g1(xi) f(xi) +i 
g
(x
)
f(x
)
2
i
i
=1
i=1
E[g(X)] = E[g1(X)] + E[g2(X)]
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.24
Adding and Subtracting
Random Variables
E(X+Y) = E(X) + E(Y)
E(X-Y) = E(X) - E(Y)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.25
Adding a constant to a variable will
add a constant to its expected value:
E(X+a) = E(X) + a
Multiplying by constant will multiply
its expected value by that constant:
E(bX) = b E(X)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.26
Variance
var(X) = average squared deviations
around the mean of X.
var(X) = expected value of the squared deviations
around the expected value of X.
2
var(X) = E [(X - E(X)) ]
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.27
2
var(X) = E [(X - EX) ]
2
var(X) = E [(X - EX) ]
2
2
= E [X - 2XEX + (EX) ]
2
2
= E(X ) - 2 EX EX + E (EX)
2
2
2
= E(X ) - 2 (EX) + (EX)
2
2
= E(X ) - (EX)
2
2
var(X) = E(X ) - (EX)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.28
variance of a discrete
random variable, X:
n
var (X) =
)
(x
EX
)
(x
f
i
 i
2
i=1
standard deviation is square root of variance
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.29
calculate the variance for a
discrete random variable, X:
2
xi
f(xi)
(xi - EX)
(xi - EX) f(xi)
2
3
4
5
6
.1
.3
.1
.2
.3
2 - 4.3 = -2.3
3 - 4.3 = -1.3
4 - 4.3 = - .3
5 - 4.3 = .7
6 - 4.3 = 1.7
5.29 (.1) =
1.69 (.3) =
.09 (.1) =
.49 (.2) =
2.89 (.3) =
.529
.507
.009
.098
.867
n
 xi f(xi) = .2 + .9 + .4 + 1.0 + 1.8 = 4.3
i=1
n
2
(xi - EX) f(xi) = .529 + .507 + .009 + .098 + .867
= 2.01
i=1
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.30
Z = a + cX
var(Z) = var(a + cX)
2
= E [(a+cX) - E(a+cX)]
2
= c var(X)
2
var(a + cX) = c var(X)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.31
Covariance
The covariance between two random
variables, X and Y, measures the
linear association between them.
cov(X,Y) = E[(X - EX)(Y-EY)]
Note that variance is a special case of covariance.
2
cov(X,X) = var(X) = E[(X - EX) ]
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.32
cov(X,Y) = E [(X - EX)(Y-EY)]
cov(X,Y) = E [(X - EX)(Y-EY)]
= E [XY - X EY - Y EX + EX EY]
= E(XY) - EX EY - EY EX + EX EY
= E(XY) - 2 EX EY + EX EY
= E(XY) - EX EY
cov(X,Y) = E(XY) - EX EY
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Y=1
X=0
.45
1A.33
Y=2
.15
.60
EX=0(.60)+1(.40)=.40
X=1
.05
.50
.35
.50
EY=1(.50)+2(.50)=1.50
EX EY = (.40)(1.50) = .60
.40
covariance
cov(X,Y) = E(XY) - EX EY
= .75 - (.40)(1.50)
= .75 - .60
= .15
E(XY) = (0)(1)(.45)+(0)(2)(.15)+(1)(1)(.05)+(1)(2)(.35)=.75
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Joint pdf
1A.34
A joint probability density function,
f(x,y), provides the probabilities
associated with the joint occurrence
of all of the possible pairs of X and Y.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.35
Survey of College City, NY
joint pdf
f(x,y)
vacation X = 0
homes
owned
X=1
college grads
in household
Y=2
Y=1
f(0,1)
.45
f(0,2)
.15
.05
f(1,1)
.35
f(1,2)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.36
Calculating the expected value of
functions of two random variables.
E[g(X,Y)] =   g(xi,yj) f(xi,yj)
i
j
E(XY) =   xi yj f(xi,yj)
i
j
E(XY) = (0)(1)(.45)+(0)(2)(.15)+(1)(1)(.05)+(1)(2)(.35)=.75
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.37
Marginal pdf
The marginal probability density functions,
f(x) and f(y), for discrete random variables,
can be obtained by summing over the f(x,y)
with respect to the values of Y to obtain f(x)
with respect to the values of X to obtain f(y).
f(xi) =  f(xi,yj)
j
f(yj) =  f(xi,yj)
i
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
marginal
Y=1
Y=2
1A.38
marginal
pdf for X:
X=0
.45
.15
.60 f(X = 0)
X=1
.05
.35
.40 f(X = 1)
.50
.50
f(Y = 2)
marginal
pdf for Y:
f(Y = 1)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Conditional pdf
1A.39
The conditional probability density
functions of X given Y=y , f(x|y),
and of Y given X=x , f(y|x),
are obtained by dividing f(x,y) by f(y)
to get f(x|y) and by f(x) to get f(y|x).
f(x,y)
f(x,y)
f(x|y) =
f(y
x)
=
|
f(y)
f(x)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.40
conditonal
f(Y=1|X = 0)=.75
Y=1
.75
.45
X=0
f(X=0|Y=1)=.90 .90
f(X=1|Y=1)=.10 .10
X=1
Y=2
f(Y=2|X= 0)=.25
.25
.60
.15
.05 .35
.30
.70
f(X=0|Y=2)=.30
f(X=1|Y=2)=.70
.40
.125 .875
f(Y=1|X = 1)=.125
.50
.50 f(Y=2|X = 1)=.875
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Independence
1A.41
X and Y are independent random
variables if their joint pdf, f(x,y),
is the product of their respective
marginal pdfs, f(x) and f(y) .
f(xi,yj) = f(xi) f(yj)
for independence this must hold for all pairs of i and j
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.42
not independent
Y=1
Y=2
.50x.60=.30
.50x.60=.30
marginal
pdf for X:
X=0
.45
.15
.60 f(X = 0)
X=1
.05
.35
.40 f(X = 1)
.50x.40=.20
marginal
pdf for Y:
.50
f(Y = 1)
.50x.40=.20
.50
f(Y = 2)
The calculations
in the boxes show
the numbers
required to have
independence.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Correlation
1A.43
The correlation between two random
variables X and Y is their covariance
divided by the square roots of their
respective variances.
(X,Y) =
cov(X,Y)
var(X) var(Y)
Correlation is a pure number falling between -1 and 1.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Y=1
Y=2
1A.44
EX=.40
2
2
2
EX=0(.60)+1(.40)=.40
X=0
.45
.05
X=1
.15
.35
.60
2
var(X) = E(X ) - (EX)
2
= .40 - (.40)
= .24
.40
cov(X,Y) = .15
.50
EY=1.50
2 2
2
.50
EY=1(.50)+2(.50)
2
2
var(Y) = E(Y ) - (EY)
= .50 + 2.0
= 2.50 - (1.50)2
= 2.50
= .25
correlation
(X,Y) =
cov(X,Y)
var(X) var(Y)
(X,Y) = .61
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
2
1A.45
Zero Covariance & Correlation
Independent random variables
have zero covariance and,
therefore, zero correlation.
The converse is not true.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Since expectation is a linear operator,
it can be applied term by term.
1A.46
The expected value of the weighted sum
of random variables is the sum of the
expectations of the individual terms.
E[c1X + c2Y] = c1EX + c2EY
In general, for random variables X1, . . . , Xn :
E[c1X1+...+ cnXn] = c1EX1+...+ cnEXn
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.47
The variance of a weighted sum of random
variables is the sum of the variances, each times
the square of the weight, plus twice the covariances
of all the random variables times the products of
their weights.
Weighted sum of random variables:
2
2
var(c1X + c2Y)=c1 var(X)+c2 var(Y) + 2c1c2cov(X,Y)
Weighted difference of random variables:
var(c1X  c2Y) = c21 var(X)+c22var(Y)  2c1c2cov(X,Y)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
The Normal Distribution
Y~
f(y) =
1A.48
2
N(, )
(y - )2
exp
2
1
2
2  2
f(y)

Copyright© 1977 John Wiley & Son, Inc. All rights reserved
y
The Standardized Normal
1A.49
Z = (y - )/
Z ~ N(,)
f(z) =
1
2
exp
- z2
2
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.50
Y ~ N(,2)
f(y)

P[Y>a]
= P
Y-

>
a
a-

= P Z >
y
a-

Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.51
Y ~ N(,2)
f(y)

a
P[a<Y<b] = P
=
P
a-

a-

b
<
y
Y-

<Z<
<
b-

b-

Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.52
Linear combinations of jointly
normally distributed random variables
are themselves normally distributed.
Y1 ~ N(1,12), Y2 ~ N(2,22), . . . , Yn ~ N(n,n2)
W = c1Y1 + c2Y2 + . . . + cnYn
W ~ N[ E(W), var(W) ]
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
Chi-Square
1A.53
If Z1, Z2, . . . , Zm denote m independent
N(0,1) random variables, and
2
2
2
2
V = Z1 + Z2 + . . . + Zm, then V ~ (m)
V is chi-square with m degrees of freedom.
mean:
E[V] = E[ (m) ] = m
2
variance:
var[V] = var[ (m) ] = 2m
2
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.54
Student - t
If Z ~ N(0,1) and V ~ (m) and if Z and V
are independent then,
Z
2
t=
Vm
~ t(m)
t is student-t with m degrees of freedom.
mean:
E[t] = E[t(m) ] = 0 symmetric about zero
variance:
var[t] = var[t(m) ] = m/(m-2)
Copyright© 1977 John Wiley & Son, Inc. All rights reserved
1A.55
F Statistic
If V1 ~ (m ) and V2 ~ (m ) and if V1 and V2
1
2
are independent, then
V1
2
2
F=
m1
V2
~ F(m1,m2)
m2
F is an F statistic with m1 numerator
degrees of freedom and m2 denominator
degrees of freedom.
Copyright© 1977 John Wiley & Son, Inc. All rights reserved