Download Properties of Estimators Outline Estimator Small

Outline
• Small sample properties of estimators.
Properties of Estimators
– Unbiasedness.
– Efficiency.
• Large sample properties of estimators.
M. S. Fadali
Professor of Electrical Engineering
University of Nevada, Reno
–
–
–
–
Consistency.
Asymptotic unbiasedness.
Asymptotic efficiency.
Asymptotic normality.
1
Estimator
2
Small-sample Properties
• Estimator of a parameter:
N finite or infinite
1. Unbiasedness (mean).
2. Efficiency (variance).
– Function of the data whose
– Value assumed close to the parameter value
• Data: random sample
• Estimator: random variable.
• Distribution of estimator: typically unknown
and complex.
• Statistical properties of an estimator.
3
4
Large-sample Properties
Small-sample properties
N 
1. Asymptotic unbiasedness.
2. Asymptotic efficiency.
3. Asymptotic normality.
4. Consistency.
Unbiased Estimator: correct “on the
average”
for random
for constant
5
6
Mean Squared Error
Example: Sample Mean
Standard Error: Standard deviation of the sampling
distribution of the estimate .
•
i.i.d. measurements with population mean
• Sample mean is an unbiased estimator
Mean Squared Error
Unbiased estimator
7
:
8
Example: Sample Mean
Efficiency
Efficient Estimator: An estimator ∗
efficient than any other estimator if
is more
Efficient estimator of
for any variance
∗
• For an unbiased estimator,
Efficient Unbiased Estimator: An unbiased estimator
∗
is more efficient than any other unbiased
estimator if ∗
9
Cramer-Rao Inequality
10
Other expressions of Cramer-Rao
•
set of data
• Characterized by the pdf f(z;) = f(z)
• Variance of an unbiased estimator
of a
deterministic is bounded below by
Fisher information matrix:
11
• Proof requires that the derivatives exist and
be absolutely integrable.
12
Example: Sample Mean
CRLB for Sample Mean
• Normal pdf:
/
set of data
• Cramer-Rao lower bound
Cramer-Rao lower bound
Efficient estimator for any variance
13
.
14
Example: Sample Mean
Consistent Estimator
as
= estimate based on
data points.
• Chebychev Inequality
• Convergence in probability.
• Using a lot of data tends to give a better
estimate.
• Convergence in probability (unbiased)
as
15
16
Example: Sample Variance
Other Asymptotic Properties
 Asymptotic unbiasedness
• Asymptotically Unbiased:
random
_|Å
→
_|Å
→
constant
 Asymptotic efficiency of consistent estimator:
more efficient that any other consistent
estimator (approaches the Cramer-Rao lower
bound as N).
 Asymptotic normality: converges in distribution
to a normal distribution.
17
References
• M. H. DeGroot and M. J. Schervish, Probability and
Statistics, Addison-Wesley, Boston, 2012.
• R. V. Hogg, J. W. McKean, and A. T. Craig,
Introduction to Mathematical Statistics, Prentice-Hall,
Upper Saddle River, NJ, 2005.
• W. L Martinez and A. R. Martinez, Computational
Statistics Handbook with MATLAB, Chapman &
Hall/CRC, Boca Raton, Fl, 2002.
• J. Mendel, Lessons in Estimation Theory and Signal
Processing, Communications, and Control, PrenticeHall, NJ, 1995.
19
• Use: (i)
(ii)
,
18