Download Hypothesis Testing

Hypothesis Testing
… understanding statistical claims
Doc Martin is at it again!
After his line of Brain Pills was
shown to be no better than a
placebo, Doc Martin decided to
devote his business genius to
marketing his new line of ski wax
– “Wax 2 the Max”.
In a recent competition between
conventional and the new spaceage W2tMax, when applied to the
skis of 12 cross country skiers, 9
of the winners were using the
Wax-2-the-Max skis!
Wax-2-the-Max!
Try it .
My wax is
proven to work!
How can we test this claim?
Either it works or it “don’t”
• The null hypothesis:
– H0: The wax makes no difference
• The alternate hypothesis:
– Ha: The wax improves performance
If we accept the null hypothesis, then it would mean that
winning the 9 (or more) races occurred merely by
chance. How likely is this?
Wax-2-the-Max
• You either win or don’t – what kind of
distribution does this suggest? B(12,0.5)
• What is the probability of this happening
by chance?
P = P(9)+P(10)+P(11)+P(12)
P = 0.0537+0.0161+0.0029+0.0002
P = 0.0729
What’s this mean?
Bad news for Doc Martin!
• The null hypothesis occurs with a
probability of about 7.3%
• If the null hypothesis occurs with 5% or
lower chance then you would reject H0
and conclude that the alternate
hypothesis is statistically significant
• If the null hypothesis occurs with 1% or
lower chance then Ha is strongly
significant
Rats! Foiled again!
Null Hypothesis and Statistical
Significance
• It is usual to create and test a null hypothesis
which essentially asks “how likely is the effect
that we are testing” due to chance alone. For
example:
– In testing the claim that my ski wax provided a
significant advantage we tested the null hypothesis
that is did not and the effect I claimed could be
explained as a result of chance.
– A probability threshold called the statistical
significance level is set to decide to accept or reject
the null hypothesis.
Significance Levels…
• Symbol a denotes the significance level.
An a of 5% or 0.05 means that events
have a 1/20 chance of occurring by
chance
• US Supreme Court sets statistical
significance at 2s or 3s away from the
mean:
– 2s  a = 0.0223
– 3s  a = 0.0013
Stats 300 Causes Stress!
• An un-named student (whose initials are Carl)
claims that Stats 300 causes stress. To prove
this he measured the blood pressure of a SRS of
100 subjects at King’s between the ages of 18
and 36 and found a mean systolic blood
pressure of 122 with a standard deviation if 12.
He then took the blood pressure of the entire
class and found a blood pressure of 128 and
assumes the same standard deviation of 12 for
each reading. Does this evidence support Carl’s
claim?
• We are making the assumption that Carl’s
original SRS was normally distributed as is
the Stats 300 class
– Null Hypothesis The Stats 300 class has the same
blood pressure as the SRS, ie: No effect on stress. :
H0: m = 122
– The Alternative Hypothesis:
A one-sided alternative
Ha: m > 122
• We will test at the significance level by first
X m
128  122
z

 3.122
s
12
39
n
So … what’s this mean?
• A blood pressure of 128 is 3.122s above the
mean. Either:
– A) H0 is true and we just got 128 by chance
– B) H0 is false – STATS 300 really does cause
stress!
• So how likely is A)?
P(Z >= 3.122) implies
that this only occurs
with a probability of
p = 1 – 0.9991
= 0.0009! H0 is false!!
(Another way of thinking about this is that 99.91% of the readings
expected would be less than 128 – getting this by chance is pretty unlikely!)
One-Sided and Two-Sided
Alternatives
• There are three possible scenarios for the
alternative hypothesis:
• Ha: m > mo
• Ha: m < mo
• Ha: m ≠ mo
One-sided
Two-sided
One-sided and two-sided alternative hypotheses have
slightly different probability formulae
Probability Formulae…
• Ha: m > mo : P-value for H0
is P(Z ≥ z)
• Ha: m < mo : P-value for H0
is P(Z ≤ z)
• Ha: m ≠ mo : P-value for H0
is 2P(Z ≥ |z|)
Closer look … example 6.13
• Make the null hypothesis: “sample
contains 0.86% of the active ingredient” or
H0: m = 0.86
• Alternative is Ha: m ≠ 0.86
• P = 2P(Z ≥ |z|)
It could be more or less
So this is a two-sided
case
z = 4.99
The probability of the null hypothesis is less than 2P(Z≥4.99) = 2(1-1)=0!