Download lecture material - Institute for Molecular Bioscience

Y EAR 2015
Institute for Molecular Bioscience
Statistics for frightened
bioresearchers
A short course organised by the AIBN Student
Association and IMB Career Advantage
You are invited to a short course in biostatistics
This six lectures series is dedicated to students and researchers in the field of
molecular biology and biomedical research with no previous experience in data analysis.
1. Summarizing and presenting data
2. Hypothesis testing and statistical inference
3. Experimental designs
4. Comparing two groups
5. Comparing more than two groups
6. Data mining and exploration
With a focus on real world examples, notes and resources you can take home,
and quick exercises to test your understanding of the material, this short course
will cover many of the essential elements of biostatistics and problems that
commonly arise in analysis in the lab.
Time: 10:00am - 12pm Monday
June 15 IMB
June 29 IMB
Aug 3 AIBN
July 13 AIBN
July 27 IMB
Aug 24 AIBN
IMB Large Seminar Room, Level 3, Queensland Bioscience Precinct
(Building 80)
AIBN Seminar Room, Level 1, Australian Institute for Bioengineering and
Nanotechnology (Building 75)
Both at The University of Queensland, St Lucia Campus
Further information and
registration on
http://www.imb.uq.edu.au/
statistics
Please forward on to colleagues
who may be interested in attending
Enquiries
Dr Kim-Anh Lê Cao (DI)
t: 3443 7069 e: k.lecao@uq.edu.au
Ms Jessica Schwaber (AIBN)
t: 3346 4219 e: j.schwaber@uq.edu.au
Dr Nick Hamilton (IMB)
t: 3346 2033 e: n.hamilton@uq.edu.au
P RESENTER :
D R K IM -A NH L Ê C AO
II Hypothesis testing and
inference
1
2
3
4
5
6
1
2
3
Populations and samples . . . . . . . . . . . . . . . . . . . . . . . . .
Random variables and probability distributions . . . . . . . . . . . . .
2.1
Important terminology . . . . . . . . . . . . . . . . . . . . . .
2.2
The normal (gaussian) distribution . . . . . . . . . . . . . . . .
2.3
Examples of other distributions . . . . . . . . . . . . . . . . .
2.4
Which distribution to choose? . . . . . . . . . . . . . . . . . .
The central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . .
3.1
Standard deviation vs. Standard error of the mean . . . . . . . .
3.2
Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hypothesis testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1
The different steps in hypothesis testing . . . . . . . . . . . . .
4.2
What is a p-value? . . . . . . . . . . . . . . . . . . . . . . . .
4.3
One and two-sided tests . . . . . . . . . . . . . . . . . . . . .
Multiple testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Statistical table for a Normal Standard Distribution N (0, 1) . . . . . . .
Statistical table for Student t−test . . . . . . . . . . . . . . . . . . . .
Statistical table for a Chi-square Distribution with df degrees of freedom
34
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
35
35
36
36
40
41
41
42
43
43
44
47
47
48
52
54
56
57
P OPULATIONS AND SAMPLES
1 Populations and samples
Definition 17. A population is a complete set of individuals or objects that we want information
about. Ideally, we would collect the information about the whole population. However, this
might be too expensive, in time or money, or simply impossible.
Instead we have to take a sample, a subset of the population, and use the data from the sample
to infer something about the population as a whole. The sample should be chosen so that it is
representative of the population of interest but also that is it not biased in any way. One way of
achieving this is to take a random sample from the population.
Definition 18. When we want to draw wider conclusions from an experiment, we need to be
clear about what the population of interest will be and we need to obtain a representative sample
from that population. Selection bias occurs when the sample itself is unrepresentative of the
population we are trying to describe.
Figure II.1: Are the results of my study valid? Do they apply to a larger group than just the participants in the
study? In Statistics, we often rely on a sample - that is, a small subset of a larger set of data to draw inferences
about the larger set. The larger set is known as the population from which the sample is drawn. We need to
define the population to which inferences are sought and design a study in which the biological samples have
been appropriately selected from the chosen population.
Remark 13. The best way to ensure that a sample will lead to realiable and valid inferences is to
use probability sampling methods such as simple random sampling, systematic sampling, stratified
sampling or cluster sampling. These concepts will not be covered in the course, but a good reference is
summary can be found in http://www.socialresearchmethods.net/kb/sampprob.
php.
2 Random variables and probability distributions
In Chapter I, we have seen the difference between parameters, which are fixed and invariant
characteristics from a population, and statistics, which are estimates of the population parameters
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
35
R ANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS
36
based on a sample. We use Greek letters for population parameters, and Roman letters for sample
statistics.
2.1
Important terminology
Definition 19. A random variable (also called stochastic variable) is a random process with a
numerical outcome, i.e. a variable whose value is subject to variations due to chance. A random
variable does not have a single fixed value, it can take a set of possible different values. Each of
them are associated to a probability.
A discrete random variable is a random variable with discrete outcome, i.e. it assumes any of
a specified list of exact values. A continuous random variable assumes any numerical value in
an interval or collection of intervals (continuous outcome).
Definition 20. The values of a random variable can be summarised in a frequency distribution
called a probability distribution.
Theoretical probability distributions are important in statistics. Those commonly encountered
include the binomial and the Poisson discrete distributions, and the normal (also called gaussian)
continuous distribution.
2.2
The normal (gaussian) distribution
(b)
0.4
0.3
0.2
normal density
0.6826 of the
total area
0.0
0.1
0.2
0.0
0.1
normal density
0.3
0.4
(a)
µ
µ−σ
µ
µ+σ
Figure II.2: (a) Density of the standard normal distribution, (b) Area under normal curve within 1 standard
deviation of mean. The normal distribution is a probability distribution, therefore, the total area under the
curve is equal to 1.
Describing the normal distribution. The normal distribution has a bell-shaped curve and is continuous (Fig. II.2 (a)). It is symmetric around the mean (denoted by µ). The standard deviation is
denoted by σ, which is the horizontal difference between the mean and the point of inflection1 on the
curve (Fig. II.2 (b)). When µ increases the distribution moves to the right, if σ is small (large) then
the distribution is steep (flat).
The parameters µ and σ completely determine the shape of a normal curve. In mathematical terms, if
1
where the curve changes from convex to concave
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
R ANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS
we assume that a random variable X follows a normal distribution with mean µ and variance σ 2 , we
will write: X ∼ N (µ, σ 2 ).
Example 7. Some examples of possibly normally distributed variables:
• In biology, the logarithm of various variables tend to have a normal distribution (e.g physiological measurements, height, length, growth, ...)
• Measurement errors in physical experiments,
• IQ scores.
Figure II.3: There exists an infinity of Normal distributions with different means and different standard
deviations. This is an example of density plots for four normally distributed random variables (source:
www.wikipedia.org).
The z (standard normal) distribution. There are numerous Normal distributions (with different
means and different standard deviations, Figure II.3) and hence it is more practical to focus on a
single Normal distribution with mean 0 and standard deviation 1 (the standard Normal distribution).
We can transform the data by substracting the mean of the data (µ) and by dividing by the standard
deviation (σ) in order to obtain a Normal distribution N (0, 1). The standardized values are now called
z-scores, according to the following formula:
Definition 21. Definition of the Z-score.
If X ∼ N (µ, σ 2 ), then
Z=
X −µ
σ
so that
Z ∼ N (0, 1).
Exercise using the standard normal distribution.
Exercise 8. We recorded the mean 24h systolic pressure in mm Hg in healthy individuals. We
assume the mean to be µ = 120 and the standard deviation σ = 10a . We assume the random
variable X = ’mean 24h systolic pressure’ to be normally distributed, X ∼ N (µ, σ 2 ).
You will describe the appropriate transformation of the data and sketch the distributions to illustrate the answers to the following questions (no exact answer is required).
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
37
R ANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS
38
1. What area of the curve is above 130 mm Hg?
2. What area of the curve is between 100 and 140 mm Hg?
3. What is the value of the systolic BP that divides the area under the curve into the lower
95% and the upper 5%?
a
In this exercise, we assume these parameters to be the true population parameters
1. Using the z-score transformation, we have a random variable Z = X−µ
σ , with Z ∼ N (0, 1).
The density plot is represented in the graph below. The z-score value for a BP = 130. We have
z = 130−µ
=1, which value is represented as a vertical line:
σ
0.2
0.3
0.4
Q1
0.0
0.1
area = 0.159
−4
−2
0
2
4
The area under the curve can be obtained using statistical software, with either the normal distribution N (120, 102 ) or the N (0, 1) distribution using the z-score transformation. With R,
we use the command line pnorm(1, mean = 0, sd = 1) = 0.841, which is the non
shaded area under the curve on the left hand side of the graph. We know that the total area
under the curve is equal to 1, therefore, the shaded area on the right hand side is 1 - 0.841 =
0.159.
Note that we could have used the N (µ = 120, σ 2 = 102 ) distribution and obtained the same value with pnorm(130,
mean = 120, sd = 10) = 0.841.
2. Similarly to answer above, we need to calculate the z-scores of the two boundary values. We
have z1 = 100−µ
=-2 and z2 = 140−µ
=2, which we represent on the N (0, 1) density plot
σ
σ
(vertical lines):
0.2
0.3
0.4
Q2
0.0
0.1
area = 0.954
−4
−2
0
2
4
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
R ANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS
39
The shaded area under the curve is computed as P(z1 ≤ Z ≤ z2 ) = pnorm(2, mean = 0,
sd = 1) - pnorm(-2, mean = 0, sd = 1) = 0.954. This is the probability that
the z-transformed BP is between the values -2 and 2.
3. For this question we need to work backward and find the z-score value that will divide the
distribution into 95% on the left hand side and 5% on the right hand side. This value can
be found in a N (0, 1) Statistical Table (Appendix 1, where the area under the curve of 0.95
is highlighted in red, and the z-score value is in blue). The z-score value is 1.645 and is
represented on the probability distribution plot:
0.2
0.3
0.4
Q3
area = 0.05
0.0
0.1
area = 0.95
−4
−2
0
2
4
From there we need to calculate the corresponding BP value, i.e. solve the equation
z=
BP − m
s
Solving the equation gives BP = 136.45. Therefore, a BP value of 136.45 is the 95th percentile
(i.e. 95% of the patients have a systolic BP of 136.45 or lower, if we assume that the BP of the
whole patient population ∼ N (µ, σ 2 )).
Remark 14. I have used the R statistical software, but there are functions in Excel such as NORM.DIST()
to obtain the probabilities in Q1 and Q2 and NORM.INV() to obtain the BP value in Q3, see
http://exceluser.com/formulas/statsnormal.htm for a tutorial.
Remark 15. The Empirical Rule is a statistical rule stating that for a normal distribution, almost all
data will fall within 3 standard deviations of the mean. It also shows that 68% of the data will fall
within the first standard deviation, that 95% of the data will fall within two standard deviations, and
that 99.7% of the data will fall within three standard deviations. We also call this the ‘Three Sigma
Rule’.
The normal distribution forms the basis of statistical inference, even if the population is not exactly normally distributed. The Empirical Rule is a good rule of thumb for approximately normal
distributions.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
R ANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS
2.3
40
Examples of other distributions
Figure II.4: Examples of common continuous and discrete (represented with histograms) probability distributions. Each type of distribution can be characterised by a set of parameters (source: Crystall Ball software).
The theoretical distributions are important in statistics for hypothesis testing and statistical modelling. Each type of distribution depends on one or several parameters (for example the mean and
variance for a Normal distribution). In Chapter I we have emphasized the importance of identifying
the right type of measurement. This is because different types of distributions exist and their use in
hypothesis testing or statistical modelling depends on whether the variable is discrete or continuous.
Some common distributions are illustrated in Figure II.4.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
T HE CENTRAL LIMIT THEOREM
2.4
Which distribution to choose?
Figure II.5: An example of distribution choice diagram from http://pages.stern.nyu.edu/
~adamodar/New_Home_Page/StatFile/statdistns.htm. Note that the Poisson distribution is
not represented, where would it fit?
3 The central limit theorem
Example 8. The average GPA at a particular school is 2.89 with a standard deviation of 0.63. If we
take a random sample of 35 students, what is the probability that the average GPA for this sample is
greater than 3.0?
The central limit theorem is one of the most important theorems in statistics and determines the
sampling distribution of the mean to answer the question posed above.
Theorem 1. Given a population with mean µ and standard deviation σ, the sampling distribution
of the mean based on repeated random samples of size n from that same population has the same
√
mean µ (the mean of the means) and a standard deviation of σ/ n (the standard error of the
mean).
The consequence of that theorem is that no matter what type of distribution the population follows,
the sampling distribution of the mean from that population will be approximately (or exactly if the
population is normally distributed) normally distributed.
Example 9. In the plots in Figure II.6, the population is assumed to be known (blue curve) and
follows either a (a) normal, (b) skewed or (c) uniform distribution. The true population mean µ from
the distribution is indicated as a vertical line. Using a statistical software, we have then sampled a
hundred times from that same distribution a number of measurements from n = 5 to n = 30 and we
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
41
T HE CENTRAL LIMIT THEOREM
42
have represented the density distribution of the mean of 100 sampling distributions (dashed curves).
We consider the following population distributions:
(a) The population is assumed to be normal with mean 0 and standard deviation 1 (N (0, 1)) and
the true population mean µ is equal to 0.
(b) The population is assumed to follow a χ2 distribution with 2 degrees of freedom, and the true
population mean is equal to 2 (theoretical result from a χ22 distribution).
(c) The population is assumed to follow a uniform distribution, and the true population mean is
equal to 0.5 (theoretical result from a uniform distribution).
From the sampling distribution of the means, we can observe that the mean of the sampling
distributions approaches the true mean when the sample size increases. The variability of the mean
decreases.
8
population
n=5
n = 10
n = 20
n = 30
population
n=5
n = 10
n = 20
n = 30
−4
−2
0
2
4
4
Density
0
0.0
0.0
0.5
2
0.5
1.0
Density
1.5
1.0
6
population
n=5
n = 10
n = 20
n = 30
2.0
(c)
1.5
(b)
2.5
(a)
−2
0
2
4
6
−0.5
0.0
0.5
Figure II.6: Distribution of the mean from a (a) N (0, 1), (b) χ22 and (c) uniform distribution (in blue). The true
population mean is represented as a vertical line.
How large should the sample size be for the Central Limit Theorem to hold? Simulation
studies have shown that the CLT holds for a sample size > 30. However, it will depend on the
population distribution (symmetric: n < 30, heavily skewed: n > 30).
Remark 16. In practice, selecting repeated samples of size n and generating a sampling distribution
for the mean is not necessary. Instead, only one sample is selected, and we calculate the sample mean
as an estimate of the sample population. We can invoke the CLT if the sample size is > 30 to justify
that the sampling distribution of the mean is known.
3.1
Standard deviation vs. Standard error of the mean
In Chapter I Section 3.2 we covered the definitions of the standard deviation and the standard
error of the mean. In fact, the CLT illustrates the difference between the two measures. If we take
the example of the sampling distribution of the mean from a normally distributed population from
Example 9, we can easily visualise the difference between the SD and the SEM (Fig. II.7). The SD
is related to the original population (or sample) whereas the SEM is related to the mean of repeated
samplings from the original population. When the number of samples increases, the SEM decreases
as the distribution of the means converges to the real mean population µ.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
1.0
1.5
H YPOTHESIS TESTING
43
population
n=5
n = 30
σ/√n"
σ"
μ"
Figure II.7: Difference between SD and SEM. Distribution of the mean from a normally distributed population
(in blue). The true population mean is represented as a vertical line.
3.2
Exercise
Exercise 9. The average GPA at a particular school is 2.89 with a standard deviation of 0.63. If
we take a random sample of 35 students, what is the probability that the average GPA for this
sample is greater than 3.07?
Let us denote X a random variable representing the GPA at a particular school. X ∼ N (µ =
2.89, σ 2 = 0.632 ). Since n = 35 > 30, we can use the CLT which states that X̄ follows a nor√
mal distribution with mean µ and standard deviation σ/ n.
We are looking for the probability P(X̄ > 3.07).
We use the z-score transformation for the mean (i.e. z =
P(X̄ > 3.07) = P(z >
3.07−2.89
√ )
0.63/ 35
X̄−µ
√ )
σ/ n
and we obtain:
= P(z > 0.04829).
Since z ∼ N (0, 1), we can use a statistical table or a statistical software to obtain the value of
that probability. Using R we obtain:
1 - pnorm(x = 3.07, mean = 2.89, sd = 0.63) = 0.3875.
Note that the function pnorm() gives the probability that a measurement is below a given value. To obtain the
probability above a given value we need to use 1 - pnorm() as the sum of the probabilities for all possible values
of a random variable is equal to 1. This property was also used in Exercise 8.
4 Hypothesis testing
Methods for making inferences about parameters fall into one of two categories: either we will
estimate the value of the population parameter of interest, or we will test a hypothesis about the
value of the parameter. This involves different procedures, answering different types of questions.
In estimating a population parameter we are answering the question: ‘What is the value of the population parameter’? We have seen how to estimate some of these parameters in Chapter I.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
H YPOTHESIS TESTING
44
In testing a hypothesis, we are for example answering the question: ‘Does the population parameter
satisfy µ > 20?’ which is the focus on hypothesis testing.
4.1
The different steps in hypothesis testing
Definition 22. A statistical test is based on the concept of proof by contradiction and is composed of the five parts listed here:
1. State the null hypothesis H0 ,
2. State the research hypothesis (alternative hypothesis): H1 ,
3. Compute the test statistic (T.S),
4. Define the rejection region to reject H0 ,
5. Check assumptions and draw conclusions.
Hypothesis test. Testing hypotheses is an essential part of statistical inference. To formulate a test,
the question of interest is simplified into two competing hypotheses: the null hypothesis, denoted H0 ,
against the alternative (also called research) hypothesis, denoted H1 .
Usually, we carry out an experiment to reject the null hypothesis. For example:
H0 : there is no difference in taste between black tea and green tea,
against
H1 : there is a difference in taste between black tea and green tea.
The hypotheses are often statements about population parameters such as sample means between
different groups or subpopulations (e.g. the mean height in ten year old boys and girls), or a statement
about the distribution of variable of interest (the height of ten year old boys is normally distributed).
The null hypothesis. H0 represents a theory is believed to be true (or has not been proved yet). For
example, in a clinical trial of a new drug, the null hypothesis might be that the new drug is no better,
on average, than the current drug:
H0 : there is no difference between the two drugs on average.
The alternative hypothesis. H1 states the statistical hypothesis test to establish. In a clinical trial
of a new drug, we would write: H1 : the two drugs have different effects, on average (two-sided test,
see Section 4.3).
It could also be:
H1 : the new drug is better than the current drug, on average (one-sided tests).
The null hypothesis is statistically tested against the alternative using a suitable distribution of a
statistic, where the statistic is computed from the experimental data. By comparing the statistic with
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
H YPOTHESIS TESTING
45
its distribution, we can draw a conclusion with respect to the null hypothesis and reject H0 or not.
The choice of a test statistic will depend on the assumed probability model and the hypotheses tested.
Figure II.8: Type I and type II errors.
The probability of wrongly rejecting H0 when in fact H0 is true is called the significance level,
generally denoted α and usually set to α = 5W (but this is an arbitrary threshold).
A type I error is committed if we reject the null hypothesis when it is true. The probability of
a type I error is α, the significance level. A type II error is committed if we do not reject the null
hypothesis when it is false and the research hypothesis is true. The probability of a type II error is β
(see Figure II.8).
The power of a statistical test is the probability that it correctly rejects the null hypothesis when the
null hypothesis is false, i.e. the ability of a test to detect an effect, if the effect actually exists. The
power is equal to 1 − β.
The critical value(s) for a hypothesis test is a threshold to which the value of the test statistic in a
sample is compared to a theoretical value to decide whether or not the null hypothesis is rejected.
The critical value for any hypothesis test depends on the significance level at which the test is
carried out, and whether the test is one-sided or two-sided (see Section 4.3).
The critical region or rejection region, is a set of values of the test statistic for which the null
hypothesis is rejected. If the observed value of the test statistic belongs to the critical region, we
conclude ‘Reject H0 ’ otherwise we ‘Do not reject H0 ’.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
H YPOTHESIS TESTING
46
The final conclusion once the test has been carried out is always given in terms of the null hypothesis. We either "Reject H0 in favour of H1 " or we ‘Do not reject H0 ’. We never conclude ‘Reject
H1 ’, or even ‘Accept H1 ’.
Remark 17. If we conclude ‘Do not reject H0 ’, this does not necessarily mean that the null hypothesis
is true, it only suggests that there is not sufficient evidence against H0 in favour of H1 . Rejecting the
null hypothesis then, suggests that the alternative hypothesis may be true.
Example 10. An agricultural service wants to determine whether the mean yield per acre (in bushels)
for a particular variety of soybeans has increased since last year. The mean yield was 520 bushels per
acre. We have a sample of n = 36 one-acre plots. This is a simple illustrative example using a z-test
and the CLT, where we assume that the standard deviation is known and is equal to s = 124. From
the current data we compute the sample mean x̄ = 573. Can we conclude that the mean yield for all
farms is above the target from last year of 520?
f (x)
area α
µ = 520
acceptance region
rejection region
Figure II.9: Rejection region for the soybean example for H1 : µ > 520
1. H0 : µ ≤ 520
2. H1 : µ > 520
3. Under H0 , T.S: z =
x̄−520
√ ,
124/ 36
with z ∼ N (0, 1), see Theorem 1.
4. Rejection region: we reject H0 if z ≥ zα , with α = 0.05.
The shaded area in Figure II.9 illustrates the rejection region with an area α in the right tail of the
distribution of x̄. Determining the location of this rejection area is equivalent to determining the
z value that has an area α to its right (we set the significance level α = 0.05). A statistics table
for the N (0, 1) (see Appendix Table 1) indicates that this value is T.Sα = 1.645 (also given by
qnorm(0.95, 0, 1)).
We obtain
x̄ − µ
573 − 520
√
T.S = z = √ =
= 2.56
σ/ n
124/ 36
5. Since we have z ≥ zα , we reject H0 in favor of the alternative hypothesis and conclude that
the average soybean yield per acre is significantly greater than last year’s target of 520.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
H YPOTHESIS TESTING
4.2
47
What is a p-value?
The rejection region can also be determined based on the p-value associated with the statistical
test. The p-value determines the significance of the results and is usually obtained using a statistical
software.
Definition 23. The p-value is the probability of obtaining a test statistic result at least as extreme
as the one that was actually observed, assuming that the null hypothesis is true.
The p-value is a number between 0 and 1 and is interpreted as follows:
• If the p-value ≤ α, we reject H0 as there is strong evidence against the null hypothesis.
• If the p-value > α, we fail to reject H0 as there is weak evidence against the null hypothesis.
Note that usually the significance level α is set to 0.05.
4.3
One and two-sided tests
(a)
f (x)
µ = 520
rejection region
(b)
f (x)
µ = 520
rejection region
rejection region
Figure II.10: (a) One and (b) two sided tests rejection regions based on the the soybean example.
Example of a one-sided test. In the example above we conducted a one-sided test where H1 : µ ≥
520. If our alternative hypothesis was instead H1 : µ ≤ 520, small values of x̄ would indicate the
rejection of null hypothesis. The rejection region would be located in the lower tail of the distribution
of x̄ (Fig. II.10 (a)), i.e. we reject H0 if z ≤ zα .
Example of a two-sided test. A two-sided test could be formulated for H1 : µ ̸= 520 where both
large and small values of x̄ would contradict the null hypothesis and the rejection region would be
located in both tails of the distribution of x̄ (Fig. II.10 (b)), i.e. we reject H0 if z ≤ zα/2 and z ≥ zα/2 ,
which we often write: we reject H0 if |z| ≥ zα/2 . In that case, we need the entire shaded region to be
equal to α, hence the zα/2 .
Remark 18. The one-sided test has the advantage over the two-tailed test of obtaining statistical
significance with a smaller departure from the hypothesized value, as there is interest in only one
direction. The use of a one-sided test must be done with caution. In medical research, we frequently
choose a two-sided test, even if we have an expectation about the direction of the test.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
M ULTIPLE TESTING
48
5 Multiple testing
Why multiple testing matters. The problem of multiple testing occurs in large data sets, such
as biological data sets, where we can perform, say, 10,000 separate hypothesis tests (one for each
measure). If we use a standard p-value cut-off of 0.05, then we expect 500 genes to be deemed
“significant" by chance.
In general, if we perform m hypothesis tests, what is the probability of at least 1 false positive
(i.e. rejecting the null hypothesis wrongly)?
• P(Making an error) = α (α is the significance level)
• P(Not making an error) = 1 - α
• P(Not making an error in m tests) = (1 − α)m
• P(Making at least 1 error in m tests) = 1 − (1 − α)m
An ANOVA test (see following Chapter) tells us if at least one mean group is different from the
other mean groups, but its does not tell us which groups have means that are significantly different.
One straightforward procedure is to perform pairwise t-tests between each of the two groups. However, when K is large there are K(K − 1)/2 comparisons to perform. If we set up a significance level
at α = 5W (i.e. there is 5% chance of making a mistake by wrongly rejecting the null hypothesis),
it means that there is a 95% chance of not making a mistake. If we perform 3 pairwise t-tests, the
probability of making no mistake is 0.95×0.95×0.95 = 0.8574 (if we assume the tests independent).
So the chance of making at least one mistake is 14.26%.
The probability of falsely rejecting at least one of the hypotheses increases as the number of tests
increases (see Figure II.11). Thus, even if we have the probability of type I error at α = 5W for each
individual test, the probability of falsely rejecting at least one of those tests is larger than 0.05.
Figure II.11: Probability of making at least one Type I error as a function of the number of independent
hypothesis tests performed on the same dataset when all null hypotheses are true and α = 0.05, from Ilakovac
(2009).
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
M ULTIPLE TESTING
49
Remark 19. The problem of multiple testing is very common to high throughput experiments (microarrays, RNA-seq, GWAS, and so on). However, multiple testing also occur in a modest number of
comparisons, as shown above for the ANOVA example.
Correcting for multiple testing: FWER, FDR, Q-value. ‘Adjusting p-values for the number of
hypothesis tests performed’ or ‘correcting for multiple testing’ means that we are controlling the
Type I error rate (see Table II.1). Many different methods have been proposed to control for multiple
testing. Although these approaches have the same goal, they go about it in fundamentally different
ways.
Table II.1: Number of errors committed when performing m hypothesis tests. m0 is the number of true null
hypothesese (‘do not reject H0 ’) and R is the number of rejected null hypotheses, V is the number of false
positives.
Not called significant
Called significant
Null true
U
V
m0
Alternative True
T
S
m − m0
Total
m−R
R
m
Correcting for multiple testing is still a very active area of statistics. We list a few methods below,
and also refer to Noble (2009) for some guidelines. We will briefly discuss the FWER and the FDR.
• Per comparison error rate (PCER): the expected value of the number of Type I errors over the
number of hypotheses, PCER = E(V )/m
• Per-family error rate (PFER): the expected number of Type I errors, PFE = E(V )
• Family-wise error rate (FWER): the probability of at least one type I error FEWR = P(V ≥ 1)
• False discovery rate (FDR) is the expected proportion of Type I errors among the rejected
hypotheses FDR = E( VR |R > 0)P(R > 0)
• Positive false discovery rate (pFDR): the rate that discoveries are false pFDR = E(V /R|R > 0)
The Bonferroni correction is a FWER approach that ensures that the overall Type I error rate
α is maintained while performing m independent hypothesis tests. It rejects any hypothesis with a
p-value ≤ α/m. For example, if m = 10, 000, then we need p-value 0.05/10000 = 5−06 to declare
significance.
The adjusted Bonferroni p−value is
p̃j = min(mpj , 1),
where pj is the raw p−value of the jth test over m tests and p̃j is the Bonferroni adjusted p-value.
The Bonferroni adjustment is very stringent and somewhat counter-intuitive as the interpretation
of findings depends on the number of other tests performed. The general null hypothesis is that all
the null hypotheses are true and it has a high probability of type II errors (i.e. not rejecting H0 when
there is a true effect).
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
M ULTIPLE TESTING
50
The False Discovery Rate (FDR, Benjamini and Hochberg 1995) is designed to control the proportion of false positives among the set of rejected hypotheses (R). To control for a FDR at level
δ:
• Order the unadjusted p-values: p(1) ≤ p(2) ≤ · · · ≤ p(m)
• Find the test with the highest rank j for which the p-value p(j) ≤
j
m
∗δ
• Declare the tests with rank 1, 2, . . . , j as significant.
An example of the FDR is illustrated in Table II.2.
Table II.2: FDR - Benjamini Hochberg example for δ = 0.05. The largest rank for which the p−value is less
than its threshold is indicated in red.
Rank
1
2
3
4
5
6
7
8
9
10
raw p-value
0.0008
0.009
0.165
0.205
0.396
0.450
0.641
0.781
0.900
0.993
j
m
∗ 0.05
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.050
Reject H0
yes
yes
no
no
no
no
no
no
no
no
The adjusted FDR Benjamini and Hochberg p−values are defined as:
p̃(1) = min(p̃(2) , mp(1) )
..
.
m
p̃(m−1) = min(p̃(m) ,
p
)
m − 1 (m−1)
p̃(m) = p(m)
where p(j) is the jth ranked raw p−value amongst the m tests (p-values) and p̃(j) is the adjusted
p−value ranked jth.
For the example above, the raw and adjusted FDR p-values are displayed in Table II.3.
The q-value (Storey 2002) is the minimum FDR at which a feature may be declared significant,
i.e. the expected proportion of false positives incurred when we call a feature significant. The estimated q-value is a function of the p-value for that test and the distribution of the entire set of p-values
from the family of tests being considered. For example, in a microarray study testing for differential
expression, if a gene X has a q−value = 0.013, it means that 1.3% of the genes with a p-value less
than that gene’s p-value (i.e. ranked before gene X with decreasing significance) are false positives.
The q-value complements the FDR.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
M ULTIPLE TESTING
51
Table II.3: Raw and FDR adjusted p-values
1
2
3
4
5
6
7
8
9
10
raw p-values
0.0008
0.0090
0.1650
0.2050
0.3960
0.4500
0.6410
0.7810
0.9000
0.9930
adjusted p-values
0.0080
0.0450
0.5125
0.5125
0.7500
0.7500
0.9157
0.9763
0.9930
0.9930
Figure II.12 shows the histograms of p-values that we expect to see when there are no significant changes in the experiment (a) and with expected changes (b). The q-value approach tries to
find the threshold where the p-value distribution flattens out and incorporates this threshold into the
calculation of the FDR adjusted p-values (c). This approach helps to establish just how many of the
significant values are actually false positives (the red portion of the green bar).
(a)$
(b)$
(c)$
Figure II.12: Histograms of p-values. (a) when there is no significant changes in the experiment (in red,
the significant p-values that are expected to be false positives), (b) when there are significant changes (in
green the significant p-values) and (c) q-value (dashed line) and proportion of false positives (red). Source:
http://www.totallab.com/.
When not to correct for multiple comparisons? The goal of multiple comparisons corrections
is to reduce the number of false positives. But the consequence is that we increase the number of
false negatives where there really is an effect but it is not detected as statistically significant. If false
negatives are very costly, you may not want to correct for multiple comparisons.
Let us assume that a knock out experiment is performed between different strains of mice developing
a tumour. The smallest p-value obtained is 0.013 for a difference in tumor size. If we were to use
either a Bonferroni correction or the Benjamini and Hochberg procedure, the adjusted p-value will
not be significant.
This kind of result can be cautiously reported as a ‘possible effect of gene X on cancer’. The cost of
a false positive–if further experiments are conducted, is a few more experiments. The cost of a false
negative, on the other hand, can be to miss an important discovery.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
S UMMARY
52
6 Summary
In this Chapter, we have seen the important definitions of random variables, the difference between population and sample and what is a probability distribution. We have mainly emphasized the
normal and standard normal distribution and how to calculate a z-score.
The central limit theorem tells us that means of observations, regardless of how they are distributed, begin to follow a normal distribution as the sample size increases. This is one of the reasons
why the normal distribution is important in statistics. Through this theorem, we have seen the difference between the standard error and the standard error of the mean.
One of the purpose of statistics is to use a sample to estimate a parameter of the population. We
have seen the different steps in hypothesis testing, the difference between one and two-sided tests,
how to define a critical region and what is a p-value.
Finally, we have discussed the topical issue of multiple testing and different approaches to adjust
for multiple testing, such as the FWER, FDR and the q-value.
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
Bibliography
Anderson, E. (1935). The irises of the gaspe peninsula. Bulletin of the American Iris Society, 5, 2–5.
Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful
approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological),
pages 289–300.
Bonita, R., Beaglehole, R., and Kjellström, T. (2006). Basic epidemiology. World Health Organization.
Colton, T. (1974). Statistics in medicine, 1974, volume 164. Little, Brown, Boston.
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics,
pages 179–188.
Ilakovac, V. (2009). Statistical hypothesis testing and some pitfalls. Biochemia Medica, 19(1), 10–16.
Noble, W. S. (2009). How does multiple testing correction work? Nature biotechnology, 27(12),
1135–1137.
Reshef, D. N., Reshef, Y., Finucane, H. K.and Grossman, S. R., G., M., P.J., T., Lander, E. S., M., M.,
and Sabeti, P. (2011). Detecting novel associations in large data sets. Science, 16, 1518–1524.
Storey, J. D. (2002). A direct approach to false discovery rates. Journal of the Royal Statistical
Society: Series B (Statistical Methodology), 64(3), 479–498.
53
I Statistical Tables
1 Statistical table for a Normal Standard Distribution
N (0, 1)
This table gives P(Z ≤ z) where Z ∼ N (0, 1). The probability is schematically represented in
blue in the diagram:
The z value (also called critical value) reads down the first column for the ones and tenths places, and
along the top row for the hundredths place.
Example 11. If the critical value z is z = 1.67, then you would split this number into 1.67 =
1.6 + 0.07. The number located in the 1.6 row and 0.07 column is 0.953. Thus 95.3% of the area
under the bell curve is to the left of z = 1.67.
The table may also be used to find the area to the left of a negative z-score. To do this, drop the
negative sign and look for the appropriate entry in the table. After locating the area, subtract 0.5 to
adjust for the fact that z is a negative value. (from http://statistics.about.com/).
S TATISTICAL TABLE FOR A N ORMAL S TANDARD D ISTRIBUTION N (0, 1)
z
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
3.0
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
Normal cumulative distribution function
second decimal place of z
0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359
0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753
0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141
0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517
0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879
0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224
0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549
0.7580 0.7611 0.7642 0.7673 0.7703 0.7734 0.7764 0.7794 0.7823 0.7852
0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133
0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389
0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621
0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830
0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015
0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177
0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319
0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441
0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545
0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633
0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706
0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767
0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817
0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857
0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890
0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916
0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936
0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952
0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964
0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974
0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981
0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986
0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990
0.9990 0.9991 0.9991 0.9991 0.9992 0.9992 0.9992 0.9992 0.9993 0.9993
0.9993 0.9993 0.9994 0.9994 0.9994 0.9994 0.9994 0.9995 0.9995 0.9995
0.9995 0.9995 0.9995 0.9996 0.9996 0.9996 0.9996 0.9996 0.9996 0.9997
0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9998
0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998
0.9998 0.9998 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999
0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999
0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999
1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
55
2 Statistical table for Student t−test
df = 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
35
40
45
50
55
60
∞
Percentage points of Student’s t distribution (right-tail probability α)
α = 60.0% 66.7% 75.0% 80.0% 87.5% 90.0% 95.0% 97.5% 99.0% 99.5% 99.9%
0.325 0.577 1.000 1.376 2.414 3.078 6.314 12.706 31.821 63.657 318.31
0.289 0.500 0.816 1.061 1.604 1.886 2.920 4.303 6.965 9.925 22.327
0.277 0.476 0.765 0.978 1.423 1.638 2.353 3.182 4.541 5.841 10.215
0.271 0.464 0.741 0.941 1.344 1.533 2.132 2.776 3.747 4.604 7.173
0.267 0.457 0.727 0.920 1.301 1.476 2.015 2.571 3.365 4.032 5.893
0.265 0.453 0.718 0.906 1.273 1.440 1.943 2.447 3.143 3.707 5.208
0.263 0.449 0.711 0.896 1.254 1.415 1.895 2.365 2.998 3.499 4.785
0.262 0.447 0.706 0.889 1.240 1.397 1.860 2.306 2.896 3.355 4.501
0.261 0.445 0.703 0.883 1.230 1.383 1.833 2.262 2.821 3.250 4.297
0.260 0.444 0.700 0.879 1.221 1.372 1.812 2.228 2.764 3.169 4.144
0.260 0.443 0.697 0.876 1.214 1.363 1.796 2.201 2.718 3.106 4.025
0.259 0.442 0.695 0.873 1.209 1.356 1.782 2.179 2.681 3.055 3.930
0.259 0.441 0.694 0.870 1.204 1.350 1.771 2.160 2.650 3.012 3.852
0.258 0.440 0.692 0.868 1.200 1.345 1.761 2.145 2.624 2.977 3.787
0.258 0.439 0.691 0.866 1.197 1.341 1.753 2.131 2.602 2.947 3.733
0.258 0.439 0.690 0.865 1.194 1.337 1.746 2.120 2.583 2.921 3.686
0.257 0.438 0.689 0.863 1.191 1.333 1.740 2.110 2.567 2.898 3.646
0.257 0.438 0.688 0.862 1.189 1.330 1.734 2.101 2.552 2.878 3.610
0.257 0.438 0.688 0.861 1.187 1.328 1.729 2.093 2.539 2.861 3.579
0.257 0.437 0.687 0.860 1.185 1.325 1.725 2.086 2.528 2.845 3.552
0.257 0.437 0.686 0.859 1.183 1.323 1.721 2.080 2.518 2.831 3.527
0.256 0.437 0.686 0.858 1.182 1.321 1.717 2.074 2.508 2.819 3.505
0.256 0.436 0.685 0.858 1.180 1.319 1.714 2.069 2.500 2.807 3.485
0.256 0.436 0.685 0.857 1.179 1.318 1.711 2.064 2.492 2.797 3.467
0.256 0.436 0.684 0.856 1.178 1.316 1.708 2.060 2.485 2.787 3.450
0.256 0.436 0.684 0.856 1.177 1.315 1.706 2.056 2.479 2.779 3.435
0.256 0.435 0.684 0.855 1.176 1.314 1.703 2.052 2.473 2.771 3.421
0.256 0.435 0.683 0.855 1.175 1.313 1.701 2.048 2.467 2.763 3.408
0.256 0.435 0.683 0.854 1.174 1.311 1.699 2.045 2.462 2.756 3.396
0.256 0.435 0.683 0.854 1.173 1.310 1.697 2.042 2.457 2.750 3.385
0.255 0.434 0.682 0.852 1.170 1.306 1.690 2.030 2.438 2.724 3.340
0.255 0.434 0.681 0.851 1.167 1.303 1.684 2.021 2.423 2.704 3.307
0.255 0.434 0.680 0.850 1.165 1.301 1.679 2.014 2.412 2.690 3.281
0.255 0.433 0.679 0.849 1.164 1.299 1.676 2.009 2.403 2.678 3.261
0.255 0.433 0.679 0.848 1.163 1.297 1.673 2.004 2.396 2.668 3.245
0.254 0.433 0.679 0.848 1.162 1.296 1.671 2.000 2.390 2.660 3.232
0.253 0.431 0.674 0.842 1.150 1.282 1.645 1.960 2.326 2.576 3.090
Remark 20. For two-tailed tests, use the value in column headed by α/2.
The first column indicate the number of degrees of freedom.
S TATISTICAL TABLE FOR A C HI - SQUARE D ISTRIBUTION WITH df
DEGREES OF FREEDOM
3 Statistical table for a Chi-square Distribution with df
degrees of freedom
The following tables gives the probability that a random variable X ∼ χ2df is less that a critical
value (i.e. the area to the left of the critical value).
For example, if X ∼ χ23 , then S`[X ≤ 7.815] = 0.95.
df
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0.01
0.000
0.020
0.115
0.297
0.554
0.872
1.239
1.646
2.088
2.558
3.053
3.571
4.107
4.660
5.229
5.812
6.408
7.015
7.633
8.260
8.897
9.542
10.196
10.856
11.524
12.198
12.879
13.565
14.256
14.953
0.025
0.001
0.051
0.216
0.484
0.831
1.237
1.690
2.180
2.700
3.247
3.816
4.404
5.009
5.629
6.262
6.908
7.564
8.231
8.907
9.591
10.283
10.982
11.689
12.401
13.120
13.844
14.573
15.308
16.047
16.791
S`[X ≤ x]
0.05
0.95
0.004
3.841
0.103
5.991
0.352
7.815
0.711
9.488
1.145 11.070
1.635 12.592
2.167 14.067
2.733 15.507
3.325 16.919
3.940 18.307
4.575 19.675
5.226 21.026
5.892 22.362
6.571 23.685
7.261 24.996
7.962 26.296
8.672 27.587
9.390 28.869
10.117 30.144
10.851 31.410
11.591 32.671
12.338 33.924
13.091 35.172
13.848 36.415
14.611 37.652
15.379 38.885
16.151 40.113
16.928 41.337
17.708 42.557
18.493 43.773
0.975
5.024
7.378
9.348
11.143
12.833
14.449
16.013
17.535
19.023
20.483
21.920
23.337
24.736
26.119
27.488
28.845
30.191
31.526
32.852
34.170
35.479
36.781
38.076
39.364
40.646
41.923
43.195
44.461
45.722
46.979
0.99
6.635
9.210
11.345
13.277
15.086
16.812
18.475
20.090
21.666
23.209
24.725
26.217
27.688
29.141
30.578
32.000
33.409
34.805
36.191
37.566
38.932
40.289
41.638
42.980
44.314
45.642
46.963
48.278
49.588
50.892
Dr Kim-Anh Lê Cao (The University of Queensland Diamantina Institute, TRI)
Statistics short course series for frightened bioresearchers, Year 2015.
57