Module 11 · Analysis of variance
Separate variation between groups from variation within groups, and use their ratio to evaluate a common population mean.
Suppose a logistics team compares average handling times for three operating procedures. The response is quantitative, and the procedure is a categorical factor with three levels. A one-way analysis of variance asks whether all population means can reasonably be treated as equal.
The null hypothesis is H₀: μA = μB = μC. The alternative is that at least one population mean differs. It does not say that every pair differs. Separate unadjusted t tests for every pair address a different collection of hypotheses and can inflate the probability of at least one false positive.
Each record should represent an independent unit. Ten measurements from the same shipment do not create ten independent shipments. Repeated measures, paired records, and clustered operations require methods that respect those structures.
Change the three illustrative groups. Watch how separating the means or widening the groups changes F.
Dots are observations; diamonds are group means; the dashed line is the grand mean. Small horizontal offsets keep overlapping dots visible and have no numerical meaning.
At α = 0.05, reject the hypothesis that all three population means are equal, provided the model assumptions are reasonable. The observed η² is 0.750. A significant omnibus result does not identify which pairs differ.
Group sizes 5, 5, 5; means 10.000, 13.000, 16.000. Between-group sum of squares 90.000, within-group sum of squares 30.000. F 18.000, degrees of freedom 2 and 12, p 0.0002.
Let x̄g be a group mean and x̄ the grand mean across all observations. The grand mean weights each group by its sample size. The between-group sum of squares is Σ ng(x̄g − x̄)². The within-group sum of squares is the sum of squared deviations of each observation from its own group mean.
These two quantities add to the total sum of squared deviations around the grand mean. Divide between-group variation by k − 1 and within-group variation by N − k to obtain the two mean squares. Their ratio is the F statistic.
Under a common population mean and the classical assumptions, both mean squares estimate the same error variance. F values substantially above 1 suggest that the group means vary more than within-group noise alone would explain. The reference distribution has k − 1 numerator and N − k denominator degrees of freedom, and the p-value is its upper-tail area.
The default data are fictional handling times. Group A is 8, 9, 10, 11, 12; B is 11, 12, 13, 14, 15; and C is 14, 15, 16, 17, 18. Their means are 10, 13, and 16, and the grand mean is 13.
The between-group sum of squares is 5(10 − 13)² + 5(13 − 13)² + 5(16 − 13)² = 90. Each group contributes 10 to within-group variation, giving 30 in total. With N = 15 and k = 3, the mean squares are 90/2 = 45 and 30/12 = 2.5.
Therefore F = 45/2.5 = 18, with degrees of freedom (2, 12) and p ≈ 0.000244. At α = 0.05, the common-mean hypothesis is rejected under the model. The sample effect-size summary η² = 90/(90 + 30) = 0.75: group membership accounts for 75% of the observed total squared variation in these data.
Classical one-way ANOVA assumes independent errors, normally distributed errors within each population for exact finite-sample inference, and a common population variance. Raw observations across all groups need not resemble one normal distribution; groups with different means can produce a multimodal combined dataset.
Inspect group spreads, outliers, and residual patterns. Strong skewness in very small groups or unequal variances combined with unequal sample sizes can undermine the usual F procedure. This lab flags a sample-variance ratio above 4 as a prompt for investigation, not as a formal assumption test. Welch ANOVA can address unequal variances; it is not implemented here.
Changing the numbers cannot repair selection bias, dependence, or confounding. If the groups are observational, a difference in means does not isolate the effect of the procedure. Random assignment helps identify treatment effects; random sampling supports generalization to a population.
Report group sample sizes and means, their spread, F with both degrees of freedom, the p-value, and a practically interpretable measure of magnitude. η² describes the observed sample; it can overestimate the corresponding population effect and does not establish causation.
A significant omnibus test leaves a follow-up question: which comparisons matter? Use prespecified contrasts or an appropriate multiple-comparison procedure, such as Tukey comparisons when their assumptions apply. Do not treat every pair as different solely because the overall test is significant.
A nonsignificant result is also limited. It means the data did not provide sufficient evidence of unequal means at the chosen threshold. Demonstrating that procedures are practically equivalent requires explicit equivalence margins and an appropriate design.
Try each question before revealing the solution.
The numerator has k − 1 = 4 − 1 = 3 degrees of freedom. The denominator has N − k = 32 − 4 = 28. Report the statistic as F with degrees of freedom (3, 28).
Between-group variation is unchanged. Squaring deviations makes within-group variation four times larger, so the denominator mean square quadruples. F is divided by four. In the worked example, 18 becomes 4.5.
No. It is evidence against all population means being equal, under the model. Identifying A versus B requires an appropriate comparison, and a causal claim requires a suitable study design. Neither follows automatically from the omnibus p-value.
Further reading