Module 10 · Correlation & regression
Use paired quantitative observations to fit a line, interpret its slope, and inspect what the line leaves unexplained.
A regression dataset contains paired measurements: one x and one y for each observational unit. For an illustrative store study, x might be weekly advertising spend in thousands of dollars and y weekly sales in tens of thousands. Keep each week's pair together; sorting the columns separately destroys the relationship.
Begin with a scatterplot. Look for direction, curvature, clusters, and unusual observations. Pearson's correlation r summarizes the strength and direction of a linear relationship. It ranges from −1 to 1 and has no units. A value near zero does not rule out a strong curved relationship.
Correlation is unchanged by shifting either variable or multiplying it by a positive constant. A negative multiplier reverses its sign. These properties make it useful for comparison, but they also mean that r alone cannot tell you the operational size of a change in y.
Each position in the x list pairs with the same position in the y list. The default values are fictional teaching data.
The fitted line minimizes squared vertical residuals. Residual view plots observed minus fitted y against x. The amber marker is a fitted value, with no prediction interval.
A one-unit increase in x changes the fitted y by 1.071 units. Compare the point pattern with the fitted line, then switch to residual view to inspect departures.
8 paired observations. Slope 1.071, intercept 1.679, correlation 0.982, R squared 0.964. At x = 5, fitted y = 7.036. Observed x range 1 to 8.
The fitted line is ŷ = b₀ + b₁x. The slope b₁ is the fitted change in y for a one-unit increase in x. Its units are y-units per x-unit. The intercept b₀ is the fitted value at x = 0, which may lie outside the observed range and may have little practical meaning.
Least squares chooses the intercept and slope to minimize the sum of squared vertical residuals. A residual is eᵢ = yᵢ − ŷᵢ, the observed response minus the fitted response. Positive residuals lie above the line. With an intercept, the residuals sum to zero, apart from numerical rounding.
The slope can be calculated as Σ(xᵢ − x̄)(yᵢ − ȳ) / Σ(xᵢ − x̄)², and the intercept as ȳ − b₁x̄. The fitted line passes through (x̄, ȳ). At least two distinct x values are needed to identify a slope.
Use the illustrative pairs (1, 2), (2, 3), and (3, 5). Then x̄ = 2 and ȳ = 10/3. The cross-product sum is 3 and the squared x-deviation sum is 2, so b₁ = 1.5 and b₀ = 1/3.
At x = 2, the fitted value is 1/3 + 1.5 × 2 = 3.333. The observed y is 3, so its residual is −0.333. The full residual sequence is approximately 0.167, −0.333, 0.167.
Here the residual sum of squares is 1/6 and total squared variation in y is 14/3. Thus R² = 1 − (1/6)/(14/3) = 27/28 ≈ 0.964. The fitted line accounts for about 96.4% of the observed variation around ȳ. This in-sample fit does not mean predictions will be 96.4% accurate on future observations.
Switch the lab to residual view. A useful linear mean model leaves no clear systematic curve in residuals versus x. A U-shape suggests a missing nonlinear pattern. A fan shape suggests changing error spread. Residuals plotted in collection order can reveal time dependence that a scatterplot against x misses.
An unusual observation at an extreme x value can strongly influence the fitted slope. Compare the outlier preset with the original data, then inspect whether the unusual record is a valid observation, a data error, or a different process. Do not remove it merely because its removal improves R².
Predictions inside the observed x range are interpolations. Predictions outside that range are extrapolations and need additional justification: an advertising-sales relationship may flatten when a market saturates. This lab reports a fitted value, not a confidence interval or an individual prediction interval.
A positive slope between advertising and sales could reflect an advertising effect, larger campaigns during already busy weeks, or a third variable such as seasonality. A regression line fitted to observational data does not by itself isolate a causal effect.
For conventional regression inference, the conditional mean should follow the specified model and errors should be independent with constant variance; normal errors support exact small-sample t procedures. Fitting and describing a line requires fewer assumptions than interpreting a p-value or interval. Use study design, diagnostics, and out-of-sample validation to decide which conclusions are justified.
Try each question before revealing the solution.
The fitted delivery time increases by 2.5 minutes per additional kilometer. At 8 km, ŷ = 12 + 2.5 × 8 = 32 minutes. This is a fitted mean relationship, not a guarantee for one delivery.
Residual = observed − fitted = 38 − 32 = 6 minutes. The point is above the fitted line. Squaring this residual contributes 36 minute² to the residual sum of squares.
The high R² does not remove the systematic curvature. The line may capture the main trend while missing an important feature of the conditional mean. Consider a substantively justified nonlinear form and validate it on data not used for fitting.
Further reading