Linear Regression Assumptions

Linear regression relies on four assumptions: linearity, independence of errors, homoscedasticity (constant error variance), and normally distributed residuals. All four are checked the same way — by plotting the residuals, not the raw data.

What Is the Linearity Assumption?

The relationship between the predictors and the outcome must be linear; check it with a residuals-vs-fitted plot, which should show a flat, patternless cloud around zero.

import statsmodels.api as sm
import matplotlib.pyplot as plt

model = sm.OLS(y, sm.add_constant(X)).fit()
resid, fitted = model.resid, model.fittedvalues

plt.scatter(fitted, resid); plt.axhline(0)
# Good: random cloud around 0
# Bad:  U-shape or curve → add a squared term or transform x

A curved band means the model is systematically wrong in parts of the range — the coefficients are biased, not just noisy. If you're new to fitting these models, start with regression analysis basics.

What Does Independence of Errors Mean?

Each observation's error must be unrelated to the others; violations usually come from time series (autocorrelation) or clustered data like repeated measures per user.

# For time-ordered data, plot residuals in order:
plt.plot(resid)          # trends or waves → autocorrelation

# Durbin-Watson: ~2 is good, < 1.5 or > 2.5 is trouble
sm.stats.durbin_watson(resid)

Violated independence doesn't bias the coefficients, but it makes standard errors far too small — you'll see significance that isn't there. Use clustered standard errors or time-series models instead.

What Is Homoscedasticity?

The spread of residuals should be constant across all fitted values; a funnel shape in the residuals-vs-fitted plot (spread growing with the prediction) means heteroscedasticity.

plt.scatter(fitted, resid)
# Funnel / megaphone shape → heteroscedasticity

# Fixes:
# 1. log-transform y (common when errors scale with size)
# 2. robust (HC) standard errors:
model.get_robustcov_results(cov_type="HC3")

Like autocorrelation, heteroscedasticity leaves the fitted line unbiased but wrecks the p-values and confidence intervals.

Do Residuals Have to Be Normal?

The residuals (not the raw variables) should be roughly normal; check with a Q-Q plot, where points should track the diagonal line.

sm.qqplot(resid, line="45", fit=True)
# S-curve      → heavy tails (outliers)
# Bowed curve  → skewed residuals; try log(y)

This matters mainly for inference on small samples. With a few hundred rows, the central limit theorem covers the coefficient tests, so mild non-normality is usually harmless.

Common Pitfalls

• Testing normality of y instead of the residuals: the assumption is about the errors after fitting, not the raw outcome.
• Relying on R² to validate the model: a high R² can coexist with badly violated assumptions; only residual plots show them.
• Ignoring clustered data: 10 observations per user is not n = 10 × users of independent data.
• Fixing heteroscedasticity by deleting points: transform the outcome or use robust errors; don't trim data to make plots pretty.