ANOVA (Analysis of Variance) tests if means of three or more groups differ significantly. It's more appropriate than multiple t-tests, which inflate Type I error rates.

One-Way ANOVA

Basic Example: Comparing Three Marketing Campaigns

from scipy import stats
import numpy as np

# Campaign results (conversion rates)
campaign_a = [23, 25, 24, 26, 22, 24]
campaign_b = [28, 30, 29, 31, 28, 30]
campaign_c = [25, 26, 24, 27, 25, 26]

# One-way ANOVA
f_stat, p_value = stats.f_oneway(campaign_a, campaign_b, campaign_c)

print(f"F-statistic: {f_stat:.3f}")
print(f"P-value: {p_value:.4f}")

if p_value < 0.05:
    print("✓ At least one campaign differs significantly")
else:
    print("✗ No significant difference between campaigns")

Using Python with DataFrame

import pandas as pd
from scipy.stats import f_oneway

# Create dataset
data = pd.DataFrame({
    'score': [23, 25, 24, 26, 22, 24, 28, 30, 29, 31, 28, 30, 25, 26, 24, 27, 25, 26],
    'group': ['A']*6 + ['B']*6 + ['C']*6
})

# Group by category
groups = [group['score'].values for name, group in data.groupby('group')]

# Run ANOVA
f_stat, p_value = f_oneway(*groups)
print(f"F-stat: {f_stat:.2f}, p-value: {p_value:.4f}")

Understanding F-Statistic

# F = Variance between groups / Variance within groups

# High F-statistic = groups differ more than expected by chance
# Low F-statistic = groups similar, differences likely random

# F-statistic formula:
# F = MSB / MSW
# MSB = Mean Square Between groups
# MSW = Mean Square Within groups

Post-Hoc Tests (Which Groups Differ?)

Tukey's HSD Test

from statsmodels.stats.multicomp import pairwise_tukeyhsd

# After significant ANOVA, find which groups differ
tukey = pairwise_tukeyhsd(data['score'], data['group'], alpha=0.05)
print(tukey)

# Output shows pairwise comparisons:
# group1 group2  meandiff  p-adj   lower   upper  reject
#      A      B     -5.0   0.001   -7.2   -2.8    True
#      A      C     -1.5   0.523   -3.7    0.7   False
#      B      C      3.5   0.045    1.3    5.7    True

Two-Way ANOVA

Testing Two Factors Simultaneously

from statsmodels.formula.api import ols
from statsmodels.stats.anova import anova_lm

# Example: Effect of Diet AND Exercise on weight loss
data = pd.DataFrame({
    'weight_loss': [5, 6, 7, 3, 4, 5, 8, 9, 10, 4, 5, 6],
    'diet': ['Low-carb', 'Low-carb', 'Low-carb', 'Keto', 'Keto', 'Keto',
             'Low-carb', 'Low-carb', 'Low-carb', 'Keto', 'Keto', 'Keto'],
    'exercise': ['None', 'None', 'None', 'None', 'None', 'None',
                 'Daily', 'Daily', 'Daily', 'Daily', 'Daily', 'Daily']
})

# Fit model
model = ols('weight_loss ~ C(diet) + C(exercise) + C(diet):C(exercise)', data).fit()

# ANOVA table
anova_table = anova_lm(model, typ=2)
print(anova_table)

# Shows:
# - Main effect of diet
# - Main effect of exercise
# - Interaction effect (diet x exercise)

Checking ANOVA Assumptions

1. Normality

from scipy.stats import shapiro

# Test each group for normality
for name, group in data.groupby('group'):
    stat, p = shapiro(group['score'])
    print(f"{name}: p={p:.3f} {'✓ Normal' if p > 0.05 else '✗ Not normal'}")

2. Homogeneity of Variance (Levene's Test)

from scipy.stats import levene

groups = [group['score'].values for name, group in data.groupby('group')]
stat, p = levene(*groups)

print(f"Levene's test p-value: {p:.3f}")
if p > 0.05:
    print("✓ Variances are equal")
else:
    print("✗ Variances differ - consider Welch's ANOVA")

3. Independence

# Ensure:
# - Random sampling
# - Each observation independent
# - No repeated measures (use repeated measures ANOVA if yes)

Non-Parametric Alternative: Kruskal-Wallis

from scipy.stats import kruskal

# Use when:
# - Data not normally distributed
# - Unequal variances
# - Ordinal data

h_stat, p_value = kruskal(campaign_a, campaign_b, campaign_c)
print(f"H-statistic: {h_stat:.3f}, p-value: {p_value:.4f}")

Effect Size: Eta-Squared

def eta_squared(groups):
    # Calculate eta-squared (proportion of variance explained)
    all_data = np.concatenate(groups)
    grand_mean = np.mean(all_data)

    ss_between = sum(len(g) * (np.mean(g) - grand_mean)**2 for g in groups)
    ss_total = sum((x - grand_mean)**2 for x in all_data)

    return ss_between / ss_total

groups = [campaign_a, campaign_b, campaign_c]
eta_sq = eta_squared(groups)
print(f"Eta-squared: {eta_sq:.3f}")

# Interpretation:
# 0.01 = small effect
# 0.06 = medium effect
# 0.14 = large effect

Practical Example: Product Testing

# Test 4 different packaging designs on sales
design_a = [45, 48, 46, 50, 47]
design_b = [52, 55, 53, 56, 54]
design_c = [48, 50, 49, 51, 48]
design_d = [44, 46, 45, 47, 46]

f_stat, p_value = stats.f_oneway(design_a, design_b, design_c, design_d)

print(f"F-statistic: {f_stat:.2f}")
print(f"P-value: {p_value:.4f}")

if p_value < 0.05:
    print("\n✓ Significant difference in packaging effectiveness")
    print("Proceed with post-hoc tests to identify best design")
else:
    print("\n✗ No significant difference - choose based on cost")

When to Use ANOVA

• Comparing 3+ groups (use t-test for 2 groups)
• Continuous outcome variable
• Independent observations
• Approximately normal distributions
• Similar variances across groups