Four Inputs You Need

1. Baseline conversion rate (current performance)
2. Minimum detectable effect (smallest change you care about)
3. Statistical power (usually 80% = 0.80)
4. Significance level (usually 5% = 0.05)

Quick Example

Current: 10% click-through rate
Want to detect: 2% absolute increase (to 12%)
Power: 80%
Significance: 5%

Sample needed: ~3,800 per group = 7,600 total

Common Mistakes

❌ Running test "until significant" - This inflates false positives

❌ Stopping early when winning - You're cherry-picking lucky noise

❌ Not deciding minimum effect size upfront - Leads to under-powered tests

Rule of Thumb

Smaller effects need bigger samples. If you want to detect a 1% lift instead of 10%, you need 100x more data.