One-Tailed vs Two-Tailed Tests: When Is One-Tailed Legitimate?

A two-tailed test asks "is there a difference in either direction?" A one-tailed test asks "is it better?" and refuses to detect "worse" at all. One-tailed tests are legitimate only when a change in the opposite direction would lead to exactly the same action as no change, and that condition is rarely true in product experiments.

What is the difference between a one-tailed and two-tailed test?

The difference is where you put your α and which alternative hypotheses you are allowed to detect. A two-tailed test splits the 5% rejection region across both ends of the distribution; a one-tailed test stacks all 5% in the direction you named before seeing data.

	Two-tailed	One-tailed (upper)
Null hypothesis	H₀: μ₁ = μ₂	H₀: μ₂ ≤ μ₁
Alternative	H₁: μ₁ ≠ μ₂	H₁: μ₂ > μ₁
α per tail	0.025 each	0.05 in one tail
Critical z (α = 0.05)	±1.960	+1.645
Critical z (α = 0.01)	±2.576	+2.326
Detects harm?	Yes	No, never
Sample needed for 80% power*	31,231 per arm	24,600 per arm

*Detecting a 5.0% → 5.5% conversion lift at α = 0.05.

How much smaller is a one-tailed p-value?

Exactly half, provided the observed effect points the way you predicted. If it points the other way, a one-sided p-value is above 0.5 and the test simply cannot reject.

Two-tailed p = 2 × P(Z > |z|)
One-tailed p =     P(Z > z)      (upper-tail test)

Observed z	Two-tailed p	One-tailed p	Significant at 0.05?
1.65	0.0989	0.0495	one-tailed only
1.70	0.0891	0.0446	one-tailed only
1.75	0.0801	0.0401	one-tailed only
1.80	0.0719	0.0359	one-tailed only
1.96	0.0500	0.0250	both
2.10	0.0357	0.0179	both

The whole band between z = 1.645 and z = 1.960 is where the choice of tails decides the outcome. That is precisely why switching to one-tailed after seeing a p-value of 0.08 is data dredging, not a methodological refinement.

When is a one-tailed test actually legitimate?

Only when a result in the opposite direction would lead to the identical decision as a null result, and you commit to the direction before collecting data. Both conditions, not just the first.

• Non-inferiority and regression checks. You are migrating infrastructure and only need to confirm the new stack is not slower. If it turns out faster, you still just ship the migration, so the upper tail is genuinely irrelevant.
• One-directional physical or logical constraints. Testing whether a cache layer reduces latency, where a mechanism producing higher latency is not plausible and would be treated as a bug, not a finding.
• Pre-registered directional hypotheses in confirmatory research, where the direction is stated in the protocol before data collection, and the analysis plan is fixed.
• Safety monitoring in one direction, where you are looking only for harm and any benefit is irrelevant to the stopping rule.

Note what all four have in common: the opposite result changes nothing about what you do next. That is the actual test for whether a one-tailed test is appropriate.

Why is one-tailed usually wrong for A/B tests?

Because in a product experiment, "the variant is significantly worse" is extremely valuable information, and a one-tailed test is structurally incapable of reporting it. You would ship a change that tanked conversion by 8% and see only "not significant."

Three more reasons it goes wrong in practice:

• Nobody is really confident about direction. Teams routinely predict a lift and get a drop. If you were certain enough to bet the analysis on it, you probably did not need the test.
• It invites post hoc tail selection. Choosing the tail after seeing which way the data went makes the real α 0.10, not 0.05, because both tails were live in your decision process.
• It breaks comparability. If some tests in your program are one-tailed and some are two-tailed, "significant at 0.05" no longer means the same thing across your experiment log, and meta-analysis of your own history becomes unreliable.

The honest way to get the power a one-tailed test promises is to increase sample size or raise your minimum detectable effect. See statistical power and our A/B test sample size guide.

How does the tail choice affect confidence intervals?

A one-tailed test corresponds to a one-sided confidence bound, not a two-sided interval, and mixing them is a common reporting error. A one-sided 95% bound and a two-sided 95% interval do not have matching endpoints.

Two-sided 95% CI:      estimate ± 1.960 × SE      → [lower, upper]
One-sided 95% bound:   estimate - 1.645 × SE      → [lower, ∞)

Same data, effect = +0.50 pp, SE = 0.28 pp:
  Two-sided 95% CI:    [-0.05 pp, +1.05 pp]   crosses zero
  One-sided 95% bound: effect > +0.04 pp      excludes zero

Publishing the two-sided interval next to a one-tailed p-value produces the contradiction where the interval contains zero but the test is "significant." Pick one framework and report it consistently. More on reading intervals correctly in confidence intervals explained.

Common mistakes with tails

• Switching to one-tailed after seeing p = 0.08. This is the single most common abuse. The direction must be fixed before the data exists.
• Using a one-tailed test to save traffic on a test where harm matters. The savings are real (roughly 21% fewer users here) but you paid for them with your ability to detect a loss.
• Halving a two-tailed p-value when the effect went the wrong way. If the observed direction is opposite your hypothesis, the one-tailed p is 1 - (two-tailed p / 2), which is greater than 0.5, not the small number you were hoping for.
• Assuming your A/B tool's default. Most experimentation platforms default to two-tailed, but some report one-tailed "chance to beat baseline" figures. Check the documentation before comparing results across tools.
• Using a one-tailed test and then reporting the effect went the other way anyway. If you are going to describe the negative direction, you needed a test that could evaluate it.