Z-Score Explained

A z-score tells you how many standard deviations a value sits from the mean: z = (x − mean) / standard deviation. A z of 2 means "two standard deviations above average," regardless of the original units.

What Is the Z-Score Formula?

Subtract the mean from the value, then divide by the standard deviation. The result is unitless, so you can compare values from completely different scales.

z = (x - μ) / σ

# A $92 order when orders average $60 with σ = $16:
z = (92 - 60) / 16   # = 2.0  → two std devs above the mean

Because the units cancel, a z-score of 2.0 for order value and a z-score of 2.0 for session length mean the same thing: equally unusual relative to their own distributions. See standard deviation explained if the denominator is fuzzy.

How Does Standardization Work?

Standardizing a column means converting every value to its z-score, giving the column a mean of 0 and standard deviation of 1. It puts features on a common scale, which many models and distance metrics require.

import numpy as np

x = np.array([60, 52, 71, 58, 92, 63])
z = (x - x.mean()) / x.std()
# array([-0.4634, -1.0812,  0.3861, -0.6178,  2.0079, -0.2317])

z.mean(), z.std()
# (~0.0, 1.0)

How Do You Flag Outliers with Z-Scores?

A common rule flags any value with |z| > 3 as an outlier, since under a normal distribution only about 0.3% of values fall that far out.

outliers = x[np.abs(z) > 3]

# On skewed data, use the median-based "robust z-score" instead:
med = np.median(x)
mad = np.median(np.abs(x - med))
robust_z = 0.6745 * (x - med) / mad

The robust version matters because ordinary z-scores use the mean and standard deviation, which the outliers themselves inflate — one huge value can hide itself by stretching σ.

Z-Score vs t-Statistic: What's the Difference?

Use a z-score when the population standard deviation is known or the sample is large; use a t-statistic when you estimate σ from a small sample. The t-distribution has fatter tails to account for that extra uncertainty.

n ≥ ~30 or σ known   → z (normal distribution)
n small, σ estimated → t (fatter tails, df = n - 1)
# As n grows, t converges to z.

Common Pitfalls

• Assuming |z| > 3 always means "bad data": in skewed or heavy-tailed data, legitimate values routinely exceed 3. The threshold assumes rough normality.
• Standardizing with test-set statistics: in ML pipelines, compute mean and σ on training data only, then apply to test data.
• Mixing population and sample σ: numpy's std() defaults to population (ddof=0), pandas' to sample (ddof=1). Pick one deliberately.
• Z-scoring tiny samples: with n < 10, the estimated σ is unstable and z-scores are barely meaningful.