Listwise Deletion: When Dropping Rows Is (and Isn't) OK

Listwise deletion (also called complete-case analysis) means removing every row that has a missing value in any of the variables used in your analysis. It's the default behavior of most statistical software — and one of the most common silent sources of bias in real-world analysis.

What is listwise deletion, exactly?

Listwise deletion keeps only "complete cases": rows where every variable in the analysis has a value. If a survey respondent skipped one question out of twenty, their entire record is excluded. In pandas this is df.dropna(); in R it's na.omit(); most regression functions do it automatically without telling you.

# pandas: listwise deletion on the columns you actually use
analysis_cols = ['age', 'income', 'tenure', 'churned']
complete = df[analysis_cols].dropna()

print(f"Kept {len(complete)} of {len(df)} rows "
      f"({len(complete)/len(df):.0%})")

Always print that percentage. The most common listwise-deletion mistake isn't statistical — it's not noticing how many rows silently disappeared.

Listwise vs pairwise deletion: what's the difference?

Listwise deletion uses only rows complete on all analysis variables. Pairwise deletion uses every row available for each specific calculation — so a correlation between age and income uses all rows where both exist, even if tenure is missing. Pairwise keeps more data but each statistic is computed on a different subsample.

	Listwise	Pairwise
Sample used	One consistent set of complete rows	Different rows per statistic
Sample size	Smaller	Larger per statistic
Consistency	All results comparable	Correlation matrices can be internally inconsistent (even non-invertible)
Typical use	Regression, ML training	Exploratory correlation tables

Pairwise deletion's inconsistency is a real problem: a correlation matrix built pairwise can imply impossible relationships, which breaks methods like factor analysis that need a valid matrix.

When is listwise deletion actually safe?

Listwise deletion gives unbiased estimates only when data is Missing Completely at Random (MCAR) — meaning the probability a value is missing has nothing to do with any variable, observed or not. Under MCAR, complete cases are a true random subsample, so you lose power but not accuracy. Under MAR or MNAR mechanisms, the remaining rows are a distorted sample and your estimates shift.

Example of the danger: if high earners are less likely to report income, dropping incomplete rows removes high earners disproportionately, and your average income estimate is biased low. No amount of extra data collection on the remaining rows fixes that. See MAR vs MCAR vs MNAR for how to reason about which mechanism you have.

One nuance worth knowing: in regression, listwise deletion is also approximately unbiased when missingness depends only on the predictor values (not on the outcome). But that's a special case — don't rely on it without thinking it through.

How much sample size does it cost?

More than intuition suggests, because missingness compounds across columns. If each of 10 columns is independently missing 5% of the time, the chance a row is fully complete is 0.95¹⁰ ≈ 0.60. You lose about 40% of your rows even though every individual column looks 95% complete.

# Expected complete-case fraction with independent missingness
p_complete = 0.95 ** 10   # 10 columns, 5% missing each
print(p_complete)          # 0.599 → ~40% of rows dropped

With 20 columns at 5% missingness, only ~36% of rows survive. That lost sample means wider confidence intervals, less power to detect effects, and — if the missingness isn't MCAR — bias on top of it.

What should I do instead of dropping rows?

When listwise deletion would cost too much data or bias your sample, the main alternatives are: imputation (filling values with estimates — see mean, median, KNN, and MICE compared), adding a "missing" indicator column alongside a simple fill, using models that handle missing values natively (XGBoost, LightGBM), or dropping the offending column instead of rows when one column carries most of the missingness.

• One column is 60% missing, others are clean: drop that column, keep the rows.
• Missingness under ~5% and plausibly MCAR: listwise deletion is fine — say so in your write-up.
• Missingness related to other variables (MAR): multiple imputation (MICE) or model-based approaches.
• Missingness in the target variable: drop those rows — never impute your target.

For the full decision process, see deletion vs imputation: how to choose and the broader guide to handling missing data.