Outliers can skew your analysis or reveal important insights. Here are 4 fast methods to detect them:

1. IQR Method (Most Robust)

Uses the Interquartile Range - works for any distribution:

# Python
Q1 = df['price'].quantile(0.25)
Q3 = df['price'].quantile(0.75)
IQR = Q3 - Q1

lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR

# Find outliers
outliers = df[(df['price'] < lower_bound) | (df['price'] > upper_bound)]
print(f"Found {len(outliers)} outliers")

Why 1.5? Standard rule that catches extreme values without being too sensitive.

Pros: Not affected by extreme outliers, works for skewed data

2. Z-Score Method (For Normal Distributions)

Measures how many standard deviations away from mean:

# Python
from scipy import stats

z_scores = np.abs(stats.zscore(df['price']))
outliers = df[z_scores > 3]

# Interpretation:
# Z > 3  → Very extreme outlier (99.7% of data within 3σ)
# Z > 2  → Moderate outlier
# Z < 2  → Normal range

Limitation: Assumes normal distribution. Use IQR method for skewed data.

3. Percentile Method (Simple & Quick)

# Flag top and bottom 1% as outliers
lower = df['price'].quantile(0.01)
upper = df['price'].quantile(0.99)

outliers = df[(df['price'] < lower) | (df['price'] > upper)]

Flexible: Adjust percentiles based on your needs (0.5%, 2.5%, etc.)

4. Visual Methods (Always Do This First!)

Box Plot - See Outliers Immediately

# Python
import matplotlib.pyplot as plt

df.boxplot(column='price')
plt.show()

# Points beyond the whiskers = outliers

Scatter Plot - Multivariate Outliers

plt.scatter(df['age'], df['salary'])
# Look for points far from the cluster

Handling Outliers: 4 Options

Option 1: Keep Them

When: Outliers are valid and important

Example: CEO salary in salary analysis (real data point)

Option 2: Remove Them

# Remove outliers
df_clean = df[z_scores <= 3]

When: Data errors or measurement mistakes

Warning: Document why you removed them!

Option 3: Cap Them (Winsorization)

# Cap at 1st and 99th percentiles
lower = df['price'].quantile(0.01)
upper = df['price'].quantile(0.99)

df['price_capped'] = df['price'].clip(lower, upper)

When: Want to reduce impact but keep all data points

Option 4: Transform Them

# Log transformation reduces impact of high outliers
df['log_price'] = np.log(df['price'] + 1)

# Or use robust statistics
median_price = df['price'].median()  # Instead of mean

Quick Decision Guide

Situation	Method
Normal distribution	Z-score (threshold: 3)
Skewed distribution	IQR method
Don't know distribution	IQR method (safest)
Need quick check	Box plot visualization
Multiple variables	Scatter plots first

Common Mistakes to Avoid

• ❌ Removing outliers without investigating why they exist
• ❌ Using Z-score on heavily skewed data
• ❌ Not documenting which outliers you removed
• ❌ Removing outliers before visualization (you might miss insights!)