Pandas concat vs merge

Use pd.concat() to stack DataFrames that share the same columns (or same rows), and pd.merge() to join DataFrames on a key column, SQL-style. Concat glues along an axis; merge matches rows by value.

When Should You Use concat?

Reach for concat when you have multiple chunks of the same shape of data — monthly files, results from a loop, train/test splits — and just need them stacked into one DataFrame. No key matching happens.

import pandas as pd

jan = pd.DataFrame({"sale_id": [1, 2], "amount": [99, 50]})
feb = pd.DataFrame({"sale_id": [3, 4], "amount": [75, 20]})

pd.concat([jan, feb], ignore_index=True)
#    sale_id  amount
# 0        1      99
# 1        2      50
# 2        3      75
# 3        4      20

What Does axis=0 vs axis=1 Do in concat?

axis=0 (default) stacks vertically, adding rows; axis=1 stacks horizontally, adding columns and aligning rows by index. Horizontal concat is index alignment, not key matching — mismatched indexes produce NaN.

prices = pd.DataFrame({"price": [10, 20]}, index=["a", "b"])
stock  = pd.DataFrame({"qty": [5, 8]},   index=["a", "c"])

pd.concat([prices, stock], axis=1)
#    price  qty
# a   10.0  5.0
# b   20.0  NaN
# c    NaN  8.0

What Do ignore_index and keys Do?

ignore_index=True throws away the original indexes and renumbers 0..n-1, which you almost always want when stacking rows. keys= instead labels each source chunk in a MultiIndex so you can tell where rows came from.

combined = pd.concat([jan, feb], keys=["jan", "feb"])
combined.loc["feb"]        # just February's rows

# Without ignore_index you get duplicate index values:
pd.concat([jan, feb]).loc[0]   # returns TWO rows — usually a bug

When Should You Use merge Instead?

Use merge when the two DataFrames hold different columns about the same entities and you need to match rows on a shared key, like joining orders to customers. Which rows survive depends on the how parameter — see the merge how parameter cheat sheet for inner/outer/left/right/cross side by side.

orders    = pd.DataFrame({"customer_id": [10, 20], "amount": [99, 50]})
customers = pd.DataFrame({"customer_id": [10, 20], "name": ["Alice", "Bob"]})

pd.merge(orders, customers, on="customer_id")
#    customer_id  amount   name
# 0           10      99  Alice
# 1           20      50    Bob

What Happened to DataFrame.append?

df.append() was removed in pandas 2.0 — old code using it now raises AttributeError. Replace it with pd.concat, and never concat inside a loop; collect frames in a list and concat once.

# Old (removed in 2.0):  df = df.append(new_rows)
# New:
frames = [process(f) for f in files]
df = pd.concat(frames, ignore_index=True)

Common Pitfalls

• Duplicate indexes after concat: forgetting ignore_index=True leaves repeated labels that break .loc lookups.
• Mismatched columns: concat unions column sets by default and fills gaps with NaN. Pass join="inner" to keep only shared columns.
• Concat in a loop is O(n²): each call copies everything. Build a list, concat once.
• Using concat axis=1 as a join: it aligns on index only. If you need key matching, that's merge.