Adding / Updating Columns
Pandas assigning: assign(), with_columns mental model, conditional assignment, .where, .mask, and the SettingWithCopyWarning patterns.
Pandas — assigning columns
EXAMPLE
import pandas as pd
import numpy as np
df = pd.DataFrame({
'name': ['Alex', 'Sam', 'Lee'],
'age': [30, 25, 40],
'spent': [1200, 800, 450],
})
# ===== Direct assignment =====
df['decade'] = (df.age // 10) * 10
df['active'] = True
df['total'] = df.spent + 100
# ===== assign() returns a new DataFrame (chainable, safer) =====
df2 = df.assign(
tier=lambda d: np.where(d.spent > 1000, 'gold', 'silver'),
age_norm=lambda d: (d.age - d.age.mean()) / d.age.std(),
)
# Chain pattern:
result = (
pd.read_csv('orders.csv')
.query('total > 100')
.assign(month=lambda d: pd.to_datetime(d.date).dt.to_period('M'))
.groupby('month').total.sum()
)
# ===== Conditional assignment =====
df.loc[df.age >= 18, 'adult'] = True
df.loc[df.age < 18, 'adult'] = False
# Or in one line:
df['adult'] = df.age >= 18
# Multi-condition:
conditions = [
df.spent < 100,
df.spent < 500,
df.spent >= 500,
]
choices = ['low', 'mid', 'high']
df['tier'] = np.select(conditions, choices, default='unknown')
# ===== where / mask =====
# where: KEEP where condition is true; replace others
df['safe_age'] = df.age.where(df.age > 0, 0)
# mask: REPLACE where condition is true
df['hidden_age'] = df.age.mask(df.age < 18, np.nan)
# ===== Assign from another DataFrame (alignment by index) =====
other = pd.DataFrame({'spent': [10000]}, index=[0])
df.update(other) # in-place, by index + column
# OR merge / join for non-trivial cases.
# ===== Add multiple cols efficiently =====
new_cols = {
'a': df.spent * 0.1,
'b': df.spent * 0.2,
}
df = df.assign(**new_cols)
# ===== Avoid chained assignment =====
# WRONG:
# df[df.age > 18]['adult'] = True # SettingWithCopyWarning; might not work
# RIGHT:
df.loc[df.age > 18, 'adult'] = True
# ===== Copying on write (Pandas 2+ option) =====
# pd.set_option('mode.copy_on_write', True)
# Makes views immutable; explicit copies needed. Removes most SettingWithCopyWarning cases.
# ===== Eval (string expressions) =====
df.eval('per_year = spent / age', inplace=True)
df.eval('flag = age > 30 and spent > 500', inplace=True)
# ===== Patterns to internalise =====
# - .assign() for chainable, immutable-feeling additions
# - .loc for conditional assignment; never chained []
# - np.select for multi-branch conditional columns
# - .where / .mask for one-condition rewrites
# ===== Pitfalls =====
# - Chained assignment df[mask].col = ... (warning + may not stick)
# - Forgetting that assign returns a NEW DataFrame; not in-place
# - dtype changes after assignment (object -> float silently)
# - Modifying a slice intended as a view (use .copy() if you must)
Why it matters
Assigning is mostly about choosing the right mechanism. .assign for chainable additions, .loc for conditional set, np.select for multi-branch, .where / .mask for one-condition rewrites. Avoid chained [][] = which is the source of the dreaded SettingWithCopyWarning.
Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.
Example
Example
import pandas as pd
df = pd.read_csv('users.csv')
df['adult'] = df['age'] >= 18
df['name_upper'] = df['name'].str.upper()
Try it Yourself »
Discussion
Loading…