Selection, Filtering, and the .loc / .iloc Trap

The single biggest source of confusion in Pandas is how to pick the rows and columns you want. The library exposes three different indexing styles, and beginners blend them in ways that produce wrong answers without errors. Here's the working framework.

The three indexers

df['col'] — column access by label. Returns a Series.
df.loc[row_label, col_label] — label-based selection. Both row and column are addressed by their labels (or boolean masks).
df.iloc[row_pos, col_pos] — position-based selection. Both row and column are addressed by integer position, like a NumPy array.

Use .loc for label-based work, .iloc for position-based work, and never mix them in your head. The legacy df.ix attribute is gone. The chained-indexing style df[df.col > 0]['other'] = ... is a classic source of SettingWithCopyWarning — which Pandas 3.0 has tightened the screws on under Copy-on-Write semantics.

The Copy-on-Write era

Pandas 3.0 made Copy-on-Write (CoW) the default. Practical effect: chained assignments that used to silently mutate the parent DataFrame now either mutate a copy (so your change disappears) or raise a clear error. The fix is the same fix it's always been — use a single .loc with both row and column on the left side of the assignment.

Code

Right and wrong ways to filter and assign·python

import pandas as pd

df = pd.DataFrame({
    'order_id':   ['A001', 'A002', 'A003', 'A004'],
    'amount_usd': [120.50,   87.30, 215.00,    9.99],
    'status':     ['shipped', 'pending', 'shipped', 'cancelled'],
})

# Label-based row selection (boolean mask is also a label index)
shipped = df.loc[df['status'] == 'shipped']                       # rows where status is 'shipped'
shipped_amounts = df.loc[df['status'] == 'shipped', 'amount_usd'] # only the amount column

# Position-based — first 2 rows, first 2 columns
topleft = df.iloc[:2, :2]

# WRONG — chained assignment, fails or no-ops under Pandas 3.0 CoW
# df[df['status'] == 'pending']['amount_usd'] = 0      # ← don't do this

# RIGHT — single .loc with row and column both addressed
df.loc[df['status'] == 'pending', 'amount_usd'] = 0

# Boolean masks combine with & | (parens required)
big_or_pending = df.loc[(df['amount_usd'] > 100) | (df['status'] == 'pending')]

Selection, Filtering, and the .loc / .iloc Trap

The three indexers

The Copy-on-Write era

Code

External links

Exercise

Progress

Comments 0