CSV — DictReader, DictWriter, and Quoting

The two reader/writer patterns

The csv module gives you two flavors of each operation. csv.reader / csv.writer work with rows-as-lists. csv.DictReader / csv.DictWriter work with rows-as-dicts using the header row for keys. The Dict variants are almost always what you want — your code reads better with field names than with column indices.

The newline argument — the one that always trips up beginners

Always open CSV files with newline=''. CSV files have their own newline conventions, and Python's text-mode newline translation interferes. The empty string disables the translation; the csv module handles its own. Forget this and you'll get extra blank rows on Windows or mangled multi-line cells.

Quoting — when commas and newlines hide inside cells

CSV's killer feature: cells can contain commas, newlines, and quotes. Quotes around a cell let you embed those characters. Doubled quotes inside a quoted cell mean a literal quote. The csv module handles all this — but only if you use it. Don't roll your own "split on comma."

The dialect parameter — different CSVs do different things

Excel uses , as separator. European Excel often uses ;. Some tools use \t (TSV). The dialect argument or delimiter/quotechar/quoting kwargs let you handle them. csv.Sniffer can guess the dialect from a sample.

Warning: If your data is more than 100KB and has a few quirks, switching to pandas.read_csv is usually the right call. The stdlib csv is for small files and code-heavy processing; pandas is for bulk data work.

Code

DictReader — rows as dicts·python

import csv
import io

data = '''name,age,role
alice,30,admin
bob,25,member
charlie,35,member'''

# Read from a string for demo; same with a file object
reader = csv.DictReader(io.StringIO(data))
for row in reader:
    print(row)
# {'name': 'alice', 'age': '30', 'role': 'admin'}
# {'name': 'bob', 'age': '25', 'role': 'member'}
# {'name': 'charlie', 'age': '35', 'role': 'member'}

# Note: all values are strings — convert manually as needed
rows = list(csv.DictReader(io.StringIO(data)))
for row in rows:
    row["age"] = int(row["age"])

DictWriter — write rows from dicts·python

import csv
import io

rows = [
    {"name": "alice", "age": 30, "role": "admin"},
    {"name": "bob", "age": 25, "role": "member"},
]

buf = io.StringIO()
writer = csv.DictWriter(buf, fieldnames=["name", "age", "role"])
writer.writeheader()
writer.writerows(rows)

print(buf.getvalue())
# name,age,role
# alice,30,admin
# bob,25,member

Reading from a real file — newline=''·python

import csv

# Always newline='' for CSV files in Python
with open("/tmp/users.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "age"])
    writer.writeheader()
    writer.writerow({"name": "alice", "age": 30})
    writer.writerow({"name": "bob", "age": 25})

with open("/tmp/users.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        print(row)

Quoting — cells with commas and quotes·python

import csv
import io

rows = [
    {"name": "alice", "note": "hello, world"},          # has a comma
    {"name": "bob", "note": 'she said "hi"'},          # has quotes
    {"name": "charlie", "note": "line1\nline2"},        # has a newline
]

buf = io.StringIO()
writer = csv.DictWriter(buf, fieldnames=["name", "note"])
writer.writeheader()
writer.writerows(rows)

print(buf.getvalue())
# name,note
# alice,"hello, world"
# bob,"she said ""hi"""           — doubled quotes for literal quote
# charlie,"line1
# line2"                            — quoted because of newline

# csv handles all this. Don't try to write CSV by hand.

Different dialects — TSV and friends·python

import csv
import io

data = '''name\tage\trole
alice\t30\tadmin
bob\t25\tmember'''

# Tab-separated — change the delimiter
for row in csv.DictReader(io.StringIO(data), delimiter="\t"):
    print(row)

# csv.Sniffer can guess
sniffer = csv.Sniffer()
dialect = sniffer.sniff(data[:200])
print("detected delimiter:", repr(dialect.delimiter))

Exercise

Take a list of three dicts representing books (title, author, year, price). Write them to /tmp/books.csv using csv.DictWriter with newline=''. Make at least one title contain a comma and at least one have an em-dash or other Unicode character. Read the file back with csv.DictReader and confirm every record matches the original (after handling that CSV gives you all values as strings — convert year and price back).

CSV — DictReader, DictWriter, and Quoting

The two reader/writer patterns

The newline argument — the one that always trips up beginners

Quoting — when commas and newlines hide inside cells

The dialect parameter — different CSVs do different things

Code

External links

Exercise

Progress

Comments 0