C.W.K.
Stream
Lesson 03 of 06 · published

JSON — load, dump, and the Real Gotchas

~20 min · json, serialization, encoding

Level 0Curious
0 XP0/93 lessons0/23 achievements
0/100 XP to next level100 XP to go0% complete

The four functions that cover 99% of JSON

json.dumps(obj) serializes to a string. json.loads(s) parses a string. json.dump(obj, file) writes to a file. json.load(file) reads from a file. The two pairs differ only in whether they touch a string or a file. Every JSON-handling code you write is one of these four.

What's serializable — and what isn't

JSON natively handles: dict, list, str, int, float, bool, None. Tuples become lists (silently — losing tuple-ness on round-trip). Sets, datetime, Path, dataclass, custom classes — none of those work without help. json.dumps({1, 2, 3}) raises TypeError. The fix is to convert before serializing or use a custom default.

The default keyword — handling unsupported types

json.dumps(obj, default=fn) calls fn on any value that json can't handle natively. Return something it CAN handle (a dict, list, or string), and it gets used. The standard pattern: default=str falls back to string representation for things like dates and Paths, with the cost that round-tripping isn't symmetric.

The four extra arguments worth knowing

indent=2 pretty-prints. sort_keys=True orders keys alphabetically — useful for diffable output. ensure_ascii=False writes non-ASCII characters as themselves rather than escaping (better for human reading; worse for old environments). separators=(',', ':') for compact output (no extra spaces).

War Story: JSON's number type is double-precision float. Round-tripping 1.1 survives, but 0.1 + 0.2 serialized and re-parsed is still 0.30000000000000004. Don't expect JSON to do exact decimal math. For currency, use Decimal and serialize as strings.

Code

The four functions·python
import json

obj = {"name": "pippa", "age": 4, "vessels": ["claude", "codex", "gemini", "ollama"]}

# To string
s = json.dumps(obj)
print(s)

# Parse string
back = json.loads(s)
print(back)

# To/from file
with open("/tmp/pippa.json", "w", encoding="utf-8") as f:
    json.dump(obj, f)

with open("/tmp/pippa.json", encoding="utf-8") as f:
    loaded = json.load(f)
print(loaded == obj)         # True
What's NOT serializable — and how to handle it·python
import json
from datetime import datetime, date
from pathlib import Path
from decimal import Decimal

obj = {
    "set": {1, 2, 3},                 # set
    "date": date(2026, 5, 2),         # date
    "path": Path("/tmp/x"),           # Path
    "money": Decimal("1.99"),         # Decimal
}

# This raises
try:
    json.dumps(obj)
except TypeError as e:
    print("raised:", e)

# Solution 1 — convert manually before dumping
cleaned = {
    "set": list(obj["set"]),
    "date": obj["date"].isoformat(),
    "path": str(obj["path"]),
    "money": str(obj["money"]),
}
print(json.dumps(cleaned))

# Solution 2 — default=str (round-trip not symmetric)
print(json.dumps(obj, default=str))
Custom default — proper handling per type·python
import json
from datetime import datetime, date
from pathlib import Path

def serialize(obj):
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    if isinstance(obj, Path):
        return str(obj)
    if isinstance(obj, set):
        return sorted(obj)               # set as a sorted list
    raise TypeError(f"not serializable: {type(obj).__name__}")

obj = {
    "created": datetime.now(),
    "path": Path("/tmp/x"),
    "tags": {"urgent", "review"},
}
print(json.dumps(obj, default=serialize, indent=2, ensure_ascii=False))
Pretty-printing and stable output·python
import json

obj = {"b": 2, "a": 1, "items": [3, 1, 2], "name": "피파"}

# Default — compact
print(json.dumps(obj))

# Pretty
print(json.dumps(obj, indent=2))

# Stable — sort keys + non-ascii passthrough
print(json.dumps(obj, indent=2, sort_keys=True, ensure_ascii=False))

# Compact (no extra whitespace)
print(json.dumps(obj, separators=(",", ":")))
Streaming JSON — for large files, use a streaming parser·python
# json doesn't stream — it reads the whole file into memory
# For 1GB JSON files, you need a streaming parser like ijson

# When the file IS a list of objects (newline-delimited JSON / JSONL),
# stream-process line by line:
import json

lines = ['{"id": 1}', '{"id": 2}', '{"id": 3}']

for line in lines:
    obj = json.loads(line)            # one record at a time
    print(obj)

# JSONL is what cwkPippa uses for session logs — one JSON object per line.
# Easy to append-only, easy to stream-read.

External links

Exercise

Build a function save_record(record, path) that takes a dict (which may contain a datetime, a Path, and a set) and writes it to a JSON file. Use a custom default function that handles those three types correctly. Then write load_record(path) that reads back a record. Verify round-tripping the simple types works perfectly. Note that the set and datetime will need to be reconstructed manually if you want full round-trip — accept that as a limitation and document it.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.