JSON Lines — One JSON Value Per Line

The format for streams, logs, and ML datasets

JSON Lines (sometimes NDJSON — newline-delimited JSON, the same thing) is a file format where each line is one independent JSON value. No enclosing array, no commas between records, no top-level structure. The file extension is .jsonl or .ndjson.

What it solves that JSON doesn't

Streaming — you can append a record by appending one line. With a regular JSON array, you'd have to rewrite the closing bracket on every append.
Partial reads — process records one at a time without loading the whole file. Critical for log files and big datasets.
Easy concat / split — cat a.jsonl b.jsonl is the union; split -l 1000 huge.jsonl splits into chunks. Both work because each line is self-contained.
grep-friendly — grep error events.jsonl finds matching lines without parsing.

Where you'll meet JSON Lines

Application log files (papertrail, vector.dev, fluentd).
ML training data (HuggingFace datasets, OpenAI fine-tuning, instruction tuning).
Database exports (MongoDB mongoexport, Firestore exports).
Anything tailing live events.

Principle: when your data is a sequence of records (log events, training samples, time-series rows), reach for JSONL, not a top-level array. Append-only is cheap, parsing is record-at-a-time, and the file scales to gigabytes without breaking the parser.

Code

JSON Lines — the file format·text

{"timestamp":"2026-05-04T01:30:11Z","level":"info","event":"server.start"}
{"timestamp":"2026-05-04T01:30:12Z","level":"info","event":"db.connected"}
{"timestamp":"2026-05-04T01:30:15Z","level":"warn","event":"slow_query","duration_ms":1240}
{"timestamp":"2026-05-04T01:30:18Z","level":"error","event":"http.5xx","path":"/api/chat"}

Append a record (no parsing required)·bash

# Plain shell — works for any size file
echo '{"event":"deploy.start","timestamp":"2026-05-04T01:30:11Z"}' >> events.jsonl

# Or pipe many at once
for i in 1 2 3; do
  echo "{\"i\":$i}"
done >> events.jsonl

Read JSON Lines line-by-line in Python·python

import json

# Process one record at a time — never loads whole file
with open('events.jsonl') as f:
    for line in f:
        line = line.strip()
        if not line:
            continue
        record = json.loads(line)
        if record.get('level') == 'error':
            print(record['event'], record.get('path'))

jq with JSON Lines·bash

# Filter errors only
jq -c 'select(.level == "error")' events.jsonl

# Aggregate — count by level (slurp the stream)
jq -s 'group_by(.level) | map({level: .[0].level, count: length})' events.jsonl

# Convert a JSON array to JSON Lines
jq -c '.[]' big-array.json > records.jsonl

JSON Lines — One JSON Value Per Line

The format for streams, logs, and ML datasets

What it solves that JSON doesn't

Where you'll meet JSON Lines

Code

External links

Exercise

Progress

Comments 0