Data Formats: JSONL, OpenAI, Alpaca, ShareGPT

Why JSONL won

Almost every fine-tuning pipeline uses JSONL (JSON Lines): one JSON object per line. It won because (1) you can stream it, (2) you can validate one line without parsing the rest, (3) bad lines do not corrupt the file, and (4) every modern dataset tool understands it natively.

The OpenAI chat format (the de facto standard)

This is what OpenAI managed fine-tuning requires, what TRL's SFTTrainer auto-detects, and what most modern open-source tools default to. Each line is one example with a messages array.

Alpaca format

Older but still appears in many open datasets. Each line: {instruction, input, output}. Easy to convert to chat format on intake.

ShareGPT format

For multi-turn conversations. Each line has a conversations array with {from, value} entries. Common in Axolotl-era datasets.

Mixing formats is a trap

Pick one format on intake and convert everything else to it. Mixing formats inside one training run is the #1 cause of "the model outputs gibberish" in week-one fine-tuning projects.

Code

OpenAI chat format (one example per JSONL line)·json

{"messages": [
  {"role": "system", "content": "You are a medical assistant. Be concise."},
  {"role": "user", "content": "What are common symptoms of type 2 diabetes?"},
  {"role": "assistant", "content": "Increased thirst, frequent urination, increased hunger, fatigue, blurred vision, slow-healing sores, frequent infections."}
]}

Alpaca format·json

{"instruction": "Summarize the key findings.", "input": "Patient presents with elevated BP (160/95)...", "output": "Stage 2 hypertension; recommend ambulatory BP monitoring."}

ShareGPT format·json

{"conversations": [
  {"from": "system", "value": "You are a coding assistant."},
  {"from": "human", "value": "How do I read a CSV in Python?"},
  {"from": "gpt", "value": "Use pandas:\n\nimport pandas as pd\ndf = pd.read_csv('data.csv')"}
]}

Data Formats: JSONL, OpenAI, Alpaca, ShareGPT

Why JSONL won

The OpenAI chat format (the de facto standard)

Alpaca format

ShareGPT format

Mixing formats is a trap

Code

External links

Exercise

Progress

Comments 0