The GPT Lineage: GPT-1 to GPT-5

OpenAI's GPT series defined the scaling paradigm. Reading the lineage in order is the quickest history of modern AI.

Model	Year	Params	Context	Key shift
GPT-1	2018	117M	512	Pretraining + fine-tuning recipe
GPT-2	2019	1.5B	1,024	Coherent multi-paragraph generation
GPT-3	2020	175B	2,048	In-context learning at scale
GPT-3.5 / ChatGPT	2022	~175B (similar)	4k → 16k	RLHF + chat format made it accessible
GPT-4	2023	~1.8T (MoE est.)	8k → 128k	Multimodal vision input; large quality jump
GPT-4o	2024	n/a (native multimodal)	128k	Audio + image native; cheaper inference
GPT-4.1	Apr 2025	Unknown	1M	1M context, coding focus
GPT-5	Aug 2025	~1.7T total	400k	Real-time router across fast / thinking / Pro modes
GPT-OSS-120b	2025	117B (5.1B active)	—	Open weights MoE, fits on a single H100

GPT-5 introduces a real-time router that dispatches each query to a different reasoning mode (fast, thinking, Pro). Total context is 400K (272K input + 128K output). The open-weights variant GPT-OSS-120b is Apache 2.0 and fits on a single H100 GPU.

Code

Reading a model card systematically·python

# When you encounter a new model, extract these in order:
fields = [
    "release date",
    "parameter count (total / active for MoE)",
    "architecture shape (d_model, n_layers, n_heads)",
    "vocabulary size and tokenizer",
    "context window (input + output)",
    "training data: source, scale, cutoff date",
    "post-training stack (SFT? DPO? GRPO? CAI?)",
    "license (commercial use? attribution?)",
    "modalities (text only? vision? audio?)",
    "stated benchmarks and known weaknesses",
]
# This is the universal template that lets you place any new model
# into the lineage without getting lost in marketing claims.

Code

External links

Exercise

Progress

Comments 0