Emergent capabilities are abilities that appear sharply at a certain scale — present in large models, absent in smaller ones, with the transition not gradual but rapid.
Classic examples
- In-context learning. GPT-3 (175B) could pick up tasks from few examples in the prompt; GPT-2 (1.5B) couldn't reliably.
- Chain-of-thought reasoning. Above a certain scale, prompting a model to "think step by step" produces dramatically better math/reasoning answers. Below that scale, CoT prompting changes little.
- Code generation from descriptions. Tiny models can complete code but rarely synthesize functional programs from English specs; large models do this routinely.
- Multilingual transfer. Training mainly on English produces models that nevertheless work well across languages once parameters cross some threshold.
Test-time compute scaling
A more recent development: models like OpenAI o1, DeepSeek-R1, and Gemini 2.5 Pro thinking-mode can be allocated more inference compute per query — visible chain-of-thought, multi-attempt reasoning, internal verification. This lets a fixed model improve on hard tasks by spending more compute at inference time, breaking the assumption that "model quality = parameter count alone."
Recent research questions whether some "emergent" capabilities are truly sudden or artifacts of evaluation — discontinuities in metrics, not in the underlying ability. The cleaner conclusion: model capability is a function of scale × data × post-training × test-time compute, with all four interacting.