The beginning is weirdly powerful
Research on attention sinks (Xiao et al. 2023, Efficient Streaming Language Models with Attention Sinks) shows that the first few tokens of a sequence can receive disproportionate attention even when they carry no semantic relevance. Softmax attention forces probabilities to sum to 1; when no other token is particularly relevant, the model dumps that mass on the first few tokens, which become information-free anchors.
Why this matters in practice
The system prompt's position matters as much as its content. The first tokens carry outsized weight; use them deliberately. A clean system prompt and stable project rules benefit from the prefix. A noisy timestamp, random banner, or stale rule at the top can also become sticky — the model keeps half-attending to the wrong thing.
Protect the prefix
The prefix should contain durable rules and stable structure. Put variable, per-turn noise later. This improves both attention behavior and prompt caching — and the two effects reinforce each other.