Tactics Against Position Bias

Beyond 'put it at the front'

Once you accept position bias, you can fight it deliberately. The standard tactics: edge-place authority (front and tail), duplicate critical rules at both ends if they really matter, separate sections with clear delimiters so the model can locate them again, and avoid scattering related facts across distant positions.

Duplication is sometimes correct

Engineers hate duplication. But when a single sentence decides the entire task ('do not push without confirmation'), repeating it once at the end of a long context can recover most of the position-bias loss. The token cost is trivial; the alignment win is real.

Section markers

Use clear, distinctive markers (XML tags, all-caps headers, --- separators) so the model can re-locate the rules section even when it is in the middle of a long input. The model uses these markers as structural anchors during attention.

If you have to put something in the middle, mark its boundaries loudly enough that the model can navigate back to it.

Code

Duplicated authority rule·markdown

# Top of prompt
<rules>
- Do not run git push without explicit human confirmation.
- Use the existing dev server.
</rules>

... (long context) ...

# Bottom of prompt, just before the question
<reminder>
Do not run git push without explicit human confirmation.
</reminder>

USER: ship the latest changes

XML section anchors·xml

<rules>...</rules>
<sources>
  <file path="...">...</file>
</sources>
<state>
  current_branch: feature/x
  pending_action: review
</state>
<task>
  Summarize the diff and propose tests.
</task>

Exercise

Take a long real prompt you use. Add explicit section markers and duplicate one critical authority rule at the very end. Run the prompt twice (with and without the changes) and observe whether behavior shifts.

Hint

If you do not see a difference, either the prompt was already short enough, or the rule was not actually critical.

Beyond 'put it at the front'

Duplication is sometimes correct

Section markers

Code

External links

Exercise

Progress

Comments 0