The 2026 frontier of attacks
Direct injection (user types adversarial instructions) is well known. The bigger problem in production is indirect injection: a tool returns content from the wider world (a search result, a customer email, a webpage), and that content contains instructions. The user didn't type them; the data did.
Where it shows up
- RAG over user-uploaded docs (PDF with hidden instructions in metadata).
- Agents that read emails (subject line tries to redirect actions).
- Web-browsing agents (page contains "if you are an AI, do X").
- Code-reading agents (comment in a repo's README acts on the model).
Defenses specific to indirect
- Tag tool outputs as untrusted in the prompt's wrapping.
- Strip imperatives from tool outputs when feasible (rare, brittle).
- Require confirmation before consequential action on data from new untrusted sources.
- Cross-source corroboration for high-stakes claims.