C.W.K.
Stream
Lesson 05 of 05 · published

What To Do When (Not If) A Secret Leaks To An LLM

~15 min · incident-response, rotation, post-mortem

Level 0Greenhorn
0 XP0/53 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete

Treat the leak as inevitable. The interesting question is what your incident response looks like, not whether you'll have one.

The response tree

What leakedFirst actionWhy now
API key (OpenAI, Anthropic, etc.)Rotate via provider dashboard immediatelyCosts you nothing; bills accrue if exploited
GitHub PATRevoke in GitHub settingsRepo write access = supply-chain compromise
Database passwordRotate, then check access logsData exfiltration window starts the moment it leaks
SSH keyRemove from authorized_keys on every server, generate new keyPersistent access until removed
App PIN / session token (this quest)Click Revoke All from Track 7Sessions die instantly
OAuth client secretRotate at the OAuth providerAffects every app instance

The three lookups after rotation

  1. Provider audit log. OpenAI/Anthropic/etc. show usage by key. Anything unusual between leak time and rotation time?
  2. Application logs. Any unexpected requests with the now-dead key/token?
  3. Billing. Spike in usage = someone else used your key. Bills usually lag a day; check tomorrow too.

What you cannot do

You cannot scrub the secret from the LLM provider's logs, training pipeline, or third-party caches. Once the secret was sent to a remote server, it exists in N copies for some retention period you don't control. Rotation is the only response that works. Apologies don't.

The post-mortem question

"What would have prevented this leak from being sent at all?" Almost always the answer is "the secret should never have been in a file the agent could read." Translate that into a permanent change: keychain migration, ignore-file update, scoped token to replace the broad one.

External links

Exercise

Pick one secret you've definitely pasted into an AI chat at some point (almost everyone has, for at least a stack trace). Rotate it today. Then check the provider's audit log for the rotation window — anything unusual? Document what you find in the audit log of your own app (Track 8). The exercise is the rotation muscle memory.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.