Audit Trails — When Something Bad Happens

~12 min · security, audit, logging

Level 0Apprentice

0 XP0/100 lessons0/14 achievements

0/120 XP to next level120 XP to go0% complete

Logs you'll need at 2 a.m.

When an incident hits — a leaked credential, an unauthorized action, a refunded amount the model invented — the logs are the only ground truth. If they don't exist, you're guessing.

What to log per request

Request id (unique, surfaced to the user for support).
Authenticated user id (not just session token).
Prompt version, model, sampler params.
Full input (with PII handling per policy).
Retrieved chunks / tool calls / tool returns.
Reasoning trace (where available).
Final output, output filter flags.
Downstream actions taken (with idempotency keys).

Storage and retention

Logs need retention long enough to investigate and short enough to comply with privacy. Encrypt at rest. Make raw logs accessible only to incident responders; aggregate metrics broadly available.

Code

Audit log shape·json

{
  "request_id": "req_abc123",
  "user_id": "u_456",
  "prompt": "support.draft@7",
  "model": "claude-opus-4-7",
  "sampler": {"temperature": 0.2, "max_tokens": 1024},
  "input_redacted": "...",
  "retrieved": ["chunk:manual-3.4", "chunk:faq-12"],
  "tool_calls": [{"id": "call_xyz", "name": "search_orders", "args": {"customer_id": "..."}}],
  "output_flags": [],
  "actions": [{"name": "send_email", "id": "act_pqr", "idem_key": "..."}],
  "created_at": "2026-05-04T12:34:56Z"
}

External links

Exercise

Sketch the audit-log schema for one of your endpoints. Identify any field missing today (prompt version, retrieved chunks, idempotency key). Add it.