You cannot improve what you cannot replay
Agent observability is more than console logs. You need traces of model calls, tool calls, inputs, outputs, costs, latency, approvals, errors, and stop reasons. You also need a way to turn failures into eval cases.
Evaluation closes the loop. Production failures become golden tests, graders, regression suites, and prompt/tool changes.
Trace the decision path
Record why the agent chose a tool, what the tool returned, what state changed, and whether a human approval or guardrail influenced the run. Redact secrets, but keep enough structure to debug.
Current OpenAI Agents docs frame traces as structured records of model calls, tool calls, handoffs, guardrails, and custom spans. Trace grading then lets you score those decision paths rather than treating the final answer as a black box.
For non-deterministic systems, evaluate behavior over datasets, not one heroic demo. Track task completion, accuracy, tool efficiency, cost, latency, and human correction rate.