Tool Results Need Summaries, Not Dumps

~24 min · tools, summaries, noise

Level 0Window Watcher

0 XP0/50 lessons0/13 achievements

0/100 XP to next level100 XP to go0% complete

Tools can poison context

A tool that returns 20,000 lines may technically be successful and operationally harmful. The model now has to carry that blob through future turns, where it competes with the actual task. Long sessions accumulate tool dumps until everything important is buried in tar -tvf output.

Task-shaped tool results

Good tools return task-shaped observations: what changed, what failed, relevant lines, counts, paths, and next suggested checks. Keep raw output in a file or log and load slices only when needed. Summaries must preserve evidence — line numbers, commands, exit codes — without dragging the entire flood through every future turn.

Compact does not mean vague

The summary should still let the model reason. 'Tests failed' is too compact. '2 failing tests in auth/session.spec.ts at lines 42 (expected 401 got 200) and 88 (timeout)' is the right density.

A giant tool dump is a context tax on every future turn.

Code

Task-shaped tool result·yaml

tool_result:
  command: "npm test"
  exit_code: 1
  summary: "2 failing tests in auth/session.spec.ts"
  evidence:
    - "auth/session.spec.ts:42 expected 401 got 200"
    - "auth/session.spec.ts:88 timeout waiting for logout"
  raw_log_path: "/tmp/run-123.log"

Wrapper for noisy tools·python

def shape_test_output(raw_stdout: str) -> dict:
    failures = parse_failures(raw_stdout)
    return {
        "exit_code": last_exit_code(),
        "summary": f"{len(failures)} failing tests",
        "evidence": [f"{f.path}:{f.line} {f.message}" for f in failures[:10]],
        "raw_log_path": dump_raw_to_tmp(raw_stdout),
    }

External links

OpenAI Responses API reference

Exercise

Take a noisy command output and rewrite it as a task-shaped tool result with summary, evidence, and raw log pointer.

Hint

Keep exact lines that matter. Drop the decorative flood.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousRAG vs Long Context Is a Routing Problem Next →Evict What Stopped Earning Attention

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.