Reading an agent's trace

Every time an agent works, it produces two things: a report at the end, and an execution trace along the way. The report is the model's writing about what it believes happened. The trace is an ordered record of what actually happened: every model message, every tool call with its input, every result or error that came back, and the cost and number of turns.

The two often match, but not always. A model can report "all tests pass" when the last test it ran still failed, or it never ran it at all. A model can write "I checked every file" when it opened three out of twenty. This is not deliberate lying; the model writes what seems a fitting end to the task. That is why the report is a claim, and the trace is evidence.

Reading a trace need not be complicated. Start from the end: what does the report claim? Then find in the trace the tool calls that should prove that claim, and read their output yourself. If the report says tests pass, find the last test output. If it says five files were changed, count the edit-file tool calls. If the evidence is not there, the claim is not proven, however confident the sentence.

Besides matching claims, a trace tells you three other useful things:

• Wasted turns. A search repeated with the same words, a file read twice, or a tool that keeps returning the same error. This signals that the goal or a tool description needs fixing.
• Touches nobody asked for. Files or folders opened that were not relevant, especially those holding personal data or secrets. This signals that permissions need narrowing.
• Cost per task. The number of turns, tokens in and out, and the cost estimate, including the cost of subagents the main agent calls. Record these numbers for every run; that is how you know whether a prompt change made the agent cheaper or more wasteful.

For one run, reading the trace does feel slow. But this habit is what lets you trust an agent later. After a few dozen runs you will know the normal pattern, and an unusual trace stands out at once. The next step is systematic evaluation: a set of test tasks rerun every time something changes. That is the project at the end of this phase.

As a worked example, the Claude Agent SDK for Python streams the trace as a series of messages from query(). AssistantMessage holds the model's writing and its tool requests; the ResultMessage at the end holds subtype (success or the kind of error), num_turns, duration_ms, total_cost_usd and usage (input tokens, output tokens and cache tokens):

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, AssistantMessage, ResultMessage

async def main():
    options = ClaudeAgentOptions(tools=["Read", "Glob", "Grep"], allowed_tools=["Read", "Glob", "Grep"], permission_mode="dontAsk", max_turns=12)
    async for message in query(prompt="I need a list of decisions with their source file names.", options=options):
        if isinstance(message, AssistantMessage):
            print(message)
        if isinstance(message, ResultMessage):
            print(message.subtype, message.num_turns, message.duration_ms)
            print(message.total_cost_usd, message.usage)

asyncio.run(main())

As in the previous lesson: tools makes only three read tools available, allowed_tools lets them run without asking, and permission_mode="dontAsk" refuses any other call that would ask. Without tools, dontAsk alone still lets through calls that need no approval, such as read-only shell commands and starting a subagent. For agents that use subagents, usage covers only the main loop; read model_usage, which covers the whole tree of calls.

Save those messages to a log file per run, without secrets, so they can be reread and compared. total_cost_usd is the library's estimate; for the official bill, use the provider's report. A step specific to Anthropic; check the latest version in its documentation, because message and field names can change.

An exercise for your agent's next run: before reading its report, write down three claims you expect to see, for example "tests pass", "two files changed", "no file outside the folder was touched". Then open the trace and look for evidence for all three. Note how many claims were proven, how many had no evidence and how many turned out wrong. After ten runs, those numbers give an honest picture of how far your agent's reports can be trusted, and which parts you always need to check yourself.