Your Agent Cost $2 Yesterday and $40 Today and You Cannot See Why

Two engineers run the same coding agent against the same repository on the same task. One of them spends two dollars. The other spends forty. Nothing changed in the prompt, the model, or the code. The only difference is that the second run got confused, started a tool call that failed, retried it, re-read the same three files trying to recover, and looped like that eleven times before it finished. Both runs returned a correct answer. Only one of them quietly cost twenty times more.

This is the thing about agents that normal services do not do to you: their cost is elastic and path-dependent. A REST endpoint does roughly the same work every time you call it, so its cost per request is a flat line you can reason about. An agent chooses its own control flow at runtime. It decides how many times to call the model, how many tools to invoke, and whether to loop when something goes wrong. The bill is a function of the path it took, and the path is invisible in your logs.

You go looking for the forty-dollar run and all you have is a wall of log lines. tool_call started, llm response received, tool_call started again, forty times over, with no structure telling you which call belongs to which step, which retry spawned which sub-call, or where in that mess the tokens actually went. Logs are a flat stream. What an agent produces is a tree. You cannot debug a tree by reading it one line at a time.

What you actually need is a span tree

The tool that fits this shape is distributed tracing, and it has existed for a decade in the world of microservices. A trace is a tree of spans. Each span is one unit of work with a start time, a duration, a parent, and a bag of attributes. The whole agent run is the root span. Each step the agent takes is a child. Each LLM call and each tool invocation inside a step is a child of that. When you look at the run in a trace viewer, you see the actual shape of what happened: the retry loop shows up as eleven sibling spans under one step, and the token usage sits right there on each of them.

The reason this is worth standardizing on rather than hand-rolling is OpenTelemetry. It is the vendor-neutral standard for traces, and in 2026 it grew a dedicated set of GenAI semantic conventions: an agreed vocabulary for the attributes that describe an LLM call. Instead of every team inventing its own field names, everyone records gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.response.finish_reasons. Any backend that speaks OpenTelemetry, and most of them now do, can then compute cost, group by model, and find the loops without you writing a custom exporter. You instrument once against the standard and stay portable across tools.

The rest of this post is how to wire that up so the forty-dollar run stops being a mystery.

Instrument the single LLM call first

Start at the leaf. Every model call becomes a span, and you put the GenAI attributes on it. This is the unit everything else is built from, so get it right once and wrap it.

from opentelemetry import trace

tracer = trace.get_tracer("agent")

# One helper wraps every model call so the same attributes land on
# every span. Centralizing the attribute names here means the GenAI
# conventions live in exactly one place you can update later.
def call_model(client, model: str, messages: list) -> dict:
    with tracer.start_as_current_span(f"chat {model}") as span:
        # Set request attributes before the call so a crash mid-call
        # still leaves you a span that says what was attempted.
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.system", "anthropic")
        span.set_attribute("gen_ai.request.model", model)

        resp = client.messages.create(model=model, messages=messages, max_tokens=1024)

        # Response attributes: token counts and why the model stopped.
        # These three lines are what make cost and loops visible later.
        usage = resp.usage
        span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", usage.output_tokens)
        span.set_attribute("gen_ai.response.finish_reasons", [resp.stop_reason])
        return resp

Notice what is not here: the prompt text and the completion text. The conventions treat message content as opt-in for a reason, and I will come back to why. The token counts alone are enough to attribute cost and spot a run that is calling the model far more than it should.

Make the run a tree, not a stream

A single span is a data point. The value shows up when you nest spans so the trace mirrors the agent's control flow. The root span is the whole run. Each iteration of the agent loop opens a step span. The model call and any tool calls happen inside that step, which makes them children automatically because OpenTelemetry tracks the current span in context.

def run_agent(client, task: str, tools: dict) -> str:
    # The root span. Everything the agent does this run hangs off it,
    # so one trace equals one run and the total cost rolls up here.
    with tracer.start_as_current_span("agent.run") as root:
        root.set_attribute("agent.task", task)
        messages = [{"role": "user", "content": task}]

        for step in range(MAX_STEPS):
            # Each loop iteration is its own span. A retry storm becomes
            # a fat pile of sibling spans under one step, which is
            # instantly visible in a trace viewer.
            with tracer.start_as_current_span(f"agent.step.{step}"):
                resp = call_model(client, "claude-sonnet-5", messages)

                if resp.stop_reason != "tool_use":
                    root.set_attribute("agent.steps", step + 1)
                    return final_text(resp)

                # Tool calls nest under the step too, so you can see
                # which step triggered which tool and how long it took.
                for block in tool_use_blocks(resp):
                    result = run_tool(tools, block)
                    messages.append(result)

    root.set_attribute("agent.halted", "max_steps")
    return "stopped: hit step limit"


def run_tool(tools: dict, block) -> dict:
    # A tool span records name, duration, and outcome. When a tool
    # keeps failing and dragging the agent into recovery loops, this
    # is the span that shows it, sitting right next to the retries.
    with tracer.start_as_current_span(f"tool {block.name}") as span:
        span.set_attribute("tool.name", block.name)
        try:
            output = tools[block.name](**block.input)
            span.set_attribute("tool.status", "ok")
            return tool_result(block.id, output)
        except Exception as exc:
            span.set_attribute("tool.status", "error")
            span.record_exception(exc)
            return tool_result(block.id, f"error: {exc}", is_error=True)

Run this once and open the trace. The two-dollar run is a tidy short tree. The forty-dollar run has one step span that fanned out into a dozen tool spans, half of them marked error, each with a model call and its token count beside it. You are no longer guessing which step looped. You are looking at it.

Turn tokens into a cost you can rank by

Token counts are the raw material. What a person actually wants is dollars, and they want it rolled up to the run so they can sort runs by spend and go straight to the worst offender. Do the conversion in a span processor, so it happens for every LLM span automatically and you never sprinkle pricing math through your business logic.

from opentelemetry.sdk.trace import SpanProcessor

# Price per million tokens. Keep this table in one place and out of
# the call sites, so a price change is one edit, not a hunt.
PRICES = {
    "claude-sonnet-5": {"in": 3.00, "out": 15.00},
    "claude-haiku-4-5": {"in": 0.80, "out": 4.00},
}

class CostProcessor(SpanProcessor):
    def on_end(self, span):
        model = span.attributes.get("gen_ai.request.model")
        price = PRICES.get(model)
        if not price:
            return
        # Derive cost from the token attributes already on the span.
        # Because the attribute names follow the GenAI conventions,
        # this same processor works for any instrumented model.
        cost = (
            span.attributes["gen_ai.usage.input_tokens"] / 1e6 * price["in"]
            + span.attributes["gen_ai.usage.output_tokens"] / 1e6 * price["out"]
        )
        # Emit a metric keyed by model so your dashboard can graph
        # spend per model, and per run through the trace it belongs to.
        cost_counter.add(cost, {"gen_ai.request.model": model})

Now cost is a first-class signal. Your backend sums the per-span cost up the tree to a total on the run span, and you can answer the question that started all of this: which runs cost the most, and inside those runs, which step and which model did the spending. The forty-dollar run sorts to the top, you open it, and the loop is right there. This pairs naturally with capping spend before it happens, which I covered in giving an agent fan-out a budget it cannot exceed: tracing tells you where the money went, and a budget stops it going there next time.

One attribute worth adding while you are here is cache usage. If you have turned on prompt prefix caching, the cached input tokens are billed at a fraction of the normal rate, and a trace that ignores them will overstate your cost and hide the fact that your caching silently broke. Record gen_ai.usage.cached_input_tokens alongside the others and price it separately, or you will chase a cost regression that is really a cache-hit-rate regression.

The tradeoffs worth naming

Tracing is not free, and the failure modes are real enough to plan for.

Content capture is a genuine decision, not a default. It is tempting to put the full prompt and completion on every span so debugging is easy. That text is user data, it is often the largest thing in the span by far, and storing it on every production request gets expensive and turns your trace store into a place PII quietly accumulates. The conventions make content opt-in precisely so you make this call deliberately. Capture it behind a debug flag or on a small sampled fraction of runs, not on all of them.

Volume needs sampling, and naive sampling hides the thing you care about. A busy agent fleet produces an enormous number of spans, and you will sample to keep the bill sane. The trap is head sampling that makes the keep-or-drop decision at the start of a run, before you know whether it turned into the forty-dollar loop. Prefer tail sampling, which decides after the run finishes, so you can keep every run that erred, exceeded a cost threshold, or ran long, and drop the boring cheap ones. The whole point is to keep the expensive outliers, and head sampling throws them away at random.

Cardinality will bite you if you are careless. Attributes with unbounded distinct values, a raw user id or a full tool argument on a metric dimension, blow up the cost of your metrics backend. Keep high-cardinality data on spans, where it belongs, and keep metric dimensions to bounded things like model name and tool name.

And the conventions are still partly in motion. The token, model, and cost attributes are stable and safe to build on. Some of the newer agent and content attributes are marked experimental and can be renamed between releases. Pin the version you target and route every attribute name through one small helper, so the day a name changes you edit one line instead of grepping your entire codebase. This is the same instinct as isolating any external contract you do not control, the way careful tool-call handling isolates provider quirks behind one layer you can fix in a single place.

The takeaway

An agent's cost and behavior are decided at runtime, which means the only way to understand a run is to see the path it actually took. Logs give you a flat stream and hide the shape. A trace gives you the tree, and once every LLM call carries its tokens and every step and tool is a nested span, the expensive run stops being a mystery and becomes a thing you open and read.

Do it in that order. Wrap the single model call with the GenAI attributes first. Nest steps and tool calls so one run is one tree. Convert tokens to cost in a processor and roll it up to the run. Then add tail sampling and a content flag so it scales without drowning you or leaking user data. That is a few hundred lines, and it is the difference between finding the loop in thirty seconds and paying the forty dollars again next week because you never found it at all.

If your agents are in production and the invoice has started to swing in ways nobody can explain, this is exactly the kind of visibility work I help teams put in place. Book a consultation call and we can trace where your runs are really spending their time and your money.