Writing / Tag / 16 of 77
Cost
Every essay tagged Cost, newest first.
Cost on this site means token spend you did not plan for. An agent that loops on the same tool seventy times, a memory store that keeps every junk observation, a fleet of agents nobody can count, a run that keeps working after the user has closed the tab. None of these are pricing problems. They are control problems, and the invoice is just where they surface.
So the essays here are mostly about putting a boundary where there was none: a deadline that propagates cancel down the whole tree, a no-progress guard that stops the loop, a per-tenant budget enforced before the call rather than after, a token budget for a long context window instead of filling it because it is there.
If you want one place to begin, read the tracing essay, because you cannot fix spend you cannot see. Then prompt prefix caching, which is usually the largest single saving, and the loop detection guard for the failure mode that bills you the fastest.
October 2026
Your Load Balancer Deletes the Prefix Cache on Every Agent Turn
You self-host inference, you turned on prefix caching, and the cost barely moved.Your GenAI Dashboards Go Blank After an OpenTelemetry Upgrade
The GenAI semantic conventions are still in Development, and they rename and remove attributes between versions.You Turned Off Thinking and Your Agent Stopped Calling Tools
Moving a step to a cheaper, faster model usually means turning the thinking knob down.Your Agent Decides at 0.8 Confidence. You Never Measured 0.8.
A model returns 0.9 and your pipeline treats it as ninety percent sure, so it auto approves, routes, or flags on that number without anyone checking what the number means.Your Token Budget Is Wrong the Day You Switch Models
You moved to a newer, cheaper model and the bill went up.Your Model Router Saves Pennies and Burns Your Prompt Cache
You added a model router to cut costs and the bill went up.When Every AI Provider Goes Down at the Same Time
Early this month three of the big model providers degraded within a few hours of each other, and a lot of AI features went dark with them.
September 2026
Nobody Can Tell You How Many Agents You Are Running
A year of shipping agents leaves most teams with a fleet nobody can inventory: no owner, no declared budget, no record of which tools each one can reach.Your Agent's Memory Is Full and Most of It Is Junk
You gave your agent persistent memory and it worked.Your Agent Fetches the Same Row Ten Times a Session
A long-running agent calls the same read tool over and over inside a single session, paying full latency and quota for answers it already had a few turns ago.Your Model Advertises 1M Tokens. It Starts Forgetting Around 600K.
A million-token context window landed in half the models this week, and the reflex is to stop retrieving and just paste everything in.
August 2026
Your Agent Called the Same Tool Seventy Times and Billed You for It
A ReAct-style agent calls the same search tool, gets the same unhelpful result, decides another identical call will help, and does it again.Your Agent Has 200 Tools and Picks the Wrong One
Connect enough MCP servers and your agent carries hundreds of tool definitions into every turn.Your Agent Kept Working After the User Left
A user closes the tab and your agent keeps going: three tool calls in flight, two subagents still reasoning, tokens still burning for an answer nobody will read.
July 2026
Your Agent Cost $2 Yesterday and $40 Today and You Cannot See Why
An agent's cost is elastic and path-dependent, so the same task runs for two dollars one day and forty the next.You Pay Full Price for the Same 8,000-Token Prompt on Every Agent Step
An agent resends the same enormous system prompt, tool list, and history on every step, and without prefix caching you pay full input price and full prefill latency each time.