Writing / Tag / 12 of 78
Observability
Every essay tagged Observability, newest first.
Observability for an agent means being able to answer three questions after the fact: what did it cost, what did it decide, and which step actually caused the failure. Plain logs answer none of them. A trace with forty-seven spans shows that something happened, but not why the run went from two dollars to forty, or why the error at step 7 was really a bad decision at step 3.
The essays here build that up in layers. Distributed tracing with the OpenTelemetry GenAI conventions gets you the where and the how much. Decision records get you the why, in a form an auditor can read. Deterministic replay gets you the ability to reproduce a failure that only one customer ever saw.
Start with tracing on OpenTelemetry GenAI, which is the base everything else sits on. Then failure attribution with counterfactual replay for finding the decisive step instead of patching the symptom, and the audit trail essay for the difference between logging everything and proving anything.
October 2026
Your Coding Agent Can Switch Off the Monitor That Watches It
The controls that watch a coding agent usually live in the same repository the agent edits: the CI workflow, the monitor config, the alert routing, the policy files.Your Agent Fails Silently While Every Test Stays Green
The failures that hurt in production are not the ones that throw.Your GenAI Dashboards Go Blank After an OpenTelemetry Upgrade
The GenAI semantic conventions are still in Development, and they rename and remove attributes between versions.Your Agent Traces Are Shipping Customer Prompts to Your Observability Vendor
You turned on GenAI content capture to debug one tool call, and now every prompt and reply is sitting in a third party's searchable storage.Your Model Router Saves Pennies and Burns Your Prompt Cache
You added a model router to cut costs and the bill went up.Your Agent Logs Everything and Can Prove Nothing
A trace with forty-seven agent steps shows an auditor that something happened.Your Agent Failed at Step 7. The Bug Was at Step 3.
When a multi-step agent produces a confidently wrong answer, the step that threw the error is almost never the step that caused it.Your Evals Are Green and the Product Is Getting Worse
A frozen eval suite stops describing your product within weeks.Your Agent's One Reliability Number Is Hiding the Failures That Matter
A single pass rate on a dashboard tells you an agent is "99% reliable" and tells you nothing about which 1% is on fire.
August 2026
Your Agent Called the Same Tool Seventy Times and Billed You for It
A ReAct-style agent calls the same search tool, gets the same unhelpful result, decides another identical call will help, and does it again.A Customer Hit a Bug Your Agent Will Never Reproduce
A user sends you a screenshot of your agent doing something wrong.