Tag:Architecture

A tool-using agent spends a surprising share of its wall clock doing nothing, just waiting for a network round trip while the model has already stalled. CPUs solved this problem decades ago with branch prediction. You can borrow the same trick: predict the next tool call, run it while the model is still reasoning, and commit the result if the guess was right. Done carefully it cuts latency by a third with zero effect on correctness. Done carelessly it fires off writes nobody asked for.

Spawning a subagent to keep the parent's context clean is the right instinct, and it is also where a lot of long-running agents quietly go wrong. The subagent does the work, returns a tidy paragraph, and the parent acts on it. But that paragraph is a lossy compression boundary nobody designed, and when it drops the one fact that mattered, the evidence is already gone. The fix is not a smarter summary. It is a return contract.

Wire a few agents together with direct handoffs and it works. Add a fifth and the wiring becomes the system, brittle and impossible to trace. Most teams drift into a shared context blob that nobody designed, then spend weeks debugging it. That blob is a blackboard, a forty-year-old architecture pattern, and building it on purpose instead of by accident is what keeps a multi-agent system auditable as it grows.

A tool call times out, your agent retries it, and the customer gets billed twice because the first request went through before the response came back. The fix is not fewer retries. It is idempotency keys on every write tool, so a repeated call with the same key returns the first result instead of doing the work again.

A user closes the tab and your agent keeps going: three tool calls in flight, two subagents still reasoning, tokens still burning for an answer nobody will read. The fix is a deadline that every layer respects and a cancel that propagates down the whole tree, tearing in-flight work down cleanly instead of leaving it to finish alone.

Most of what an agent does is safe to let run. A few things are not: the refund, the production deploy, the email to a customer. The answer is not to make the agent slower everywhere. It is an approval gate that pauses the run before the risky action, persists the pending decision, and resumes exactly where it stopped once a human says yes or no.