Tag:Production

A2A hit v1.0 this year and teams are wiring their agents to other teams' and vendors' agents. The moment your agent fetches an Agent Card and acts on it, you have extended your trust boundary to identity and code you do not control. The spec advertises auth schemes but leaves verification to you. Here is the admission gate that makes an outbound A2A call safe: verify the signed card, scope the credential you hand over, and block replays.

Reasoning traces are the feature everyone turned on this month and nobody planned for. Handled wrong, they quietly triple your token bill, dump intermediate reasoning full of customer data into your logs, and break your tool loop in a way that looks like the model got dumber. Here is how to treat extended thinking as something you manage at the boundary, not something you print.

Article 50 of the EU AI Act took effect on August 2, and it says any image, audio, video, or text your system generates has to carry a machine-readable mark that it was made by AI. A checkbox in your UI does not satisfy that. The mark has to live inside the file, survive a re-upload, and be verifiable. Here is how to build that into your generation pipeline as a gate, not a bolt-on.

A ReAct-style agent calls the same search tool, gets the same unhelpful result, decides another identical call will help, and does it again. Seventy times. It never crashes and never finishes, it just burns tokens going nowhere until a timeout or your bill catches it. A hard step ceiling is a backstop, not a fix. What you want is a guard that notices the agent has stopped making progress and breaks the cycle in seconds.

You pulled a skill off a registry, your agent read its instructions, and it is now ready to execute whatever code shipped in the folder. Nobody signed it, nobody diffed it, and the manifest that says "read-only" is just a text file the author wrote. Treat every third-party skill as untrusted code and put a gate in front of it: pin and hash it, check what it actually does against what it claims, and run it in a box that can only reach what it declared.

Your eval suite is green. Every case passes. Then a real user has a six-turn conversation with your agent and it forgets what they said in turn one, asks for information they already gave, and quietly breaks a policy under pressure. Single-shot evals test a single prompt. Production is a conversation. Drive your agent with a simulated user and you can test the thing users actually do.

A user sends you a screenshot of your agent doing something wrong. You have the logs, you have the trace, and you still cannot make it happen again, because the model sampled differently and a tool returned something new. You are debugging a ghost. Record every source of nondeterminism during the run and you can replay that exact failure on demand.

Your retrieval is good. The right chunks come back, the citation links resolve, and the answer reads clean. Then a user clicks the citation and the document says nothing of the kind. The model wrote a confident claim, stapled a real source to it, and shipped. A grounding gate catches that before the user does.

Your agent does real work before it answers: it searches, calls three tools, reasons about the results, then writes a reply. From the user's side that is twenty seconds of a spinning circle and no idea whether anything is happening. Stream the steps and the tokens as they land, and the same slow agent starts to feel fast.

You wired an LLM up as the grader for your evals, and now every release looks green. The problem is the judge is not scoring quality, it is scoring length, order, and answers that sound like its own. Fix the three biases that matter and calibrate the judge against human labels, so the number your pipeline gates on actually means what you think it means.

A tool call times out, your agent retries it, and the customer gets billed twice because the first request went through before the response came back. The fix is not fewer retries. It is idempotency keys on every write tool, so a repeated call with the same key returns the first result instead of doing the work again.

Connect enough MCP servers and your agent carries hundreds of tool definitions into every turn. It pays for all of them on every request and still reaches for the wrong one, because the model is choosing from a wall of near-duplicate schemas. The fix is to stop shipping the whole toolbox and retrieve the handful that matches the task instead.

A user closes the tab and your agent keeps going: three tool calls in flight, two subagents still reasoning, tokens still burning for an answer nobody will read. The fix is a deadline that every layer respects and a cancel that propagates down the whole tree, tearing in-flight work down cleanly instead of leaving it to finish alone.

Most of what an agent does is safe to let run. A few things are not: the refund, the production deploy, the email to a customer. The answer is not to make the agent slower everywhere. It is an approval gate that pauses the run before the risky action, persists the pending decision, and resumes exactly where it stopped once a human says yes or no.

Input defenses stop bad instructions from getting in. They do nothing about what your agent says on the way out. One generated reply that leaks another customer's data or makes a promise you never authorized is a message you cannot unsend. A fail-closed egress layer that checks every output before it ships is how you keep that message from ever leaving.

LLM spend is per request and wildly variable, so a single runaway agent or one heavy tenant can externalize its cost straight onto your margin. Cheaper models and caching lower the average, but nothing stops the bill. A per-tenant spend ledger with the cap enforced before the call does.

An agent that spawns subtasks and fans out tool calls builds an unbounded work queue, hammers your rate limits, and burns the token budget before the run finishes. Backpressure, bounded concurrency, and circuit breakers keep the fan-out from taking down the systems it depends on.

A long agent run that crashes halfway restarts from scratch, re-spending every token it already paid for and re-firing side effects you never wanted twice. Durable execution fixes that with checkpoints, replay-safe steps, and idempotency keys.

Most of what your agent gets asked, it has already answered. A semantic cache reuses those answers on near-identical questions, cutting cost and latency, as long as you build the guardrails that stop it from serving the wrong one.

Once an LLM feeds a real workflow, "mostly valid" JSON is a failed handoff, not a formatting quirk. Here is how to get schema-valid output every time with a validation gateway that parses, repairs, and fails loudly.

The moment your agent reads untrusted content and can also call tools, a hidden instruction in a web page or email can hijack it. Here is the containment strategy that keeps a landed prompt injection from doing damage.

Most agent failures in production are not bad reasoning, they are bad tool calls: wrong arguments, expired tokens, timeouts. Here is how to validate, classify, and recover from tool failures instead of retrying blindly.