Your Agent Keeps Retrying a Tool Call That Will Never Work

Watch an agent fail in production and it rarely looks like bad reasoning. The model picks the right tool. It fills in arguments that look correct. Then the call comes back with a 400, or a 401, or nothing at all because the service timed out, and the agent does the one thing you did not want: it tries again with the exact same arguments, gets the exact same error, and loops until it runs out of budget or patience.

The model was never the problem. The tool call was, and the layer around the tool call did not know what to do with a failure.

This is where a lot of agent projects quietly stall. The demo works because in a demo the token has not expired, the API is up, and the arguments happen to be valid. Production is the opposite of a demo. Tokens expire, APIs rate limit, arguments drift out of range, and a system that treats every failure as "try again" turns a small hiccup into a cascading outage.

The business problem: your agent is only as reliable as its worst tool

An agent is a model wrapped around a set of tools. The model is the easy part to reason about, because it is the part you prompt. The hard part is everything the tools touch: your database, a payments API, a search index, someone else's flaky third-party service. Every one of those is a place the agent can fail for reasons that have nothing to do with how smart the model is.

The industry data lines up with this. Surveys through 2026 keep putting quality and reliability at the top of the list of reasons agents do not make it to production, well ahead of model capability. And when teams dig into their traces, the pattern repeats: most agent failures in production are not bugs in the agent logic, they are external failures. Provider timeouts. Malformed tool responses. Expired credentials. Partial executions interrupted by an infrastructure blip.

Framed as a business problem, unreliable tool calls cost you in three ways. They cost money, because a retry loop burns tokens and downstream API quota on calls that were never going to work. They cost trust, because an agent that fails halfway through a task and cannot recover is worse than no agent at all, since now a human has to figure out what it did and did not do. And they cost availability, because naive retries against a struggling dependency are how you turn a five second blip into a fifteen minute outage.

The fix is not a smarter model. It is a disciplined layer between the model and your tools that does three things: validate arguments before you run anything, classify failures when they happen, and recover in the way that matches the failure.

Validate before you execute

Here is the thing the provider docs are honest about and a lot of teams skip: the model does its best to produce valid arguments, but it does not guarantee they match your schema. A required field can go missing. A number can land out of range. An enum can come back as a value you never defined. If you pass those straight into your tool, you find out the hard way, usually after the call has already caused a side effect.

Validate first. The cheapest place to catch a bad argument is before it leaves your process, where you can hand the model a precise correction and let it try again without spending a real API call.

# pip install anthropic pydantic
from pydantic import BaseModel, Field, ValidationError

class RefundArgs(BaseModel):
    """Schema for the issue_refund tool. Validation lives here, not in the tool body."""
    order_id: str = Field(min_length=1)
    amount_cents: int = Field(gt=0, le=100_00)  # refunds capped at $100
    reason: str = Field(min_length=3, max_length=200)

def validate_args(schema: type[BaseModel], raw: dict) -> tuple[BaseModel | None, str | None]:
    """Return (parsed_args, None) on success, or (None, error_message) on failure.

    The error string is written to be read by the model, so it can repair the
    call on the next turn instead of you guessing what went wrong.
    """
    try:
        return schema(**raw), None
    except ValidationError as e:
        problems = "; ".join(
            f"{'.'.join(str(p) for p in err['loc'])}: {err['msg']}"
            for err in e.errors()
        )
        return None, f"Invalid arguments: {problems}. Fix these and call the tool again."

That error message is the important part. When validation fails, you do not raise and die, and you do not silently drop the call. You return the error into the model's context as the tool result, and the model gets a chance to correct itself. A missing order_id or an amount_cents of 0 becomes a one turn self repair instead of a stack trace.

Classify the failure, then pick the recovery

Once the arguments are valid and you actually run the tool, a different class of failure shows up: the ones that come from the outside world. This is where blind retrying does the most damage, because failures that look identical at the execution layer need completely different responses.

A timeout and a rate limit are transient. Wait and retry. A 400 for a malformed request and a 422 for a business rule violation are permanent for those arguments. Retrying is pointless, but repairing the arguments might work. A 401 is auth, and it is worth being precise here, because an expired access token and a revoked refresh token can look the same and demand opposite responses: one needs a refresh, the other needs a human. Everything else, treat as unrecoverable until you have a reason to believe otherwise.

Give each failure a category, and the recovery logic falls out of it.

from enum import Enum
import httpx

class Failure(str, Enum):
    TRANSIENT = "transient"      # retry with backoff
    BAD_ARGS = "bad_args"        # send error back to the model to repair
    AUTH = "auth"                # refresh or escalate, do not retry blindly
    UNRECOVERABLE = "unrecoverable"  # stop, report clearly

def classify(err: Exception) -> Failure:
    """Map a raw exception to a recovery category. This is the switchboard the
    rest of the agent routes on, so keep it explicit and easy to extend."""
    if isinstance(err, (httpx.TimeoutException, httpx.ConnectError)):
        return Failure.TRANSIENT
    if isinstance(err, httpx.HTTPStatusError):
        code = err.response.status_code
        if code in (408, 429, 502, 503, 504):
            return Failure.TRANSIENT
        if code in (400, 422):
            return Failure.BAD_ARGS
        if code in (401, 403):
            return Failure.AUTH
        if code >= 500:
            return Failure.TRANSIENT
    return Failure.UNRECOVERABLE

Notice 429 and the 5xx codes route to TRANSIENT, while 400 and 422 route to BAD_ARGS. That single distinction is the difference between an agent that recovers gracefully and one that hammers a dead endpoint. The classifier is also the one place you extend as you learn: the first time a specific provider returns a weird status, you add a line here, and every tool inherits the fix.

Recover, with a budget

Now wire the pieces into an execution loop that runs a tool, and on failure does the right thing per category, with a hard cap so it can never loop forever.

import time

def run_tool(fn, args, *, max_attempts: int = 3):
    """Execute a validated tool call with classification-driven recovery.

    Returns (ok, result_or_message). On BAD_ARGS the message goes back to the
    model to repair; on exhaustion it goes back to the caller as a clean failure.
    """
    for attempt in range(1, max_attempts + 1):
        try:
            return True, fn(**args)
        except Exception as err:
            kind = classify(err)

            if kind is Failure.TRANSIENT and attempt < max_attempts:
                # Exponential backoff with a fixed base. Add jitter in real code
                # so a fleet of agents does not retry in lockstep.
                time.sleep(0.5 * (2 ** (attempt - 1)))
                continue

            if kind is Failure.BAD_ARGS:
                # No point retrying the same args. Hand the reason to the model.
                return False, f"Tool rejected the request: {err}. Adjust the arguments."

            if kind is Failure.AUTH:
                # Do not retry a credential failure in a loop. Escalate once.
                return False, "Authorization failed. This tool needs re-authentication."

            # Unrecoverable, or transient attempts exhausted.
            return False, f"Tool failed after {attempt} attempt(s): {err}"

    return False, "Tool failed: retry budget exhausted."

Three things make this loop safe. It counts attempts, so the transient path cannot spin forever. It only retries the category that benefits from retrying, so a bad argument does not consume the budget on identical failing calls. And every non retry path returns a clear message, either to the model so it can repair, or to the caller so a human sees an honest failure instead of a silent hang.

For a tool that hits a genuinely flaky dependency, add a circuit breaker on top: track recent failures for that specific tool, and once they cross a threshold, stop calling it for a cooling off period and return the failure immediately. That is what stops one sick downstream service from dragging the whole agent down with it, and it is the same instinct as containing failures inside a worker so they do not cascade, which I covered in the orchestrator worker split.

The tradeoffs and the pitfalls

Idempotency is not optional for anything with side effects. The refund tool above is the scary case. If a call times out, you do not actually know whether the refund went through, only that you did not get a response. Retry it and you might refund twice. Any tool that changes state needs an idempotency key so a retry is safe, and if a tool cannot be made idempotent, it should not be on the transient retry path at all. Decide this per tool, not globally.

Do not swallow errors into the happy path. A tempting shortcut is to catch everything and return an empty result so the agent keeps going. That produces the worst failure mode there is: the agent proceeds confidently on missing data and gives a wrong answer that looks right. A failed tool should surface as a failure the model can see and reason about, not as a silent empty string.

Classification is a living thing, and it deserves tests. Your first classifier will be wrong about some provider's quirks. That is fine, as long as the mapping is one explicit function you can correct and, more importantly, test. The recovery logic is exactly the kind of thing worth pinning down with evals in CI, so a well meaning change to the retry policy does not quietly reintroduce the retry storm you just fixed.

More layers mean more latency, so match the effort to the tool. A read only search tool that fails cheaply does not need a circuit breaker and an idempotency key. A payments call does. Reliability engineering has a cost, and spending it uniformly across every tool is its own kind of waste. Put the heavy machinery where a failure actually hurts.

The takeaway

Agents do not mostly fail because the model reasoned badly. They fail because a tool call went wrong and the code around it treated every failure the same, usually as a reason to try again. The result is an agent that loops on dead endpoints, burns budget on calls that will never succeed, and occasionally does real damage by retrying something it should not have.

The fix is three disciplined steps that live entirely in your code, not in the model. Validate arguments before you execute, so bad calls become cheap self corrections instead of side effects. Classify failures when they happen, so a timeout and a bad argument stop getting the same response. Recover per category with a hard budget, so retries are bounded, repairs go back to the model, and unrecoverable failures surface honestly. None of it requires a smarter model. It requires treating the tool layer like the production system it actually is.

If your agent works in demos and keeps falling over on the tool calls in production, book a consultation call and we will make its failure handling boring, which is exactly what you want it to be.