One Customer Burned Your Month of LLM Budget by 2pm

A single customer signed up on a Tuesday, pointed our agent at their entire document store, and left it running. By 2pm their workspace alone had spent more on model calls than the whole product was supposed to spend in a month. Nobody noticed until the provider dashboard updated the next morning, by which point the money was gone and the invoice was real.

Nothing was broken. The agent worked exactly as designed. It retrieved context, called the model, called some tools, looped, and did it again, thousands of times, because that customer's data made every task expand into a long chain of expensive steps. There was no bug to fix. There was just no ceiling. The cost of one tenant's usage landed directly on our margin because nothing in the request path was counting dollars and nothing was allowed to say no.

This is the part of running LLM agents in production that the tutorials skip. The model call is metered by the provider per token, the cost per request swings by two orders of magnitude depending on context size and completion length, and any single tenant can generate unbounded work. If you do not meter spend yourself, per tenant, and enforce a cap before the call goes out, you have handed every customer a blank check drawn on your account.

The levers you already have do not stop the bill

If you have been optimizing agent cost, you have probably reached for three things, and all three are worth doing. None of them is a spend cap.

Model routing sends easy calls to a cheaper model. That lowers the average cost per call. It does not stop a tenant from making a hundred thousand of them. Semantic caching avoids repeat calls entirely, which is the cheapest call there is, but only for work you have seen before. Novel work still hits the model at full price. Backpressure and concurrency control bounds how many requests are in flight so you do not stampede a dependency, but a rate limiter counts requests, not dollars, and ten cheap requests look exactly like ten that each drag fifty thousand tokens of context.

Every one of these lowers your average cost. A spend guardrail is a different kind of control. It sets a hard number, this tenant may spend at most this many dollars in this window, and it enforces that number in the request path before the expensive call happens. Averages protect your overall bill. A cap protects you from the specific tenant, or the specific runaway loop, that ignores the average.

The shape of the fix: meter, then gate

Two pieces. A ledger that records what each tenant has spent in the current window, and a guardrail that reads the ledger before every model call and decides whether to allow it, degrade it, or block it.

Start with cost itself. Cost is not a mystery, it is token counts times a per-model price. Keep the prices in one table so a provider price change is a one-line edit, and compute a call's cost from the usage the response reports.

from dataclasses import dataclass

# Prices in USD per 1M tokens. Keep this in one place; it is the only
# thing that has to change when a provider adjusts pricing.
PRICES = {
    "claude-opus-4-8":   {"input": 5.00, "output": 25.00},
    "claude-sonnet-4-6": {"input": 3.00, "output": 15.00},
    "claude-haiku-4-5":  {"input": 0.80, "output": 4.00},
}

@dataclass
class Usage:
    input_tokens: int
    output_tokens: int

def cost_usd(model: str, usage: Usage) -> float:
    p = PRICES[model]
    return (usage.input_tokens  * p["input"]  / 1_000_000
          + usage.output_tokens * p["output"] / 1_000_000)

Now the ledger. It has to do one thing well: increment a tenant's spend for the current window atomically, so two concurrent calls cannot both read the old total and each think there is room. Redis gives you this directly with INCRBYFLOAT and a key that expires at the end of the window. The atomic increment is the whole point. A read-modify-write in application code races, and under load it races exactly when you most need the cap to hold.

import redis

r = redis.Redis()

def window_key(tenant_id: str, window: str) -> str:
    # e.g. spend:acme:2026-08-06  -> a daily budget bucket that
    # rolls over on its own because the key expires.
    return f"spend:{tenant_id}:{window}"

def record_spend(tenant_id: str, window: str, amount: float, ttl_s: int) -> float:
    key = window_key(tenant_id, window)
    # Atomic: no two callers can interleave a read and a write here.
    total = r.incrbyfloat(key, amount)
    # Set the expiry once, on first write, so the bucket self-cleans.
    if total == amount:
        r.expire(key, ttl_s)
    return total

def current_spend(tenant_id: str, window: str) -> float:
    return float(r.get(window_key(tenant_id, window)) or 0.0)

Reserve the worst case, then settle the real number

Here is the detail that separates a guardrail that holds from one that leaks. Before the call you do not know the output length, and output tokens are the expensive ones. If you wait until after the call to record cost, every concurrent request passes the budget check first and records its cost second, and a burst sails straight past the cap together.

So you charge in two steps. Before the call, reserve the worst case: input tokens you can count exactly, plus max_tokens priced as output, because that is the most the model can bill you. Check that reservation against the remaining budget. If it fits, write the reservation to the ledger now, before the call goes out. After the call returns, you know the real usage, so you release the gap between what you reserved and what it actually cost.

from datetime import date

DAY_TTL = 24 * 60 * 60

def guarded_call(tenant_id: str, model: str, messages, max_tokens: int,
                 daily_budget: float, count_input_tokens):
    window = date.today().isoformat()

    # 1. Reserve the worst case. Input is known; output is capped at max_tokens.
    input_tokens = count_input_tokens(model, messages)
    reservation = cost_usd(model, Usage(input_tokens, max_tokens))

    already_spent = current_spend(tenant_id, window)
    if already_spent + reservation > daily_budget:
        # Over budget for the worst case. Caller decides: degrade or fail.
        raise BudgetExceeded(tenant_id, already_spent, daily_budget, reservation)

    # 2. Commit the reservation BEFORE the call, so concurrent callers
    #    see the money as already spoken for.
    record_spend(tenant_id, window, reservation, DAY_TTL)

    try:
        resp = call_model(model=model, messages=messages, max_tokens=max_tokens)
    except Exception:
        # Call never billed us: release the whole reservation and re-raise.
        record_spend(tenant_id, window, -reservation, DAY_TTL)
        raise

    # 3. Settle. Refund the gap between the reservation and the real cost.
    actual = cost_usd(model, Usage(resp.usage.input_tokens, resp.usage.output_tokens))
    record_spend(tenant_id, window, actual - reservation, DAY_TTL)
    return resp

That reserve-then-settle flow is the same instinct a payment processor uses when it places a hold on a card before the final amount is known. You authorize the maximum, capture the real figure, and release the rest. It costs you one extra ledger write per call and buys you a cap that holds under concurrency instead of one that is right on average and wrong exactly when a tenant floods you.

Blocking is the crude option. Degrading is usually the right one.

When guarded_call raises BudgetExceeded, you have a decision, and defaulting to a hard failure is often the wrong one. A hard block on a paying customer in the middle of a conversation is a support ticket and a reason to churn. The better move for most tenants is to degrade: try the cheaper path before you refuse the work entirely.

def call_with_degradation(tenant_id, messages, max_tokens, tier):
    # Ordered cheapest-capable-last so we downgrade, not upgrade, under pressure.
    ladder = ["claude-opus-4-8", "claude-sonnet-4-6", "claude-haiku-4-5"]
    budget = tier.daily_budget

    for model in ladder:
        try:
            return guarded_call(tenant_id, model, messages, max_tokens,
                                budget, count_input_tokens)
        except BudgetExceeded:
            continue  # too expensive at this model; drop to a cheaper one

    # Even the cheapest model does not fit. Now a hard stop is honest.
    raise BudgetExhausted(tenant_id, budget)

The tenant who would have blown their budget on Opus gets moved to Sonnet, then Haiku, and only hits a wall when even the cheapest model cannot fit under the cap. You can push the same idea further by shrinking their context or dropping optional tool calls before you refuse outright. The rule is to make the degraded behavior explicit and tiered, not a silent quality drop that nobody can explain three weeks later when the customer complains the answers got worse.

The pitfalls that actually bite

Metering the call is not metering the run. An agent turn is rarely one model call. It is a retrieval, a model call, three tool calls, another model call, in a loop. If you meter only the top-level request, you undercount by the loop's fan-out and the cap never trips until the damage is done. Charge every model call inside the run to the same tenant key, and if you trace your agent, attach spend to the trace so an expensive run is visible as one thing, not scattered across a hundred log lines.

A guardrail on the hot path must be fast. You are adding a store round trip before every model call. Against a model call that takes seconds, a sub-millisecond Redis increment is free, but do not reach for a relational database with a transaction per check, and do not put the ledger a continent away from the agent. If the ledger is slow, people delete the guardrail, and then you are back to blank checks.

Fail open or fail closed on purpose, not by accident. If the ledger itself is down, what happens? Fail closed and a Redis blip takes your whole product offline. Fail open and an outage becomes an unmetered spending window. Pick deliberately per tenant tier: fail open for trusted internal traffic, fail closed for the free tier where the abuse risk lives, and alert loudly either way so a degraded ledger does not quietly become no ledger.

The window has to match the commitment. A daily bucket that resets at midnight caps a bad day but says nothing about the month. If you sell a monthly plan, meter a monthly window too, and gate on whichever is tighter. Two cheap keys are less work than one surprise invoice.

The takeaway

LLM spend is variable, per request, and unbounded per tenant, which means the cost of one customer's usage or one runaway loop lands on your margin unless something in the request path is counting dollars and is allowed to say no. Cheaper models and caching lower the average and are worth doing, but the average is not what hurts you. The specific tenant who ignores the average is. The fix is small: an atomic per-tenant ledger, a reservation of the worst-case cost before each call, a settlement to the real figure after, and a degrade ladder so being over budget means a cheaper answer rather than a broken one. It is a middleware and a Redis key, and it is the difference between reading your provider dashboard with interest and reading it with dread.

If your agent costs are unpredictable and you have no per-tenant ceiling standing between a heavy customer and your margin, this is exactly the kind of production work I help teams put in place before it shows up on an invoice. Book a consultation call and we can find where your spend is running without a cap.