LLM

The LLM tag collects the essays that are about the model call itself rather than the agent around it: what you send, what comes back, what it costs and how long it takes. That includes structured output that is almost valid JSON, prompts that resend the same 8,000 tokens on every step, routing between models of different prices, and evals that catch a prompt regression before a support ticket does.

A lot of these started as cost problems. Token spend is per request and elastic, so the same task can cost two dollars one day and forty the next, and the fixes tend to improve latency and reliability at the same time. The guide this tag maps to walks through that in order; the essays go deeper on each piece.

Start with model routing to see the cascade pattern that sends each request to the cheapest model that can handle it, then prompt prefix caching for the single change that cuts both the bill and prefill time. Evals in CI is the third one to read, because none of the savings hold if you cannot tell when quality dropped.