Your Agent Is One Prompt Doing Six Jobs

Most agents start as one prompt. You give a model a system message, wire in a few tools, wrap it in a loop, and it works. It classifies what the user wants, decides which tool to call, reads the result, decides the next step, and eventually writes an answer. One model, one context window, one loop doing all of it. For a demo, that is exactly the right amount of machinery.

Then you put it in front of real traffic and the cracks show. It handles the common path fine and falls apart on the edges. It nails four steps of a task and botches the fifth, and because the fifth was wrong, the answer is wrong. Its context fills up with the debris of everything it has done so far, and by step ten it is reasoning over a window so cluttered it starts to lose the thread. You add instructions to patch each failure, the system prompt swells to two thousand words of "and also remember to," and every fix makes the next one harder.

This is the monolithic agent problem, and it is the single most common reason agents that looked great in a notebook fall over in production.

The business problem: one prompt is a single point of failure

A monolithic agent is one LLM doing several distinct jobs inside one prompt. It is the router, deciding what kind of request came in. It is the retriever, deciding what to fetch. It is the reasoner, working through the logic. It is the executor, calling tools and writing the result. All of that shares one context window and one set of instructions.

The problem is that these jobs have nothing in common except that you asked one model to do all of them. Routing wants to be fast and cheap. Reasoning wants to be careful and slow. Retrieval wants a big pile of documents in context. Execution wants a clean, small context so it does not get distracted. Cram them together and every job drags on the others.

The reliability math is unforgiving. A monolithic agent's success rate is capped by the one subtask it does worst. If it routes well, retrieves well, reasons well, and executes badly, you have a badly executing agent, because that last weak step poisons the output no matter how good the rest was. There is no isolation. A stumble anywhere becomes a failure everywhere.

And the failure is hard to debug, because there is nothing to inspect between the input and the wrong answer. It was all one opaque model call. You cannot point at the routing decision, or the retrieval step, or the reasoning, because none of them exist as separate, observable things. They are all tangled inside a single generation.

This is why teams report roughly 40 percent of multi-agent pilots failing inside six months, and it is also why the ones that succeed did not start with multiple agents. They started with a monolith, watched exactly where it broke, and split along those fracture lines.

The pattern: an orchestrator that delegates to workers

The orchestrator-worker architecture takes the jobs that were fighting inside one prompt and gives each its own agent. A coordinator, the orchestrator, owns the plan. It reads the request, breaks it into subtasks, and hands each subtask to a worker built for exactly that job. Each worker has its own narrow instructions, its own tools, and crucially its own fresh context. It does its one thing, returns a distilled result, and the orchestrator assembles the pieces.

The shift is from one model that knows everything to a small team where each member knows one thing well. Here is the monolith we are moving away from, so the contrast is concrete:

// Monolithic agent: one loop, one context, every job crammed together.
// Routing, retrieval, reasoning, and execution all share this window,
// and it grows with every step until the model is reasoning over noise.
async function monolithicAgent(request: string) {
  const messages = [{ role: "system", content: HUGE_SYSTEM_PROMPT }, { role: "user", content: request }];
  while (true) {
    const res = await model.chat({ messages, tools: ALL_TOOLS });
    if (res.stop) return res.content;
    // Every tool result, from every kind of step, piles into the same context.
    const toolResult = await runTool(res.toolCall);
    messages.push(res.message, { role: "tool", content: toolResult });
  }
}

Now the orchestrator version. The coordinator plans and delegates. It never touches a tool itself, so its context stays small and readable:

// Orchestrator: decomposes the request into typed subtasks and dispatches
// each to the worker that owns that job. It holds the plan, not the details.
async function orchestrate(request: string) {
  // Step 1: the coordinator plans. It only decides WHAT needs doing,
  // returning a structured list of subtasks, not prose.
  const plan = await model.chat({
    messages: [{ role: "system", content: PLANNER_PROMPT }, { role: "user", content: request }],
    responseFormat: PlanSchema, // { subtasks: [{ type, input }] }
  });

  // Step 2: hand each subtask to its worker. Independent subtasks run
  // in parallel; each worker gets a clean context scoped to its job.
  const results = await Promise.all(
    plan.subtasks.map((task) => runWorker(task)),
  );

  // Step 3: the coordinator sees only distilled worker outputs, never the
  // raw tool payloads. It composes the final answer from summaries.
  return model.chat({
    messages: [
      { role: "system", content: SYNTHESIS_PROMPT },
      { role: "user", content: JSON.stringify({ request, results }) },
    ],
  });
}

Each worker is small and boring on purpose. A worker built for database lookups knows about SQL and nothing else. Its context does not carry the reasoning from three steps ago or the retrieval from a different part of the task. It gets a clean input, does its job, and returns a short result:

// A worker: one job, its own tools, its own fresh context. Whatever mess
// it makes internally stays inside its window and never leaks back up.
const WORKERS = {
  retrieval: { prompt: RETRIEVAL_PROMPT, tools: [vectorSearch, keywordSearch] },
  database: { prompt: DATABASE_PROMPT, tools: [runSql] },
  writer: { prompt: WRITER_PROMPT, tools: [] },
};

async function runWorker(task: { type: keyof typeof WORKERS; input: string }) {
  const worker = WORKERS[task.type];
  const messages = [{ role: "system", content: worker.prompt }, { role: "user", content: task.input }];
  // Same tool loop as before, but scoped to ONE job on a small context.
  // The 4,000-row query result lives and dies in this worker; the
  // orchestrator only ever sees the one number the worker distilled out.
  const res = await toolLoop(messages, worker.tools);
  return { type: task.type, summary: res.content };
}

The important thing is not that there are more agents. It is that each agent works on a small, relevant context and owns exactly one job. The retrieval worker can pull four thousand rows and boil them down to a number, and only the number crosses back to the orchestrator. That is the same discipline behind context compaction for long-running agents, applied at the agent boundary: the cheapest token, and the least distracting one, is the one a given agent never has to see.

Why the split fixes reliability

Three things change the moment you decompose.

Failures get isolated. When a worker botches its job, the failure is contained to that worker's output. The orchestrator can see that the database worker returned something malformed and retry just that subtask, instead of the whole task collapsing because one step went sideways. The weak-link problem does not vanish, but a weak link is now something you can catch and rerun, not a silent poison in one giant generation.

Each agent gets a clean context. A worker reasoning over five hundred tokens of relevant input makes better decisions than a monolith reasoning over fifteen thousand tokens of accumulated history, most of which has nothing to do with the step in front of it. Small, focused context is not just cheaper, it is more accurate, because the model is not spending attention filtering out its own past.

The system becomes observable. Now there is something to look at between input and output. You can trace which subtasks the orchestrator created, what each worker returned, and where the answer went wrong. That is what makes agents testable. You can write an eval for the router's classifications separately from an eval for the writer's output, which is exactly the kind of gate that catches regressions before your users do. A monolith gives you one number, pass or fail. A decomposed system gives you a failure you can localize.

There is a cost angle too. Because each worker runs on a small context, and because you can send the cheap subtasks to cheap models and reserve the expensive model for the reasoning that needs it, per-task cost often drops even though you are making more model calls. This is model routing applied inside your own agent: not every worker needs your best model, and paying frontier rates for a classification step is waste.

The tradeoffs nobody mentions

Decomposition is not free, and the failure statistics for multi-agent pilots are a warning, not a footnote.

The orchestrator is a new single point of failure. You moved the risk, you did not delete it. If the coordinator misclassifies a task, the wrong worker runs, and that error cascades through everything downstream. Worse, misclassification compounds as you add workers, because more categories means more ways to route wrong. The orchestrator's planning step is now the most important prompt in your system, and it deserves the most testing. A cheap, sloppy router at the top will sink an otherwise good architecture.

The orchestrator's context still grows. It accumulates a summary from every worker it calls. Keep the workers few and their outputs tight and this is fine. Let the orchestrator fan out to a dozen workers that each return verbose results and its window overflows, and you have recreated the monolith's context problem one level up. The rule that keeps this honest is that workers return distilled results, not transcripts. If a worker hands back everything it saw, the split bought you nothing.

More agents means more latency and more moving parts. Every hop between orchestrator and worker is a round trip. A task that was one model call is now a plan, several worker calls, and a synthesis. Parallelizing independent subtasks hides some of that, but a deeply sequential task will feel slower. You are trading latency for reliability, and that is only a good trade when you actually needed the reliability.

Most agents should not be split at all. This is the one teams get wrong most often. If your agent calls a model and two tools and returns a small answer, it is fine as a monolith, and wrapping it in an orchestrator just adds coordination overhead and new failure modes for no benefit. The split earns its keep when a single agent is genuinely juggling several distinct jobs, when its accuracy is bottlenecked by its worst subtask, or when its context is drowning in irrelevant history. Absent one of those, you are adding architecture to a problem you do not have.

The takeaway

A monolithic agent is one prompt doing the work of a router, a retriever, a reasoner, and an executor at once. It prototypes beautifully and breaks in production because those jobs fight over one context window, and because its success rate is hostage to whichever one it does worst, with no way to isolate or inspect the failure.

The orchestrator-worker split fixes that by giving each job its own agent, its own tools, and its own clean context, with a coordinator that plans and delegates instead of doing everything itself. Failures get contained, contexts stay small and accurate, and the whole thing becomes observable enough to test. The price is a new single point of failure at the orchestrator, more latency, and more parts to maintain, which is why you split along the fracture lines a real monolith showed you, not preemptively.

Start monolithic. Watch where it breaks. Split there, and nowhere else.

If your agent works in demos and keeps surprising you in production, and you are not sure whether the fix is a better prompt or a different architecture, book a consultation call and we will find the fracture lines together.