Multi-agent systems: when one AI agent is not enough

Use more than one AI agent only when the work splits into parts that can be done independently with little shared context, such as researching many sources at once, or when a single agent keeps choosing the wrong tool even after you have cleaned up its instructions and tool descriptions. For most business workflows, one agent with well-designed tools is cheaper, easier to debug and makes fewer contradictory decisions. A second agent should be the answer to a specific problem you have measured, not the starting design.

Start by making one agent good

The practical agent guides from two model vendors open with the same advice. OpenAI's guide to building agents puts it as "maximize a single agent's capabilities first", noting that more agents bring "additional complexity and overhead, so often a single agent with tools is sufficient." Anthropic's guide to effective agents recommends "finding the simplest solution possible, and only increasing complexity when needed."

The reason is mechanical. A single agent works in one context: the task, the conversation, every tool result so far, so each decision can take every earlier one into account. Split the work across two agents and each sees only part of that history. Whatever you leave out of the message between them is something one agent will not know when it acts.

So before splitting, I would do the boring fixes. Rename tools so their purpose is obvious, tighten their parameters, remove near-duplicates, and move fixed sequences out of the prompt into code. A lot of what looks like "this agent is overloaded" turns out to be "this agent has three ways to look up a customer and no idea which one to use."

What the published numbers say

The most detailed public account of a multi-agent system in production is Anthropic's write-up of the research feature in Claude, published in June 2025. It used an orchestrator-worker design: a lead agent plans the research, starts subagents that search in parallel, and then combines what they found.

On Anthropic's own internal research evaluation, a setup with Claude Opus 4 as the lead and Claude Sonnet 4 subagents "outperformed single-agent Claude Opus 4 by 90.2%." Read the rest of the article before copying the design. The authors report that agents typically use about 4 times more tokens than chat interactions, and multi-agent systems about 15 times more. In their analysis of the BrowseComp benchmark, token usage alone explained 80% of the variance in performance. Put plainly, a big part of the gain came from spending more compute on the problem, spread across several context windows.

They are also direct about where the design does not fit: "some domains that require all agents to share the same context are not a good fit for multi-agent systems today", and they note that most coding tasks have fewer truly parallel parts than research does.

That gives a usable test. Multi-agent helps when the task is breadth-first, the answer is worth the token bill, and the threads do not need to know about each other until the end. Research across fifty documents fits. One customer's refund does not.

The shapes multi-agent systems take

OpenAI's guide describes two broad patterns. In the manager pattern, a central agent calls specialist agents as if they were tools, then combines their results; it stays in control and is the only one talking to the user. In the decentralized pattern, agents hand off to each other: a handoff is "a one way transfer", and the receiving agent takes over the conversation. Anthropic's orchestrator-workers workflow is close to the manager pattern, with the orchestrator deciding at run time how to break the task down. Anthropic also describes a simpler relative, parallelization, where the same model works on independent sections at once or on the same task several times so the outputs can be compared or voted on.

Context is what gets lost between agents

Walden Yan of Cognition wrote the clearest argument against splitting too early, in a June 2025 post titled "Don't Build Multi-Agents". It rests on two principles. The first is to share context, meaning full agent traces rather than individual messages. The second: "Actions carry implicit decisions, and conflicting decisions carry bad results."

His example is a small game split into two subtasks, the background and the bird, given to two subagents. One builds a background in a completely different visual style, the other builds a bird that does not match the game, and the final agent is left trying to combine two parts that were never designed to fit. Each subagent did a reasonable job of the brief it got. The brief just did not carry the decisions the other one was making.

The same problem in an operations setting, as an illustration: A distributor in Pune uses an orchestrator to handle a disputed invoice. Worker A reads the customer's email and the contract, and concludes the customer is owed a 5% volume discount. Worker B checks the ledger and finds an unpaid earlier invoice, and drafts a reminder. The orchestrator receives two tidy summaries and sends both: a credit note and a payment chase, to the same customer, on the same afternoon. Neither worker was wrong about its own piece. The conflict only exists in the combination, and nobody had the whole picture when it mattered.

A single agent doing both steps in sequence would most likely have noticed.

How multi-agent systems fail

The most systematic study I know of is "Why Do Multi-Agent LLM Systems Fail?" by Mert Cemri and colleagues, first posted in March 2025 and revised in October 2025. The authors built a failure taxonomy from 150 traces, checked it with human annotators (agreement of kappa 0.88), and released a dataset of over 1,600 annotated traces from seven multi-agent frameworks. They found 14 failure modes in three groups: system design issues, misalignment between agents, and task verification.

The individual modes read like an incident log: agents disobey their task or role, repeat steps, do not know when to stop, fail to ask for clarification, ignore each other's input, say one thing and do another, or verify the result badly or not at all. Anthropic's write-up lists the same kind of thing from its own early versions: spawning 50 subagents for a simple query, and subagents duplicating each other's searches because the briefs did not divide the work clearly.

Two of those groups are about communication and checking, not model ability, so spend effort on the briefs that go out, the results that come back, and a check on the combined answer.

You also need to see the whole run: the orchestrator's plan, each worker's brief, and each worker's tool calls and results, in one place. CloudOEngine's AI runtime traces every step and tool call an agent makes and makes them inspectable, which is the record you need to find the step where two branches went different ways. Our article on agent observability covers what such a trace should hold.

Signals that one agent has outgrown itself

OpenAI's guide gives two practical triggers for splitting. The first is complex logic: prompts full of if-then-else branches that have become hard to maintain. The second is tool overload, and the guide makes a point that is easy to miss: "The issue isn't solely the number of tools, but their similarity or overlap." Some setups handle more than 15 well-defined, distinct tools while others struggle with fewer than 10 overlapping ones. The guide's advice is to split only if clearer tool names, parameters and descriptions do not fix the problem.

I would add three signals of my own: the work fans out into many independent lookups and waiting for them one by one is the slow part; different parts of the job need very different permissions; or the context window fills up because the agent reads far more raw material than it needs to keep.

Notice what is not on the list: "a specialist for each department would be cleaner." That is how we organise people, but agents do not share a hallway. Every boundary you draw is a place where context has to be packed into a message and unpacked again.

Designing the split when you need one

If the signals are there, a few design rules prevent most of the failures above.

Keep one agent in charge. The manager pattern is easier to reason about than peers handing work around, because one agent owns the plan, sees every result and talks to the user. Use handoffs where the specialist genuinely takes over, such as moving a conversation from sales to support, and give the receiving agent the full conversation, not a summary of it.

Write briefs like work orders. Each worker needs the objective, the format of what it returns, which tools and sources to use, and where its job ends. Anthropic's early briefs were too vague, which is how its duplicated searches happened.

Make workers read, and let one agent write. Fan-out works best for gathering: searching, reading, checking. Actions that change something (sending an email, issuing a credit, closing a ticket) should come from a single agent that has the combined picture, ideally after a person has looked. In the invoice illustration, the fix is that neither worker sends anything; the orchestrator decides once, with both findings in front of it. CloudOEngine's Autopilot runs triggered agents with approval rules, so a person approves the action, in the app or through an email approve link, before it runs. Our article on human-in-the-loop AI covers which actions should wait for that approval.

Put limits in code: a cap on workers per run, steps per worker and total tokens. A prompt asking for few subagents helps, but only a hard ceiling stops the run that decides 50 are needed.

Add a verification step that reads the combined answer against the original request. Since so many documented failures involve weak verification, this is the cheapest improvement I know of.

Permissions multiply with agents

Every agent can read data and call tools. If all of them share one broad service account, you have built several copies of your most privileged component. Each worker should get only the tools and data its brief needs, and the run as a whole should never be allowed more than the person or system it acts for. The rules in our article on role-based access control for AI agents apply to each agent separately, which is why the permission setup tends to grow with the number of agents.

This is easier when the agents, their tools and the data they read sit under the same rules. CloudOEngine has one permission model and one audit trail across data connections, pipelines and the AI runtime, so a worker agent is limited by the same roles as everything else.

Agents that belong to someone else

Sometimes the second agent is not yours: a supplier's order agent, or a partner's scheduling agent. That is a different problem from splitting your own work, and it is the one the Agent2Agent (A2A) protocol addresses. A2A was started by Google and is now hosted by the Linux Foundation; its documentation describes agents as "opaque", meaning they collaborate without sharing their internal memory, tools or logic, and it positions A2A as complementary to the Model Context Protocol: MCP connects an agent to tools, A2A connects agents to each other.

Opacity is the right default between companies, but it makes the lost-context problem worse: you see the other agent's answer, never its reasoning, so check what comes back before acting on it.

What I would do first

If you are running a single agent and wondering whether to go multi-agent, this order avoids most wasted effort:

1. Collect 30 to 50 real tasks the agent handles, with the expected result for each, so you can compare designs on your own work.
2. Fix the single agent first: clearer tool names and descriptions, no overlapping tools, fixed sequences moved into code.
3. Look at where it still fails. If the failures are wrong tool choices, tangled instructions or a context window that fills up on broad reading, splitting may help. If they are bad data or vague requests, it will not.
4. Try the smallest split: one orchestrator, read-only workers, one agent that takes actions, hard caps on workers and tokens.
5. Run the same task set on both designs and compare success rate, cost per task and time per task.
6. Keep the multi-agent version only if it wins clearly on tasks that matter, by a margin that covers its extra tokens and the extra effort to debug it.

Many teams will stop at step two or three with a better single agent, which is a good outcome.

How CloudOEngine helps

CloudOEngine is one platform and console for connecting data (databases, files, camera streams), pipelines, an AI runtime (LLM gateway, agents with tool access, RAG), Vision AI and live video. Everything sits under one permission model and one audit trail, so one agent or an orchestrator with several workers is bound by the same roles as the rest of your systems, and what each one did lands in the same record.

In the AI runtime, agents run real tools, for example filing a ticket, and every step and tool call is traced and inspectable. When a run with several agents produces a strange result, you can open it and follow each step and tool call to the point where the work went wrong, instead of guessing from the final answer.

Autopilot runs triggered agents with approval rules. A person approves an action, in the app or through an email approve link, before it runs, which is how you keep the one step that changes something under human control while the reading and checking happen automatically.

Governance is included, not upsold: tenant isolation, role-based access, audit logs and per-tenant encrypted secrets. Tool credentials stay scoped to the tenant that owns them, however many agents call those tools.

CloudOEngine is built in Bengaluru, India.

Book a demo or see the AI runtime.

Frequently asked questions

What is a multi-agent system in AI?

A multi-agent system is an application where two or more LLM-driven agents, each with its own instructions, tools and context, work on parts of one task. The two common shapes are a manager or orchestrator that delegates to worker agents, and peers that hand control to each other.

When should you use multiple AI agents instead of one?

When the work splits into independent parts that can run in parallel without much shared context, such as broad research, or when a single agent keeps choosing the wrong tool or failing complicated instructions even after you have improved its prompt and tool descriptions. For most business workflows one agent with good tools is enough.

Are multi-agent systems more expensive to run?

Usually, yes. Anthropic reported that in its own data agents used about 4 times more tokens than chat interactions, and multi-agent systems about 15 times more. The extra spend pays off only when the task is valuable enough and splits well.

Why do multi-agent systems fail?

Research that studied over 1,600 traces from seven open-source frameworks grouped the failures into three categories: system design issues, misalignment between agents, and weak task verification. In practice most of these come from agents working on partial context and nobody checking the combined result.

What is the difference between the manager pattern and handoffs?

In the manager pattern one agent stays in control and calls specialist agents like tools, then combines their answers. With handoffs, one agent transfers the whole conversation to another, which then takes over and talks to the user directly.

Do multi-agent systems need different security controls?

They need the same controls applied per agent. Give each agent only the tools and data its part of the job requires, keep write actions with as few agents as possible, put approvals on the actions themselves, and keep one trace that covers every agent in the run.

Sources

1. How we built our multi-agent research system (Anthropic, June 2025)
2. Building effective agents (Anthropic, December 2024)
3. A practical guide to building agents (OpenAI)
4. Don't Build Multi-Agents (Cognition, June 2025)
5. Why Do Multi-Agent LLM Systems Fail? (Cemri et al., arXiv 2503.13657)
6. Agent2Agent (A2A) Protocol documentation

#ai agents #multi-agent systems #orchestration #agent architecture #llmops