I expected a team of specialized agents to outperform a single one. 🤖
On paper, the architecture looked sensible: an orchestrator routes work to specialists, each with its own role and model.
Then I ran the evaluation.
Same quality, more coordination
Across 1,499 ecommerce cases, the two architectures reached comparable quality. The distributed approach, however, took longer and consumed more tokens. It also introduced more routing, context handoffs, and places for things to fail.
Using a cheaper model for specialists did lower model cost in this configuration. But that saving came from model selection, not from multiplying agents.
Delegation did not produce a meaningful quality gain in this setting. It mostly added 21.7 seconds of latency.
When distribution earns its keep
This does not mean agents should never be distributed. Separation becomes valuable when it preserves real independence: ownership, deployment, security, state, or context isolation.
Otherwise, the specialist you think you need may simply be a tool.
We already know this trade-off from monoliths and microservices: splitting a system does not automatically create value. It creates a coordination cost first.
The decision criterion
The useful question is not: “Would several agents look cleaner on the diagram?” It is: “What meaningful separation does this new agent preserve?”
A practical decision guide
Before adding a specialist, work through three checks:
- Name the boundary. State exactly what must remain independent: ownership, deployment, security, state, or context.
- Try a tool first. If the capability can stay within one context and deployment boundary, a tool is usually the simpler fit.
- Measure coordination. Compare quality, latency, tokens, and failure modes before turning a split into an architectural default.
That keeps the architectural decision tied to evidence rather than to the appeal of a cleaner diagram.
The methodology, results, and the decision guide behind this comparison.