Writing a decent essay in one pass, with no backspace, is nearly impossible. Not because you can’t write, but because the normal process —draft, reread, delete, check a source, reorder, ask someone to tear it apart— is forbidden by the rules of the game. That is exactly what we ask of a model in zero-shot mode: the perfect essay, first try, no undo.
That is where the whole of agent architecture comes from. A loop is nothing more than giving the model back its right to delete. And the thesis that made Andrew Ng famous in 2024 was precisely that: the design of the workflow matters more than the model running it.
Two and a half years later, the thesis still holds. The evidence it used to stand on does not.
The number worth retiring
Everyone quotes the same figure: GPT-3.5 solved 48.1 % of HumanEval zero-shot, GPT-4 reached 67 %, and GPT-3.5 wrapped in an agentic workflow hit 95.1 %. The conclusion —the model upgrade is dwarfed by the workflow upgrade— has been repeated in a thousand decks, including a few of mine.
Time to stop using it. HumanEval is saturated: 164 single-file Python functions, frontier models clustered somewhere between 90 and 98 %, and well-documented training-set contamination. Defending a 2026 argument with a ruler that no longer measures anything is the fastest way to lose a technical audience three minutes in.
What’s interesting is that the modern evidence supports the thesis better. Hold the model fixed and change only the harness —how you slice the task, which tools you expose, what you carry between steps— and the performance gap is of the same order as a model generation. Comparisons that run identical models through different orchestrations show gaps of seven points on research tasks, and broader framework comparisons report swings of up to thirty.
Translated into architecture-committee language: that variance is yours. Your model vendor doesn’t control it. You do.
The real axis isn’t agents. It’s state
Ng’s four patterns —reflection, tool use, planning, multi-agent collaboration— are still the best vocabulary available, and the five workflows Anthropic formalised (prompt chaining, routing, parallelisation, orchestrator-workers, evaluator-optimizer) are still the best grammar. But they get taught badly: as a menu of recipes you pick from.
They aren’t recipes. They are stages in the externalisation of cognition, and every one of them answers the same question: where does the state live, and who pays to move it?
- Loop. State lives in the context window. One agent inspecting and revising its own work. Compact, cheap, perfect as long as everything fits.
- Chain. State —the ordering— is externalised into your application code. Each stage gets its own prompt, model and validation. Predictable, but you have to anticipate the odd cases.
- Network. Specialised workers and an orchestrator handing out work. Here comes the first real pain: the orchestrator turns into a conversational hub receiving every worker’s full output.
- Graph. State moves outside, into a durable, queryable structure. Each worker reads the subgraph it needs and writes back new entities and relations. The orchestrator’s window stays small because the state no longer travels — it gets queried.
This is not a maturity ladder. Plenty of systems should stay a direct call or a fixed chain forever. It’s a sequence of decisions, and every step up is paid for in cost, latency, non-determinism and debugging surface.
What almost nobody was saying in 2024: context rots
If I had to name the biggest hole in that generation of agent material, this is it. The discipline that decides whether your system survives production is no longer called prompt engineering. It’s context engineering, and its working definition is uncomfortably simple: curate and maintain the smallest set of high-signal tokens at every inference.
The finding that changes everything is that context degrades before it fills up. As the token count in the window grows, the model’s ability to recall information accurately drops. You never need to hit the hard limit: each additional token simply returns less. An agent that reaches step 47 dragging the residue of steps 1 through 46 decides worse than one that starts from a curated eight-hundred-token summary.
There’s a corollary that stings in enterprise work: tool definitions are context too. One complex schema eats five hundred tokens; connect a handful of MCP servers and you’re burning tens of thousands of tokens of definitions before the model starts reasoning. Hence the practical ceiling of roughly twenty tools per agent, with measurable degradation well before that.
The three fixes 2026 has produced are the same idea in three wrappers —progressive disclosure—: load tool catalogues on demand, package repeatable workflows as skills whose body only loads when relevant, and let the agent write code that calls the tools so intermediate data never passes through its context. Reported savings run from 78 % to 98 % of input tokens. That isn’t an optimisation. It’s the difference between an agent that reasons and one that merely administers its own inventory.
The twenty-four-hour debate
In June 2025, one day apart, Cognition published Don’t Build Multi-Agents and Anthropic explained how it had built its multi-agent research system. Opposite headlines, identical underlying worry: context.
Cognition works on code: tight dependencies, coupled decisions, a single artefact. There, parallel subagents make independent calls on partial information and the result is brittle. Anthropic works on research: naturally independent threads. Their subagents get a self-contained task, an output format and a clean window; they don’t know the others exist. That isolation is the mechanism, and it buys them a 90 % improvement over the single agent on their internal research eval — at roughly fifteen times the tokens.
Both are right inside their own domain. The task picks the architecture, not the other way round: wide and shallow goes multi-agent; deep and narrow goes single agent with continuous context.
The synthesis the field has landed on is narrow and very specific: one orchestrator that keeps continuous context, spawning ephemeral read-only subagents that return compressed summaries. Swarms of agents writing in parallel over the same artefact remain fragile. If your design has several agents writing to the same thing at once, you already know where it will break.
And one criterion you’ll be glad to have in the budget conversation: multi-agent only makes sense when the value of the task justifies the overhead. A 15× token factor doesn’t pay for itself in tier-one support. It might pay for itself in due diligence.
The failures aren’t about intelligence
Since 2025 there has been an empirical taxonomy of why multi-agent systems fail, built on more than sixteen hundred annotated execution traces across seven frameworks. Fourteen failure modes in three categories: specification and design problems, inter-agent misalignment, and weak verification. Roughly four in ten failures come from bad specs, almost four from broken coordination, two from thin verification.
The authors’ conclusion is the uncomfortable one: improving the base model will not be enough to cover the taxonomy. These are not intelligence failures, they are distributed-systems engineering failures. Write verifiable specs, enforce structured communication, add independent validation, instrument everything. None of that shows up in a demo.
One figure I find decisive when choosing topology: uncoordinated multi-agent systems can amplify errors by up to 17×, while centralised architectures with a validation gate hold amplification to around 4×. That’s a quantitative argument for the orchestrator-with-a-gate over the swarm.
The graph, and why it isn’t automatic truth
The graph shows up when state has to outlive the session. It does three distinct jobs, and they’re worth keeping apart: it’s shared memory so the orchestrator doesn’t drown, it’s a grounding layer so the evaluator can say “the triple (X, works_at, Y) does not exist” instead of “this feels off”, and it’s a persistent world model for long-running loops. The agent forgets; the graph does not.
A minimal schema holds up surprisingly well: entities, claims, sources, artefacts and runs, with edges for mentions, supports, contradicts, derived_from and supersedes. The golden rule is that writes stay additive. Don’t overwrite a claim — create a new version and link it. Don’t delete contradictory evidence — keep both and attach an evaluation. That’s what makes a long-running system auditable.
Two technical points that are solved today and weren’t two years ago. First, bitemporality: serious temporal graph implementations store on every edge both when the fact was true and when the agent observed it, which handles contradictory or updated facts without losing information. Second, latency: classic GraphRAG, with community detection and LLM-precomputed summaries, is excellent over static corpora and unusable as agent memory, because every update triggers recomputation and multi-step synthesis takes tens of seconds. What works in production is hybrid retrieval — lexical search, embeddings and graph traversal, with no model calls at read time.
And now the part that rarely makes it into the architecture deck: a graph preserves errors just as efficiently as facts. Entity resolution can merge two different organisations. Extraction can attach the wrong date. A confident evaluator can mark a weak source as sufficient. Without schema validation, canonical identifiers, provenance, explicit conflict representation and periodic review, what you’ve built isn’t memory — it’s a very low-latency store of mistaken convictions.
The sentence I use as a test
When I review an agent architecture, the first thing I check is whether this sentence is true:
Every important output can be traced to a task, a plan, an artefact, a source, an evaluator decision and a bounded execution record.
When it’s false, more autonomy only adds uncertainty. When it’s true, loops, tools, plans, workers and graphs behave like composable engineering parts rather than opaque behaviour.
And there’s a layer that almost never makes it onto the diagram. If the four patterns are the vocabulary, the five workflows the grammar and the graph the language, then evals are the spelling: the boring part, the one nobody brags about mastering, and the only thing separating a readable text from one that merely looks like it says something. Ng himself keeps repeating it in his recent material: after working with many teams, the single biggest predictor of whether someone executes well isn’t technical skill, it’s their ability to sustain a disciplined process of evals and error analysis.
The road from loops to graphs isn’t a road from simple to complex. It goes from implicit state to explicit state, from volatile memory to durable memory, and from estimation to evidence. Everything else is decoration.
Main references: A. Ng, “What’s Next for AI Agentic Workflows” (Sequoia AI Ascent, 2024) and the “Agentic Design Patterns” series (The Batch, 2024) · E. Schluntz and B. Zhang, “Building Effective Agents” (Anthropic, 2024) · Anthropic, “Effective context engineering for AI agents” (2025) · W. Yan, “Don’t Build Multi-Agents” (Cognition, 2025) · Anthropic, “How we built our multi-agent research system” (2025) · M. Cemri et al., “Why Do Multi-Agent LLM Systems Fail?” — the MAST taxonomy (NeurIPS 2025) · P. Rasmussen et al., “Zep: a temporal knowledge graph architecture for agent memory” (2025).
Leave a Reply