Agent orchestration, the hard parts

An agent framework gives you a loop. Production gives you everything the loop ignores: runs that succeed once and fail the next time, side effects that happen twice, contexts that grow until they cost more than the answer, and tool output that turns into instructions. This note is about those parts.

Measure reliability as pass^k, not pass@1

Benchmarks usually report pass@1, or pass@k, where a task counts as solved if any of k attempts succeeds [1]. Users don’t experience pass@k. They experience the same agent on the same kind of task, again and again. τ-bench made this explicit with pass^k, the probability that all k trials of a task succeed. In its retail domain, state-of-the-art function-calling agents such as GPT-4o solved fewer than half the tasks, and pass^8 fell below 25% [2].

The non-obvious part: pass^k depends on how failures are distributed across tasks, not only on the average.

System A looks fine on a leaderboard and is unusable in a workflow, because no task is dependable. System B is honest about what it can’t do, and you can route around it. When you improve an agent, check whether pass^k moved, not only pass@1. METR’s time-horizon work makes the same point from another angle: the length of task frontier agents complete at 50% success has doubled roughly every seven months since 2019, and the horizon at higher reliability thresholds is much shorter [3]. “Can do” and “reliably does” are different curves.

Plan as a graph, execute in parallel

ReAct interleaves reasoning and tool calls in one loop [4]. It is flexible and pays for it: every observation goes back through the model, and every step waits for the one before. ReWOO separates planning from observation: a planner writes the whole plan with placeholders, workers fill them, and a solver composes the answer, which cuts token use substantially [5]. LLMCompiler compiles the plan into a dependency graph of function calls and runs them in parallel as soon as their inputs are ready [6].

Planning up front has a second benefit that matters more than speed. If the plan is fixed before any untrusted data is read, a malicious tool result can’t add steps to it. Plan-then-execute is one of the core patterns in the prompt-injection design work below [7]. Re-planning is still allowed, but it becomes an explicit, logged transition rather than something a web page can trigger.

Treat the context window as a cache

For long-running agents the prompt is mostly input. Manus reported about 100 input tokens per output token and called KV-cache hit rate the single most important metric for a production agent; with Claude Sonnet at the time, cached input cost $0.30 per million tokens against $3 uncached [8]. Cache hits need byte-identical prefixes, which turns some innocent habits into expensive bugs.

Three more Manus lessons are worth stealing. Use the file system as unbounded, restorable memory: drop a page’s content from context but keep its URL or path. Leave failed actions and stack traces in context, because erasing them removes the evidence the model needs to avoid repeating the mistake. And have the agent rewrite its task list near the end of the context, which keeps the global goal where models attend most reliably [8, 9].

Temperature zero is not reproducibility

The usual explanation for nondeterminism at temperature 0 is floating-point arithmetic under concurrency. Thinking Machines showed the bigger cause in inference servers is a lack of batch invariance: kernels return slightly different numbers depending on batch size, and batch size depends on whoever else is using the server at that moment [10].

Two consequences for agents. Replay has to use recorded model outputs and tool results, not re-execution, or the debugger will show you a different run. And every eval needs repeated trials per task, which is pass^k again.

Constrain the interface, not the thinking

Constrained decoding guarantees syntactically valid output by masking tokens that would break a grammar or schema [11]. It fixes parsing, not correctness, and it has a cost: Tam et al. found a significant decline in reasoning under strict format restrictions [12]. Two patterns keep both properties. Let the model reason in free text and emit the structured result in a second, constrained step. Or put a free-text reasoning field before the answer fields in the schema, so generation order follows thought order. Then validate semantically (the ID exists, the amount is in range) and return failures to the model as tool results; one retry with the error attached fixes a large share of bad calls.

Side effects need idempotency or compensation

Retries are the right response to transient failure and dangerous around side effects. The classic answer is idempotency: key each operation so that doing it again returns the first result [13]. Derive the key from the run and the step, and store the result before acknowledging.

Some actions can’t be made idempotent, and some multi-step tasks have to be undone halfway. Databases solved this decades ago with sagas: split a long transaction into steps, give each step a compensating action, and on failure run the compensations in reverse [14]. For agents the rule becomes: every tool with a side effect declares either an idempotency key or a compensation, and the planner schedules irreversible steps, such as sending a message or charging a card, as late as possible, after everything that can fail has already succeeded.

Multi-agent: spend tokens on breadth, keep one writer

Anthropic’s research system, a lead agent delegating to parallel subagents, beat a single agent by 90.2% on their internal research eval. It also used about 15 times the tokens of a chat interaction, and on BrowseComp, token usage alone explained 80% of the variance in performance [15]. Cognition argued the other side in “Don’t Build Multi-Agents”: subagents that can’t see each other’s context make conflicting implicit decisions, so context should be shared in full and actions kept single-threaded [16].

The two positions are consistent once you separate reading from writing. Research is breadth-first reading: independent subtasks, results merged by a lead, no conflicting side effects. Coding and operations are writing: every action carries decisions the next one depends on. Fan out to read; keep a single writer for decisions and side effects.

That taxonomy comes from Cemri et al. [17]. The uncomfortable finding is how many failures are organisational, not model errors: unclear roles, missing verification, information one agent had and never passed on.

Security: break the lethal trifecta

Simon Willison’s framing is the most useful threat model I know for agents: an agent that has access to private data, is exposed to untrusted content, and can communicate externally can be tricked into exfiltrating that data [18]. Remove any one of the three and that class of attack closes.

When you can’t remove one, design for it. AgentDojo benchmarks agents against realistic injection attacks [19]. CaMeL separates a privileged model that plans from a quarantined model that parses untrusted data, while an interpreter tracks where every value came from and enforces which values may flow into which tools [20]. Beurer-Kellner et al. catalogue patterns with provable properties: action selectors, plan-then-execute, map-reduce over untrusted inputs, dual LLMs, code-then-execute and context minimisation [7]. Each trades flexibility for a guarantee, which is the right trade wherever money, data or reputation is involved.

Workflows first, still

Anthropic’s guide draws the line between workflows, where code decides the path, and agents, where the model does, and recommends the simplest option that works [21]. Everything above is the cost of moving right on that spectrum.

An agent is a distributed system with a probabilistic component. Engineer it like one: measure pass^k, protect the cache, key every side effect, and assume tool output is hostile.

References

1. 01Chen et al., 2021. Evaluating Large Language Models Trained on Code. arXiv.
2. 02Yao et al., 2024. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv.
3. 03Kwa et al., 2025. Measuring AI Ability to Complete Long Tasks. METR.
4. 04Yao et al., 2023. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023.
5. 05Xu et al., 2023. ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models. arXiv.
6. 06Kim et al., 2024. An LLM Compiler for Parallel Function Calling. ICML 2024.
7. 07Beurer-Kellner et al., 2025. Design Patterns for Securing LLM Agents against Prompt Injections. arXiv.
8. 08Ji, 2025. Context Engineering for AI Agents: Lessons from Building Manus. Manus blog.
9. 09Liu et al., 2024. Lost in the Middle: How Language Models Use Long Contexts. TACL 2024.
10. 10He & Thinking Machines Lab, 2025. Defeating Nondeterminism in LLM Inference. Connectionism blog.
11. 11Willard & Louf, 2023. Efficient Guided Generation for Large Language Models. arXiv.
12. 12Tam et al., 2024. Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models. EMNLP 2024 Industry.
13. 13Helland, 2012. Idempotence Is Not a Medical Condition. ACM Queue.
14. 14Garcia-Molina & Salem, 1987. Sagas. SIGMOD 1987.
15. 15Anthropic, 2025. How We Built Our Multi-Agent Research System. Engineering blog.
16. 16Yan, 2025. Don’t Build Multi-Agents. Cognition blog.
17. 17Cemri et al., 2025. Why Do Multi-Agent LLM Systems Fail?. NeurIPS 2025 Datasets and Benchmarks.
18. 18Willison, 2025. The Lethal Trifecta for AI Agents. simonwillison.net.
19. 19Debenedetti et al., 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. NeurIPS 2024 Datasets and Benchmarks.
20. 20Debenedetti et al., 2025. Defeating Prompt Injections by Design. arXiv.
21. 21Anthropic, 2024. Building Effective Agents. Engineering blog.