When not to use an agent
Agents cost latency, money and predictability. When one call or a fixed workflow is the better answer, and the tests that justify reaching for an agent.
Start from the cost
An agent is the most flexible thing you can build from an LLM and also the most expensive to run and the hardest to predict. Anthropic’s advice is to find the simplest solution possible and increase complexity only when needed, which might mean not building an agentic system at all1. They note that agentic systems often trade latency and cost for better task performance1.
Diagram: my own and qualitative. The bars show ordering, not measurements.
Each extra autonomous step adds latency and tokens, and gives an early mistake more room to carry forward. Anthropic describes the cost of autonomy as higher spend and the potential for compounding errors1.
A decision order
Diagram: my own decision order, built from the guidance in 1 and 2.
- Try one well-prompted call first. Often a single call with retrieval and a few examples is enough1. See Prompt design.
- If the steps are known, write a workflow. Anthropic says workflows give predictability and consistency for well-defined tasks, whereas agents suit cases that need flexibility and model-driven decisions at scale1. Patterns are in Workflows vs agents.
- Use an agent when you cannot hardcode the path. Anthropic describes agents as fitting open-ended problems where it is hard to predict the number of steps, and where you cannot fix the path in advance. They also require some trust in the model’s decisions1.
- Only if you can verify progress. An agent that cannot check its work is guessing at length. Tests, tool results and state checks are what let it correct itself.
Where agents do earn their place
OpenAI’s guide suggests looking at the workflows where conventional automation struggles. It names three kinds of candidate: tasks needing complex decision-making with judgement and exceptions, such as refund approval; systems with rule sets so large and tangled they are hard to maintain, such as vendor security reviews; and work that leans on unstructured data like documents or conversations2. Where none of that applies, a deterministic solution may do the job2.
Anthropic points to customer support and coding as two areas where agents have proved valuable in practice1. In both, the path varies from case to case and the results can be checked, through a resolved ticket or passing tests.
Signs you picked too much agent
- The steps never change between runs. That is a workflow.
- You cannot say what “done” looks like, so you cannot test it.
- Cost per task is high and latency is user-visible, but one call would have worked.
- Mistakes are expensive or irreversible, and no human approves them. Raise the guard, or lower the autonomy level.
- You are debugging the framework instead of the task. Anthropic advises starting with direct API calls, since many patterns need only a few lines, and understanding any framework code you adopt1.
If you do build one
Bound it (Stopping conditions), instrument it (Tracing and monitoring), and measure it (Agent evals) before widening what it may do. Choosing a pattern continues from here.
Sources
- [1]Building effective agents — Erik Schluntz and Barry Zhang · Anthropic · 2024PARAPHRASED, SHORT QUOTE
- [2]A practical guide to building agents — OpenAI · 2025PARAPHRASED
- [3]Levels of Autonomy for AI Agents — K. J. Kevin Feng, David W. McDonald, Amy X. Zhang · arXiv · 2025FURTHER READING