Case study: coding agents
How the harness ideas come together in a coding agent: the gather, act, verify loop, the tools it needs, why tests make good feedback, and where such agents struggle.
Coding is the best-understood agent use case, so it is a good place to see every idea in this section working together.
The loop: gather, act, verify
Claude Code’s documentation describes its work as three phases that blend together: gather context, take action and verify results1. Each tool use gives the model new information that informs the next step, and you can interrupt at any point1. A typical bug fix reads as a story: run the tests and see the failure, search and read the relevant files, edit them, run the tests again1.
This is the execution loop with a very specific set of tools.
Diagram: my own.
The tools that matter
- Search and read files: the “gather” phase. Good search (by name, by content, by symbol) saves context compared with reading whole files.
- Edit files: precise edits rather than rewriting whole files.
- Run shell commands: builds, linters, tests, scripts.
- Git: versioning gives checkpoints you can roll back to (see State and checkpoints).
Why the interface matters
The SWE-agent paper showed this clearly. Raw shell access was a poor fit for a language model; adding an agent-computer interface with commands designed for the model improved its ability to create and edit files, navigate repositories and run tests2. The paper reports that the agent with the custom interface solved 10.7 percentage points more instances than a baseline using only the default Linux shell2. The takeaway matches Tool design: shape what the model sees and does. Guarded edits that reject syntactically broken changes, concise search output and clear errors all matter.
Tests are the verifier
Coding has an advantage most domains lack: a cheap, objective check. Running the test suite tells the agent whether it made progress. Anthropic’s long-running agent work leans on this. The agent is asked to test as a human would and to treat a feature as done only after verification, and a feature list is used to define what “done” means, so the agent cannot claim success early3. When you build agents for other domains, ask what plays the role of the tests, and if there is nothing, build it: that is what evals are for.
Extending the harness
Coding agents are where many harness features were refined:
- Hooks run formatters and block risky commands;
- subagents isolate noisy exploration;
- MCP connects issue trackers and databases;
- project instruction files hold conventions the model should always know (see Context engineering);
- permissions and sandboxing limit what a mistaken command can do.
Failure modes
Anthropic describes two recurring failures in long coding tasks: trying to do everything in one go and running out of context mid-feature, and declaring the job finished too early3. Others to watch for: editing code without reading enough of it, getting stuck repeating a failing command, and changing tests to make them pass. Each suggests a harness fix rather than a longer prompt: smaller units of work, a hook that flags edits to test files, a turn limit.
Autonomy is a dial
Anthropic’s general advice is that agents suit open-ended problems where you cannot predict the steps, and that you should keep a human in the loop for consequential actions, with testing in sandboxed environments and appropriate guardrails4. For coding that translates to read-only by default, approval for writes or risky commands, and a sandbox for anything autonomous. See Levels of autonomy.
Sources
- [1]How Claude Code works — Anthropic, Claude Code documentationPARAPHRASED
- [2]SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig and others · NeurIPS 2024 (arXiv:2405.15793) · 2024PARAPHRASED
- [3]Effective harnesses for long-running agents — Justin Young and others · Anthropic Engineering · 2025PARAPHRASED
- [4]Building effective agents — Erik Schluntz and Barry Zhang · Anthropic · 2024PARAPHRASED