NOTES / HARNESSES

Case study: coding agents

How the harness ideas come together in a coding agent: the gather, act, verify loop, the tools it needs, why tests make good feedback, and where such agents struggle.

UPDATED 18 NOV 2026#harness#coding-agents#claude-code#swe-agent

Coding is the best-understood agent use case, so it is a good place to see every idea in this section working together.

The loop: gather, act, verify

Claude Code’s documentation describes its work as three phases that blend together: gather context, take action and verify results1. Each tool use gives the model new information that informs the next step, and you can interrupt at any point1. A typical bug fix reads as a story: run the tests and see the failure, search and read the relevant files, edit them, run the tests again1.

This is the execution loop with a very specific set of tools.

Diagram: a task goes to the model, which uses tools to read and search files, edit files, run shell and tests, and use git, with test results as feedback

Diagram: my own.

The tools that matter

  • Search and read files: the “gather” phase. Good search (by name, by content, by symbol) saves context compared with reading whole files.
  • Edit files: precise edits rather than rewriting whole files.
  • Run shell commands: builds, linters, tests, scripts.
  • Git: versioning gives checkpoints you can roll back to (see State and checkpoints).

Why the interface matters

The SWE-agent paper showed this clearly. Raw shell access was a poor fit for a language model; adding an agent-computer interface with commands designed for the model improved its ability to create and edit files, navigate repositories and run tests2. The paper reports that the agent with the custom interface solved 10.7 percentage points more instances than a baseline using only the default Linux shell2. The takeaway matches Tool design: shape what the model sees and does. Guarded edits that reject syntactically broken changes, concise search output and clear errors all matter.

Tests are the verifier

Coding has an advantage most domains lack: a cheap, objective check. Running the test suite tells the agent whether it made progress. Anthropic’s long-running agent work leans on this. The agent is asked to test as a human would and to treat a feature as done only after verification, and a feature list is used to define what “done” means, so the agent cannot claim success early3. When you build agents for other domains, ask what plays the role of the tests, and if there is nothing, build it: that is what evals are for.

Extending the harness

Coding agents are where many harness features were refined:

Failure modes

Anthropic describes two recurring failures in long coding tasks: trying to do everything in one go and running out of context mid-feature, and declaring the job finished too early3. Others to watch for: editing code without reading enough of it, getting stuck repeating a failing command, and changing tests to make them pass. Each suggests a harness fix rather than a longer prompt: smaller units of work, a hook that flags edits to test files, a turn limit.

Autonomy is a dial

Anthropic’s general advice is that agents suit open-ended problems where you cannot predict the steps, and that you should keep a human in the loop for consequential actions, with testing in sandboxed environments and appropriate guardrails4. For coding that translates to read-only by default, approval for writes or risky commands, and a sandbox for anything autonomous. See Levels of autonomy.

Sources

  1. [1]How Claude Code works — Anthropic, Claude Code documentationPARAPHRASED
  2. [2]SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig and others · NeurIPS 2024 (arXiv:2405.15793) · 2024PARAPHRASED
  3. [3]Effective harnesses for long-running agents — Justin Young and others · Anthropic Engineering · 2025PARAPHRASED
  4. [4]Building effective agents — Erik Schluntz and Barry Zhang · Anthropic · 2024PARAPHRASED

© Ashish Kumar. All rights reserved. Quoted and cited material belongs to its owners. How I attribute sources.

Product, company and framework names are trademarks of their respective owners. Mentioning them does not imply affiliation or endorsement.