NOTES / EVALS AND OBSERVABILITY
Experiments and eval-driven development
Turning scores into decisions: run versions of your agent against the same dataset, compare them case by case, fix one thing at a time, and guard against regressions.
From scores to experiments
A single set of scores tells you where you are. To know whether a change helped, you run an experiment: one version of the system, over a fixed dataset, graded by fixed evaluators. Then you run another version the same way and compare.
Diagram: my own.
Tools support this directly. Phoenix, for example, offers versioned datasets and experiments for tracking and evaluating changes to prompts, models and retrieval1. The idea is tool-independent. What matters is that only one thing varies between the two runs.
Record what changed
For each experiment, keep a record of:
- the dataset and its version;
- the agent or workflow version, including code and tool definitions;
- the model and its settings;
- the prompt version;
- the evaluators and rubric versions;
- the scores and the individual case results.
If the dataset, the agent and the judge all change at once, you can’t say what moved the number. Change one, and keep the rest fixed.
Comparing versions
Suppose you tighten the router prompt so the lab tool is chosen more reliably. The comparison might look like this.
Diagram: my own. The numbers are made up to show the idea, not measured results.
Tool selection rose a lot, arguments barely moved, and the final answer improved somewhat, which is the pattern you’d expect if the change really only touched routing. But the averages are the start of the analysis, not the end. A score tells you how often something passed; reading cases tells you why. Look at three groups: cases that got fixed, cases that got worse, and cases that didn’t change. The middle group is where regressions hide, and a net improvement can easily contain a new failure in a case that used to work.
Non-determinism matters here as well. Agents don’t always give the same output twice, so a small difference between two runs can be noise. Repeat runs where it’s cheap, and be cautious about reading much into a one-point change.
Eval-driven development
Eval-driven development means using examples and measurable criteria through the whole cycle, not as a final gate:
- Define the task, e.g. answer questions about the latest lab result.
- Define success: right patient and test, latest result identified, stated accurately.
- Create examples, including normal cases, missing data and failures.
- Run the agent.
- Inspect traces and evaluate outputs.
- Group failures by cause: wrong tool, wrong date, missing result, unsupported claim.
- Make one targeted change.
- Re-run and look for regressions.
Anthropic describes evals as a way to define expected behaviour up front and to uphold a consistent quality bar later2, and notes that teams without them can spend weeks re-testing when a new model arrives while teams with them can adopt it faster2.
A worked turn
Say you notice the agent often reports the oldest lab result as the latest.
Diagram: my own.
The trace shows records arriving in no particular order and the agent trusting the first. Rather than adding “pick the newest” to the prompt and hoping, fix it in code:
from datetime import date
def sort_newest_first(rows: list[dict]) -> list[dict]:
return sorted(rows, key=lambda r: date.fromisoformat(r["date"]), reverse=True)
Then pin the behaviour with a regression test that tries several orderings:
import itertools
ROWS = [
{"date": "2026-03-01", "value": 6.4},
{"date": "2026-04-01", "value": 5.8},
{"date": "2026-07-01", "value": 5.7},
]
def test_latest_is_selected_in_any_order():
for perm in itertools.permutations(ROWS):
assert sort_newest_first(list(perm))[0]["value"] == 5.7
This is stronger than editing a prompt after one bad answer. It targets a cause, it is checked mechanically, and it keeps protecting you afterwards. Finally, re-run the full suite, because a fix for one case can break another.
Regression suites
A regression is a change that breaks behaviour which used to work. A regression set should include important past failures, the most common user requests, edge cases, high-impact requirements, and cases touched by recent changes. Anthropic separates capability evals, which ask how well an agent can do something hard, from regression evals, which should stay near a 100% pass rate so that any drop is a signal2. Treat them differently: improving capability scores is progress, while a regression failing is a problem to stop and investigate.
Common mistakes
- Changing the agent, the dataset and the judge together.
- Tuning to the development set until it stops meaning anything; keep a held-out set.
- Celebrating an average without reading the cases that got worse.
- Fixing with a longer prompt what a line of code would fix reliably.
- Forgetting to add the new failure as a case, so it can come back.
The experiment framing and the lab-lookup examples are adapted from the course3.
Sources
- [1]Arize Phoenix documentation — Arize AI · Arize AIPARAPHRASED
- [2]Demystifying evals for AI agents — Anthropic · Anthropic Engineering · 2026PARAPHRASED
- [3]Evaluating AI Agents (short course) — John Gilhuly and Aman Khan · DeepLearning.AI, with Arize AISTUDY SOURCE; EXAMPLES, CODE AND DIAGRAMS ARE MY OWN