Eval-Driven Development (Part 4)

From agentic maturity to deterministic, self-sustaining systems

Part 4: Eval-driven development

Part 3 showed how golden master tests keep AI-written code honest. That works because plain code gives the same output for the same input.

Some work should stay with the AI. Open questions, messy inputs, judgment calls. Ask the same question twice and you get two different answers, and both can be fine.

An exact-match test fails both of them. You need a different kind of test.

Who this is for

Any team running software where AI makes decisions or writes answers at runtime: a support assistant, a document reviewer, an agent that triages requests. It stands on its own, at any maturity level.

 

Animated infographic: eval-driven development. A grader pyramid of code checks, model as judge and human review, and 40 graded cases against a threshold; the change ships only when all cases pass. Golden set, ship rule, grow it.

From pass/fail to scores

A golden master asks one question: did the output match? An eval asks how good the output was, against a set of criteria, and whether the whole set of cases clears a bar.

Golden masters check for an exact match. Evals score each case and look at the whole set against a threshold.

 

Golden master (Part 3)

Eval-driven (Part 4)

What counts as right

One approved output

An approved reference plus grading criteria

How it passes

Output matches exactly

Scores across the set clear the threshold

Variation

A failure, unless masked

Expected, and graded

Who grades

An equality check

Code checks, an AI judge, a person

What it protects

Plain code

Prompts, agents, retrieval, model choice

 

The rule doesn’t change

No change ships until it’s been measured against a curated, approved set of cases. Golden masters and evals are two ways of keeping that one promise.

 


What a golden case looks like

The golden dataset is the eval version of the golden master harness. Each case has four parts.

Every golden case has an input, a reference, the criteria it’s graded on, and metadata about why it exists.

GD  Building the golden dataset

Small, approved and drawn from real life

START SMALL

A few dozen cases per capability catch most regressions. Grow toward a few hundred as the feature matures.

COVER THE SPREAD

Main intents, known edge cases and past failures. Ten different behaviors beat a hundred copies of one.

APPROVE EVERY REFERENCE

A case is only golden once someone who knows the subject has signed off the reference.

TAG THE ORIGIN

Seeded by the team, captured from real use, or added after an incident. Incident cases get the highest severity.

 

A worked example: the support assistant

Here’s a made-up but typical case. A software company runs an AI assistant that answers customer billing questions from its help-center articles. These are three of its golden cases.

G1  Refund after 45 days

Facts have to be exact

INPUT

“Can I get a refund on my annual plan? I bought it 45 days ago.”

REFERENCE

Pro-rated refund is available up to 60 days. Link to the refund policy.

CRITERIA

What gets scored:

• Correct facts (code check against the policy numbers)

• Grounded in the policy article (AI judge)

• Cites the policy link (code check)

METADATA

Behavior: refunds. Origin: incident. Severity: high.

WHY IT MATTERS

An earlier version said “no refunds after 30 days”. That cost real money in goodwill credits, so this case can never fail again.

 

G2  Angry customer, double charge

Tone counts as much as facts

INPUT

“You charged me twice this month. Fix it now.”

REFERENCE

Apologize, confirm the duplicate charge will be reversed in 5 to 7 business days, and offer a hand-off to a person.

CRITERIA

What gets scored:

• Correct timeline (code check)

• Offers a person (code check)

• Calm, respectful tone (AI judge, calibrated against human ratings)

METADATA

Behavior: billing errors. Origin: real use. Severity: high.

 

G3  Question it shouldn’t answer

Knowing when to stop

INPUT

“What will your prices be next year?”

REFERENCE

Says it can’t share future pricing and points to the sales team.

CRITERIA

What gets scored:

• Doesn’t guess or invent prices (AI judge, plus a banned-phrase check)

• Offers the right next step (code check)

METADATA

Behavior: out of scope. Origin: seeded. Severity: medium.

 

The team then rewrites the assistant’s instructions to make it friendlier, and runs the full set before shipping.

Golden case

Current version

New version

Result

G1 Refund after 45 days

0.92

0.94

Better

G2 Angry customer

0.71

0.89

Much better tone

G3 Future pricing

0.95

0.62

Starts guessing at prices

 

The average went up, and the change still doesn’t ship. One criterion fell below its bar, and a friendlier assistant that invents prices is worse than the one you have. The team fixes the instruction, runs the set again, and ships when all three clear. G3’s failing answer becomes a new golden case.

The lesson from the example

Look at every criterion. An average can hide the one that broke. Evals earn their keep when they stop a change that looks like an improvement.

 


Graders: use the cheapest one that’s reliable

Each criterion needs a grader. Some are code, some are an AI judge, and a few need a person.

The grader pyramid. Push as much as you can down to code checks, which are cheap, fast and give the same answer every time.

Criterion

Best grader

Why

Valid format, required fields

Code check

Same answer every time, costs nothing

Right tool called with the right inputs

Code check

Easy to compare against the reference

Banned content or phrases

Code check

It’s a list lookup

Grounded in the source documents

AI judge

Someone, or something, has to read the answer

Complete and helpful

AI judge

A rubric works well here

Tone in sensitive situations

AI judge, checked by people

Calibrate it before you trust it

New references and high-stakes calls

A person

Someone has to own what “right” means

 

Test the judge too

An AI judge is software, so it gets the same treatment. Pin its model version, compare its scores with a sample of human ratings, and re-check it whenever you change it. Part 2’s rule applies here as well: anything a grader does the same way every time belongs in a code check.

 

The eval flywheel

The loop from Part 3 carries over, adjusted for answers that vary.

The eval flywheel. AI does most of the work. A person agrees the fix and approves new golden cases.

  1. Monitor. Capture real conversations, ratings, corrections and escalations.
  2. Diagnose. The AI groups the failures and finds the cause: missing content, a vague instruction, a missing tool or a model limit. It shows a person the evidence.
  3. Negotiate. The person and the AI agree on the change. That might be the instructions, the content it reads, a tool, the model, or moving a stable sub-task into plain code.
  4. Implement. The AI makes the change.
  5. Evaluate. The full golden set runs. The change ships only if it improves the target without pushing any criterion below its bar.
  6. Grow the set. The failures, with approved references, become new golden cases.
  7. Release and watch. Deploy, then check that the offline gains hold up with real users.

Every failure becomes a test, and every test raises the bar. The assistant gets measurably better the more it’s used.

Evals make the L5 cost moves safe

Part 2 was about cutting cost: cheaper models, smarter routing, and moving known work into code. Every one of those moves is a quality risk unless you can measure it.

Before switching to a cheaper model, run the golden set. The answer is either a safe switch or a precise list of what would break.

Cost move

What the evals tell you

Swap to a cheaper model

Whether every criterion holds, and if not, exactly which behaviors would regress

Tune the router

Which request types a small model handles reliably, so thresholds come from data

Upgrade to a new model version

Whether it’s better on your cases, before anyone depends on it

Move a sub-task into plain code

When a sub-task scores the same way every time, it’s ready. From then on, Part 3’s golden masters protect it

 

Without evals, every saving is a bet on quality. With them, it’s a decision you can show your work on.

What to measure

Measure

What it tells you

Score per criterion, over time

Whether quality is improving or drifting

Golden set size and growth per month

Whether failures are turning into tests

Agreement between AI judges and people

Whether you can trust the automated grading

Ratings, escalations and corrections from real use

Whether offline gains show up for users

Cost per good outcome, by model

Whether the cheaper option is really cheaper

Share of sub-tasks moved into plain code

How far along the L5 path you are

 

The series in one picture

The whole series. Climb the scale, then move known work into code, and protect both sides with tests.

P1  The AI adoption maturity scale

Where each AI solution sits

MAIN POINT

Four levels by what the AI can do: Answer, Know, Act, Coordinate. Capability, cost and unpredictability all peak at L4.

USE IT TO

Place each solution and check it has the controls its level needs.

 

P2  L5, the self-improving mesh

Turn what AI learns into code

MAIN POINT

Move known work into plain code, protect it with a test harness, and send the rest to the cheapest model that can do it.

USE IT TO

Cut AI usage costs month over month without losing quality.

 

P3  Golden master testing

Keep AI-written code safe

MAIN POINT

Lock in approved behavior, let the AI diagnose and fix, and add every fix to the harness.

USE IT TO

Generate code quickly without piling up a maintenance problem.

 

P4  Eval-driven development

Keep AI behavior safe

MAIN POINT

Score answers against approved golden cases, and gate every change and every cost move on the results.

USE IT TO

Improve AI features with evidence, and catch the change that only looks like an improvement.

 

Put together, you get a system that uses AI to work out how a job should be done, uses code to do it once it’s known, and uses tests and evals to keep both honest. It gets cheaper and more predictable the longer it runs.

Where would evals have caught a problem in one of your AI features? We’re happy to help you build the first golden set.

Leave a Reply

© Copyright 2021 Distributed Logic Corp.

contact via web form  |  info@distributedlogic.com   |    (866) 229-8621