Guide · Continuous evaluation

Continuous evaluation

An agent that passed evaluation in March is not an agent that works in September. The model underneath it shipped two new versions, the corpus grew, and users started asking things nobody anticipated. Nothing broke. It just stopped being the same system — and a suite frozen at launch measures a distribution that no longer exists.

ProductionDrift DetectionSamplingRegressionAgent SLOsOpenTelemetry
01 The difference

Not the same job on a cron.

Pre-release evaluation asks should we ship this. Continuous evaluation asks is the thing we shipped still working.

The conditions invert, and that changes the method rather than the schedule. Before launch you control the inputs: a curated suite, known-good outcomes, every case scored, cost bounded by the size of the suite. In production you control none of it. Traffic is open-ended, most runs have no reference answer, and scoring everything at real volume costs more than the agent does.

The consequence is that continuous evaluation is fundamentally a sampling and prioritisation problem, not a scheduling one. Running the launch suite nightly answers a question nobody is asking — whether the agent still handles cases you already know it handles.

A nightly run of your launch suite tells you the past still works. It is silent about the present.
02 Economics

What to check on everything, and what to sample.

The design decision that makes production evaluation affordable is sorting checks by cost, then applying each at the widest scope it can afford. Three tiers:

100% of runs

Deterministic checks

Forbidden tools, schema conformance, budget ceilings, loop detection, citation IDs resolving to something retrieved. These are code. They are effectively free, they are exact, and there is no reason not to run them on every single run.

Sampled

Model-judged checks

Groundedness, completeness, semantic tool-argument correctness. Expensive and non-deterministic. Run on a stratified sample — by task type, by user segment, by novelty — so the sample stays representative as traffic shifts.

Triggered

Full investigation

Everything, on runs already flagged: a tripped deterministic check, an outlier cost, a user complaint, an unusual path. Cheap in aggregate because the population is small and self-selecting.

The tiering matters more than the sample rate. A team scoring 2% of runs on everything learns less than a team running exact checks on 100% and deep checks on 2% — because the exact checks are what select the 2% worth looking at closely.

03 Drift

Four things move, and only one is your deploy.

The model moves

Providers update models continuously, and behaviour changes without a version bump you controlled. This is the drift teams are least prepared for, because nothing in their own change log explains it.

The corpus moves

Documents get added, superseded, reorganised. Retrieval quality changes underneath an agent whose prompt and code are byte-identical to last week's.

The traffic moves

Users learn what the agent is good at and ask harder things; a new team onboards with different questions. The input distribution shifts away from anything the launch suite represented.

The agent moves

Prompt edits, new tools, orchestration changes. The only one with a commit attached — and therefore the only one most teams actually test.

Three of the four leave no trace in your repository. That is the argument for continuous measurement in one sentence: most of what changes your agent's behaviour is not a change you made.

04 Comparability

Drift is only visible if the criteria held still.

Detecting drift means comparing this week's scores to last week's and attributing the difference to the agent rather than to the measurement. That requires the criteria to be fixed, versioned, and identical across both windows. If the rubric was edited in between, the comparison is meaningless and — worse — it will look like a finding.

This is the practical case for a written contract, beyond governance:

  • Versioned criteria make time series legitimate. A score change means the agent changed, because the ruler didn't.
  • Criteria changes become explicit events on the same timeline, so a step change has a cause you can point at rather than argue about.
  • Nobody re-litigates the bar during an incident. The argument about what "good" means happened once, in advance, when nothing was on fire.
Watch the variance

A stable mean can hide a widening spread.

The most useful early signal is usually not the average moving — it is the spread widening while the average holds. That is an agent becoming less predictable before it becomes less correct, and it is invisible to any dashboard that plots means. Track percentiles, and track per-category rates: an aggregate score can sit perfectly still while one task type collapses and another improves.

05 Closing the loop

An evaluation nobody acts on is a cost centre.

The failure mode of continuous evaluation is not technical. It is a dashboard that goes yellow, that nobody owns, that everyone learns to ignore inside a month. Three things prevent it:

  • Findings are specific. Not "groundedness declined 4%" but "these eleven runs made claims their retrieved context does not support, here they are, here are the spans." A number is not actionable; a list of runs is.
  • Failures route to the fix, not the dashboard. Thin telemetry is instrumentation work. Ungrounded claims are retrieval or prompt work. Wrong tool arguments are schema or description work. Different owners, different queues.
  • Findings feed the suite. Every real production failure becomes a case in the pre-release suite, so the same failure has to be novel to happen twice. This is the loop that makes the system get cheaper over time instead of more expensive.
06 Where it leads

Clearance expires. This is what renews it.

A clearance is a decision made at a moment, against a scope, on evidence available then. Since three of the four drift sources have nothing to do with your deploys, that decision has a shelf life whether or not anyone assigned it one.

Continuous evaluation is what makes renewal cheap. If criteria are already written, evidence is already attached, and failures are already named, then re-clearing an agent is reading a current report — not re-running a project.

Know when it stops working.

Agent TrustKit scores production runs against a contract that holds still, so a change in the score means a change in the agent.

Book a call [email protected]Next case study Get started