Why Aggregate AI Eval Scores Hide Regressions

A higher average is not the same as a safer release

Suppose a candidate raises an aggregate score from 0.82 to 0.86. That looks like an improvement—until the tool-permission slice falls from 0.96 to 0.69. If that slice protects whether an agent may perform a privileged action, the release is a regression regardless of the average.

Why averages hide important failures

Aggregate metrics weight every included case into one number. A large gain in a common, easy behavior can overwhelm a severe decline in a smaller but critical group. The math is doing exactly what you asked; the policy is incomplete.

Common hidden slices include:

• privileged tool calls and authorization boundaries;
• refusal and escalation behavior;
• languages, regions, or customer tiers;
• long-context and multi-step trajectories;
• provider failures and malformed tool responses.

Define protected slices before the candidate runs

A protected slice is a named subset with its own acceptance rule. Select it because failure would be materially harmful, not because the candidate already performed poorly. Retrofitting slices after seeing results turns a gate into an argument.

Use precedence, not vibes

A defensible release decision follows a stable order:

1. Reject invalid or incomplete execution as inconclusive.
2. Block confirmed protected-slice regressions.
3. Surface quality, reliability, latency, and cost tradeoffs.
4. Promote only when the intended gain clears policy without a blocker.

This prevents an attractive aggregate number from overriding a known failure and prevents infrastructure noise from being mislabeled as product quality.

Make the report complete even when evidence is not

Every gate should report quality, protected slices, reliability, latency, and cost. An explicit “not measured” is more useful than an invented metric because it tells the reviewer exactly where uncertainty remains.

Inspect the cases behind the slice

A slice failure is a signal to inspect evidence, not an invitation to lower the threshold. Review failed cases, trajectory deviations, tool arguments, judge disagreement, and whether the baseline and candidate were executed under comparable conditions.

Turn the policy into CI

Store the baseline immutably, run the impacted scope, and make protected slice rules part of the reviewed gate configuration. EvalGate documents this workflow in its experiment guide and CI integration guide.