# Why Aggregate AI Eval Scores Hide Regressions

A candidate can raise its aggregate score while making a critical behavior
worse. If the overall score moves from 0.82 to 0.86 while a protected
tool-permission slice falls from 0.96 to 0.69, the release is a regression
regardless of the average.

## Why averages hide failures

A large gain in a common behavior can outweigh a severe decline in a smaller,
high-risk group. Common protected slices include privileged tool calls,
refusals, escalation, long-context trajectories, and provider-failure handling.

## Define protected slices before the run

A protected slice is a named subset with its own acceptance rule. Choose it
because failure would be materially harmful, not because the candidate already
performed poorly.

Use a stable decision order:

1. Reject invalid execution as inconclusive.
2. Block confirmed protected-slice regressions.
3. Surface quality, reliability, latency, and cost tradeoffs.
4. Promote only when the intended gain clears policy without a blocker.

Every report should cover quality, protected slices, reliability, latency, and
cost. An explicit `not_measured` value is better than a fabricated metric.

Read the [experiment guide](https://www.evalgate.com/docs/platform/experiments)
and [CI integration guide](https://www.evalgate.com/docs/guides/cicd-integration).