Install an evaluation pack
Evaluation packs give you realistic, versioned coverage without starting from
an empty dataset. EvalGate ships first-party packs for healthcare, legal,
support, coding, and financial agents. Every pack contains multi-step
scenarios, fictional source fixtures, criterion-level rubrics, adversarial
variants, and critical-failure release rules.
If you are starting from an existing codebase, first
scan the repository, review the
evidence-linked recommendations, and then choose a pack.
Install from the web app
- Open Evaluation packs at
/eval-packs.
- Filter by domain and inspect the pack’s scenarios, evidence fixtures,
criteria, and release-gate thresholds.
- Select Install pack.
- Open the created evaluation and run it against the candidate AI change.
Installation is atomic and retry-safe. EvalGate creates a published Dataset
Hub version, immutable row snapshots, an active domain-pack evaluation,
active-gate test cases, and a snapshot binding. It does not create a partial
evaluation if installation fails.
First-party fixtures are fictional evaluation evidence. Packs help test a
system; they are not legal, clinical, or financial advice and do not prove
regulatory compliance by themselves.
Install from the CLI
The TypeScript package exposes the same workflow through client.evalPacks.
The Python package exposes it through client.eval_packs.
Interpret a result
Domain packs score each criterion with its declared deterministic, trajectory,
or semantic evaluator. Semantic criteria use the configured judge ensemble and
must satisfy both the score and agreement thresholds. A candidate fails the
release gate when a critical criterion fails, judge agreement is insufficient,
or the required scenario pass rate is missed.
Next step
Run the created evaluation, inspect failing criteria and their cited fixtures,
then use that evidence in the release decision.