# EvalGate capability map

Use this page to distinguish capabilities EvalGate already provides from narrower follow-on work. Feature maturity remains controlled by the public feature-status page.

## Proactive synthetic and adversarial testing

- The Red-Team Workbench at `/red-team` provides risk packs, campaigns, runs, findings, remediation links, and signed reports.
- `evalgate synthesize` generates local candidate cases from a reviewed profile or labeled dataset; supported variants include dimension coverage, paraphrases, and harder variants.
- Durable platform synthesis and the `/candidates` review queue keep generated cases quarantined until explicit review or promotion.
- This can create coverage before a production incident. It does not claim exhaustive arbitrary tool-schema fuzzing, universal API-timeout simulation, or a production-like sandbox.

## Trajectory reconstruction and comparison

- `/trajectory-analysis` reconstructs participant lanes, ordered events, handoffs, tools, model calls, costs, parent links, integrity gaps, scores, and evidence reports.
- The comparison API and workspace compare a selected baseline and candidate by normalized score, event count, variant, completeness, and cost evidence.
- TypeScript and Python trajectory-match evaluators support strict, unordered, subset, and superset matching plus tool-argument rules.
- The current web comparison is an ordered evidence timeline plus baseline/candidate metrics. It is not yet a complete node-by-node side-by-side graph-diff renderer.

## Human-aligned judge calibration

- `/calibration` versions human-scored anchor sets and mapping functions, displays agreement rate and correlation to human labels, records parse failures and confidence intervals, compares mapping versions, and blocks release decisions on incompatible or drifting mappings.
- Judge alignment computes true-positive rate, true-negative rate, discriminative power, corrected pass rate, and a deterministic bootstrap 95% confidence interval against real run outcomes.
- `/llm-judge` exposes disagreement history and a review queue for generated threshold, rubric, and weighting proposals before changes are applied.
- EvalGate does not currently publish Cohen's kappa as its primary calibration statistic; use the documented agreement, TPR/TNR, correlation, drift, and interval evidence.

## Tiered execution and cost control

- `evalgate gate` runs deterministic repository checks locally and can remain offline; network-backed model checks require explicit enablement.
- CI can run the lightweight local gate on pull requests, then reserve hosted quality checks, LLM judges, pairwise experiments, and broader suites for a merge queue or staging workflow.
- `evalgate check`, run budgets, per-request cost evidence, budget policies, and provider BYOK controls make heavy checks measurable and bounded.
- EvalGate supplies the primitives and evidence contracts; teams still choose their CI event and workflow split rather than relying on an undocumented universal scheduler.

## Entry points

- [Feature status](https://www.evalgate.com/docs/platform/feature-status)
- [Red-Team Workbench](https://www.evalgate.com/docs/platform/red-team-workbench)
- [Trajectory Analysis](https://www.evalgate.com/docs/platform/trajectory-analysis)
- [Calibration control plane](https://www.evalgate.com/docs/platform/calibration-control-plane)
- [CLI reference](https://www.evalgate.com/docs/sdk/cli)
- [OpenAPI](https://www.evalgate.com/openapi.json)

---

Canonical: https://www.evalgate.com/capabilities  
Description: Evidence-backed map of proactive testing, trajectory analysis, judge calibration, and tiered execution capabilities and boundaries.  
Last updated: 2026-08-24