| Credential-free cloud-agent local gate | Experimental | None; repository CLI execution | None; local execution deliberately creates no hosted identity | TypeScript and Python CLIs in @evalgate/sdk@3.10.0 / current Python source expose evalgate init --local, baseline update, status --json (and --format json), and gate --format json; JSON/non-TTY init requires explicit --apply. Each CLI evaluates only its own runtime’s specs, so a Node project is not runnable by the Python CLI and the reverse also holds. No REST surface. | Packed-candidate cold start (no auth, no remote, network denied, path with spaces) and a real published-3.8.2→candidate upgrade both pass and fail closed (pnpm agent-experience:cold-start, pnpm agent-experience:upgrade); zero network attempts counted with the guard proving activation. TypeScript/Python parity is NOT complete — see DISC-20260903-003 remaining items 1–2. PyPI remains separately governed | Developer Experience |
| GitHub repository access and Repository Intelligence | Experimental | /setup → Connect a GitHub repository, then /repository-intelligence | get_integrations_github, post_integrations_github, activation preview/apply, and the repository list/scan/ask operations | Generated TypeScript, Python, and CLI REST contracts. @evalgate/sdk@3.10.0 separates durable link identity from exact-commit activation snapshots, reports commit relationships without unlinking, and adds matching login, whoami, link, status, open, and logout | Commit-graph service/route tests, DOM, workspace DOM, Unit, PostgreSQL, and focused TypeScript/Python CLI source tests; live device flow, App installation, private-repository scan/Copilot, exact-head CI, and production browser proof remain open | Integrations |
| Playground | Beta | /playgrounds, then /evaluations/{id}/playground | get_playgrounds, get/post/patch_evaluations_id_playground, plus documented winner, variant, scorer, run, proposal, case, assistant, cancellation, and export operations | No dedicated client methods; use REST | API and persistence tests; exact-head CI | Evaluation Workflows |
| Prompt Hub | Beta | /prompts and an evaluation detail page | get_prompts and the evaluation prompt/version operation family | CLI sync supports prompt pull/push; no complete convenience-client surface | Prompt route tests; exact-head CI | Evaluation Workflows |
| Scorer Studio | Beta | /scorers and /scorers/{scorerId} | /api/scorers plus version, example, test, alignment, calibration, publish, binding, and usage operations | No dedicated SDK or CLI; use the authenticated REST workflow | Scorer workflow proof and hostile-code OCI boundary proof; exact-head CI | Evaluation Workflows |
| Red-Team Workbench | Beta | /red-team | /api/red-team risk-pack, campaign, run, release-evidence, finding, remediation, and signed-report operations | Generated TypeScript and Python clients plus source CLI commands: evalgate red-team evidence --run <run-id> compiles, verifies, and atomically persists exact-run evidence; evalgate red-team finding --finding <finding-id> reads persisted source states and commit-pinned graph/locator evidence. Strict gates pin organization, campaign scope, target, and trust anchor; campaign orchestration remains an explicit REST workflow | Red-team workflow, durable compiler, route, TS/Python CLI and failure taxonomy, persisted finding-to-source DB/API/DOM proof, and gate test source pass locally; exact-head CI and authenticated browser proof remain required, and these changes are not claimed in a published package | Evaluation Workflows |
| Logs & Trace Explorer | Beta | /logs | /api/logs query, detail continuation, aggregate, saved-view, cohort, bulk-action, and audited-export operations | No dedicated TypeScript, Python, or CLI convenience workflow; use authenticated REST | Unit, API, database, DOM, and golden-path proof source; exact-head CI | Production Intelligence |
| EvalGate Copilot | Beta | /copilot and supported product surfaces | /api/copilot thread, retained-context, message, action, proposal, accept/apply, and reject operations | No dedicated SDK or CLI convenience workflow; use authenticated REST and the review UI | Unit, API, database, DOM, and golden-path proof source; exact-head CI | Evaluation Workflows |
| Remote Runners | Beta | /remote-runners | /api/remote-runners registration, credentials, lifecycle, enqueue, claim, heartbeat, stream, cancellation, lease recovery, completion, and reconciliation operations | TypeScript and Python worker SDKs implement the signed protocol; no complete operator CLI | Unit, API, PostgreSQL, DOM, and golden-path proof source; exact-head CI | Evaluation Runtime |
| Deployable Assets | Beta | /deployable-assets | /api/deployable-assets list/create, approval, transition, rollout, health, and governed invocation operations | Generated contract types only; no complete TypeScript, Python, or CLI deployment client | Unit, API, PostgreSQL, DOM, and golden-path proof source; exact-head CI | Evaluation Runtime |
| Trajectory Analysis | Beta | /trajectory-analysis | /api/trajectory-analysis ingestion, detail, scoring, comparison, report, and retention operations | TypeScript and Python clients cover ingestion, inspection, scoring, reporting, and comparison; no dedicated CLI command | Unit, API, PostgreSQL, DOM, and golden-path proof sources; exact-head CI | Evaluation Quality |
| Dashboards and monitors | Beta | /dashboards and pinned /dashboards/shared/{token} views | Metric-definition, metric-query, dashboard/version/widget/render/share, and stateful alert/event/delivery operation families | No complete TypeScript, Python, or CLI convenience workflow; use the authenticated REST contract | Unit, API, PostgreSQL, DOM, and golden-path proof source; exact-head CI | Evidence & Reporting |
| Failure Topics | Beta | /insights | Production-insights summary, topic discovery/detail/version controls, merge/split, Gateway label proposals, and governed corrective actions | No dedicated TypeScript, Python, or CLI convenience workflow; use authenticated REST | Unit, API, PostgreSQL, DOM, and golden-path proof source; exact-head CI | Production Intelligence |
| Dataset Hub | Beta | Evaluation detail: test cases and labeled cases | get_evaluations_id_test_cases, post_evaluations_id_test_cases, get_evaluations_id_labeled_cases, post_evaluations_id_labeled_cases | TypeScript and Python workflows read canonical JSONL; CLI label, cluster, and sync commands | Golden lifecycle DB tests; exact-head CI | Evaluation Workflows |
| Experiments | Beta | /evaluations/{id} run and Auto panels | get_evaluations_id_runs, post_evaluations_id_runs, and the Auto-session operation family | CLI run, auto, and report artifacts; web Auto orchestration has no dedicated convenience client | Experiment runner tests; exact-head CI | Evaluation Runtime |
| Continuous Eval Studio | Beta | /online-evals and /online-evals/{monitorId} for draft, simulation, activation, operation, and remediation | Monitor/version CRUD, simulate/activate, pause/resume, bounded backfill/cancel/resume, sample/run/health/topic/candidate reads, alert delivery, and verified remediation operations under /api/online-evals/* | No dedicated TypeScript, Python, or CLI monitor workflow; use authenticated REST | Unit, API, PostgreSQL lifecycle, DOM, and golden-path proof source; exact-head CI | Evaluation Runtime |
| Review Queue | Beta | /review | get_evaluations_id_human_review, post_evaluations_id_human_review, post_evaluations_id_human_review_reviews | CLI review supports local/gate workflows; no complete queue client | Review route tests; exact-head CI | Trust & Review |
| Synthetic and golden lifecycle | Beta | /candidates and evaluation Synthesize/Test Cases panels | get_candidates, get_candidates_id, patch_candidates_id, post_candidates_id_replay, post_candidates_id_promote, and evaluation-scoped review/promotion operations | CLI synthesize, promote, and replay; TypeScript and Python canonical golden-case support | Promotion DB tests; exact-head CI | Trust & Review |
| Model Gateway | Beta | /settings → Model Gateway | get/post_model_gateway_configs, config detail/health/rotation, routing-profile, model-sync, call-ledger, and test-call operations | No dedicated generated gateway client or CLI | Gateway ledger DB tests; exact-head CI | Gateway & Providers |
| Evals-as-Code | Beta | CLI/CI first; evaluation pages show applied results | post_evals_as_code_validate, post_evals_as_code_plan, post_evals_as_code_diff, post_evals_as_code_apply, post_evals_as_code_gate, manifest/apply/drift operations | TypeScript CLI is the primary client; Python parity remains incomplete | Rollback and drift DB tests; exact-head CI | Developer Experience |
| MCP review surface | Experimental | /api/mcp for coding/chat agents, public /api/mcp/docs for reviewed product documentation; legacy /api/mcp/tools remains documented separately | Stateless MCP 2026-07-28 product endpoint with public protocol and schema-only tool/resource discovery, authenticated tool calls, eight read-only P0 review tools, a scoped run-evidence MCP App, stateless 2025-era initialization fallback, and a separate anonymous docs server with native Markdown resources plus a docs App | RFC 9728 metadata links to first-party RFC 8414 authorization-code/PKCE and RFC 8628 device flows. Public clients use exact registered redirects; a signed-in person chooses the organization and scopes before EvalGate issues a revocable 90-day API-key token. No refresh token or production-like sandbox is offered | Source conformance, schema parity, PKCE validation, App-resource, and public-docs isolation tests; status remains experimental until the migration, exact deployment, and real-client compatibility matrix pass | Agent Native Surfaces |
| Agent Skills distribution | Experimental | npx skills add evalgate/skills, the public evalgate/skills repository, and /.well-known/agent-skills/ | One primary evaluate-ai-change decision framework plus five focused setup, gate, trace, repository, and MCP Skills; progressive references cover behavioral impact, authoring, regressions, experiments, cost, evidence, invalid runs, and the canonical nine-value decision contract with required quality, protected-slice, reliability, latency, and cost reporting | Skills invoke documented CLI/API/MCP contracts but do not enforce policy, grant credentials, expose private scoring logic, or make an unevaluated release safe | Pinned skills CLI discovery and selected-install proof for Codex, Cursor, GitHub Copilot, OpenCode, and Zed; eleven provider-neutral decision scenarios cover every classification and release decision, including a mixed bug sweep that scores decision correctness separately from reporting completeness. Default-branch promotion, hosted CI, broader multi-model trials, and skills.sh re-indexing remain pending | Agent Native Surfaces |
| Provider onboarding | Beta | /settings → Provider Keys and Model Gateway | Provider-key REST plus Model Gateway config, health-check, model-sync, and test-call operations | Generated contract types only; no dedicated onboarding convenience client or CLI | Onboarding DOM tests; exact-head CI | Gateway & Providers |
| Calibration control plane | Beta | /calibration, plus evaluation-detail and /llm-judge judge workflows | Anchor-set/version/review, mapping/review/diff/drift, workspace, release-gate, scorer-binding, and existing evaluation calibration-proposal operation families | Judge execution exists in both SDKs; no complete SDK or CLI control-plane workflow | Unit, API, PostgreSQL, DOM, and golden-path proof source; exact-head CI | Trust & Review |
| Evaluator measurement integrity | Experimental | /evaluator-validation measurement workspace, plus run summaries through existing evaluation APIs | Release/run lifecycle, coverage-aware policy sampling, automatic post-run validation, delayed outcome attribution/recomputation, canonical AB/BA judge execution, interval-bearing causal metrics, observation execution, metric reads, and measurement-bound release/Evals-as-Code gates | TypeScript and Python SDK measurement clients, including delayed outcomes; evalgate measurement lists releases/runs, inspects metrics, and executes validation runs | Measurement API and database tests; production browser proof remains release-gated | Evaluation Quality |
| Golden Release / AI regression bot | Experimental | /golden-release candidate inbox, decision header, and publication panel (flag-gated) | Sealed OTEL revisions, canonical measurement revisions, GitHub App webhook/publication/check APIs behind EVALGATE_* flags | No dedicated generated SDK convenience client yet; composite Action and sticky comment remain supported | Golden Release vertical proof; sandbox GitHub App and production browser proof outstanding | Evaluation Quality |
| Governed Full-Harness Improvement Loop | Experimental | /improvement | Ten organization-scoped /api/improvement operations: cycle list/detail, hypotheses, candidates, candidate diff, attempts, partitions, events, partition members, and one governed authorization write | Generated TypeScript and Python operations; evalgate improve covers the read path. No client can trigger a governed action — execute, shadow, canary, promote, and rollback are driven by the orchestrator | Real-execution proof (17 tests spawning real child processes), non-prompt acceptance scenario (15 tests, 120 real executions), mutation adapters, statistical decision rules, and learning compilers, OpenRouter model broker (22 tests against a real loopback HTTP server), and comparison statistics, cross-organization isolation, CLI, and workspace DOM. No released-availability claim: no exact-head CI run, no browser proof, and no live-model or live-traffic evidence is claimed. A real OpenRouter adapter exists and is proven against a loopback HTTP server, but no run against a live model has been performed: this environment’s network policy denies openrouter.ai. The default broker is a deterministic seeded behavioral model, not a language model. Scope and deferred items: ADR-009 and the operator reference | Platform Governance |
| Reports | Beta | Evaluation Export/Report actions and /reports; anonymous public links require an explicit organization-admin opt-in and default off | get_reports, post_reports, public verification, and revocation operations return a stable policy denial when public links are disabled | CLI emits regression and machine-readable reports; signed report administration has no complete convenience client | Signing keyring and default-deny policy DB/API tests; exact-head CI | Evidence & Reporting |
| Asset Portability | Beta | /portability inventories prompt, dataset, calibration, scorer, failure_memory, approval, and institutional_context; evaluations, golden cases, red-team, deployments, experiments, and eval-loop workflows are omitted | /api/portability export/import for those seven kinds | Generated contract types only; no complete TypeScript, Python, or CLI convenience client | Portability DOM tests; exact-head CI | Developer Experience |
| Deployment | Beta | CI and release workflow; no single deployment dashboard | get_releases, post_releases, plus Evals-as-Code gate/apply operations | TypeScript CLI and CI integrations are primary; Python parity remains incomplete | Golden path; exact-head CI | Developer Experience |