dev, staging, or prod) and keeps an exact serving version, policy snapshot, rollout assignment, revision, and evidence history.
Use this control plane when an asset has passed offline evaluation and must serve application traffic without losing the connection between the request, release version, model-gateway call, cost, policy decision, and rollout state.
Supported assets
The same deployment contract applies to five release kinds:
All deployable versions require a SHA-256 content hash and object input/output schemas. A deployment never resolves a mutable “latest” alias.
Permissions
API keys can invoke only when their organization and scopes match the deployment. An approval in one environment never authorizes another environment.
Prepare a release version
A release version payload contains a runtime and JSON object schemas:Approve and deploy
First approve the exact version for the target environment:Invoke an exact deployment
idempotencyKey is unique within the deployment. Repeating it returns the original invocation instead of creating a second gateway call or charge. requestKey controls deterministic canary assignment; use a stable customer, conversation, or session key when cohort stickiness matters.
When stream is true, the response contains ordered partial evidence:
version: exact artifact version and cohort chosen before execution.output: schema-validated serving output.evidence: the model-gateway call identifier.
- organization, deployment, artifact, exact version, environment, and cohort;
- input and output hashes plus the schema-validated values;
- gateway model-call ID, request/response hashes, latency, and token usage;
- reported or explicitly unavailable cost source plus the budget reservation;
- deployment and gateway policy snapshots;
- shadow version, model call, and output hash when shadowing is enabled.
Policy enforcement order
Before gateway execution, EvalGate verifies:- caller authentication, organization, and
runs:writescope; - deployment status and immutable environment/version binding;
- idempotency replay;
- serialized input size and allowed data classification;
- per-minute request count;
- atomic daily budget reservation;
- input schema.
Canary rollout
Approve the candidate for the same environment, then assign a bounded percentage:Shadow rollout
Shadow mode evaluates the candidate for every serving request but never changes the serving output:Health evidence and promotion
Health windows are cohort- and version-bound. Submit the same interval for serving and canary cohorts with invocation volume, success, quality, p95 latency, cost, and policy violations.Pause, resume, and rollback
Every control-plane transition includesexpectedRevision. If another operator changes the deployment first, EvalGate returns a conflict and requires a refresh instead of overwriting the newer decision.
Pause immediately blocks new invocations. Resume preserves the current exact active version. Rollback swaps the active and recorded previous versions inside one database transaction and clears in-progress canary assignment:
Operator workspace
Open Deployable Assets from the EvalGate navigation to:- compare active, canary, shadow, and paused environments;
- inspect immutable hashes and policy budgets;
- approve and deploy an exact version;
- configure canary or shadow rollout;
- compare quality, reliability, latency, cost, and violations by cohort;
- inspect exact invocation and model-call provenance;
- promote a healthy canary, pause/resume traffic, or confirm an atomic rollback;
- follow the revisioned rollout history and rationale.