Red-Team Workbench

Red-Team Workbench is EvalGate’s organization-scoped control plane for adversarial evaluation. It turns a safety test from a collection of prompt strings into a reproducible evidence chain: reviewed risk-pack version, exact attack cases, immutable target and scorer versions, Model Gateway calls, stable findings, human decisions, remediation artifacts, a fixed-target rerun, active golden coverage, a release gate, and a signed report. Open the workbench at /red-team.

A campaign never resolves “latest.” Its snapshot stores the target artifact ID, exact version ID, SHA-256 content hash, approved risk-pack and case hashes, variants, scorer bindings, optional dataset version, budget, concurrency, and seed. Approval and execution revalidate those identities.

Permissions

Action	Required scope	Default role access
Read packs, campaigns, runs, findings, decisions, and evidence	`red_team:read`	Viewer, member, admin, owner
Generate proposed cases, create campaigns/versions, run, cancel, rerun, compare	`red_team:run`	Member, admin, owner
Review cases/packs, approve campaign versions, transition findings, link remediation, promote regressions, sign reports	`red_team:approve`	Admin and owner

API keys receive only the scopes explicitly granted to them. All database reads and writes are organization-scoped. Composite references are revalidated in the service and database, so a same-ID resource from another organization is rejected rather than disclosed.

Evidence model

Object	Immutable evidence	Mutable lifecycle fields
Risk-pack version	Taxonomy/version, pack snapshot, artifact/version IDs, content hash	Draft review to approved or rejected
Attack case	Category, technique, redacted input, expected safe behavior, indicators, severity, source, content hash, generated model-call ID	Proposed review to approved or rejected
Campaign version	Exact target/version/hash, cases, variants, scorers, dataset, budget, seed, snapshot hash	Draft approval; an older approved version may become superseded
Campaign result	Execution key, response hash/preview, scorer/model-call evidence, cost, finding fingerprint	None; append-only
Finding decision	From/to state, rationale, evidence, actor, decision time	None; append-only
Remediation link	Resource identity/version, server-verified evidence hash, actor	None; append-only
Run comparison	Baseline/candidate runs and content-hashed comparison	None; append-only

Risk packs and campaign versions are backed by the normal release-artifact/version registry. Their content remains addressable after a newer version is created.

Create and review a risk pack

Choose New risk pack and provide the complete governed JSON payload. Each case requires:

A stable caseKey
One supported risk category
Technique and adversarial input
Expected safe behavior
Explicit attack-success indicators
Severity
built_in or imported source

The server redacts secrets and personal-data patterns before hashing and storing case text. Every imported case starts proposed.

Generate proposed cases

POST /api/red-team/attack-cases/generate sends the reviewed objective, category, taxonomy, count, and explicit provider/model intent through the canonical Model Gateway with the synthesis use case. It accepts only strict JSON, rejects cases outside the requested category, records the model-call ID, redacts content, and stores the candidates as proposed. Generated content is never auto-approved. An admin or owner must inspect the exact input, safe behavior, indicators, severity, source, and hash before choosing Approve case or rejecting it with a rationale. The pack can be approved only after every included case is approved. Approval transitions the backing release version to a candidate and freezes the risk-pack review state.

Edit a case by creating a new risk-pack version. Reviewed case content and historical risk-pack snapshots cannot be rewritten or deleted.

Create an immutable campaign

Choose New campaign and bind:

Target type, release artifact ID, exact version ID, and content hash.
One or more approved risk-pack version IDs.
One or more execution variants with stable keys, configured Gateway models/routing profiles, and parameters.
Exact scorer artifact/version/hash bindings when scoring is required.
An optional immutable dataset-version ID.
Maximum cost, maximum model calls, concurrency, and deterministic seed.
A unique idempotency key.

The service expands the cross-product of approved cases and variants into stable execution keys. A reused campaign idempotency key returns the original campaign only when the complete immutable intent matches; different intent returns a conflict.

Approve the snapshot

Approve reruns target, pack, case, and scorer validation before storing the actor, rationale, and time. Creating a draft does not supersede the currently approved version. Only successful approval supersedes the prior approved version. The approval dialog identifies that the complete target/version/hash, attack evidence, variants, scorers, dataset, budget, concurrency, and seed are being frozen.

Execute within reviewed bounds

Choose Run on an approved campaign version. The request repeats the accepted maximum cost and concurrency. Execution is refused when:

The version is not human-approved.
Target or scorer content no longer matches the approved hash.
Expansion exceeds the reviewed model-call ceiling.
The caller does not explicitly accept the approved cost or concurrency bound.
A rerun baseline is incomplete, outside the organization, or belongs to another campaign.

Every provider/model request goes through the Model Gateway with eval_executor. The result retains the model-call ID, response hash, redacted preview, target/case/scorer hashes, provider status, cost source, and error code. Calls execute at the snapshot’s bounded concurrency. The worker stops scheduling new work when cancellation is requested, known actual cost exceeds the ceiling, or provider cost is unavailable. Unknown cost fails closed with COST_UNAVAILABLE; known overage fails with BUDGET_EXCEEDED. Provider failures remain explicit failed evidence rather than successful safe results.

Cancel, rerun, and compare

Cancel requires a human rationale. In-flight calls may finish and remain durable; no new work is scheduled after the cancellation flag is observed. Rerun uses a completed baseline from the same campaign and an approved campaign version, normally one that binds the fixed target. The workbench automatically requests a baseline/candidate comparison after a completed rerun. Comparison reports:

Baseline and candidate attack-success rate
Rate delta
Resolved fingerprints
Newly introduced fingerprints
Persisting fingerprints
Regression state

Findings and stable dedupe

A successful attack creates or updates a campaign-scoped finding. Its fingerprint contains organization, exact target version, case content hash, category, variant, and normalized evidence signature. The same governed failure recurs into one finding; evidence from another campaign cannot become its latest result. Severity is rank-preserving (critical > high > medium > low) rather than lexicographic. Confidence retains the maximum supported value. High and critical findings always require an explicit owner. The finding lifecycle is:

open → triaged → confirmed → remediating → resolved
  └────────────→ false_positive
triaged/confirmed/remediating → accepted_risk
resolved/accepted_risk → remediating

Every transition requires a fresh append-only decision created after the current state began. The decision records actor, rationale, structured evidence, and from/to state in the same transaction as the state change. Historical decisions cannot be replayed after a finding is reopened. Confirming a finding creates or reuses an organization-scoped adjudication item and links it to the finding. High/critical accepted risk requires an explicit owner and a verified release-gate record.

Prove the remediation chain

A resolved finding requires seven distinct verified links:

Link	Server verification
Playground	Completed immutable Playground run snapshot in the same organization
Experiment	Completed immutable experiment snapshot and hash
Candidate case	Durable reviewed candidate with provenance and dedupe hash
Golden case	Active `active_gate` regression case in the target evaluation
Release gate	Completed organization-scoped evaluation run
Rerun	Completed rerun of the same campaign, with zero successful attacks
Signed report	Active organization-scoped shared report signed by the current report key

The reviewer supplies the durable resource ID. EvalGate reads the resource, verifies organization and lifecycle state, derives server evidence, hashes the complete evidence envelope, and appends the link. Caller-supplied metadata cannot make an incomplete resource verified.

Promote a finding to golden coverage

Enter the target evaluation ID and choose Promote. EvalGate:

Reads the latest immutable attack/result/campaign evidence.
Creates a deterministic candidate with source run, failure mode, fingerprint cluster, exact target version, deterministic generator version, and dedupe hash.
Records a normal synthetic human-review decision.
Promotes through the canonical candidate-to-golden lifecycle.
Appends candidate and active-golden remediation links.

The flow is available only for confirmed or remediating findings. It never silently auto-promotes generated attack content.

Sign the evidence report

After Playground, Experiment, Candidate, Golden, Release gate, and Rerun are verified, enter the evaluation and release-gate run IDs and choose Sign report. The report includes:

Finding ID, fingerprint, severity, and state
Exact fixed target artifact/version/content hash and campaign snapshot hash
Every remediation resource/version and evidence hash
Baseline/rerun comparison
Active golden-case and release-gate identity
Signing algorithm, key version, audience, scope, replay policy, creation time, and optional expiry

Report persistence and signing-key selection occur transactionally. The signed report becomes the seventh verified link.

Resolve

Choose Resolve, enter the decision rationale, and confirm. Both the service and database reject resolution unless all seven link types are verified and the linked campaign rerun contains zero successful attacks. Reopening clears the resolution timestamp but retains all historical decisions and evidence.

API reference

Method	Endpoint	Purpose
`GET`	`/api/red-team/workspace`	Read packs, campaigns, versions, runs, and findings
`GET`, `POST`	`/api/red-team/risk-packs`	List or create versioned risk packs
`POST`	`/api/red-team/attack-cases/generate`	Generate Model Gateway proposals
`POST`	`/api/red-team/attack-cases/{id}/review`	Approve or reject one proposed case
`POST`	`/api/red-team/risk-packs/{id}/review`	Approve or reject a pack version
`POST`	`/api/red-team/campaigns`	Create campaign and immutable draft version
`POST`	`/api/red-team/campaigns/{id}/versions`	Create a new immutable draft
`POST`	`/api/red-team/campaigns/{id}/versions/{versionId}/approve`	Revalidate and approve a snapshot
`POST`	`/api/red-team/campaigns/{id}/runs`	Execute or rerun through Model Gateway
`GET`	`/api/red-team/runs/{id}`	Read run and append-only results
`POST`	`/api/red-team/runs/{id}/cancel`	Request attributable cancellation
`POST`	`/api/red-team/runs/{id}/compare`	Persist baseline/candidate comparison
`GET`	`/api/red-team/findings/{id}`	Read finding evidence and decisions
`POST`	`/api/red-team/findings/{id}/transition`	Append a human decision and change state
`POST`	`/api/red-team/findings/{id}/remediation-links`	Verify and append downstream evidence
`POST`	`/api/red-team/findings/{id}/promote-regression`	Create reviewed candidate and golden coverage
`POST`	`/api/red-team/findings/{id}/signed-report`	Sign the completed remediation evidence

Failure behavior

Condition	Response or run state
Referenced resource is missing or outside the organization	`404 NOT_FOUND` without cross-org disclosure
Invalid body, category, lifecycle, or evidence type	`400 VALIDATION_ERROR`
Idempotency key reused for different intent	`409 CONFLICT`
Target/scorer hash changed before approval or execution	`409 CONFLICT` with stale-target detail
Model-call plan or actual cost exceeds reviewed budget	`QUOTA_EXCEEDED` before start or failed run with `BUDGET_EXCEEDED`
Provider omits cost	Failed run with `COST_UNAVAILABLE`
Gateway blocks/fails a call	Failed run with `PROVIDER_FAILURE` and model-call evidence
Finding decision is stale or transition invalid	`409 CONFLICT` or database rejection
Remediation chain is incomplete	`409 CONFLICT` with missing evidence list
Severe accepted risk lacks release gate or owner	`409 CONFLICT` / validation rejection

Troubleshooting

If campaign approval reports stale evidence, reload the exact target, pack, cases, and scorers; create a new campaign version for intentional changes.
If execution is denied by budget, reduce case/variant expansion or approve a new campaign version with the intended bounded ceiling.
If a run fails with unknown cost, inspect the Model Gateway ledger and configure pricing before rerunning. EvalGate does not treat unknown cost as zero.
If promotion fails, confirm the finding and verify that the target evaluation belongs to the same organization.
If signing fails, verify all six prerequisite links, the active golden lifecycle, and that the evaluation run exactly matches the release-gate link.
If resolution is blocked, inspect the seven remediation cards and the candidate rerun’s successful-attack count.

The implementation is covered by red-team contract/unit tests, route tests, real-Postgres immutability and organization-lineage tests, DOM interaction tests, and a non-skipped authenticated golden browser path under tests/unit/red-team, tests/api/red-team, tests/integration/red-team, tests/dom/red-team, and e2e/red-team-golden-path.spec.ts.

Get Started

Core Concepts

Guides

SDK Reference

Platform

Red-Team Workbench

Red-Team Workbench

Permissions

Evidence model

Create and review a risk pack

Generate proposed cases

Create an immutable campaign

Approve the snapshot

Execute within reviewed bounds

Cancel, rerun, and compare

Findings and stable dedupe

Prove the remediation chain

Promote a finding to golden coverage

Sign the evidence report

Resolve

API reference

Failure behavior

Troubleshooting

​Red-Team Workbench

​Permissions

​Evidence model

​Create and review a risk pack

​Generate proposed cases

​Create an immutable campaign

​Approve the snapshot

​Execute within reviewed bounds

​Cancel, rerun, and compare

​Findings and stable dedupe

​Prove the remediation chain

​Promote a finding to golden coverage

​Sign the evidence report

​Resolve

​API reference

​Failure behavior

​Troubleshooting

Red-Team Workbench

Permissions

Evidence model

Create and review a risk pack

Generate proposed cases

Create an immutable campaign

Approve the snapshot

Execute within reviewed bounds

Cancel, rerun, and compare

Findings and stable dedupe

Prove the remediation chain

Promote a finding to golden coverage

Sign the evidence report

Resolve

API reference

Failure behavior

Troubleshooting