Red-Team Workbench

Red-Team Workbench is EvalGate’s organization-scoped control plane for adversarial evaluation. It turns a safety test from a collection of prompt strings into a reproducible evidence chain: reviewed risk-pack version, exact attack cases, immutable target and scorer versions, Model Gateway calls, stable findings, human decisions, remediation artifacts, a fixed-target rerun, active golden coverage, a release gate, and a signed report. Open the workbench at /red-team.

A campaign never resolves “latest.” Its snapshot stores the target artifact ID, exact version ID, SHA-256 content hash, approved risk-pack and case hashes, variants, scorer bindings, optional dataset version, budget, concurrency, and seed. Approval and execution revalidate those identities.

Permissions

Action	Required scope	Default role access
Read packs, campaigns, runs, findings, decisions, and evidence	`red_team:read`	Viewer, member, admin, owner
Generate proposed cases, create campaigns/versions, run, cancel, rerun, compare	`red_team:run`	Member, admin, owner
Review cases/packs, approve campaign versions, transition findings, link remediation, promote regressions, sign reports	`red_team:approve`	Admin and owner

API keys receive only the scopes explicitly granted to them. All database reads and writes are organization-scoped. Composite references are revalidated in the service and database, so a same-ID resource from another organization is rejected rather than disclosed.

Evidence model

Object	Immutable evidence	Mutable lifecycle fields
Risk-pack version	Taxonomy/version, pack snapshot, artifact/version IDs, content hash	Draft review to approved or rejected
Attack case	Category, technique, redacted input, expected safe behavior, indicators, severity, source, content hash, generated model-call ID	Proposed review to approved or rejected
Campaign version	Exact target/version/hash, cases, variants, scorers, dataset, budget, seed, snapshot hash	Draft approval; an older approved version may become superseded
Campaign result	Execution key, response hash/preview, scorer/model-call evidence, cost, finding fingerprint	None; append-only
Finding decision	From/to state, rationale, evidence, actor, decision time	None; append-only
Remediation link	Resource identity/version, server-verified evidence hash, actor	None; append-only
Run comparison	Baseline/candidate runs and content-hashed comparison	None; append-only

Risk packs and campaign versions are backed by the normal release-artifact/version registry. Their content remains addressable after a newer version is created.

Create an assessment from risk

The current workbench retains its governed risk-pack and campaign builders while the persisted assessment-plan contract is developed. The intended risk-first sequence begins with an exact published target and a plain-language failure mode, then reviews applicable defensive controls, evidence methods, bounded execution depth, and the complete plan. EvalGate does not expose that sequence as a primary production route until it can persist the plan through a shared API without executing provider calls or mutating the target. The control-first foundation is available in the AIDEFEND defense library. AIDEFEND is a defensive-control reference and remediation framework, not an attack engine. EvalGate owns the attack scenarios, assessment planning, immutable campaigns, evidence, findings, remediation proof, regression gates, release decisions, and reports. The working model is:

risk or failure mode → attack scenario → applicable defense → verification plan
→ immutable campaign → evidence → finding → remediation proof → regression gate

Parent families are browsable organizers and may summarize child coverage, but cannot be assessed or receive independent credit. Standalone techniques and leaf sub-techniques are actionable. A control without a reviewed EvalGate verification profile remains an evidence obligation labelled Verification design required; EvalGate does not fabricate a generic prompt. Supported methods are adversarial probe, deterministic assertion, configuration review, trace/telemetry requirement, provenance or artifact review, resilience exercise, and human evidence review. A control may require several methods, and attack resistance remains separate from benign utility.

A mapping does not prove implementation. Missing evidence is not a pass. EvalGate never presents AIDEFEND compliance, certification, approval, endorsement, partnership, or affiliation.

Advanced risk-pack implementation objects

Risk packs and campaigns remain the governed implementation objects behind the workbench. Advanced users can create a structured risk-pack draft with stable case keys, supported risk categories, explicit safe behavior and success indicators. Every framework-derived or model-generated suggestion remains proposed, editable, and human-reviewed before approval.

Generate proposed cases

POST /api/red-team/attack-cases/generate sends the reviewed objective, category, taxonomy, count, and explicit provider/model intent through the canonical Model Gateway with the synthesis use case. It accepts only strict JSON, rejects cases outside the requested category, records the model-call ID, redacts content, and stores the candidates as proposed. Generated content is never auto-approved. An admin or owner must inspect the exact input, safe behavior, indicators, severity, source, and hash before choosing Approve case or rejecting it with a rationale. The pack can be approved only after every included case is approved. Approval transitions the backing release version to a candidate and freezes the risk-pack review state.

Edit a case by creating a new risk-pack version. Reviewed case content and historical risk-pack snapshots cannot be rewritten or deleted.

Create an immutable campaign

Choose New campaign and bind:

Target type, release artifact ID, exact version ID, and content hash.
One or more approved risk-pack version IDs.
One or more execution variants with stable keys, configured Gateway models/routing profiles, and parameters.
Exact scorer artifact/version/hash bindings when scoring is required.
An optional immutable dataset-version ID.
Maximum cost, maximum model calls, concurrency, and deterministic seed.
A unique idempotency key.

The service expands the cross-product of approved cases and variants into stable execution keys. A reused campaign idempotency key returns the original campaign only when the complete immutable intent matches; different intent returns a conflict.

Approve the snapshot

Approve reruns target, pack, case, and scorer validation before storing the actor, rationale, and time. Creating a draft does not supersede the currently approved version. Only successful approval supersedes the prior approved version. The approval dialog identifies that the complete target/version/hash, attack evidence, variants, scorers, dataset, budget, concurrency, and seed are being frozen.

Execute within reviewed bounds

Choose Run on an approved campaign version. The request repeats the accepted maximum cost and concurrency. Execution is refused when:

The version is not human-approved.
Target or scorer content no longer matches the approved hash.
Expansion exceeds the reviewed model-call ceiling.
The caller does not explicitly accept the approved cost or concurrency bound.
A rerun baseline is incomplete, outside the organization, or belongs to another campaign.

Every provider/model request goes through the Model Gateway with eval_executor. The result retains the model-call ID, response hash, redacted preview, target/case/scorer hashes, provider status, cost source, and error code. Calls execute at the snapshot’s bounded concurrency. The worker stops scheduling new work when cancellation is requested, known actual cost exceeds the ceiling, or provider cost is unavailable. Unknown cost fails closed with COST_UNAVAILABLE; known overage fails with BUDGET_EXCEEDED. Provider failures remain explicit failed evidence rather than successful safe results.

Cancel, rerun, and compare

Cancel requires a human rationale. In-flight calls may finish and remain durable; no new work is scheduled after the cancellation flag is observed. Rerun uses a completed baseline from the same campaign and an approved campaign version, normally one that binds the fixed target. The workbench automatically requests a baseline/candidate comparison after a completed rerun. Comparison reports:

Baseline and candidate attack-success rate
Rate delta
Resolved fingerprints
Newly introduced fingerprints
Persisting fingerprints
Regression state

Compile release evidence

The release-admission artifact is compiled from one organization-scoped, completed campaign run. It is not a second execution path and it does not discover a run by recency. Fetch the run you intend to use:

evalgate red-team evidence --run <completed-run-id>

The command exists in both the TypeScript and Python CLIs and calls the same get_red_team_runs_id_release_evidence operation exposed by both generated clients. It validates the strict schema and canonical hash, verifies the server’s Ed25519 signature against the repository-pinned public-key fingerprint, checks the configured organization, snapshot, and exact target, then atomically writes .evalgate/red-team-evidence.json. The response contains the exact artifact/version/content-hash identity, campaign snapshot and run identity, execution completeness, provider state, attack and benign-utility outcomes, finding governance, remediation comparison, active golden coverage, signed report identity, framework references, and a content hash for the evidence envelope. It contains decision metadata rather than raw prompts or model outputs. Pin the target the gate is deciding on in repository configuration. For example:

{
  "redTeam": {
    "required": true,
    "policyProfile": "pre_release_strict",
    "exactTargetRequired": true,
    "expectedTarget": {
      "artifactId": "42",
      "versionId": "7",
      "contentHash": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
    },
    "expectedOrganizationId": "org_123",
    "expectedSnapshotHash": "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789",
    "expectedAttestationKeyId": "red-team-prod-v1",
    "expectedAttestationPublicKeySha256": "fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210"
  }
}

For a required gate, pre_release_strict is the default profile. It requires compiler attestation, a complete and fresh campaign, attributable costs, zero successful attacks, and a 100% benign-utility pass rate. It does not require a finding-scoped report, so a clean first campaign can pass. Use post_remediation_strict for a repaired finding; that profile additionally requires the baseline/fixed rerun comparison, no introduced fingerprints, active golden coverage, and an exact-hash signed report. The legacy strict name remains an alias for pre_release_strict; use custom only for a reviewed exception. Both strict lifecycle profiles require exact organization, target, campaign-snapshot, and compiler-key pins. The control plane must configure the server-only RED_TEAM_EVIDENCE_SIGNING_PRIVATE_KEY and RED_TEAM_EVIDENCE_SIGNING_KEY_ID; CI receives no signing secret. Use the artifact and version identifiers from the release registry, not a source checkout digest. A required gate fails closed when the evidence is missing, malformed, stale, cross-organization, bound to another target or snapshot, incomplete, provider-failed, cost-unattributed, or has an invalid evidence hash or compiler signature. Hand-authored JSON and a convenient “latest run” lookup are not admissible substitutes. Campaign creation and execution remain explicit human-approved actions; compiling evidence only reads durable state. In a fresh CI checkout, fetch the exact artifact before the normal gate:

evalgate red-team evidence --run "$RED_TEAM_RUN_ID" --format json
evalgate gate --format json

Findings and stable dedupe

A successful attack creates or updates a campaign-scoped finding. Its fingerprint contains organization, exact target version, case content hash, category, variant, and normalized evidence signature. The same governed failure recurs into one finding; evidence from another campaign cannot become its latest result. Each new finding result occurrence also records an explicit source state. When the campaign target is linked to Repository Intelligence, the occurrence binds the organization-owned repository, exact commit snapshot, graph version, and one or more typed locators (test_target, observed_execution, or suspected_contributor). Node identity and revision are included only when the locator is verified as evidence for that graph member. These relationships are observations, not automatic causal attribution. Legacy, inaccessible, and black-box targets remain explicitly unlinked, unavailable, or not_applicable; EvalGate never substitutes today’s repository HEAD. Read the same evidence used by the finding detail UI through either generated client or the TypeScript/Python source CLI:

evalgate red-team finding --finding <finding-id> --format json

Linked responses include commit-pinned GitHub and exact-graph navigation. The original observation remains distinct from later repeat, rerun, and remediation source contexts. Severity is rank-preserving (critical > high > medium > low) rather than lexicographic. Confidence retains the maximum supported value. High and critical findings always require an explicit owner. The finding lifecycle is:

open → triaged → confirmed → remediating → resolved
  └────────────→ false_positive
triaged/confirmed/remediating → accepted_risk
resolved/accepted_risk → remediating

Every transition requires a fresh append-only decision created after the current state began. The decision records actor, rationale, structured evidence, and from/to state in the same transaction as the state change. Historical decisions cannot be replayed after a finding is reopened. Confirming a finding creates or reuses an organization-scoped adjudication item and links it to the finding. High/critical accepted risk requires an explicit owner and a verified release-gate record.

Prove the remediation chain

A resolved finding requires seven distinct verified links:

Link	Server verification
Playground	Completed immutable Playground run snapshot in the same organization
Experiment	Completed immutable experiment snapshot and hash
Candidate case	Durable reviewed candidate with provenance and dedupe hash
Golden case	Active `active_gate` regression case in the target evaluation
Release gate	Completed organization-scoped evaluation run
Rerun	Completed rerun of the same campaign, with zero successful attacks
Signed report	Active organization-scoped shared report signed by the current report key

The reviewer supplies the durable resource ID. EvalGate reads the resource, verifies organization and lifecycle state, derives server evidence, hashes the complete evidence envelope, and appends the link. Caller-supplied metadata cannot make an incomplete resource verified.

Promote a finding to golden coverage

Enter the target evaluation ID and choose Promote. EvalGate:

Reads the latest immutable attack/result/campaign evidence.
Creates a deterministic candidate with source run, failure mode, fingerprint cluster, exact target version, deterministic generator version, and dedupe hash.
Records a normal synthetic human-review decision.
Promotes through the canonical candidate-to-golden lifecycle.
Appends candidate and active-golden remediation links.

The flow is available only for confirmed or remediating findings. It never silently auto-promotes generated attack content.

Sign the evidence report

After Playground, Experiment, Candidate, Golden, Release gate, and Rerun are verified, enter the evaluation and release-gate run IDs and choose Sign report. The report includes:

Finding ID, fingerprint, severity, and state
Exact fixed target artifact/version/content hash and campaign snapshot hash
Every remediation resource/version and evidence hash
Baseline/rerun comparison
Active golden-case and release-gate identity
Signing algorithm, key version, audience, scope, replay policy, creation time, and optional expiry

Report persistence and signing-key selection occur transactionally. The signed report becomes the seventh verified link.

Resolve

Choose Resolve, enter the decision rationale, and confirm. Both the service and database reject resolution unless all seven link types are verified and the linked campaign rerun contains zero successful attacks. Reopening clears the resolution timestamp but retains all historical decisions and evidence.

Versioned framework references and attribution

The active AIDEFEND defense-library snapshot records framework/data version, public schema version, generation time, source URL and commit, SHA-256 source digest, computed parent/standalone/leaf/actionable/total counts, license, attribution, and the EvalGate change notice. A reviewed framework-derived risk-pack case also persists the complete control reference, including canonical ID/title, hierarchy, relationship, mapping source and rationale, and review state, in that immutable risk-pack snapshot. Persisting that edition-qualified reference through assessment plans, campaign results, findings, remediation, regression gates, and signed reports remains tracked platform work. The current catalog does not pin migration-registry identity or digest, and the current OWASP and MITRE mappings remain derived display strings rather than reviewed structured references. Exact-target control-evidence states and semantic edition migration therefore remain intended assessment-contract behavior, not an end-to-end persistence claim. AIDEFEND AI Defense Framework, created by Edward Lee, https://aidefend.net, licensed under CC BY 4.0. EvalGate classifies catalog nodes, summarizes purposes, and adds reviewed mappings and verification metadata; these are EvalGate modifications and adaptations. No logo or badge is used.

API reference

Method	Endpoint	Purpose
`GET`	`/api/red-team/workspace`	Read packs, campaigns, versions, runs, and findings
`GET`, `POST`	`/api/red-team/risk-packs`	List or create versioned risk packs
`POST`	`/api/red-team/attack-cases/generate`	Generate Model Gateway proposals
`POST`	`/api/red-team/attack-cases/{id}/review`	Approve or reject one proposed case
`POST`	`/api/red-team/risk-packs/{id}/review`	Approve or reject a pack version
`POST`	`/api/red-team/campaigns`	Create campaign and immutable draft version
`POST`	`/api/red-team/campaigns/{id}/versions`	Create a new immutable draft
`POST`	`/api/red-team/campaigns/{id}/versions/{versionId}/approve`	Revalidate and approve a snapshot
`POST`	`/api/red-team/campaigns/{id}/runs`	Execute or rerun through Model Gateway
`GET`	`/api/red-team/runs/{id}`	Read run and append-only results
`GET`	`/api/red-team/runs/{id}/release-evidence`	Compile canonical release-admission evidence for one run
`POST`	`/api/red-team/runs/{id}/cancel`	Request attributable cancellation
`POST`	`/api/red-team/runs/{id}/compare`	Persist baseline/candidate comparison
`GET`	`/api/red-team/findings/{id}`	Read finding evidence and decisions
`POST`	`/api/red-team/findings/{id}/transition`	Append a human decision and change state
`POST`	`/api/red-team/findings/{id}/remediation-links`	Verify and append downstream evidence
`POST`	`/api/red-team/findings/{id}/promote-regression`	Create reviewed candidate and golden coverage
`POST`	`/api/red-team/findings/{id}/signed-report`	Sign the completed remediation evidence

Failure behavior

Condition	Response or run state
Referenced resource is missing or outside the organization	`404 NOT_FOUND` without cross-org disclosure
Invalid body, category, lifecycle, or evidence type	`400 VALIDATION_ERROR`
Idempotency key reused for different intent	`409 CONFLICT`
Target/scorer hash changed before approval or execution	`409 CONFLICT` with stale-target detail
Model-call plan or actual cost exceeds reviewed budget	`QUOTA_EXCEEDED` before start or failed run with `BUDGET_EXCEEDED`
Provider omits cost	Failed run with `COST_UNAVAILABLE`
Gateway blocks/fails a call	Failed run with `PROVIDER_FAILURE` and model-call evidence
Finding decision is stale or transition invalid	`409 CONFLICT` or database rejection
Remediation chain is incomplete	`409 CONFLICT` with missing evidence list
Severe accepted risk lacks release gate or owner	`409 CONFLICT` / validation rejection

Troubleshooting

If campaign approval reports stale evidence, reload the exact target, pack, cases, and scorers; create a new campaign version for intentional changes.
If execution is denied by budget, reduce case/variant expansion or approve a new campaign version with the intended bounded ceiling.
If a run fails with unknown cost, inspect the Model Gateway ledger and configure pricing before rerunning. EvalGate does not treat unknown cost as zero.
If promotion fails, confirm the finding and verify that the target evaluation belongs to the same organization.
If signing fails, verify all six prerequisite links, the active golden lifecycle, and that the evaluation run exactly matches the release-gate link.
If resolution is blocked, inspect the seven remediation cards and the candidate rerun’s successful-attack count.

The implementation is covered by red-team contract/unit tests, route tests, real-Postgres immutability and organization-lineage tests, DOM interaction tests, and a non-skipped authenticated golden browser path under tests/unit/red-team, tests/api/red-team, tests/integration/red-team, tests/dom/red-team, and e2e/red-team-golden-path.spec.ts.

Get Started

Core Concepts

Guides

SDK Reference

Platform

Red-Team Workbench

Red-Team Workbench

Permissions

Evidence model

Create an assessment from risk

Advanced risk-pack implementation objects

Generate proposed cases

Create an immutable campaign

Approve the snapshot

Execute within reviewed bounds

Cancel, rerun, and compare

Compile release evidence

Findings and stable dedupe

Prove the remediation chain

Promote a finding to golden coverage

Sign the evidence report

Resolve

Versioned framework references and attribution

API reference

Failure behavior

Troubleshooting

​Red-Team Workbench

​Permissions

​Evidence model

​Create an assessment from risk

​Advanced risk-pack implementation objects

​Generate proposed cases

​Create an immutable campaign

​Approve the snapshot

​Execute within reviewed bounds

​Cancel, rerun, and compare

​Compile release evidence

​Findings and stable dedupe

​Prove the remediation chain

​Promote a finding to golden coverage

​Sign the evidence report

​Resolve

​Versioned framework references and attribution

​API reference

​Failure behavior

​Troubleshooting

Red-Team Workbench

Permissions

Evidence model

Create an assessment from risk

Advanced risk-pack implementation objects

Generate proposed cases

Create an immutable campaign

Approve the snapshot

Execute within reviewed bounds

Cancel, rerun, and compare

Compile release evidence

Findings and stable dedupe

Prove the remediation chain

Promote a finding to golden coverage

Sign the evidence report

Resolve

Versioned framework references and attribution

API reference

Failure behavior

Troubleshooting