> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create orchestration evals runs > Requires scopes: runs:write ## OpenAPI ````yaml /api-reference/openapi.json post /api/orchestration-evals/runs openapi: 3.1.2 info: title: EvalGate API version: 5.1.0 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/orchestration-evals/runs: post: tags: - orchestration evals summary: Create orchestration evals runs description: 'Requires scopes: runs:write' operationId: post_orchestration_evals_runs requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostApiOrchestrationEvalsRunsRequest' responses: '200': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/PostApiOrchestrationEvalsRunsResponse200ApplicationJson '202': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/PostApiOrchestrationEvalsRunsResponse202ApplicationJson '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '503': $ref: '#/components/responses/ServiceUnavailable' security: - bearerAuth: [] components: schemas: PostApiOrchestrationEvalsRunsRequest: type: object properties: taskAttemptId: type: string programVersionId: type: string executionStateSessionId: type: string interpreterProfileVersionId: type: string capabilityLeaseId: type: string input: $ref: '#/components/schemas/JsonValue' idempotencyKey: type: string sourceWorkSetHash: type: string workItemIds: type: array items: type: string additionalProperties: false required: - taskAttemptId - programVersionId - executionStateSessionId - interpreterProfileVersionId - capabilityLeaseId PostApiOrchestrationEvalsRunsResponse200ApplicationJson: type: object properties: accepted: type: boolean enum: - true idempotent: type: boolean enum: - true run: type: object properties: id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' schemaVersion: type: number enum: - 1 stateVersion: type: number evidence: type: array items: type: object properties: id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' schemaVersion: type: number enum: - 1 kind: oneOf: - type: string enum: - trace_revision - type: string enum: - trajectory - type: string enum: - evidence_item - type: string enum: - evaluation_run - type: string enum: - test_result - type: string enum: - measurement_revision - type: string enum: - evaluator_release - type: string enum: - repair_decision - type: string enum: - experiment_decision - type: string enum: - objective_decision - type: string enum: - release_plan - type: string enum: - release_decision - type: string enum: - production_outcome - type: string enum: - human_adjudication - type: string enum: - privacy_manifest - type: string enum: - task_environment_snapshot - type: string enum: - task_attempt - type: string enum: - state_delta - type: string enum: - context_epoch - type: string enum: - compaction_event - type: string enum: - model_handoff - type: string enum: - memory_decision - type: string enum: - orchestration_dispatch - type: string enum: - subagent_run - type: string enum: - trial_batch - type: string enum: - harness_comparison contentHash: type: string occurredAt: type: string privacyManifestId: type: string privacyViewId: type: string additionalProperties: false required: - id - organizationId - schemaVersion - kind - contentHash - occurredAt idempotencyKey: type: string inputHash: type: string state: oneOf: - type: string enum: - failed - type: string enum: - completed - type: string enum: - cancelled - type: string enum: - running - type: string enum: - blocked - type: string enum: - lost - type: string enum: - planned - type: string enum: - authorized - type: string enum: - synthesizing - type: string enum: - evaluating taskAttemptId: type: string programVersionId: type: string executionStateSessionId: type: string interpreterProfileVersionId: type: string capabilityLeaseId: type: string dispatchCount: type: number bridgeCallCount: type: number modelCallCount: type: number toolCallCount: type: number startedAt: type: string completedAt: type: string costUsd: type: number trajectoryId: type: string wallTimeMs: type: number sourceWorkSetHash: type: string interpreterSnapshotRef: type: object properties: organizationId: $ref: '#/components/schemas/OrganizationId' schemaVersion: type: number enum: - 1 kind: oneOf: - type: string enum: - prompt - type: string enum: - judge_config - type: string enum: - scorer - type: string enum: - policy_pack - type: string enum: - ruleset - type: string enum: - model_routing_profile - type: string enum: - model_binding - type: string enum: - retrieval_corpus - type: string enum: - retrieval_index - type: string enum: - retrieval_config - type: string enum: - tool_contract - type: string enum: - tool_inventory - type: string enum: - workflow - type: string enum: - agent_topology - type: string enum: - memory_entry_set - type: string enum: - skill - type: string enum: - runtime_config - type: string enum: - dataset - type: string enum: - evaluation - type: string enum: - repository_commit - type: string enum: - agent_task - type: string enum: - task_environment - type: string enum: - task_verifier - type: string enum: - harness_profile - type: string enum: - context_policy - type: string enum: - execution_state_schema - type: string enum: - orchestration_program - type: string enum: - subagent_profile - type: string enum: - eval_catalog - type: string enum: - experiment_plan - type: string enum: - tool_policy - type: string enum: - mcp_config - type: string enum: - memory_policy - type: string enum: - prompt_overlay - type: string enum: - release_constraint artifactId: type: string versionId: type: string contentHash: type: string environment: oneOf: - type: string enum: - local - type: string enum: - ci - type: string enum: - dev - type: string enum: - staging - type: string enum: - prod sourceControl: type: object properties: repositoryId: type: string provider: oneOf: - type: string enum: - github - type: string enum: - gitlab - type: string enum: - bitbucket - type: string enum: - azure_devops - type: string enum: - other commitSha: type: string path: type: string additionalProperties: false required: - repositoryId - provider - commitSha dependencySnapshotHash: type: string additionalProperties: false required: - organizationId - schemaVersion - kind - artifactId - versionId - contentHash additionalProperties: false required: - id - organizationId - schemaVersion - stateVersion - evidence - idempotencyKey - inputHash - state - taskAttemptId - programVersionId - executionStateSessionId - interpreterProfileVersionId - capabilityLeaseId - dispatchCount - bridgeCallCount - modelCallCount - toolCallCount additionalProperties: false required: - accepted - idempotent - run PostApiOrchestrationEvalsRunsResponse202ApplicationJson: oneOf: - type: object properties: accepted: type: boolean enum: - true idempotent: type: boolean enum: - true run: type: object properties: id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' schemaVersion: type: number enum: - 1 stateVersion: type: number evidence: type: array items: type: object properties: id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' schemaVersion: type: number enum: - 1 kind: oneOf: - type: string enum: - trace_revision - type: string enum: - trajectory - type: string enum: - evidence_item - type: string enum: - evaluation_run - type: string enum: - test_result - type: string enum: - measurement_revision - type: string enum: - evaluator_release - type: string enum: - repair_decision - type: string enum: - experiment_decision - type: string enum: - objective_decision - type: string enum: - release_plan - type: string enum: - release_decision - type: string enum: - production_outcome - type: string enum: - human_adjudication - type: string enum: - privacy_manifest - type: string enum: - task_environment_snapshot - type: string enum: - task_attempt - type: string enum: - state_delta - type: string enum: - context_epoch - type: string enum: - compaction_event - type: string enum: - model_handoff - type: string enum: - memory_decision - type: string enum: - orchestration_dispatch - type: string enum: - subagent_run - type: string enum: - trial_batch - type: string enum: - harness_comparison contentHash: type: string occurredAt: type: string privacyManifestId: type: string privacyViewId: type: string additionalProperties: false required: - id - organizationId - schemaVersion - kind - contentHash - occurredAt idempotencyKey: type: string inputHash: type: string state: oneOf: - type: string enum: - failed - type: string enum: - completed - type: string enum: - cancelled - type: string enum: - running - type: string enum: - blocked - type: string enum: - lost - type: string enum: - planned - type: string enum: - authorized - type: string enum: - synthesizing - type: string enum: - evaluating taskAttemptId: type: string programVersionId: type: string executionStateSessionId: type: string interpreterProfileVersionId: type: string capabilityLeaseId: type: string dispatchCount: type: number bridgeCallCount: type: number modelCallCount: type: number toolCallCount: type: number startedAt: type: string completedAt: type: string costUsd: type: number trajectoryId: type: string wallTimeMs: type: number sourceWorkSetHash: type: string interpreterSnapshotRef: type: object properties: organizationId: $ref: '#/components/schemas/OrganizationId' schemaVersion: type: number enum: - 1 kind: oneOf: - type: string enum: - prompt - type: string enum: - judge_config - type: string enum: - scorer - type: string enum: - policy_pack - type: string enum: - ruleset - type: string enum: - model_routing_profile - type: string enum: - model_binding - type: string enum: - retrieval_corpus - type: string enum: - retrieval_index - type: string enum: - retrieval_config - type: string enum: - tool_contract - type: string enum: - tool_inventory - type: string enum: - workflow - type: string enum: - agent_topology - type: string enum: - memory_entry_set - type: string enum: - skill - type: string enum: - runtime_config - type: string enum: - dataset - type: string enum: - evaluation - type: string enum: - repository_commit - type: string enum: - agent_task - type: string enum: - task_environment - type: string enum: - task_verifier - type: string enum: - harness_profile - type: string enum: - context_policy - type: string enum: - execution_state_schema - type: string enum: - orchestration_program - type: string enum: - subagent_profile - type: string enum: - eval_catalog - type: string enum: - experiment_plan - type: string enum: - tool_policy - type: string enum: - mcp_config - type: string enum: - memory_policy - type: string enum: - prompt_overlay - type: string enum: - release_constraint artifactId: type: string versionId: type: string contentHash: type: string environment: oneOf: - type: string enum: - local - type: string enum: - ci - type: string enum: - dev - type: string enum: - staging - type: string enum: - prod sourceControl: type: object properties: repositoryId: type: string provider: oneOf: - type: string enum: - github - type: string enum: - gitlab - type: string enum: - bitbucket - type: string enum: - azure_devops - type: string enum: - other commitSha: type: string path: type: string additionalProperties: false required: - repositoryId - provider - commitSha dependencySnapshotHash: type: string additionalProperties: false required: - organizationId - schemaVersion - kind - artifactId - versionId - contentHash additionalProperties: false required: - id - organizationId - schemaVersion - stateVersion - evidence - idempotencyKey - inputHash - state - taskAttemptId - programVersionId - executionStateSessionId - interpreterProfileVersionId - capabilityLeaseId - dispatchCount - bridgeCallCount - modelCallCount - toolCallCount additionalProperties: false required: - accepted - idempotent - run - type: object properties: accepted: type: boolean enum: - true idempotent: type: boolean enum: - false jobId: type: number additionalProperties: false required: - accepted - idempotent - jobId JsonValue: oneOf: - type: 'null' - type: boolean - type: number - type: string - type: array items: $ref: '#/components/schemas/JsonValue' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' OrganizationId: type: string format: uuid ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - INVALID_REFERENCE - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - FEATURE_LIMIT_REACHED - NO_ORG_MEMBERSHIP - SHARE_REVOKED - SHARE_EXPIRED - SHARE_UNAVAILABLE - PROVIDER_KEY_MISSING - PROVIDER_KEY_EXPIRED - PROVIDER_RATE_LIMITED - PROVIDER_UNAUTHORIZED - PROVIDER_UNAVAILABLE - AGENT_HANDOFF_INVALID - AGENT_HANDOFF_EXPIRED - AGENT_HANDOFF_DENIED - AGENT_HANDOFF_CONSUMED - PAYLOAD_TOO_LARGE - METHOD_NOT_ALLOWED message: type: string details: {} requestId: type: string format: uuid redaction: type: object properties: applied: type: boolean enum: - true fieldCount: type: integer minimum: 0 secretCount: type: integer minimum: 0 required: - applied - fieldCount - secretCount required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: WWW-Authenticate: description: >- Bearer challenge. Agent-facing routes include the RFC 9728 resource_metadata URL. schema: type: string Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: Retry-After: description: Delay in seconds before retrying when present. schema: type: integer minimum: 1 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' ServiceUnavailable: description: Service unavailable content: application/json: schema: $ref: '#/components/schemas/ApiError' securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: EvalGate API key description: >- Organization-scoped EvalGate API key. Discover the supported agent scopes at /.well-known/oauth-protected-resource. Per-operation grants are listed in x-evalgate-required-scopes; OpenAPI bearer security requirement arrays remain empty as required for non-OAuth schemes. x-evalgate-scopes-supported: - eval:read - eval:write - objectives:read - objectives:write - runs:read - runs:write - traces:read - traces:write - exports:download - reports:write - prompt:publish - scorer:publish - red_team:read - red_team:run - red_team:approve - docs:read - agent:read - agent:execute - admin:keys - admin:org x-evalgate-protected-resource-metadata: https://www.evalgate.com/.well-known/oauth-protected-resource ````