> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create harness compare > Requires scopes: eval:read ## OpenAPI ````yaml /api-reference/openapi.json post /api/harness/experiments/compare openapi: 3.1.2 info: title: EvalGate API version: 5.1.0 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/harness/experiments/compare: post: tags: - harness summary: Create harness compare description: 'Requires scopes: eval:read' operationId: post_harness_experiments_compare requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostApiHarnessExperimentsCompareRequest' responses: '200': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/PostApiHarnessExperimentsCompareResponse200ApplicationJson '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '503': $ref: '#/components/responses/ServiceUnavailable' security: - bearerAuth: [] components: schemas: PostApiHarnessExperimentsCompareRequest: type: object properties: policy: type: object properties: confidenceLevel: type: number minimumMeaningfulDelta: type: number minimumSampleSize: type: number qualityFloor: type: number nonRegressionMargin: type: number efficiencyMetrics: type: array items: type: string additionalProperties: false required: - confidenceLevel candidate: type: object properties: schemaVersion: type: number enum: - 1 batchId: type: string correctness: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev 'n': type: number hardFailureCounts: type: object additionalProperties: type: number categoryScores: type: object additionalProperties: type: number summaryHash: type: string costUsd: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev latencyMs: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev solveRate: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev stepRatio: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev toolCallRatio: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev additionalProperties: false required: - schemaVersion - batchId - correctness - 'n' - hardFailureCounts - categoryScores - summaryHash baseline: type: object properties: schemaVersion: type: number enum: - 1 batchId: type: string correctness: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev 'n': type: number hardFailureCounts: type: object additionalProperties: type: number categoryScores: type: object additionalProperties: type: number summaryHash: type: string costUsd: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev latencyMs: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev solveRate: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev stepRatio: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev toolCallRatio: type: object properties: max: type: number min: type: number mean: type: number median: type: number sampleStdDev: oneOf: - type: 'null' - type: number additionalProperties: false required: - max - min - mean - median - sampleStdDev additionalProperties: false required: - schemaVersion - batchId - correctness - 'n' - hardFailureCounts - categoryScores - summaryHash qualityMetric: type: string additionalProperties: false required: - policy - candidate - baseline PostApiHarnessExperimentsCompareResponse200ApplicationJson: type: object properties: automaticApply: type: boolean decision: oneOf: - type: string enum: - candidate_better - type: string enum: - equivalent_within_policy - type: string enum: - inconclusive - type: string enum: - baseline_better policy: type: object properties: confidenceLevel: type: number minimumSampleSize: type: number qualityFloor: type: number nonRegressionMargin: type: number minimumMeaningfulDelta: type: number additionalProperties: false required: - confidenceLevel - minimumSampleSize - qualityFloor - nonRegressionMargin - minimumMeaningfulDelta quality: allOf: - type: object properties: metric: type: string baselineEstimate: oneOf: - type: 'null' - type: number candidateEstimate: oneOf: - type: 'null' - type: number delta: oneOf: - type: 'null' - type: number baselineInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number confidenceLevel: type: number additionalProperties: false required: - lower - upper - confidenceLevel candidateInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number confidenceLevel: type: number additionalProperties: false required: - lower - upper - confidenceLevel deltaInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number confidenceLevel: type: number additionalProperties: false required: - lower - upper - confidenceLevel evidenceSufficient: type: boolean improved: type: boolean regressed: type: boolean additionalProperties: false required: - metric - baselineEstimate - candidateEstimate - delta - baselineInterval - candidateInterval - deltaInterval - evidenceSufficient - improved - regressed - type: object properties: passesQualityFloor: type: boolean nonRegression: type: boolean meaningfulImprovement: type: boolean meaningfullyRegressed: type: boolean additionalProperties: false required: - passesQualityFloor - nonRegression - meaningfulImprovement - meaningfullyRegressed efficiency: type: object properties: metrics: type: object additionalProperties: type: object properties: metric: type: string baselineEstimate: oneOf: - type: 'null' - type: number candidateEstimate: oneOf: - type: 'null' - type: number delta: oneOf: - type: 'null' - type: number baselineInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number confidenceLevel: type: number additionalProperties: false required: - lower - upper - confidenceLevel candidateInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number confidenceLevel: type: number additionalProperties: false required: - lower - upper - confidenceLevel deltaInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number confidenceLevel: type: number additionalProperties: false required: - lower - upper - confidenceLevel evidenceSufficient: type: boolean improved: type: boolean regressed: type: boolean additionalProperties: false required: - metric - baselineEstimate - candidateEstimate - delta - baselineInterval - candidateInterval - deltaInterval - evidenceSufficient - improved - regressed savingsRecommendation: type: object properties: supported: type: boolean metrics: type: array items: type: string additionalProperties: false required: - supported - metrics additionalProperties: false required: - metrics - savingsRecommendation confidence: type: object properties: qualityEvidenceSufficient: type: boolean efficiencyEvidenceSufficient: type: boolean baseN: type: number candidateN: type: number additionalProperties: false required: - qualityEvidenceSufficient - efficiencyEvidenceSufficient - baseN - candidateN reasonCodes: type: array items: type: string additionalProperties: false required: - automaticApply - decision - policy - quality - efficiency - confidence - reasonCodes ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - INVALID_REFERENCE - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - FEATURE_LIMIT_REACHED - NO_ORG_MEMBERSHIP - SHARE_REVOKED - SHARE_EXPIRED - SHARE_UNAVAILABLE - PROVIDER_KEY_MISSING - PROVIDER_KEY_EXPIRED - PROVIDER_RATE_LIMITED - PROVIDER_UNAUTHORIZED - PROVIDER_UNAVAILABLE - AGENT_HANDOFF_INVALID - AGENT_HANDOFF_EXPIRED - AGENT_HANDOFF_DENIED - AGENT_HANDOFF_CONSUMED - PAYLOAD_TOO_LARGE - METHOD_NOT_ALLOWED message: type: string details: {} requestId: type: string format: uuid redaction: type: object properties: applied: type: boolean enum: - true fieldCount: type: integer minimum: 0 secretCount: type: integer minimum: 0 required: - applied - fieldCount - secretCount required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: WWW-Authenticate: description: >- Bearer challenge. Agent-facing routes include the RFC 9728 resource_metadata URL. schema: type: string Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: Retry-After: description: Delay in seconds before retrying when present. schema: type: integer minimum: 1 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' ServiceUnavailable: description: Service unavailable content: application/json: schema: $ref: '#/components/schemas/ApiError' securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: EvalGate API key description: >- Organization-scoped EvalGate API key. Discover the supported agent scopes at /.well-known/oauth-protected-resource. Per-operation grants are listed in x-evalgate-required-scopes; OpenAPI bearer security requirement arrays remain empty as required for non-OAuth schemes. x-evalgate-scopes-supported: - eval:read - eval:write - objectives:read - objectives:write - runs:read - runs:write - traces:read - traces:write - exports:download - reports:write - prompt:publish - scorer:publish - red_team:read - red_team:run - red_team:approve - docs:read - agent:read - agent:execute - admin:keys - admin:org x-evalgate-protected-resource-metadata: https://www.evalgate.com/.well-known/oauth-protected-resource ````