> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create llm judge test > Requires scopes: eval:write ## OpenAPI ````yaml /api-reference/openapi.json post /api/llm-judge/test openapi: 3.1.0 info: title: EvalGate API version: 3.7.6 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/llm-judge/test: post: tags: - llm judge summary: Create llm judge test description: 'Requires scopes: eval:write' operationId: post_llm_judge_test requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostApiLlmJudgeTestRequest' responses: '200': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/PostApiLlmJudgeTestResponse200ApplicationJson '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' security: - bearerAuth: - eval:write components: schemas: PostApiLlmJudgeTestRequest: type: object properties: input: type: string output: type: string promptTemplate: type: string expectedOutput: type: string provider: oneOf: - type: string enum: - local - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot aggregation: oneOf: - type: string enum: - all_pass - type: string enum: - any_pass - type: string enum: - weighted - type: string enum: - primary_fallback - type: string enum: - escalate_on_disagreement context: oneOf: - type: string - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' model: type: string criteria: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' settings: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' behavior: type: string taskType: type: string passThreshold: type: number judges: type: array items: type: object properties: type: oneOf: - type: string enum: - rule - type: string enum: - llm id: type: string provider: oneOf: - type: string enum: - local - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot threshold: type: number model: type: string temperature: type: number weight: type: number seed: type: number topP: type: number maxTokens: type: number rule: oneOf: - type: string enum: - json_schema - type: string enum: - exact_match - type: string enum: - contains_expected top_p: type: number timeoutMs: type: number cacheEnabled: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false instabilityThreshold: type: number stabilityRuns: type: number additionalProperties: false required: - input - output - promptTemplate PostApiLlmJudgeTestResponse200ApplicationJson: type: object properties: promptPreview: type: string judgement: type: object properties: judgeResultId: type: number judgeConfigHash: type: string comparabilityStatus: oneOf: - type: string enum: - not_comparable - type: string enum: - exact_comparable - type: string enum: - calibrated_comparable - type: string enum: - trend_discontinuity provider: oneOf: - type: string enum: - local - type: string enum: - rule - type: string enum: - aggregate - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot score: type: number latencyMs: type: number aggregation: oneOf: - type: string enum: - all_pass - type: string enum: - any_pass - type: string enum: - weighted - type: string enum: - primary_fallback - type: string enum: - escalate_on_disagreement model: type: string reasoning: type: string flags: type: array items: type: string passed: type: boolean signals: type: object additionalProperties: oneOf: - type: string - type: number - type: boolean enum: - false - type: boolean enum: - true retries: type: number judgeResults: type: array items: type: object properties: type: oneOf: - type: string enum: - rule - type: string enum: - llm - type: string enum: - hybrid latencyMs: type: number judgeId: type: string error: type: string provider: oneOf: - type: string enum: - local - type: string enum: - rule - type: string enum: - aggregate - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot result: type: object properties: provider: oneOf: - type: string enum: - local - type: string enum: - rule - type: string enum: - aggregate - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot score: type: number latencyMs: type: number model: type: string reasoning: type: string passed: type: boolean signals: type: object additionalProperties: oneOf: - type: string - type: number - type: boolean enum: - false - type: boolean enum: - true retries: type: number judgeId: type: string parseStatus: oneOf: - type: string enum: - failed - type: string enum: - ok - type: string enum: - retried confidence: type: number modelCallId: type: string tokenUsage: type: object properties: input: type: number output: type: number total: type: number additionalProperties: false required: - input - output modelProvenance: type: object properties: provider: oneOf: - type: string enum: - local - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot model: type: string temperature: type: number prompt_hash: type: string top_p: type: number max_tokens: type: number seed: type: number additionalProperties: false required: - provider - model - temperature - prompt_hash - top_p - max_tokens cached: oneOf: - type: boolean enum: - false - type: boolean enum: - true judgeFingerprint: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - provider - score - latencyMs - model - reasoning - passed - signals - retries - judgeId - parseStatus model: type: string weight: type: number modelProvenance: type: object properties: provider: oneOf: - type: string enum: - local - type: string enum: - openai - type: string enum: - anthropic - type: string enum: - google - type: string enum: - xai - type: string enum: - cursor - type: string enum: - windsurf - type: string enum: - moonshot model: type: string temperature: type: number prompt_hash: type: string top_p: type: number max_tokens: type: number seed: type: number additionalProperties: false required: - provider - model - temperature - prompt_hash - top_p - max_tokens additionalProperties: false required: - type - latencyMs - judgeId judgeId: type: string parseStatus: oneOf: - type: string enum: - failed - type: string enum: - ok - type: string enum: - retried disagreement: type: object properties: rate: type: number reasoningDivergence: type: number outlierJudgeIds: type: array items: type: string requiresEscalation: type: boolean additionalProperties: false required: - rate - reasoningDivergence - outlierJudgeIds - requiresEscalation confidence: type: number totalCostUsd: type: number tokenUsage: type: object properties: input: type: number output: type: number total: type: number additionalProperties: false required: - input - output variance: type: number additionalProperties: false required: - provider - score - latencyMs - aggregation - model - reasoning - flags - passed - signals - retries - judgeResults - judgeId - parseStatus - disagreement additionalProperties: false required: - promptPreview - judgement JsonValue: oneOf: - type: 'null' - type: boolean - type: number - type: string - type: array items: $ref: '#/components/schemas/JsonValue' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - NO_ORG_MEMBERSHIP - CROSS_ORG_REFERENCE - INCOMPLETE - MALFORMED message: type: string details: {} requestId: type: string format: uuid required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: API Key or Session Token ````