Skip to main content

Documentation Index

Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

​
MCP API — tool discovery and execution

Discover Evalgate tools anonymously and execute them from any MCP-compatible AI agent, IDE assistant, or custom client using a bearer token.
Evalgate exposes an MCP-style API that lets AI agents and IDE assistants (Cursor, Claude Desktop, ChatGPT) discover and call Evalgate operations as structured tools. Tool discovery is public — no authentication required. Tool execution requires a bearer token.

​
GET /api/mcp/tools — list available tools

Returns all tools available for MCP execution. This endpoint requires no authentication and is designed for tool discovery by agent runtimes and IDE plugins. 
curl https://evalgate.com/api/mcp/tools

​
Response

{
  "mcpVersion": "1",
  "tools": [
    {
      "name": "eval.quality.latest",
      "description": "Get the latest quality score for an evaluation.",
      "inputSchema": {
        "type": "object",
        "properties": {
          "evaluationId": {
            "type": "number",
            "description": "ID of the evaluation"
          },
          "baseline": {
            "type": "string",
            "enum": ["published", "previous", "production"]
          }
        },
        "required": ["evaluationId"]
      },
      "version": "1.0",
      "longRunning": false
    }
  ]
}
​
mcpVersion
string
The MCP protocol version implemented by this server.
​
tools
array

​
POST /api/mcp/call — execute a tool

Executes a named tool with the provided arguments. Requires a bearer token. 
curl https://evalgate.com/api/mcp/call \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool": "eval.quality.latest",
    "arguments": {
      "evaluationId": 42,
      "baseline": "published"
    }
  }'

​
Request body

​
tool
string
required
The name of the tool to execute, exactly as returned by GET /api/mcp/tools.
​
arguments
object
required
Arguments matching the tool’s inputSchema. Required fields in the schema must be present.

​
Success response (200)

{
  "ok": true,
  "content": [
    {
      "type": "json",
      "json": {
        "score": 85,
        "baselineScore": 82,
        "regressionDetected": false,
        "evaluationId": 42
      }
    }
  ]
}
​
ok
boolean
Always true on success.
​
content
array
Array of result items. Each item has a type field (json or text) and carries the tool’s result in either json (an object) or text (a string).

​
Error response (4xx / 5xx)

{
  "ok": false,
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "evaluationId is required",
    "requestId": "550e8400-e29b-41d4-a716-446655440000"
  }
}
​
ok
boolean
Always false on error.
​
error.code
string
Machine-readable error code. See API error codes for the full list.
​
error.message
string
Human-readable error description.
​
error.requestId
string
UUID for support diagnostics.

​
Available tools

​
eval.quality.latest

Get the latest quality score for an evaluation, optionally compared to a baseline.
ParameterTypeRequiredDescription
evaluationIdnumberYesNumeric ID of the evaluation
baselinestringNoComparison baseline: published, previous, or production

​
eval.run.create

Start a new evaluation run.
ParameterTypeRequiredDescription
evaluationIdnumberYesNumeric ID of the evaluation to run
environmentstringNoTarget environment: dev, staging, or prod

​
eval.trace.create

Create a distributed trace record.
ParameterTypeRequiredDescription
namestringYesDisplay name for the trace
metadataobjectNoArbitrary JSON context to store with the trace

​
eval.testcase.list

List test cases for an evaluation.
ParameterTypeRequiredDescription
evaluationIdnumberYesNumeric ID of the evaluation
limitnumberNoMaximum number of test cases to return

​
Connecting an MCP client

{
  "mcpServers": {
    "evalgate": {
      "url": "https://evalgate.com/api/mcp/tools",
      "auth": "Bearer YOUR_API_KEY"
    }
  }
}
The GET /api/mcp/tools discovery call does not need an API key. Only include the auth field in your MCP client configuration — the client will send it automatically when calling POST /api/mcp/call.