> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations

# Evaluations API — create and run evals

> Create evaluation definitions, retrieve them with test cases and runs, and start new runs against dev, staging, or production environments.

The Evaluations API is the core of EvalGate's quality loop. Use it to define what you want to measure, add test cases, and trigger runs that compute a pass/fail score you can gate on in CI or track over time in the dashboard.

## List evaluations

Returns all evaluations for the authenticated organization. Use the `status` query parameter to filter by lifecycle stage.

```bash theme={null} theme={null}
curl "https://evalgate.com/api/evaluations" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### Query parameters

<ParamField query="status" type="string">
  Filter by evaluation status. Accepted values: `draft`, `active`, `archived`.
</ParamField>

<ParamField query="limit" type="integer">
  Maximum number of results to return. Defaults to 50, maximum 100.
</ParamField>

<ParamField query="offset" type="integer">
  Number of results to skip for pagination. Defaults to 0.
</ParamField>

### Response

```json theme={null} theme={null}
{
  "evaluations": [
    {
      "id": 42,
      "name": "Chatbot regression",
      "description": null,
      "type": "unit_test",
      "status": "draft",
      "organizationId": "00000000-0000-4000-8000-000000000001",
      "createdAt": "2026-03-15T10:30:00.000Z",
      "updatedAt": "2026-03-15T10:30:00.000Z"
    }
  ]
}
```

<ResponseField name="evaluations" type="array">
  <Expandable title="Evaluation object fields">
    <ResponseField name="id" type="integer">Unique numeric ID of the evaluation.</ResponseField>
    <ResponseField name="name" type="string">Display name.</ResponseField>
    <ResponseField name="description" type="string | null">Optional description.</ResponseField>
    <ResponseField name="type" type="string">One of `unit_test`, `human_eval`, `model_eval`, `ab_test`.</ResponseField>
    <ResponseField name="status" type="string">One of `draft`, `active`, `archived`.</ResponseField>
    <ResponseField name="organizationId" type="string">UUID of the owning organization.</ResponseField>
    <ResponseField name="createdAt" type="string">ISO 8601 creation timestamp.</ResponseField>
    <ResponseField name="updatedAt" type="string">ISO 8601 last-updated timestamp.</ResponseField>
  </Expandable>
</ResponseField>

***

## Create an evaluation

Creates a new evaluation definition. The evaluation starts in `draft` status.

```bash theme={null} theme={null}
curl https://evalgate.com/api/evaluations \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Evaluation",
    "type": "unit_test"
  }'
```

### Request body

<ParamField body="name" type="string" required>
  Display name for the evaluation.
</ParamField>

<ParamField body="type" type="string" required>
  Evaluation type. One of `unit_test`, `human_eval`, `model_eval`, `ab_test`.
</ParamField>

<ParamField body="description" type="string">
  Optional description explaining the purpose of this evaluation.
</ParamField>

<ParamField body="executionSettings" type="object">
  Optional settings controlling how the evaluation is executed (parallelism, timeout, etc.).
</ParamField>

<ParamField body="modelSettings" type="object">
  Optional model configuration applied when the evaluation runner makes LLM calls.
</ParamField>

<ParamField body="customMetrics" type="array">
  Optional array of custom metric definitions to compute alongside built-in scoring.
</ParamField>

<ParamField body="testCases" type="array">
  Optional inline test cases to attach at creation time.
</ParamField>

### Response

Returns the created evaluation object:

```json theme={null} theme={null}
{
  "id": 43,
  "name": "My Evaluation",
  "type": "unit_test",
  "status": "draft",
  "organizationId": "00000000-0000-4000-8000-000000000001",
  "createdAt": "2026-03-15T10:35:00.000Z",
  "updatedAt": "2026-03-15T10:35:00.000Z"
}
```

***

## Get a single evaluation

Retrieve one evaluation by its numeric ID. The response includes the evaluation's `testCases` and recent `runs` arrays, which the list endpoint omits.

```bash theme={null} theme={null}
curl "https://evalgate.com/api/evaluations?id=42" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### Query parameters

<ParamField query="id" type="integer" required>
  Numeric ID of the evaluation to retrieve.
</ParamField>

### Response

```json theme={null} theme={null}
{
  "id": 42,
  "name": "Chatbot regression",
  "type": "unit_test",
  "status": "active",
  "organizationId": "00000000-0000-4000-8000-000000000001",
  "createdAt": "2026-03-15T10:30:00.000Z",
  "updatedAt": "2026-03-15T10:30:00.000Z",
  "testCases": [
    {
      "id": 1,
      "name": "Case A",
      "input": "What is your refund policy?"
    }
  ],
  "runs": [
    {
      "id": 9001,
      "status": "completed",
      "evaluationId": 42
    }
  ]
}
```

<ResponseField name="testCases" type="array">
  Test cases attached to this evaluation.
</ResponseField>

<ResponseField name="runs" type="array">
  Recent evaluation runs. Ordered by creation time, most recent first.
</ResponseField>

***

## Start an evaluation run

Triggers a new run for an existing evaluation. Pass an `environment` value to tag the run for filtering in the dashboard and in CI comparisons.

```bash theme={null} theme={null}
curl https://evalgate.com/api/evaluations/42/runs \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment": "dev"}'
```

### Path parameters

<ParamField path="id" type="integer" required>
  Numeric ID of the evaluation to run.
</ParamField>

### Request body

<ParamField body="environment" type="string">
  Target environment for this run. Accepted values: `dev`, `staging`, `prod`. You can also pass the environment via the `x-evalgate-env` request header instead of the body.
</ParamField>

### Response

```json theme={null} theme={null}
{
  "id": 9002,
  "evaluationId": 42,
  "status": "running",
  "environment": "dev",
  "createdAt": "2026-03-15T10:40:00.000Z"
}
```

<ResponseField name="id" type="integer">Unique ID of the new run.</ResponseField>
<ResponseField name="evaluationId" type="integer">ID of the parent evaluation.</ResponseField>
<ResponseField name="status" type="string">Initial status — `running`. Poll or use webhooks to detect completion.</ResponseField>
<ResponseField name="environment" type="string">The environment value this run was tagged with.</ResponseField>
<ResponseField name="createdAt" type="string">ISO 8601 timestamp when the run was created.</ResponseField>

<Note>
  To import results from your own test runner instead of triggering a managed run, use `POST /api/evaluations/{id}/runs/import` with an optional `Idempotency-Key` header to prevent duplicate runs on CI retry.
</Note>