> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-agent trajectory analysis

> Normalize, inspect, score, compare, and report multi-agent runs without losing model-call, cost, tool, or parentage evidence.

Multi-agent trajectory analysis turns a hosted workflow or remote execution stream into one versioned, queryable evidence graph. Use it when outcome scores alone cannot explain which agent acted, why control changed hands, which tool or model call was responsible, or where cost accumulated.

Open **Evaluation → Trajectories** at `/trajectory-analysis` to inspect runs. The workspace keeps partial evidence visible, but marks it `incomplete` or `malformed` until the contract is satisfied. It never creates placeholder model calls, costs, tool evidence, parents, or participants.

## What the contract records

The version 2 contract normalizes these event classes:

| Event                     | Required identity or evidence                  | Typical question                                 |
| ------------------------- | ---------------------------------------------- | ------------------------------------------------ |
| `message`                 | participant, timestamp, content                | What context did an agent receive?               |
| `handoff`                 | source participant, target participant, parent | Who transferred control and when?                |
| `tool_call`               | participant, tool name, evidence item, parent  | Which governed tool invocation occurred?         |
| `tool_result`             | participant, parent tool call                  | What result returned to the workflow?            |
| `model_call`              | Model Gateway ledger ID, participant, parent   | Which provider/model request produced this step? |
| `cost`                    | cost record ID, participant, parent            | Which persisted charge belongs to the call?      |
| `environment_observation` | environment, key/value, participant, parent    | What runtime condition affected execution?       |
| `final`                   | participant, timestamp, content, parent        | Which event completed the trajectory?            |

Participants have stable keys, display names, roles, optional framework identifiers, and metadata. Events have stable keys, source timestamps, optional source sequence numbers, explicit parent keys, and typed evidence references.

## Ingest a trajectory

Send batches to `POST /api/trajectory-analysis` with `runs:write`. One batch accepts up to 5,000 events and 10 MiB. Reuse the trajectory key across late batches; use a new idempotency key for each distinct batch.

```json theme={null}
{
  "schemaVersion": 2,
  "trajectoryKey": "support-triage:run-482",
  "name": "Support triage run 482",
  "sourceVariant": "remote",
  "sourceRunId": "runner-job-482",
  "idempotencyKey": "runner-job-482:events:0-7",
  "retentionDays": 90,
  "finalize": true,
  "participants": [
    {
      "participantKey": "coordinator",
      "displayName": "Coordinator",
      "role": "orchestrator",
      "framework": "custom",
      "metadata": {}
    },
    {
      "participantKey": "researcher",
      "displayName": "Researcher",
      "role": "specialist",
      "metadata": {}
    }
  ],
  "events": [
    {
      "eventKey": "evt-001",
      "kind": "message",
      "participantKey": "coordinator",
      "occurredAt": "2026-07-13T16:00:00.000Z",
      "content": "Investigate the billing discrepancy.",
      "metadata": {}
    },
    {
      "eventKey": "evt-002",
      "kind": "handoff",
      "participantKey": "coordinator",
      "targetParticipantKey": "researcher",
      "parentEventKey": "evt-001",
      "occurredAt": "2026-07-13T16:00:01.000Z",
      "metadata": {}
    }
  ]
}
```

The response contains the run, normalized participants, ordered events, scores, and reports. If a referenced participant or parent arrives later, EvalGate retains the raw key and repairs the durable relationship when the missing object is ingested.

### Idempotency and conflicts

* Replaying the same idempotency key and identical body returns the existing trajectory.
* Reusing an idempotency key with different content returns `409 CONFLICT`.
* Reusing an event key with different content returns `409 CONFLICT`.
* A trajectory key cannot switch its source run or hosted/remote variant.
* Evidence IDs from another organization return `403 CROSS_ORG_REFERENCE`.

These rules apply to large streams and late events. A batch is committed atomically: no participant, event, receipt, or completion state is partially accepted.

## Understand completion states

`complete` means the run is finalized and every event class, participant, parent, target participant, tool evidence item, Model Gateway call, and cost record is present.

`incomplete` is a valid persisted state. The response lists exact missing parts such as:

```json theme={null}
{
  "state": "incomplete",
  "missingParts": [
    "cost:evt-006",
    "evidence:evt-003",
    "parent:evt-002"
  ]
}
```

`malformed` indicates a structural contradiction such as a duplicate participant identity or self-parent event. Malformed runs remain inspectable for diagnosis but cannot be scored or reported.

<Warning>
  EvalGate does not estimate or synthesize a missing price, gateway call, evidence item, participant, or parent. Ingest the durable source record and then send a late trajectory batch that references it.
</Warning>

## Inspect the explorer

The **Timeline** tab separates named agent lanes from the event stream. Events are ordered by source timestamp, then source sequence, then stable event key. Each row exposes its parent and participant so concurrency, handoffs, loops, and premature termination remain visible.

The **Evidence** tab counts exact Model Gateway, cost, and tool-evidence references. A complete run shows zero orphaned parent or participant relationships. Incomplete runs list each unresolved contract part next to the affected workflow.

The **Quality** tab displays versioned trajectory scorer results and evidence reports. The **Compare** tab compares a selected baseline against another hosted or remote variant without merging their provenance.

## Use the SDK clients

<CodeGroup>
  ```typescript TypeScript theme={null}
  import { TrajectoryAnalysisClient } from "@evalgate/sdk";

  const trajectories = new TrajectoryAnalysisClient({
    baseUrl: process.env.EVALGATE_BASE_URL!,
    apiKey: process.env.EVALGATE_API_KEY!,
  });

  const detail = await trajectories.ingest({
    schemaVersion: 2,
    trajectoryKey: "support-triage:run-482",
    name: "Support triage run 482",
    sourceVariant: "remote",
    sourceRunId: "runner-job-482",
    idempotencyKey: "runner-job-482:events:0-7",
    participants,
    events,
    finalize: true,
    retentionDays: 90,
  });
  ```

  ```python Python theme={null}
  import os
  from evalgate_sdk.trajectory_analysis import TrajectoryAnalysisClient

  async with TrajectoryAnalysisClient(
      base_url=os.environ["EVALGATE_BASE_URL"],
      api_key=os.environ["EVALGATE_API_KEY"],
  ) as trajectories:
      detail = await trajectories.ingest({
          "schemaVersion": 2,
          "trajectoryKey": "support-triage:run-482",
          "name": "Support triage run 482",
          "sourceVariant": "remote",
          "sourceRunId": "runner-job-482",
          "idempotencyKey": "runner-job-482:events:0-7",
          "participants": participants,
          "events": events,
          "finalize": True,
          "retentionDays": 90,
      })
  ```
</CodeGroup>

Both clients also provide `list`, `get`, `score`, `createReport` / `create_report`, and `compare`. Structured API failures preserve the HTTP status and EvalGate error code so callers can distinguish incomplete evidence from identity conflicts or authorization failures.

## Score a complete trajectory

Call `POST /api/trajectory-analysis/{trajectoryId}/score` with `runs:write`:

```json theme={null}
{
  "scorerKey": "trajectory-quality",
  "scorerVersion": "1.0.0",
  "ideal": {
    "expectedSteps": 8,
    "expectedToolCalls": 1,
    "requiredTools": ["lookup_invoice"],
    "forbiddenTools": ["issue_refund_without_approval"],
    "allowParallel": true,
    "maxRedundantSteps": 0
  }
}
```

Scoring uses the platform's authoritative trajectory scorer. The stored input hash binds the normalized trajectory content hash, scorer identity/version, and ideal specification. Identical scoring requests are idempotent.

Incomplete and malformed runs fail closed with `409 INCOMPLETE` or `409 MALFORMED`.

## Build an evidence report

After scoring, call `POST /api/trajectory-analysis/{trajectoryId}/reports` with `reports:write`:

```json theme={null}
{
  "title": "Support triage run 482 evidence report",
  "scoreResultId": "d899121d-70b8-4d87-a585-c9af3dd6d68d"
}
```

The report freezes:

* trajectory identity, source variant, source run, and content hash;
* participant identities and roles;
* every event ID, parent ID, and participant ID;
* exact model-call, cost-record, and evidence-item references;
* scorer identity, version, metrics, failures, and score;
* an evidence hash over the complete report payload;
* an empty orphan list, enforced before report creation.

If evidence becomes incomplete, report generation stops instead of emitting a partial certificate.

## Compare hosted and remote variants

Use `GET /api/trajectory-analysis/compare?baselineId={hostedId}&candidateId={remoteId}` with `runs:read`. The comparison returns source variant, completion state, latest score, event count, cost-evidence count, and score/event deltas for each side.

The comparison never treats two variants as interchangeable. Each side preserves its own source run, content hash, participants, events, and durable evidence references.

## Retention

Each trajectory sets a retention period from 1 to 3,650 days. Organization administrators can run `POST /api/trajectory-analysis/retention` with `admin:org` to delete expired trajectories. Cascades remove normalized participants, ingestion receipts, events, score results, and reports together; referenced gateway, cost, and platform evidence records remain governed by their own retention policies.

## Operational checklist

1. Create Model Gateway calls, cost records, and tool evidence before referencing them.
2. Use stable participant and event keys from the source system.
3. Preserve source timestamps and explicit parents; do not infer order from arrival time.
4. Use one idempotency key per distinct batch and retain it through retries.
5. Send late participants or parents under the same trajectory key.
6. Finalize only after the source believes streaming is terminal.
7. Require `complete` and zero orphans before scoring or reporting.
8. Compare hosted and remote variants as separate evidence graphs.
9. Review retention periods against organization policy.

## Failure handling

| Response                  | Meaning                                                              | Action                                             |
| ------------------------- | -------------------------------------------------------------------- | -------------------------------------------------- |
| `400 VALIDATION_ERROR`    | Body, timestamp, event kind, or size is invalid                      | Correct the contract; do not retry unchanged       |
| `403 CROSS_ORG_REFERENCE` | A run or evidence reference is missing or outside the organization   | Verify tenant-scoped IDs and source credentials    |
| `404 NOT_FOUND`           | The requested trajectory or score does not exist in the organization | Refresh the run list or correct the ID             |
| `409 CONFLICT`            | Idempotency, event identity, or source binding changed               | Reuse the original body or create a new stable key |
| `409 INCOMPLETE`          | Required evidence is unresolved                                      | Ingest durable missing evidence and a late batch   |
| `409 MALFORMED`           | Parentage or identity is contradictory                               | Repair the source event graph under new event keys |

## Authorization

| Operation                  | Minimum scope or role         |
| -------------------------- | ----------------------------- |
| List, inspect, compare     | `runs:read`                   |
| Ingest, finalize, score    | member and `runs:write`       |
| Build report               | member and `reports:write`    |
| Prune expired trajectories | administrator and `admin:org` |