Multi-agent trajectory analysis

Multi-agent trajectory analysis turns a hosted workflow or remote execution stream into one versioned, queryable evidence graph. Use it when outcome scores alone cannot explain which agent acted, why control changed hands, which tool or model call was responsible, or where cost accumulated. Open Evaluation → Trajectories at /trajectory-analysis to inspect runs. The workspace keeps partial evidence visible, but marks it incomplete or malformed until the contract is satisfied. It never creates placeholder model calls, costs, tool evidence, parents, or participants.

What the contract records

The version 2 contract normalizes these event classes:

Event	Required identity or evidence	Typical question
`message`	participant, timestamp, content	What context did an agent receive?
`handoff`	source participant, target participant, parent	Who transferred control and when?
`tool_call`	participant, tool name, evidence item, parent	Which governed tool invocation occurred?
`tool_result`	participant, parent tool call	What result returned to the workflow?
`model_call`	Model Gateway ledger ID, participant, parent	Which provider/model request produced this step?
`cost`	cost record ID, participant, parent	Which persisted charge belongs to the call?
`environment_observation`	environment, key/value, participant, parent	What runtime condition affected execution?
`final`	participant, timestamp, content, parent	Which event completed the trajectory?

Participants have stable keys, display names, roles, optional framework identifiers, and metadata. Events have stable keys, source timestamps, optional source sequence numbers, explicit parent keys, and typed evidence references.

Ingest a trajectory

Send batches to POST /api/trajectory-analysis with runs:write. One batch accepts up to 5,000 events and 10 MiB. Reuse the trajectory key across late batches; use a new idempotency key for each distinct batch.

{
  "schemaVersion": 2,
  "trajectoryKey": "support-triage:run-482",
  "name": "Support triage run 482",
  "sourceVariant": "remote",
  "sourceRunId": "runner-job-482",
  "idempotencyKey": "runner-job-482:events:0-7",
  "retentionDays": 90,
  "finalize": true,
  "participants": [
    {
      "participantKey": "coordinator",
      "displayName": "Coordinator",
      "role": "orchestrator",
      "framework": "custom",
      "metadata": {}
    },
    {
      "participantKey": "researcher",
      "displayName": "Researcher",
      "role": "specialist",
      "metadata": {}
    }
  ],
  "events": [
    {
      "eventKey": "evt-001",
      "kind": "message",
      "participantKey": "coordinator",
      "occurredAt": "2026-07-13T16:00:00.000Z",
      "content": "Investigate the billing discrepancy.",
      "metadata": {}
    },
    {
      "eventKey": "evt-002",
      "kind": "handoff",
      "participantKey": "coordinator",
      "targetParticipantKey": "researcher",
      "parentEventKey": "evt-001",
      "occurredAt": "2026-07-13T16:00:01.000Z",
      "metadata": {}
    }
  ]
}

The response contains the run, normalized participants, ordered events, scores, and reports. If a referenced participant or parent arrives later, EvalGate retains the raw key and repairs the durable relationship when the missing object is ingested.

Idempotency and conflicts

Replaying the same idempotency key and identical body returns the existing trajectory.
Reusing an idempotency key with different content returns 409 CONFLICT.
Reusing an event key with different content returns 409 CONFLICT.
A trajectory key cannot switch its source run or hosted/remote variant.
Evidence IDs from another organization return 403 CROSS_ORG_REFERENCE.

These rules apply to large streams and late events. A batch is committed atomically: no participant, event, receipt, or completion state is partially accepted.

Understand completion states

complete means the run is finalized and every event class, participant, parent, target participant, tool evidence item, Model Gateway call, and cost record is present. incomplete is a valid persisted state. The response lists exact missing parts such as:

{
  "state": "incomplete",
  "missingParts": [
    "cost:evt-006",
    "evidence:evt-003",
    "parent:evt-002"
  ]
}

malformed indicates a structural contradiction such as a duplicate participant identity or self-parent event. Malformed runs remain inspectable for diagnosis but cannot be scored or reported.

EvalGate does not estimate or synthesize a missing price, gateway call, evidence item, participant, or parent. Ingest the durable source record and then send a late trajectory batch that references it.

Inspect the explorer

The Timeline tab separates named agent lanes from the event stream. Events are ordered by source timestamp, then source sequence, then stable event key. Each row exposes its parent and participant so concurrency, handoffs, loops, and premature termination remain visible. The Evidence tab counts exact Model Gateway, cost, and tool-evidence references. A complete run shows zero orphaned parent or participant relationships. Incomplete runs list each unresolved contract part next to the affected workflow. The Quality tab displays versioned trajectory scorer results and evidence reports. The Compare tab compares a selected baseline against another hosted or remote variant without merging their provenance.

Use the SDK clients

import { TrajectoryAnalysisClient } from "@evalgate/sdk";

const trajectories = new TrajectoryAnalysisClient({
  baseUrl: process.env.EVALGATE_BASE_URL!,
  apiKey: process.env.EVALGATE_API_KEY!,
});

const detail = await trajectories.ingest({
  schemaVersion: 2,
  trajectoryKey: "support-triage:run-482",
  name: "Support triage run 482",
  sourceVariant: "remote",
  sourceRunId: "runner-job-482",
  idempotencyKey: "runner-job-482:events:0-7",
  participants,
  events,
  finalize: true,
  retentionDays: 90,
});

import os
from evalgate_sdk.trajectory_analysis import TrajectoryAnalysisClient

async with TrajectoryAnalysisClient(
    base_url=os.environ["EVALGATE_BASE_URL"],
    api_key=os.environ["EVALGATE_API_KEY"],
) as trajectories:
    detail = await trajectories.ingest({
        "schemaVersion": 2,
        "trajectoryKey": "support-triage:run-482",
        "name": "Support triage run 482",
        "sourceVariant": "remote",
        "sourceRunId": "runner-job-482",
        "idempotencyKey": "runner-job-482:events:0-7",
        "participants": participants,
        "events": events,
        "finalize": True,
        "retentionDays": 90,
    })

Both clients also provide list, get, score, createReport / create_report, and compare. Structured API failures preserve the HTTP status and EvalGate error code so callers can distinguish incomplete evidence from identity conflicts or authorization failures.

Score a complete trajectory

Call POST /api/trajectory-analysis/{trajectoryId}/score with runs:write:

{
  "scorerKey": "trajectory-quality",
  "scorerVersion": "1.0.0",
  "ideal": {
    "expectedSteps": 8,
    "expectedToolCalls": 1,
    "requiredTools": ["lookup_invoice"],
    "forbiddenTools": ["issue_refund_without_approval"],
    "allowParallel": true,
    "maxRedundantSteps": 0
  }
}

Scoring uses the platform’s authoritative trajectory scorer. The stored input hash binds the normalized trajectory content hash, scorer identity/version, and ideal specification. Identical scoring requests are idempotent. Incomplete and malformed runs fail closed with 409 INCOMPLETE or 409 MALFORMED.

Build an evidence report

After scoring, call POST /api/trajectory-analysis/{trajectoryId}/reports with reports:write:

{
  "title": "Support triage run 482 evidence report",
  "scoreResultId": "d899121d-70b8-4d87-a585-c9af3dd6d68d"
}

The report freezes:

trajectory identity, source variant, source run, and content hash;
participant identities and roles;
every event ID, parent ID, and participant ID;
exact model-call, cost-record, and evidence-item references;
scorer identity, version, metrics, failures, and score;
an evidence hash over the complete report payload;
an empty orphan list, enforced before report creation.

If evidence becomes incomplete, report generation stops instead of emitting a partial certificate.

Compare hosted and remote variants

Use GET /api/trajectory-analysis/compare?baselineId={hostedId}&candidateId={remoteId} with runs:read. The comparison returns source variant, completion state, latest score, event count, cost-evidence count, and score/event deltas for each side. The comparison never treats two variants as interchangeable. Each side preserves its own source run, content hash, participants, events, and durable evidence references.

Retention

Each trajectory sets a retention period from 1 to 3,650 days. Organization administrators can run POST /api/trajectory-analysis/retention with admin:org to delete expired trajectories. Cascades remove normalized participants, ingestion receipts, events, score results, and reports together; referenced gateway, cost, and platform evidence records remain governed by their own retention policies.

Operational checklist

Create Model Gateway calls, cost records, and tool evidence before referencing them.
Use stable participant and event keys from the source system.
Preserve source timestamps and explicit parents; do not infer order from arrival time.
Use one idempotency key per distinct batch and retain it through retries.
Send late participants or parents under the same trajectory key.
Finalize only after the source believes streaming is terminal.
Require complete and zero orphans before scoring or reporting.
Compare hosted and remote variants as separate evidence graphs.
Review retention periods against organization policy.

Failure handling

Response	Meaning	Action
`400 VALIDATION_ERROR`	Body, timestamp, event kind, or size is invalid	Correct the contract; do not retry unchanged
`403 CROSS_ORG_REFERENCE`	A run or evidence reference is missing or outside the organization	Verify tenant-scoped IDs and source credentials
`404 NOT_FOUND`	The requested trajectory or score does not exist in the organization	Refresh the run list or correct the ID
`409 CONFLICT`	Idempotency, event identity, or source binding changed	Reuse the original body or create a new stable key
`409 INCOMPLETE`	Required evidence is unresolved	Ingest durable missing evidence and a late batch
`409 MALFORMED`	Parentage or identity is contradictory	Repair the source event graph under new event keys

Authorization

Operation	Minimum scope or role
List, inspect, compare	`runs:read`
Ingest, finalize, score	member and `runs:write`
Build report	member and `reports:write`
Prune expired trajectories	administrator and `admin:org`

Get Started

Core Concepts

Guides

SDK Reference

Platform

Multi-agent trajectory analysis

What the contract records

Ingest a trajectory

Idempotency and conflicts

Understand completion states

Inspect the explorer

Use the SDK clients

Score a complete trajectory

Build an evidence report

Compare hosted and remote variants

Retention

Operational checklist

Failure handling

Authorization

​What the contract records

​Ingest a trajectory

​Idempotency and conflicts

​Understand completion states

​Inspect the explorer

​Use the SDK clients

​Score a complete trajectory

​Build an evidence report

​Compare hosted and remote variants

​Retention

​Operational checklist

​Failure handling

​Authorization

What the contract records

Ingest a trajectory

Idempotency and conflicts

Understand completion states

Inspect the explorer

Use the SDK clients

Score a complete trajectory

Build an evidence report

Compare hosted and remote variants

Retention

Operational checklist

Failure handling

Authorization