---
title: "How to Compare Two AI Agent Trajectories"
description: "Find the first meaningful divergence in plans, tools, evidence, retries, costs, and outcomes."
canonical: "https://www.evalgate.com/guides/compare-ai-agent-trajectories"
date-published: "2026-09-02"
last-updated: "2026-09-02"
category: "Tools and trajectories"
keywords: "compare AI agent trajectories, agent run diff, trajectory evaluation"
---

# How to Compare Two AI Agent Trajectories

Find the first meaningful divergence in plans, tools, evidence, retries, costs, and outcomes.

**Question:** How do I compare a baseline and candidate AI agent run?

## Short answer

Normalize both trajectories into ordered events, align them by task milestones rather than raw token position, and find the first meaningful divergence. Compare tool choices, arguments, evidence, errors, recovery, cost, and final state; allow multiple valid paths when the contract permits them.

## Key takeaways

- Align on semantic milestones, not identical hidden reasoning.
- The first divergence is more actionable than the final difference.
- Compare only runs with compatible task and environment evidence.

## Signs this is the problem

- Reviewers place two raw transcripts side by side manually.
- Different but valid tool order is marked as failure.
- A candidate looks cheaper because it stopped before completing the task.

## Step-by-step approach

### 1. Check comparability

Confirm task, inputs, environment, permissions, source versions, and outcome contracts match.

### 2. Normalize events

Represent models, tools, handoffs, approvals, state changes, errors, and evidence with stable types and IDs.

### 3. Align milestones

Match equivalent goals and state transitions while preserving extra, missing, or reordered steps.

### 4. Explain the decision

Connect divergences to quality, safety, latency, cost, and verified final outcome.

## What to measure

| Metric | What it measures | How to use it |
| --- | --- | --- |
| Milestone match | Required states reached by both runs, including order constraints where meaningful. | Allow flexible paths when the product contract allows them. |
| First-divergence location | Earliest event where the candidate departs from expected evidence or action. | Use it to select the owning layer for a fix. |
| Outcome-adjusted efficiency | Resource use among trajectories that satisfy the same outcome. | Do not reward incomplete candidates. |

## Common mistakes

- Comparing private chain-of-thought text.
- Assuming identical event counts mean equivalent behavior.
- Diffing runs from different data or permission states.

## Practical checklist

- [ ] Check comparability: Confirm task, inputs, environment, permissions, source versions, and outcome contracts match.
- [ ] Normalize events: Represent models, tools, handoffs, approvals, state changes, errors, and evidence with stable types and IDs.
- [ ] Align milestones: Match equivalent goals and state transitions while preserving extra, missing, or reordered steps.
- [ ] Explain the decision: Connect divergences to quality, safety, latency, cost, and verified final outcome.

## Where EvalGate fits

EvalGate's trajectory workspace reconstructs ordered evidence and compares selected baseline and candidate runs with completeness, score, variant, and cost context.

[Open trajectory analysis](https://www.evalgate.com/docs/platform/trajectory-analysis) or [start with the EvalGate quickstart](https://www.evalgate.com/docs/quickstart).

## Frequently asked questions

### What should I do first?

Confirm task, inputs, environment, permissions, source versions, and outcome contracts match

### How should the result be measured?

Required states reached by both runs, including order constraints where meaningful. Allow flexible paths when the product contract allows them.

### When is the change ready to ship?

Ship only after the protected cases pass, the primary metric clears its agreed boundary, and the team reviews the remaining failure modes instead of relying on one aggregate score.

## Related guides

- [https://www.evalgate.com/guides/investigate-ai-agent-failures](https://www.evalgate.com/guides/investigate-ai-agent-failures)
- [https://www.evalgate.com/guides/evaluate-multi-step-ai-agents](https://www.evalgate.com/guides/evaluate-multi-step-ai-agents)
- [https://www.evalgate.com/guides/balance-ai-quality-latency-cost](https://www.evalgate.com/guides/balance-ai-quality-latency-cost)