Import OpenAI Agents API runs

The OpenAI Agents API runs the agent loop for you, in an OpenAI-hosted or a self-hosted environment. OpenAI does not push its traces anywhere: you export them per session with GET /v1/agents/sessions/{session_id}/traces. EvalGate reads the saved pages offline into the same evalgate.agent-run.v2 runs as Claude Code, so a view or report can compare them.

This profile is Experimental. OpenAI documents the export page but not the span names or attributes inside it, so EvalGate reads only OpenTelemetry GenAI convention names, and the profile is tested against a synthetic capture, not a real one. It ships in the TypeScript 4.0.0 source candidate of @evalgate/sdk, which is not published yet. EvalGate never calls the OpenAI API and never holds an OpenAI key: you export the pages yourself.

1. Export a session’s traces

Trace export must be enabled for your organization, and the key needs api.traces.read (or api.agents.read) on the session’s project. Keep the key in your shell; EvalGate never reads it. Save each page as one line. Follow has_more and last_id until the last page:

session=sess_123
after=""
: > traces.jsonl
while :; do
  page=$(curl -sS "https://api.openai.com/v1/agents/sessions/$session/traces?limit=20&order=asc${after:+&after=$after}" \
    -H "Authorization: Bearer $OPENAI_API_KEY" -H "OpenAI-Beta: agents=v1")
  printf '%s\n' "$page" | jq -c . >> traces.jsonl
  [ "$(printf '%s' "$page" | jq -r .has_more)" = "true" ] || break
  after=$(printf '%s' "$page" | jq -r .last_id)
done

The export includes only the traces that exist when you call it, so wait until the session is idle. The pages can contain prompts, outputs, tool arguments and tool results; treat the file as sensitive and delete it after the import.

2. Import the pages

npx @evalgate/sdk harness import --from ./traces.jsonl \
  --agent openai-agents-api \
  --runtime-host openai --runtime-hosting vendor_hosted \
  --run-id invoice-triage-1 --tier light --task invoice-triage --apply

--run-id folds every trace in the file into one run. Without it, EvalGate groups spans by gen_ai.conversation.id when present, else by trace id. Use --runtime-hosting self_hosted and --sandbox-provider <id> when the session ran in your own environment; EvalGate never infers either from the traces.

What the run contains

Measure	Where it comes from
Input, output and cached tokens	`chat` spans’ `gen_ai.usage.*` counts. The reported input includes cached tokens, so EvalGate subtracts them to get uncached input.
API requests and errors	One per `chat` span; an error status or `error.type` counts as an API error. A span repeated across pages counts once.
Tool calls and failures	`execute_tool` spans and their status, counted by tool type (`function`, `extension`, `datastore`). Function names are chosen by you and are never kept.
Time	Span start and end times; wall-clock time runs to the end of the last agent span.

OpenAI reports usage best-effort: a null count means unknown, not zero. If any request in the run lacks a count, that run total stays n/a with reason not_exported instead of an undercount. Cost stays n/a because the offline price snapshot covers Anthropic models only. File reads and edits, subagent tokens and depth, time waiting on a person, prompts and refusals also stay not_exported. Messages, system instructions, agent names and descriptions, tool arguments and results, user.id and URLs are dropped while decoding. Every number is self-reported by OpenAI’s export and is not verified against billing data. To compare runs, continue with cost of quality for coding agents.

Get Started

Core Concepts

Guides

SDK Reference

Platform

Import OpenAI Agents API runs

1. Export a session’s traces

2. Import the pages

What the run contains

​1. Export a session’s traces

​2. Import the pages

​What the run contains

1. Export a session’s traces

2. Import the pages

What the run contains