> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluate your subagents

> Compare final task quality, delegation completeness, and total resource use across context strategies.

Do your subagents improve the final result, or only add work? Evaluate the
whole task, including supervision, handoffs, retries, and final synthesis.
Ten successful children do not count as ten successful tasks.

<Warning>
  This local workflow requires a build containing the delegation evidence
  extension. The fixture is synthetic and makes no model calls. It does not
  demonstrate that any strategy improves a real model, automatically observe
  private subagents in a chat, or certify a provider's native fork behavior.
</Warning>

## Choose the question before the strategy

Keep roles separate from context delivery and experimental treatments:

| Role       | Question to test                                                                            |
| ---------- | ------------------------------------------------------------------------------------------- |
| Worker     | Does inherited context reduce repeated investigation without losing requirements?           |
| Reviewer   | Does independent context improve defect detection without increasing false positives?       |
| Researcher | Is a focused task sufficient to answer a self-contained question accurately?                |
| Supervisor | Does delegation improve the final artifact after accounting for coordination and synthesis? |

Compare the same frozen tasks across four treatments:

1. **No delegation:** the root does the work and explicitly closes an empty fan-out.
2. **Isolated delegation:** children receive complete instructions and necessary evidence.
3. **Inherited-context delegation:** use only when the runtime supports and reports inheritance.
4. **Compact handoff:** an isolated child receives a bounded summary and necessary artifacts.

`isolated` and `fork` are the native modes described in
[LangChain's context article](https://www.langchain.com/blog/organizing-context-in-a-multi-agent-harness).
Compact handoff is a treatment, not another native mode. A requested fork is
not evidence that conversation inheritance occurred.

## Run the deterministic recipe

Build and install a package containing this extension into a temporary consumer
directory. The repository's `examples/evaluate-subagents/README.md` gives the
tarball and wheel build steps. With the TypeScript package installed there:

```bash theme={null}
node /path/to/repo/examples/evaluate-subagents/recipe.mjs \
  --cli ./node_modules/@evalgate/sdk/dist/cli/index.js --output ./complete
node /path/to/repo/examples/evaluate-subagents/recipe.mjs \
  --cli ./node_modules/@evalgate/sdk/dist/cli/index.js \
  --scenario missing-child --output ./incomplete
```

Use `--python /path/to/venv/bin/python` instead of `--cli` for an installed
Python wheel. The generator uses Node for identical fixture inputs; actual
ingestion and reporting use the selected package's CLI.

Each run creates `frame.json`, metrics-only observations, CLI receipts, and
`report.json`. The complete fixture has 12 expected root trials and nine child
observations. The missing-child case omits three children without shrinking
the expected frame. Other scenarios exercise failed roots despite passing
children, lost constraints, biased reviews and stale handoffs.

For an explicit metadata hook, submit one record on stdin and analyze it against
the independently frozen frame:

```bash theme={null}
evalgate harness record --adapter generic-jsonl --input - --format json
evalgate harness optimize --input .evalgate/harness-events.jsonl \
  --frame frame.json --output report.json --format json
```

The outer metrics schema stays at version 1; the optional `delegation` extension
has its own version 1. Existing flat sessions remain valid but do not become
proven root tasks. Inspect `delegationAnalysis.rootTrials`, `treatments`,
`comparisons`, and `decisionBindingHash`. A complete descriptive comparison
still has `eligible: false`; it does not fabricate an inferential interval.

For the synthetic complete control, the report shows:

| Treatment                    | Expected / complete roots | Known cost | Cost per successful root |
| ---------------------------- | ------------------------- | ---------- | ------------------------ |
| No delegation                | 3 / 3                     | \$0.03     | \$0.01                   |
| Isolated delegation          | 3 / 3                     | \$0.06     | \$0.02                   |
| Inherited-context delegation | 3 / 3                     | \$0.06     | \$0.02                   |
| Compact handoff              | 3 / 3                     | \$0.06     | \$0.02                   |

These are fixed fixture counters, not measured model economics. Isolated and
compact comparisons contain three matched root trials across three tasks;
the inherited comparison is inconclusive because native context attribution
is unverified. Every comparison has `eligible: false` and `interval: null`.
The missing-child fixture retains all 12 expected roots, marks three trees
incomplete, and withholds their total-cost and savings claims. The separate
`synthetic-evaluations.json` file records synthetic protected-constraint retention,
reviewer defect detection/false positives and current-artifact checks.

## Preserve the task boundary

### Minimal parent and child

The runnable fixture contains a parent with a closed fan-out issuing one required
`delegationId`, and a child with that same edge ID and `parentSessionId`.
Both bind the same task version, root execution, variant and logical trial.
The child has its own session and attempt IDs. To inspect the smallest complete
tree, run this in the repository root:

```js theme={null}
// Save as mini.mjs in the repository root, then run: node mini.mjs
import fs from "node:fs";
const base = "examples/evaluate-subagents/fixtures/";
const frame = JSON.parse(fs.readFileSync(base + "frame.json", "utf8"));
const root = frame.roots.find(r => r.treatment === "isolated_delegation");
const rows = fs.readFileSync(base + "complete.jsonl", "utf8").trim()
  .split("\n").map(JSON.parse)
  .filter(r => r.delegation.rootExecutionId === root.rootExecutionId);
fs.writeFileSync("mini-frame.json", JSON.stringify({...frame, roots: [root]}));
fs.writeFileSync("mini.jsonl", rows.map(r => JSON.stringify(r)).join("\n"));
```

```bash theme={null}
evalgate harness capture --adapter generic-jsonl --input mini.jsonl --output mini-recorded.jsonl
evalgate harness optimize --input mini-recorded.jsonl --frame mini-frame.json --format json
```

This yields one root trial and two physical attempts, with no matched baseline
comparison. Keep the complete four-treatment frame for comparing strategies.

Freeze the task versions and intended root executions before collecting
observations. Record issued child work before waiting for its result. Close
each parent's fan-out explicitly when no more children will be issued.
Otherwise a missing child—or an entirely missing root—could disappear from
the comparison.

The root's final evaluated outcome determines task quality. Required child
work must be resolved for the evidence to be complete. A cancelled or missing
child is different from a completed child with an attributable behavioral
failure. Optional work must be declared by the issuer; a missing result cannot
retroactively declare itself optional.

Record physical retries separately and deduplicate retransmitted observations.
Never sum repeated cumulative snapshots. Repetitions of one task and child
attempts do not increase the number of independent tasks.

## Interpret cost and latency

Exclusive session usage can be added across the task tree. Inclusive root
totals already contain descendant usage and must not be added to it again.
Include unsuccessful attempts, tools, context preparation and final synthesis
in the reported coverage. If the exporter cannot account for a component, mark
it unknown rather than zero.

Known subtotal and complete total are different results. Cached input tokens
are not automatically free. Root elapsed time is distinct from the sum of
parallel child durations. Cost per successful task includes unsuccessful work
in the intended cohort and is undefined when no verified root succeeds.

## Capture only what your runtime exposes

Use an explicit hook or reviewed export. Do not read private session stores or
scrape transcripts to fill missing fields. Keep IDs bounded and opaque; never
put prompts, answers, tool arguments, source snapshots, credentials or private
paths in metrics JSONL.

| Source                       | What it establishes                                                             |
| ---------------------------- | ------------------------------------------------------------------------------- |
| Manual declaration           | What a person reports; not independently verified runtime behavior              |
| Adapter report               | What the exporter reports; an adapter name does not certify inheritance         |
| Execution evidence reference | A reference for verification; a hash alone does not prove its contents or truth |
| Unknown                      | The runtime did not expose enough information; preserve this limitation         |

Codex, Claude Code, Cursor and CoWork hook names do not guarantee that any of
them exports every delegation field. A generic metadata export can exercise
the contract without pretending to be a native provider integration.
Conversation isolation also does not isolate shared files, tools, caches or
external state. An isolated reviewer can still encounter a biased shared note.

The existing `harness record` hook also accepts a bounded, hook-only
`traceLineage: { traceId, spanId, parentSpanId? }` projection from an existing
WorkflowTracer or OpenTelemetry span. It maps to `rootExecutionId`, `sessionId`
and `parentSessionId` respectively and rejects conflicting supplied identities.
Only those IDs are accepted: do not pass the whole span or its metadata.
The exporter must separately supply the required delegation contract, issued
fan-out, attempts, actual context observations and metrics. This mapping does
not infer outcome, inheritance, role, or cost coverage from a span.

If a session ID is reused across supervisor retries, provide `parentAttemptId`
when the same delegation edge is present in more than one parent attempt.
Ambiguous parentage is invalid evidence. Multiple inclusive root attempts yield
an observed resource lower bound, not a verified total; their end-to-end elapsed
time remains unknown unless an execution-wide measurement is available.

## Design a fair live experiment

Keep models/settings, tools/permissions, evaluator identity, task versions and
budget policy fixed. Record intended scheduling and fan-out differences.
Interleave or randomize treatments and describe warm/cold cache conditions.
Do not give one arm privileged requirements or leak held-out answers into any
arm's inputs.

Score protected-constraint retention, final correctness, reviewer true and
false positives, and evidence completeness before efficiency. Agreement with
the supervisor is not reviewer quality. Measure duplicate investigation only
when the runtime directly exposes it.

A compact handoff should include the objective, protected constraints,
verified facts, artifact versions, unresolved questions and completion
criteria. Keep its actual text in your own permitted workspace. The metrics
channel carries only its permitted reference and observed counters. A stale
reference or a dropped requirement still needs an actual task assertion;
hash equality does not certify semantic correctness.

Local descriptive comparisons are exploratory. They do not grant approval,
publish a winner, or change your delegation configuration. Missing evidence,
unknown resource coverage, hard failures and unverified context claims must
remain visible even when an observed subtotal looks cheaper.

For existing instrumentation, see [TypeScript workflows](/docs/sdk/typescript),
[Python workflows](/docs/sdk/python), and
[multi-agent trajectories](/docs/platform/trajectory-analysis).