> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Tracing setup

# Instrument your LLM app with EvalGate tracing

> Add distributed tracing to your LLM app to track every call, measure latency, monitor token usage, and debug failures before they affect users.

Distributed tracing gives you full visibility into your AI application's behavior in production. Every LLM call, retrieval step, and tool invocation becomes a searchable, filterable event with timing, token counts, and cost data attached. This guide walks you through installing the SDK, creating traces and spans, nesting spans for multi-step workflows, and following tracing best practices.

## Install the SDK

<CodeGroup>
  ```bash TypeScript theme={null} theme={null}
  npm install @evalgate/sdk
  # or
  yarn add @evalgate/sdk
  # or
  pnpm add @evalgate/sdk
  ```

  ```bash Python theme={null} theme={null}
  pip install evalgate-sdk
  ```
</CodeGroup>

## Set environment variables

Create a `.env` file in your project root and add your credentials:

```bash .env theme={null} theme={null}
EVALGATE_API_KEY=sk_test_your_api_key_here
EVALGATE_ORGANIZATION_ID=00000000-0000-4000-8000-000000000001
```

Get your API key from the [Developer Dashboard](https://evalgate.com/developer).

## Initialize the client and tracer

<CodeGroup>
  ```typescript TypeScript theme={null} theme={null}
  import { AIEvalClient, WorkflowTracer } from '@evalgate/sdk'

  const client = new AIEvalClient({
    apiKey: process.env.EVALGATE_API_KEY
  })

  const tracer = new WorkflowTracer(client)
  ```

  ```python Python theme={null} theme={null}
  import os
  from evalgate_sdk import AIEvalClient, WorkflowTracer

  client = AIEvalClient(api_key=os.environ["EVALGATE_API_KEY"])
  tracer = WorkflowTracer(client)
  ```
</CodeGroup>

## Create traces

A trace represents one logical operation — a user query, a support ticket, or a content generation request. Create a trace with a descriptive name and attach metadata that will help you filter it later:

<CodeGroup>
  ```typescript TypeScript theme={null} theme={null}
  const trace = await client.traces.create({
    name: 'Customer Support Query',
    traceId: 'trace-' + Date.now(),
    metadata: {
      userId: 'user_123',
      sessionId: 'session_456'
    }
  })
  ```

  ```python Python theme={null} theme={null}
  import time
  from evalgate_sdk.types import CreateTraceParams

  trace = await client.traces.create(CreateTraceParams(
      name="Customer Support Query",
      trace_id=f"trace-{int(time.time() * 1000)}",
      metadata={"userId": "user_123", "sessionId": "session_456"}
  ))
  ```
</CodeGroup>

## Add spans

Spans represent individual steps within a trace — an LLM call, a vector search, or a function execution. Attach each span to its parent trace:

<CodeGroup>
  ```typescript TypeScript theme={null} theme={null}
  const span = await client.traces.createSpan(trace.id, {
    name: 'LLM Call',
    spanId: 'span-' + Date.now(),
    type: 'llm',
    startTime: new Date().toISOString(),
    input: userQuery,
    output: response,
    metadata: {
      model: 'gpt-5.2-chat-latest',
      tokens: 150
    }
  })
  ```

  ```python Python theme={null} theme={null}
  from datetime import datetime
  from evalgate_sdk.types import CreateSpanParams

  span = await client.traces.create_span(trace.id, CreateSpanParams(
      name="LLM Call",
      span_id=f"span-{int(time.time() * 1000)}",
      type="llm",
      start_time=datetime.now().isoformat(),
      input=user_query,
      output=response,
      metadata={"model": "gpt-5.2-chat-latest", "tokens": 150}
  ))
  ```
</CodeGroup>

## Nested spans for multi-step workflows

For pipelines with multiple sequential steps — like a RAG workflow with embedding, retrieval, and generation — use `traceWorkflowStep` to create properly nested spans automatically:

<CodeGroup>
  ```typescript TypeScript theme={null} theme={null}
  import { traceWorkflowStep } from '@evalgate/sdk'

  await tracer.startWorkflow('RAG Pipeline')

  const embedding = await traceWorkflowStep(tracer, 'embed-query', async () => {
    return await openai.embeddings.create({ /* ... */ })
  })

  const docs = await traceWorkflowStep(tracer, 'retrieve-docs', async () => {
    return await vectorDb.search(embedding)
  })

  const response = await traceWorkflowStep(tracer, 'generate-response', async () => {
    return await openai.chat.completions.create({ /* ... */ })
  })

  await tracer.endWorkflow({ status: 'success' })
  ```

  ```python Python theme={null} theme={null}
  from evalgate_sdk.workflows import trace_workflow_step

  await tracer.start_workflow("RAG Pipeline")

  embedding = await trace_workflow_step(
      tracer, "embed-query",
      lambda: openai.embeddings.create(...)
  )

  docs = await trace_workflow_step(
      tracer, "retrieve-docs",
      lambda: vector_db.search(embedding)
  )

  response = await trace_workflow_step(
      tracer, "generate-response",
      lambda: openai.chat.completions.create(...)
  )

  await tracer.end_workflow({"status": "success"})
  ```
</CodeGroup>

## Adding custom metadata

Attach business context to traces to make them filterable and useful for debugging:

<CodeGroup>
  ```typescript TypeScript theme={null} theme={null}
  await tracer.startWorkflow('content-generation', undefined, {
    userId: user.id,
    contentType: 'blog-post',
    targetAudience: 'developers',
    keywords: ['AI', 'evaluation', 'testing']
  })

  const span = await tracer.startAgentSpan('ContentAgent', { input: '...' })
  // Your LLM call here
  await tracer.endAgentSpan(span, { result: '...' })

  await tracer.endWorkflow({ status: 'success' })
  ```

  ```python Python theme={null} theme={null}
  await tracer.start_workflow("content-generation", metadata={
      "userId": user.id,
      "contentType": "blog-post",
      "targetAudience": "developers",
      "keywords": ["AI", "evaluation", "testing"]
  })

  span = await tracer.start_agent_span("ContentAgent", {"input": "..."})
  # Your LLM call here
  await tracer.end_agent_span(span, {"result": "..."})

  await tracer.end_workflow({"status": "success"})
  ```
</CodeGroup>

## What gets tracked automatically

Every trace and span captures the following without any extra code:

| Field              | Description                                         |
| ------------------ | --------------------------------------------------- |
| **Input / Output** | Full prompts and model responses                    |
| **Timing**         | Start time, end time, and total latency             |
| **Tokens**         | Input tokens, output tokens, and estimated cost     |
| **Model**          | Model name, version, and parameters                 |
| **Metadata**       | User ID, session ID, and any custom tags you attach |
| **Errors**         | Stack traces and error messages on failure          |

## Viewing traces

Once your application is instrumented, open the [Traces](https://evalgate.com/traces) page in your dashboard to:

* Search and filter traces by metadata, tags, or time range
* View detailed timelines showing nested spans
* Analyze token usage and costs across requests
* Debug failures with full stack traces
* Identify latency bottlenecks across pipeline steps

## Best practices

<CardGroup cols={2}>
  <Card title="Use descriptive names" icon="tag">
    Name traces after the user action, not the implementation. `customer-support-query` is more useful than `llm-call`.
  </Card>

  <Card title="Attach relevant metadata" icon="database">
    Include `userId`, `sessionId`, environment, and feature flags so you can slice and debug traces effectively.
  </Card>

  <Card title="Sample for high volume" icon="filter">
    For high-throughput applications, configure sampling to trace 10–20% of requests rather than every call.
  </Card>

  <Card title="Never log PII" icon="shield">
    Anonymize or redact sensitive user data before it appears in trace inputs, outputs, or metadata fields.
  </Card>
</CardGroup>

## Troubleshooting

**Traces not appearing in the dashboard?**

Verify your `EVALGATE_API_KEY` is correct and that `AIEvalClient` is initialized before any traces are created.

**Noticing added latency?**

The SDK adds roughly 10ms of overhead. Make sure you are not `await`-ing trace upload calls in the critical path — they run asynchronously by default.

**Spans missing data?**

Ensure every `async` function inside a `traceWorkflowStep` callback is properly `await`-ed. Unawaited promises can resolve after the span closes.