Skip to main content

Agent framework integrations

EvalGate accepts quality evidence at three boundaries: an SDK wrapper around a runtime call, an OpenTelemetry/OpenInference trace export, or an authenticated MCP/API call. Pick the narrowest boundary your application already supports.
Hosted trace, MCP, and API calls require an EvalGate account, an organization, and a scoped API key. Your EvalGate key does not pay for model inference; model-backed workflows use a provider credential owned and billed by your organization.

Support matrix

The framework packages remain your dependencies. EvalGate’s structural wrappers do not install them or take over their model-provider configuration.

CrewAI

Wrap the crew object after creating an authenticated client and workflow tracer:
The wrapper records workflow and agent-span completion or failure around the existing kickoff call. It does not change the crew’s tools or provider keys.

AutoGen

Use the corresponding conversation wrapper when your runtime exposes initiate_chat:

Vercel AI SDK

The Vercel integration wraps a Language Model v3-compatible object without adding the ai package as an EvalGate runtime dependency:
Trace-delivery failures do not mask the model result. If trace persistence is a release requirement, verify the trace separately before promoting a baseline.

MCP product and documentation servers

Use the public server cards to inspect the tools before connecting:
  • Product card: https://www.evalgate.com/.well-known/mcp/server-card.json
  • Documentation card: https://www.evalgate.com/.well-known/mcp/docs-server-card.json
  • Product transport: https://www.evalgate.com/api/mcp
  • Documentation transport: https://www.evalgate.com/api/mcp/docs
The documentation server requires docs:read. Product tools are filtered by the scopes on the organization API key. See MCP integration and agent authentication.

DSPy and other OpenTelemetry-compatible runtimes

EvalGate does not currently ship a first-class DSPy wrapper. Use an existing OpenTelemetry or OpenInference instrumentation path to export trace evidence, or call the Python SDK explicitly at the application boundary. Keep model calls and tool side effects in your application; EvalGate receives the trace and evaluation evidence you choose to send. See TypeScript SDK: OTLP/OpenInference and trace setup. Do not label a generic trace export as a dedicated framework integration in architecture or procurement documents.

Verify the integration

Before relying on a framework path for a release gate:
  1. Send one successful and one failing run from a non-production environment.
  2. Confirm workflow, agent, model, tool, and error fields preserve the evidence needed by your assertions.
  3. Turn one reviewed failure into a permanent evaluation case.
  4. Run the gate against the exact commit and baseline used for the release.
  5. Confirm provider, budget, parser, and missing-trace failures remain explicit.