EvalGate: CI for AI behavior
EvalGate helps AI teams stop the same failure from shipping twice. Sign in, connect the repository that owns the AI behavior, and keep its exact non-secret context separate from the revocable human CLI session. Then prove one local regression gate before adding traces, evaluation history, LLM judges, reviews, cost controls, and governance. The product is one operating loop: trace to eval to gate. Real behavior produces evidence. Reviewed failures become test cases. Promoted test cases become CI gates that give reviewers evidence before a prompt, model, retriever, or agent change ships.Model-backed workflows are bring your own provider key (BYOK). EvalGate does not bundle model inference credits: connect your own provider or gateway credential, and that provider bills model usage directly. Understand BYOK and model-provider setup.
Quick start
Copy two commands, create your first baseline, and see a local regression gate fail before it reaches CI.
Trace to eval to gate
Understand the operating loop before you wire in platform features.
SDK and CLI
Install the TypeScript or Python SDK and use the same assertions locally, in app code, and in CI.
Framework recipes
Copy setup paths for Node, Next.js, Python, FastAPI, LangChain, and RAG apps.
API reference
Integrate directly with the EvalGate platform for traces, runs, projects, and keys.
Feature status
Check what is Beta or Experimental, which interfaces are supported, and where to find acceptance evidence.
The adoption path
1
Activate one repository and prove one gate
Sign in with GitHub or Google, connect the repository in Setup, complete the human-approved CLI session, and install the SDK. Snapshot the current test/eval health, then add a CI step that fails when the reviewed baseline regresses.
2
Capture real failures
Add tracing when local gates are not enough. EvalGate captures production and staging behavior with inputs, outputs, tool calls, latency, token usage, cost, and metadata.
3
Promote failures into coverage
Convert repeated or high-risk failures into reusable eval cases. Label, cluster, synthesize, and review cases so your suites track actual user pain.
4
Gate releases with evidence
Run the eval suite in CI, compare against the baseline, and give reviewers clear pass/fail evidence before changes merge.
What to use first
EvalGate is currently a controlled beta. Review Feature status before depending on a platform surface in production.Explore next
CI/CD integration
Wire EvalGate into GitHub Actions or GitLab CI to gate every PR.
Tracing setup
Capture the real AI behavior that should become eval coverage.
LLM judge
Add judge-backed scoring when assertions alone are not enough.
Model providers and BYOK
Connect your provider or gateway credential and understand the separate authentication, billing, and data boundaries.
Agent governance
Scale from one gate to governed AI release workflows.