Create an evaluation
An evaluation is a named set of test cases plus the scorers that decide pass or fail. You will create one, add a couple of cases, and pick a scorer.Step 1 — Open evaluations
In the sidebar, click Evaluations, then New evaluation.
Step 2 — Name it
Give the evaluation a name that says what it checks, for examplecheckout-assistant-hallucination. Add a short description. Click Create.
Step 3 — Pick a model and scorer
On the evaluation page:- Under Model, choose the model this eval runs against.
- Under Scorers, add at least one. Start with a deterministic scorer (for example, “output contains the order ID”) before reaching for an LLM judge.
Model-backed scorers need a connected provider
key.