Add test cases
A test case is one input, the expected behavior, and any metadata the scorer needs. You can add cases four ways.Option A — Type one in
On the evaluation’s Test cases tab, click Add case, fill in the input and expected fields, and save.
Option B — Paste many
Click Paste cases and paste rows from a spreadsheet. The first row must be the column headers (for example,input,expected). EvalGate maps the columns to
case fields automatically.
Option C — Import a JSONL file
Click Import, choose a.jsonl file where each line is one case object, and
upload it. This is the fastest path for hundreds of cases.
Option D — Promote a real failure
This is the most valuable source. From the Logs explorer, open a trace that failed, click Promote to case, pick the target evaluation, and confirm. The real production input becomes a test case with its source trace recorded in lineage.What makes a good case
- Specific. A case that always passes teaches you nothing. Pick cases that have failed, or that you fear will fail.
- Labeled. If your scorer needs an expected output, fill it in. Unlabeled cases can still run but cannot be scored deterministically.
- Traceable. Promoted cases keep their source trace, so a future regression points back to the original incident.