> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Create a scorer

> Deterministic, code, composite, or LLM judge.

# Create a scorer

A scorer turns a case's output into a score. EvalGate supports four kinds. Start
simple and add complexity only when you need it.

## Pick the right kind

| Kind              | When to use it                                  | Example                             |
| ----------------- | ----------------------------------------------- | ----------------------------------- |
| **Deterministic** | Exact, rule-based checks                        | "output contains the order ID"      |
| **Code**          | Logic too complex for a rule, but no model call | A regex + parse + compare function  |
| **Composite**     | Combine other scorers with weights              | 0.5 × deterministic + 0.5 × judge   |
| **LLM judge**     | A model grades the output                       | "is this answer safe and on-topic?" |

## Step 1 — Open Scorer Studio

In the sidebar, click **Scorers**, then **New scorer**.

## Step 2 — Define the scorer

Pick the kind, then fill in the definition:

* **Deterministic** — choose a comparator and the expected value.
* **Code** — paste a TypeScript or Python function. Production runs inside the
  OCI sandbox; local runs are for development only.
* **Composite** — add component scorers and weights. Each component references an
  exact version.
* **LLM judge** — write the judge prompt, pick a model, and set the score
  mapping.

## Step 3 — Add examples

On the **Examples** tab, add labeled input/output pairs. These are the
ground-truth cases the scorer must agree with.

## Step 4 — Test

On the **Test** tab, run the scorer against the examples. Every example must
match its label before you submit for review.

## Step 5 — Submit and publish

Submit for review. A reviewer with `scorer:publish` approves the version, which
becomes immutable and content-hashed. Bind it to an evaluation to use it.

<Warning>
  Code scorers run in a restricted, digest-pinned OCI sandbox in production. Do
  not assume your local Node or Python environment matches production exactly.
</Warning>

## Up next

[Calibrate the scorer](./calibrate-scorer) so its scores are
comparable across versions.