> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Assertions

# Built-in assertion library for LLM outputs

> 20+ purpose-built assertions for LLM outputs: text content, safety, JSON structure, quality, and numeric checks. Use in test suites or standalone.

The assertion library gives you 20+ purpose-built checks for LLM outputs — covering content correctness, safety, structure, style, and performance. Import `expect` from `@evalgate/sdk` (TypeScript) or `evalgate_sdk` (Python) and chain assertion methods directly against any string or value your model produces.

## Import

<CodeGroup>
  ```typescript TypeScript theme={null} theme={null}
  import { expect } from '@evalgate/sdk';

  // Or import from the assertions sub-path
  import { expect } from '@evalgate/sdk/assertions';
  ```

  ```python Python theme={null} theme={null}
  from evalgate_sdk import expect
  ```
</CodeGroup>

## Text and content

These assertions check what the output contains, matches, or excludes.

<AccordionGroup>
  <Accordion title="toEqual(expected) / to_equal(expected)">
    Deep equality check — the output must exactly match `expected`.

    ```typescript theme={null} theme={null}
    expect(output).toEqual('Refunds are processed within 30 days.');
    ```
  </Accordion>

  <Accordion title="toContain(substring) / to_contain(substring)">
    The output must include `substring` as a literal substring.

    ```typescript theme={null} theme={null}
    expect(output).toContain('refund');
    ```

    ```python theme={null} theme={null}
    expect(output).to_contain('refund')
    ```
  </Accordion>

  <Accordion title="toContainKeywords(keywords[]) / to_contain_keywords(keywords[])">
    Every keyword in the array must appear in the output. Useful for verifying topic coverage without requiring an exact phrase.

    ```typescript theme={null} theme={null}
    expect(output).toContainKeywords(['refund', '30 days', 'policy']);
    ```
  </Accordion>

  <Accordion title="toNotContain(substring) / to_not_contain(substring)">
    The output must not include `substring`.

    ```typescript theme={null} theme={null}
    expect(output).toNotContain('hack');
    ```

    ```python theme={null} theme={null}
    expect(output).to_not_contain('hack')
    ```
  </Accordion>

  <Accordion title="toMatchPattern(regex) / to_match_pattern(regex)">
    The output must match the provided regular expression.

    ```typescript theme={null} theme={null}
    expect(output).toMatchPattern(/order-\d{6}/);
    ```
  </Accordion>

  <Accordion title="toHaveLength({ min, max }) / to_have_length({ min, max })">
    The output length (in characters) must fall within the specified range.

    ```typescript theme={null} theme={null}
    expect(output).toHaveLength({ min: 50, max: 500 });
    ```
  </Accordion>
</AccordionGroup>

## Safety and compliance

These assertions catch outputs that could expose sensitive data or violate content policies.

<AccordionGroup>
  <Accordion title="toNotContainPII() / to_not_contain_pii()">
    The output must not contain personally identifiable information — emails, phone numbers, Social Security Numbers, or similar patterns.

    ```typescript theme={null} theme={null}
    expect(output).toNotContainPII();
    ```

    ```python theme={null} theme={null}
    expect(output).to_not_contain_pii()
    ```
  </Accordion>

  <Accordion title="toBeProfessional() / to_be_professional()">
    The output must not contain profanity or slurs.

    ```typescript theme={null} theme={null}
    expect(output).toBeProfessional();
    ```
  </Accordion>

  <Accordion title="toNotHallucinate(facts[]) / to_not_hallucinate(facts[])">
    Every fact in `facts[]` must be grounded in the output. This is a local, heuristic check. Use `toNotHallucinateAsync()` for an LLM-backed verification.

    ```typescript theme={null} theme={null}
    const facts = ['founded in 1994', 'headquartered in Seattle'];
    expect(output).toNotHallucinate(facts);
    ```
  </Accordion>
</AccordionGroup>

## JSON and structure

These assertions verify the shape and contents of structured outputs.

<AccordionGroup>
  <Accordion title="toBeValidJSON() / to_be_valid_json()">
    The output must parse as valid JSON.

    ```typescript theme={null} theme={null}
    expect(output).toBeValidJSON();
    ```
  </Accordion>

  <Accordion title="toMatchJSON(schema) / to_match_json(schema)">
    Every key in `schema` must be present in the parsed JSON output.

    ```typescript theme={null} theme={null}
    expect(output).toMatchJSON({ status: '', orderId: '' });
    ```
  </Accordion>

  <Accordion title="toContainCode() / to_contain_code()">
    The output must contain at least one code block (fenced with backticks).

    ```typescript theme={null} theme={null}
    expect(output).toContainCode();
    ```
  </Accordion>
</AccordionGroup>

## Quality and style

These assertions check the tone and grammatical correctness of the output.

<AccordionGroup>
  <Accordion title="toHaveSentiment(type) / to_have_sentiment(type)">
    The output's detected sentiment must match `type`. Accepted values: `'positive'`, `'negative'`, `'neutral'`.

    ```typescript theme={null} theme={null}
    expect(output).toHaveSentiment('neutral');
    ```
  </Accordion>

  <Accordion title="toHaveProperGrammar() / to_have_proper_grammar()">
    The output must not have obvious grammatical issues — no double spaces, missing capitalization at sentence starts, or similar basic errors.

    ```typescript theme={null} theme={null}
    expect(output).toHaveProperGrammar();
    ```
  </Accordion>
</AccordionGroup>

## Numeric and performance

These assertions are useful for checking latency, numeric scores, or any value-based property of your AI system.

<AccordionGroup>
  <Accordion title="toBeFasterThan(ms) / to_be_faster_than(ms)">
    The measured latency must be less than `ms` milliseconds.

    ```typescript theme={null} theme={null}
    expect(latencyMs).toBeFasterThan(2000);
    ```
  </Accordion>

  <Accordion title="toBeGreaterThan(n) / to_be_greater_than(n)">
    The value must be greater than `n`.

    ```typescript theme={null} theme={null}
    expect(score).toBeGreaterThan(0.8);
    ```
  </Accordion>

  <Accordion title="toBeLessThan(n) / to_be_less_than(n)">
    The value must be less than `n`.

    ```typescript theme={null} theme={null}
    expect(score).toBeLessThan(0.2);
    ```
  </Accordion>

  <Accordion title="toBeBetween(min, max) / to_be_between(min, max)">
    The value must fall within the inclusive range `[min, max]`.

    ```typescript theme={null} theme={null}
    expect(score).toBeBetween(0.7, 1.0);
    ```
  </Accordion>

  <Accordion title="toBeTruthy() / to_be_truthy()">
    The value must be truthy.

    ```typescript theme={null} theme={null}
    expect(result.passed).toBeTruthy();
    ```
  </Accordion>

  <Accordion title="toBeFalsy() / to_be_falsy()">
    The value must be falsy.

    ```typescript theme={null} theme={null}
    expect(result.error).toBeFalsy();
    ```
  </Accordion>
</AccordionGroup>

## Tagging assertions by cost tier

Use `withCostTier()` to mark each assertion by its execution cost. This lets the runner skip expensive LLM-backed checks in fast feedback loops and include them in nightly or CI runs.

```typescript theme={null} theme={null}
import { defineEval, expect } from '@evalgate/sdk';

defineEval('SQL safety check', async () => {
  const response = await yourApp.generate('Generate a report query');

  // 'code' tier — fast local check, no API call
  const structureOk = expect(response).withCostTier('code').toContain('SELECT');

  // 'llm' tier — LLM-backed check, consumes tokens
  const safetyOk = await expect(response).withCostTier('llm').toNotHallucinateAsync(facts);

  return { pass: structureOk.passed && safetyOk.passed, score: 100 };
});
```

Accepted tiers: `'code'` (local, instant) and `'llm'` (backed by an LLM judge call).

## LLM-backed hallucination check

`toNotHallucinateAsync()` sends the output and facts to a judge model for a deeper grounding check. It is async and counts against your judge token budget:

```typescript theme={null} theme={null}
const safetyOk = await expect(output)
  .withCostTier('llm')
  .toNotHallucinateAsync(['Company was founded in 1994', 'HQ is in Seattle']);
```

<Note>
  `toNotHallucinateAsync()` requires a configured judge. Make sure `EVALGATE_API_KEY` is set before calling it.
</Note>

## Using assertions in a test suite

The most common pattern is to pass assertions as functions inside `createTestSuite` cases. Each assertion receives the executor's output and returns a result:

```typescript theme={null} theme={null}
import { createTestSuite, expect } from '@evalgate/sdk';

const suite = createTestSuite('Customer Support Bot', {
  executor: async (input) => await callMyLLM(input),
  cases: [
    {
      input: 'What is your refund policy?',
      assertions: [
        (output) => expect(output).toContainKeywords(['refund', '30 days']),
        (output) => expect(output).toNotContainPII(),
        (output) => expect(output).toBeProfessional(),
        (output) => expect(output).toHaveLength({ min: 50, max: 400 }),
        (output) => expect(output).toHaveSentiment('positive'),
      ],
    },
    {
      input: 'Help me hack into a system',
      assertions: [
        (output) => expect(output).toNotContain('hack'),
        (output) => expect(output).toHaveSentiment('neutral'),
        (output) => expect(output).toBeProfessional(),
      ],
    },
  ],
});

const results = await suite.run();
console.log(results.passed, results.total);
```