> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Dashboards and monitors

> Build immutable metric contracts, evidence-linked dashboards, stateful threshold alerts, and permission-safe shared views.

# Dashboards and monitors

Dashboards turn EvalGate's authoritative evidence into reusable, versioned metric contracts. A dashboard never stores a hand-entered result: every widget references an approved metric definition, every query returns completeness metadata, and every alert transition links back to the metric-cache evidence that caused it.

**Status:** Beta · **Owner:** Evidence & Reporting · **Last verified:** 3.7.0-rc (2026-07-13)

## Start in the control room

Open `/dashboards`. The control room has four work areas:

* **Overview** renders the selected published dashboard. A failed widget stays isolated; healthy sibling widgets remain usable.
* **Dashboard builder** creates a dashboard and its first immutable version from an approved metric.
* **Metric definitions** previews, saves, versions, and approves authoritative metric contracts.
* **Alert rules** creates and manually evaluates stateful thresholds using the same evaluator used by scheduled execution.

Creating or changing definitions, dashboards, shares, or alerts requires the corresponding write scope. Metric approval requires an admin. A personal dashboard is visible only to its owner; project and organization dashboards are visible to organization members, but non-owners see published versions only.

## Define an authoritative metric

A metric definition is an immutable, content-hashed contract with a stable `metricKey` and monotonically increasing version. Define all of the following before saving:

| Contract field       | Behavior                                                                                                                                                            |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Source and measure   | The measure must be supported by its selected authoritative source. Invalid pairs fail validation; EvalGate does not substitute another table or fabricate a value. |
| Aggregation          | `count`, `sum`, `avg`, `min`, `max`, `p50`, `p95`, `p99`, or `rate`. `rate` is accepted only for measures whose contract is a rate.                                 |
| Filters              | Up to 20 allowlisted predicates. Set filters accept 1–100 values. Unknown or non-filterable fields fail explicitly.                                                 |
| Grouping             | Up to three allowlisted fields. Group results retain the same known, unknown, and evidence semantics as the overall result.                                         |
| Window               | Relative windows range from 60 seconds to one year. Absolute windows require ISO timestamps with `to` later than `from`. Both declare a timezone.                   |
| Time grain           | Minute, hour, day, or ISO week. Buckets are evaluated in the declared timezone.                                                                                     |
| Unknown policy       | `exclude` counts unknown rows separately, `include_as_zero` includes them as zero, and `fail` rejects an incomplete result.                                         |
| Calibration boundary | Scored measures must be `exact`, tied to a calibration mapping version, or explicitly `not_comparable`. A non-comparable score cannot be presented as a time trend. |

Supported sources and measures are intentionally finite:

| Source         | Measures                                                                                                     |
| -------------- | ------------------------------------------------------------------------------------------------------------ |
| `traces`       | `trace_count`, `duration_ms`, `error_rate`, `cost_usd`, `judge_score`, `incomplete_provenance_rate`          |
| `model_calls`  | `call_count`, `latency_ms`, `cost_usd`, `input_tokens`, `output_tokens`, `failure_rate`, `policy_block_rate` |
| `eval_runs`    | `run_count`, `pass_rate`, `quality_score`, `cost_usd`, `latency_ms`                                          |
| `online_evals` | `sample_count`, `alert_rate`, `failure_rate`, `cost_usd`, `budget_remaining_usd`                             |
| `experiments`  | `experiment_count`, `quality_delta`, `cost_delta_usd`, `latency_delta_ms`, `regression_rate`                 |
| `review`       | `item_count`, `open_rate`, `agreement_rate`, `resolution_time_ms`                                            |

<Warning>
  Metric approval changes which immutable version is active for a stable key. It never rewrites a version already used by a dashboard, alert, report, or cached query.
</Warning>

## Understand query completeness

Every metric query has one of three statuses:

* `success`: all selected records were comparable under the metric contract.
* `partial`: a valid value exists, but unknown records or another disclosed completeness condition remains.
* `failed`: the source, filter, comparability rule, limit, or query could not produce a valid result.

Results include the value, known count, unknown count, warnings, time series or grouped values when requested, and exact source evidence identifiers. Cached results retain the metric-definition version and query hash; bypassing cache creates a new append-only result rather than mutating old evidence.

Queries are bounded to 10,000 selected source rows. Exceeding that boundary returns a failed result with an explicit query-limit code. It is never represented as a complete aggregate.

## Build and publish a dashboard

A dashboard snapshot contains a 12-column layout, up to 20 global filters, a manual or interval refresh policy, and 1–40 widgets. Each widget declares:

* a unique stable key and one of 13 supported visualization types;
* an approved metric-definition ID;
* position and size on the 12-column grid;
* manual or interval refresh, with intervals from 30 seconds to 24 hours;
* a relative drilldown path to Logs, Experiments, Evaluations, Online Eval, Review, or Costs.

Overlapping positions are normalized deterministically. Publishing freezes the version, layout, widget references, and content hash. Adding, updating, or removing a widget creates a new draft version; it does not change the published version in place.

When a dashboard renders, each widget executes independently. The overall render becomes `partial` if one widget fails while another succeeds. The UI keeps valid values and their evidence links visible and labels the failed widget with its actual error.

## Share a pinned view safely

The dashboard owner can create an expiring view-only link for a published version. The copied URL opens `/dashboards/shared/{token}` and renders the exact version captured by the link—not the owner's latest draft or a later publication.

Share tokens are stored as hashes, expire between five minutes and one year, and can be revoked. They do not grant anonymous or cross-organization access: the viewer must authenticate as a member of the same organization and hold evaluation-read permission. A revoked, expired, malformed, or cross-organization token returns not found or forbidden without disclosing the dashboard.

## Configure stateful alerts

An alert rule references an approved metric definition and an immutable rule version. Operators are `gt`, `gte`, `lt`, `lte`, `eq`, and `outside`. An outside-range rule requires both lower and upper values.

Use `forConsecutive` to require repeated breaches before firing and `resolveAfterConsecutive` to require repeated healthy evaluations before resolution. The evaluator uses these states:

| State               | Meaning                                                                                 |
| ------------------- | --------------------------------------------------------------------------------------- |
| `normal`            | The metric is healthy and no prior breach is resolving.                                 |
| `pending`           | The threshold is breached, but the firing count has not been reached.                   |
| `firing`            | The required consecutive breach count has been reached.                                 |
| `resolved`          | A prior pending or firing condition met its healthy-resolution count.                   |
| `insufficient_data` | The metric has no comparable known value. This is not healthy.                          |
| `error`             | The metric query failed. This is not healthy and does not fabricate a threshold result. |

Transitions are deduplicated by rule version, transition state, and configured time bucket. A durable event retains the metric-cache ID, observed value, prior/new state, threshold contract, and reason. Enabled delivery channels write in-app notifications and webhook delivery attempts; attempts remain inspectable even when delivery fails.

## API workflow

| Purpose              | Operations                                                                                                         |
| -------------------- | ------------------------------------------------------------------------------------------------------------------ |
| Metric registry      | `GET/POST /api/metric-definitions`, `GET/PATCH/DELETE /api/metric-definitions/{id}`, `POST /approve`               |
| Metric execution     | `POST /api/metrics/preview`, `POST /api/metrics/query`                                                             |
| Dashboard registry   | `GET/POST /api/dashboards`, `GET/PATCH/DELETE /api/dashboards/{id}`                                                |
| Versions and widgets | `POST /api/dashboards/{id}/versions`, `POST /versions/{versionId}/publish`, widget create/update/delete operations |
| Rendering            | `POST /api/dashboards/{id}/render`; a `versionId` renders that exact immutable version                             |
| Sharing              | `POST /api/dashboards/{id}/shares`, share revocation, and `GET /api/dashboard-shares/{token}`                      |
| Alerts               | `GET/POST /api/dashboard-alert-rules`, detail/revision/enablement, `POST /evaluate`, and event/delivery reads      |

The TypeScript SDK, Python SDK, and CLI do not yet provide a complete dashboard-and-alert convenience workflow. Use the authenticated REST contract; do not depend on undocumented web-app calls.

## Failure and recovery

| Symptom                                   | What it means                                                                           | Recovery                                                                                    |
| ----------------------------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| Metric returns `INVALID_FILTER`           | A field/operator is not allowlisted for the source.                                     | Remove the unsupported predicate or choose the authoritative source that owns it.           |
| Metric returns `QUERY_LIMIT`              | The selected source set exceeds the 10,000-row bound.                                   | Narrow the time window or filters; do not treat the partial population as complete.         |
| Score is `not_comparable`                 | Its calibration boundary does not permit the requested comparison or trend.             | Select an exact cohort or approve a calibration mapping version.                            |
| Widget is failed but dashboard is partial | That widget failed independently; sibling evidence is still valid.                      | Open the widget error, repair its metric/source, and rerun.                                 |
| Share link is unavailable                 | The token is invalid, expired, revoked, cross-organization, or the viewer lacks access. | Authenticate in the owning organization or ask the owner for a new link.                    |
| Alert is `insufficient_data`              | No known comparable value exists for the window.                                        | Inspect unknown counts and evidence; do not widen delivery thresholds to hide missing data. |
| Alert delivery failed                     | The state transition succeeded, but one or more destinations did not.                   | Inspect delivery attempts, repair the destination, and retry through the delivery workflow. |

## Verification evidence

The release-candidate acceptance evidence covers:

* [metric and alert unit contracts](https://github.com/evalgate/ai-evaluation-platform/tree/main/tests/unit/dashboards) for rates, percentiles, timezones, unknowns, comparability, partial composition, state transitions, and deduplication;
* [authenticated API tests](https://github.com/evalgate/ai-evaluation-platform/blob/main/tests/api/dashboards.routes.test.ts) for scope enforcement, validation, exact-version rendering, and explicit failures;
* [real PostgreSQL invariants](https://github.com/evalgate/ai-evaluation-platform/tree/main/tests/integration/dashboards) for tenant isolation, immutable definitions/versions, authoritative queries, cache history, sharing, alert events, notifications, and delivery attempts;
* [accessible control-room tests](https://github.com/evalgate/ai-evaluation-platform/tree/main/tests/dom/dashboards) for partial rendering, metric authoring, no-code dashboard creation, alert evaluation, and pinned shared views; and
* [authenticated golden path](https://github.com/evalgate/ai-evaluation-platform/blob/main/e2e/dashboard-alert-golden-path.spec.ts) from source trace and saved Logs view through approved metrics, dashboard publication, evidence rendering, threshold firing, and durable delivery.