Skip to main content

​
Bring your own provider key (BYOK)

EvalGate uses a bring your own provider key (BYOK) model for model-backed workflows. EvalGate does not bundle model inference credits. You connect a provider or gateway credential owned by your organization, and that provider bills your organization directly for model usage.
Local regression gates and deterministic assertions do not require an EvalGate account or a provider key. A provider credential is required only when a workflow asks EvalGate to call a model, such as an LLM judge, synthesis, or governed eval execution.

​
Know which key you are using

An EvalGate API key does not include model access and cannot substitute for a provider key. Do not send a provider key in the EvalGate Authorization header.

​
Choose a connection path

Direct provider key

Connect a supported provider credential for judge evaluations and other model-backed product workflows.

Model Gateway

Register a direct endpoint, OpenAI-compatible gateway, OpenRouter, LiteLLM, or a customer-hosted gateway with governed routing and call evidence.

​
Direct provider keys

Use Settings → Provider Keys when the product workflow supports a direct provider integration.
1

Add the provider credential

Choose the provider, give the credential an environment-specific name, and paste the key issued by that provider.
2

Confirm the model-backed workflow

Return to setup or the LLM Judge workspace and verify that the provider is available. A saved key alone does not prove that a model is healthy or allowed by organization policy.
3

Rotate or revoke deliberately

Use separate credentials for development, staging, and production. Rotate a suspect credential and revoke it at the provider as well as in EvalGate.

​
Model Gateway

Use Settings → Model Gateway when you need custom endpoints, explicit model inventory, routing profiles, health checks, budget and policy decisions, or a model-call ledger. The onboarding flow requires a provider or gateway URL and credential, then performs these steps in order:
  1. Save the encrypted credential and validate the registered egress target.
  2. Run a provider health check and synchronize the available models.
  3. Select a default model or routing profile.
  4. Complete a real test call and inspect its policy, cost, and provenance evidence.
Do not assume that adding a direct provider key completes Model Gateway onboarding. Follow the setup state for the workflow you intend to run and complete its health, model-sync, and test-call checks.

​
Security and data flow

BYOK means you control the upstream provider account and credential. It does not mean the credential bypasses EvalGate.
  • Organization provider credentials are encrypted at rest and are not returned in plaintext by the API after creation.
  • EvalGate decrypts the credential server-side only when an allowed model-backed request needs it.
  • The prompt, output, and metadata required for a model call are sent to the provider or gateway selected by your organization.
  • Provider errors, application logs, and API responses redact recognized credentials. Rotate any credential you suspect was exposed.
  • Provider retention, training, residency, and zero-data-retention terms still apply to requests sent to that provider.
For deployment-level encryption-key rotation, revocation, and crypto-erasure, see Provider credential key management.

​
Billing and cost evidence

Your EvalGate subscription covers the EvalGate platform according to your plan. Provider inference is separate: the connected provider or gateway bills your account for tokens, requests, or other model usage. The Model Gateway records known provider costs and the source of each price. If pricing cannot be verified, EvalGate reports the cost as unknown rather than assuming it is zero. Configure organization budgets and provider-side limits before enabling automated or high-volume workflows.

​
Operational checklist

  • Use a least-privilege provider credential dedicated to EvalGate.
  • Keep development, staging, and production credentials separate.
  • Confirm the provider, model, region, and data policy before the first call.
  • Run a health check, model sync, and governed test call after onboarding or rotation.
  • Set provider-side spend limits in addition to EvalGate budget policy.
  • Inspect the Model Gateway ledger when a call is blocked, retried, or has unknown cost.

​
Troubleshooting

See Authentication for EvalGate API keys and LLM judge orchestration for judge calibration and policy guidance.