Provider API Keys
To call models from the Playground and run LLM-as-a-Judge evaluators, connect your LLM provider credentials. These are stored per project and encrypted at rest.
Manage provider connections under Settings → LLM Connections (provider API keys). They power the Playground and evaluators — not trace ingestion, which uses your project API keys.
Supported providers
| Provider | Notes |
|---|---|
| OpenAI | API key; optional custom base URL |
| Anthropic | API key |
| AWS Bedrock | Access key + secret; region |
| Google Vertex AI | Service account; project & region |
| Google AI Studio | API key |
| Azure OpenAI | API key + endpoint/base URL |
Connecting a provider
Add a connection
In Settings → LLM Connections, choose a provider and enter its credentials (and any required config — region for Bedrock, project/region for Vertex, endpoint for Azure).
Add custom models (optional)
If you use models beyond the provider's defaults (e.g. a fine-tuned or newly released model), add their identifiers so they appear in model pickers.
Use it
The connection now appears as a model option in the Playground and as a selectable judge model for evaluations.
Credentials are encrypted at rest and never returned in full after saving. Use least-privilege keys, and rotate them per your security policy. Restrict who can manage connections via Roles & Access.
Next steps
- Playground — test prompts against your connected models.
- LLM-as-a-Judge — pick a connected model as the grader.