mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-08 22:21:35 +00:00
ce2582e9d0
* feat(terraform): vendor terraform-provider-litellm as source of truth with endpoint drift CI * fix(terraform): address review feedback on vendored provider Replace deprecated io/ioutil with io. Remove the unused org/team CRUD client methods so the endpoint audit only tracks live call sites (54 -> 46). Redact request/response logs by parsing the JSON and recursively masking sensitive fields, which fixes the nested-object leak in the old credential_values regex, with a regex fallback for non-JSON payloads; covered by new unit tests. Docs: stop showing api_key inside vector store litellm_params and document that Sensitive attributes still persist in plaintext state, recommending litellm_credential_name and an encrypted state backend. * fix(terraform): stop persisting server-returned litellm_params into vector store state The vector store Read wrote litellm_params straight back from the API response into state. The proxy redacts secrets in those responses, so the readback overwrote user config with redaction sentinels and caused perpetual diffs, and against a server that returns raw values it would persist secrets into a non-Sensitive attribute. Read now preserves the config value like the credential and model resources do, litellm_params is marked Sensitive, and a regression test pins that a server-returned api_key never lands in state * fix(terraform): send role on team member update and stop persisting server env into MCP state The team member update payload omitted role, and the proxy leaves role unchanged when the field is absent, so a role downgrade reported as applied by Terraform never took effect on the proxy. The update now always sends the configured role (the attribute is Required). The MCP server resource wrote env straight back from API responses into a non-Sensitive attribute, pulling admin-visible secrets into state and, for sanitized responses, blanking user config. Read now preserves the config value, env is marked Sensitive, and the docs warn against passing secrets via args. Regression tests cover both fixes and fail against the previous behavior.
117 lines
4.1 KiB
Markdown
117 lines
4.1 KiB
Markdown
# LiteLLM Provider
|
|
|
|
The LiteLLM provider allows Terraform to manage LiteLLM resources. LiteLLM is a proxy service that standardizes the input/output across different LLM APIs, providing a unified interface for various language model providers.
|
|
|
|
## Example Usage
|
|
|
|
```hcl
|
|
terraform {
|
|
required_providers {
|
|
litellm = {
|
|
source = "registry.terraform.io/BerriAI/litellm"
|
|
}
|
|
}
|
|
}
|
|
|
|
provider "litellm" {
|
|
api_base = "https://your-litellm-proxy.com"
|
|
api_key = var.litellm_api_key
|
|
}
|
|
|
|
# Basic model configuration
|
|
resource "litellm_model" "gpt4" {
|
|
model_name = "gpt-4-proxy"
|
|
custom_llm_provider = "openai"
|
|
model_api_key = var.openai_api_key
|
|
base_model = "gpt-4"
|
|
tier = "paid"
|
|
mode = "chat"
|
|
|
|
input_cost_per_million_tokens = 30.0
|
|
output_cost_per_million_tokens = 60.0
|
|
}
|
|
|
|
# Team configuration
|
|
resource "litellm_team" "dev_team" {
|
|
team_alias = "development-team"
|
|
models = [litellm_model.gpt4.model_name]
|
|
max_budget = 100.0
|
|
}
|
|
```
|
|
|
|
## Available Resources
|
|
|
|
The LiteLLM provider supports the following resources:
|
|
|
|
* [`litellm_model`](./resources/model) - Manage LiteLLM model configurations
|
|
* [`litellm_team`](./resources/team) - Manage teams and their permissions
|
|
* [`litellm_team_member`](./resources/team_member) - Manage team member configurations
|
|
* [`litellm_team_member_add`](./resources/team_member_add) - Add members to teams
|
|
* [`litellm_key`](./resources/key) - Manage API keys
|
|
* [`litellm_mcp_server`](./resources/mcp_server) - Manage MCP (Model Context Protocol) servers
|
|
* [`litellm_credential`](./resources/credential) - Manage credentials for various providers
|
|
* [`litellm_vector_store`](./resources/vector_store) - Manage vector stores
|
|
|
|
## Available Data Sources
|
|
|
|
The LiteLLM provider supports the following data sources:
|
|
|
|
* [`litellm_credential`](./data-sources/credential) - Retrieve credential information
|
|
* [`litellm_vector_store`](./data-sources/vector_store) - Retrieve vector store information
|
|
|
|
## Authentication
|
|
|
|
The LiteLLM provider requires an API key and base URL for authentication. These can be provided in the provider configuration block or via environment variables.
|
|
|
|
### Environment Variables
|
|
|
|
- `LITELLM_API_BASE` - The base URL of your LiteLLM instance
|
|
- `LITELLM_API_KEY` - Your LiteLLM API key
|
|
|
|
### Example with Environment Variables
|
|
|
|
```bash
|
|
export LITELLM_API_BASE="https://your-litellm-proxy.com"
|
|
export LITELLM_API_KEY="your-api-key"
|
|
```
|
|
|
|
```hcl
|
|
terraform {
|
|
required_providers {
|
|
litellm = {
|
|
source = "registry.terraform.io/BerriAI/litellm"
|
|
}
|
|
}
|
|
}
|
|
|
|
# Provider will automatically use environment variables
|
|
provider "litellm" {}
|
|
```
|
|
|
|
## Provider Arguments
|
|
|
|
The following arguments are supported in the provider block:
|
|
|
|
* `api_base` - (Required) The base URL of your LiteLLM instance. This can also be provided via the `LITELLM_API_BASE` environment variable.
|
|
* `api_key` - (Required) The API key used to authenticate with LiteLLM. This can also be provided via the `LITELLM_API_KEY` environment variable.
|
|
|
|
## Getting Started
|
|
|
|
1. Install the provider by adding it to your Terraform configuration
|
|
2. Configure your LiteLLM instance URL and API key
|
|
3. Start creating resources like models, teams, and credentials
|
|
4. Use data sources to reference existing configurations
|
|
|
|
For detailed examples and configuration options, see the individual resource and data source documentation pages.
|
|
|
|
## Examples
|
|
|
|
This repository includes an `examples/` directory with curated, ready-to-run HCL examples that demonstrate common and advanced usages of the provider. Examples are grouped by resource and illustrate provider-specific configuration, handling of sensitive values, and advanced options such as `additional_litellm_params`.
|
|
|
|
See:
|
|
* `examples/model_additional_params.tf` — demonstrates how to use `additional_litellm_params` (booleans, integers, floats, and strings).
|
|
* Other example files will be added to `examples/` for credentials, vector stores, and MCP servers.
|
|
|
|
You can reference these examples directly or copy snippets into your Terraform configurations for quick starts.
|
|
|
|
For detailed examples and configuration options, see the individual resource and data source documentation pages.
|