mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
b5823d5894
The provider's release gate in project-releaser publishes only when the topmost released heading in terraform/provider/CHANGELOG.md moves past the tag the mirror already carries. That heading has been 0.2.2 since 2026-05-13, so every stable release since has correctly decided there was nothing to publish and the registry has gone stale. Two things were blocking a release: 1. The mirror shipped 0.3.0 out-of-band on 2026-07-13 (pricing_base_model, BerriAI/terraform-provider-litellm#47) after the source move, so that code exists only in the mirror. The publish rsyncs monorepo -> mirror with --delete, so publishing without this port would have deleted a released feature from the registry. 2. Nothing here declared a new version. Port #47 verbatim (resource_model.go and resource_model_crud.go are now byte-identical to the mirror's released files), backfill the 0.3.0 changelog entry it shipped under, and cut 0.4.0 covering the changes made here since the source move. 0.3.0 is not reusable as the next version -- the mirror holds that tag and the publish workflow's tag guard rejects it.
240 lines
10 KiB
Markdown
240 lines
10 KiB
Markdown
# litellm_model Resource
|
|
|
|
Manages a LiteLLM model configuration. This resource allows you to create, update, and delete model configurations in your LiteLLM instance.
|
|
|
|
## Example Usage
|
|
|
|
### Basic OpenAI Model
|
|
|
|
```hcl
|
|
resource "litellm_model" "gpt4" {
|
|
model_name = "gpt-4-proxy"
|
|
custom_llm_provider = "openai"
|
|
model_api_key = var.openai_api_key
|
|
base_model = "gpt-4"
|
|
tier = "paid"
|
|
mode = "chat"
|
|
|
|
input_cost_per_million_tokens = 30.0
|
|
output_cost_per_million_tokens = 60.0
|
|
}
|
|
```
|
|
|
|
### Advanced Model with All Features
|
|
|
|
```hcl
|
|
resource "litellm_model" "advanced_gpt4" {
|
|
model_name = "gpt-4-advanced"
|
|
custom_llm_provider = "openai"
|
|
model_api_key = var.openai_api_key
|
|
model_api_base = "https://api.openai.com/v1"
|
|
api_version = "2023-05-15"
|
|
base_model = "gpt-4"
|
|
tier = "paid"
|
|
team_id = "team-123"
|
|
mode = "chat"
|
|
reasoning_effort = "medium"
|
|
thinking_enabled = true
|
|
thinking_budget_tokens = 1024
|
|
merge_reasoning_content_in_choices = true
|
|
tpm = 100000
|
|
rpm = 1000
|
|
|
|
# Cost configuration (per million tokens)
|
|
input_cost_per_million_tokens = 30.0 # $0.03 per 1k tokens = $30 per million
|
|
output_cost_per_million_tokens = 60.0 # $0.06 per 1k tokens = $60 per million
|
|
}
|
|
```
|
|
|
|
### AWS Bedrock Model with Cross-Account Access
|
|
|
|
```hcl
|
|
resource "litellm_model" "bedrock_claude" {
|
|
model_name = "bedrock-claude-proxy"
|
|
custom_llm_provider = "bedrock"
|
|
base_model = "anthropic.claude-3-sonnet-20240229-v1:0"
|
|
tier = "paid"
|
|
mode = "chat"
|
|
|
|
# AWS configuration with cross-account access
|
|
aws_access_key_id = var.aws_access_key_id
|
|
aws_secret_access_key = var.aws_secret_access_key
|
|
aws_region_name = "us-east-1"
|
|
aws_session_name = "litellm-cross-account-session"
|
|
aws_role_name = "arn:aws:iam::123456789012:role/LiteLLMCrossAccountRole"
|
|
|
|
input_cost_per_million_tokens = 3.0
|
|
output_cost_per_million_tokens = 15.0
|
|
}
|
|
```
|
|
|
|
### Anthropic Model
|
|
|
|
```hcl
|
|
resource "litellm_model" "claude" {
|
|
model_name = "claude-proxy"
|
|
custom_llm_provider = "anthropic"
|
|
model_api_key = var.anthropic_api_key
|
|
base_model = "claude-3-sonnet-20240229"
|
|
tier = "paid"
|
|
mode = "chat"
|
|
|
|
input_cost_per_million_tokens = 3.0
|
|
output_cost_per_million_tokens = 15.0
|
|
}
|
|
```
|
|
|
|
### Azure OpenAI Model
|
|
|
|
```hcl
|
|
resource "litellm_model" "azure_gpt4" {
|
|
model_name = "azure-gpt4-proxy"
|
|
custom_llm_provider = "azure"
|
|
model_api_key = var.azure_openai_key
|
|
model_api_base = var.azure_openai_endpoint
|
|
api_version = "2023-12-01-preview"
|
|
base_model = "gpt-4"
|
|
tier = "paid"
|
|
mode = "chat"
|
|
|
|
input_cost_per_million_tokens = 30.0
|
|
output_cost_per_million_tokens = 60.0
|
|
}
|
|
```
|
|
|
|
## Argument Reference
|
|
|
|
The following arguments are supported:
|
|
|
|
* `model_name` - (Required) string. The name of the model configuration used to identify the model in API calls.
|
|
|
|
* `custom_llm_provider` - (Required) string. The LLM provider for this model (e.g., "openai", "anthropic", "azure", "bedrock").
|
|
|
|
* `model_api_key` - (Optional) string (Sensitive). The API key for the underlying model provider. Sensitive attributes are hidden from Terraform output but still stored in plaintext in the state file; prefer storing provider secrets in a `litellm_credential` and referencing it via `litellm_credential_name`, and secure your state backend.
|
|
|
|
* `model_api_base` - (Optional) string. The base URL for the model provider's API.
|
|
|
|
* `api_version` - (Optional) string. The API version to use for the model provider.
|
|
|
|
* `base_model` - (Required) string. The actual model identifier from the provider (e.g., "gpt-4", "claude-2").
|
|
|
|
* `pricing_base_model` - (Optional) string. A pricing key fed to `model_info.base_model` **independently of routing**. When set, `litellm_params.model` still routes via `base_model`, but LiteLLM looks up cost against this key. Useful when the routing/deployment name differs from the cost-map key — e.g. an Azure deployment routed as `azure/gpt-4.1` whose real tier is Data Zone: set `pricing_base_model = "us/gpt-4.1-2025-04-14"` so it is billed at the Data Zone rate. When unset, `base_model` drives pricing as before.
|
|
|
|
* `litellm_credential_name` - (Optional) string. Name of a LiteLLM credential to use for this model.
|
|
|
|
* `tier` - (Optional) string. The usage tier for this model. Valid values are `"free"` or `"paid"`. Default: `"free"`.
|
|
|
|
* `team_id` - (Optional) string. Associate the model with a specific team.
|
|
|
|
* `mode` - (Optional) string. The intended use of the model. Valid values are:
|
|
* `completion`
|
|
* `embedding`
|
|
* `image_generation`
|
|
* `chat`
|
|
* `moderation`
|
|
* `audio_transcription`
|
|
* `audio_speech`
|
|
* `rerank`
|
|
|
|
* `tpm` - (Optional) integer. Tokens per minute limit for this model.
|
|
|
|
* `rpm` - (Optional) integer. Requests per minute limit for this model.
|
|
|
|
* `reasoning_effort` - (Optional) string. Configures the model's reasoning effort level. Valid values are:
|
|
* `low`
|
|
* `medium`
|
|
* `high`
|
|
|
|
* `thinking_enabled` - (Optional) boolean. Enables the model's thinking capability. Default: `false`.
|
|
|
|
* `thinking_budget_tokens` - (Optional) integer. Sets the token budget for the model's thinking capability. Default: `1024`. Note: this field is only relevant when `thinking_enabled = true`.
|
|
|
|
* `merge_reasoning_content_in_choices` - (Optional) boolean. When set to `true`, merges reasoning content into the model's choices.
|
|
|
|
* `input_cost_per_million_tokens` - (Optional) float. Cost per million input tokens. The provider converts this to a per-token cost sent to the API.
|
|
|
|
* `output_cost_per_million_tokens` - (Optional) float. Cost per million output tokens. The provider converts this to a per-token cost sent to the API.
|
|
|
|
* `input_cost_per_pixel` - (Optional) float. Cost applied per input pixel for models that charge by image size.
|
|
|
|
* `output_cost_per_pixel` - (Optional) float. Cost applied per output pixel for image-generation models.
|
|
|
|
* `input_cost_per_second` - (Optional) float. Cost applied per input second for audio/transcription models.
|
|
|
|
* `output_cost_per_second` - (Optional) float. Cost applied per output second for audio/transcription models.
|
|
|
|
* `vertex_project` - (Optional) string. Vertex AI project id (for `custom_llm_provider = "vertex"`).
|
|
|
|
* `vertex_location` - (Optional) string. Vertex AI location (e.g., `us-central1`).
|
|
|
|
* `vertex_credentials` - (Optional) string. Vertex credentials (JSON string or path depending on your setup).
|
|
|
|
* `additional_litellm_params` - (Optional) map(string). A map of arbitrary additional parameters that will be merged into the `litellm_params` object sent to the LiteLLM API. This is intended for provider-specific or experimental options not exposed as dedicated arguments.
|
|
|
|
Conversion and behavior rules (how the provider handles values):
|
|
* When values in the map are strings the provider will attempt to coerce them:
|
|
* `"true"` / `"false"` (strings) -> boolean true / false
|
|
* Numeric strings are parsed first as integers; if integer parsing fails, parsed as floats (e.g., `"16384"` -> 16384, `"0.75"` -> 0.75)
|
|
* JSON strings (starting with `[` or `{`) are parsed as JSON objects/arrays
|
|
* Non-convertible strings remain strings
|
|
* Non-string map values (if supplied) are passed through unchanged.
|
|
* The provider merges these keys into the `litellm_params` payload sent to the API.
|
|
* Note: the remote API may not echo back all custom parameters; this provider preserves `additional_litellm_params` in state when present in configuration.
|
|
|
|
**Special parameter: `additional_drop_params`**
|
|
* When `additional_drop_params` is provided as a JSON array string, it specifies parameters to remove from the final `litellm_params` before sending to the API
|
|
* This allows you to override or remove built-in parameters if needed
|
|
* The `additional_drop_params` key itself is not included in the final parameters
|
|
|
|
Example showing booleans, integers, floats, strings, and parameter dropping:
|
|
|
|
```hcl
|
|
resource "litellm_model" "with_additional" {
|
|
model_name = "custom-model"
|
|
custom_llm_provider = "openai"
|
|
model_api_key = var.openai_api_key
|
|
base_model = "gpt-4"
|
|
mode = "chat"
|
|
|
|
additional_litellm_params = {
|
|
"use_fine_tune" = "true" # becomes boolean true
|
|
"max_context" = "16384" # becomes integer 16384
|
|
"scale" = "0.75" # becomes float 0.75
|
|
"note" = "for testing" # stays string
|
|
"complex_config" = "{\"nested\": {\"value\": 42}}" # parsed as JSON object
|
|
"additional_drop_params" = "[\"reasoningEffort\"]" # removes reasoningEffort parameter
|
|
}
|
|
}
|
|
```
|
|
|
|
### AWS-specific Configuration
|
|
|
|
* `aws_access_key_id` - (Optional) string (Sensitive). AWS access key ID for AWS-based models.
|
|
|
|
* `aws_secret_access_key` - (Optional) string (Sensitive). AWS secret access key for AWS-based models. As with `model_api_key`, the value is stored in plaintext in the state file; prefer a `litellm_credential` referenced via `litellm_credential_name` and secure your state backend.
|
|
|
|
* `aws_region_name` - (Optional) string. AWS region name for AWS-based models.
|
|
|
|
* `aws_session_name` - (Optional) string (Sensitive). AWS session name for cross-account access scenarios.
|
|
|
|
* `aws_role_name` - (Optional) string (Sensitive). AWS IAM role name for cross-account access scenarios.
|
|
|
|
## Attribute Reference
|
|
|
|
In addition to the arguments above, the following attributes are exported:
|
|
|
|
* `id` - The ID of the model configuration.
|
|
|
|
## Import
|
|
|
|
Model configurations can be imported using the model ID:
|
|
|
|
```shell
|
|
terraform import litellm_model.gpt4 <model-id>
|
|
```
|
|
|
|
Note: The model ID is generated when the model is created and is different from the `model_name`.
|
|
|
|
## Security Note
|
|
|
|
When using this resource, ensure that sensitive information such as API keys and AWS credentials are stored securely. It's recommended to use environment variables or a secure secret management solution rather than hardcoding these values in your Terraform configuration files.
|