litellm/terraform/provider/docs/resources/model.md

litellm_model Resource

Manages a LiteLLM model configuration. This resource allows you to create, update, and delete model configurations in your LiteLLM instance.

Example Usage

Basic OpenAI Model

resource "litellm_model" "gpt4" {
  model_name          = "gpt-4-proxy"
  custom_llm_provider = "openai"
  model_api_key       = var.openai_api_key
  base_model          = "gpt-4"
  tier                = "paid"
  mode                = "chat"
  
  input_cost_per_million_tokens  = 30.0
  output_cost_per_million_tokens = 60.0
}

Advanced Model with All Features

resource "litellm_model" "advanced_gpt4" {
  model_name          = "gpt-4-advanced"
  custom_llm_provider = "openai"
  model_api_key       = var.openai_api_key
  model_api_base      = "https://api.openai.com/v1"
  api_version         = "2023-05-15"
  base_model          = "gpt-4"
  tier                = "paid"
  team_id             = "team-123"
  mode                = "chat"
  reasoning_effort    = "medium"
  thinking_enabled    = true
  thinking_budget_tokens = 1024
  merge_reasoning_content_in_choices = true
  tpm                 = 100000
  rpm                 = 1000
  
  # Cost configuration (per million tokens)
  input_cost_per_million_tokens  = 30.0    # $0.03 per 1k tokens = $30 per million
  output_cost_per_million_tokens = 60.0    # $0.06 per 1k tokens = $60 per million
}

AWS Bedrock Model with Cross-Account Access

resource "litellm_model" "bedrock_claude" {
  model_name          = "bedrock-claude-proxy"
  custom_llm_provider = "bedrock"
  base_model          = "anthropic.claude-3-sonnet-20240229-v1:0"
  tier                = "paid"
  mode                = "chat"
  
  # AWS configuration with cross-account access
  aws_access_key_id     = var.aws_access_key_id
  aws_secret_access_key = var.aws_secret_access_key
  aws_region_name       = "us-east-1"
  aws_session_name      = "litellm-cross-account-session"
  aws_role_name         = "arn:aws:iam::123456789012:role/LiteLLMCrossAccountRole"
  
  input_cost_per_million_tokens  = 3.0
  output_cost_per_million_tokens = 15.0
}

Anthropic Model

resource "litellm_model" "claude" {
  model_name          = "claude-proxy"
  custom_llm_provider = "anthropic"
  model_api_key       = var.anthropic_api_key
  base_model          = "claude-3-sonnet-20240229"
  tier                = "paid"
  mode                = "chat"
  
  input_cost_per_million_tokens  = 3.0
  output_cost_per_million_tokens = 15.0
}

Azure OpenAI Model

resource "litellm_model" "azure_gpt4" {
  model_name          = "azure-gpt4-proxy"
  custom_llm_provider = "azure"
  model_api_key       = var.azure_openai_key
  model_api_base      = var.azure_openai_endpoint
  api_version         = "2023-12-01-preview"
  base_model          = "gpt-4"
  tier                = "paid"
  mode                = "chat"
  
  input_cost_per_million_tokens  = 30.0
  output_cost_per_million_tokens = 60.0
}

Argument Reference

The following arguments are supported:

• model_name - (Required) string. The name of the model configuration used to identify the model in API calls.

• custom_llm_provider - (Required) string. The LLM provider for this model (e.g., "openai", "anthropic", "azure", "bedrock").

• model_api_key - (Optional) string (Sensitive). The API key for the underlying model provider. Sensitive attributes are hidden from Terraform output but still stored in plaintext in the state file; use model_api_key_wo or a litellm_credential referenced via litellm_credential_name to keep it out of state. Conflicts with model_api_key_wo.

• model_api_key_wo - (Optional, Write-only) string (Sensitive). Same as model_api_key, but sent to the proxy on every apply and never written to plan or state. Requires Terraform 1.11 or later and model_api_key_wo_version.

• model_api_key_wo_version - (Optional) integer. Bump it whenever model_api_key_wo changes to trigger an update, since write-only values produce no diff on their own.

• model_api_base - (Optional) string. The base URL for the model provider's API.

• api_version - (Optional) string. The API version to use for the model provider.

• base_model - (Required) string. The actual model identifier from the provider (e.g., "gpt-4", "claude-2").

• pricing_base_model - (Optional) string. A pricing key fed to model_info.base_model independently of routing. When set, litellm_params.model still routes via base_model, but LiteLLM looks up cost against this key. Useful when the routing/deployment name differs from the cost-map key — e.g. an Azure deployment routed as azure/gpt-4.1 whose real tier is Data Zone: set pricing_base_model = "us/gpt-4.1-2025-04-14" so it is billed at the Data Zone rate. When unset, base_model drives pricing as before.

• litellm_credential_name - (Optional) string. Name of a LiteLLM credential to use for this model.

• tier - (Optional) string. The usage tier for this model. Valid values are "free" or "paid". Default: "free".

• team_id - (Optional) string. Associate the model with a specific team.

• display_name - (Optional) string. Human-readable name stored in model_info.display_name and returned as display_name by /v1/models, so clients such as Claude Code and Claude Desktop show it in their model picker instead of model_name. When unset, clients fall back to model_name.

• mode - (Optional) string. The intended use of the model. Valid values are:

  • completion
  • embedding
  • image_generation
  • chat
  • moderation
  • audio_transcription
  • audio_speech
  • rerank

• tpm - (Optional) integer. Tokens per minute limit for this model.

• rpm - (Optional) integer. Requests per minute limit for this model.

• reasoning_effort - (Optional) string. Configures the model's reasoning effort level. Valid values are:

  • low
  • medium
  • high

• thinking_enabled - (Optional) boolean. Enables the model's thinking capability. Default: false.

• thinking_budget_tokens - (Optional) integer. Sets the token budget for the model's thinking capability. Default: 1024. Note: this field is only relevant when thinking_enabled = true.

• merge_reasoning_content_in_choices - (Optional) boolean. When set to true, merges reasoning content into the model's choices.

• input_cost_per_million_tokens - (Optional) float. Cost per million input tokens. The provider converts this to a per-token cost sent to the API.

• output_cost_per_million_tokens - (Optional) float. Cost per million output tokens. The provider converts this to a per-token cost sent to the API.

• input_cost_per_pixel - (Optional) float. Cost applied per input pixel for models that charge by image size.

• output_cost_per_pixel - (Optional) float. Cost applied per output pixel for image-generation models.

• input_cost_per_second - (Optional) float. Cost applied per input second for audio/transcription models.

• output_cost_per_second - (Optional) float. Cost applied per output second for audio/transcription models.

• vertex_project - (Optional) string. Vertex AI project id (for custom_llm_provider = "vertex").

• vertex_location - (Optional) string. Vertex AI location (e.g., us-central1).

• vertex_credentials - (Optional) string (Sensitive). Vertex credentials (JSON string or path depending on your setup). Stored in plaintext in the state file. Conflicts with vertex_credentials_wo.

• vertex_credentials_wo - (Optional, Write-only) string (Sensitive). Same as vertex_credentials, but never written to plan or state. Requires Terraform 1.11 or later and vertex_credentials_wo_version.

• vertex_credentials_wo_version - (Optional) integer. Bump it whenever vertex_credentials_wo changes to trigger an update.

• additional_litellm_params - (Optional) map(string). A map of arbitrary additional parameters that will be merged into the litellm_params object sent to the LiteLLM API. This is intended for provider-specific or experimental options not exposed as dedicated arguments.

  Conversion and behavior rules (how the provider handles values):

  • When values in the map are strings the provider will attempt to coerce them:
    • "true" / "false" (strings) -> boolean true / false
    • Numeric strings are parsed first as integers; if integer parsing fails, parsed as floats (e.g., "16384" -> 16384, "0.75" -> 0.75)
    • JSON strings (starting with [ or {) are parsed as JSON objects/arrays
    • Non-convertible strings remain strings
  • The provider merges these keys into the litellm_params payload sent to the API.
  • Note: the remote API may not echo back all custom parameters; this provider preserves additional_litellm_params in state when present in configuration.

  additional_drop_params

  • Give it a JSON array string, such as jsonencode(["store"]). The provider sends the list to the proxy as the deployment's additional_drop_params, and the proxy applies it the same as additional_drop_params in a model_list entry. terraform plan fails on any other value, an empty value, or an entry that is not a name, and null entries are skipped
  • Other keys on the resource, such as vertex_location, are sent unchanged
  • When the configuration sets the list, refresh reads the stored list back from the proxy, so a list changed outside Terraform shows up as drift. A model whose configuration has no list keeps a list set outside Terraform out of the plan, but the proxy clears it on the next update
  • When the proxy has no stored list, as on a model an older provider created, the next plan shows the list and every listed key the configuration sets as changes, since the next apply writes them

  Example showing booleans, integers, floats, strings, JSON, and dropped request parameters:

  resource "litellm_model" "with_additional" {
    model_name          = "custom-model"
    custom_llm_provider = "openai"
    model_api_key       = var.openai_api_key
    base_model          = "gpt-4"
    mode                = "chat"
  
    additional_litellm_params = {
      "use_fine_tune"          = "true"                    # becomes boolean true
      "max_context"            = "16384"                   # becomes integer 16384
      "scale"                  = "0.75"                    # becomes float 0.75
      "note"                   = "for testing"             # stays string
      "complex_config"         = "{\"nested\": {\"value\": 42}}"  # parsed as JSON object
      "additional_drop_params" = "[\"reasoning_effort\"]"  # proxy drops reasoning_effort from requests
    }
  }
  

AWS-specific Configuration

• aws_access_key_id - (Optional) string (Sensitive). AWS access key ID for AWS-based models.

• aws_secret_access_key - (Optional) string (Sensitive). AWS secret access key for AWS-based models. As with model_api_key, the value is stored in plaintext in the state file; use aws_secret_access_key_wo or a litellm_credential referenced via litellm_credential_name to keep it out of state. Conflicts with aws_secret_access_key_wo.

• aws_secret_access_key_wo - (Optional, Write-only) string (Sensitive). Same as aws_secret_access_key, but never written to plan or state. Requires Terraform 1.11 or later and aws_secret_access_key_wo_version.

• aws_secret_access_key_wo_version - (Optional) integer. Bump it whenever aws_secret_access_key_wo changes to trigger an update.

• aws_region_name - (Optional) string. AWS region name for AWS-based models.

• aws_session_name - (Optional) string (Sensitive). AWS session name for cross-account access scenarios.

• aws_role_name - (Optional) string (Sensitive). AWS IAM role name for cross-account access scenarios.

Attribute Reference

In addition to the arguments above, the following attributes are exported:

• id - The ID of the model configuration.

Import

Model configurations can be imported using the model ID:

terraform import litellm_model.gpt4 <model-id>

Note: The model ID is generated when the model is created and is different from the model_name.

Security Note

When using this resource, ensure that sensitive information such as API keys and AWS credentials are stored securely. It's recommended to use environment variables or a secure secret management solution rather than hardcoding these values in your Terraform configuration files.