litellm/terraform/provider/docs/resources/model.md

litellm_model Resource

Manages a LiteLLM model configuration. This resource allows you to create, update, and delete model configurations in your LiteLLM instance.

Example Usage

Basic OpenAI Model

resource "litellm_model" "gpt4" {
  model_name          = "gpt-4-proxy"
  custom_llm_provider = "openai"
  model_api_key       = var.openai_api_key
  base_model          = "gpt-4"
  tier                = "paid"
  mode                = "chat"
  
  input_cost_per_million_tokens  = 30.0
  output_cost_per_million_tokens = 60.0
}

Advanced Model with All Features

resource "litellm_model" "advanced_gpt4" {
  model_name          = "gpt-4-advanced"
  custom_llm_provider = "openai"
  model_api_key       = var.openai_api_key
  model_api_base      = "https://api.openai.com/v1"
  api_version         = "2023-05-15"
  base_model          = "gpt-4"
  tier                = "paid"
  team_id             = "team-123"
  mode                = "chat"
  reasoning_effort    = "medium"
  thinking_enabled    = true
  thinking_budget_tokens = 1024
  merge_reasoning_content_in_choices = true
  tpm                 = 100000
  rpm                 = 1000
  
  # Cost configuration (per million tokens)
  input_cost_per_million_tokens  = 30.0    # $0.03 per 1k tokens = $30 per million
  output_cost_per_million_tokens = 60.0    # $0.06 per 1k tokens = $60 per million
}

AWS Bedrock Model with Cross-Account Access

resource "litellm_model" "bedrock_claude" {
  model_name          = "bedrock-claude-proxy"
  custom_llm_provider = "bedrock"
  base_model          = "anthropic.claude-3-sonnet-20240229-v1:0"
  tier                = "paid"
  mode                = "chat"
  
  # AWS configuration with cross-account access
  aws_access_key_id     = var.aws_access_key_id
  aws_secret_access_key = var.aws_secret_access_key
  aws_region_name       = "us-east-1"
  aws_session_name      = "litellm-cross-account-session"
  aws_role_name         = "arn:aws:iam::123456789012:role/LiteLLMCrossAccountRole"
  
  input_cost_per_million_tokens  = 3.0
  output_cost_per_million_tokens = 15.0
}

Anthropic Model

resource "litellm_model" "claude" {
  model_name          = "claude-proxy"
  custom_llm_provider = "anthropic"
  model_api_key       = var.anthropic_api_key
  base_model          = "claude-3-sonnet-20240229"
  tier                = "paid"
  mode                = "chat"
  
  input_cost_per_million_tokens  = 3.0
  output_cost_per_million_tokens = 15.0
}

Azure OpenAI Model

resource "litellm_model" "azure_gpt4" {
  model_name          = "azure-gpt4-proxy"
  custom_llm_provider = "azure"
  model_api_key       = var.azure_openai_key
  model_api_base      = var.azure_openai_endpoint
  api_version         = "2023-12-01-preview"
  base_model          = "gpt-4"
  tier                = "paid"
  mode                = "chat"
  
  input_cost_per_million_tokens  = 30.0
  output_cost_per_million_tokens = 60.0
}

Argument Reference

The following arguments are supported:

• model_name - (Required) string. The name of the model configuration used to identify the model in API calls.

• custom_llm_provider - (Required) string. The LLM provider for this model (e.g., "openai", "anthropic", "azure", "bedrock").

• model_api_key - (Optional) string (Sensitive). The API key for the underlying model provider. Sensitive attributes are hidden from Terraform output but still stored in plaintext in the state file; prefer storing provider secrets in a litellm_credential and referencing it via litellm_credential_name, and secure your state backend.

• model_api_base - (Optional) string. The base URL for the model provider's API.

• api_version - (Optional) string. The API version to use for the model provider.

• base_model - (Required) string. The actual model identifier from the provider (e.g., "gpt-4", "claude-2").

• pricing_base_model - (Optional) string. A pricing key fed to model_info.base_model independently of routing. When set, litellm_params.model still routes via base_model, but LiteLLM looks up cost against this key. Useful when the routing/deployment name differs from the cost-map key — e.g. an Azure deployment routed as azure/gpt-4.1 whose real tier is Data Zone: set pricing_base_model = "us/gpt-4.1-2025-04-14" so it is billed at the Data Zone rate. When unset, base_model drives pricing as before.

• litellm_credential_name - (Optional) string. Name of a LiteLLM credential to use for this model.

• tier - (Optional) string. The usage tier for this model. Valid values are "free" or "paid". Default: "free".

• team_id - (Optional) string. Associate the model with a specific team.

• display_name - (Optional) string. Human-readable name stored in model_info.display_name and returned as display_name by /v1/models, so clients such as Claude Code and Claude Desktop show it in their model picker instead of model_name. When unset, clients fall back to model_name.

• mode - (Optional) string. The intended use of the model. Valid values are:

  • completion
  • embedding
  • image_generation
  • chat
  • moderation
  • audio_transcription
  • audio_speech
  • rerank

• tpm - (Optional) integer. Tokens per minute limit for this model.

• rpm - (Optional) integer. Requests per minute limit for this model.

• reasoning_effort - (Optional) string. Configures the model's reasoning effort level. Valid values are:

  • low
  • medium
  • high

• thinking_enabled - (Optional) boolean. Enables the model's thinking capability. Default: false.

• thinking_budget_tokens - (Optional) integer. Sets the token budget for the model's thinking capability. Default: 1024. Note: this field is only relevant when thinking_enabled = true.

• merge_reasoning_content_in_choices - (Optional) boolean. When set to true, merges reasoning content into the model's choices.

• input_cost_per_million_tokens - (Optional) float. Cost per million input tokens. The provider converts this to a per-token cost sent to the API.

• output_cost_per_million_tokens - (Optional) float. Cost per million output tokens. The provider converts this to a per-token cost sent to the API.

• input_cost_per_pixel - (Optional) float. Cost applied per input pixel for models that charge by image size.

• output_cost_per_pixel - (Optional) float. Cost applied per output pixel for image-generation models.

• input_cost_per_second - (Optional) float. Cost applied per input second for audio/transcription models.

• output_cost_per_second - (Optional) float. Cost applied per output second for audio/transcription models.

• vertex_project - (Optional) string. Vertex AI project id (for custom_llm_provider = "vertex").

• vertex_location - (Optional) string. Vertex AI location (e.g., us-central1).

• vertex_credentials - (Optional) string. Vertex credentials (JSON string or path depending on your setup).

• additional_litellm_params - (Optional) map(string). A map of arbitrary additional parameters that will be merged into the litellm_params object sent to the LiteLLM API. This is intended for provider-specific or experimental options not exposed as dedicated arguments.

  Conversion and behavior rules (how the provider handles values):

  • When values in the map are strings the provider will attempt to coerce them:
    • "true" / "false" (strings) -> boolean true / false
    • Numeric strings are parsed first as integers; if integer parsing fails, parsed as floats (e.g., "16384" -> 16384, "0.75" -> 0.75)
    • JSON strings (starting with [ or {) are parsed as JSON objects/arrays
    • Non-convertible strings remain strings
  • Non-string map values (if supplied) are passed through unchanged.
  • The provider merges these keys into the litellm_params payload sent to the API.
  • Note: the remote API may not echo back all custom parameters; this provider preserves additional_litellm_params in state when present in configuration.

  Special parameter: additional_drop_params

  • When additional_drop_params is provided as a JSON array string, it specifies parameters to remove from the final litellm_params before sending to the API
  • This allows you to override or remove built-in parameters if needed
  • The additional_drop_params key itself is not included in the final parameters

  Example showing booleans, integers, floats, strings, and parameter dropping:

  resource "litellm_model" "with_additional" {
    model_name          = "custom-model"
    custom_llm_provider = "openai"
    model_api_key       = var.openai_api_key
    base_model          = "gpt-4"
    mode                = "chat"
  
    additional_litellm_params = {
      "use_fine_tune"          = "true"                    # becomes boolean true
      "max_context"            = "16384"                   # becomes integer 16384
      "scale"                  = "0.75"                    # becomes float 0.75
      "note"                   = "for testing"             # stays string
      "complex_config"         = "{\"nested\": {\"value\": 42}}"  # parsed as JSON object
      "additional_drop_params" = "[\"reasoningEffort\"]"   # removes reasoningEffort parameter
    }
  }
  

AWS-specific Configuration

• aws_access_key_id - (Optional) string (Sensitive). AWS access key ID for AWS-based models.

• aws_secret_access_key - (Optional) string (Sensitive). AWS secret access key for AWS-based models. As with model_api_key, the value is stored in plaintext in the state file; prefer a litellm_credential referenced via litellm_credential_name and secure your state backend.

• aws_region_name - (Optional) string. AWS region name for AWS-based models.

• aws_session_name - (Optional) string (Sensitive). AWS session name for cross-account access scenarios.

• aws_role_name - (Optional) string (Sensitive). AWS IAM role name for cross-account access scenarios.

Attribute Reference

In addition to the arguments above, the following attributes are exported:

• id - The ID of the model configuration.

Import

Model configurations can be imported using the model ID:

terraform import litellm_model.gpt4 <model-id>

Note: The model ID is generated when the model is created and is different from the model_name.

Security Note

When using this resource, ensure that sensitive information such as API keys and AWS credentials are stored securely. It's recommended to use environment variables or a secure secret management solution rather than hardcoding these values in your Terraform configuration files.