# litellm_model Resource Manages a LiteLLM model configuration. This resource allows you to create, update, and delete model configurations in your LiteLLM instance. ## Example Usage ### Basic OpenAI Model ```hcl resource "litellm_model" "gpt4" { model_name = "gpt-4-proxy" custom_llm_provider = "openai" model_api_key = var.openai_api_key base_model = "gpt-4" tier = "paid" mode = "chat" input_cost_per_million_tokens = 30.0 output_cost_per_million_tokens = 60.0 } ``` ### Advanced Model with All Features ```hcl resource "litellm_model" "advanced_gpt4" { model_name = "gpt-4-advanced" custom_llm_provider = "openai" model_api_key = var.openai_api_key model_api_base = "https://api.openai.com/v1" api_version = "2023-05-15" base_model = "gpt-4" tier = "paid" team_id = "team-123" mode = "chat" reasoning_effort = "medium" thinking_enabled = true thinking_budget_tokens = 1024 merge_reasoning_content_in_choices = true tpm = 100000 rpm = 1000 # Cost configuration (per million tokens) input_cost_per_million_tokens = 30.0 # $0.03 per 1k tokens = $30 per million output_cost_per_million_tokens = 60.0 # $0.06 per 1k tokens = $60 per million } ``` ### AWS Bedrock Model with Cross-Account Access ```hcl resource "litellm_model" "bedrock_claude" { model_name = "bedrock-claude-proxy" custom_llm_provider = "bedrock" base_model = "anthropic.claude-3-sonnet-20240229-v1:0" tier = "paid" mode = "chat" # AWS configuration with cross-account access aws_access_key_id = var.aws_access_key_id aws_secret_access_key = var.aws_secret_access_key aws_region_name = "us-east-1" aws_session_name = "litellm-cross-account-session" aws_role_name = "arn:aws:iam::123456789012:role/LiteLLMCrossAccountRole" input_cost_per_million_tokens = 3.0 output_cost_per_million_tokens = 15.0 } ``` ### Anthropic Model ```hcl resource "litellm_model" "claude" { model_name = "claude-proxy" custom_llm_provider = "anthropic" model_api_key = var.anthropic_api_key base_model = "claude-3-sonnet-20240229" tier = "paid" mode = "chat" input_cost_per_million_tokens = 3.0 output_cost_per_million_tokens = 15.0 } ``` ### Azure OpenAI Model ```hcl resource "litellm_model" "azure_gpt4" { model_name = "azure-gpt4-proxy" custom_llm_provider = "azure" model_api_key = var.azure_openai_key model_api_base = var.azure_openai_endpoint api_version = "2023-12-01-preview" base_model = "gpt-4" tier = "paid" mode = "chat" input_cost_per_million_tokens = 30.0 output_cost_per_million_tokens = 60.0 } ``` ## Argument Reference The following arguments are supported: * `model_name` - (Required) string. The name of the model configuration used to identify the model in API calls. * `custom_llm_provider` - (Required) string. The LLM provider for this model (e.g., "openai", "anthropic", "azure", "bedrock"). * `model_api_key` - (Optional) string (Sensitive). The API key for the underlying model provider. Sensitive attributes are hidden from Terraform output but still stored in plaintext in the state file; prefer storing provider secrets in a `litellm_credential` and referencing it via `litellm_credential_name`, and secure your state backend. * `model_api_base` - (Optional) string. The base URL for the model provider's API. * `api_version` - (Optional) string. The API version to use for the model provider. * `base_model` - (Required) string. The actual model identifier from the provider (e.g., "gpt-4", "claude-2"). * `pricing_base_model` - (Optional) string. A pricing key fed to `model_info.base_model` **independently of routing**. When set, `litellm_params.model` still routes via `base_model`, but LiteLLM looks up cost against this key. Useful when the routing/deployment name differs from the cost-map key — e.g. an Azure deployment routed as `azure/gpt-4.1` whose real tier is Data Zone: set `pricing_base_model = "us/gpt-4.1-2025-04-14"` so it is billed at the Data Zone rate. When unset, `base_model` drives pricing as before. * `litellm_credential_name` - (Optional) string. Name of a LiteLLM credential to use for this model. * `tier` - (Optional) string. The usage tier for this model. Valid values are `"free"` or `"paid"`. Default: `"free"`. * `team_id` - (Optional) string. Associate the model with a specific team. * `mode` - (Optional) string. The intended use of the model. Valid values are: * `completion` * `embedding` * `image_generation` * `chat` * `moderation` * `audio_transcription` * `audio_speech` * `rerank` * `tpm` - (Optional) integer. Tokens per minute limit for this model. * `rpm` - (Optional) integer. Requests per minute limit for this model. * `reasoning_effort` - (Optional) string. Configures the model's reasoning effort level. Valid values are: * `low` * `medium` * `high` * `thinking_enabled` - (Optional) boolean. Enables the model's thinking capability. Default: `false`. * `thinking_budget_tokens` - (Optional) integer. Sets the token budget for the model's thinking capability. Default: `1024`. Note: this field is only relevant when `thinking_enabled = true`. * `merge_reasoning_content_in_choices` - (Optional) boolean. When set to `true`, merges reasoning content into the model's choices. * `input_cost_per_million_tokens` - (Optional) float. Cost per million input tokens. The provider converts this to a per-token cost sent to the API. * `output_cost_per_million_tokens` - (Optional) float. Cost per million output tokens. The provider converts this to a per-token cost sent to the API. * `input_cost_per_pixel` - (Optional) float. Cost applied per input pixel for models that charge by image size. * `output_cost_per_pixel` - (Optional) float. Cost applied per output pixel for image-generation models. * `input_cost_per_second` - (Optional) float. Cost applied per input second for audio/transcription models. * `output_cost_per_second` - (Optional) float. Cost applied per output second for audio/transcription models. * `vertex_project` - (Optional) string. Vertex AI project id (for `custom_llm_provider = "vertex"`). * `vertex_location` - (Optional) string. Vertex AI location (e.g., `us-central1`). * `vertex_credentials` - (Optional) string. Vertex credentials (JSON string or path depending on your setup). * `additional_litellm_params` - (Optional) map(string). A map of arbitrary additional parameters that will be merged into the `litellm_params` object sent to the LiteLLM API. This is intended for provider-specific or experimental options not exposed as dedicated arguments. Conversion and behavior rules (how the provider handles values): * When values in the map are strings the provider will attempt to coerce them: * `"true"` / `"false"` (strings) -> boolean true / false * Numeric strings are parsed first as integers; if integer parsing fails, parsed as floats (e.g., `"16384"` -> 16384, `"0.75"` -> 0.75) * JSON strings (starting with `[` or `{`) are parsed as JSON objects/arrays * Non-convertible strings remain strings * Non-string map values (if supplied) are passed through unchanged. * The provider merges these keys into the `litellm_params` payload sent to the API. * Note: the remote API may not echo back all custom parameters; this provider preserves `additional_litellm_params` in state when present in configuration. **Special parameter: `additional_drop_params`** * When `additional_drop_params` is provided as a JSON array string, it specifies parameters to remove from the final `litellm_params` before sending to the API * This allows you to override or remove built-in parameters if needed * The `additional_drop_params` key itself is not included in the final parameters Example showing booleans, integers, floats, strings, and parameter dropping: ```hcl resource "litellm_model" "with_additional" { model_name = "custom-model" custom_llm_provider = "openai" model_api_key = var.openai_api_key base_model = "gpt-4" mode = "chat" additional_litellm_params = { "use_fine_tune" = "true" # becomes boolean true "max_context" = "16384" # becomes integer 16384 "scale" = "0.75" # becomes float 0.75 "note" = "for testing" # stays string "complex_config" = "{\"nested\": {\"value\": 42}}" # parsed as JSON object "additional_drop_params" = "[\"reasoningEffort\"]" # removes reasoningEffort parameter } } ``` ### AWS-specific Configuration * `aws_access_key_id` - (Optional) string (Sensitive). AWS access key ID for AWS-based models. * `aws_secret_access_key` - (Optional) string (Sensitive). AWS secret access key for AWS-based models. As with `model_api_key`, the value is stored in plaintext in the state file; prefer a `litellm_credential` referenced via `litellm_credential_name` and secure your state backend. * `aws_region_name` - (Optional) string. AWS region name for AWS-based models. * `aws_session_name` - (Optional) string (Sensitive). AWS session name for cross-account access scenarios. * `aws_role_name` - (Optional) string (Sensitive). AWS IAM role name for cross-account access scenarios. ## Attribute Reference In addition to the arguments above, the following attributes are exported: * `id` - The ID of the model configuration. ## Import Model configurations can be imported using the model ID: ```shell terraform import litellm_model.gpt4 ``` Note: The model ID is generated when the model is created and is different from the `model_name`. ## Security Note When using this resource, ensure that sensitive information such as API keys and AWS credentials are stored securely. It's recommended to use environment variables or a secure secret management solution rather than hardcoding these values in your Terraform configuration files.