mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-16 04:29:31 +00:00
fix(terraform)!: send additional_drop_params to the proxy instead of deleting the keys it names (#46782)
* fix(terraform): send additional_drop_params to the proxy instead of deleting the keys it names litellm_model treated additional_drop_params in additional_litellm_params as a provider-side delete list: it never sent the list and removed every key it named from litellm_params, so vertex_location vanished and requests still carried store. The list now goes to the proxy like any other JSON value, so the proxy drops those params per request. Approach credited to #38345. Fixes #38349 * fix(terraform)!: validate additional_drop_params and read it back from the proxy on refresh * docs(terraform): note that a non-list additional_drop_params now fails the apply * fix(terraform)!: check additional_drop_params at plan time and refresh listed keys and outside lists * fix(terraform)!: show every listed key an older provider deleted in the upgrade plan and leave outside drop lists alone Refresh no longer reads listed keys back from the proxy, which left a plan diff that never cleared when a listed key was removed from config. When the proxy has no stored list, the plan now shows the list and each listed key the config sets as changes. A drop list set outside Terraform on a model whose config has none stays out of the plan, and an empty value now fails the plan * fix(terraform): show a listed merge_reasoning_content_in_choices in the upgrade plan The refresh now clears merge_reasoning_content_in_choices when it is listed and the proxy has no stored list, since an older provider deleted it, and no longer clears litellm_credential_name, which the older provider always sent
This commit is contained in:
parent
f13d0430e8
commit
dfb949d437
7 changed files with 394 additions and 91 deletions
|
|
@ -51,6 +51,7 @@ longer signal it.
|
|||
|
||||
### Fixed
|
||||
|
||||
- **model**: `additional_drop_params` in `litellm_model.additional_litellm_params` is now sent to the proxy as the deployment's `additional_drop_params` list, the same list a `model_list` entry takes. Before, the provider never sent the list and instead deleted every key it named from `litellm_params`, so a `store` or `vertex_location` listed there was lost and requests still carried the parameters (#38349). The value must be a JSON list of parameter names such as `jsonencode(["store"])`, which `terraform plan` now checks, and `null` entries are skipped as before. When the configuration sets the list, refresh reads the stored list back from the proxy, so a list changed outside Terraform shows up as drift. On a model an older provider created, the proxy has no stored list, so the first plan after upgrading shows the list and every listed key the configuration sets as changes, since that apply writes them. A write-only secret such as `model_api_key_wo` is never shown in a plan
|
||||
- **user**: `litellm_user` updates no longer send `blocked`, which `/user/update` writes into the user table where no such column exists, so every update failed with a 400 (`Could not find field at upsertOneLiteLLM_UserTable.create.blocked`). Create still sends `blocked`, including `false`, so a `default_internal_user_params.blocked` on the proxy cannot block the auto-created key. Changing `blocked` after creation has no effect, since the proxy has no user-level block; use `litellm_key_block` instead
|
||||
- **model**: `litellm_model` refresh now reads the `{"data": [...]}` envelope `/model/info` returns, so `model_info` fields changed outside Terraform show up as drift instead of silently keeping the previous state
|
||||
- **key**: An update that changes `team_id` and fails because the key was already cascade-deleted along with its previous team now recovers by recreating the key under the new team, instead of aborting the apply. The key's absence is confirmed against the proxy first, so an unrelated failure still errors out, and a `team_id` change between two teams that both still exist stays a plain in-place update
|
||||
|
|
@ -69,6 +70,7 @@ longer signal it.
|
|||
|
||||
### Changed
|
||||
|
||||
- **model** (breaking): a key named in `additional_litellm_params.additional_drop_params` is no longer deleted from the deployment's `litellm_params`. A configuration that also sets that key, such as `vertex_location`, stores it on the deployment at its next apply, and the proxy applies the drop list to requests instead. For `vertex_location` this changes routing. `/v1/messages` calls now go to the configured region, while chat completions drop a listed `vertex_location` and use the default region. To keep such a key off a deployment, remove its attribute and replace the model with `terraform apply -replace`, which gives it a new model id, since an update keeps a stored value the provider leaves out. `model`, `custom_llm_provider` and `merge_reasoning_content_in_choices` are always sent. A value that is not a JSON list of names, such as a bare `"store"` or an empty string, now fails the plan, and with it `terraform destroy`, instead of being stored as a string the proxy ignores. Rewrite it as `jsonencode([...])`, or remove the model without fixing it through `terraform destroy -refresh=false`
|
||||
- **key** (breaking): `model_max_budget` on `litellm_key` is now a JSON string of per-model budget objects (`jsonencode({"gpt-4o-mini" = {budget_limit = 50, time_period = "30d"}})`), matching `litellm_user`, `litellm_budget` and `litellm_tag`. The old `map(number)` form sent bare numbers to `/key/generate`, which the proxy rejects with a 500 (`'int' object is not iterable`), so every key with a non-empty `model_max_budget` failed to apply. Existing state upgrades automatically (schema version 1) and the attribute is refilled from the proxy on the next read; configurations still using the map form must be rewritten
|
||||
- **Versioning**: the provider is now published at the LiteLLM version, from the same commit as the proxy, on every LiteLLM release (dev, rc, stable). The `0.x` line ends at `0.4.0`; a `~> 0.4` constraint will not receive further releases, so re-pin to the LiteLLM version your proxy runs (for example `~> 1.99.0`). Existing `0.x` versions remain in the registry and keep verifying
|
||||
|
||||
|
|
|
|||
|
|
@ -183,16 +183,16 @@ The following arguments are supported:
|
|||
* Numeric strings are parsed first as integers; if integer parsing fails, parsed as floats (e.g., `"16384"` -> 16384, `"0.75"` -> 0.75)
|
||||
* JSON strings (starting with `[` or `{`) are parsed as JSON objects/arrays
|
||||
* Non-convertible strings remain strings
|
||||
* Non-string map values (if supplied) are passed through unchanged.
|
||||
* The provider merges these keys into the `litellm_params` payload sent to the API.
|
||||
* Note: the remote API may not echo back all custom parameters; this provider preserves `additional_litellm_params` in state when present in configuration.
|
||||
|
||||
**Special parameter: `additional_drop_params`**
|
||||
* When `additional_drop_params` is provided as a JSON array string, it specifies parameters to remove from the final `litellm_params` before sending to the API
|
||||
* This allows you to override or remove built-in parameters if needed
|
||||
* The `additional_drop_params` key itself is not included in the final parameters
|
||||
**`additional_drop_params`**
|
||||
* Give it a JSON array string, such as `jsonencode(["store"])`. The provider sends the list to the proxy as the deployment's `additional_drop_params`, and the proxy applies it the same as `additional_drop_params` in a `model_list` entry. `terraform plan` fails on any other value, an empty value, or an entry that is not a name, and `null` entries are skipped
|
||||
* Other keys on the resource, such as `vertex_location`, are sent unchanged
|
||||
* When the configuration sets the list, refresh reads the stored list back from the proxy, so a list changed outside Terraform shows up as drift. A model whose configuration has no list keeps a list set outside Terraform out of the plan, but the proxy clears it on the next update
|
||||
* When the proxy has no stored list, as on a model an older provider created, the next plan shows the list and every listed key the configuration sets as changes, since the next apply writes them
|
||||
|
||||
Example showing booleans, integers, floats, strings, and parameter dropping:
|
||||
Example showing booleans, integers, floats, strings, JSON, and dropped request parameters:
|
||||
|
||||
```hcl
|
||||
resource "litellm_model" "with_additional" {
|
||||
|
|
@ -208,7 +208,7 @@ The following arguments are supported:
|
|||
"scale" = "0.75" # becomes float 0.75
|
||||
"note" = "for testing" # stays string
|
||||
"complex_config" = "{\"nested\": {\"value\": 42}}" # parsed as JSON object
|
||||
"additional_drop_params" = "[\"reasoningEffort\"]" # removes reasoningEffort parameter
|
||||
"additional_drop_params" = "[\"reasoning_effort\"]" # proxy drops reasoning_effort from requests
|
||||
}
|
||||
}
|
||||
```
|
||||
|
|
|
|||
|
|
@ -9,8 +9,7 @@ provider "litellm" {
|
|||
# - Numeric strings will be parsed to integer (if possible) otherwise float.
|
||||
# - JSON strings (starting with [ or {) will be parsed as JSON objects/arrays.
|
||||
# - Non-convertible strings remain strings.
|
||||
# - Non-string map values are passed through unchanged.
|
||||
# - Use "additional_drop_params" as a JSON array to remove parameters from the final request.
|
||||
# - "additional_drop_params" takes a JSON array of request parameters the proxy drops before calling the model provider.
|
||||
|
||||
resource "litellm_model" "with_additional" {
|
||||
model_name = "custom-model"
|
||||
|
|
@ -21,14 +20,12 @@ resource "litellm_model" "with_additional" {
|
|||
|
||||
# Additional parameters not exposed as first-class arguments
|
||||
additional_litellm_params = {
|
||||
"use_fine_tune" = "true" # becomes boolean true
|
||||
"max_context" = "16384" # becomes integer 16384
|
||||
"temperature_scale" = "0.75" # becomes float 0.75
|
||||
"experimental_feature" = "enabled" # stays string "enabled"
|
||||
"complex_config" = "{\"nested\": {\"value\": 42}}" # parsed as JSON object
|
||||
"additional_drop_params" = "[\"reasoningEffort\"]" # removes reasoningEffort parameter
|
||||
# You may also pass non-string values (they will be passed through unchanged)
|
||||
# "raw_flag" = true
|
||||
"use_fine_tune" = "true" # becomes boolean true
|
||||
"max_context" = "16384" # becomes integer 16384
|
||||
"temperature_scale" = "0.75" # becomes float 0.75
|
||||
"experimental_feature" = "enabled" # stays string "enabled"
|
||||
"complex_config" = "{\"nested\": {\"value\": 42}}" # parsed as JSON object
|
||||
"additional_drop_params" = "[\"reasoning_effort\"]" # proxy drops reasoning_effort from requests
|
||||
}
|
||||
|
||||
# Cost configuration (optional)
|
||||
|
|
@ -47,9 +44,9 @@ resource "litellm_model" "azure_with_drop_params" {
|
|||
tier = "paid"
|
||||
mode = "completion"
|
||||
|
||||
# Drop the reasoningEffort parameter that might be automatically added
|
||||
# Have the proxy drop reasoning_effort from requests before they reach Azure
|
||||
additional_litellm_params = {
|
||||
"additional_drop_params" = "[\"reasoningEffort\"]"
|
||||
"additional_drop_params" = "[\"reasoning_effort\"]"
|
||||
}
|
||||
|
||||
input_cost_per_million_tokens = 0.25
|
||||
|
|
|
|||
|
|
@ -7,10 +7,11 @@ import (
|
|||
|
||||
func resourceLiteLLMModel() *schema.Resource {
|
||||
return &schema.Resource{
|
||||
Create: resourceLiteLLMModelCreate,
|
||||
Read: resourceLiteLLMModelRead,
|
||||
Update: resourceLiteLLMModelUpdate,
|
||||
Delete: resourceLiteLLMModelDelete,
|
||||
Create: resourceLiteLLMModelCreate,
|
||||
Read: resourceLiteLLMModelRead,
|
||||
Update: resourceLiteLLMModelUpdate,
|
||||
Delete: resourceLiteLLMModelDelete,
|
||||
CustomizeDiff: validateAdditionalDropParams,
|
||||
Importer: &schema.ResourceImporter{
|
||||
StateContext: schema.ImportStatePassthroughContext,
|
||||
},
|
||||
|
|
@ -237,7 +238,8 @@ func resourceLiteLLMModel() *schema.Resource {
|
|||
Elem: &schema.Schema{
|
||||
Type: schema.TypeString,
|
||||
},
|
||||
Description: "Additional parameters to pass to litellm_params beyond the standard ones",
|
||||
DiffSuppressFunc: suppressEquivalentDropParams,
|
||||
Description: "Additional parameters to pass to litellm_params beyond the standard ones",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,13 @@
|
|||
package litellm
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"maps"
|
||||
"net/url"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
|
@ -68,6 +71,119 @@ func secretFromResource(d *schema.ResourceData, attr string) string {
|
|||
return d.Get(attr).(string)
|
||||
}
|
||||
|
||||
const additionalDropParamsKey = "additional_drop_params"
|
||||
|
||||
func parseAdditionalDropParams(raw string) ([]string, error) {
|
||||
var items []interface{}
|
||||
if err := json.Unmarshal([]byte(raw), &items); err != nil {
|
||||
return nil, fmt.Errorf("additional_litellm_params.%s must be a JSON list of parameter names, such as jsonencode([\"store\"]): %w", additionalDropParamsKey, err)
|
||||
}
|
||||
names := []string{}
|
||||
for _, item := range items {
|
||||
if item == nil {
|
||||
continue
|
||||
}
|
||||
name, ok := item.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("additional_litellm_params.%s entries must be parameter names, got %v", additionalDropParamsKey, item)
|
||||
}
|
||||
names = append(names, name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func validateAdditionalDropParams(_ context.Context, d *schema.ResourceDiff, _ interface{}) error {
|
||||
params, _ := d.Get("additional_litellm_params").(map[string]interface{})
|
||||
raw, set := params[additionalDropParamsKey].(string)
|
||||
if !set {
|
||||
return nil
|
||||
}
|
||||
_, err := parseAdditionalDropParams(raw)
|
||||
return err
|
||||
}
|
||||
|
||||
func suppressEquivalentDropParams(key, oldValue, newValue string, _ *schema.ResourceData) bool {
|
||||
if key != "additional_litellm_params."+additionalDropParamsKey {
|
||||
return false
|
||||
}
|
||||
oldNames, oldErr := parseAdditionalDropParams(oldValue)
|
||||
newNames, newErr := parseAdditionalDropParams(newValue)
|
||||
return oldErr == nil && newErr == nil && slices.Equal(oldNames, newNames)
|
||||
}
|
||||
|
||||
var attributeSentAsLiteLLMParam = map[string]string{
|
||||
"custom_llm_provider": "custom_llm_provider",
|
||||
"tpm": "tpm",
|
||||
"rpm": "rpm",
|
||||
"input_cost_per_token": "input_cost_per_million_tokens",
|
||||
"output_cost_per_token": "output_cost_per_million_tokens",
|
||||
"api_key": "model_api_key",
|
||||
"api_base": "model_api_base",
|
||||
"api_version": "api_version",
|
||||
"input_cost_per_pixel": "input_cost_per_pixel",
|
||||
"output_cost_per_pixel": "output_cost_per_pixel",
|
||||
"input_cost_per_second": "input_cost_per_second",
|
||||
"output_cost_per_second": "output_cost_per_second",
|
||||
"aws_access_key_id": "aws_access_key_id",
|
||||
"aws_secret_access_key": "aws_secret_access_key",
|
||||
"aws_region_name": "aws_region_name",
|
||||
"aws_session_name": "aws_session_name",
|
||||
"aws_role_name": "aws_role_name",
|
||||
"vertex_project": "vertex_project",
|
||||
"vertex_location": "vertex_location",
|
||||
"vertex_credentials": "vertex_credentials",
|
||||
"reasoning_effort": "reasoning_effort",
|
||||
"thinking": "thinking_enabled",
|
||||
"merge_reasoning_content_in_choices": "merge_reasoning_content_in_choices",
|
||||
}
|
||||
|
||||
func refreshAdditionalDropParams(d *schema.ResourceData, stored interface{}) {
|
||||
params, _ := d.Get("additional_litellm_params").(map[string]interface{})
|
||||
raw, managed := params[additionalDropParamsKey].(string)
|
||||
if !managed {
|
||||
return
|
||||
}
|
||||
refreshed := maps.Clone(params)
|
||||
if stored != nil {
|
||||
encoded, _ := json.Marshal(stored)
|
||||
refreshed[additionalDropParamsKey] = string(encoded)
|
||||
d.Set("additional_litellm_params", refreshed)
|
||||
return
|
||||
}
|
||||
delete(refreshed, additionalDropParamsKey)
|
||||
listed, _ := parseAdditionalDropParams(raw)
|
||||
for _, key := range listed {
|
||||
delete(refreshed, key)
|
||||
if attribute, ok := attributeSentAsLiteLLMParam[key]; ok {
|
||||
d.Set(attribute, nil)
|
||||
}
|
||||
}
|
||||
d.Set("additional_litellm_params", refreshed)
|
||||
}
|
||||
|
||||
func coerceAdditionalParamValue(strValue string) interface{} {
|
||||
trimmedValue := strings.TrimSpace(strValue)
|
||||
if strings.HasPrefix(trimmedValue, "[") || strings.HasPrefix(trimmedValue, "{") {
|
||||
var parsedValue interface{}
|
||||
if err := json.Unmarshal([]byte(strValue), &parsedValue); err == nil {
|
||||
return parsedValue
|
||||
}
|
||||
}
|
||||
switch strValue {
|
||||
case "true":
|
||||
return true
|
||||
case "false":
|
||||
return false
|
||||
}
|
||||
if intValue, err := strconv.Atoi(strValue); err == nil {
|
||||
return intValue
|
||||
}
|
||||
if floatValue, err := strconv.ParseFloat(strValue, 64); err == nil {
|
||||
return floatValue
|
||||
}
|
||||
return strValue
|
||||
}
|
||||
|
||||
func createOrUpdateModel(d *schema.ResourceData, m interface{}, isUpdate bool) error {
|
||||
client, ok := m.(*Client)
|
||||
if !ok {
|
||||
|
|
@ -175,76 +291,17 @@ func createOrUpdateModel(d *schema.ResourceData, m interface{}, isUpdate bool) e
|
|||
litellmParams["thinking"] = thinking
|
||||
}
|
||||
|
||||
// Add additional parameters if provided
|
||||
if additionalParams, ok := d.GetOk("additional_litellm_params"); ok {
|
||||
var dropParams []string
|
||||
|
||||
for key, value := range additionalParams.(map[string]interface{}) {
|
||||
// Convert string values to appropriate types where possible
|
||||
if strValue, ok := value.(string); ok {
|
||||
// Check if it's JSON (starts with [ or {)
|
||||
trimmedValue := strings.TrimSpace(strValue)
|
||||
if strings.HasPrefix(trimmedValue, "[") || strings.HasPrefix(trimmedValue, "{") {
|
||||
var parsedValue interface{}
|
||||
if err := json.Unmarshal([]byte(strValue), &parsedValue); err == nil {
|
||||
// Successfully parsed JSON
|
||||
if key == "additional_drop_params" {
|
||||
// Handle drop params specially
|
||||
if dropList, ok := parsedValue.([]interface{}); ok {
|
||||
for _, item := range dropList {
|
||||
if paramStr, ok := item.(string); ok {
|
||||
dropParams = append(dropParams, paramStr)
|
||||
}
|
||||
}
|
||||
}
|
||||
continue // Don't add to litellmParams
|
||||
} else {
|
||||
litellmParams[key] = parsedValue
|
||||
}
|
||||
} else {
|
||||
// Not valid JSON, apply existing conversion logic
|
||||
if strValue == "true" {
|
||||
litellmParams[key] = true
|
||||
} else if strValue == "false" {
|
||||
litellmParams[key] = false
|
||||
} else {
|
||||
// Try to convert numeric strings
|
||||
if intValue, err := strconv.Atoi(strValue); err == nil {
|
||||
litellmParams[key] = intValue
|
||||
} else if floatValue, err := strconv.ParseFloat(strValue, 64); err == nil {
|
||||
litellmParams[key] = floatValue
|
||||
} else {
|
||||
// Keep as string
|
||||
litellmParams[key] = strValue
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Apply existing conversion logic for non-JSON strings
|
||||
if strValue == "true" {
|
||||
litellmParams[key] = true
|
||||
} else if strValue == "false" {
|
||||
litellmParams[key] = false
|
||||
} else {
|
||||
// Try to convert numeric strings
|
||||
if intValue, err := strconv.Atoi(strValue); err == nil {
|
||||
litellmParams[key] = intValue
|
||||
} else if floatValue, err := strconv.ParseFloat(strValue, 64); err == nil {
|
||||
litellmParams[key] = floatValue
|
||||
} else {
|
||||
// Keep as string
|
||||
litellmParams[key] = strValue
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
litellmParams[key] = value
|
||||
if key != additionalDropParamsKey {
|
||||
litellmParams[key] = coerceAdditionalParamValue(value.(string))
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// Apply drop params at the end
|
||||
for _, paramToDrop := range dropParams {
|
||||
delete(litellmParams, paramToDrop)
|
||||
dropParams, err := parseAdditionalDropParams(value.(string))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
litellmParams[key] = dropParams
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -419,6 +476,7 @@ func resourceLiteLLMModelRead(d *schema.ResourceData, m interface{}) error {
|
|||
if _, ok := d.GetOk("additional_litellm_params"); ok {
|
||||
d.Set("additional_litellm_params", d.Get("additional_litellm_params"))
|
||||
}
|
||||
refreshAdditionalDropParams(d, modelResp.LiteLLMParams.AdditionalDropParams)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,8 +3,11 @@ package litellm
|
|||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"maps"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"reflect"
|
||||
"slices"
|
||||
"testing"
|
||||
|
||||
"github.com/hashicorp/go-cty/cty"
|
||||
|
|
@ -116,6 +119,246 @@ func TestResourceLiteLLMModelCreateOmitsUnsetDisplayName(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestResourceLiteLLMModelCreateSendsAdditionalLiteLLMParams(t *testing.T) {
|
||||
var createPayload map[string]interface{}
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/model/new":
|
||||
if err := json.NewDecoder(r.Body).Decode(&createPayload); err != nil {
|
||||
t.Errorf("failed to decode create payload: %v", err)
|
||||
}
|
||||
w.Write([]byte(modelInfoBody("")))
|
||||
case "/model/info":
|
||||
w.Write([]byte(modelInfoBody("")))
|
||||
default:
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
d := schema.TestResourceDataRaw(t, resourceLiteLLMModel().Schema, map[string]interface{}{
|
||||
"model_name": "vertex-claude-haiku",
|
||||
"custom_llm_provider": "vertex_ai",
|
||||
"base_model": "claude-haiku-4-5",
|
||||
"vertex_project": "test-project",
|
||||
"vertex_location": "eu",
|
||||
"additional_litellm_params": map[string]interface{}{
|
||||
"additional_drop_params": `["store", "vertex_location"]`,
|
||||
"tags": ` ["prod", "eu"]`,
|
||||
"headers": `{"x-team": "search"}`,
|
||||
"use_in_pass_through": "true",
|
||||
"merge_reasoning_content_in_choices": "false",
|
||||
"max_parallel_requests": "8",
|
||||
"input_cost_per_token": "0.5",
|
||||
"description": "[draft] haiku",
|
||||
},
|
||||
})
|
||||
|
||||
if err := resourceLiteLLMModelCreate(d, NewClient(srv.URL, "test-key", true)); err != nil {
|
||||
t.Fatalf("create failed: %v", err)
|
||||
}
|
||||
|
||||
litellmParams, ok := createPayload["litellm_params"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("expected litellm_params object in create payload, got %v", createPayload["litellm_params"])
|
||||
}
|
||||
want := map[string]interface{}{
|
||||
"additional_drop_params": []interface{}{"store", "vertex_location"},
|
||||
"vertex_project": "test-project",
|
||||
"vertex_location": "eu",
|
||||
"tags": []interface{}{"prod", "eu"},
|
||||
"headers": map[string]interface{}{"x-team": "search"},
|
||||
"use_in_pass_through": true,
|
||||
"merge_reasoning_content_in_choices": false,
|
||||
"max_parallel_requests": float64(8),
|
||||
"input_cost_per_token": 0.5,
|
||||
"description": "[draft] haiku",
|
||||
}
|
||||
for key, wantValue := range want {
|
||||
if got := litellmParams[key]; !reflect.DeepEqual(got, wantValue) {
|
||||
t.Errorf("litellm_params[%q] = %#v, want %#v", key, got, wantValue)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const terraformUnknownValue = "74D93920-ED26-11E3-AC10-0800200C9A66"
|
||||
|
||||
func TestResourceLiteLLMModelChecksAdditionalDropParams(t *testing.T) {
|
||||
cases := map[string]struct {
|
||||
dropParams string
|
||||
want []interface{}
|
||||
wantErr bool
|
||||
}{
|
||||
"null entry from an unset variable is skipped": {dropParams: `["store", null]`, want: []interface{}{"store"}},
|
||||
"number entry is rejected": {dropParams: `["store", 1]`, wantErr: true},
|
||||
"object entry is rejected": {dropParams: `["store", {"name": "temperature"}]`, wantErr: true},
|
||||
"object is rejected": {dropParams: `{"store": true}`, wantErr: true},
|
||||
"bare name is rejected": {dropParams: "store", wantErr: true},
|
||||
"empty value is rejected": {dropParams: "", wantErr: true},
|
||||
}
|
||||
for name, tc := range cases {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
var createPayload map[string]interface{}
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/model/new":
|
||||
if err := json.NewDecoder(r.Body).Decode(&createPayload); err != nil {
|
||||
t.Errorf("failed to decode create payload: %v", err)
|
||||
}
|
||||
w.Write([]byte(modelInfoBody("")))
|
||||
case "/model/info":
|
||||
w.Write([]byte(modelInfoBody("")))
|
||||
default:
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
res := resourceLiteLLMModel()
|
||||
config := map[string]interface{}{
|
||||
"model_name": "vertex-claude-haiku",
|
||||
"custom_llm_provider": "vertex_ai",
|
||||
"base_model": "claude-haiku-4-5",
|
||||
"additional_litellm_params": map[string]interface{}{"additional_drop_params": tc.dropParams},
|
||||
}
|
||||
if _, err := res.Diff(context.Background(), nil, terraform.NewResourceConfigRaw(config), nil); (err != nil) != tc.wantErr {
|
||||
t.Errorf("plan error = %v, want error %v", err, tc.wantErr)
|
||||
}
|
||||
|
||||
d := schema.TestResourceDataRaw(t, res.Schema, config)
|
||||
err := resourceLiteLLMModelCreate(d, NewClient(srv.URL, "test-key", true))
|
||||
if tc.wantErr {
|
||||
if err == nil {
|
||||
t.Fatal("expected create to fail")
|
||||
}
|
||||
if createPayload != nil {
|
||||
t.Errorf("expected no /model/new request, got %v", createPayload)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("create failed: %v", err)
|
||||
}
|
||||
litellmParams := createPayload["litellm_params"].(map[string]interface{})
|
||||
if got := litellmParams["additional_drop_params"]; !reflect.DeepEqual(got, tc.want) {
|
||||
t.Errorf("additional_drop_params = %#v, want %#v", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceLiteLLMModelPlanAcceptsDropParamsUnknownUntilApply(t *testing.T) {
|
||||
config := terraform.NewResourceConfigRaw(map[string]interface{}{
|
||||
"model_name": "vertex-claude-haiku",
|
||||
"custom_llm_provider": "vertex_ai",
|
||||
"additional_litellm_params": map[string]interface{}{"additional_drop_params": terraformUnknownValue},
|
||||
})
|
||||
if _, err := resourceLiteLLMModel().Diff(context.Background(), nil, config, nil); err != nil {
|
||||
t.Errorf("plan failed on a drop list not known until apply: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceLiteLLMModelPlanComparesAdditionalDropParamsWithProxy(t *testing.T) {
|
||||
const (
|
||||
listKey = "additional_litellm_params.additional_drop_params"
|
||||
parallelKey = "additional_litellm_params.max_parallel_requests"
|
||||
)
|
||||
cases := map[string]struct {
|
||||
configured string
|
||||
configLocation string
|
||||
stored interface{}
|
||||
storedLocation string
|
||||
extraConfig map[string]interface{}
|
||||
wantChanged []string
|
||||
}{
|
||||
"list never sent by an older provider": {configured: `["store"]`, configLocation: "eu", storedLocation: "eu", wantChanged: []string{listKey}},
|
||||
"listed keys an older provider deleted": {
|
||||
configured: `["store", "vertex_location", "tpm", "max_parallel_requests"]`,
|
||||
configLocation: "eu",
|
||||
wantChanged: []string{listKey, "vertex_location", "tpm", parallelKey},
|
||||
},
|
||||
"listed flag an older provider deleted": {
|
||||
configured: `["merge_reasoning_content_in_choices"]`,
|
||||
configLocation: "eu",
|
||||
storedLocation: "eu",
|
||||
extraConfig: map[string]interface{}{"merge_reasoning_content_in_choices": true},
|
||||
wantChanged: []string{listKey, "merge_reasoning_content_in_choices"},
|
||||
},
|
||||
"listed key an older provider still sent": {
|
||||
configured: `["litellm_credential_name"]`,
|
||||
configLocation: "eu",
|
||||
storedLocation: "eu",
|
||||
extraConfig: map[string]interface{}{"litellm_credential_name": "vertex-cred"},
|
||||
wantChanged: []string{listKey},
|
||||
},
|
||||
"list changed in the Admin UI": {configured: `["store"]`, configLocation: "eu", stored: []string{"temperature"}, storedLocation: "eu", wantChanged: []string{listKey}},
|
||||
"list set outside Terraform": {configLocation: "eu", stored: []string{"store"}, storedLocation: "eu"},
|
||||
"same list written with spaces": {configured: `["store", "vertex_location"]`, configLocation: "eu", stored: []string{"store", "vertex_location"}, storedLocation: "eu"},
|
||||
"null entry the proxy never received": {configured: `["store", null]`, configLocation: "eu", stored: []string{"store"}, storedLocation: "eu"},
|
||||
"listed key left out of the config": {configured: `["vertex_location"]`, stored: []string{"vertex_location"}, storedLocation: "us-east5"},
|
||||
"no list on the model or in the proxy": {configLocation: "eu", storedLocation: "eu"},
|
||||
}
|
||||
for name, tc := range cases {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
litellmParams := map[string]interface{}{"model": "vertex_ai/claude-haiku-4-5", "custom_llm_provider": "vertex_ai"}
|
||||
if tc.stored != nil {
|
||||
litellmParams["additional_drop_params"] = tc.stored
|
||||
}
|
||||
if tc.storedLocation != "" {
|
||||
litellmParams["vertex_location"] = tc.storedLocation
|
||||
}
|
||||
body, _ := json.Marshal(map[string]interface{}{
|
||||
"model_name": "vertex-claude-haiku",
|
||||
"litellm_params": litellmParams,
|
||||
"model_info": map[string]interface{}{"id": "model-123", "db_model": true, "base_model": "claude-haiku-4-5"},
|
||||
})
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Write(body)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
res := resourceLiteLLMModel()
|
||||
additionalParams := map[string]interface{}{"max_parallel_requests": "8"}
|
||||
if tc.configured != "" {
|
||||
additionalParams["additional_drop_params"] = tc.configured
|
||||
}
|
||||
config := map[string]interface{}{
|
||||
"model_name": "vertex-claude-haiku",
|
||||
"custom_llm_provider": "vertex_ai",
|
||||
"base_model": "claude-haiku-4-5",
|
||||
"tpm": 1000,
|
||||
"additional_litellm_params": additionalParams,
|
||||
}
|
||||
if tc.configLocation != "" {
|
||||
config["vertex_location"] = tc.configLocation
|
||||
}
|
||||
maps.Copy(config, tc.extraConfig)
|
||||
d := schema.TestResourceDataRaw(t, res.Schema, config)
|
||||
d.SetId("model-123")
|
||||
if err := resourceLiteLLMModelRead(d, NewClient(srv.URL, "test-key", true)); err != nil {
|
||||
t.Fatalf("read failed: %v", err)
|
||||
}
|
||||
|
||||
diff, err := res.Diff(context.Background(), d.State(), terraform.NewResourceConfigRaw(config), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("diff failed: %v", err)
|
||||
}
|
||||
for _, key := range []string{listKey, "vertex_location", "tpm", parallelKey, "merge_reasoning_content_in_choices", "litellm_credential_name"} {
|
||||
attr := (*terraform.ResourceAttrDiff)(nil)
|
||||
if diff != nil {
|
||||
attr = diff.Attributes[key]
|
||||
}
|
||||
got := attr != nil && attr.Old != attr.New
|
||||
if want := slices.Contains(tc.wantChanged, key); got != want {
|
||||
t.Errorf("plan shows %s change = %v, want %v", key, got, want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceLiteLLMModelReadDisplayName(t *testing.T) {
|
||||
cases := map[string]struct {
|
||||
serverBody string
|
||||
|
|
|
|||
|
|
@ -120,6 +120,7 @@ type LiteLLMParams struct {
|
|||
VertexProject string `json:"vertex_project,omitempty"`
|
||||
VertexLocation string `json:"vertex_location,omitempty"`
|
||||
VertexCredentials string `json:"vertex_credentials,omitempty"`
|
||||
AdditionalDropParams interface{} `json:"additional_drop_params,omitempty"`
|
||||
}
|
||||
|
||||
// ModelInfo represents information about a model.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue