scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-15 04:19:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 121

feat(terraform): add team_member_model_max_budget to litellm_team (#46274)

Browse source 
* feat(terraform): add team_member_model_max_budget to litellm_team

The proxy now accepts per-model budgets for team members, so a member can
be capped on one model and keep using the team's others. The provider had
no way to set or read it, so it could only be managed by hand.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(terraform): tighten team_member_model_max_budget per review

Reject tpm_limit/rpm_limit at plan time since the proxy rejects them on team
budgets, skip sending an empty map on create so no empty member budget row
is made, and document the router-alias matching and minimum proxy version.

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
This commit is contained in:
Matthew Howard 2026-10-14 18:52:54 -04:00 • committed by GitHub
parent 220bbfc0ce
commit a114356542
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 352 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
terraform/provider/CHANGELOG.md
View file

@ -16,6 +16,7 @@ longer signal it.
### Added
- **team**: Optional `team_member_model_max_budget` argument on `litellm_team`, a JSON string of per-model budgets applied to every team member and sent as `team_member_model_max_budget` on `/team/new` and `/team/update`; read back from `team_member_budget_table.model_max_budget` on `/team/info` so out-of-band changes show as drift, with `budget_limit`/`time_period` treated as equal to the `max_budget`/`budget_duration` spelling the proxy stores; removing it sends `{}`, which clears every per-model member budget; `tpm_limit`/`rpm_limit` are rejected at plan time because the proxy rejects them on team budgets. Needs a proxy that includes [#43887](https://github.com/BerriAI/litellm/pull/43887): v1.105.0 and v1.106.0-rc.1 accept the field but silently drop it, so every plan shows a diff
- **model**: Optional `display_name` argument on `litellm_model`, sent as `model_info.display_name` and returned as `display_name` by `/v1/models`, so client model pickers show a readable name; changes are persisted through `/model/{id}/update` since `/model/update` ignores `model_info`; also exported by the `litellm_model` and `litellm_models` data sources
- **key**: Computed `server_metadata` attribute on `litellm_key` exposing every metadata entry the proxy stores, so metadata created outside Terraform is visible in state and drift on it shows on refresh, while `metadata` keeps tracking only the declared entries and updates keep preserving undeclared ones
- **team_member_add**: `tpm_limit`, `rpm_limit`, `budget_duration`, and `allowed_models` attributes on `litellm_team_member_add`, applied to every member of the resource; `budget_duration` and `allowed_models` ride on `/team/member_add`, while the limits are sent through `/team/member_update`, which is where the proxy accepts them

20
terraform/provider/docs/resources/team.md
View file

@ -98,6 +98,24 @@ resource "litellm_team" "model_dependent_team" {
}
```
### Team with Per-Member Model Budgets
```hcl
resource "litellm_team" "engineering" {
team_alias = "engineering"
models = ["claude-opus", "claude-sonnet"]
# Each member may spend $500 a day in total
team_member_budget = 500
team_member_budget_duration = "1d"
# ...of which at most $400 on Opus. Past that, Sonnet keeps working.
team_member_model_max_budget = jsonencode({
"claude-opus" = { budget_limit = 400, time_period = "1d" }
})
}
```
## Argument Reference
The following arguments are supported:
@ -150,6 +168,8 @@ The following arguments are supported:
* `team_member_key_duration` - (Optional) Lifetime for keys created by team members (e.g. `1d`, `1w`).
* `team_member_model_max_budget` - (Optional) JSON string of per-model budgets applied to each individual team member, e.g. `jsonencode({"claude-opus" = {budget_limit = 400, time_period = "1d"}})`. Each model maps to an object with `budget_limit` (or `max_budget`) and `time_period` (or `budget_duration`). The proxy rejects `tpm_limit` and `rpm_limit` on team budgets, so the provider rejects them at plan time. A member who reaches a model's cap is blocked on that model only and can keep calling the team's other models until `team_member_budget` runs out. The key is matched against the model name on the request, so for a router alias the cap applies to the alias, not the model the router picks. Remove the attribute to clear every per-model member budget. Requires a LiteLLM Enterprise license on the proxy, and a proxy that includes [#43887](https://github.com/BerriAI/litellm/pull/43887). Older proxies, including v1.105.0 and v1.106.0-rc.1, accept the field but silently drop it, so every plan shows a diff.
* `model_rpm_limit` - (Optional) Map of model name to requests per minute limit for that model.
* `model_tpm_limit` - (Optional) Map of model name to tokens per minute limit for that model.

90
terraform/provider/litellm/resource_team.go
View file

@ -6,6 +6,7 @@ import (
"io"
"log"
"net/http"
"reflect"
"github.com/google/uuid"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
@ -135,6 +136,13 @@ func ResourceLiteLLMTeam() *schema.Resource {
Type: schema.TypeString,
Optional: true,
},
"team_member_model_max_budget": {
Type: schema.TypeString,
Optional: true,
ValidateFunc: validateTeamMemberModelMaxBudget,
DiffSuppressFunc: teamMemberModelMaxBudgetEquivalent,
Description: "JSON string of per-model budgets applied to every team member (e.g. '{\"claude-opus\": {\"budget_limit\": 400, \"time_period\": \"1d\"}}'). Each entry takes budget_limit (or max_budget) and time_period (or budget_duration). Caps match the model name on the request, so behind a router alias the cap must be keyed on the alias, not the model the router picks. Requires a LiteLLM Enterprise license and a proxy that includes https://github.com/BerriAI/litellm/pull/43887; older proxies drop the value, so every plan shows a diff",
},
"model_rpm_limit": {
Type: schema.TypeMap,
Optional: true,
@ -280,6 +288,11 @@ func resourceLiteLLMTeamRead(d *schema.ResourceData, m interface{}) error {
if teamResp.AllowedPassthroughRoutes != nil {
d.Set("allowed_passthrough_routes", teamResp.AllowedPassthroughRoutes)
}
var memberModelMaxBudget map[string]interface{}
if teamResp.TeamMemberBudgetTable != nil {
memberModelMaxBudget = teamResp.TeamMemberBudgetTable.ModelMaxBudget
}
d.Set("team_member_model_max_budget", keyModelMaxBudgetJSON(memberModelMaxBudget))
// Explicitly fetch the current permissions from the API
permResp, err := getTeamPermissions(client, d.Id())
@ -362,6 +375,75 @@ func resourceLiteLLMTeamDelete(d *schema.ResourceData, m interface{}) error {
return nil
}
// The proxy rejects tpm_limit/rpm_limit on team budgets, so only spend fields are allowed here.
var teamMemberModelBudgetFields = map[string]bool{
"budget_limit": true,
"max_budget": true,
"time_period": true,
"budget_duration": true,
}
func validateTeamMemberModelMaxBudget(v interface{}, k string) ([]string, []error) {
var parsed map[string]json.RawMessage
if err := json.Unmarshal([]byte(v.(string)), &parsed); err != nil || parsed == nil {
return nil, []error{fmt.Errorf("%q must be a JSON object keyed by model name, got %s", k, v)}
}
for model, cfg := range parsed {
var budget map[string]json.RawMessage
if err := json.Unmarshal(cfg, &budget); err != nil || len(budget) == 0 {
return nil, []error{fmt.Errorf("%q[%q] must be a budget object such as {\"budget_limit\": 400, \"time_period\": \"1d\"}, got %s", k, model, cfg)}
}
for field := range budget {
if !teamMemberModelBudgetFields[field] {
return nil, []error{fmt.Errorf("%q[%q] has unsupported field %q; team member budgets accept budget_limit, max_budget, time_period and budget_duration only", k, model, field)}
}
}
}
return nil, nil
}
// The proxy stores budget_limit/time_period as max_budget/budget_duration, so both spellings must compare equal.
var modelBudgetFieldAliases = map[string]string{
"budget_limit": "max_budget",
"time_period": "budget_duration",
}
func normalizeModelMaxBudget(raw string) (map[string]interface{}, bool) {
if raw == "" {
return map[string]interface{}{}, true
}
var parsed map[string]interface{}
if err := json.Unmarshal([]byte(raw), &parsed); err != nil {
return nil, false
}
normalized := make(map[string]interface{}, len(parsed))
for model, cfg := range parsed {
budget, ok := cfg.(map[string]interface{})
if !ok {
normalized[model] = cfg
continue
}
entry := make(map[string]interface{}, len(budget))
for field, value := range budget {
if value == nil {
continue
}
if canonical, isAlias := modelBudgetFieldAliases[field]; isAlias {
field = canonical
}
entry[field] = value
}
normalized[model] = entry
}
return normalized, true
}
func teamMemberModelMaxBudgetEquivalent(k, oldValue, newValue string, d *schema.ResourceData) bool {
oldParsed, okOld := normalizeModelMaxBudget(oldValue)
newParsed, okNew := normalizeModelMaxBudget(newValue)
return okOld && okNew && reflect.DeepEqual(oldParsed, newParsed)
}
func resolveTeamID(d *schema.ResourceData) string {
if v, ok := d.GetOk("team_id"); ok {
return v.(string)
@ -402,6 +484,14 @@ func buildTeamData(d *schema.ResourceData, teamID string) map[string]interface{}
teamData["tags"] = v
}
// The proxy treats a missing field as "leave unchanged", so an empty map is sent only to clear a value that was set.
memberModelMaxBudget := parseKeyModelMaxBudget(d.Get("team_member_model_max_budget").(string))
if len(memberModelMaxBudget) > 0 {
teamData["team_member_model_max_budget"] = memberModelMaxBudget
} else if old, _ := d.GetChange("team_member_model_max_budget"); len(parseKeyModelMaxBudget(old.(string))) > 0 {
teamData["team_member_model_max_budget"] = memberModelMaxBudget
}
if metadata := buildTeamMetadata(d); metadata != nil {
teamData["metadata"] = metadata
}

235
terraform/provider/litellm/resource_team_test.go
View file

@ -472,3 +472,238 @@ func TestHandleResponseAcceptsFullSuccessRange(t *testing.T) {
})
}
}
func TestTeamCreateSendsTeamMemberModelMaxBudget(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
d := newTeamResourceData(t, map[string]interface{}{
"team_alias": "eng",
"team_member_model_max_budget": `{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`,
})
if err := resourceLiteLLMTeamCreate(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("create failed: %v", err)
}
want := map[string]interface{}{"claude-opus": map[string]interface{}{"budget_limit": 400.0, "time_period": "1d"}}
if got := captured["team_member_model_max_budget"]; !reflect.DeepEqual(got, want) {
t.Fatalf("payload team_member_model_max_budget = %v, want %v", got, want)
}
}
func TestTeamReadMapsTeamMemberModelMaxBudgetFromBudgetTable(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{
"team_id": "team-1",
"team_info": {
"team_id": "team-1",
"team_alias": "eng",
"team_member_budget_table": {
"max_budget": 500.0,
"model_max_budget": {"claude-opus": {"max_budget": 400.0, "budget_duration": "1d"}}
}
}
}`)
defer srv.Close()
d := newTeamResourceData(t, map[string]interface{}{"team_alias": "eng"})
d.SetId("team-1")
if err := resourceLiteLLMTeamRead(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("read returned error: %v", err)
}
var got map[string]interface{}
if err := json.Unmarshal([]byte(d.Get("team_member_model_max_budget").(string)), &got); err != nil {
t.Fatalf("team_member_model_max_budget in state is not JSON: %v", err)
}
want := map[string]interface{}{"claude-opus": map[string]interface{}{"max_budget": 400.0, "budget_duration": "1d"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("team_member_model_max_budget = %v, want %v", got, want)
}
}
func TestTeamReadClearsTeamMemberModelMaxBudgetWhenAbsent(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng","team_member_budget_table":{"model_max_budget":null}}}`)
defer srv.Close()
d := newTeamResourceData(t, map[string]interface{}{
"team_alias": "eng",
"team_member_model_max_budget": `{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`,
})
d.SetId("team-1")
if err := resourceLiteLLMTeamRead(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("read returned error: %v", err)
}
if got := d.Get("team_member_model_max_budget").(string); got != "" {
t.Errorf("team_member_model_max_budget = %q, want empty so drift shows up", got)
}
}
func TestTeamUpdateClearsRemovedTeamMemberModelMaxBudget(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
res := ResourceLiteLLMTeam()
priorData := schema.TestResourceDataRaw(t, res.Schema, map[string]interface{}{
"team_alias": "eng",
"team_member_model_max_budget": `{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`,
})
priorData.SetId("team-1")
prior := priorData.State()
config := terraform.NewResourceConfigRaw(map[string]interface{}{"team_alias": "eng"})
diff, err := res.Diff(context.Background(), prior, config, nil)
if err != nil {
t.Fatalf("diff failed: %v", err)
}
d, err := schema.InternalMap(res.Schema).Data(prior, diff)
if err != nil {
t.Fatalf("data failed: %v", err)
}
if err := resourceLiteLLMTeamUpdate(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("update failed: %v", err)
}
if got, ok := captured["team_member_model_max_budget"]; !ok || !reflect.DeepEqual(got, map[string]interface{}{}) {
t.Errorf("payload team_member_model_max_budget = %v (present=%v), want explicit empty object", got, ok)
}
}
func TestTeamMemberModelMaxBudgetAliasesAreEquivalent(t *testing.T) {
configured := `{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`
stored := `{"claude-opus":{"max_budget":400,"budget_duration":"1d"}}`
if !teamMemberModelMaxBudgetEquivalent("", stored, configured, nil) {
t.Fatalf("budget_limit/time_period should match the stored max_budget/budget_duration spelling")
}
if teamMemberModelMaxBudgetEquivalent("", stored, `{"claude-opus": {"budget_limit": 300, "time_period": "1d"}}`, nil) {
t.Fatalf("a changed limit must not be suppressed")
}
}
func TestTeamMemberModelMaxBudgetRejectsUnknownFields(t *testing.T) {
schema := ResourceLiteLLMTeam().Schema["team_member_model_max_budget"]
for _, bad := range []string{
`{"claude-opus": {"limit": 400}}`,
`{"claude-opus": {"budget_limit": 400, "time_period": "1d", "tpm_limit": 1000}}`,
`{"claude-opus": {"budget_limit": 400, "time_period": "1d", "rpm_limit": 10}}`,
} {
if _, errs := schema.ValidateFunc(bad, "team_member_model_max_budget"); len(errs) == 0 {
t.Errorf("expected %s to be rejected", bad)
}
}
if _, errs := schema.ValidateFunc(`{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`, "team_member_model_max_budget"); len(errs) != 0 {
t.Fatalf("valid config rejected: %v", errs)
}
}
func TestTeamCreateOmitsTeamMemberModelMaxBudgetWhenUnset(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
d := newTeamResourceData(t, map[string]interface{}{"team_alias": "eng"})
if err := resourceLiteLLMTeamCreate(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("create failed: %v", err)
}
if got, ok := captured["team_member_model_max_budget"]; ok {
t.Fatalf("payload team_member_model_max_budget = %v, want key absent", got)
}
}
func TestTeamCreateOmitsEmptyTeamMemberModelMaxBudget(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
d := newTeamResourceData(t, map[string]interface{}{
"team_alias": "eng",
"team_member_model_max_budget": `{}`,
})
if err := resourceLiteLLMTeamCreate(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("create failed: %v", err)
}
if got, ok := captured["team_member_model_max_budget"]; ok {
t.Fatalf("payload team_member_model_max_budget = %v, want key absent so no empty member budget row is created", got)
}
}
func teamUpdateData(t *testing.T, prior, config map[string]interface{}) *schema.ResourceData {
t.Helper()
res := ResourceLiteLLMTeam()
priorData := schema.TestResourceDataRaw(t, res.Schema, prior)
priorData.SetId("team-1")
state := priorData.State()
diff, err := res.Diff(context.Background(), state, terraform.NewResourceConfigRaw(config), nil)
if err != nil {
t.Fatalf("diff failed: %v", err)
}
d, err := schema.InternalMap(res.Schema).Data(state, diff)
if err != nil {
t.Fatalf("data failed: %v", err)
}
return d
}
func TestTeamUpdateSendsChangedTeamMemberModelMaxBudget(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
d := teamUpdateData(t,
map[string]interface{}{"team_alias": "eng", "team_member_model_max_budget": `{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`},
map[string]interface{}{"team_alias": "eng", "team_member_model_max_budget": `{"claude-opus": {"budget_limit": 300, "time_period": "1d"}}`},
)
if err := resourceLiteLLMTeamUpdate(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("update failed: %v", err)
}
want := map[string]interface{}{"claude-opus": map[string]interface{}{"budget_limit": 300.0, "time_period": "1d"}}
if got := captured["team_member_model_max_budget"]; !reflect.DeepEqual(got, want) {
t.Fatalf("payload team_member_model_max_budget = %v, want %v", got, want)
}
}
func TestTeamUpdateSendsMemberBudgetWithPerModelBudget(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
d := teamUpdateData(t,
map[string]interface{}{"team_alias": "eng"},
map[string]interface{}{
"team_alias": "eng",
"team_member_budget": 500.0,
"team_member_budget_duration": "1d",
"team_member_model_max_budget": `{"claude-opus": {"budget_limit": 400, "time_period": "1d"}}`,
},
)
if err := resourceLiteLLMTeamUpdate(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("update failed: %v", err)
}
if got := captured["team_member_budget"]; got != 500.0 {
t.Errorf("payload team_member_budget = %v, want 500", got)
}
if got := captured["team_member_budget_duration"]; got != "1d" {
t.Errorf("payload team_member_budget_duration = %v, want 1d", got)
}
want := map[string]interface{}{"claude-opus": map[string]interface{}{"budget_limit": 400.0, "time_period": "1d"}}
if got := captured["team_member_model_max_budget"]; !reflect.DeepEqual(got, want) {
t.Errorf("payload team_member_model_max_budget = %v, want %v", got, want)
}
}
func TestTeamReadWithoutMemberBudgetTableLeavesFieldEmpty(t *testing.T) {
var captured map[string]interface{}
srv := newTeamTestServer(t, &captured, `{"team_id":"team-1","team_info":{"team_id":"team-1","team_alias":"eng"}}`)
defer srv.Close()
d := newTeamResourceData(t, map[string]interface{}{"team_alias": "eng"})
d.SetId("team-1")
if err := resourceLiteLLMTeamRead(d, NewClient(srv.URL, "test-key", true)); err != nil {
t.Fatalf("read returned error: %v", err)
}
if got := d.Get("team_member_model_max_budget").(string); got != "" {
t.Errorf("team_member_model_max_budget = %q, want empty", got)
}
}

6
terraform/provider/litellm/types.go
View file

@ -73,6 +73,12 @@ type TeamResponse struct {
ModelRPMLimit map[string]interface{} `json:"model_rpm_limit,omitempty"`
ModelTPMLimit map[string]interface{} `json:"model_tpm_limit,omitempty"`
AllowedPassthroughRoutes []string `json:"allowed_passthrough_routes,omitempty"`
TeamMemberBudgetTable *TeamMemberBudgetTable `json:"team_member_budget_table,omitempty"`
}
// TeamMemberBudgetTable is the team's default member budget row, the only place /team/info returns team_member_model_max_budget.
type TeamMemberBudgetTable struct {
ModelMaxBudget map[string]interface{} `json:"model_max_budget,omitempty"`
}
// OrganizationResponse represents a response from the API containing organization information.