mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
567383c503
* feat(ui): let the per-model budget field take a model pattern or access group * feat(proxy): let model_max_budget entries name a model family or access group, shared across replicas * fix(proxy): open each shared model budget window once for every replica, and match patterns in linear time Without Redis, the first replica to charge an entry in a new window now opens it in the window spend table and the others join it, so no flush replaces spend another replica already added. Budget patterns match by splitting on `*` instead of a regular expression, a refused request names the entry that refused it, and the key form no longer offers a typed pattern another row already budgets. * fix(proxy): read the shared budget window from the writer, and keep concurrent first charges on one replica Opening a shared window now reads the row from the writer, since a lagging read replica could hand a replica its own start and let a flush replace the others' spend. A charge that waited on the open no longer resets a local window another charge on the same replica opened meanwhile. Window tests use fixed timestamps. * fix(proxy): stop a non proxy admin from lifting a team model cap with a looser group or pattern entry Group and pattern entries now take precedence over other entries, so a team update by anyone but a proxy admin must keep each new or changed entry within every existing cap it can take precedence over. A pattern can match a provider-prefixed or wildcard-routed name, so any two patterns count as overlapping, and a model or group entry counts as outranking every pattern. * fix(proxy): narrow model budget pattern overlap, keep team caps under key patterns, and accept only typed patterns in the UI * fix(proxy): keep model family budgets through database blips, team models and key entries A failed window open no longer restarts the window at this replica's clock, so it cannot wipe shared spend. Spend charged meanwhile stays local and joins the shared window once the database answers A key entry named after a model replaces only the team entry of that name, so team pattern and group caps still apply. Team-scoped models are gated and charged under their public name, and group lookups see team deployments The team update guard compares caps by window length and only ranks patterns that can match the same name, including Bedrock vendor patterns behind a bedrock wildcard route * fix(proxy): refuse team entries that take precedence over a capped model_max_budget entry A non-proxy-admin could add a carve-out (an exact model, a narrower pattern or a group) at or under the family cap, and its own counter then multiplied the team's spend on that family. Any new entry that takes precedence over a capped entry now needs a proxy admin, and group members include the team's own public model names so team deployments cannot hide a group overlap. * test(router): cover team_public_model_names_of * docs(proxy): name pattern and access group keys in model_max_budget docstrings * fix(proxy): name the entry holding the cap in the team model budget raise refusal * fix(proxy): check precedence when a team edit makes an unenforced model_max_budget entry take effect * fix(proxy): let an org admin give a legacy model_max_budget row a budget_duration at the same or a lower cap * fix(proxy): keep a legacy model_max_budget row's stored cap when an org admin gives it a budget_duration |
||
|---|---|---|
| .. | ||
| litellm_skills | ||
| __init__.py | ||
| test_async_post_call_streaming_iterator_hook.py | ||
| test_autorouter_baseline_cache.py | ||
| test_banned_keyword_list.py | ||
| test_batch_enqueued_tokens.py | ||
| test_batch_file_validation.py | ||
| test_batch_rate_limiter.py | ||
| test_dynamic_rate_limiter.py | ||
| test_dynamic_rate_limiter_v3.py | ||
| test_image_generation_guardrails.py | ||
| test_key_management_event_hooks.py | ||
| test_max_budget_per_session_limiter.py | ||
| test_max_iterations_limiter.py | ||
| test_model_budget_shared_windows.py | ||
| test_model_max_budget_limiter.py | ||
| test_parallel_request_limiter.py | ||
| test_parallel_request_limiter_v3.py | ||
| test_post_call_failure_hook_integration.py | ||
| test_post_call_response_headers_hook.py | ||
| test_post_call_streaming_hook_integration.py | ||
| test_post_call_success_hook_integration.py | ||
| test_prompt_cache_observer.py | ||
| test_prompt_injection_detection.py | ||
| test_proxy_hooks_init.py | ||
| test_proxy_rate_limit_provider_field.py | ||
| test_proxy_track_cost_callback.py | ||
| test_rate_limiter_toctou.py | ||
| test_search_free_tier.py | ||
| test_send_invite_email.py | ||
| test_sensitive_data_routing.py | ||
| test_tpm_concurrent.py | ||
| test_unit_test_max_model_budget_limiter.py | ||
| test_user_management_event_hooks.py | ||