mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-17 04:34:25 +00:00
a76e723b26
* feat(proxy): report when each per-model budget window resets model_max_budget_usage on /key/info, /user/info and /team/info now carries budget_reset_at per model, the open window's start plus its period, or null when no window is open. The window start is read in the same batched lookup as the spend counter * fix(proxy): count a monthly budget window from its own start when reporting its reset duration_in_seconds measures Nmo from now, so a window opened on January 15 and read on February 10 reported February 12, then null while the counter still refused requests until February 15. duration_in_seconds_from measures the month from the window start, and duration_in_seconds now shares that month arithmetic * fix(proxy): measure months from timezone-aware datetimes and assert whole usage values datetime.fromtimestamp without a tz tripped DTZ006. The month arithmetic now runs on an aware local datetime, which keeps the same wall-clock result. The duration tests move to parametrized functions, since subTest payloads carrying datetimes cannot cross the xdist worker boundary, and the new usage tests compare the whole entry * fix(budgets): end a monthly budget window a calendar month after it opens Each charge recounted the month from the charge date, so a monthly window opened in a month of another length expired days before or after the reset /key/info reports. The window is now measured from its start, by the same helper the usage report uses. A period ending after year 9999 reports no reset instead of failing the read. * fix(budgets): count monthly budget windows in UTC for every router budget Months were counted on the host's local calendar, so replicas with different TZ settings disagreed on when a monthly window ends. Provider, deployment, tag and per-model budgets now all measure a window from its start in UTC, and a test covers the router budgets. A reset past year 9999 is documented as null. * test(budgets): cover monthly tag budgets in the router window test * test(budgets): add integration cells for budget_reset_at and monthly budget windows * test(budgets): find a monthly window in another span on every calendar day --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| adaptive_router | ||
| complexity_router | ||
| __init__.py | ||
| test_auto_router.py | ||
| test_base_routing_strategy.py | ||
| test_budget_limiter.py | ||
| test_budget_limiter_hotpath.py | ||
| test_complexity_router.py | ||
| test_complexity_tier_predictor.py | ||
| test_fuse_presets.py | ||
| test_lar1_routing.py | ||
| test_least_busy.py | ||
| test_litellm_encoder.py | ||
| test_llm_v2.py | ||
| test_lowest_cost.py | ||
| test_lowest_latency.py | ||
| test_lowest_tpm_rpm.py | ||
| test_lowest_tpm_rpm_v2.py | ||
| test_quality_router.py | ||
| test_router_routing_groups.py | ||
| test_router_routing_plugins.py | ||
| test_router_tag_regex_routing.py | ||
| test_router_tag_routing.py | ||
| test_savings_baseline.py | ||
| test_simple_shuffle.py | ||
| test_stall_detector.py | ||