mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
89086db282
The reset job evicts the cached end-user object only from its own worker's in-memory cache (plus Redis), so every other uvicorn worker and replica keeps the pre-reset spend for up to user_api_key_cache_ttl (60s by default). Those workers pass that stale spend as fallback_spend, and since the authoritative floor read returned None for spend:end_user: keys, get_current_spend handed the stale value straight back and the end user kept getting 429 after the rollover on every worker but the one that ran the reset. The floor read now consults LiteLLM_EndUserTable.spend for end-user counters, the same way keys, teams, users, and orgs already read their rows. It runs only when the shared counter sits below the cached spend (a reset or a Redis restart) and stays behind the existing 5s in-process marker, so the normal request path still does no DB read. Cold end-user counters keep seeding from the cached object rather than the row, so from_db is unchanged for them. |
||
|---|---|---|
| .. | ||
| .coverage_baseline | ||
| __init__.py | ||
| _coverage_check.py | ||
| _pin_check.py | ||
| conftest.py | ||
| test_background_health.py | ||
| test_exception_handlers.py | ||
| test_harness_smoke.py | ||
| test_lifecycle.py | ||
| test_openapi_customization.py | ||
| test_proxy_config.py | ||
| test_routes_anthropic_beta.py | ||
| test_routes_assistants.py | ||
| test_routes_audio.py | ||
| test_routes_chat_completions.py | ||
| test_routes_completions.py | ||
| test_routes_config.py | ||
| test_routes_embeddings.py | ||
| test_routes_invitation.py | ||
| test_routes_login_sso.py | ||
| test_routes_misc.py | ||
| test_routes_model_cost_map.py | ||
| test_routes_model_info.py | ||
| test_routes_model_metrics.py | ||
| test_routes_models.py | ||
| test_routes_moderations.py | ||
| test_routes_onboarding.py | ||
| test_routes_queue.py | ||
| test_routes_threads.py | ||
| test_routes_utils.py | ||
| test_spend_counters.py | ||
| test_streaming_helpers.py | ||
| test_team_model_name_translation.py | ||