mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
7ba47a5b6e
The budget-tier reset read every customer linked to an expiring tier into one result set before the write, then invalidated their caches one key at a time. Both of those scale with the customer count, so a large enough deployment can OOM the proxy pod on the read, and the tail of the population sits on a stale spend counter while the per-key invalidations drain PR #40639 moved the reset write itself to a link-based UPDATE, so that pre-commit read no longer feeds the write. It only fed cache invalidation and the service-logging counts, which means it can move after the commit. This replaces it with a keyset walk over litellm_endusertable ordered by user_id, taking RESET_BUDGET_JOB_BATCH_SIZE rows per page, the same shape _reset_windows_for_source already uses, with no per-run page cap for the same reason that walk has none: the cursor cannot survive the run, so a cap would restart at the first customer on every tick and never reach the tail Each page's counter and cache keys now go out as one batched delete through a new DualCache.async_delete_cache_keys, which drops the in-memory entries and chunks the Redis DELETE at DEFAULT_MAX_REDIS_BATCH_CACHE_SIZE num_endusers_found and num_endusers_updated now report the customers whose caches were invalidated after the commit rather than the rows read before it, so both read 0 when the cascade write fails |
||
|---|---|---|
| .. | ||
| html_forms | ||
| test_auth_cache_invalidation_pubsub.py | ||
| test_cache_codec.py | ||
| test_callback_config_validation.py | ||
| test_callback_utils.py | ||
| test_config_sync_pubsub.py | ||
| test_custom_openapi_spec.py | ||
| test_debug_utils.py | ||
| test_encrypt_decrypt_utils.py | ||
| test_expired_ui_session_key_cleanup_manager.py | ||
| test_get_routes.py | ||
| test_http_parsing_utils.py | ||
| test_json_merge_patch.py | ||
| test_key_rotation_e2e.py | ||
| test_key_rotation_integration.py | ||
| test_key_rotation_lock.py | ||
| test_key_rotation_manager.py | ||
| test_load_config_utils.py | ||
| test_model_deprecation.py | ||
| test_model_listing_utils.py | ||
| test_openai_endpoint_utils.py | ||
| test_openai_error_payload.py | ||
| test_path_utils.py | ||
| test_periodic_reload_schedule.py | ||
| test_prompt_cache_pricing.py | ||
| test_rbac_utils.py | ||
| test_registry_read_through.py | ||
| test_reset_budget_job.py | ||
| test_scheduled_job_stagger.py | ||
| test_sse_keepalive.py | ||
| test_static_asset_utils.py | ||
| test_timezone_utils.py | ||
| test_upsert_budget_membership.py | ||
| test_user_api_key_cache.py | ||