mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
66a89f5a6e
Every pod schedules the budget reset job, so a fleet re-read the whole due population and wrote it back against one Postgres at the same calendar boundary, multiplying a single sweep by its replica count. The job now takes the shared PodLockManager lease, so one pod sweeps per tick. A deployment with no Redis keeps its previous behavior, and a Redis that cannot answer sweeps unguarded rather than stranding every expired budget at its cap. The per-window scan read every row carrying budget_limits in one statement, so its cost grew with the deployment's key count. It is now keyset-paginated and walks to the end of the table on every sweep. A per-run cap would need a resume position, and no pod can hold one because the lease rotates between ticks, so the strictly advancing cursor is what terminates the walk. Found and updated rows were also JSON-serialized into the service hook's metadata and into debug lines on every chunk, on the event loop, whether or not anything consumed them. The hooks now carry counts, and the debug payload is deferred until a record is actually emitted. Resolves LIT-4793 Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| html_forms | ||
| test_auth_cache_invalidation_pubsub.py | ||
| test_cache_codec.py | ||
| test_callback_utils.py | ||
| test_config_sync_pubsub.py | ||
| test_custom_openapi_spec.py | ||
| test_encrypt_decrypt_utils.py | ||
| test_expired_ui_session_key_cleanup_manager.py | ||
| test_get_routes.py | ||
| test_http_parsing_utils.py | ||
| test_json_merge_patch.py | ||
| test_key_rotation_e2e.py | ||
| test_key_rotation_integration.py | ||
| test_key_rotation_lock.py | ||
| test_key_rotation_manager.py | ||
| test_load_config_utils.py | ||
| test_model_deprecation.py | ||
| test_openai_endpoint_utils.py | ||
| test_path_utils.py | ||
| test_periodic_reload_schedule.py | ||
| test_rbac_utils.py | ||
| test_registry_read_through.py | ||
| test_reset_budget_job.py | ||
| test_scheduled_job_stagger.py | ||
| test_sse_keepalive.py | ||
| test_static_asset_utils.py | ||
| test_timezone_utils.py | ||
| test_upsert_budget_membership.py | ||
| test_user_api_key_cache.py | ||