scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

fix: use async_set_cache (not async_increment_cache) to seed spend counter on cold-start in auth check

Browse source
This commit is contained in:
Ishaan Jaffer 2026-04-24 20:12:20 -07:00
parent 09362b525d
commit d9067925f5
No known key found for this signature in database
1 changed files with 8 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
litellm/proxy/auth/auth_checks.py
View file

@ -3400,14 +3400,15 @@ async def _check_team_member_model_budget(
)
if model_spend < 0:
# Cache cold (pod restart / Redis flush) — fetch authoritative value from DB.
# Seed spend_counter_cache immediately so concurrent auth calls for the same
# (user_id, team_id, model) slot use the cached value instead of each
# issuing another DB query (O(1) vs O(concurrent-requests) DB hits).
# Use async_set_cache (idempotent overwrite) not async_increment_cache:
# two concurrent requests that both see a miss would each call INCRBYFLOAT,
# doubling the seed value and producing false budget-exceeded 429s.
# Seed unconditionally (including zero spend) so new users don't hit DB
# on every auth request until their first request's cost callback fires.
model_spend = await _reseed_spend_from_db(counter_key)
if model_spend > 0:
await spend_counter_cache.async_increment_cache(
key=counter_key, value=model_spend
)
await spend_counter_cache.async_set_cache(
key=counter_key, value=model_spend
)
if model_spend >= max_budget:
raise litellm.BudgetExceededError(