mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
fix(rate-limiting): stop leaving a permanent Redis key on an expired-reservation release
Releasing a reservation whose key had already expired made INCRBY recreate it, and the floor-to-zero SET left that recreated key permanently in Redis (SET clears any TTL). Verified against a real Redis instance: DEL removes the key outright instead, which reads back identically to 0 everywhere this key is read.
This commit is contained in:
parent
f6f65e181b
commit
79e66d9754
1 changed files with 6 additions and 2 deletions
|
|
@ -105,13 +105,17 @@ return { 1, new_value }
|
|||
# release a concurrency reservation -- floors at 0 so a decrement that can't
|
||||
# be attributed to the exact reservation that caused it (see
|
||||
# `_release_keys`'s docstring) degrades to under-counting rather than a
|
||||
# negative counter that would admit unlimited requests.
|
||||
# negative counter that would admit unlimited requests. Floors via DEL, not
|
||||
# `SET key 0`: releasing a reservation whose key already expired makes
|
||||
# INCRBY recreate it with no TTL, and a plain SET would leave that recreated
|
||||
# key permanently in Redis (SET clears any TTL); DEL removes it outright,
|
||||
# which reads back identically to 0 everywhere this key is read (`GET key or 0`).
|
||||
TAG_RL_DECR_FLOOR_ZERO_SCRIPT: Final = """
|
||||
local key = KEYS[1]
|
||||
local delta = tonumber(ARGV[1])
|
||||
local new_value = redis.call('INCRBY', key, delta)
|
||||
if new_value < 0 then
|
||||
redis.call('SET', key, 0)
|
||||
redis.call('DEL', key)
|
||||
new_value = 0
|
||||
end
|
||||
return new_value
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue