scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 85

fix(cache): InMemoryCache.async_get_ttl returns remaining seconds, not absolute expiry (gh-37261)

Browse source 
Fixes budget_reset_at showing ~57 years in the future when no Redis is configured. InMemoryCache stored absolute expiry timestamps in ttl_dict, but budget_limiter.py adds them directly to datetime.now(). Now async_get_ttl returns remaining seconds like RedisCache does.
This commit is contained in:
Siraj637909 2026-08-18 21:31:32 +05:30
parent bd0d13566e
commit 0b2c69be64
1 changed files with 5 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
litellm/caching/in_memory_cache.py
View file

@ -264,10 +264,13 @@ class InMemoryCache(BaseCache):
self._remove_key(key)
async def async_get_ttl(self, key: str) -> int | None:
"""
"""Get the remaining TTL of a key in in-memory cache"""
Get the remaining TTL of a key in in-memory cache
"""
return self.ttl_dict.get(key, None)
expiry = self.ttl_dict.get(key, None)
if expiry is None:
return None
return max(0, expiry - time.time())
async def async_get_oldest_n_keys(self, n: int) -> list[str]:
"""