scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-30 06:43:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 249
litellm/tests/unit/caching
History
Kerry Lu 0f786fa1c7
feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits (#48875) 
* feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): route rate limit cache increments through typed helpers

* chore(ui): regenerate schema.d.ts for deployment rpd and tpd fields

* test(types): add rpd and tpd to the owned litellm_params inventory

* fix(router): run daily rpd/tpd checks after the minute checks and increment rpm and rpd in one cache trip

* fix(router): flatten the async rpm reject branch for the ruff strict gate

* fix(router): hand rpm and rpd back on any rate limit reject

A request the rpm, rpd or tpd check rejects no longer keeps its request counts, so retries against a deployment whose rpd is spent cannot fill its rpm and block the in-group fallover with the minute 429.

* test(router): a minute-rejected request no longer keeps its rpm count

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-28 15:52:44 -07:00
..
__init__.py
test_azure_blob_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cache_preset_key.py
test_caching.py feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
test_caching_handler.py test(caching): assert sync cache-hit logging without depending on thread order (#47093) 2026-10-17 14:00:03 -07:00
test_check_and_fix_namespace_none_guard.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_disk_cache.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_dual_cache.py fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488) 2026-10-14 15:27:00 -07:00
test_embedding_router.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_evicted_client_closer.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_gcs_cache.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_in_memory_cache.py feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits (#48875) 2026-10-28 15:52:44 -07:00
test_llm_caching_handler.py fix(aiohttp_transport): close an owned aiohttp session when its transport is collected (#48650) 2026-10-27 20:33:03 +00:00
test_llm_client_cache_e2e.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_qdrant_semantic_cache.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_redis_batch.py fix(otel): nest cache spans under their operation and name service spans by purpose (#44150) 2026-10-03 09:20:27 -07:00
test_redis_cache.py test(caching): baseline breaker metrics from live state after gc (#46652) 2026-10-15 05:53:30 -07:00
test_redis_cluster_cache.py fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488) 2026-10-14 15:27:00 -07:00
test_redis_cluster_node_isolation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_redis_connection_pool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_redis_semantic_cache.py fix(caching): skip the cache past max_messages and keep tool_result text in semantic prompts (#43878) 2026-10-07 17:32:39 +00:00
test_request_redis_batch_post_call.py fix(proxy)!: stop update_cache from resurrecting evicted auth cache entries (#46997) 2026-10-20 17:09:53 +00:00
test_request_redis_batch_pre_call.py fix: cap cold MCP schema workers at usable CPUs and deflake wall-clock unit tests (#45791) 2026-10-15 00:47:59 -07:00
test_responses_stream_cache_keys.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_s3_cache.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_unit_test_caching.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_valkey_semantic_cache.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00