mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-04 07:27:51 +00:00
e4b92bdd1a
Rebase onto latest main and address review feedback from @lets-order-some-fries: - DualCache resolves kwargs per tier: the in-memory tier gets default_in_memory_ttl, the redis tier gets default_redis_ttl and falls back to the in-memory default when no Redis-specific TTL is set, so the two tiers expire together unless an operator asks for otherwise. An explicit ttl= still reaches both tiers unchanged. The previous version mutated one shared kwargs dict, so the in-memory TTL leaked into the redis tier and the redis tier could not be configured independently. - Those per-tier dict copies carry `# mutable-ok:` reasons so the type-discipline budget in scripts/type-discipline-budget.json stays within its ceiling. - UserApiKeyCache computes effective_redis_ttl = default_redis_ttl if set else default_in_memory_ttl in __init__ and passes that to super().__init__ and key_object_cache, so the auth-cache contract (Redis mirrors in-memory TTL when no explicit Redis TTL is configured) is preserved without special-casing DualCache. - _attach_redis_usage_cache no longer passes the global litellm.default_redis_ttl to user_api_key_cache.attach_redis_cache; the auth cache already mirrors its own in-memory TTL, and overriding it here made the effective TTL depend on YAML key order. - _init_cache reads default_redis_ttl without popping the caller's dict and filters a copy out before constructing Cache, so redis-py Redis() does not receive an unexpected keyword argument and DB reloads do not force a cache rebuild on every poll. - Add regression tests covering per-tier TTL writes and the auth-cache attach path. Refs #43187 Signed-off-by: apex-mochen <2756823972@qq.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.