mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
c9a5777739
* test(azure): audit #47348 Add proxy integration tests for #47348's cached Entra token providers and off-loop token refresh, and fix the token herd the audit found: once the fetch moved to a worker thread, concurrent cold requests each fetched their own token. Every cached bearer token provider now serializes its calls so the first fetch fills the cache and the rest reuse it * fix(azure): build each cached Entra token provider once under concurrent first calls lru_cache does not hold back concurrent misses, so worker threads that miss together each built their own credential and token provider, and each fetched its own token. A lock around each provider cache makes the first caller build it and the rest reuse it * fix(azure): share Entra token fetches only while no valid token is held Concurrent callers now share one result (or one error) from the cold fetch, while callers inside the refresh window keep the still-valid token. The oidc/azure provider build and the client-assertion exchange are single-flighted per audience, and the shared helpers no longer call subscripted generic aliases, which crashed on Python 3.10. * fix(azure): refresh Entra tokens once while other requests keep the valid token MSAL treats a token with less than five minutes left as expired, so every request in that window fetched a new one. Now one request refreshes while the others keep using the still-valid token. Requests that waited on a fetch that failed retry once through a fresh fetch instead of all failing with the first error. Credentials that only implement get_token keep working, and OIDC token exchanges are cached per scope. * fix(azure): bound Entra token waits during an outage Requests waiting on a failed token fetch now retry only when the fetch before it succeeded, so during an Entra outage later bursts fail after one fetch instead of two. Requests during a refresh only get the held token while it has more than 60 seconds left, and a failed refresh falls back to that token instead of failing its request. * fix(azure): share one retry per failed Entra token fetch and raise on a failed refresh Waiters of a failed fetch now retry the flight they joined, so a fast-failing Entra makes one retry instead of one per waiter. A failed refresh raises again instead of returning the held token, which Postgres token auth read as a fresh one. The DefaultAzureCredential fallback reuses one provider per scope. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.