mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-13 03:59:12 +00:00
52403d7a8d
A JWKS fetch had no retry, so a single connect timeout to the identity provider failed authentication outright, and once the cached copy expired there was nothing to fall back on. How that surfaced depended on the outage shape: httpx.ConnectTimeout was missing from DB_CONNECTION_ERROR_TYPES so it fell through to the generic auth handler as a 401 with an empty detail, while a read timeout took the database path and reported a healthy database as unreachable. Transport failures are now retried three times with a short backoff, and the last-known-good JWKS stays usable for a bounded window past public_key_ttl. That window is public_key_stale_ttl, a new config field defaulting to 3600s and settable to 0 to fail closed. It is checked on every read against the current setting rather than baked into the cache entry when it is written, so lowering it binds immediately instead of waiting for entries written under the old value to age out, which matters because a shared cache survives the restart an operator performs to make the change take effect. A copy whose write time cannot be established is not servable. Only httpx.TransportError unlocks the stale copy, so an identity provider that answers at all, including with a narrowed key set, revokes on the next refresh. Every stale serve logs the kid it authenticated, how long ago that copy was refreshed, and how long until it stops being trusted. A sustained outage is remembered for 30s per key url, so it costs one fetch per window instead of three timeouts per request serialised behind the refresh lock. Non-200 JWKS responses now raise instead of being cached as the key set, which previously let an error body overwrite the last-known-good copy. An unreachable identity provider with no cached copy left returns 503 auth_provider_unavailable. Resolves LIT-5524 Co-authored-by: Yassin Kortam <yassin@berri.ai> |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm/proxy/_experimental/mcp_server | ||
| litellm-proxy-extras | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.