mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-17 23:51:30 +00:00
59d42d36e6
* fix(headroom): bound the /v1/compress and /v1/retrieve calls with a timeout The headroom guardrail builds its client with get_async_httpx_client(GuardrailCallback) and no params, and passes no timeout on either outbound call. That client's read, write and pool legs are 600s (litellm.request_timeout when set explicitly, default 6000s), so an unreachable or stalled compression service holds the caller's pre-call request open for the whole window before unreachable_fallback ever runs. Because the client is shared with every other no-params guardrail, each stalled call also pins a pooled connection for the same window, so a saturated pool makes unrelated requests block on the pool leg. Bound both calls at 60s by default, honoring litellm_params.timeout when set (the field already exists and documents itself as the per-guardrail API timeout; headroom accepted it and ignored it). The connect leg stays at the http_handler default, or the configured budget when that is shorter, so a dead host still fails fast. Live on a proxy against a stalled /v1/compress: 600.4s -> 60.2s before the 502, and 5.2s with timeout: 5 configured. * fix(headroom): reject non-finite timeouts and trim the timeout commentary `timeout: .inf` on a Headroom guardrail reached httpx and the aiohttp transport raised OverflowError, so every request came back as a raw 500 instead of going through unreachable_fallback. Reject non-finite values the same way as non-positive ones, and cut the comments and docstrings back to what the code does not already say. |
||
|---|---|---|
| .. | ||
| guardrail_hooks | ||
| test_content_filter_path_traversal.py | ||
| test_content_utils.py | ||
| test_custom_code_security.py | ||
| test_deferred_guardrail_logging.py | ||
| test_guardrail_coverage.py | ||
| test_guardrail_endpoints.py | ||
| test_guardrail_registry.py | ||
| test_init_guardrails.py | ||
| test_llm_as_a_judge.py | ||
| test_mcp_jwt_signer.py | ||
| test_pillar_guardrails.py | ||
| test_prompt_security_guardrails.py | ||
| test_qostodian_nexus_guardrail.py | ||
| test_usage_endpoints.py | ||
| test_usage_tracking.py | ||