The previous fix only skipped cooldown counter increments. The health state
cache was still marking 429/408 endpoints as is_healthy=False, causing the
binary health check filter to exclude them from routing.
Now, when health_check_ignore_transient_errors=True, 429/408 endpoints are
also excluded from the unhealthy list passed to build_deployment_health_states(),
so the binary filter treats them as unaffected (not unhealthy).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When enabled, health check failures with 429 (rate limit) or 408 (timeout)
status codes are skipped from the cooldown pipeline. These are transient
load issues, not broken deployments. Auth errors (401), 404, and 5xx errors
still increment counters and trigger cooldown as before.
Config (general_settings):
health_check_ignore_transient_errors: true
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Health check failures now increment the same per-deployment failure
counters used by allowed_fails_policy, so users can control how many
health check failures of each error type are required before a
deployment enters cooldown.
- ahealth_check() preserves the original exception in its return dict
- run_with_timeout() returns a litellm.Timeout on health check timeout
- _perform_health_check() propagates exceptions to unhealthy endpoints
- _write_health_state_to_router_cache() calls _set_cooldown_deployments
for each unhealthy endpoint that has an exception
- When allowed_fails_policy is set, the binary health check filter is
bypassed so cooldown is the sole routing exclusion mechanism
- Safety net: if all deployments are in cooldown with
enable_health_check_routing=True, the cooldown filter is bypassed
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Background health checks now feed deployment health state into the
router candidate-filtering pipeline. Unhealthy deployments are excluded
proactively instead of waiting for request failures to trigger cooldown.
Gated by `enable_health_check_routing: true` in general_settings.
Off by default — zero behavior change for existing users.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When fallbacks use non-standard formats (e.g. ["claude-3-haiku"] or
[{"model": "...", "messages": [...]}]), detect them with
_check_non_standard_fallback_format and pass them through directly
instead of trying to parse with get_fallback_model_group which only
handles the standard dict-keyed format.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When order=1 deployments fail, the router now automatically tries order=2,
then order=3, etc. before falling through to external fallbacks. This removes
the need for enable_pre_call_checks and makes order work as a true priority-based
fallback chain within a model group.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- GCPIAMCredentialProvider now inherits from redis.credentials.CredentialProvider
so redis-py's async path calls get_credentials_async() properly
- move _redis_credential_provider import to top of _redis.py (PEP 8)
- remove dead else-branch that silently no-oped (gcp_service_account from
redis_kwargs.get() was always None since it's popped by _get_redis_client_logic)
- remove mid-function 'from litellm import get_secret_str' inline import
- remove unused 'call' import from test_redis.py
Extract GCPIAMCredentialProvider and _generate_gcp_iam_access_token
into litellm/_redis_credential_provider.py. _redis.py imports them
from there, keeping the public API unchanged.
Async RedisCluster was generating the IAM token once at startup and
storing it as a static password. After the 1-hour GCP token TTL, any
new connection (including to newly-discovered cluster nodes) would fail
to authenticate.
Fix: introduce GCPIAMCredentialProvider that implements redis-py's
CredentialProvider protocol. It calls _generate_gcp_iam_access_token()
on every new connection, matching what the sync redis_connect_func
already does. async_redis.RedisCluster accepts a credential_provider
kwarg which is invoked per-connection.
Addresses Greptile feedback that test assertions were weakened when
removing summary: "detailed" expectations — now every default-behavior
test explicitly asserts that "summary" is absent from the result.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(redis): add circuit breaker to RedisCache to fast-fail when Redis is down (#24181)
* feat(redis): add circuit breaker env var constants
* feat(redis): add RedisCircuitBreaker and apply guard decorator to all async ops
* fix(dual_cache): fall back to L1 instead of re-raising on Redis increment failures
* test(caching): add circuit breaker unit tests
* fix(redis): fast-fail concurrent HALF_OPEN probes — only one probe at a time
* fix(dual_cache): return None fallback when in_memory_cache is absent and Redis fails
* test(caching): add regression tests for HALF_OPEN concurrency and None fallback
* Fix blocking sync next in __anext__ (#24177)
* Fix blocking sync next
* Update tests/test_litellm/litellm_core_utils/test_streaming_handler.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix PEP 479 regression in __anext__ sync iterator exhaustion
asyncio.to_thread re-raises thread exceptions inside a coroutine, where
PEP 479 converts StopIteration to RuntimeError before any except clause
can catch it. Add _next_sync_or_exhausted() module-level helper that
catches StopIteration in the thread and returns a sentinel instead, then
raise StopAsyncIteration in the coroutine.
Also rewrites the non-blocking test to use asyncio.gather() instead of
asyncio.create_task() (which returned None on Python 3.9 / pytest-asyncio
in CI), and adds an exhaustion regression test that drains the wrapper
fully and asserts no RuntimeError leaks out.
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: add git-subdir source type to claude-code/plugins API (#24223)
Support a third plugin source type `git-subdir` alongside the existing
`github` and `url` types, as documented in the official Claude Code
plugin marketplaces spec.
New format: {"source": "git-subdir", "url": "...", "path": "subdir/path"}
- Validates url and path fields are present and non-empty
- Rejects absolute paths, '..' segments, backslashes, and percent-encoded
traversal sequences (including double-encoded variants via regex check)
- Extracts path validation into _validate_git_subdir_path() helper
- Updates Pydantic field description to document all three source types
- Adds isValidUrl() check for url/git-subdir source types in the UI form
- Adds "Git Subdir" option to the UI form with a required Path field
- Adds unit tests covering success, update, missing/empty fields,
path traversal variants, and unknown source type
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* [FEAT] add extract_header and extract_footer to Mistral OCR supported params (#24213)
* docs: add git-subdir source type to claude-code plugin marketplace docs (#24289)
* fix(ui): swap J/K keyboard navigation in log details drawer (#24279) (#24286)
J should navigate down (next) and K should navigate up (previous),
matching vim/standard conventions.
* fix: use async_set_cache in user_api_key_auth hot path (#24302)
* fix: use async_set_cache in auth hot path to avoid blocking event loop
* test: assert no blocking set_cache call in _user_api_key_auth_builder
* test: broaden blocking call check to all sync DualCache methods
* test: fix regression test to actually catch blocking cache calls
* fix: ruff lint unused variable + UI build MessageManager error
- litellm/caching/redis_cache.py: remove unused variable 'e' in circuit
breaker exception handler (F841)
- add_plugin_form.tsx: use MessageManager.error() instead of undefined
message.error() for git URL validation
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs: add REDIS_CIRCUIT_BREAKER env vars to config_settings reference
Add REDIS_CIRCUIT_BREAKER_FAILURE_THRESHOLD and
REDIS_CIRCUIT_BREAKER_RECOVERY_TIMEOUT to the environment variables
reference table so test_env_keys.py passes.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Vincenzo Barrea <manamana88@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Robert Kirscht <rkirscht242@gmail.com>
Co-authored-by: Imgyu Kim <kimimgo@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Add key-name-based regex patterns (master_key, database_url, auth_token,
etc.) to SecretRedactionFilter so secrets embedded in dict/config dumps
are redacted by key name, regardless of value format.
Fixes a leak where general_settings containing master_key and
database_url was logged in full because the secret values didn't match
any existing value-format regex pattern.
Address Greptile review feedback:
1. Replace opt-out `disable_default_reasoning_summary` with existing opt-in
`reasoning_auto_summary` flag — avoids backwards-incompatible change where
all users routing thinking-enabled requests would silently get a changed
reasoning_effort shape (string -> dict) on upgrade.
2. Add default summary injection to `_translate_thinking_to_openai` — this path
was the only one missing it, causing inconsistent behavior for
litellm.completion() callers using the Anthropic adapter.
3. Narrow `except Exception` to `except (ValueError, TypeError, AttributeError)`
in tests to avoid masking genuine failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove incorrect supports_prompt_caching from gpt-4-0314 (predates the feature)
- Make data-URL detection case-insensitive in Gemini tool call result conversion
- Mock show_banner/generate_feedback_box in max_budget tests to prevent real I/O
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(moonshot): preserve reasoning_content on Pydantic Message objects in multi-turn tool calls
The condition 'reasoning_content not in msg' doesn't work correctly for
Pydantic Message objects because they don't support the 'in' operator
like dicts do. This caused reasoning_content to be stripped from
assistant messages in multi-turn conversation history.
Changed the condition to use msg.get('reasoning_content') instead,
which works correctly for both dicts and Pydantic models.
Fixes#23765
* added newline eof
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/llms/moonshot/test_moonshot_chat_transformation.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Simplify assertions in test_moonshot_chat_transformation
Removed redundant assertions for non-assistant messages.
---------
Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com>
Co-authored-by: Aarish Alam <arishalam121@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
- Set output_cost_per_second to 0.0 (was 0.0001) for whisper-1 and
azure/whisper-1: transcription is billed on input duration only,
not output duration
- Fix cost_per_second() in openai/cost_calculation.py: change elif to if
so input_cost_per_second is evaluated independently of output_cost_per_second,
and remove the erroneous completion_cost = 0.0 assignment that masked
any previously-set output cost
- Add TestCostPerSecondArithmetic unit tests covering both cost fields,
the None-guard, and zero-duration edge case
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The Gemini REST API documents the embedding task type parameter as
camelCase `taskType`. The existing transformation functions convert
`dimensions` to `outputDimensionality` but miss the parallel
`task_type` to `taskType` conversion. This adds that conversion to
both `transform_openai_input_gemini_content` (batchEmbedContents path)
and `transform_openai_input_gemini_embed_content` (embedContent path).
Fixes#24190
Non-streaming paths call _process_hidden_params_and_response_cost; streaming
assembles the full response later and skipped that, so litellm_params.metadata
lacked hidden_params (e.g. response_cost for OTEL/OpenSearch).
- Add _merge_hidden_params_from_response_into_metadata and call it from
success_handler and async_success_handler after cost is set, before
_build_standard_logging_payload.
- Unit tests for merge helper.
Tests: pytest tests/test_litellm/litellm_core_utils/test_litellm_logging.py
Made-with: Cursor
