Greptile follow-ups on the prior commit:
- (P1) ``is_request_body_safe`` recursed into ``litellm_embedding_config``
with no depth bound, so a request body 1000 levels deep could exhaust
Python's call stack and surface a 500 ``RecursionError``. Refactored
the check to be iterative (single-level descent into a fixed list of
nested-config keys) and extracted the per-dict banned-param scan into
a helper that's shared between the root and the nested call sites.
Also fixes the ``recursive_detector`` CI job that was triggered by
the recursive-by-name pattern.
- (P2) ``assert_same_origin`` error messages identified the mismatching
component but echoed the ``expected`` host and the candidate
hostname back to the caller. In the SSRF threat model the caller is
the attacker, so reflecting that information was a secondary leak of
operator infrastructure. Messages now identify only *which*
component mismatched (scheme / host / port) without naming names.
- (P2) ``_NESTED_CONFIG_KEYS`` was defined after the function that used
it. Hoisted the constant (and the new ``_BANNED_REQUEST_BODY_PARAMS``
tuple) above the function for readability.
Adds a 1000-level-deep nested config test that asserts no
``RecursionError`` and a hostname-leak test that asserts no operator
host appears in the rejection message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The async CustomStreamWrapper.__anext__ caught httpx.TimeoutException
and re-raised it raw, bypassing _handle_stream_fallback_error. The
Router's FallbackStreamWrapper only reacts to MidStreamFallbackError,
so stream_timeout firing mid-stream never triggered fallbacks the way
connection-phase timeout does.
Route timeouts through _handle_stream_fallback_error like every other
exception so they wrap into MidStreamFallbackError and the Router can
switch to a fallback model.
Two SSRF findings were OPEN with no in-flight fix; both are closed
now using narrow defenses that key off existing trust boundaries.
VERIA-6 (Milvus ``litellm_embedding_config``):
``is_request_body_safe`` already blocks ``api_base`` / ``api_key`` /
``langfuse_host`` / ``s3_endpoint_url`` / etc. at the *root* of the
request body, gated by an admin opt-in (``allow_client_side_credentials``
or per-deployment ``configurable_clientside_auth_params``). The bug is
that the Milvus vector-store transformer unpacks
``litellm_embedding_config`` into ``litellm.embedding(**embedding_config)``,
so a caller can smuggle the same banned params in via nesting and bypass
the check. Fix: ``is_request_body_safe`` now recurses into a known list
of nested-config dicts (``litellm_embedding_config`` for now) and applies
the same banned-param check with the same admin opt-in. Admin-side
vector-store config flows through ``litellm_params`` rather than the
request body, so it's unaffected.
VERIA-51 (polling URLs returned by upstream APIs):
Azure DALL-E 2, Azure Document Intelligence, and Black Forest Labs
all blindly fetched a polling URL returned by the upstream and
attached the operator's API key to the request. A compromised upstream
or a future API contract change could redirect credentials anywhere.
New ``url_utils.assert_same_origin(candidate, expected)`` helper checks
scheme, host (case-insensitive), and port (with default-port
normalization). Applied at all five polling sites: Azure DALL-E
sync+async, Azure DI sync+async, BFL image generation sync+async, BFL
image edit sync+async. Cross-origin polling URLs now raise rather than
forward credentials. The Azure DALL-E ``Expected 'status' in response``
exception no longer reflects the raw response body — that path turned
Blind SSRF into Full-Read SSRF for the limited window before the
origin check fully closed it.
Tests: 7 ``assert_same_origin`` unit tests, 6 ``is_request_body_safe``
nested-config tests, 5 polling-site rejection tests + 1 same-origin
sanity check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile: LITELLM_DEFAULT_EMBEDDING_ENCODING_FORMAT=none (case-insensitive)
strips encoding_format so OpenAI-compatible backends can use provider defaults.
Preserves optional_params passthrough when env is unset.
Co-authored-by: Cursor <cursoragent@cursor.com>
Stop forcing Gemini 3 thinkingLevel for Anthropic-style thinking params by default, and gate legacy low/minimal mapping behind an explicit feature flag to avoid provider-default confusion.
Made-with: Cursor
- Route publisher/model ids (e.g. xai/grok) to .../endpoints/openapi; keep model in JSON body
- Add model_prices keys for vertex_ai/openai/xai/grok-*
- Document xAI Grok on vertex_partner (aligned with GPT-OSS)
- Add tests for create_vertex_url and body-model heuristic
Made-with: Cursor
Trailing slashes on custom API base examples cause double-slash in
get_complete_url. Also fixes inconsistent list indentation in
test_crusoe_models_configuration.
Crusoe's vLLM-based endpoint accepts max_tokens, not max_completion_tokens.
Without this mapping, callers using the OpenAI-standard param would get errors.
- Remove trailing slash from docs Base URL to match providers.json
- Wrap model_cost mutations in try/finally to prevent test state leakage
- Add missing __init__.py to crusoe test package
Replace hand-written CrusoeChatConfig class and manual registrations
across constants.py, __init__.py, get_llm_provider_logic.py, and
_lazy_imports_registry.py with a single entry in
litellm/llms/openai_like/providers.json, consistent with the
recommended pattern for OpenAI-compatible providers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When aembedding=True, api_version was not passed to self.aembedding(), causing
get_azure_openai_client() to receive None instead of "v1". This made
_is_azure_v1_api_version() return False, so AsyncAzureOpenAI was selected
instead of AsyncOpenAI, constructing the wrong request URL and returning 404.
Fixes#24848
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(gemini): handle Gemini Files API URIs without fetching
Fixes#24907
When a file is uploaded via the Gemini Files API, the returned URI
(https://generativelanguage.googleapis.com/v1beta/files/...) starts
with 'https://' and hits the generic HTTPS handler in
_process_gemini_media(). That handler calls
_get_image_mime_type_from_url() which tries to fetch the URL — but
Gemini Files API URLs return 403 when accessed directly, causing:
'Unable to determine mime type for file_id: ...'
Fix: add an early elif that matches Gemini Files API URLs and passes
them through as file_data without trying to fetch the URL. When an
explicit format is provided it's included; otherwise the Gemini API
infers the MIME type from its stored metadata.
Exactly matches the fix direction suggested by the issue reporter
(rodriciru).
* fix: anchor Gemini Files API URL check with startswith
Address greptile P2: replace `in` substring check with `startswith`
to prevent query-string injection bypass (e.g.
`https://evil.com/?ref=https://generativelanguage...`).
Also adds trailing slash to match only valid file URIs.
---------
Co-authored-by: voidborne-d <voidborne-d@users.noreply.github.com>
service_tier (priority/flex) was not forwarded to generic_cost_per_token
for azure and azure_ai providers, so tier-specific pricing was ignored
and standard pricing was always returned. Other providers (openai,
bedrock, gemini, vertex_ai) already pass it correctly.
* fix(proxy): use actual request start_time for failed spend logs
async_post_call_failure_hook was calling datetime.now() for both
start_time and end_time, making every failed request show Duration: 0.000s.
litellm_logging_obj (already fetched in the same method for trace ID
propagation) carries the real request start_time — use it as
actual_start_time with a datetime.now() fallback when absent.
Add two regression tests covering the fix and the fallback path.
Fixes#24888
* fix(llm translation): redact Gemini API key from URL query params in error traces
Gemini API requests authenticate via a ?key=<api_key> URL query param.
When a provider call fails, httpx.Response.raise_for_status() embeds the full
URL in the error message, leaking the key in exception traces and logs.
Changes:
- Extract secret-redaction logic from litellm/_logging.py into a new public
utility module litellm/litellm_core_utils/secret_redaction.py, exposing
redact_string() as a proper public API instead of a private helper
- Add (?<=[?&])key=[^\s&'"]{8,} pattern to _SECRET_RE so ?key=VALUE and
&key=VALUE fragments are caught by the existing SecretRedactionFilter
- Apply redact_string() to error_str in exception_mapping_utils.py so the
key is also stripped from the mapped exception message surfaced to callers
- Add 5 regression tests covering: ?key=, &key=, short-value no-op, httpx
raise_for_status path, and end-to-end logger output
- Keep _redact_string = redact_string alias in _logging.py for backward compat
Fixes#24902
* revert: undo start_time fix for failed spend logs
* fix: gate exception redaction on _ENABLE_SECRET_REDACTION opt-out flag
- Apply redact_string() conditionally in exception_mapping_utils.py,
matching the same _ENABLE_SECRET_REDACTION guard used by SecretRedactionFilter
so that LITELLM_DISABLE_REDACT_SECRETS=true is honoured for exception messages
- Rewrite test_redact_string_applied_to_httpx_error_message to use pytest.raises
so assertions cannot be silently skipped if raise_for_status() doesn't raise
- Add test_exception_mapping_respects_redaction_opt_out to verify the flag is
respected end-to-end through exception_type()
Success handlers already run when CustomStreamWrapper or
CachedResponsesAPIStreamingIterator finishes replay. Logging at
cache-hit time for acompletion/completion streaming duplicated spend
and callbacks. Align tests with deferred behavior.
Made-with: Cursor
