When aembedding=True, api_version was not passed to self.aembedding(), causing
get_azure_openai_client() to receive None instead of "v1". This made
_is_azure_v1_api_version() return False, so AsyncAzureOpenAI was selected
instead of AsyncOpenAI, constructing the wrong request URL and returning 404.
Fixes#24848
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(gemini): handle Gemini Files API URIs without fetching
Fixes#24907
When a file is uploaded via the Gemini Files API, the returned URI
(https://generativelanguage.googleapis.com/v1beta/files/...) starts
with 'https://' and hits the generic HTTPS handler in
_process_gemini_media(). That handler calls
_get_image_mime_type_from_url() which tries to fetch the URL — but
Gemini Files API URLs return 403 when accessed directly, causing:
'Unable to determine mime type for file_id: ...'
Fix: add an early elif that matches Gemini Files API URLs and passes
them through as file_data without trying to fetch the URL. When an
explicit format is provided it's included; otherwise the Gemini API
infers the MIME type from its stored metadata.
Exactly matches the fix direction suggested by the issue reporter
(rodriciru).
* fix: anchor Gemini Files API URL check with startswith
Address greptile P2: replace `in` substring check with `startswith`
to prevent query-string injection bypass (e.g.
`https://evil.com/?ref=https://generativelanguage...`).
Also adds trailing slash to match only valid file URIs.
---------
Co-authored-by: voidborne-d <voidborne-d@users.noreply.github.com>
service_tier (priority/flex) was not forwarded to generic_cost_per_token
for azure and azure_ai providers, so tier-specific pricing was ignored
and standard pricing was always returned. Other providers (openai,
bedrock, gemini, vertex_ai) already pass it correctly.
* fix(proxy): use actual request start_time for failed spend logs
async_post_call_failure_hook was calling datetime.now() for both
start_time and end_time, making every failed request show Duration: 0.000s.
litellm_logging_obj (already fetched in the same method for trace ID
propagation) carries the real request start_time — use it as
actual_start_time with a datetime.now() fallback when absent.
Add two regression tests covering the fix and the fallback path.
Fixes#24888
* fix(llm translation): redact Gemini API key from URL query params in error traces
Gemini API requests authenticate via a ?key=<api_key> URL query param.
When a provider call fails, httpx.Response.raise_for_status() embeds the full
URL in the error message, leaking the key in exception traces and logs.
Changes:
- Extract secret-redaction logic from litellm/_logging.py into a new public
utility module litellm/litellm_core_utils/secret_redaction.py, exposing
redact_string() as a proper public API instead of a private helper
- Add (?<=[?&])key=[^\s&'"]{8,} pattern to _SECRET_RE so ?key=VALUE and
&key=VALUE fragments are caught by the existing SecretRedactionFilter
- Apply redact_string() to error_str in exception_mapping_utils.py so the
key is also stripped from the mapped exception message surfaced to callers
- Add 5 regression tests covering: ?key=, &key=, short-value no-op, httpx
raise_for_status path, and end-to-end logger output
- Keep _redact_string = redact_string alias in _logging.py for backward compat
Fixes#24902
* revert: undo start_time fix for failed spend logs
* fix: gate exception redaction on _ENABLE_SECRET_REDACTION opt-out flag
- Apply redact_string() conditionally in exception_mapping_utils.py,
matching the same _ENABLE_SECRET_REDACTION guard used by SecretRedactionFilter
so that LITELLM_DISABLE_REDACT_SECRETS=true is honoured for exception messages
- Rewrite test_redact_string_applied_to_httpx_error_message to use pytest.raises
so assertions cannot be silently skipped if raise_for_status() doesn't raise
- Add test_exception_mapping_respects_redaction_opt_out to verify the flag is
respected end-to-end through exception_type()
Success handlers already run when CustomStreamWrapper or
CachedResponsesAPIStreamingIterator finishes replay. Logging at
cache-hit time for acompletion/completion streaming duplicated spend
and callbacks. Align tests with deferred behavior.
Made-with: Cursor
_base_process_chunk only encodes response IDs when parsed_chunk contains
a top-level "response" key. Align test_process_chunk_completed_response_
updates_id_and_usage_cost with that contract and test_base_responses_api_streaming_iterator.
Made-with: Cursor
The test supplies a minimal PDF base64 payload but expected the wrong
constant (base64 for "test"). Assert against the same pdf_b64 value
and drop the unused import.
Made-with: Cursor
CodeQL flagged the previous ``from litellm.types.router import
MockRouterTestingParams`` at module top-level — ``litellm.types.router``
indirectly imports back into proxy modules, so the dataclass may not
exist yet when ``route_llm_request`` is being imported.
Hardcode the three flag names instead, with a guard test
(``test_mock_testing_kwarg_names_matches_dataclass``) that asserts the
hardcoded list matches ``MockRouterTestingParams.fields`` so drift is
caught at test time rather than missed in production.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes that together prevent a caller from smuggling unauthorized
models past the API key's allowlist via per-request router overrides.
1. ``_enforce_key_and_fallback_model_access``: also walk fallback models
nested inside ``router_settings_override.fallbacks`` /
``context_window_fallbacks`` / ``content_policy_fallbacks``.
``route_llm_request.py`` promotes those to per-request kwargs after
auth, so without this they bypassed the model allowlist entirely.
New ``iter_router_fallback_model_names`` helper extracts leaf names
from both the simple top-level shape (str | {"model": str}) and the
nested router-config shape ({primary: [fallbacks]}). The two fallback
validation loops are unified — every name (top-level + override) is
deduplicated and validated once via ``can_key_call_model`` +
``is_valid_fallback_model``.
2. ``route_request``: strip router-internal ``mock_testing_*`` flags
from user-supplied data. These are testing-only flags that
deterministically force the router into fallback logic by raising a
synthetic ``InternalServerError`` etc. Combined with override
fallbacks they made the smuggling path trivially exploitable. Test
code that calls the router directly bypasses the strip and is
unaffected. The strip list is derived from ``MockRouterTestingParams``
so a new ``mock_testing_*`` flag added to that dataclass is
automatically covered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-release detector in create-release.yml uses `\.dev` (literal dot
before `dev`), which matches PEP 440 canonical tags like `1.84.0.dev2`
but misses the SemVer/Docker form `1.84.0-dev.2` (hyphen-dev). Per the
release design doc's PyPI<->Docker mapping rule, both forms are valid
production-track release tags and both are pre-releases (opt-in via
`pip install --pre litellm`), so the workflow should mark them as
GitHub pre-releases either way.
Change the regex to `[-.]dev` so it accepts `.dev` and `-dev`.
* fix(bedrock): handle document content blocks in Converse API message conversion
Document content blocks (used for PDF support) were silently dropped
during message conversion for Bedrock's Converse API. The content block
processing loop only handled text, image_url, and file types — document
blocks were skipped without warning, causing the model to respond as if
no document was provided.
Adds document block handling in three locations:
- Sync user message processing (_bedrock_converse_messages_pt)
- Async user message processing (_bedrock_converse_messages_pt_async)
- Tool result conversion (_convert_to_bedrock_tool_call_result)
Fixes#24641
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use _validate_format for proper MIME type to Bedrock format mapping
Address Greptile review: naive media_type.split("/")[1] produced invalid
Bedrock format names for complex MIME types (e.g. OOXML → docx, text/plain
→ txt, text/markdown → md). Now reuses BedrockImageProcessor._validate_format
which handles all MIME types correctly via mimetypes + fallback.
Also fixes test assertions to expect correct Bedrock format values and adds
text/plain and text/markdown test cases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reject non-base64 document sources with a clear error
URL-type document sources (e.g. {"type": "url", "url": "..."}) would
crash with an opaque KeyError on missing 'media_type'. Guard at the top
of _process_document_message and raise a clear ValueError since Bedrock
Converse only supports base64-encoded document sources.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The /v2/model/info endpoint (used by the UI's Models + Endpoints page)
was not resolving access group names when filtering models by team.
When a team has models: ["Group-A"] where "Group-A" is an access group,
_filter_models_by_team_id() passed it as a literal model name to
get_model_list(), which found no deployments with that name. This caused
the UI to show all models instead of only team-accessible ones.
The request-time auth path (model_in_access_group in auth_checks.py)
correctly resolves access groups via get_model_access_groups(). This
fix applies the same resolution in _filter_models_by_team_id() for both
the in-memory router lookup and the database fallback query.
Tests added:
- test_filter_resolves_access_group_names
- test_filter_resolves_mix_of_access_groups_and_literal_names
- test_filter_excludes_models_from_other_access_group
- test_filter_db_fallback_receives_resolved_model_names
Orgs with no MCP permissions configured (the common default) previously
returned None without writing to cache, meaning every subsequent MCP
request triggered a fresh find_unique against litellm_organizationtable.
Cache a sentinel string on the negative path so the DB is queried at most
once per cache TTL per org, regardless of whether the org has an
object_permission or not.
Made-with: Cursor
Greptile P2: ``asyncio.gather`` without ``return_exceptions=True`` lets
the first failing call propagate immediately, leaving the other in-flight
inspections running until they complete on their own. Pass
``return_exceptions=True`` so every inspection finishes, then re-raise
the first exception encountered while iterating results.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Cache org object_permission in user_api_key_cache to avoid a DB hit on
every MCP request (was: raw find_unique on every call).
- Expand org mcp_servers list via expand_permission_list() so name-based
entries resolve to canonical IDs, consistent with key/team/end-user path.
- Expand org mcp_tool_permissions via expand_tool_permissions() in both
_get_allowed_mcp_servers_for_org and get_allowed_tools_for_server,
closing the silent name-vs-ID mismatch that could let restricted tools
through.
- The second _get_org_object_permission call in get_allowed_tools_for_server
now hits the cache (warm from the earlier server-list check), resolving
the double DB round-trip without changing the call structure.
Made-with: Cursor
Greptile P1: Aim's ``_anonymize_request`` and Lakera v2's mask-PII path
both wrote redacted content only to ``data["messages"]``. The Responses
API backend reads ``data["input"]``, so when a request arrived via
``/v1/responses`` with a plain string ``input`` the hook would update
``messages`` (which the backend ignores) and leave ``input`` carrying
the original unredacted text. Net effect: anonymize/mask silently passed
PII through to the LLM.
Add ``apply_redacted_messages_back`` to ``_content_utils`` — it writes
the redacted messages back to ``data["messages"]`` AND, when present,
re-flattens the redacted content into ``data["input"]``. Aim and
Lakera v2 now route their mask writeback through this helper. List
``input`` (multimodal) is still handled by the upstream
block-on-multimodal guard.
Adds unit tests for the helper and regression tests asserting
``data["input"]`` is redacted for both hooks on Responses-API string
input.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
