1. Cross-account false match: Added _parse_arn_account_and_role_name()
helper that compares partition + account ID + role name (not just
role name) to prevent same-name-different-account false matches.
2. SSL verify: _is_already_running_as_role() now passes ssl_verify to
the STS client via self._get_ssl_verify(), consistent with all other
boto3 client creation in this module.
3. Overbroad AccessDenied fallback: The catch in _auth_with_aws_role now
only falls back to ambient credentials when _is_already_running_as_role
positively confirms the caller is the target role. Genuine trust-policy
or permission misconfigurations are re-raised.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When aws_role_name is configured but the environment (ECS task role, EC2 instance
profile) is already running as that role, AssumeRole is unnecessary and can fail
with AccessDenied. This adds same-role detection for ECS/EC2 (extending existing
IRSA support) and a fallback to ambient credentials when AssumeRole returns
AccessDenied.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Arize Phoenix and Arize loggers now create dedicated TracerProviders
instead of fighting over the global singleton, and the otel callback
dedup check no longer incorrectly matches Arize subclasses.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: redaction headers ignored when sent via proxy
When requests go through the proxy, `litellm_params["litellm_metadata"]`
is always set (even when `None`), so `get_metadata_variable_name_from_kwargs`
always returns "litellm_metadata". The redaction code then reads `None`
instead of the actual metadata dict that contains the headers.
Add a fallback to read from `metadata` when `litellm_metadata` is not a
dict, so `x-litellm-enable-message-redaction` and related headers work
correctly in the proxy flow.
Fixes#20739
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: normalize non-dict metadata after fallback in redact_messages
After falling back from litellm_metadata to metadata, ensure the value
is always a dict so .get("headers") never raises on None/non-dict inputs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(batch_completion): submit all model futures before waiting
* test: add batch_completion all responses concurrency regression
* fix(batch_completion): continue collecting responses on per-model failures
* fix(batch_completion): handle empty and string models in all responses
* test(batch_completion): avoid blocking wait in concurrency regression
The provider-prefixed entries (deepseek/deepseek-chat, deepseek/deepseek-reasoner)
in the model cost map were missing supports_response_schema, supports_system_messages,
supports_native_streaming, supports_parallel_function_calling, and had stale
max_input_tokens / max_output_tokens values. This caused supports_response_schema()
to return False for DeepSeek models regardless of calling convention.
Changes:
- Sync deepseek/deepseek-chat and deepseek/deepseek-reasoner entries with
their canonical bare-name counterparts in both JSON files
- Add a bare-model-name fallback in _supports_factory so that when a
provider-prefixed entry is missing a capability field, the bare model
entry is consulted before returning False
- Fix pre-existing unused-import lint error (F401) in policy_resolve_endpoints.py
- Add 14 regression tests covering data consistency, API-level correctness,
and the new fallback logic
* feat: add opus 4.5 and 4.6 to use outout_format param
* generate poetry lock with 2.3.2 poetry
* restore poetry lock
* e2e tests, key delete, update tpm rpm, and regenerate
* Split e2e ui testing for browser
* new login with sso button in login page
* option to hide usage indicator
* fix(cloudzero): update CBF field mappings per LIT-1907 (#20906)
* fix(cloudzero): update CBF field mappings per LIT-1907
Phase 1 field updates for CloudZero integration:
ADD/UPDATE:
- resource/account: Send concat(api_key_alias, '|', api_key_prefix)
- resource/service: Send model_group instead of service_type
- resource/usage_family: Send provider instead of hardcoded 'llm-usage'
- action/operation: NEW - Send team_id
- resource/id: Send model name instead of CZRN
- resource/tag:organization_alias: Add if exists
- resource/tag:project_alias: Add if exists
- resource/tag:user_alias: Add if exists
REMOVE:
- resource/tag:total_tokens: Removed
- resource/tag:team_id: Removed (team_id now in action/operation)
Fixes LIT-1907
* Update litellm/integrations/cloudzero/transform.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: define api_key_alias variable, update CBFRecord docstring
- Fix F821 lint error: api_key_alias was used but not defined
- Update CBFRecord docstring to reflect LIT-1907 field mappings
- Remove unused Optional import
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add banner notifying of breaking change
* Add semgrep & Fix OOMs (#20912)
* [Feat] Policies - Allow connecting Policies to Tags, Simulating Policies, Viewing how many keys, teams it applies on (#20904)
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: type error & better error handling (#20689)
* [Docs] Add docs guide for using policies (#20914)
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs v1 guide with UI imgs
* docs fix
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add dashscope/qwen3-max model with tiered pricing (#20919)
Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix linting
* docs: add Greptile review requirement to PR template (#20762)
* fix(azure): preserve content_policy_violation error details from Azure OpenAI
Closes#20811
Azure OpenAI returns rich error payloads for content policy violations
(inner_error with ResponsibleAIPolicyViolation, content_filter_results,
revised_prompt). Previously these details were lost when:
1. The top-level error code was not "content_policy_violation" but the
inner_error.code was "ResponsibleAIPolicyViolation" -- the structured
check only examined the top-level code.
2. The DALL-E image generation polling path stringified the error JSON
into the message field instead of setting the structured body, making
it impossible for exception_type() to extract error details.
3. The string-based fallback detector used "invalid_request_error" as a
content-policy indicator, which is too broad and could misclassify
regular bad-request errors.
Changes:
- exception_mapping_utils.py: Check inner_error.code for
ResponsibleAIPolicyViolation when top-level code is not
content_policy_violation. Replace overly broad "invalid_request_error"
string match with specific Azure safety-system messages.
- azure.py: Set structured body on AzureOpenAIError in both async and
sync DALL-E polling paths so exception_type() can inspect error details.
- test_azure_exception_mapping.py: Add regression tests covering the
exact error payloads from issue #20811.
- Fix pre-existing lint: duplicate PerplexityResponsesConfig dict key,
unused RouteChecks top-level import.
---------
Co-authored-by: Kelvin Tran <kelvin-tran@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: ken <122603020@qq.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
* fix: reasoning_effort=None returns None for Opus 4.6
Previously, _map_reasoning_effort would return adaptive thinking
for Opus 4.6 even when reasoning_effort was None, which breaks the
expected contract where None means no thinking is sent.
* fix: handle reasoning_effort="none" string for Opus 4.6
The string "none" is a valid OpenAI reasoning_effort value meaning
"disable thinking". Previously it was mapped to adaptive for Opus 4.6.
* Update tests/litellm/llms/anthropic/test_anthropic_reasoning_effort.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Generic Guardrails: Forward request headers + litellm_version to generic guardrail API
* Generic Guardrail: Change the request headers addition to be with allowlist instead denylist
* fix(scheduler): remove timed-out requests from queue to prevent memory leak
Fixes#20059
* fix(scheduler): use actual model param instead of hardcoded gpt-3.5-turbo in schedule_acompletion
* trigger CLA recheck
---------
Co-authored-by: Piyush Bhawsar <piyush100x@Piyushs-MacBook-Pro-3.local>
Adds a new config option to exclude specific fields from StandardLoggingPayload
before any callback receives it. This provides a general approach to control
what data is logged across ALL integrations (S3, GCS, Datadog, etc.).
## Changes
1. **litellm/__init__.py**: Added new global setting
`standard_logging_payload_excluded_fields: Optional[List[str]] = None`
2. **litellm/integrations/custom_logger.py**: Modified
`redact_standard_logging_payload_from_model_call_details()` to:
- Remove specified fields entirely from the StandardLoggingPayload
- Works alongside existing `turn_off_message_logging` feature
- Excluded fields take precedence (removed rather than redacted)
3. **tests/**: Added comprehensive test suite with 17 tests covering:
- Single/multiple field exclusion
- Interaction with turn_off_message_logging
- Original payload immutability
- Config loading via setattr (proxy pattern)
- Edge cases (empty list, non-existent fields, None standard_logging_object)
## Usage
```yaml
litellm_settings:
success_callback: ["s3"]
standard_logging_payload_excluded_fields: ["response", "messages"]
```
This removes the `response` and `messages` fields from logs before any
callback processes them, reducing log size and improving privacy compliance.
## Available Fields
The fields match StandardLoggingPayload TypedDict keys including:
- messages, response (large payload fields)
- metadata, hidden_params, model_parameters
- error_str, error_information
- And all other StandardLoggingPayload fields
Closes the need for per-integration flags like `s3_log_response`.
* [Fix] handle metadata=None in SDK path retry/error logic (utils.py)
Fixes#20871
Same class of bug as #9717 (fixed by #9764 for the proxy path).
The SDK path in utils.py has the same fragile pattern at 7 locations.
Replace `kwargs.get("metadata", {})` with `(kwargs.get("metadata") or {})`
to handle the case where metadata key exists with value None (e.g. from
Azure OpenAI streaming responses).
This is consistent with the existing correct pattern at line 602:
`metadata = kwargs.get("metadata") or {}`
Adds TestMetadataNoneHandling with 6 unit tests in test_utils.py.
* fix: remove duplicate PerplexityResponsesConfig key in lazy imports registry
Removes duplicate dictionary key added in commit be0ebb15 (PR #20860).
The entry at line 1042 is identical to the existing entry at line 906.
This causes ruff F601 lint failure on all PRs targeting main.
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* feat(guardrail_hooks/): add guardrail logging to all unified guardrails
ensures unified guardrails use the 'log_guardrail_information' decorator for logging
* fix(custom_guardrail.py): don't log inputs on guardrail response - just emit state
* refactor: don't double log bedrock guardrail information
* feat: add in-product nudges for contributing + trying community custom code guardrails
allows users to contribute / share custom code guardrails
* fix(aiohttp): respect ssl_verify with shared sessions
* fix(aiohttp): resolve mypy error for ssl parameter type
Pass ssl kwarg conditionally to aiohttp request() only when explicitly
configured, since None is not a valid value for the ssl parameter
(expected SSLContext | bool | Fingerprint).
Replace 3 full tree traversals with 1 Pydantic dump + 3 dict lookups
per choice. Deletes _path_matches_pattern, _build_preserved_paths, and
_remove_none_except_preserved helpers (~140 lines removed). Adds 13
regression tests including full output structure snapshots.
When OpenTelemetry is configured via the UI, only OTEL_ENDPOINT and
OTEL_HEADERS are set, but OTEL_EXPORTER is not specified. This caused
the exporter to default to "console", meaning traces were printed to
stdout instead of being sent to the configured endpoint.
This fix adds logic in OpenTelemetryConfig.__post_init__ to automatically
infer "otlp_http" as the exporter when an endpoint is specified but the
exporter is still the default "console".
Fixes issue reported by Elastic team where traces weren't being sent
to their OTEL endpoint when configured through the LiteLLM UI.
* fix:Parse embedded JSON in the message field of logs
* Update litellm/_logging.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
When transforming chat completion responses to Responses API format,
image_tokens from completion_tokens_details was not being included
in output_tokens_details. This affected Vertex AI/Gemini models that
return image token counts in candidatesTokensDetails with modality="IMAGE".
The fix adds image_tokens handling alongside existing reasoning_tokens
and text_tokens transformation.
