1. Cross-account false match: Added _parse_arn_account_and_role_name()
helper that compares partition + account ID + role name (not just
role name) to prevent same-name-different-account false matches.
2. SSL verify: _is_already_running_as_role() now passes ssl_verify to
the STS client via self._get_ssl_verify(), consistent with all other
boto3 client creation in this module.
3. Overbroad AccessDenied fallback: The catch in _auth_with_aws_role now
only falls back to ambient credentials when _is_already_running_as_role
positively confirms the caller is the target role. Genuine trust-policy
or permission misconfigurations are re-raised.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When aws_role_name is configured but the environment (ECS task role, EC2 instance
profile) is already running as that role, AssumeRole is unnecessary and can fail
with AccessDenied. This adds same-role detection for ECS/EC2 (extending existing
IRSA support) and a fallback to ambient credentials when AssumeRole returns
AccessDenied.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add opus 4.5 and 4.6 to use outout_format param
* generate poetry lock with 2.3.2 poetry
* restore poetry lock
* e2e tests, key delete, update tpm rpm, and regenerate
* Split e2e ui testing for browser
* new login with sso button in login page
* option to hide usage indicator
* fix(cloudzero): update CBF field mappings per LIT-1907 (#20906)
* fix(cloudzero): update CBF field mappings per LIT-1907
Phase 1 field updates for CloudZero integration:
ADD/UPDATE:
- resource/account: Send concat(api_key_alias, '|', api_key_prefix)
- resource/service: Send model_group instead of service_type
- resource/usage_family: Send provider instead of hardcoded 'llm-usage'
- action/operation: NEW - Send team_id
- resource/id: Send model name instead of CZRN
- resource/tag:organization_alias: Add if exists
- resource/tag:project_alias: Add if exists
- resource/tag:user_alias: Add if exists
REMOVE:
- resource/tag:total_tokens: Removed
- resource/tag:team_id: Removed (team_id now in action/operation)
Fixes LIT-1907
* Update litellm/integrations/cloudzero/transform.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: define api_key_alias variable, update CBFRecord docstring
- Fix F821 lint error: api_key_alias was used but not defined
- Update CBFRecord docstring to reflect LIT-1907 field mappings
- Remove unused Optional import
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add banner notifying of breaking change
* Add semgrep & Fix OOMs (#20912)
* [Feat] Policies - Allow connecting Policies to Tags, Simulating Policies, Viewing how many keys, teams it applies on (#20904)
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: type error & better error handling (#20689)
* [Docs] Add docs guide for using policies (#20914)
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs v1 guide with UI imgs
* docs fix
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add dashscope/qwen3-max model with tiered pricing (#20919)
Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix linting
* docs: add Greptile review requirement to PR template (#20762)
* fix(azure): preserve content_policy_violation error details from Azure OpenAI
Closes#20811
Azure OpenAI returns rich error payloads for content policy violations
(inner_error with ResponsibleAIPolicyViolation, content_filter_results,
revised_prompt). Previously these details were lost when:
1. The top-level error code was not "content_policy_violation" but the
inner_error.code was "ResponsibleAIPolicyViolation" -- the structured
check only examined the top-level code.
2. The DALL-E image generation polling path stringified the error JSON
into the message field instead of setting the structured body, making
it impossible for exception_type() to extract error details.
3. The string-based fallback detector used "invalid_request_error" as a
content-policy indicator, which is too broad and could misclassify
regular bad-request errors.
Changes:
- exception_mapping_utils.py: Check inner_error.code for
ResponsibleAIPolicyViolation when top-level code is not
content_policy_violation. Replace overly broad "invalid_request_error"
string match with specific Azure safety-system messages.
- azure.py: Set structured body on AzureOpenAIError in both async and
sync DALL-E polling paths so exception_type() can inspect error details.
- test_azure_exception_mapping.py: Add regression tests covering the
exact error payloads from issue #20811.
- Fix pre-existing lint: duplicate PerplexityResponsesConfig dict key,
unused RouteChecks top-level import.
---------
Co-authored-by: Kelvin Tran <kelvin-tran@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: ken <122603020@qq.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
* fix(aiohttp): respect ssl_verify with shared sessions
* fix(aiohttp): resolve mypy error for ssl parameter type
Pass ssl kwarg conditionally to aiohttp request() only when explicitly
configured, since None is not a valid value for the ssl parameter
(expected SSLContext | bool | Fingerprint).
Vertex AI requires Anthropic beta flags in the request body
(anthropic_beta array), not as HTTP headers. The Bedrock handler
already extracts user-specified beta headers from the headers dict,
but the Vertex handler was missing this, causing extra_headers like
interleaved-thinking-2025-05-14 to be silently dropped.
This extracts anthropic-beta values from optional_params extra_headers
and merges them into the anthropic_beta request body field, and also
removes extra_headers from the request body since the parent's
transform_request spreads optional_params into data.
Set cached tokenizer config directly and mock both sync and async
tokenizer functions to avoid race conditions when running with
parallel test execution (-n 16).
The issue was that parallel tests could populate the
litellm.known_tokenizer_config cache between clearing it and
when the code checked it. This caused the sync code path to be
used instead of the async path, bypassing the mocked async functions.
Fix:
1. Set cache directly instead of clearing it
2. Also mock sync versions _get_tokenizer_config and _get_chat_template_file
This ensures the test is deterministic regardless of test execution order.
* _translate_openai_content_to_anthropic
* test_response_format_consistency
* test fixes unit tests
* test fix
* fix: use dict access for anthropic content blocks in tests (#20447)
The translate_openai_response_to_anthropic method returns dicts, not objects.
Changed .type/.text/.thinking attribute access to dict ['key'] access.
---------
Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
AsyncHTTPHandler.__del__ was closing httpx clients still in use by
AsyncOpenAI/AsyncAzureOpenAI due to independent cache lifecycles.
Restores standalone httpx client creation for OpenAI/Azure providers.
* fix: check for model_response_choices before guardrail input
* test: add tests for responses api translation
* fix: protect other guardrail translations
* refactor: remove type ignores
* anthropic request body got mutated fix
* add warning when extra_body is provided but user is non premium
* fix: resolve mypy union-attr errors in anthropic guardrail handler
Cast choices[0] to Choices type before accessing .message attribute
to satisfy mypy's union type checking for Choices | StreamingChoices.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* add logger when model response has no choices for streaming /response and /messages
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 541a2b075a.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 716ea0caa1.
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Add x-api-key header to CountTokens handler to match chat completion
authentication. Azure AI Anthropic requires this header per Microsoft's
native API format.
When using LiteLLM's Anthropic /v1/messages endpoint to route requests to
OpenAI models, requests fail if any tool name exceeds OpenAI's 64-character
limit. Anthropic API has no such limit, causing compatibility issues.
Changes:
- Add truncate_tool_name() function using {55-char-prefix}_{8-char-hash} format
- Modify translate_anthropic_tools_to_openai() to truncate and return mapping
- Modify translate_anthropic_tool_choice_to_openai() to truncate tool name
- Restore original tool names in responses using the mapping
- Support tool name restoration in streaming responses
- Add backwards-compatible API (existing methods still work)
The fix only applies when routing Anthropic requests to OpenAI models.
Native Anthropic/Claude requests pass through unchanged.
Fixes#19788
- Add `supported_regions: ["global"]` to Qwen MaaS models in model_prices_and_context_window.json
- Update `get_supported_regions()` to read directly from `model_cost` dict
- Update `get_complete_vertex_url()` to use `get_vertex_region()` for global-only models
- Update `create_vertex_url()` to generate correct URL for global location (without region prefix)
- Add tests for Qwen global endpoint support
