* fix(batch_completion): submit all model futures before waiting
* test: add batch_completion all responses concurrency regression
* fix(batch_completion): continue collecting responses on per-model failures
* fix(batch_completion): handle empty and string models in all responses
* test(batch_completion): avoid blocking wait in concurrency regression
The provider-prefixed entries (deepseek/deepseek-chat, deepseek/deepseek-reasoner)
in the model cost map were missing supports_response_schema, supports_system_messages,
supports_native_streaming, supports_parallel_function_calling, and had stale
max_input_tokens / max_output_tokens values. This caused supports_response_schema()
to return False for DeepSeek models regardless of calling convention.
Changes:
- Sync deepseek/deepseek-chat and deepseek/deepseek-reasoner entries with
their canonical bare-name counterparts in both JSON files
- Add a bare-model-name fallback in _supports_factory so that when a
provider-prefixed entry is missing a capability field, the bare model
entry is consulted before returning False
- Fix pre-existing unused-import lint error (F401) in policy_resolve_endpoints.py
- Add 14 regression tests covering data consistency, API-level correctness,
and the new fallback logic
* feat: add opus 4.5 and 4.6 to use outout_format param
* generate poetry lock with 2.3.2 poetry
* restore poetry lock
* e2e tests, key delete, update tpm rpm, and regenerate
* Split e2e ui testing for browser
* new login with sso button in login page
* option to hide usage indicator
* fix(cloudzero): update CBF field mappings per LIT-1907 (#20906)
* fix(cloudzero): update CBF field mappings per LIT-1907
Phase 1 field updates for CloudZero integration:
ADD/UPDATE:
- resource/account: Send concat(api_key_alias, '|', api_key_prefix)
- resource/service: Send model_group instead of service_type
- resource/usage_family: Send provider instead of hardcoded 'llm-usage'
- action/operation: NEW - Send team_id
- resource/id: Send model name instead of CZRN
- resource/tag:organization_alias: Add if exists
- resource/tag:project_alias: Add if exists
- resource/tag:user_alias: Add if exists
REMOVE:
- resource/tag:total_tokens: Removed
- resource/tag:team_id: Removed (team_id now in action/operation)
Fixes LIT-1907
* Update litellm/integrations/cloudzero/transform.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: define api_key_alias variable, update CBFRecord docstring
- Fix F821 lint error: api_key_alias was used but not defined
- Update CBFRecord docstring to reflect LIT-1907 field mappings
- Remove unused Optional import
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add banner notifying of breaking change
* Add semgrep & Fix OOMs (#20912)
* [Feat] Policies - Allow connecting Policies to Tags, Simulating Policies, Viewing how many keys, teams it applies on (#20904)
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: type error & better error handling (#20689)
* [Docs] Add docs guide for using policies (#20914)
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs v1 guide with UI imgs
* docs fix
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add dashscope/qwen3-max model with tiered pricing (#20919)
Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix linting
* docs: add Greptile review requirement to PR template (#20762)
* fix(azure): preserve content_policy_violation error details from Azure OpenAI
Closes#20811
Azure OpenAI returns rich error payloads for content policy violations
(inner_error with ResponsibleAIPolicyViolation, content_filter_results,
revised_prompt). Previously these details were lost when:
1. The top-level error code was not "content_policy_violation" but the
inner_error.code was "ResponsibleAIPolicyViolation" -- the structured
check only examined the top-level code.
2. The DALL-E image generation polling path stringified the error JSON
into the message field instead of setting the structured body, making
it impossible for exception_type() to extract error details.
3. The string-based fallback detector used "invalid_request_error" as a
content-policy indicator, which is too broad and could misclassify
regular bad-request errors.
Changes:
- exception_mapping_utils.py: Check inner_error.code for
ResponsibleAIPolicyViolation when top-level code is not
content_policy_violation. Replace overly broad "invalid_request_error"
string match with specific Azure safety-system messages.
- azure.py: Set structured body on AzureOpenAIError in both async and
sync DALL-E polling paths so exception_type() can inspect error details.
- test_azure_exception_mapping.py: Add regression tests covering the
exact error payloads from issue #20811.
- Fix pre-existing lint: duplicate PerplexityResponsesConfig dict key,
unused RouteChecks top-level import.
---------
Co-authored-by: Kelvin Tran <kelvin-tran@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: ken <122603020@qq.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
* fix: reasoning_effort=None returns None for Opus 4.6
Previously, _map_reasoning_effort would return adaptive thinking
for Opus 4.6 even when reasoning_effort was None, which breaks the
expected contract where None means no thinking is sent.
* fix: handle reasoning_effort="none" string for Opus 4.6
The string "none" is a valid OpenAI reasoning_effort value meaning
"disable thinking". Previously it was mapped to adaptive for Opus 4.6.
* Update tests/litellm/llms/anthropic/test_anthropic_reasoning_effort.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Generic Guardrails: Forward request headers + litellm_version to generic guardrail API
* Generic Guardrail: Change the request headers addition to be with allowlist instead denylist
* fix(scheduler): remove timed-out requests from queue to prevent memory leak
Fixes#20059
* fix(scheduler): use actual model param instead of hardcoded gpt-3.5-turbo in schedule_acompletion
* trigger CLA recheck
---------
Co-authored-by: Piyush Bhawsar <piyush100x@Piyushs-MacBook-Pro-3.local>
Replace asyncio.wait_for() with anyio.fail_after() in _fetch_tools_with_timeout()
to fix conflict with MCP SDK's anyio TaskGroup that causes 0 tools to be returned
for external StreamableHTTP MCP backends.
Root cause: asyncio.wait_for() wrapping anyio-managed code causes inconsistent
CancelledError propagation, resulting in false cancellations even when the
operation hasn't timed out.
Fixes#20715
Adds a new config option to exclude specific fields from StandardLoggingPayload
before any callback receives it. This provides a general approach to control
what data is logged across ALL integrations (S3, GCS, Datadog, etc.).
## Changes
1. **litellm/__init__.py**: Added new global setting
`standard_logging_payload_excluded_fields: Optional[List[str]] = None`
2. **litellm/integrations/custom_logger.py**: Modified
`redact_standard_logging_payload_from_model_call_details()` to:
- Remove specified fields entirely from the StandardLoggingPayload
- Works alongside existing `turn_off_message_logging` feature
- Excluded fields take precedence (removed rather than redacted)
3. **tests/**: Added comprehensive test suite with 17 tests covering:
- Single/multiple field exclusion
- Interaction with turn_off_message_logging
- Original payload immutability
- Config loading via setattr (proxy pattern)
- Edge cases (empty list, non-existent fields, None standard_logging_object)
## Usage
```yaml
litellm_settings:
success_callback: ["s3"]
standard_logging_payload_excluded_fields: ["response", "messages"]
```
This removes the `response` and `messages` fields from logs before any
callback processes them, reducing log size and improving privacy compliance.
## Available Fields
The fields match StandardLoggingPayload TypedDict keys including:
- messages, response (large payload fields)
- metadata, hidden_params, model_parameters
- error_str, error_information
- And all other StandardLoggingPayload fields
Closes the need for per-integration flags like `s3_log_response`.
When a guardrail (e.g. Zscaler AI Guard) or other exception is raised
during streaming, the error handler in `async_data_generator` includes
the full Python traceback in the SSE response sent to clients:
error_msg = f"{str(e)}\n\n{traceback.format_exc()}"
This leaks internal server details (file paths, line numbers, call
stacks) to end users. The traceback is already logged server-side via
`verbose_proxy_logger.exception()`, so including it in the client
response is unnecessary.
Change to only include the exception message (`str(e)`) in the SSE
error payload, consistent with how `StreamingCallbackError` is already
handled.
Fixes#20610
* [Fix] handle metadata=None in SDK path retry/error logic (utils.py)
Fixes#20871
Same class of bug as #9717 (fixed by #9764 for the proxy path).
The SDK path in utils.py has the same fragile pattern at 7 locations.
Replace `kwargs.get("metadata", {})` with `(kwargs.get("metadata") or {})`
to handle the case where metadata key exists with value None (e.g. from
Azure OpenAI streaming responses).
This is consistent with the existing correct pattern at line 602:
`metadata = kwargs.get("metadata") or {}`
Adds TestMetadataNoneHandling with 6 unit tests in test_utils.py.
* fix: remove duplicate PerplexityResponsesConfig key in lazy imports registry
Removes duplicate dictionary key added in commit be0ebb15 (PR #20860).
The entry at line 1042 is identical to the existing entry at line 906.
This causes ruff F601 lint failure on all PRs targeting main.
Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs v1 guide with UI imgs
* docs fix
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* init schema with TAGS
* ui: add policy test
* resolvePoliciesCall
* add_policy_sources_to_metadata + headers
* types Policy
* preview Impact
* def _describe_match_reason(
* match based on TAGs
* TestTagBasedAttachments
* test fixes
* add policy_resolve_router
* add_guardrails_from_policy_engine
* TestMatchAttribution
* refactor
* fix
* fix: address Greptile review feedback on policy resolve endpoints
- Track unnamed keys/teams as separate counts instead of inflating
affected_keys_count with duplicate "(unnamed key)" placeholders.
Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
converts exact patterns to Prisma "in" and suffix wildcards to
"startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
force_sync query param (default false) to avoid 2 DB round-trips
on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: eliminate duplicate DB queries and fix header delimiter ambiguity
- Fetch teams table once in estimate_attachment_impact and reuse for
both tag-based and alias-based lookups (was querying teams twice when
both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
filters that operate on pre-fetched data (_filter_keys_by_tags,
_filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
(e.g. "tag:healthcare+team:health-team") to avoid conflict with
header delimiters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
The template literal in the tags query parameter concatenation included
`${url}` inside a `+=` assignment, causing the full URL to be doubled.
Supersedes #8793 (original fix by @Mte90, now stale).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(guardrail_hooks/): add guardrail logging to all unified guardrails
ensures unified guardrails use the 'log_guardrail_information' decorator for logging
* fix(custom_guardrail.py): don't log inputs on guardrail response - just emit state
* refactor: don't double log bedrock guardrail information
* feat: add in-product nudges for contributing + trying community custom code guardrails
allows users to contribute / share custom code guardrails
