- Replace raw find_unique with get_user_object in
_build_team_list_where_conditions for cache/metrics consistency
- Remove over-complex OR clause for org admin + user_id: when user_id
is provided, filter by that user's direct team memberships (same as
regular users) since the access control gate already verified the
org admin's authority
- Preserve caller-supplied organization_id instead of overwriting with
org_admin_org_ids
- Update test mock to match get_user_object call path
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add _is_reasoning_effort_level_explicitly_disabled to use opt-out semantics
for minimal effort: unknown/unlisted models pass through, only blocked when
the model map explicitly sets supports_minimal_reasoning_effort=false.
xhigh keeps opt-in semantics (must be explicitly supported).
Adds test for unknown-model passthrough and explicit-disabled detection.
Made-with: Cursor
- Add try/except httpx.HTTPStatusError blocks in _async_cancel_batch for
both POST cancel and GET retrieve calls, with verbose_logger error logging
- Fix endpoint extraction inconsistency: compute endpoint from URL without
:cancel suffix so it matches behaviour of create_batch/retrieve_batch
- Add explicit validation that api_base ends with ':cancel' before
stripping it, raising a descriptive error for unsupported custom proxy
URL rewriting scenarios
- Use string-based patch() in test instead of patch.object() for robustness
against import order changes
Made-with: Cursor
_get_token_and_url_context_caching() was hardcoding model=None when
calling _check_custom_proxy(), which raises ValueError when api_base
is set because Gemini proxy URLs need the model name:
{api_base}/models/{model}:cachedContents
Fixes#23846
Azure GPT-5.4+ models now get the same auto-routing treatment as OpenAI
when both `reasoning_effort` and `tools` are used in `litellm.completion()`.
Previously, `reasoning_effort` was silently dropped for Azure; now the
request is bridged to the Responses API which supports both parameters.
Fixes#23914
Fixes#23890 — Mistral's Voxtral transcription with `diarize=true` returns
`segments` (with speaker_id, timestamps) and `language`, but these fields
were dropped when mapping the response to TranscriptionResponse.
Address review feedback from greptile — use new_callable=AsyncMock
on the concurrent test's patch.object to ensure the mock is properly
typed as async, even though side_effect already handles the coroutine.
The count_tokens handler unconditionally overrode vertex_location to
us-central1 for Claude models, ignoring the user-configured
vertex_count_tokens_location parameter. Also, us-central1 is no longer
a supported region — Google now supports us-east5, europe-west1, and
asia-southeast1.
Now vertex_count_tokens_location takes precedence, vertex_location is
used as fallback, and us-east5 is the default only when neither is set.
Fixes#23872
* akto guardrails support in litellm
* docs(guardrails): add akto to supported values in types/guardrails.py
* frontend changes + fixes
* feat(akto): update Akto guardrail integration with new configuration options and modes
* docs(akto): enhance Akto documentation and configuration descriptions for clarity
* feat(tests): add proxy server request headers to sample request data
* refactor(akto): remove optional account and VXLAN IDs; update documentation and tests
* feat(akto): add event_type parameter for enhanced observability in guardrail logging
* refactor(akto): update environment variable references
* refactor the python codes
* refactor and fix linting
* refactor(akto): remove unused event hook and clean up imports
* refactor(akto): enhance AktoGuardrail with async support and improved logging
* fix: Register DynamoAI guardrail initializer and enum entry (#23752)
* fix: Register DynamoAI guardrail initializer and enum entry
Fix the "Unsupported guardrail: dynamoai" error by:
1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum
2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py
The DynamoAI guardrail was added in PR #15920 but never properly registered
in the initialization system. The __init__.py was missing the
guardrail_initializer_registry and guardrail_class_registry dictionaries
that the dynamic discovery mechanism looks for at module load time.
Fixes#22773
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* test: Add tests for DynamoAI guardrail registration
Verifies enum entry, initializer registry, class registry,
instance creation, and global registry discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* docs: add v1.82.3 release notes and update provider_endpoints_support.json (#23816)
* Revert "docs: add v1.82.3 release notes and update provider_endpoints_support…" (#23817)
This reverts commit 966124966f.
* Refactor Akto guardrail configuration and tests; update UI description and tags
* add account and vxlan ID parameters to Akto guardrail initialization; update Akto logo format
* enhance Akto guardrail documentation and improve error handling for non-JSON responses
* address greptile issues
* fix: update payload handling to use 'data' instead of 'json' in AktoGuardrail and adjust tests accordingly
---------
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Anthropic's 'refusal' stop_reason was missing from _FINISH_REASON_MAP,
causing it to fall through to the default 'stop' — hiding the fact that
the model refused to respond due to safety policies.
Fixes#23793
Models like gemini-3.1-flash-lite-preview send the final streaming chunk
with empty content (text:"") alongside finishReason:"STOP", instead of
omitting content entirely. The existing fix (PR #21577) only handled
chunks without content, so this case was missed.
Now, after processing candidates, if tool_calls were seen in earlier
chunks and a choice has finish_reason="stop", it is overridden to
"tool_calls" to match the OpenAI spec.
Fixes#22900
- Add WARNING docstring to _get_shared_session_lock() about not resetting
the lock to None while coroutines may be in the recovery path
- Remove redundant proxy_server_module.shared_aiohttp_session assignment
in mock_init (add_shared_session_to_data overwrites it synchronously)
- Add try/except around _initialize_shared_aiohttp_session call to catch
and log exceptions (instead of letting them bubble to outer handler)
- Fix warning message when re-checked session is None (was incorrectly
logging closed session ID on a None session)
- Add debug logging to outer except handler instead of bare pass
- Add test for _initialize_shared_aiohttp_session raising exception
Strip internal logging ids from emulated sub-calls, dedupe included search_results by file_id, clean unused imports, and add unit coverage for dedupe behavior.
Made-with: Cursor
Address Greptile P1 review: tests that exercise the closed-session code
path need to reset the module-level lock to avoid RuntimeError on
Python < 3.10 when asyncio.Lock is reused across different event loops.
Include all function_call items when building emulated follow-up input and update tests to assert real emulated routing + Responses-format function tool structure.
Made-with: Cursor
When multiple requests detect a closed shared session simultaneously,
they would each create a new aiohttp.ClientSession, leaking intermediate
sessions and their TCP connectors. Added double-checked locking pattern
with asyncio.Lock to ensure only one coroutine recreates the session.
Added concurrent recreation test case.
Add Vertex batch cancellation support in LiteLLM batch APIs, route proxy cancel fallback using request provider headers, and return post-cancel batch state via retrieve to keep response shape compatible.
Made-with: Cursor
* fix(gemini): support images in tool_results for /v1/messages routing
convert_to_gemini_tool_call_result() dropped images in two cases:
- data-URL strings (data:image/...;base64,...) treated as plain text
- Anthropic image blocks in list content skipped
Add detection and convert both to Gemini inline_data BlobType so image
bytes are preserved.
Fixes#23712.
* fix(gemini): support images in tool_results for /v1/messages routing
convert_to_gemini_tool_call_result() dropped images in two cases:
- data-URL strings (data:image/...;base64,...) treated as plain text
- Anthropic image blocks in list content skipped
Add detection and convert both to Gemini inline_data BlobType so image
bytes are preserved.
Fixes#23712.
* fix(gemini): support images in tool_results for /v1/messages routing
convert_to_gemini_tool_call_result() dropped images in two cases:
- data-URL strings (data:image/...;base64,...) treated as plain text
- Anthropic image blocks in list content skipped
Add detection and convert both to Gemini inline_data BlobType so image
bytes are preserved.
Fixes#23712.
* fix(fireworks): skip #transform=inline for base64 data URLs
Closes#23583
* fix(fireworks): skip #transform=inline for base64 data URLs
Closes#23583
Appending #transform=inline to a data: URL corrupted the base64 payload,
causing binascii.Error (Incorrect padding) when Fireworks AI attempted to
decode the image. Data URLs are already inlined so the fragment is a no-op
anyway — guard both the str and dict image_url branches to skip the suffix
when the URL starts with "data:".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(fireworks): skip #transform=inline for base64 data URLs
Closes#23583
* fix(fireworks): skip #transform=inline for base64 data URLs
Closes#23583
* fix(fireworks): skip #transform=inline for base64 data URLs
Closes#23583
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix: Register DynamoAI guardrail initializer and enum entry
Fix the "Unsupported guardrail: dynamoai" error by:
1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum
2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py
The DynamoAI guardrail was added in PR #15920 but never properly registered
in the initialization system. The __init__.py was missing the
guardrail_initializer_registry and guardrail_class_registry dictionaries
that the dynamic discovery mechanism looks for at module load time.
Fixes#22773
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* test: Add tests for DynamoAI guardrail registration
Verifies enum entry, initializer registry, class registry,
instance creation, and global registry discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Add disable_custom_api_keys UI setting that prevents users from specifying
custom key values during key generation and regeneration. When enabled, all
keys must be auto-generated, eliminating the risk of key hash collisions
in multi-tenant environments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the shared aiohttp session closes (due to network interruption,
idle timeout, or Redis failover side effects), the proxy permanently
falls back to creating a new HTTPS connection per request, losing the
benefit of connection pooling for the entire pod lifetime.
Fix: make add_shared_session_to_data() async and recreate the session
when it is found closed, restoring connection pooling automatically.
Fixes#23806
When updating or regenerating a key without changing its key_alias, the
existing alias was being re-validated against current format rules. This
caused keys with legacy aliases (created before stricter validation) to
become uneditable. Now validation only runs when the alias actually changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
