- Fix async path: call async_get_chat_completion_prompt in aresponses()
before executor dispatch, mirroring acompletion() in main.py. Discard
merged_optional_params in async path (sync responses() handles them
via local_vars), avoiding TypeError from duplicate kwargs in partial().
- Fix provider re-resolution: replace "/" in model heuristic with
model != original_model comparison so bare model names are handled.
- Add 3 async tests covering hook invocation, optional param
propagation, and non-message item filtering in aresponses().
Made-with: Cursor
- Restore backward compat for leading assistant(tool_calls) — always prepend user_continue
- Replace partial assertions with full list assertion in trailing tool-call test
Made-with: Cursor
Avoid inserting assistant continue messages in the middle of assistant tool_call->tool chains by inserting before the next counted user turn, and add regression coverage for this edge case.
Made-with: Cursor
These were only used in block_key/unblock_key for cache refresh,
which now uses _delete_cache_key_object instead.
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
- test_block_key_nonexistent_key_returns_404: verifies block_key returns
404 (not misleading 401) when the key doesn't exist in the DB
- test_unblock_key_nonexistent_key_returns_404: same for unblock_key
- test_block_key_existing_key_succeeds: verifies block_key succeeds and
invalidates cache for existing keys
- Update test_unblock_key_supports_both_sk_and_hashed_tokens to reflect
the new cache invalidation pattern (_delete_cache_key_object instead
of get_key_object + _cache_key_object)
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
- Promote _fetch_managed_vector_stores_by_uuids from @staticmethod to a module-level
async helper get_managed_vector_store_rows_by_uuids, following the same standalone
helper pattern as get_team_object / get_key_object so the hot-path DB read is a
named importable function rather than an inline prisma_client.db.* call
- Pass no-log=True to both inner _call_aresponses sub-calls so they do not fire
independent billing/monitoring callbacks; cost is accumulated in the synthesized
response's _hidden_params for the outer responses() call
- Add test_H11b covering the primary queries (plural array) function-tool schema,
complementing H11 which exercises only the backward-compat singular query path
Made-with: Cursor
On /key/block, /key/unblock, and /key/update, the request body 'key'
field could contaminate the api_key Security dependency, causing the
auth layer to authenticate against the target key instead of the
caller's bearer token. This returned 401 for a nonexistent body key
even when the Authorization header contained a valid master key.
Added a guard in user_api_key_auth that re-reads the Authorization
header directly from the request, ensuring the header is always the
authoritative source for authentication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Re-add should_use_emulated_file_search() to emulated_handler.py so H5/H6/H7/H13 tests don't fail with ImportError
- Remove per-file-id deduplication from _build_search_results_for_include so all chunks are returned (matching OpenAI native file_search behaviour); update test_H14 to assert 2 results
- Extract raw prisma DB query in check_vector_store_ids_access into a static _fetch_managed_vector_stores_by_uuids helper so the hot request path uses a named, testable function instead of an inline prisma_client.db.* call
- Remove developer-local path from test module docstring
Made-with: Cursor
- Document intentional legacy-matching behavior: when user_id is
provided to an org admin, no org filter is applied (returns all of
that user's teams across all orgs, same as legacy endpoint)
- Fix two existing security tests to properly patch user_api_key_cache,
proxy_logging_obj, and get_user_object instead of relying on
incidental error handling
- Add three new org admin test cases:
- Org admin sees org-scoped teams (200 with correct where clause)
- Org admin rejected when filtering by other org (403)
- Org admin with user_id filter returns target user's teams
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace raw find_unique with get_user_object in
_build_team_list_where_conditions for cache/metrics consistency
- Remove over-complex OR clause for org admin + user_id: when user_id
is provided, filter by that user's direct team memberships (same as
regular users) since the access control gate already verified the
org admin's authority
- Preserve caller-supplied organization_id instead of overwriting with
org_admin_org_ids
- Update test mock to match get_user_object call path
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add _is_reasoning_effort_level_explicitly_disabled to use opt-out semantics
for minimal effort: unknown/unlisted models pass through, only blocked when
the model map explicitly sets supports_minimal_reasoning_effort=false.
xhigh keeps opt-in semantics (must be explicitly supported).
Adds test for unknown-model passthrough and explicit-disabled detection.
Made-with: Cursor
- Add try/except httpx.HTTPStatusError blocks in _async_cancel_batch for
both POST cancel and GET retrieve calls, with verbose_logger error logging
- Fix endpoint extraction inconsistency: compute endpoint from URL without
:cancel suffix so it matches behaviour of create_batch/retrieve_batch
- Add explicit validation that api_base ends with ':cancel' before
stripping it, raising a descriptive error for unsupported custom proxy
URL rewriting scenarios
- Use string-based patch() in test instead of patch.object() for robustness
against import order changes
Made-with: Cursor
- Fix bug where only last text item's annotations were preserved when
thread.message.completed contained multiple text content items
- Accumulate annotations via extend() instead of overwriting
- Add test_azure_ai_agents_streaming_annotations_from_completed_message
- Add test_azure_ai_agents_streaming_accumulates_annotations_from_multiple_text_items
Addresses Greptile review on PR #23849
Made-with: Cursor
_get_token_and_url_context_caching() was hardcoding model=None when
calling _check_custom_proxy(), which raises ValueError when api_base
is set because Gemini proxy URLs need the model name:
{api_base}/models/{model}:cachedContents
Fixes#23846
Azure GPT-5.4+ models now get the same auto-routing treatment as OpenAI
when both `reasoning_effort` and `tools` are used in `litellm.completion()`.
Previously, `reasoning_effort` was silently dropped for Azure; now the
request is bridged to the Responses API which supports both parameters.
Fixes#23914
Fixes#23890 — Mistral's Voxtral transcription with `diarize=true` returns
`segments` (with speaker_id, timestamps) and `language`, but these fields
were dropped when mapping the response to TranscriptionResponse.
Address review feedback from greptile — use new_callable=AsyncMock
on the concurrent test's patch.object to ensure the mock is properly
typed as async, even though side_effect already handles the coroutine.
The count_tokens handler unconditionally overrode vertex_location to
us-central1 for Claude models, ignoring the user-configured
vertex_count_tokens_location parameter. Also, us-central1 is no longer
a supported region — Google now supports us-east5, europe-west1, and
asia-southeast1.
Now vertex_count_tokens_location takes precedence, vertex_location is
used as fallback, and us-east5 is the default only when neither is set.
Fixes#23872
* akto guardrails support in litellm
* docs(guardrails): add akto to supported values in types/guardrails.py
* frontend changes + fixes
* feat(akto): update Akto guardrail integration with new configuration options and modes
* docs(akto): enhance Akto documentation and configuration descriptions for clarity
* feat(tests): add proxy server request headers to sample request data
* refactor(akto): remove optional account and VXLAN IDs; update documentation and tests
* feat(akto): add event_type parameter for enhanced observability in guardrail logging
* refactor(akto): update environment variable references
* refactor the python codes
* refactor and fix linting
* refactor(akto): remove unused event hook and clean up imports
* refactor(akto): enhance AktoGuardrail with async support and improved logging
* fix: Register DynamoAI guardrail initializer and enum entry (#23752)
* fix: Register DynamoAI guardrail initializer and enum entry
Fix the "Unsupported guardrail: dynamoai" error by:
1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum
2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py
The DynamoAI guardrail was added in PR #15920 but never properly registered
in the initialization system. The __init__.py was missing the
guardrail_initializer_registry and guardrail_class_registry dictionaries
that the dynamic discovery mechanism looks for at module load time.
Fixes#22773
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* test: Add tests for DynamoAI guardrail registration
Verifies enum entry, initializer registry, class registry,
instance creation, and global registry discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* docs: add v1.82.3 release notes and update provider_endpoints_support.json (#23816)
* Revert "docs: add v1.82.3 release notes and update provider_endpoints_support…" (#23817)
This reverts commit 966124966f.
* Refactor Akto guardrail configuration and tests; update UI description and tags
* add account and vxlan ID parameters to Akto guardrail initialization; update Akto logo format
* enhance Akto guardrail documentation and improve error handling for non-JSON responses
* address greptile issues
* fix: update payload handling to use 'data' instead of 'json' in AktoGuardrail and adjust tests accordingly
---------
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Anthropic's 'refusal' stop_reason was missing from _FINISH_REASON_MAP,
causing it to fall through to the default 'stop' — hiding the fact that
the model refused to respond due to safety policies.
Fixes#23793
Models like gemini-3.1-flash-lite-preview send the final streaming chunk
with empty content (text:"") alongside finishReason:"STOP", instead of
omitting content entirely. The existing fix (PR #21577) only handled
chunks without content, so this case was missed.
Now, after processing candidates, if tool_calls were seen in earlier
chunks and a choice has finish_reason="stop", it is overridden to
"tool_calls" to match the OpenAI spec.
Fixes#22900
