Commit 99c62ca40e removed "azure" from _RESPONSES_API_PROVIDERS,
routing Azure models through litellm.completion instead of
litellm.responses. The test was not updated to match, causing it
to assert against the wrong mock.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests were asserting no response.create/conversation.item.create sent to
backend when guardrail blocks, but the implementation intentionally sends
these to have the LLM voice the guardrail violation message to the user.
Updated assertions to verify the correct guardrail flow:
- response.cancel is sent to stop any in-progress response
- conversation.item.create with violation message is injected
- response.create is sent to voice the violation
- original blocked content is NOT forwarded
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a gunicorn worker exits (e.g. from max_requests recycling), its
per-process prometheus .db files remain on disk. For gauges using
livesum/liveall mode, this means the dead worker's last-known values
persist as if the process were still alive. Wire gunicorn's child_exit
hook to call mark_process_dead() so live-tracking gauges accurately
reflect only running workers.
* fix: add missing LiteLLM_ClaudeCodePluginTable to schema.prisma
- Claude Code Plugin Marketplace endpoints (/claude-code/marketplace.json,
/claude-code/plugins) were returning 500 errors because
LiteLLM_ClaudeCodePluginTable model was missing from both schema.prisma files
- Prisma client was generated without this table causing AttributeError:
'Prisma' object has no attribute 'litellm_claudecodeplugintable'
- Added missing model definition to root schema.prisma and
litellm/proxy/schema.prisma
Fixes#21310
* test: add regression test for LiteLLM_ClaudeCodePluginTable schema
* fix: address greptile review - add @updatedAt, clean up test imports
The /project/list endpoint was not returning created_at and updated_at timestamps because these fields were not defined in LiteLLM_ProjectTable. Added these fields to the model so FastAPI includes them in the response (values come from the database). This allows the UI to display project creation and last-updated times.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Tests go through the real get_async_httpx_client() code path to verify
clients remain usable after both capacity eviction and TTL expiry.
Regression tests for PR #22247.
In a multi-worker Uvicorn setup, a client that reconnects to a different
worker sends an mcp-session-id that the new worker has never seen. The
MCP SDK returns 400 because the session is unknown.
Fix: add _handle_stale_mcp_session() which inspects the inbound
mcp-session-id header before the request reaches the SDK. If the
session is not in this worker's _server_instances:
- Non-DELETE: strip the header so the SDK creates a fresh session
- DELETE: return 200 immediately (idempotent, session already gone)
No new dependencies, no Redis, no latency added to the hot path.
Fixes https://github.com/BerriAI/litellm/issues/20992
* [Docs] Enable local file support
Implemented internal handling for converting file-type documents to the required format for OCR processing, ensuring seamless integration with various providers.
* Refactor OCR file handling and improve security checks
Removed deprecated MIME type mapping and file conversion functions, replacing them with updated implementations. Enhanced security by rejecting 'file' document types in JSON requests, ensuring file uploads are handled via multipart/form-data. Updated tests to reflect these changes and ensure proper functionality.
* Enhance MIME type validation in OCR processing
Added a regular expression check to validate MIME types in the convert_file_document_to_url_document function, raising a ValueError for invalid types. Updated tests to ensure proper error handling for unsupported MIME types.
* Enhance type safety in OCR file handling
Added type casting for the uploaded file in the _parse_multipart_form function to ensure proper handling of UploadFile instances. This change improves type safety and reduces potential runtime errors during file processing.
* Refactor MIME type handling in document uploads
Updated the MIME type extraction logic to strip parameters from the Content-Type header, ensuring only the base type is used. Added tests to verify that MIME parameters are correctly handled and stripped in various scenarios.
* Update OCR documentation for MIME type recommendations and remove unnecessary tips
Clarified the recommended usage of MIME types for raw bytes in document uploads. Simplified the documentation by removing the tip about multipart file uploads from tools like Postman, ensuring a more concise and focused guide.
* Enhance multipart form handling in OCR endpoints
Updated the _parse_multipart_form function to ignore both 'file' and 'document' fields during form parsing, ensuring that the document built from the uploaded file is not overridden. Added a new test to verify that injected document fields do not affect the constructed document, improving security and robustness of the file upload process.
When a client sends null for assigned_team_ids or assigned_key_ids, ensure
the DB receives [] instead of null, preventing null from being stored where
empty list is expected. Extend test to verify the DB call uses [].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
set(None) raises TypeError when a client sends null for assigned_team_ids or
assigned_key_ids. Add `or []` to handle null safely, consistent with create.
Add test covering this case.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When creating, updating, or deleting access groups, automatically keep
team and key access_group_ids in sync with the access group's assigned_team_ids
and assigned_key_ids. Includes transaction-based DB updates, cache patching,
and handles out-of-sync data by unioning assigned_* fields with hasSome queries.
Adds 12 new tests covering sync behavior across all three CRUD operations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add new /public/endpoints endpoint that returns which providers support each LiteLLM
endpoint (e.g., chat_completions, embeddings). The endpoint reads from a local backup
JSON file bundled with the package, caches the result in-process, and transforms the
raw provider-centric data into an endpoint-centric response format.
Changes:
- Add litellm/provider_endpoints_support_backup.json (copy of root source file)
- Add Pydantic response models (EndpointProvider, SupportedEndpoint, SupportedEndpointsResponse)
- Add /public/endpoints route with transformation and caching logic
- Add 16 comprehensive tests covering HTTP layer and transformation functions
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix(gemini): enable inputAudioTranscription and handle transcription events for realtime guardrails
Gemini sends inputTranscription/outputTranscription inside serverContent separately from modelTurn/turnComplete. This adds handling to convert them into OpenAI-compatible events so the guardrail pipeline can inspect voice input, and enables inputAudioTranscription in the session setup config.
Made-with: Cursor
* fix(vertex_ai): enable inputAudioTranscription in realtime session config
Add inputAudioTranscription to the Vertex AI realtime setup so the backend returns transcripts of user speech, allowing guardrails to inspect voice input.
Made-with: Cursor
* fix(realtime): pass user_api_key_dict and guardrail metadata through async_realtime handler
The base LLM HTTP handler's async_realtime method was not accepting or forwarding user_api_key_dict and litellm_metadata to RealTimeStreaming. This meant guardrails configured with default_on=false were silently skipped for all provider_config-based realtime connections (Gemini, Vertex AI, etc). Also fixes wss:// connections when SSL_VERIFY=False by overriding ssl=False for secure WebSocket URLs.
Made-with: Cursor
* fix(realtime): forward guardrail metadata for generic provider_config and vertex_ai paths
The _arealtime function was not passing user_api_key_dict or litellm_metadata to base_llm_http_handler.async_realtime() for the generic provider_config path and the vertex_ai-specific path. This broke guardrail resolution since RealTimeStreaming.request_data was empty, causing should_run_guardrail to return False.
Made-with: Cursor
* fix(realtime): voice guardrail responses and block duplicate response.create on text input
When a guardrail blocks voice input, send a conversation.item.create + response.create to the backend so the LLM voices the guardrail message as audio instead of only returning text. Also adds pending_guardrail_message tracking to suppress the automatic response.create the client sends after a blocked text message, and broadens _has_audio_transcription_guardrails to match pre_call/post_call modes.
Made-with: Cursor
* test(realtime): update guardrail tests for broadened audio transcription check and add integration tests
Update existing tests to reflect that pre_call guardrails now correctly trigger the audio/VAD session.update injection. Add integration test file for live OpenAI realtime guardrail testing.
Made-with: Cursor
* fix(realtime): instruct LLM to say exact guardrail message verbatim
The previous prompt gave the LLM creative freedom to paraphrase the guardrail violation message. Now it instructs the LLM to repeat the exact configured message word for word.
Made-with: Cursor
* fix(realtime): preserve wss ssl semantics and move live guardrail test
Keep TLS enabled for wss realtime sessions while honoring SSL_VERIFY=False via a no-verify SSLContext, move the OpenAI live guardrail test into llm_translation, and dedupe duplicated guardrail-detection helpers to prevent drift.
Made-with: Cursor
Same bug as team budget: _assemble_user_object fetched user info from DB
but only used budget_reset_at, discarding max_budget. When the key cache
has a stale None for user_max_budget, _safe_get_remaining_budget returns
+Inf. Now falls back to DB max_budget when metadata value is None.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The file was at the repo root and excluded from pip distributions. Moving it to litellm/proxy/public_endpoints/ alongside the other provider JSON files ensures it is packaged correctly. Updates all references in the endpoint handler, coverage tests, and release notes instructions.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Return all supported endpoints and which providers support them. Includes endpoint display names, URL paths, and per-provider support lists. Results are cached for the process lifetime.
Also adds comprehensive test coverage for the new endpoint.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
