* [Feat] Add model parameter to Generic Guardrail API
Add model information to guardrail requests, allowing guardrails to make
model-specific security decisions.
Changes:
- Add `model` field to GenericGuardrailAPIInputs TypedDict
- Add `model` field to GenericGuardrailAPIRequest Pydantic model
- Update OpenAI and Anthropic handlers to pass model from request/response
- Add unit tests for model parameter handling
* [Feat] Add model parameter to all guardrail_translation handlers
Extend model parameter support to all guardrail handlers for consistent
implementation across all endpoint types:
- OpenAI Responses API (input/output + streaming)
- OpenAI Image Generation (input only)
- OpenAI Text Completion (input/output)
- OpenAI Text-to-Speech (input only)
- OpenAI Audio Transcription (output only)
- Cohere Rerank (input only)
- Pass-through Endpoints (input/output)
- MCP Server (input only)
This addresses the review feedback requesting consistent model parameter
handling across all guardrail_translation/handler.py files.
---------
Co-authored-by: Igal Boxerman <igal@pillar.security>
- Add @lru_cache decorator to get_model_info() and _cached_get_model_info_helper()
- Update _invalidate_model_cost_lowercase_map() to clear these caches when model_cost changes
- Update test to call cache invalidation after modifying litellm.model_cost
Reduces get_model_cost_information from 46% to <1% of request handling time.
Add support for /embeddings endpoint via Vercel AI Gateway.
Closes#19658
Changes:
- Add VercelAIGatewayEmbeddingConfig in litellm/llms/vercel_ai_gateway/embedding/
- Register provider in utils.py and main.py
- Add unit tests for embedding transformation
- Update documentation with embeddings examples
Usage:
```python
from litellm import embedding
response = embedding(
model="vercel_ai_gateway/openai/text-embedding-3-small",
input="Hello world",
api_key="your-api-key"
)
```
* Fix: SSO user roles are not updated for existing users
Fixes#19620
* Refactor: Remove redundant user_info retrieval in SSOAuthenticationHandler
* Test: add new tests for user creation and updates in get_user_info_from_db
Fixes#19508
The cost calculation for gpt-image-1.5 was not including image tokens
from output_tokens_details, causing costs to be underreported
(e.g., $0.046 instead of $0.14).
Root cause: The OpenAI image generation API uses Responses API naming
(input_tokens, output_tokens, output_tokens_details) but the cost
calculator expected Chat Completions API naming (prompt_tokens,
completion_tokens, completion_tokens_details).
Changes:
- convert_dict_to_response.py: Map Responses API fields to Chat
Completions API fields and convert dicts to wrapper objects
- cost_calculator.py: Use usage directly if already transformed,
avoiding double transformation that lost the wrapper objects
- Added test for gpt-image-1.5 output image token cost calculation
* Enhance error handling for num_retries in Router class to support string values. Add test case to verify conversion from string to int for deployment num_retries.
* Refactor Router class for improved readability by formatting long lines and enhancing exception handling tests for num_retries. Ensure consistent style in test cases for better maintainability.
* Update exception handling for num_retries in Router class to suppress mypy warnings. Add type ignore comment for clarity in type conversion from string to int.
* added support for nova grounding for amazon nova model
* added citations support
* added integration tests
* removing test file
* refactor: Use web_search_options for Nova grounding instead of system_tool
---------
Co-authored-by: Juhie <juhiechandra@gmail.com>
Co-authored-by: Juhie <75068056+juhiechandra@users.noreply.github.com>
The websearch interception handler was passing internal flags like
`_websearch_interception_converted_stream` to the follow-up LLM request.
This caused "Extra inputs are not permitted" errors from providers like
Bedrock that use strict Pydantic validation.
Fix: Filter out all kwargs starting with `_websearch_interception` prefix
before making the follow-up anthropic_messages.acreate() call.
The Azure Responses API uses a different schema (flattened) for tools compared to the standard OpenAI/Azure Chat Completions API (nested). This caused a `BadRequestError` when users passed standard tool definitions.
Changes:
- Implemented tool flattening logic in `AzureOpenAIResponsesAPIConfig.transform_responses_api_request`.
- Added comprehensive unit tests in test_azure_transformation.py to verify nested-to-flat transformation, pass-through of flat tools, and immutability.
- Ensures cross-provider compatibility for tool definitions.
Fixes#19523
* Consolidated change
* fix(prompt_security): update message processing to persist sanitized files and filter for API calls
* fix per krrishdholakia suggestion
Forward static_headers from /mcp-rest/test/* routes into the MCP client so headers are present during session.initialize() and tool discovery.
Also add a shared merge_mcp_headers() helper to keep header precedence consistent and ensure OpenAPI-to-MCP generated tools include static_headers.
Tests:
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_rest_endpoints.py
- pytest tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py -k register_openapi_tools_includes_static_headers
Fixes#19341
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
