* cache control for user messages and system messages
* add cache createion tokens in reponse
* cache controls in tool calls and assistant turns
* refactor with _should_preserve_cache_control
* add cache control unit tests
* use simpler cache creation token count logic
* use helper function
* remove unused function
* fix unit tests
* fix: make HTTPHandler mockable in OIDC secret manager tests
- Add _get_oidc_http_handler() factory function to make HTTPHandler
easily mockable in tests
- Update test_oidc_github_success to patch factory function instead
of HTTPHandler directly
- Update Google OIDC tests for consistency
- Fixes test_oidc_github_success failure where mock was bypassed
This change allows tests to properly mock HTTPHandler instances used
for OIDC token requests, fixing the test failure where the mock was
not being used.
* fix: patch base_llm_http_handler method directly in container tests
- Use patch.object to patch container_create_handler method directly
on the base_llm_http_handler instance instead of patching the module
- Fixes test_provider_support[openai] failure where mock wasn't applied
- Also fixes test_error_handling_integration with same approach
The issue was that patching 'litellm.containers.main.base_llm_http_handler'
didn't work because the module imports it with 'from litellm.main import',
creating a local reference. Using patch.object patches the method on the
actual object instance, which works regardless of import style.
* fix: resolve flaky test_openai_env_base by clearing cache
- Add cache clearing at start of test_openai_env_base to prevent cache pollution
- Ensures no cached clients from previous tests interfere with respx mocks
- Fixes intermittent failures where aiohttp transport was used instead of httpx
- Test-only change with low risk, no production code modifications
Resolves flaky test marked with @pytest.mark.flaky(retries=3, delay=1)
Both parametrized versions (OPENAI_API_BASE and OPENAI_BASE_URL) now pass consistently
* test: add explicit mock verification in test_provider_support
- Capture mock handler with 'as mock_handler' for explicit validation
- Add assert_called_once() to verify mock was actually used
- Ensures test verifies no real API calls are made
- Follows same pattern as test_openai_env_base validation
Mock _create_mcp_client to avoid network calls in health checks.
This prevents asyncio.CancelledError when the test teardown closes
the event loop while health checks are still pending.
The test focuses on conversion logic (access_groups, description)
not health check functionality, so mocking the network call is appropriate.
- Add test_get_valid_args in test_router_helper_utils.py to cover get_valid_args
- Use encoding='utf-8' in router_code_coverage.py for cross-platform file reads
- Create a test user with auto_create_key=False to ensure known starting state
- Filter get_users by user_ids to target only the test user
- Verify initial key count is 0 before creating a key
- Clean up test user after test completes
- This ensures consistent behavior across CI and local environments
- Added a Pydantic validator to convert empty string inputs for max_budget to None, preventing float parsing errors from the frontend.
- Modified the internal user update logic to explicitly allow max_budget to be None, ensuring the value isn't filtered out and can be reset to unlimited in the database.
- Added unit tests for validation and logic.
Closes#19781
Adds support for Anthropic-style 'thinking' parameter in hosted_vllm,
converting it to OpenAI-style 'reasoning_effort' since vLLM is
OpenAI-compatible.
This enables users to use Claude Code CLI with hosted vLLM models
like GLM-4.6/4.7 through the /v1/messages endpoint.
Mapping (same as Anthropic adapter):
- budget_tokens >= 10000 -> "high"
- budget_tokens >= 5000 -> "medium"
- budget_tokens >= 2000 -> "low"
- budget_tokens < 2000 -> "minimal"
Fixes#19761
* fix(proxy): use return value from CustomLogger.async_post_call_success_hook
Previously the return value was ignored for CustomLogger callbacks,
preventing users from modifying responses. Now the return value is
captured and used to replace the response (if not None), consistent
with CustomGuardrail and streaming iterator hook behavior.
Fixes issue with custom_callbacks not being able to inject data into
LLM responses.
* fix(proxy): also fix async_post_call_streaming_hook to use return value
Previously the streaming hook only used return values that started with
"data: " (SSE format). Now any non-None return value is used, consistent
with async_post_call_success_hook and streaming iterator hook behavior.
Added tests for streaming hook transformation.
---------
Co-authored-by: Gabriele Michelli <michelligabriele0@gmail.com>
* feat: add clientip and user agent in metrics
* fix: lint errors
* Add model id and other req labels
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)
OAuth discovery endpoints now support both URL patterns:
- Standard MCP pattern: /mcp/{server_name} (new)
- Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)
The standard pattern is required by MCP-compliant clients like
mcp-inspector and VSCode Copilot, which expect resource URLs
following the /mcp/{server_name} convention per RFC 9728.
Changes:
- Add _build_oauth_protected_resource_response() helper
- Add oauth_protected_resource_mcp_standard() endpoint
- Add oauth_authorization_server_mcp_standard() endpoint
- Keep legacy endpoints for backward compatibility
- Add tests for both URL patterns
Fixes#17272
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)
OAuth discovery endpoints now support both URL patterns:
- Standard MCP pattern: /mcp/{server_name} (new)
- Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)
The standard pattern is required by MCP-compliant clients like
mcp-inspector and VSCode Copilot, which expect resource URLs
following the /mcp/{server_name} convention per RFC 9728.
Changes:
- Add _build_oauth_protected_resource_response() helper
- Add oauth_protected_resource_mcp_standard() endpoint
- Add oauth_authorization_server_mcp_standard() endpoint
- Keep legacy endpoints for backward compatibility
- Add tests for both URL patterns
Fixes#17272
* Test was relocated
* refactor(mcp): Extract helper methods from run_with_session to fix PLR0915
Split the large run_with_session method (55 statements) into smaller
helper methods to satisfy ruff's PLR0915 rule (max 50 statements):
- _create_transport_context(): Creates transport based on type
- _execute_session_operation(): Handles session lifecycle
Also changed cleanup exception handling from Exception to BaseException
to properly catch asyncio.CancelledError (which is a BaseException subclass
in Python 3.8+).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(mcp): Fix flaky test by mocking health_check_server
The test_mcp_server_manager_config_integration_with_database test was
making real network calls to fake URLs which caused timeouts and
CancelledError exceptions.
Fixed by mocking health_check_server to return a proper
LiteLLM_MCPServerTable object instead of making network calls.
* test(mcp): Fix skip condition to properly detect claude model names
The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.
Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(mcp): Fix skip condition to properly detect claude model names
The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.
Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(proxy): support slashes in google route params
* fix(proxy): extract google model ids with slashes
* test(proxy): cover google model ids with slashes
* [Feat] Add model parameter to Generic Guardrail API
Add model information to guardrail requests, allowing guardrails to make
model-specific security decisions.
Changes:
- Add `model` field to GenericGuardrailAPIInputs TypedDict
- Add `model` field to GenericGuardrailAPIRequest Pydantic model
- Update OpenAI and Anthropic handlers to pass model from request/response
- Add unit tests for model parameter handling
* [Feat] Add model parameter to all guardrail_translation handlers
Extend model parameter support to all guardrail handlers for consistent
implementation across all endpoint types:
- OpenAI Responses API (input/output + streaming)
- OpenAI Image Generation (input only)
- OpenAI Text Completion (input/output)
- OpenAI Text-to-Speech (input only)
- OpenAI Audio Transcription (output only)
- Cohere Rerank (input only)
- Pass-through Endpoints (input/output)
- MCP Server (input only)
This addresses the review feedback requesting consistent model parameter
handling across all guardrail_translation/handler.py files.
---------
Co-authored-by: Igal Boxerman <igal@pillar.security>
