- Added a Pydantic validator to convert empty string inputs for max_budget to None, preventing float parsing errors from the frontend.
- Modified the internal user update logic to explicitly allow max_budget to be None, ensuring the value isn't filtered out and can be reset to unlimited in the database.
- Added unit tests for validation and logic.
Closes#19781
The issue was in `register_model()` where `split_string[1]` assumed the
model name always contained '/'. For custom model names like "glm" or
UUIDs, the split would only produce one element, causing an IndexError.
Changed `split_string[1]` to `split_string[-1]` which always returns
the last element, working correctly for both cases:
- "openrouter/gpt-4" → ["openrouter", "gpt-4"] → [-1] = "gpt-4"
- "my-custom-alias" → ["my-custom-alias"] → [-1] = "my-custom-alias"
Adds support for Anthropic-style 'thinking' parameter in hosted_vllm,
converting it to OpenAI-style 'reasoning_effort' since vLLM is
OpenAI-compatible.
This enables users to use Claude Code CLI with hosted vLLM models
like GLM-4.6/4.7 through the /v1/messages endpoint.
Mapping (same as Anthropic adapter):
- budget_tokens >= 10000 -> "high"
- budget_tokens >= 5000 -> "medium"
- budget_tokens >= 2000 -> "low"
- budget_tokens < 2000 -> "minimal"
Fixes#19761
* fix(proxy): use return value from CustomLogger.async_post_call_success_hook
Previously the return value was ignored for CustomLogger callbacks,
preventing users from modifying responses. Now the return value is
captured and used to replace the response (if not None), consistent
with CustomGuardrail and streaming iterator hook behavior.
Fixes issue with custom_callbacks not being able to inject data into
LLM responses.
* fix(proxy): also fix async_post_call_streaming_hook to use return value
Previously the streaming hook only used return values that started with
"data: " (SSE format). Now any non-None return value is used, consistent
with async_post_call_success_hook and streaming iterator hook behavior.
Added tests for streaming hook transformation.
---------
Co-authored-by: Gabriele Michelli <michelligabriele0@gmail.com>
LiteLLM's pyproject.toml specifies `python = ">=3.9,<4.0"`, supporting
Python 3.9 and above. However, the OpenAIContainerConfig class was
failing to load on Python 3.10 with:
TypeError: Cannot subclass typing.Any
This occurred because `BaseContainerConfig = Any` was used as a
runtime placeholder, and the class then inherited from it.
In Python 3.11+, subclassing `typing.Any` is allowed (added to support
dynamic classes like unittest.Mock where type checkers should skip
verification). However, in Python 3.10 and earlier, this raises a
TypeError.
The fix imports the actual BaseContainerConfig class at runtime instead
of using Any as a placeholder.
Fixes#19727
* feat: add clientip and user agent in metrics
* fix: lint errors
* Add model id and other req labels
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)
OAuth discovery endpoints now support both URL patterns:
- Standard MCP pattern: /mcp/{server_name} (new)
- Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)
The standard pattern is required by MCP-compliant clients like
mcp-inspector and VSCode Copilot, which expect resource URLs
following the /mcp/{server_name} convention per RFC 9728.
Changes:
- Add _build_oauth_protected_resource_response() helper
- Add oauth_protected_resource_mcp_standard() endpoint
- Add oauth_authorization_server_mcp_standard() endpoint
- Keep legacy endpoints for backward compatibility
- Add tests for both URL patterns
Fixes#17272
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)
OAuth discovery endpoints now support both URL patterns:
- Standard MCP pattern: /mcp/{server_name} (new)
- Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)
The standard pattern is required by MCP-compliant clients like
mcp-inspector and VSCode Copilot, which expect resource URLs
following the /mcp/{server_name} convention per RFC 9728.
Changes:
- Add _build_oauth_protected_resource_response() helper
- Add oauth_protected_resource_mcp_standard() endpoint
- Add oauth_authorization_server_mcp_standard() endpoint
- Keep legacy endpoints for backward compatibility
- Add tests for both URL patterns
Fixes#17272
* Test was relocated
* refactor(mcp): Extract helper methods from run_with_session to fix PLR0915
Split the large run_with_session method (55 statements) into smaller
helper methods to satisfy ruff's PLR0915 rule (max 50 statements):
- _create_transport_context(): Creates transport based on type
- _execute_session_operation(): Handles session lifecycle
Also changed cleanup exception handling from Exception to BaseException
to properly catch asyncio.CancelledError (which is a BaseException subclass
in Python 3.8+).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(mcp): Fix flaky test by mocking health_check_server
The test_mcp_server_manager_config_integration_with_database test was
making real network calls to fake URLs which caused timeouts and
CancelledError exceptions.
Fixed by mocking health_check_server to return a proper
LiteLLM_MCPServerTable object instead of making network calls.
* test(mcp): Fix skip condition to properly detect claude model names
The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.
Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(mcp): Fix skip condition to properly detect claude model names
The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.
Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(proxy): support slashes in google route params
* fix(proxy): extract google model ids with slashes
* test(proxy): cover google model ids with slashes
