- Fix F821: add BaseTokenCounter TYPE_CHECKING import in gpt_transformation.py
- Remove duplicate auth invocation in count_response_input_tokens endpoint
- Preserve `strict` field during chat→Responses API tool conversion
- Fix docs tools example to use chat completions format (not Responses API format)
- Return None early for system-only messages to avoid noisy error logs
* fix: complexity_router fails on list-format message content (OpenAI multi-part messages)
When a client sends messages with list-format content
(e.g. [{"type": "text", "text": "..."}] as used by the OpenAI JS SDK
and other clients), the complexity_router's async_pre_routing_hook
skipped those messages because it only handled str content. This caused
user_message to be None, the hook returned None, and the router fell
through to selecting the complexity_router deployment itself
(model="auto_router/complexity_router") which litellm cannot dispatch,
resulting in LiteLLMUnknownProvider.
Fixes:
- Extract text from list-format content parts (type=text) before
classifying
- Return default_model instead of None when no user message can be
extracted, preventing the crash fallthrough
- Loosen PreRoutingHookResponse.messages type from Dict[str, str] to
Dict[str, Any] to accommodate list-format content values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update messages type annotation in async_pre_routing_hook to Dict[str, Any]
Consistent with PreRoutingHookResponse.messages type change and the
list-format content support added in the previous commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: normalize None content to empty string in complexity_router message parsing
msg.get("content", "") returns None when the key exists with value None
(e.g. assistant messages with tool calls). Use `or ""` to normalize
None to an empty string explicitly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: strip whitespace from joined list content parts in complexity_router
Prevents leading/trailing spaces when some content parts have empty
text values (e.g. " ".join(["", "hello"]) → " hello").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The OpenAPI-to-MCP feature (PR #21575) added spec_path to the code
(_types.py, mcp_server_manager.py) but missed adding the column to
the Prisma schema files. This causes "Could not find field spec_path"
errors when creating OpenAPI-based MCP servers via the UI or API.
Adds `spec_path String?` to LiteLLM_MCPServerTable in all three
schema files (root, litellm/proxy, litellm-proxy-extras).
Made-with: Cursor
* (sap) ensure tool parameters have type='object' for SAP compatibility
Fix SAP GenAI Hub Orchestration Service rejecting tool calls with error:
"400 - LLM Module: tools.0.custom.input_schema.type: Input should be 'object'"
Root cause: When Claude Code uses tools (like web_search) with the SAP provider
through LiteLLM's Anthropic experimental pass-through adapter, Anthropic's
input_schema format doesn't always include the required type="object" field.
The adapter's translate_anthropic_tools_to_openai() function was directly
copying input_schema to OpenAI's parameters field without ensuring the
type="object" requirement that SAP's API strictly enforces.
Changes:
- Modified translate_anthropic_tools_to_openai() to check if input_schema
is missing the type field and add type="object" if absent
- Preserves existing type field if already present
- Added comprehensive test suite (6 tests) covering:
- Missing type field scenario (now adds type="object")
- Existing type preservation
- Empty input_schema handling
- Multiple tools transformation
- Additional schema properties preservation
- SAP-specific compatibility regression test
Testing:
- All new tests pass (6/6 in test_anthropic_tool_schema_fix.py)
- All existing Anthropic tool tests pass (57/57 tool-related tests)
- SAP tool parameter validation tests pass (9/9 in test_sap_tool_parameters.py)
* (sap) enable native response_format for anthropic models
* (sap) filter strict param from model_params for GPT models only
* (sap) revert Anthropic adapter type='object' fix
The SAP FunctionTool Pydantic validator in litellm/llms/sap/chat/models.py
already ensures type='object' is added to all tool parameters for SAP
API compatibility.
The Anthropic adapter change affected ALL consumers, not just SAP, which
was broader scope than intended for this PR.
- Revert input_schema modification in Anthropic adapter
- Remove Anthropic-specific test file (SAP tests still cover this case)
* (sap) gate markdown stripping to Anthropic models only
SAP GenAI Hub with Anthropic models sometimes returns JSON wrapped in
markdown code blocks. GPT/Gemini/Mistral models don't exhibit this
behavior, so stripping is now gated to avoid accidentally modifying
valid responses that may contain markdown in JSON string values.
* fix(proxy): add guardrails list routes for internal users
* fix(ui): add guardrails fetch with v1/v2 fallback in networking
* fix(ui): allow internal users/team admins to select guardrails in create key modal
* fix(ui): show guardrails selector for internal users in key edit view
* fix(ui): pass canEditGuardrails flag to key info view
* test(ui): add tests for role-based guardrails access in key info view
* test(ui): update key edit view test for guardrails
The gemini-live-2.5-flash-preview-native-audio-09-2025 model only works
with WebSocket (Live API), not REST endpoints. Changed supported_endpoints
from /v1/chat/completions to /vertex_ai/live to reflect the actual
passthrough endpoint available in LiteLLM proxy.
Match the error handling pattern used in the Anthropic count_tokens
endpoint: catch ProxyException separately to surface its status code
and message, and include error details in the generic 500 fallback.
The user-specified async client was being overwritten by
`litellm.module_level_aclient` in `streaming_handler.py` when using
async+streaming with Gemini.
This fix adds a `gemini_client` parameter to `make_call()` (matching
the existing pattern in `make_sync_call()`) so the user's custom client
is preserved and not overwritten.
Fixes#17148
- Log provider token counting failures instead of silently swallowing
- Fall back to local tokenizer when provider returns error response
- Map assistant tool_calls to Responses API function_call items
- Concatenate multiple system messages instead of overwriting
- Hide internal error details from proxy API responses
- Narrow exception catch in handler to network/JSON errors only
- Update test to match new fallback behavior
- Add OpenAITokenCounter using POST /v1/responses/input_tokens endpoint
- Add litellm.acount_tokens() public async API that auto-routes to provider APIs
- Add proxy endpoint POST /v1/responses/input_tokens for OpenAI-compatible counting
- Transform chat tools format to Responses API format for correct token counting
- Fall back to local tiktoken when provider API unavailable
Fixes#22302
- Remove token field from JWTKeyMappingResponse to prevent hashed key exposure
- Use _to_response() helper on all CRUD endpoints to control returned fields
- Return 409 for unique constraint violations, 400 for FK violations, 404 for not found
- Add response_model to endpoint decorators
- Add 8 new unit tests covering error handling and token redaction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Kontext models (flux-kontext-pro, flux-kontext-max) support both
text-to-image and image editing. Add them to IMAGE_GENERATION_MODELS
and update supported_endpoints in model prices JSON.
- Change mode from "image_generation" to "image_edit" for all 4 BFL
image edit models (flux-kontext-pro, flux-kontext-max, flux-pro-1.0-fill,
flux-pro-1.0-expand)
- Rename shadowed api_base variable to complete_url in async handler
for consistency with sync path
- Remove response_format from supported params (BFL always returns URLs)
- Remove n and size from image edit supported params (not mapped)
- Raise ValueError on unknown model names instead of silently defaulting
