- Keep Anthropic-native tools (tool_search_tool_regex, web_search, bash, etc.) in original format when translating to OpenAI format for guardrails
- Convert guardrail-returned tools back from OpenAI to Anthropic format (type=custom for user tools)
- Add TOOL_SEARCH_TOOL to ANTHROPIC_HOSTED_TOOLS enum; use prefix matching for native tool detection
- Set type=custom explicitly when mapping OpenAI function tools to AnthropicMessagesTool
- Add test for Anthropic native tools with guardrails
Made-with: Cursor
1. Add missing __init__.py files in tests/test_litellm/llms/gemini/ and
subdirectories (realtime/, image_edit/) to fix ModuleNotFoundError
with pytest-xdist parallel workers.
2. Update test_transform_request_uses_dynamic_max_tokens to use
claude-3-7-sonnet-20250219 (max_output_tokens=64000) since
claude-3-5-sonnet-20241022 was removed from model_prices JSON
during deprecated model cleanup. The test assertion was outdated.
3. Update context caching TTL tests to use gemini-2.5-pro instead of
gemini-1.5-pro. The old model was removed from model_prices JSON,
causing supports_system_messages to return False, which prevented
system_instruction from appearing in the transformation output.
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
Update transform_image_generation_response test calls to pass required
explicit params (request_data, optional_params, litellm_params, encoding)
that replaced **kwargs in the method signature.
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
The empty-line filter in __next__/__anext__ called .strip() without
checking the type first. When the Responses API yields Pydantic
BaseModel events (e.g. ResponseCreatedEvent), this raises
AttributeError. Add an isinstance(str_line, str) guard so non-string
objects pass through to _handle_string_chunk as intended.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Lint fixes (check_code_and_doc_quality job):
- Remove unused variable reasoning_effort in gpt_5_transformation.py (F841)
- Remove unused timezone imports in mcp_server rest_endpoints.py and server.py (F401)
- Remove unused ProxyBaseLLMRequestProcessing import in realtime endpoints.py (F401)
- Add BaseRealtimeHTTPConfig to TYPE_CHECKING block in utils.py (F821)
- Add PLR0915 per-file-ignore for mcp_server/rest_endpoints.py in ruff.toml
Test fixes (litellm_mapped_tests_llms job):
- Gemini video cost tests: pass explicit model_info to video_generation_cost()
instead of relying on gemini/veo-3.0-generate-preview being in model_prices JSON
- Anthropic max_tokens tests: mock get_max_tokens() to return expected values
instead of depending on claude-3-5-sonnet-20241022 being in model_prices JSON
- Vertex AI pydantic obj test: update from removed gemini-1.5-pro to gemini-2.5-flash,
update expected request body to use response_json_schema format
- Vertex AI/Bedrock file_content integration tests: update mocks to target
base_llm_http_handler.retrieve_file_content (the new code path via
ProviderConfigManager) instead of the old vertex_ai_files_instance/
bedrock_files_instance paths
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
OpenAI rejects any reasoning_effort (even 'none') with tools in
/v1/chat/completions for gpt-5.4. Update the guard to drop reasoning_effort
regardless of value. Add docs explaining the auto-drop behavior.
Keep both sets of tests: upstream's OAuth2 token injection test and
our case-insensitive tool matching tests. Use upstream's version of
the bedrock output_config test (more comprehensive).
- Azure Model Router transform_response: let parent extract actual model from raw response
- common_request_processing: skip model override for Azure Model Router requests
- proxy_server: skip streaming chunk model restamp for Azure Model Router
- Add _is_azure_model_router_request helper
- Add tests for non-streaming and streaming
Made-with: Cursor
Snowflake's Cortex LLM API (like Anthropic) requires tool_choice as an
object with a "type" field, not as a bare string. Passing tool_choice="auto"
(or "required"/"none") results in error 390142 "invalid payload".
This fix transforms OpenAI string tool_choice values to the Snowflake
object format:
- "auto" -> {"type": "auto"}
- "required" -> {"type": "any"} (Snowflake/Anthropic convention)
- "none" -> {"type": "none"}
The dict-to-dict transformation for specific function tool choices
({"type": "function", "function": {"name": "..."}} -> {"type": "tool",
"name": [...]}) remains unchanged.
Fixes#23284
Co-authored-by: gambletan <tan@echooo.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
- MCP tests: set mock_mcp_server.oauth2_flow = None to prevent MagicMock
leaking into Pydantic Literal validation for MCPServer
- AgentCore tests: pass api_key="test-jwt-token" to bypass SigV4 credential
lookup that fails in CI without AWS credentials
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add verbose_logger.warning when user-specified region is overridden by
supported_regions. Remove now-unused is_global_only_vertex_model function
and its tests since get_vertex_region handles all region logic directly.
- get_vertex_region now overrides user-specified region when it's not in
the model's supported_regions list (prevents 404 for users with a
global VERTEXAI_LOCATION default hitting global-only models)
- Add supported_regions: ["global"] to glm-5-maas in both JSON files
- Update tests to cover the override behavior
- Remove redundant get_vertex_region() call in partner models main.py
(already called inside get_complete_vertex_url)
- Rewrite test mocks to use patch.dict(litellm.model_cost) instead of
patching the removed is_global_only_vertex_model symbol
- Align test assertions with new behavior: user-specified region is
preserved (not overridden) for global-only models
Implement Anthropic Files API (upload, retrieve, list, delete, content)
using the BaseFilesConfig provider pattern. Adds multipart form-data
support to BaseLLMHTTPHandler for file uploads.
* fix(anthropic): enforce type:'object' on tool input schemas
Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.
The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.
Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.
Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.
Related issues: #12020, #64, #1671
* fix(anthropic): deduplicate tool_result messages by tool_call_id
Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.
This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().
Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).
Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.
Related issues: #11804, #11029, #6836, #1782, #151
* fix: shallow copy input_schema to avoid caller mutation + add mutation guard test
Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call
* feat: add qwen3.5 series for openrouter
* fix: typo on max_output_tokens and max_tokens from qwen3.5 series
* chore: fix
* chore: fix
* [Test] UI - Logs: Add unit tests for 5 untested view_logs components
Add vitest tests for TypeBadges, ErrorViewer, ConfigInfoMessage, TimeCell, and TruncatedValue covering rendering, user interactions, and edge cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(chat-ui): responses API + MCP tool execution in /chat (#23297)
* feat(ui): add Chat UI v0 — standalone LiteLLM-branded chat window
Adds a full chat UI accessible from the sidebar Chat link (opens in new tab).
- Standalone route at /chat (outside dashboard layout — no Navbar/Sidebar chrome)
- Claude.ai-style layout: model selector top-left, LiteLLM logo center, settings top-right
- Greeting with time-of-day, centered input card, suggestion chips (Write/Learn/Code/Brainstorm)
- Sliding conversation history sidebar with Cmd+K search, rename, delete, date grouping
- localStorage-backed conversation persistence (litellm_chat_history_v1)
- Streaming completions via makeOpenAIChatCompletionRequest with AbortController stop support
- MCP server picker (toggle servers on/off per conversation)
- LiteLLM aesthetic: white/light-gray background, Ant Design blue (#1677ff) primary, system font
- Sidebar2: Chat menu item opens in new tab via window.open
* feat(chat-ui): responses API + MCP tool execution display
- Switch /chat from chat completions to responses API (previous_response_id session chaining)
- Add MCP server picker with search filter in chat input bar
- Show MCP tool call events (list_tools + call_tool) inline in chat via MCPEventsDisplay
- Add tool chip strip showing available tools when MCP servers are selected
- Non-blocking MCP toggle: server added immediately, verification in background (works for no-auth MCPs like deepwiki)
- Add truncateAfterMessage to useChatHistory for edit/retry
- Sync activeConversationId on URL change (fixes stale conversation on new chat)
- Add "Open Chat" shortcut button to sidebar
* fix(chat-ui): switch to responses API, remove dead code, add tests
- Switch handleSend from makeOpenAIChatCompletionRequest to makeOpenAIResponsesRequest with previous_response_id session chaining
- Add responsesSessionId state; reset to null when starting a new conversation
- Remove unused ChatInputBar.tsx and ModelSelector.tsx (dead code)
- Add tests/test_litellm/test_chat_ui_responses_session.py covering previous_response_id forwarding and signature validation
* fix(chat-ui): address greptile review issues
- Reset responsesSessionId when activeConversationId changes (not just on new conversation)
- Wire onMCPEvent callback into makeOpenAIResponsesRequest; render MCPEventsDisplay below messages
- Clear mcpEvents on each new send
- Explicitly filter history to user/assistant roles only (no tool-role casting)
- Remove duplicate "Chat" menu item from sidebar (pinned button serves same purpose)
- Make Sider a flex column so "Open Chat" button actually pins to bottom
- Fix tests to intercept real HTTP requests and assert previous_response_id in body
* fix(chat-ui): address greptile review feedback (greploop iteration 1)
- Fix duplicate context: when responsesSessionId is set, only send the
new user message as input (prior context is already server-side via
session chaining). Full history is still sent on the first turn.
- Fix ephemeral MCP events: store events per-message in ChatMessage.mcpEvents
instead of ephemeral component state. Events now survive across turns
and render inline below each assistant response via MCPEventsDisplay.
- Remove stale mcpEvents useState and ephemeral panel at bottom of chat.
* fix(chat-ui): address greptile review feedback (greploop iteration 2)
- Fix stale session on edit/retry: derive previousResponseId as null when
historyOverride is set so edit/retry always starts a fresh Responses API
session rather than chaining off a now-invalid prior session
- Fix unsafe MCPEvent cast: import MCPEvent directly from MCPEventsDisplay
into types.ts and type ChatMessage.mcpEvents as MCPEvent[], eliminating
the bare 'as MCPEvent[]' cast in ChatMessages.tsx
* fix(chat-ui): fix MCPEvent layering, batch localStorage writes, module-level test imports
- Move MCPEvent interface definition into chat/types.ts (single source of truth)
- MCPEventsDisplay.tsx now imports MCPEvent from types.ts instead of defining it locally
- Batch MCP event localStorage writes: accumulate during stream, persist once in finally
- Move test imports to module level per PEP 8 convention
* fix(chat-ui): fix MCPEvent import path and rename truncateFromMessage
- responses_api.tsx now imports MCPEvent directly from chat/types (not via MCPEventsDisplay re-export)
- Remove the now-unnecessary MCPEvent re-export from MCPEventsDisplay.tsx
- Rename truncateAfterMessage → truncateFromMessage: the function removes the target message and all subsequent ones (not just what comes after), so the new name accurately describes the behavior
* fix(responses-api): fix whitespace token filter and MCP server URL construction
- Drop the delta.trim() whitespace filter that was silently swallowing spaces
and newlines during streaming, causing words to concatenate and paragraphs
to collapse. Only skip truly empty strings (delta.length > 0).
- Use proxyBaseUrl for MCP server_url construction instead of the hardcoded
relative path "litellm_proxy/mcp", so non-root deployments route correctly.
* fix(responses-api): use unique server_label per MCP server to prevent tool routing collisions
* fix(chat-ui): move MCPEvent to shared mcp_tools/types, skip partial events on abort
- Move MCPEvent interface to mcp_tools/types.tsx (shared with MCPServer/MCPTool),
eliminating the playground→chat cross-module dependency. chat/types.ts and
both playground components now import from mcp_tools/types.
- Only persist accumulated MCP events when the stream completes cleanly; aborted
or errored turns drop partial events to avoid showing incomplete tool calls.
* fix(responses-api): use server_name for MCP URL routing, fix test path
- Use server_name (not alias) as the URL path segment for MCP server_url;
alias is a display name that may differ from the registered proxy route.
URL-encode the path to handle names with spaces/special characters.
- Fix sys.path.insert in tests to use __file__-relative path so tests pass
regardless of which directory pytest is invoked from.
* fix(chat-ui): fix stale session after failed edit, clean MCP event persistence, unique server_label
- Eagerly call setResponsesSessionId(null) when historyOverride is set so a
failed/aborted edit does not leave a stale session contaminating the next turn
- Replace abort-signal check with streamCompletedCleanly flag to correctly skip
MCP event persistence on both abort and non-abort errors (network/API failures)
- Use server_name (unique) as server_label instead of alias to prevent silent
tool-routing failures when two MCP servers share the same display name
* [Feat] UI - Show logos on MCP Apps page (#23320)
* feat(ui): add MCP server logo support across admin and chat UIs
- New MCPLogoSelector component with grid of well-known logos (GitHub,
Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering
- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
including /ui/chat/ (absolute paths resolve correctly everywhere)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(codeql): remove ruby from language matrix (#23227)
* Add team-scoped MCP server filtering for key creation and fix UnboundLocalError
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix cross-team MCP server info disclosure and restricted key bypass
The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix mcp_tool_permissions JSON string deserialization in _resolve_team_allowed_mcp_servers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Feature] UI - MCP Servers: Add per-server health recheck
Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix Anthropic docs link for beta endpoint
Update the Anthropic /v1/messages beta endpoint docstring to point to
its current pass-through documentation.
This keeps the change scoped to the incorrect URL and avoids changing
unverified wording in the surrounding comment.
---------
Co-authored-by: netbrah <162479981+netbrah@users.noreply.github.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
Perplexity requires type='message' on input items. Add _ensure_message_type
to automatically set it when missing, matching the old _format_input behavior.
Remove the isinstance(cost, dict) guard from Usage.__init__ — ResponseAPIUsage.parse_cost
validator already converts Perplexity's cost dict to float before it reaches Usage.
Add test_streaming_cost_dict_to_float_via_validator to verify the dict→float conversion
works end-to-end through transform_streaming_response (pydantic recursive construction).
- Remove Perplexity-specific `models` field from ResponsesAPIOptionalRequestParams
TypedDict to avoid polluting the shared OpenAI type. The param is still passed
through via get_supported_openai_params + map_openai_params at runtime.
- Add transform_response_api_response override to catch Perplexity's HTTP 200
with status:"failed" and raise BaseLLMException instead of silently succeeding.
- Update test_all_supported_params_declared to match the simplified
8-param list (was 24 params from old 410-line implementation)
- Replace test_cost_transformation tests: use ResponseAPIUsage
validator directly instead of removed _transform_usage method
(cost parsing moved to generic parse_cost validator)
- Add 'models' field to ResponsesAPIOptionalRequestParams TypedDict so
the Perplexity fallback chain param is not silently filtered out by
get_requested_response_api_optional_param(). Verified with curl that
Perplexity's models param works (tries models in order until one
succeeds).
- Fix test_python_class_takes_priority_over_json: save and restore the
original JSONProviderRegistry entry instead of deleting it, preventing
state corruption for subsequent tests.
Resolve conflict in perplexity/responses/transformation.py by keeping
the simplified ~50 line version (PR's goal) instead of main's ~410 line
version. Added supports_native_websocket() -> False from main.
Keep unified _FINISH_REASON_MAP dict approach, discard upstream's
inconsistent _VALID_OPENAI_FINISH_REASONS frozenset that mapped to
values not in the OpenAIChatCompletionFinishReason Literal.
Gemini 2.0+ natively accepts JSON Schema in tool parameters, including
bare {} (TYPE_UNSPECIFIED), anyOf with null, and lowercase types. The
existing _build_vertex_schema pipeline was coercing {} to {"type": "object"},
breaking JsonValue/Any field semantics (issue #22391).
Add _build_vertex_schema_for_gemini_2() that only resolves $ref (which
Gemini doesn't support in tools) and filters unsupported fields. Use it
for Gemini 2.0+ models, keeping the full transform for Gemini 1.5.
