* fix(anthropic): enforce type:'object' on tool input schemas
Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.
The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.
Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.
Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.
Related issues: #12020, #64, #1671
* fix(anthropic): deduplicate tool_result messages by tool_call_id
Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.
This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().
Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).
Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.
Related issues: #11804, #11029, #6836, #1782, #151
* fix: shallow copy input_schema to avoid caller mutation + add mutation guard test
Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call
* feat: add qwen3.5 series for openrouter
* fix: typo on max_output_tokens and max_tokens from qwen3.5 series
* chore: fix
* chore: fix
* [Test] UI - Logs: Add unit tests for 5 untested view_logs components
Add vitest tests for TypeBadges, ErrorViewer, ConfigInfoMessage, TimeCell, and TruncatedValue covering rendering, user interactions, and edge cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(chat-ui): responses API + MCP tool execution in /chat (#23297)
* feat(ui): add Chat UI v0 — standalone LiteLLM-branded chat window
Adds a full chat UI accessible from the sidebar Chat link (opens in new tab).
- Standalone route at /chat (outside dashboard layout — no Navbar/Sidebar chrome)
- Claude.ai-style layout: model selector top-left, LiteLLM logo center, settings top-right
- Greeting with time-of-day, centered input card, suggestion chips (Write/Learn/Code/Brainstorm)
- Sliding conversation history sidebar with Cmd+K search, rename, delete, date grouping
- localStorage-backed conversation persistence (litellm_chat_history_v1)
- Streaming completions via makeOpenAIChatCompletionRequest with AbortController stop support
- MCP server picker (toggle servers on/off per conversation)
- LiteLLM aesthetic: white/light-gray background, Ant Design blue (#1677ff) primary, system font
- Sidebar2: Chat menu item opens in new tab via window.open
* feat(chat-ui): responses API + MCP tool execution display
- Switch /chat from chat completions to responses API (previous_response_id session chaining)
- Add MCP server picker with search filter in chat input bar
- Show MCP tool call events (list_tools + call_tool) inline in chat via MCPEventsDisplay
- Add tool chip strip showing available tools when MCP servers are selected
- Non-blocking MCP toggle: server added immediately, verification in background (works for no-auth MCPs like deepwiki)
- Add truncateAfterMessage to useChatHistory for edit/retry
- Sync activeConversationId on URL change (fixes stale conversation on new chat)
- Add "Open Chat" shortcut button to sidebar
* fix(chat-ui): switch to responses API, remove dead code, add tests
- Switch handleSend from makeOpenAIChatCompletionRequest to makeOpenAIResponsesRequest with previous_response_id session chaining
- Add responsesSessionId state; reset to null when starting a new conversation
- Remove unused ChatInputBar.tsx and ModelSelector.tsx (dead code)
- Add tests/test_litellm/test_chat_ui_responses_session.py covering previous_response_id forwarding and signature validation
* fix(chat-ui): address greptile review issues
- Reset responsesSessionId when activeConversationId changes (not just on new conversation)
- Wire onMCPEvent callback into makeOpenAIResponsesRequest; render MCPEventsDisplay below messages
- Clear mcpEvents on each new send
- Explicitly filter history to user/assistant roles only (no tool-role casting)
- Remove duplicate "Chat" menu item from sidebar (pinned button serves same purpose)
- Make Sider a flex column so "Open Chat" button actually pins to bottom
- Fix tests to intercept real HTTP requests and assert previous_response_id in body
* fix(chat-ui): address greptile review feedback (greploop iteration 1)
- Fix duplicate context: when responsesSessionId is set, only send the
new user message as input (prior context is already server-side via
session chaining). Full history is still sent on the first turn.
- Fix ephemeral MCP events: store events per-message in ChatMessage.mcpEvents
instead of ephemeral component state. Events now survive across turns
and render inline below each assistant response via MCPEventsDisplay.
- Remove stale mcpEvents useState and ephemeral panel at bottom of chat.
* fix(chat-ui): address greptile review feedback (greploop iteration 2)
- Fix stale session on edit/retry: derive previousResponseId as null when
historyOverride is set so edit/retry always starts a fresh Responses API
session rather than chaining off a now-invalid prior session
- Fix unsafe MCPEvent cast: import MCPEvent directly from MCPEventsDisplay
into types.ts and type ChatMessage.mcpEvents as MCPEvent[], eliminating
the bare 'as MCPEvent[]' cast in ChatMessages.tsx
* fix(chat-ui): fix MCPEvent layering, batch localStorage writes, module-level test imports
- Move MCPEvent interface definition into chat/types.ts (single source of truth)
- MCPEventsDisplay.tsx now imports MCPEvent from types.ts instead of defining it locally
- Batch MCP event localStorage writes: accumulate during stream, persist once in finally
- Move test imports to module level per PEP 8 convention
* fix(chat-ui): fix MCPEvent import path and rename truncateFromMessage
- responses_api.tsx now imports MCPEvent directly from chat/types (not via MCPEventsDisplay re-export)
- Remove the now-unnecessary MCPEvent re-export from MCPEventsDisplay.tsx
- Rename truncateAfterMessage → truncateFromMessage: the function removes the target message and all subsequent ones (not just what comes after), so the new name accurately describes the behavior
* fix(responses-api): fix whitespace token filter and MCP server URL construction
- Drop the delta.trim() whitespace filter that was silently swallowing spaces
and newlines during streaming, causing words to concatenate and paragraphs
to collapse. Only skip truly empty strings (delta.length > 0).
- Use proxyBaseUrl for MCP server_url construction instead of the hardcoded
relative path "litellm_proxy/mcp", so non-root deployments route correctly.
* fix(responses-api): use unique server_label per MCP server to prevent tool routing collisions
* fix(chat-ui): move MCPEvent to shared mcp_tools/types, skip partial events on abort
- Move MCPEvent interface to mcp_tools/types.tsx (shared with MCPServer/MCPTool),
eliminating the playground→chat cross-module dependency. chat/types.ts and
both playground components now import from mcp_tools/types.
- Only persist accumulated MCP events when the stream completes cleanly; aborted
or errored turns drop partial events to avoid showing incomplete tool calls.
* fix(responses-api): use server_name for MCP URL routing, fix test path
- Use server_name (not alias) as the URL path segment for MCP server_url;
alias is a display name that may differ from the registered proxy route.
URL-encode the path to handle names with spaces/special characters.
- Fix sys.path.insert in tests to use __file__-relative path so tests pass
regardless of which directory pytest is invoked from.
* fix(chat-ui): fix stale session after failed edit, clean MCP event persistence, unique server_label
- Eagerly call setResponsesSessionId(null) when historyOverride is set so a
failed/aborted edit does not leave a stale session contaminating the next turn
- Replace abort-signal check with streamCompletedCleanly flag to correctly skip
MCP event persistence on both abort and non-abort errors (network/API failures)
- Use server_name (unique) as server_label instead of alias to prevent silent
tool-routing failures when two MCP servers share the same display name
* [Feat] UI - Show logos on MCP Apps page (#23320)
* feat(ui): add MCP server logo support across admin and chat UIs
- New MCPLogoSelector component with grid of well-known logos (GitHub,
Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering
- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
including /ui/chat/ (absolute paths resolve correctly everywhere)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(codeql): remove ruby from language matrix (#23227)
* Add team-scoped MCP server filtering for key creation and fix UnboundLocalError
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix cross-team MCP server info disclosure and restricted key bypass
The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix mcp_tool_permissions JSON string deserialization in _resolve_team_allowed_mcp_servers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Feature] UI - MCP Servers: Add per-server health recheck
Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix Anthropic docs link for beta endpoint
Update the Anthropic /v1/messages beta endpoint docstring to point to
its current pass-through documentation.
This keeps the change scoped to the incorrect URL and avoids changing
unverified wording in the surrounding comment.
---------
Co-authored-by: netbrah <162479981+netbrah@users.noreply.github.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
* fix(codeql): switch to security-extended query suite
The security-and-quality suite produces result sets > 2 GiB on this
codebase, causing fatal OOM failures and blocking CI. Switching to
security-extended reduces query scope to security-only checks, which
still complete successfully. Quality/maintainability checks are
already covered by the existing lint pipeline.
* fix(codeql): exclude OOM queries from security-extended
Cast Polars Decimal columns to Float64 before calling .to_dicts() in
vantage_dry_run_export so the response contains JSON-serializable float
values instead of decimal.Decimal objects that FastAPI cannot encode.
Also cast summary totals to float for the same reason.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DELETE /vantage/delete now removes the in-memory VantageLogger from
litellm.callbacks via remove_callbacks_by_type, preventing the
scheduler from continuing to fire exports with stale credentials
- Add test_should_cast_decimal_columns_to_float covering the
Decimal→Float64 cast in FocusCsvSerializer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Claude Opus 4.6, Sonnet 4.6, Opus 4.5, Sonnet 4.5, and Haiku 4.5
to the web fetch supported models documentation. These models were
missing from the list despite supporting the web_fetch tool.
- Use timedelta(hours=1) instead of replace(hour=hour+1) in _window()
to avoid ValueError when hour=23
- Add test_should_batch_by_row_count covering the >10K rows batching
path (previously only the 2 MB size-limit path was tested)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matches the existing validator on VantageInitRequest so that empty or
whitespace-only api_key/integration_token values are rejected at update
time rather than silently persisted and failing at the next export.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Override initialize_focus_export_job in VantageLogger to use
VANTAGE_USAGE_DATA_JOB_NAME as the Redis pod lock key, preventing
silent export skips when both Focus and Vantage loggers are configured
- Add field_validator to VantageInitRequest rejecting empty-string
api_key and integration_token at init time instead of at export time
- Guard FocusLogger FOCUS_INTERVAL_SECONDS with try/except matching
the pattern already used in VantageLogger
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use `type(cb) is FocusLogger` instead of `isinstance(cb, FocusLogger)`
in both _init_custom_logger_compatible_class and
get_custom_logger_compatible_class so that a VantageLogger already in
_in_memory_loggers is not incorrectly returned for a "focus" lookup.
This ensures users with both "vantage" and "focus" in success_callbacks
get separate loggers for each export destination.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes issue where multipart file uploads through passthrough endpoints failed with RequestValidationError. The proxy was consuming the request body stream and FastAPI was trying to parse multipart bodies as JSON dicts.
Changes:
- Try JSON parsing first for multipart content-type (handles misconfigured clients)
- Skip multipart parsing if JSON succeeds to avoid stream consumption
- Remove custom_body parameter from endpoint_func to prevent FastAPI auto-parsing
- Check for parsed body before using multipart handler
- Add regression test for multipart boundary preservation
Handles both actual multipart uploads and JSON bodies with incorrect multipart content-type headers.
Made-with: Cursor
- Guard FocusLogger.init_focus_export_background_job with exact type
check (type(cb) is FocusLogger) to exclude VantageLogger subclass,
preventing duplicate hourly exports when VantageLogger is registered
programmatically before startup
- Cast pl.Decimal columns to Float64 in FocusCsvSerializer so CSV
output uses standard floating-point notation instead of fixed-point
strings that Vantage's parser may reject
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Align dry-run summary to use pre-transform columns (spend, total_tokens,
team_id, model) matching FocusExportEngine internals
- Reduce token exposure in debug logs to first 4 chars
- Wrap raise_for_status in try/except to log httpx errors before re-raising
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Split VantageExportRequest (limit=None) and VantageDryRunRequest
(limit=500) so actual exports don't silently truncate large datasets
- Add try/except around each sub-batch upload in _upload_size_limited,
consistent with _upload_batched's continue-on-failure guarantee
- Guard VANTAGE_EXPORT_INTERVAL_SECONDS against non-numeric values
with try/except instead of bare int() cast
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vantage destination:
- Skip individual CSV rows exceeding 2MB limit with a warning instead of
uploading an oversized batch that Vantage would reject
- Wrap each batch upload in try/except so remaining batches continue on
failure; re-raise the first error after all batches are attempted
Callback registration:
- Use litellm.logging_callback_manager.add_litellm_callback() instead of
raw litellm.callbacks.append() for DB-bootstrapped VantageLogger, ensuring
proper dedup and manager visibility
- Add "vantage" init handler in litellm_logging.py (both creation and
lookup branches) so config.yaml string callbacks are properly resolved
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Polars write_csv already handles empty DataFrames correctly (outputs
header-only CSV), so the conditional branch was a no-op.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add missing except HTTPException: raise in vantage_dry_run_export
(consistent with all other endpoints in the file)
- Fix is_vantage_setup_in_config() to detect both the string "vantage"
and VantageLogger instances in litellm.callbacks, preventing duplicate
logger registration on startup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add missing `except HTTPException: raise` guard so intentional 404
responses (e.g. when settings are not configured) are not caught by the
generic Exception handler and re-raised as 500 errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Revert FocusExportEngine.dry_run_export_usage_data to use original raw
column names (spend, total_tokens, team_id, model) preserving backward
compatibility for existing callers
- Vantage dry-run endpoint computes its own summary from FOCUS columns
independently, avoiding coupling to the engine method
- Set VantageExportRequest.limit default to 500 (was None) matching docstring
- Add empty-settings guard in /vantage/export returning 404 instead of
deferring ValueError to runtime
- Fix misleading docstring in _build_tags_expr about Rust-level execution
- Clarify Tags schema comment: parquet is self-describing so existing
exports are unaffected
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Mask integration_token in GET /vantage/settings response (renamed field to integration_token_masked)
- Reuse single httpx.AsyncClient across all batch uploads in deliver()
- Align FocusExportEngine.dry_run_export_usage_data to use post-transform FOCUS columns (BilledCost, SubAccountId, ResourceType) matching the Vantage dry-run endpoint
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Enforce 10K row limit in single-shot upload path (not just 2MB size)
- Fix KeyError crash in update_vantage_settings when no settings exist
- Remove unreachable status_code==400 dead code branch
- Make dry-run endpoint work without Vantage credentials by using
FOCUS database + transformer directly instead of VantageLogger
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a pluggable Vantage destination to the existing FOCUS export pipeline,
enabling LiteLLM to export spend data in FOCUS format directly to Vantage's
cost-import API. Supports automatic hourly exports via scheduled background job,
with admin API endpoints for manual control and configuration. Includes CSV
serializer, batching for 10K row / 2MB API limits, and enriched Tags JSON with
team/user/key metadata for Vantage Token Allocation feature.
- Add CSV serializer (FocusCsvSerializer) for FOCUS data
- Add Vantage API destination with automatic batching
- Add VantageLogger that wraps FocusLogger with Vantage defaults
- Add proxy endpoints: /vantage/{init,settings,export,dry-run,delete}
- Register "vantage" callback in logger registry and literal type
- Wire up background job in proxy_server.py startup
- Populate Tags column with JSON metadata (team_id, user_id, user_email, etc.)
- Add 14 unit tests covering serializer, destination, and factory
All tests pass (23 focus tests total, no regressions).
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
