Commit graph

1215 commits

Author SHA1 Message Date
Krish Dholakia
ca4329aeb9
Root cause fix - migrate all logging update to use 1 function - for centralized kwarg updates (#23659)
* fix: Fixes https://github.com/BerriAI/litellm/issues/23185

* fix(responses/main.py): ensure litellm metadata custom cost works

* refactor: move all logging updates to a common function, to have just 1 place to update logging kwarg updates
2026-03-15 23:21:01 -07:00
yuneng-jiang
82fc819abf Merge remote-tracking branch 'origin' into litellm_internal_dev_03_14_2026 2026-03-14 18:35:03 -07:00
yuneng-jiang
460f6208c5 Fix __main__ exit bug, close litellm_async_client, restore global state
- Fix __main__ block: test returns None now, so always exited 1
- Close litellm_async_client in test_ssl_verification_with_aiohttp_transport
- Save/restore litellm.force_ipv4 and litellm.disable_aiohttp_transport
  in test_force_ipv4_transport and test_aiohttp_disabled_transport

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:14:42 -07:00
yuneng-jiang
062cd4a2a1 Remove gc.get_objects() from test_gemini_session_leak.py
count_aiohttp_sessions() iterates every object in the Python GC,
which hangs in CI when xdist workers have millions of loaded objects.
Replace with direct session.closed checks — same coverage, no hang.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:00:26 -07:00
yuneng-jiang
b14eadef86 Fix hanging CI tests in test_http_handler.py
- Remove real HTTP call to example.com in test_force_ipv4_transport
  (hangs in CI when network is slow/unavailable)
- Close leaked aiohttp.ClientSession in test_ssl_verification_with_aiohttp_transport
- Add cleanup for transports in test_aiohttp_transport_trust_env_setting
- Add cleanup for handler in test_ssl_security_level
- Add cleanup for transport in test_ssl_context_transport

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 17:40:33 -07:00
ryanh-ai
374c3458d5
feat: add sagemaker_nova provider for Amazon Nova models on SageMaker (#21542)
* feat: add sagemaker_nova provider for Nova models on SageMaker

Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite,
Nova 2 Lite) deployed on SageMaker Inference real-time endpoints.

Nova uses OpenAI-compatible request/response format with additional
Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids,
truncate_prompt_tokens) and requires stream:true in the request body.
Nova endpoints also reject 'model' in the request body.

Changes:
- New provider: sagemaker_nova/<endpoint-name>
- SagemakerNovaConfig inherits from SagemakerChatConfig
- Override transform_request to strip 'model' from request body
- Override supports_stream_param_in_request_body (True for Nova)
- Extend get_supported_openai_params with Nova-specific params
- Refactored SagemakerChatConfig to use custom_llm_provider param
  instead of hardcoded strings (backwards-compatible)
- Consolidated main.py routing for sagemaker_chat and sagemaker_nova
- 22 unit tests + 9 integration tests (skip-gated)
- Documentation with SDK, streaming, multimodal, and proxy examples
- All tests verified against live SageMaker Nova endpoint

* fix: move integration tests to tests/local_testing/ per test directory policy

* fix: remove unused module-level SagemakerNovaConfig instance

The sagemaker_nova_config singleton was never imported or used — the
ProviderConfigManager creates its own instance via the lambda registered
in utils.py. Removing this leftover boilerplate.

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-14 15:10:01 -07:00
yuneng-jiang
94eabffe01
Merge pull request #22390 from Harshit28j/litellm_langfuse-session-trace-fix
Fix Langfuse failure path kwargs inconsistency
2026-03-14 12:49:04 -07:00
yuneng-jiang
0066ad7291
Merge pull request #23649 from Harshit28j/litellm_tests_leak
add tests for fix
2026-03-14 12:32:39 -07:00
Krish Dholakia
8abf2d8e34
fix: Fixes https://github.com/BerriAI/litellm/issues/23185 (#23647) 2026-03-14 12:30:52 -07:00
Ishaan Jaff
b87d1f8dad
[Feat] - Ishaan main merge branch (#23596)
* fix(bedrock): respect s3_region_name for batch file uploads (#23569)

* fix(bedrock): respect s3_region_name for batch file uploads (GovCloud fix)

* fix: s3_region_name always wins over aws_region_name for S3 signing (Greptile feedback)

* fix: _filter_headers_for_aws_signature - Bedrock KB (#23571)

* fix: _filter_headers_for_aws_signature

* fix: filter None header values in all post-signing re-merge paths

Addresses Greptile feedback: None-valued headers were being filtered
during SigV4 signing but re-merged back into the final headers dict
afterward, which would cause downstream HTTP client failures.

Made-with: Cursor

* feat(router): tag_regex routing — route by User-Agent regex without per-developer tag config (#23594)

* feat(router): add tag_regex support for header-based routing

Adds a new `tag_regex` field to litellm_params that lets operators route
requests based on regex patterns matched against request headers — primarily
User-Agent — without requiring per-developer tag configuration.

Use case: route all Claude Code traffic (User-Agent: claude-code/x.y.z) to
a dedicated deployment by setting:

  tag_regex:
    - "^User-Agent: claude-code\\/"

in the deployment's litellm_params. Works alongside existing `tags` routing;
exact tag match takes precedence over regex match. Unmatched requests fall
through to deployments tagged `default`.

The matched deployment, pattern, and user_agent are recorded in
`metadata["tag_routing"]` so they flow through to SpendLogs automatically.

* fix(tag_regex): address backwards-compat, metadata overwrite, and warning noise

Three issues from code review:

1. Backwards-compat: `has_tag_filter` was widened to activate on any non-empty
   User-Agent, which would raise ValueError for existing deployments using plain
   tags without a `default` fallback. Fix: only activate header-based regex
   filtering when at least one candidate deployment has `tag_regex` configured.

2. Metadata overwrite: `metadata["tag_routing"]` was overwritten for every
   matching deployment in the loop, leaving inaccurate provenance when multiple
   deployments match. Fix: write only for the first match.

3. Warning noise: an invalid regex pattern logged one warning per header string
   rather than once per pattern. Fix: compile first (catching re.error once),
   then iterate over header strings.

Also adds two new tests covering these cases, and adds docs page for
tag_regex routing with a Claude Code walk-through.

* refactor(tag_regex): remove unnecessary _healthy_list copy

* docs: merge tag_regex section into tag_routing.md, remove standalone page

- Add ## Regex-based tag routing (tag_regex) section to existing
  tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
  page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
  routing fires under strict mode (regex always uses OR semantics)

* fix(tag_regex): address greptile review - security docs, strict-mode enforcement, validation order

- Strengthen security note in tag_routing.md: explicitly state User-Agent
  is client-supplied and can be set to any value; frame tag_regex as a
  traffic classification hint, not an access-control mechanism
- Move tag_regex startup validation before _add_deployment() so an invalid
  pattern never leaves partial router state
- Enforce match_any=False strict-tag policy: when a deployment has both
  tags and tag_regex and the strict tag check fails, skip the regex fallback
  rather than silently bypassing the operator's intent
- Extract per-deployment match logic into _match_deployment() helper to
  keep get_deployments_for_tag() readable
- Add two new tests: strict-mode blocks regex fallback, regex-only
  deployment still matches under match_any=False

* fix(ci): apply Black formatting to 14 files and stabilize flaky caplog tests

- Run Black formatter on 14 files that were failing the lint check
- Replace caplog-based assertions in TestAliasConflicts with
  unittest.mock.patch on verbose_logger.warning for xdist compatibility
- The caplog fixture can produce empty text in pytest-xdist workers
  in certain CI environments, causing flaky test failures

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-14 09:40:00 -07:00
yuneng-jiang
36c076d56c
Merge pull request #23550 from Sameerlite/litellm_azure-model-router-cost-breakdown
feat(azure): Azure Model Router cost breakdown in UI + additional_costs from hidden_params
2026-03-14 09:39:04 -07:00
yuneng-jiang
dd1a3e15e1
Merge pull request #23526 from Sameerlite/litellm_anthropic-guardrail-tools
fix(anthropic): preserve native tool format when guardrails convert tools for Anthropic Messages API
2026-03-14 09:38:43 -07:00
Pradyumna Yadav
a94b961c18
fix: auto-fill reasoning_content for moonshot kimi reasoning models in multi-turn tool calling (#23580) 2026-03-13 22:48:45 -07:00
Awais Qureshi
c7ba7948bc
PR #22867 added _remove_scope_from_cache_control for Bedrock and Azur… (#23183)
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."

* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."

* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
 but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
  inherit it, and removed the now-redundant copy from the Azure AI subclass.

* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
 but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
  inherit it, and removed the now-redundant copy from the Azure AI subclass.

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-13 22:41:25 -07:00
Sameer Kankute
7dce61ce48 fix(tests): update TestGPT5ReasoningEffortPreservation for dict normalization
Made-with: Cursor
2026-03-14 00:13:16 +05:30
Sameer Kankute
408b717cb9 Fix gpt 5 transformation tests 2026-03-14 00:00:02 +05:30
Sameer Kankute
30645d683f Reserve reasoning for responses via chat completion 2026-03-13 23:59:57 +05:30
Sameer Kankute
3596464d11 Revert "feat(openai): drop reasoning_effort for gpt-5.4 when tools present"
This reverts commit 14b52b1318.
2026-03-13 23:59:48 +05:30
yuneng-jiang
2b71b0fb25
Revert "QA: improve gpt-5.4 code/bugs" 2026-03-13 10:15:47 -07:00
yuneng-jiang
8dc198eccf
Merge pull request #23535 from Sameerlite/litellm_improve_qa-5.4
QA:  improve gpt-5.4 code/bugs
2026-03-13 09:37:30 -07:00
yuneng-jiang
61cee53200 fix(tests): fix flaky qwen global endpoint test by mocking AsyncHTTPHandler at class level
The test was creating a real AsyncHTTPHandler instance and patching its
post method, but the internal code creates its own handler, bypassing
the mock. This caused real API calls to Vertex AI, resulting in 401
auth errors in CI. Switched to patching AsyncHTTPHandler at the class
level, matching the pattern used by the passing GPT-OSS test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:07:48 -07:00
Sameer Kankute
7e662afe01 feat(azure): Azure Model Router cost breakdown in UI + additional_costs from hidden_params
- Backend: Use request model from hidden_params for Azure Model Router additional_costs when response has actual model
- Backend: Add additional_costs to total cost calculation
- UI: Show all non-null/non-zero additional_costs in CostBreakdownViewer
- UI: Render cost breakdown when only additional_costs exist
- Tests: Backend test for hidden_params flow; frontend tests for additional_costs

Made-with: Cursor
2026-03-13 18:25:29 +05:30
Sameer Kankute
9b3ffd04ef Reserve reasoning for responses via chat completion 2026-03-13 14:19:40 +05:30
Sameer Kankute
90b03f6c67 Revert "feat(openai): drop reasoning_effort for gpt-5.4 when tools present"
This reverts commit 14b52b1318.
2026-03-13 13:40:29 +05:30
Sameer Kankute
45ba9e1f7e fix(anthropic): preserve native tool format when guardrails convert tools for Anthropic Messages API
- Keep Anthropic-native tools (tool_search_tool_regex, web_search, bash, etc.) in original format when translating to OpenAI format for guardrails
- Convert guardrail-returned tools back from OpenAI to Anthropic format (type=custom for user tools)
- Add TOOL_SEARCH_TOOL to ANTHROPIC_HOSTED_TOOLS enum; use prefix matching for native tool detection
- Set type=custom explicitly when mapping OpenAI function tools to AnthropicMessagesTool
- Add test for Anthropic native tools with guardrails

Made-with: Cursor
2026-03-13 11:34:18 +05:30
yuneng-jiang
0235aafb20
Merge pull request #23500 from BerriAI/litellm_litellm-mypy-errors-28de
[Fix] MyPy Errors
2026-03-12 22:46:10 -07:00
Cursor Agent
9a356644bf
fix(tests): stabilize 3 failing CI tests
1. Add missing __init__.py files in tests/test_litellm/llms/gemini/ and
   subdirectories (realtime/, image_edit/) to fix ModuleNotFoundError
   with pytest-xdist parallel workers.

2. Update test_transform_request_uses_dynamic_max_tokens to use
   claude-3-7-sonnet-20250219 (max_output_tokens=64000) since
   claude-3-5-sonnet-20241022 was removed from model_prices JSON
   during deprecated model cleanup. The test assertion was outdated.

3. Update context caching TTL tests to use gemini-2.5-pro instead of
   gemini-1.5-pro. The old model was removed from model_prices JSON,
   causing supports_system_messages to return False, which prevented
   system_instruction from appearing in the transformation output.

Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
2026-03-13 00:26:31 +00:00
Cursor Agent
003e841737
fix(tests): update BFL image generation tests for new signature
Update transform_image_generation_response test calls to pass required
explicit params (request_data, optional_params, litellm_params, encoding)
that replaced **kwargs in the method signature.

Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
2026-03-13 00:20:52 +00:00
yuneng-jiang
d16c8c5590 [Fix] BaseModelResponseIterator crashes on non-string stream chunks
The empty-line filter in __next__/__anext__ called .strip() without
checking the type first. When the Responses API yields Pydantic
BaseModel events (e.g. ResponseCreatedEvent), this raises
AttributeError. Add an isinstance(str_line, str) guard so non-string
objects pass through to _handle_string_chunk as intended.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 16:16:32 -07:00
Cursor Agent
e242356570
fix(ci): fix ruff lint errors and 9 failing unit tests on main
Lint fixes (check_code_and_doc_quality job):
- Remove unused variable reasoning_effort in gpt_5_transformation.py (F841)
- Remove unused timezone imports in mcp_server rest_endpoints.py and server.py (F401)
- Remove unused ProxyBaseLLMRequestProcessing import in realtime endpoints.py (F401)
- Add BaseRealtimeHTTPConfig to TYPE_CHECKING block in utils.py (F821)
- Add PLR0915 per-file-ignore for mcp_server/rest_endpoints.py in ruff.toml

Test fixes (litellm_mapped_tests_llms job):
- Gemini video cost tests: pass explicit model_info to video_generation_cost()
  instead of relying on gemini/veo-3.0-generate-preview being in model_prices JSON
- Anthropic max_tokens tests: mock get_max_tokens() to return expected values
  instead of depending on claude-3-5-sonnet-20241022 being in model_prices JSON
- Vertex AI pydantic obj test: update from removed gemini-1.5-pro to gemini-2.5-flash,
  update expected request body to use response_json_schema format
- Vertex AI/Bedrock file_content integration tests: update mocks to target
  base_llm_http_handler.retrieve_file_content (the new code path via
  ProviderConfigManager) instead of the old vertex_ai_files_instance/
  bedrock_files_instance paths

Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
2026-03-12 19:58:43 +00:00
Chesars
690ad4c45b fix(openai): drop all reasoning_effort for gpt-5.4 + tools, including 'none'
OpenAI rejects any reasoning_effort (even 'none') with tools in
/v1/chat/completions for gpt-5.4. Update the guard to drop reasoning_effort
regardless of value. Add docs explaining the auto-drop behavior.
2026-03-12 16:22:40 -03:00
Cesar Garcia
e01d722803
Merge branch 'main' into litellm_oss_staging_03_11_2026 2026-03-12 13:53:14 -03:00
Sameer Kankute
d507f840d3
Merge pull request #23432 from BerriAI/litellm_azure-model-router-show-actual-model
feat(azure_ai): show actual model used in Azure Model Router response
2026-03-12 22:18:50 +05:30
Chesars
4e6e1d8de8 merge: resolve conflicts with upstream staging (bedrock + mcp tests)
Keep both sets of tests: upstream's OAuth2 token injection test and
our case-insensitive tool matching tests. Use upstream's version of
the bedrock output_config test (more comprehensive).
2026-03-12 13:40:16 -03:00
Chesars
feed274aa3 Reapply "feat: add model_cost aliases expansion support"
This reverts commit 3d2df7e8b5.
2026-03-12 13:36:57 -03:00
Cesar Garcia
6bd7cd7573
Merge branch 'main' into litellm_oss_staging_03_11_2026 2026-03-12 10:43:08 -03:00
Sameer Kankute
f5be79419c Fix test_claude_agent_sdk_streaming 2026-03-12 18:36:04 +05:30
Sameer Kankute
412a283569 Revert "fix(vertex): skip harmful schema transforms for Gemini 2.0+ tool parameters"
This reverts commit a9c3095cc5.
2026-03-12 18:26:11 +05:30
Chesars
1be6b31e2f merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
Sameer Kankute
5b83aae715 feat(azure_ai): show actual model used in Azure Model Router response
- Azure Model Router transform_response: let parent extract actual model from raw response
- common_request_processing: skip model override for Azure Model Router requests
- proxy_server: skip streaming chunk model restamp for Azure Model Router
- Add _is_azure_model_router_request helper
- Add tests for non-streaming and streaming

Made-with: Cursor
2026-03-12 11:41:19 +05:30
Alvin Tang
2b7b7d3086
fix(snowflake): transform string tool_choice to object format (#23318)
Snowflake's Cortex LLM API (like Anthropic) requires tool_choice as an
object with a "type" field, not as a bare string. Passing tool_choice="auto"
(or "required"/"none") results in error 390142 "invalid payload".

This fix transforms OpenAI string tool_choice values to the Snowflake
object format:
- "auto"     -> {"type": "auto"}
- "required" -> {"type": "any"}  (Snowflake/Anthropic convention)
- "none"     -> {"type": "none"}

The dict-to-dict transformation for specific function tool choices
({"type": "function", "function": {"name": "..."}} -> {"type": "tool",
"name": [...]}) remains unchanged.

Fixes #23284

Co-authored-by: gambletan <tan@echooo.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-11 21:29:59 -07:00
yuneng-jiang
b5dfde892c
Merge pull request #23377 from BerriAI/litellm_fix_flaky_mcp_and_agentcore_tests
[Fix] Flaky MCP Server and AgentCore Streaming Tests
2026-03-11 13:20:59 -07:00
yuneng-jiang
64d3d7626f [Fix] Flaky MCP server and AgentCore streaming tests in CI
- MCP tests: set mock_mcp_server.oauth2_flow = None to prevent MagicMock
  leaking into Pydantic Literal validation for MCPServer
- AgentCore tests: pass api_key="test-jwt-token" to bypass SigV4 credential
  lookup that fails in CI without AWS credentials

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 13:08:48 -07:00
Cesar Garcia
cb24b8b05e
Merge pull request #19104 from Chesars/fix/vertex-ai-zai-org-global-region
feat(vertex_ai): route region for partner models and add GLM support
2026-03-11 15:19:33 -03:00
Chesars
3948513a4c fix(vertex-ai): warn on region override and remove dead is_global_only_vertex_model
Add verbose_logger.warning when user-specified region is overridden by
supported_regions. Remove now-unused is_global_only_vertex_model function
and its tests since get_vertex_region handles all region logic directly.
2026-03-11 15:12:53 -03:00
Chesars
f3ceb69e9f fix(vertex-ai): override unsupported user region for models with supported_regions
- get_vertex_region now overrides user-specified region when it's not in
  the model's supported_regions list (prevents 404 for users with a
  global VERTEXAI_LOCATION default hitting global-only models)
- Add supported_regions: ["global"] to glm-5-maas in both JSON files
- Update tests to cover the override behavior
2026-03-11 14:55:51 -03:00
Chesars
689cbaa6c1 fix(vertex-ai): update tests to match new get_vertex_region model_cost lookup
- Remove redundant get_vertex_region() call in partner models main.py
  (already called inside get_complete_vertex_url)
- Rewrite test mocks to use patch.dict(litellm.model_cost) instead of
  patching the removed is_global_only_vertex_model symbol
- Align test assertions with new behavior: user-specified region is
  preserved (not overridden) for global-only models
2026-03-11 14:25:08 -03:00
Cesar Garcia
ebe75a1bc6
Merge pull request #16594 from Chesars/feat/anthropic-files-api
feat: Add Anthropic Files API support
2026-03-11 13:45:53 -03:00
Chesars
9eff611b1a feat(anthropic): add Files API support for SDK
Implement Anthropic Files API (upload, retrieve, list, delete, content)
using the BaseFilesConfig provider pattern. Adds multipart form-data
support to BaseLLMHTTPHandler for file uploads.
2026-03-11 12:45:19 -03:00
Peter Dave Hello
3f18cd2fdc
[Docs] Fix "Page Not Found" link for Anthropic endpoint (#23349)
* fix(anthropic): enforce type:'object' on tool input schemas

Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.

The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.

Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.

Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.

Related issues: #12020, #64, #1671

* fix(anthropic): deduplicate tool_result messages by tool_call_id

Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.

This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().

Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).

Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.

Related issues: #11804, #11029, #6836, #1782, #151

* fix: shallow copy input_schema to avoid caller mutation + add mutation guard test

Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call

* feat: add qwen3.5 series for openrouter

* fix: typo on max_output_tokens and max_tokens from qwen3.5 series

* chore: fix

* chore: fix

* [Test] UI - Logs: Add unit tests for 5 untested view_logs components

Add vitest tests for TypeBadges, ErrorViewer, ConfigInfoMessage, TimeCell, and TruncatedValue covering rendering, user interactions, and edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* feat(chat-ui): responses API + MCP tool execution in /chat (#23297)

* feat(ui): add Chat UI v0 — standalone LiteLLM-branded chat window

Adds a full chat UI accessible from the sidebar Chat link (opens in new tab).
- Standalone route at /chat (outside dashboard layout — no Navbar/Sidebar chrome)
- Claude.ai-style layout: model selector top-left, LiteLLM logo center, settings top-right
- Greeting with time-of-day, centered input card, suggestion chips (Write/Learn/Code/Brainstorm)
- Sliding conversation history sidebar with Cmd+K search, rename, delete, date grouping
- localStorage-backed conversation persistence (litellm_chat_history_v1)
- Streaming completions via makeOpenAIChatCompletionRequest with AbortController stop support
- MCP server picker (toggle servers on/off per conversation)
- LiteLLM aesthetic: white/light-gray background, Ant Design blue (#1677ff) primary, system font
- Sidebar2: Chat menu item opens in new tab via window.open

* feat(chat-ui): responses API + MCP tool execution display

- Switch /chat from chat completions to responses API (previous_response_id session chaining)
- Add MCP server picker with search filter in chat input bar
- Show MCP tool call events (list_tools + call_tool) inline in chat via MCPEventsDisplay
- Add tool chip strip showing available tools when MCP servers are selected
- Non-blocking MCP toggle: server added immediately, verification in background (works for no-auth MCPs like deepwiki)
- Add truncateAfterMessage to useChatHistory for edit/retry
- Sync activeConversationId on URL change (fixes stale conversation on new chat)
- Add "Open Chat" shortcut button to sidebar

* fix(chat-ui): switch to responses API, remove dead code, add tests

- Switch handleSend from makeOpenAIChatCompletionRequest to makeOpenAIResponsesRequest with previous_response_id session chaining
- Add responsesSessionId state; reset to null when starting a new conversation
- Remove unused ChatInputBar.tsx and ModelSelector.tsx (dead code)
- Add tests/test_litellm/test_chat_ui_responses_session.py covering previous_response_id forwarding and signature validation

* fix(chat-ui): address greptile review issues

- Reset responsesSessionId when activeConversationId changes (not just on new conversation)
- Wire onMCPEvent callback into makeOpenAIResponsesRequest; render MCPEventsDisplay below messages
- Clear mcpEvents on each new send
- Explicitly filter history to user/assistant roles only (no tool-role casting)
- Remove duplicate "Chat" menu item from sidebar (pinned button serves same purpose)
- Make Sider a flex column so "Open Chat" button actually pins to bottom
- Fix tests to intercept real HTTP requests and assert previous_response_id in body

* fix(chat-ui): address greptile review feedback (greploop iteration 1)

- Fix duplicate context: when responsesSessionId is set, only send the
  new user message as input (prior context is already server-side via
  session chaining). Full history is still sent on the first turn.
- Fix ephemeral MCP events: store events per-message in ChatMessage.mcpEvents
  instead of ephemeral component state. Events now survive across turns
  and render inline below each assistant response via MCPEventsDisplay.
- Remove stale mcpEvents useState and ephemeral panel at bottom of chat.

* fix(chat-ui): address greptile review feedback (greploop iteration 2)

- Fix stale session on edit/retry: derive previousResponseId as null when
  historyOverride is set so edit/retry always starts a fresh Responses API
  session rather than chaining off a now-invalid prior session
- Fix unsafe MCPEvent cast: import MCPEvent directly from MCPEventsDisplay
  into types.ts and type ChatMessage.mcpEvents as MCPEvent[], eliminating
  the bare 'as MCPEvent[]' cast in ChatMessages.tsx

* fix(chat-ui): fix MCPEvent layering, batch localStorage writes, module-level test imports

- Move MCPEvent interface definition into chat/types.ts (single source of truth)
- MCPEventsDisplay.tsx now imports MCPEvent from types.ts instead of defining it locally
- Batch MCP event localStorage writes: accumulate during stream, persist once in finally
- Move test imports to module level per PEP 8 convention

* fix(chat-ui): fix MCPEvent import path and rename truncateFromMessage

- responses_api.tsx now imports MCPEvent directly from chat/types (not via MCPEventsDisplay re-export)
- Remove the now-unnecessary MCPEvent re-export from MCPEventsDisplay.tsx
- Rename truncateAfterMessage → truncateFromMessage: the function removes the target message and all subsequent ones (not just what comes after), so the new name accurately describes the behavior

* fix(responses-api): fix whitespace token filter and MCP server URL construction

- Drop the delta.trim() whitespace filter that was silently swallowing spaces
  and newlines during streaming, causing words to concatenate and paragraphs
  to collapse. Only skip truly empty strings (delta.length > 0).
- Use proxyBaseUrl for MCP server_url construction instead of the hardcoded
  relative path "litellm_proxy/mcp", so non-root deployments route correctly.

* fix(responses-api): use unique server_label per MCP server to prevent tool routing collisions

* fix(chat-ui): move MCPEvent to shared mcp_tools/types, skip partial events on abort

- Move MCPEvent interface to mcp_tools/types.tsx (shared with MCPServer/MCPTool),
  eliminating the playground→chat cross-module dependency. chat/types.ts and
  both playground components now import from mcp_tools/types.
- Only persist accumulated MCP events when the stream completes cleanly; aborted
  or errored turns drop partial events to avoid showing incomplete tool calls.

* fix(responses-api): use server_name for MCP URL routing, fix test path

- Use server_name (not alias) as the URL path segment for MCP server_url;
  alias is a display name that may differ from the registered proxy route.
  URL-encode the path to handle names with spaces/special characters.
- Fix sys.path.insert in tests to use __file__-relative path so tests pass
  regardless of which directory pytest is invoked from.

* fix(chat-ui): fix stale session after failed edit, clean MCP event persistence, unique server_label

- Eagerly call setResponsesSessionId(null) when historyOverride is set so a
  failed/aborted edit does not leave a stale session contaminating the next turn
- Replace abort-signal check with streamCompletedCleanly flag to correctly skip
  MCP event persistence on both abort and non-abort errors (network/API failures)
- Use server_name (unique) as server_label instead of alias to prevent silent
  tool-routing failures when two MCP servers share the same display name

* [Feat] UI - Show logos on MCP Apps page (#23320)

* feat(ui): add MCP server logo support across admin and chat UIs

- New MCPLogoSelector component with grid of well-known logos (GitHub,
  Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
  auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
  graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering

- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
  Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
  GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
  asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
  including /ui/chat/ (absolute paths resolve correctly everywhere)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(codeql): remove ruby from language matrix (#23227)

* Add team-scoped MCP server filtering for key creation and fix UnboundLocalError

When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix cross-team MCP server info disclosure and restricted key bypass

The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix mcp_tool_permissions JSON string deserialization in _resolve_team_allowed_mcp_servers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [Feature] UI - MCP Servers: Add per-server health recheck

Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix Anthropic docs link for beta endpoint

Update the Anthropic /v1/messages beta endpoint docstring to point to
its current pass-through documentation.

This keeps the change scoped to the incorrect URL and avoids changing
unverified wording in the surrounding comment.

---------

Co-authored-by: netbrah <162479981+netbrah@users.noreply.github.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
2026-03-11 20:17:41 +05:30