Commit graph

89 commits

Author SHA1 Message Date
YutaSaito
bb7aad9de1
Merge pull request #19319 from BerriAI/litellm_test_mcp_integration
[test] mcp integration test
2026-01-19 14:38:02 +09:00
Yuta Saito
44a166a792 fix: ci mcp version up 2026-01-19 14:27:00 +09:00
Yuta Saito
a141aa6026 test: temporary skip 2026-01-19 13:57:40 +09:00
Yuta Saito
1fbbe0a983 test: restore global MCP server manager after access-group test 2026-01-19 12:29:37 +09:00
Yuta Saito
30c4a38179 test: const 2026-01-19 12:03:26 +09:00
Yuta Saito
20b6468222 test: refactor 2026-01-19 11:17:44 +09:00
Yuta Saito
c2b5e9c669 test: MCP E2E streamable_http 2026-01-19 11:12:36 +09:00
Yuta Saito
737fec600f test: add mcp e2e test 2026-01-19 10:49:39 +09:00
Yuta Saito
d31c609600 test: Let MCP tool-execution mock accept new kwargs for streaming tests 2026-01-19 07:00:14 +09:00
Yuta Saito
4ad78236ab test: Fail MCP streaming test when LiteLLM logs errors during follow-up calls 2026-01-19 06:46:39 +09:00
Yuta Saito
cd19039e39 test: Parametrize MCP streaming test to cover OpenAI and Anthropic models 2026-01-19 06:22:05 +09:00
Yuta Saito
1c2942d808 test: add mcp completions test 2026-01-15 15:47:45 +09:00
Yuta Saito
df37770a70 test: add permission test 2026-01-14 07:58:21 +09:00
Yuta Saito
35f9a75d55 feat: add UI support for configuring meta URLs 2026-01-02 15:07:37 +09:00
Yuta Saito
34b500c7f5 feat: support MCP stdio header env overrides 2025-12-22 12:58:31 +09:00
Yuta Saito
ed5cbdac2f feat: add support for using MCPs on /chat/completions 2025-12-10 10:55:33 +09:00
YutaSaito
b5133c4c7d
Feat/mcp preserve tool metadata calltoolresult (#17561)
* feat(mcp): preserve tool metadata and full CallToolResult in MCP gateway

This PR fixes two issues that prevented ChatGPT from rendering MCP UI widgets
when proxied through LiteLLM:

1. Preserve Tool Metadata in tools/list
   - Modified _create_prefixed_tools() to mutate tools in place instead of
     reconstructing them, preserving all fields including metadata/_meta
   - This ensures ChatGPT can see 'openai/outputTemplate' URIs in tools/list
     and will call resources/read to fetch widgets

2. Preserve Full CallToolResult (structuredContent + metadata)
   - Changed call_mcp_tool() and _handle_managed_mcp_tool() to return full
     CallToolResult objects instead of just content
   - Updated error handlers to return CallToolResult with isError flag
   - Wrapped local tool results in CallToolResult objects
   - This preserves structuredContent and metadata fields needed for widget rendering

Files changed:
- litellm/proxy/_experimental/mcp_server/mcp_server_manager.py
- litellm/proxy/_experimental/mcp_server/server.py

Fixes issues where ChatGPT could not render MCP UI widgets when using
LiteLLM as an MCP gateway.

* feat(mcp): Preserve tool metadata and return full CallToolResult for ChatGPT UI widgets

- Preserve metadata and _meta fields when creating prefixed tools
- Return full CallToolResult instead of just content list
- Ensures ChatGPT can discover and render UI widgets via openai/outputTemplate
- Fixes metadata stripping that prevented widget rendering in ChatGPT

Changes:
- mcp_server_manager.py: Mutate tools in place to preserve all fields including metadata
- server.py: Return CallToolResult with structuredContent and metadata preserved
- Added test to verify metadata preservation

* fix: guard cost calculator when BaseModel lacks _hidden_params

---------

Co-authored-by: Afroz Ahmad <aahmad@Afrozs-MacBook-Pro.local>
Co-authored-by: Afroz Ahmad <aahmad@KNDMCPTMZH3.sephoraus.com>
2025-12-05 16:15:22 -08:00
Ishaan Jaffer
6530749c89 test fixes 2025-11-26 11:42:12 -08:00
YutaSaito
b72b49757e
feat: add backend support for OAuth2 auth_type registration via UI (#17006) 2025-11-23 21:52:18 -08:00
YutaSaito
f0b10b854b
chore: remove unused MCP_PROTOCOL_VERSION_HEADER_NAME constant (#17008) 2025-11-23 21:51:11 -08:00
Ishaan Jaffer
0c28af8705 test MCP server 2025-11-22 10:02:15 -08:00
Ishaan Jaffer
fc0eac2d10 test_get_tools_from_mcp_servers 2025-11-22 10:02:15 -08:00
YutaSaito
93affcb732
[Feat] mcp resources support (#16800)
* feat: mcp prompts support

* feat: mcp resources support
2025-11-20 14:53:44 -08:00
YutaSaito
0b586d26fc
refactor: drop MCPClient.connect and use run_with_session lifecycle (#16696)
Surface detailed connection errors by handling HTTP failures
2025-11-15 17:54:27 -08:00
yuneng-jiang
00ecb99f3e
[Fix] UI MCP Tool Test Regression (#16695)
* Fix UI MCP testing tool regression

* Fixed linting
2025-11-15 15:31:04 -08:00
YutaSaito
f487f4e3a9
feat: add dynamic OAuth2 metadata discovery for MCP servers (#16676)
* feat: add dynamic OAuth2 metadata discovery for MCP servers

* fix: lint error
2025-11-14 18:14:43 -08:00
YutaSaito
8e0b66a814
fix: exclude unauthorized MCP servers from allowed server list (#16551)
* fix: exclude unauthorized MCP servers from allowed server list

* fix: test after resolving merge conflicts
2025-11-13 12:33:54 -08:00
YutaSaito
2843dab7fe
fix: allow tool call even when server name prefix is missing (#16425)
* fix: allow tool call even when server name prefix is missing

* fix: test

* fix: test

* fix: test
2025-11-12 13:50:52 -08:00
YutaSaito
6eb74bd62a
Feat/persist mcp credentials in db (#16308)
* feat: persist mcp credentials in db

* feat: remove Auth Value field from MCP Tool Testing Playground

* fix: test
2025-11-07 19:22:49 -08:00
YutaSaito
8e27b6c0b4
[MCP] configure static mcp header (#16179)
* feat: configure extra mcp headers in ui

* doc: static header

* build: add new migration file

* chore: add missing image file

* fix: test
2025-11-03 21:06:36 -08:00
YutaSaito
c0890e7d33
[Feat] add support for dynamic client registration (#15921) (enables Atlassian MCP to work via Oauth on LiteLLM)
* feat: add support for dynamic client registration #13856

* fix: test

* feat: return 401 when oauth2_header is missing for OAuth2-based MCP servers
2025-10-26 10:13:46 -07:00
Vinod Singh
d4aadda692
Auth Header Fix for MCP Tool Call (#15736)
* fixed the Auth header for MCP Tool Call

* Final fix for Auth header

* testcase for mcp_auth_header_extraction, insensitive_alias_matching, insensitive_servername_matching added
2025-10-21 13:58:03 -07:00
Yuta Saito
5359a0d6a6 fix: test 2025-09-30 07:25:32 +09:00
Yuta Saito
be5010a675 test: fix tests 2025-09-29 07:21:32 +09:00
Krrish Dholakia
9403efa511 test: fix tests 2025-09-28 08:54:15 -07:00
Krrish Dholakia
1256dd53d9 test: update tests 2025-09-28 08:53:11 -07:00
Krrish Dholakia
340262c85f feat(server.py): only show allowed mcp tools 2025-09-27 20:57:12 -07:00
Krrish Dholakia
5d6889dc92 test: update tests 2025-09-27 10:40:06 -07:00
Ishaan Jaffer
a6795a6560 test fix 2025-09-19 18:05:36 -07:00
Ishaan Jaffer
dec15a80a2 test: mcp test fix 2025-09-19 17:52:32 -07:00
Ishaan Jaffer
c3f150b13d mcp test fix 2025-09-19 17:41:15 -07:00
Yuta Saito
654f1d3290 fix: stop including spec_version in MCP server registration inserts 2025-09-19 07:06:15 +09:00
Yuta Saito
6c291093e9 fix: remove adding Mcp-Protocol-Version header (#14069)
The Mcp-Protocol-Version header is already handled in the MCP Python SDK, so the explicit addition on LiteLLM Proxy was redundant.
2025-09-19 07:05:20 +09:00
Ishaan Jaff
a13aa4740a
[Fixes] Bug fixes to using LiteLLM MCP Gateway (#14392)
* fix: use _get_mcp_servers_in_path

* fix checks for using litellm_proxy as MCP tool provider

* fix: fix mcp_tools_with_litellm_proxy

* fix: fix aresponses_api_with_mcp

* aresponses_api_with_mcp

* test_mcp_allowed_tools_filtering

* fix: _filter_mcp_tools_by_allowed_tools

* fix: _filter_mcp_tools_by_allowed_tools

* test_streaming_responses_api_with_mcp_tools

* fixes: test tools transfrom MCP->OpenaI spec

* test_streaming_responses_api_with_mcp_tools

* fix: chat ui allow multi select with allowed tools

* fix: use correct MCP events with litellm proxy response API

* fix get_event_model_class

* fix litellm proxy MCP handler

* fix MCPEnhancedStreamingIterator

* chat ui show list tools result

* UI: show MCP events

* fix stream iterator

* fixes: litellm proxy mcp handler

* test responses + mcp

* fix: update responses api with mcp handling

* ruff check fix

* central: _process_mcp_tools_to_openai_format

* fix: refactor code

* test_mcp_allowed_tools_filtering

* test mcp with litellm proxy

* fix mcp call

* demo: video using MCP ui

* fixes for using stream iterator

* test_no_duplicate_mcp_tools_in_streaming_e2e

* docs fix

* fix code snippet
2025-09-10 19:12:11 -07:00
Ishaan Jaff
1f42e41c8d
[Bug]: Fix Authorization header not being sent to configured MCP servers (#14422)
* test: test_mcp_server_config_auth_value_header_used

* fix: authentication_token

* docs: fix instructions on using responses api with MCPs

* mcp fixes
2025-09-10 16:41:08 -07:00
Ishaan Jaff
68d250973c
[Feature]: Support "Token" MCPAuth (#14359)
* add MCPAuth.authorization type

* test: Token Auth MCP

* fix: _get_auth_headers to handle auth_type == MCPAuth.authorization

* docs auth types with MCP on config.yaml

* test_mcp_server_works_without_config_auth_value
2025-09-08 18:25:41 -07:00
Ishaan Jaff
b9132968b2
[Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905)
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f4.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
2025-08-23 13:13:23 -07:00
Jugal D. Bhatt
b8fe5f7b17
[MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e390.
2025-08-14 16:32:18 -07:00
Jugal Bhatt
f487816b9f [fix] Enhance MCPServerManager with access groups and description support
* Added access_groups and description fields to MCPServerManager for better server configuration.
* Implemented tests to verify integration of config-based servers with database servers, ensuring correct handling of access_groups and description.
* Updated add_update_server method to accommodate new fields and validate server addition in the registry.
2025-08-12 14:22:40 -07:00
Thiago Salvatore
c2ad858c83
fix(access group): allow access group on mcp tool retrieval (#13425)
* fix(access group): allow access group on mcp tool retrieval

* fix(test): fix broken tests and add test case for access group

* fix(mypy): fix typing issues
2025-08-08 08:55:46 -07:00