Commit graph

16 commits

Author SHA1 Message Date
yuneng-jiang
6a0d03914c
test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)
* test: drop the cwd-relative sys.path.insert calls from the test suite

TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.

Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.

Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.

* test: drop the duplicate imports the sys.path sweep exposed to F811

* test(pre-call-utils): restore the os import the new bedrock tests need
2026-08-22 09:25:58 -07:00
ryan-crabbe-berri
21e9632713
test: add six ruff rules that catch tests which cannot fail (#37709)
`assert False` inside a `try:` raises AssertionError, which the `except
Exception` right below it catches, so several tests reported green no matter
what the code did. `pytest.fail` raises Failed, a BaseException, and escapes.

A bare `a == b` statement is evaluated and discarded. Nine of those sat in
tests, and one was comparing against a model name the router never produces.

Selects B011, B015, B018, PT015, PLR0133 and PLW0127 in ruff-tests.toml
alongside F821, with all 50 existing violations fixed, so no budget file or
ratchet is needed. CI already runs this config over tests/.
2026-08-20 14:21:26 -07:00
Tin Chi Lo
cf23df9431 fix(mcp): require every reference to opt in before auto-executing tools
_should_auto_execute_tools returned True as soon as any MCP reference set
require_approval="never", so a request that mixed a "never" reference with an
"always" or "manual" one auto-executed every tool call the model produced,
including the approval-gated ones. A prompt could name the approval-required
tool and have it run with no approval.

Make the gate fail closed: auto-execute only when every reference opts in with
"never". A single approval-required reference (including the object form or an
unset value) returns the model's tool calls to the caller instead of running
them, so an approval-gated tool can never be auto-invoked. This is the shared
decision behind /chat/completions, /responses, the streaming iterator and the
new /v1/messages path, so all four fail closed from one change. The common case,
every reference "never", is unchanged.

The alternative, executing the "never" calls and returning only the
approval-required ones, needs partial execution that the Anthropic tool loop
cannot express without fabricating tool_result blocks for the calls it withheld,
so the whole-request fail-closed gate is the safe minimum. A future change can
add per-call partial execution if a caller needs it.

Test covers the mixed and manual cases; reverting to "any never" fails it.
2026-07-17 11:34:08 -07:00
Ishaan Jaffer
e8461b5b97
style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
Julio Quinteros Pro
e8301829cd Fix flaky MCP streaming test by properly mocking inner aresponses call
The test_streaming_mcp_events_validation test was flaky because:
1. It didn't mock the nested aresponses() call inside the iterator's
   _create_initial_response_iterator(), causing real API calls that fail
   without credentials
2. The iterator silently swallowed exceptions and set phase="finished",
   discarding pre-generated MCP discovery events
3. The _execute_tool_calls mock had wrong signature (missing tool_server_map)

Production fix: MCPEnhancedStreamingIterator no longer sets phase="finished"
on LLM call failure — it falls through to emit MCP discovery events first.

Test fix: Added mock for litellm.responses.main.aresponses returning a fake
async streaming iterator, fixed mock signatures, removed try/except that
masked failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 11:09:24 -03:00
Sameer Kankute
f878923d26 Add test for correct streaming chunks and responses id consistency 2026-03-04 12:19:54 +05:30
Shivam Rawat
d49abf8577
[Fix] Pass MCP auth headers from request into tool fetch for /v1/responses and chat completions (#22291)
* fixed dynamic auth for /responses with mcp

* fixed greptile concern
2026-02-27 19:15:51 -08:00
jquinter
199fbabfb3 Fix MCP streaming test skip logic and metadata None check (#20428)
- Fix skip condition to detect claude models (was only checking for
    "anthropic" in model name, missing "claude-haiku-4-5")
  - Add missing skip for OpenAI tests when OPENAI_API_KEY is not set
  - Fix TypeError in utils.py when metadata is explicitly None instead
    of missing (use `or {}` fallback)
2026-02-12 19:39:05 +05:30
jquinter
5666c725ce
Fix/non standard mcp url pattern (#19738)
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)

  OAuth discovery endpoints now support both URL patterns:
  - Standard MCP pattern: /mcp/{server_name} (new)
  - Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)

  The standard pattern is required by MCP-compliant clients like
  mcp-inspector and VSCode Copilot, which expect resource URLs
  following the /mcp/{server_name} convention per RFC 9728.

  Changes:
  - Add _build_oauth_protected_resource_response() helper
  - Add oauth_protected_resource_mcp_standard() endpoint
  - Add oauth_authorization_server_mcp_standard() endpoint
  - Keep legacy endpoints for backward compatibility
  - Add tests for both URL patterns

  Fixes #17272

* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)

  OAuth discovery endpoints now support both URL patterns:
  - Standard MCP pattern: /mcp/{server_name} (new)
  - Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)

  The standard pattern is required by MCP-compliant clients like
  mcp-inspector and VSCode Copilot, which expect resource URLs
  following the /mcp/{server_name} convention per RFC 9728.

  Changes:
  - Add _build_oauth_protected_resource_response() helper
  - Add oauth_protected_resource_mcp_standard() endpoint
  - Add oauth_authorization_server_mcp_standard() endpoint
  - Keep legacy endpoints for backward compatibility
  - Add tests for both URL patterns

  Fixes #17272

* Test was relocated

* refactor(mcp): Extract helper methods from run_with_session to fix PLR0915

Split the large run_with_session method (55 statements) into smaller
helper methods to satisfy ruff's PLR0915 rule (max 50 statements):

- _create_transport_context(): Creates transport based on type
- _execute_session_operation(): Handles session lifecycle

Also changed cleanup exception handling from Exception to BaseException
to properly catch asyncio.CancelledError (which is a BaseException subclass
in Python 3.8+).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(mcp): Fix flaky test by mocking health_check_server

The test_mcp_server_manager_config_integration_with_database test was
making real network calls to fake URLs which caused timeouts and
CancelledError exceptions.

Fixed by mocking health_check_server to return a proper
LiteLLM_MCPServerTable object instead of making network calls.

* test(mcp): Fix skip condition to properly detect claude model names

The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.

Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(mcp): Fix skip condition to properly detect claude model names

The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.

Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 23:00:53 -08:00
Yuta Saito
1ae9189ff8 test: Skip anthropic model test when ANTHROPIC_API_KEY is not set 2026-01-23 15:50:56 +09:00
Yuta Saito
d31c609600 test: Let MCP tool-execution mock accept new kwargs for streaming tests 2026-01-19 07:00:14 +09:00
Yuta Saito
4ad78236ab test: Fail MCP streaming test when LiteLLM logs errors during follow-up calls 2026-01-19 06:46:39 +09:00
Yuta Saito
cd19039e39 test: Parametrize MCP streaming test to cover OpenAI and Anthropic models 2026-01-19 06:22:05 +09:00
YutaSaito
2843dab7fe
fix: allow tool call even when server name prefix is missing (#16425)
* fix: allow tool call even when server name prefix is missing

* fix: test

* fix: test

* fix: test
2025-11-12 13:50:52 -08:00
Ishaan Jaff
a13aa4740a
[Fixes] Bug fixes to using LiteLLM MCP Gateway (#14392)
* fix: use _get_mcp_servers_in_path

* fix checks for using litellm_proxy as MCP tool provider

* fix: fix mcp_tools_with_litellm_proxy

* fix: fix aresponses_api_with_mcp

* aresponses_api_with_mcp

* test_mcp_allowed_tools_filtering

* fix: _filter_mcp_tools_by_allowed_tools

* fix: _filter_mcp_tools_by_allowed_tools

* test_streaming_responses_api_with_mcp_tools

* fixes: test tools transfrom MCP->OpenaI spec

* test_streaming_responses_api_with_mcp_tools

* fix: chat ui allow multi select with allowed tools

* fix: use correct MCP events with litellm proxy response API

* fix get_event_model_class

* fix litellm proxy MCP handler

* fix MCPEnhancedStreamingIterator

* chat ui show list tools result

* UI: show MCP events

* fix stream iterator

* fixes: litellm proxy mcp handler

* test responses + mcp

* fix: update responses api with mcp handling

* ruff check fix

* central: _process_mcp_tools_to_openai_format

* fix: refactor code

* test_mcp_allowed_tools_filtering

* test mcp with litellm proxy

* fix mcp call

* demo: video using MCP ui

* fixes for using stream iterator

* test_no_duplicate_mcp_tools_in_streaming_e2e

* docs fix

* fix code snippet
2025-09-10 19:12:11 -07:00
Ishaan Jaff
2d6751a396
[Feat] MCP Gateway - allow using MCPs with all LLM APIs when using /responses with LiteLLM (#12546)
* add MCPResponsesAPIHelper

* rename LiteLLM_Proxy_MCP_Handler

* aresponses_api_with_mcp

* mock_responses_api_response

* test response with litellm proxy MCP

* add _should_use_litellm_mcp_gateway

* fix transform_mcp_tool_to_openai_responses_api_tool

* use correct _transform_mcp_tools_to_openai

* fix config.yaml

* fixes for native MCP handling

* docs MCP with litellm proxy

* aresponses_api_with_mcp

* fix linting

* fix mypy

* fix linting

* test_aresponses_api_with_mcp_mock_integration

* docs How it works when server_url="litellm_proxy"
2025-07-15 14:06:31 -07:00