litellm/tests/e2e/coverage_registry/mcp.yaml
yuneng-jiang 44fcbc30b0
test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154)
* test(mcp): move the MCP OAuth happy path from a live Linear e2e to an integration test

GitHub Actions now runs unit tests only, and the MCP OAuth happy path needed a
captured Linear browser login to click through a real consent page. The OAuth
flows it exercised are already integration tests against an owned
authorization-server double, except one behavior: the stored per-user token
must be served by a proxy process that never saw the consent, with no shared
Redis to fall back on. That case is now in test_mcp_oauth_flows.py

The e2e test, its owned-gateway helper, its workflow, the mcp_oauth_live marker
and the JUnit gate script only that workflow used are removed

* test(mcp): chat completions run an OAuth server tool with the caller's own stored token

* test(e2e): drop the Linear chat-completion MCP OAuth e2e and its helpers

The behavior it checked, a chat completion running an OAuth server's tool with
the caller's stored per-user token, is now an integration test in
test_mcp_llm_endpoints.py that needs no saved Linear browser login

* test(mcp): decode the stored per-user credential as urlsafe base64

The plaintext check used b64decode, which drops the - and _ of a urlsafe value
and raised on roughly one ciphertext in five

* test(mcp): drop the chat OAuth integration test that duplicates test_mcp_slots F2-chat

security/test_mcp_slots.py already sends a per-user OAuth token through
/v1/chat/completions and asserts the peer receives it. Every mutant the new test
killed was also killed by the existing suite, so it added no signal
2026-10-17 14:29:59 -07:00

145 lines
4.6 KiB
YAML

# MCP module. Grounded in litellm/proxy/_experimental/mcp_server/. See tests/e2e/AGENTS.md for the grammar.
- id: mcp.list_tools.api_key.succeeds
module: mcp
tier: P0
operation: list_tools
auth_family: api_key
assertions: [succeeds]
source: "server.py:637"
rationale: Core operation; most common auth path; high usage
- id: mcp.list_tools.api_key.access_group_scoped
module: mcp
tier: P1
operation: list_tools
auth_family: api_key
assertions: [access_group_scoped]
source: "test_mcp_access_group_e2e.py"
rationale: "A key granted an MCP access group sees the tagged server's tools; a key with a different group does not. Access-group-scoped tool selection at key creation"
- id: mcp.list_tools.api_key.denied_without_permission
module: mcp
tier: P0
operation: list_tools
auth_family: api_key
assertions: [denied_without_permission]
source: "mcp_server_manager.py:1409"
rationale: Permission guard is high blast-radius; multi-tenant safety
- id: mcp.call_tool.api_key.succeeds
module: mcp
tier: P0
operation: call_tool
auth_family: api_key
assertions: [succeeds]
source: "server.py:849"
rationale: Primary operation; customer-critical; high usage
- id: mcp.call_tool.api_key.denied_without_permission
module: mcp
tier: P0
operation: call_tool
auth_family: api_key
assertions: [denied_without_permission]
source: "rest_endpoints.py:305-386"
rationale: Tool-level permission guard; multi-tenant safety
- id: mcp.tool_search.api_key.oversized_description_omitted
module: mcp
tier: P0
operation: tool_search
auth_family: api_key
assertions: [oversized_description_omitted]
source: "test_mcp_tool_search_e2e.py"
rationale: Native semantic search omits tools whose descriptions exceed the embedding input limit
- id: mcp.list_tools.bearer.succeeds
module: mcp
tier: P1
operation: list_tools
auth_family: bearer
assertions: [succeeds]
source: "server.py:662"
rationale: OAuth/bearer token flow; upstream delegation
- id: mcp.call_tool.bearer.succeeds
module: mcp
tier: P1
operation: call_tool
auth_family: bearer
assertions: [succeeds]
source: "server.py:886"
rationale: Bearer token forwarding for tool invocation
- id: mcp.list_tools.oauth.succeeds
module: mcp
tier: P1
operation: list_tools
auth_family: oauth
assertions: [succeeds]
source: "rest_endpoints.py:138-188"
rationale: Interactive OAuth2 flow; live token management
- id: mcp.call_tool.oauth.succeeds
module: mcp
tier: P1
operation: call_tool
auth_family: oauth
assertions: [succeeds]
source: "db.py user_oauth_credential lookup"
rationale: OAuth2 token passthrough; per-user credential storage
- id: mcp.list_tools.none.succeeds
module: mcp
tier: P1
operation: list_tools
auth_family: none
assertions: [succeeds]
source: "mcp_server_manager.py:1485-1492"
rationale: Explicitly anonymous true_passthrough servers
- id: mcp.call_tool.none.succeeds
module: mcp
tier: P1
operation: call_tool
auth_family: none
assertions: [succeeds]
source: "rest_endpoints.py:305-334"
rationale: No upstream auth required; demo servers
- id: mcp.get_prompt.api_key.succeeds
module: mcp
tier: P1
operation: get_prompt
auth_family: api_key
assertions: [succeeds]
source: "server.py:1042"
rationale: Prompt op; same auth stack as tools
- id: mcp.read_resource.api_key.succeeds
module: mcp
tier: P1
operation: read_resource
auth_family: api_key
assertions: [succeeds]
source: "server.py:1177"
rationale: Resource op; same permission model as tools
- id: mcp.list_prompts.api_key.succeeds
module: mcp
tier: P2
operation: list_prompts
auth_family: api_key
assertions: [succeeds]
source: "server.py:993"
rationale: Smoke-level; same auth stack as list_tools
- id: mcp.list_resources.api_key.succeeds
module: mcp
tier: P2
operation: list_resources
auth_family: api_key
assertions: [succeeds]
source: "server.py:1089"
rationale: Smoke; rarely used; same auth model as tools
- id: mcp.list_tools.api_key.toolset_scoped
module: mcp
tier: P0
operation: list_tools
auth_family: api_key
assertions: [toolset_scoped]
source: "user_api_key_auth_mcp.py:2137"
rationale: "A key granted a toolset lists exactly the toolset's tools: the rest of the server's catalog stays hidden and every stored name resolves"
- id: mcp.list_tools.api_key.semantic_filtered
module: mcp
tier: P1
operation: list_tools
auth_family: api_key
assertions: [semantic_filtered]
source: "litellm/proxy/hooks/mcp_semantic_filter/hook.py"
rationale: The semantic tool filter must cut a large MCP catalog to top_k and keep the tool the prompt needs