mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
feat(mcp): add opt-in local code execution
This commit is contained in:
moe-berri
parent
e81dd348a4
commit
b6c5d623eb
17 changed files with 1417 additions and 24 deletions
|
|
@ -501,6 +501,7 @@ public_skills_index: bool = False
|
|||
public_agent_groups: Optional[List[str]] = None
|
||||
agent_search_embedding_model: Optional[str] = None
|
||||
mcp_tool_search: Optional[Mapping[str, object]] = None
|
||||
mcp_code_mode: bool | Mapping[str, object] = False
|
||||
skill_search_embedding_model: Optional[str] = None
|
||||
# Supports both old format (Dict[str, str]) and new format (Dict[str, Dict[str, Any]])
|
||||
# New format: { "displayName": { "url": "...", "index": 0 } }
|
||||
|
|
|
|||
|
|
@ -2382,6 +2382,7 @@ class ProxyBaseLLMRequestProcessing:
|
|||
rate_limited_model: str | None = None,
|
||||
skip_guardrails: bool = False,
|
||||
logging_auth: UserAPIKeyAuth | None = None,
|
||||
guardrails_only: bool = False,
|
||||
) -> tuple[dict, LiteLLMLoggingObj]:
|
||||
start_time: Final = datetime.now() # start before calling guardrail hooks
|
||||
|
||||
|
|
@ -2592,6 +2593,7 @@ class ProxyBaseLLMRequestProcessing:
|
|||
data=self.data,
|
||||
call_type=route_type,
|
||||
skip_guardrails=skip_guardrails,
|
||||
guardrails_only=guardrails_only,
|
||||
)
|
||||
await _enforce_guardrail_added_tag_budgets(
|
||||
data=self.data,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,56 @@
|
|||
# MCP code mode
|
||||
|
||||
Code mode lets an MCP client run JavaScript that combines tool calls and returns only the output it needs. Enable it in the gateway configuration and restart the proxy. The QuickJS-NG runtime is included in the proxy installation on platforms with supported wheels; no separate sandbox service or client integration is required.
|
||||
|
||||
```yaml
|
||||
litellm_settings:
|
||||
mcp_code_mode: true
|
||||
```
|
||||
|
||||
Reconnect the MCP client or refresh its tool list. It should discover `litellm_execute_code` alongside its existing tools. The setting is off by default. Turning it off also rejects calls from clients that cached the code tool.
|
||||
|
||||
Intel macOS and Windows ARM installations need a manual source installation of `quickjs-ng` before enabling code mode. A missing runtime produces a startup error. The original `quickjs` package conflicts with `quickjs-ng`; use an environment without that package. These requirements do not affect the gateway when code mode is disabled.
|
||||
|
||||
The code tool accepts a `code` string. Discover a tool and its argument schema normally, then use its exact catalog name or the opaque ID returned by `search_tools` in `tools.call_tool`. Calls return the full MCP result, including `content`, `structuredContent` when provided, and `isError`. Use `text(value)` to return a JSON value to the model. For example, after discovering a tool named `example-add` that accepts `a` and `b`:
|
||||
|
||||
```javascript
|
||||
const result = await tools.call_tool({
|
||||
tool_id: "example-add",
|
||||
arguments: { a: 2, b: 3 }
|
||||
});
|
||||
if (result.isError) {
|
||||
text({ error: result.content });
|
||||
} else {
|
||||
text(result.structuredContent ?? result.content);
|
||||
}
|
||||
```
|
||||
|
||||
Use `await` for every call, including calls grouped with `Promise.all`. Scripts run in fresh local worker processes. They have no persistent state, general network, filesystem, package imports, or shell API. The parent gateway performs tool calls using the original caller's permissions, upstream authentication, guardrails, and accounting. A worker process isolates normal runtime failures; it is not an operating-system sandbox against an exploit of the embedded JavaScript engine.
|
||||
|
||||
## Tool compatibility
|
||||
|
||||
When code mode is enabled, tool listings include `_meta["litellm.ai/code_mode"]`; search and schema results include `code_mode`. `available` means the tool can be attempted through the JSON call bridge, not that the gateway has tested every behavior of that upstream. `unsupported` includes a reason. Existing tools remain callable directly.
|
||||
|
||||
Task-only tools and nested orchestration are unsupported. Interactive continuations cannot be resumed inside a script. An unexpected interaction requirement returns an error instead of bypassing approval or replaying the script. Approving the outer code tool in an MCP client is not per-call approval of every nested action.
|
||||
|
||||
An administrator can exclude tools known to need direct interaction. Use the exact tool name from the gateway catalog:
|
||||
|
||||
```yaml
|
||||
litellm_settings:
|
||||
mcp_code_mode:
|
||||
enabled: true
|
||||
excluded_tools:
|
||||
example-approve: "Requires interactive approval. Call this tool directly."
|
||||
```
|
||||
|
||||
Keep large identifiers as strings. Values crossing the JavaScript bridge must be JSON-compatible, and integer values must fit JavaScript's safe integer range. Tool errors and unsupported data are not silently treated as successful results.
|
||||
|
||||
## Failures and cancellation
|
||||
|
||||
Each script has a 120-second deadline, a 64 MiB JavaScript heap limit, and a maximum of 100 tool calls with eight in flight. Each proxy process admits four scripts at once; additional requests receive a busy error. Code is limited to 128 KiB, individual bridge messages to 2 MiB, and returned output to 256 KiB. Use direct calls for tools whose results or duration exceed these limits.
|
||||
|
||||
A script that exceeds a limit fails explicitly. The result includes `output`, `error`, and `calls_started`; the count records attempted tool calls, not confirmed upstream actions. Cancellation stops further dispatch and terminates the worker, but cannot undo a request already sent upstream. A failed script can therefore have partial effects. Check the upstream state before retrying a write; the gateway does not automatically retry whole scripts.
|
||||
|
||||
# Gateway catalog pagination
|
||||
|
||||
Tools, prompts, resources, and resource templates retain upstream page boundaries. Follow the gateway's opaque `nextCursor` until it is absent. Each request checks the caller's current access and tool permissions. A cursor can be replayed and can be sent to another replica serving the same configuration
|
||||
|
|
|
|||
|
|
@ -928,11 +928,13 @@ async def get_filtered_server_tools(
|
|||
else:
|
||||
listing_updates.callback(record)
|
||||
|
||||
if mcp_proxy_mode:
|
||||
from litellm.proxy.mcp_server.code_execution import settings as code_mode_settings
|
||||
|
||||
if mcp_proxy_mode or code_mode_settings().enabled:
|
||||
from litellm.proxy.mcp_server.tool_search import with_mcp_proxy_identity
|
||||
|
||||
filtered_tools = [with_mcp_proxy_identity(tool, server.server_id) for tool in filtered_tools]
|
||||
else:
|
||||
if not mcp_proxy_mode:
|
||||
filtered_tools = apply_display_name_overrides(filtered_tools, server)
|
||||
|
||||
verbose_logger.debug(
|
||||
|
|
|
|||
284
litellm/proxy/mcp_server/code_execution.py
Normal file
284
litellm/proxy/mcp_server/code_execution.py
Normal file
|
|
@ -0,0 +1,284 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Mapping, Sequence
|
||||
from dataclasses import replace
|
||||
from datetime import datetime
|
||||
from typing import Final
|
||||
from uuid import uuid4
|
||||
|
||||
from mcp.shared.exceptions import MCPError
|
||||
from mcp.types import CallToolResult, TextContent, Tool
|
||||
from pydantic import ConfigDict, JsonValue, TypeAdapter
|
||||
|
||||
import litellm
|
||||
from litellm.proxy.mcp_server.contracts import OperationContext
|
||||
from litellm.types.llms.base import LiteLLMBaseModel
|
||||
from litellm.types.mcp import MCPCodeModeSettings
|
||||
|
||||
CODE_TOOL_NAME: Final = "litellm_execute_code"
|
||||
CODE_MODE_META: Final = "litellm.ai/code_mode"
|
||||
_JSON_OBJECT: Final = TypeAdapter(dict[str, JsonValue])
|
||||
|
||||
|
||||
class CodeInput(LiteLLMBaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
code: str
|
||||
|
||||
|
||||
def settings() -> MCPCodeModeSettings:
|
||||
configured: Final = litellm.mcp_code_mode
|
||||
return (
|
||||
MCPCodeModeSettings(enabled=configured)
|
||||
if isinstance(configured, bool)
|
||||
else MCPCodeModeSettings.model_validate(configured)
|
||||
)
|
||||
|
||||
|
||||
def validate_settings(configured: object) -> MCPCodeModeSettings:
|
||||
validated: Final = (
|
||||
MCPCodeModeSettings(enabled=configured)
|
||||
if isinstance(configured, bool)
|
||||
else MCPCodeModeSettings.model_validate(configured)
|
||||
)
|
||||
if validated.enabled:
|
||||
from litellm.proxy.mcp_server.code_runtime import check_runtime
|
||||
|
||||
check_runtime()
|
||||
return validated
|
||||
|
||||
|
||||
def tool_names(tool: Tool) -> frozenset[str]:
|
||||
"""Recognize canonical, bare and configured display names for one identity."""
|
||||
from litellm.proxy.mcp_server.mcp_server_manager import global_mcp_server_manager
|
||||
from litellm.proxy.mcp_server.tool_search import (
|
||||
_mcp_proxy_identity, # pyright: ignore[reportPrivateUsage] # shared identity metadata contract
|
||||
)
|
||||
from litellm.proxy.mcp_server.utils import strip_known_server_prefix
|
||||
|
||||
try:
|
||||
identity: Final = _mcp_proxy_identity(tool)
|
||||
except TypeError:
|
||||
return frozenset((tool.name,))
|
||||
canonical: Final = identity["tool_name"]
|
||||
server: Final = next(
|
||||
(item for item in global_mcp_server_manager.get_registry().values() if item.server_id == identity["server_id"]),
|
||||
None,
|
||||
)
|
||||
if server is None:
|
||||
return frozenset((tool.name, canonical))
|
||||
bare: Final = strip_known_server_prefix(canonical, server)
|
||||
display: Final = (server.tool_name_to_display_name or {}).get(bare, canonical)
|
||||
return frozenset((tool.name, canonical, bare, display))
|
||||
|
||||
|
||||
def compatibility(tool: Tool) -> Mapping[str, JsonValue]:
|
||||
from litellm.proxy.mcp_server.tool_search import MCP_PROXY_TOOL_NAMES, VIRTUAL_TOOL_NAMES
|
||||
|
||||
names: Final = tool_names(tool)
|
||||
if names.intersection((*MCP_PROXY_TOOL_NAMES, *VIRTUAL_TOOL_NAMES, CODE_TOOL_NAME)):
|
||||
return {"status": "unsupported", "reason": "Nested orchestration is unsupported"}
|
||||
if tool.execution is not None and tool.execution.task_support == "required":
|
||||
return {"status": "unsupported", "reason": "Task-based execution is unsupported in code mode"}
|
||||
excluded: Final = next((reason for name, reason in settings().excluded_tools.items() if name in names), None)
|
||||
return (
|
||||
{"status": "unsupported", "reason": excluded}
|
||||
if excluded is not None
|
||||
else {
|
||||
"status": "available",
|
||||
"reason": "JSON tool calls only. Interactive continuations are unsupported; upstream behavior has not been verified.",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def code_tool() -> Tool:
|
||||
from litellm.proxy.mcp_server.code_runtime import CodeExecutionLimits
|
||||
|
||||
limits: Final = CodeExecutionLimits()
|
||||
return Tool(
|
||||
name=CODE_TOOL_NAME,
|
||||
description=(
|
||||
"Run JavaScript to compose MCP tool calls. Use await tools.call_tool({tool_id, arguments}); "
|
||||
"tool_id is an ID from search_tools or an exact tool name from the catalog. "
|
||||
"Each call returns the full MCP envelope: structuredContent, content and isError. "
|
||||
"Inspect isError before using results. Emit only needed results with text(value). "
|
||||
"No filesystem, network, imports, persistent state or interactive continuations. "
|
||||
"A failed script may have completed some calls; never blindly replay writes. "
|
||||
f"Limits: {limits.timeout_seconds:g} seconds, {limits.max_calls} calls, "
|
||||
f"{limits.max_concurrent_calls} concurrent calls, {limits.output_bytes // 1024} KiB output."
|
||||
),
|
||||
input_schema=CodeInput.model_json_schema(),
|
||||
)
|
||||
|
||||
|
||||
def with_code_tool(
|
||||
tools: Sequence[Tool], *, first_page: bool = True
|
||||
) -> list[Tool]: # mutable-ok: MCP SDK requires a list
|
||||
if not settings().enabled:
|
||||
return list(tools)
|
||||
if any(tool.name == CODE_TOOL_NAME for tool in tools):
|
||||
raise MCPError(code=-32602, message="MCP tool name conflicts with the reserved code execution tool")
|
||||
annotated: Final = [
|
||||
tool.model_copy(update={"meta": {**(tool.meta or {}), CODE_MODE_META: compatibility(tool)}}) for tool in tools
|
||||
]
|
||||
return [*annotated, *([code_tool()] if first_page else [])]
|
||||
|
||||
|
||||
def error_result(message: str) -> CallToolResult:
|
||||
return CallToolResult(content=[TextContent(type="text", text=message)], is_error=True)
|
||||
|
||||
|
||||
def serialize_tool_result(
|
||||
result: CallToolResult,
|
||||
) -> dict[str, JsonValue]: # mutable-ok: runtime JSON callback contract
|
||||
return _JSON_OBJECT.validate_python(result.model_dump(mode="json", by_alias=True, exclude_none=True))
|
||||
|
||||
|
||||
def with_code_trace(context: OperationContext) -> OperationContext:
|
||||
from litellm.proxy.litellm_pre_call_utils import get_chain_id_from_headers
|
||||
|
||||
headers: Final = dict(context.raw_headers or {})
|
||||
if get_chain_id_from_headers(headers):
|
||||
return context
|
||||
return replace(context, raw_headers={**headers, "x-litellm-trace-id": str(uuid4())})
|
||||
|
||||
|
||||
async def run_code_tool(arguments: Mapping[str, object] | None, context: OperationContext) -> CallToolResult:
|
||||
from litellm.proxy.mcp_server.operations import (
|
||||
_fire_mcp_tool_call_logging,
|
||||
fire_mcp_tool_call_failure_logging,
|
||||
prepare_virtual_mcp_call,
|
||||
)
|
||||
|
||||
if not settings().enabled:
|
||||
return error_result("MCP code mode is disabled")
|
||||
caller: Final = context.user_api_key_auth
|
||||
if caller is None:
|
||||
return error_result("MCP code execution requires an authenticated caller")
|
||||
traced: Final = with_code_trace(context)
|
||||
started: Final = datetime.now() # noqa: DTZ005 # MCP logging uses naive timestamps
|
||||
data, logging_obj = await prepare_virtual_mcp_call(
|
||||
CODE_TOOL_NAME,
|
||||
dict(arguments or {}),
|
||||
caller,
|
||||
raw_headers=traced.raw_headers,
|
||||
client_ip=traced.client_ip,
|
||||
guardrails_only=True,
|
||||
)
|
||||
try:
|
||||
processed: Final = TypeAdapter(dict[str, object]).validate_python(
|
||||
data.get("modified_arguments", data.get("mcp_arguments", data.get("arguments", {})))
|
||||
)
|
||||
result: Final = await _execute_script(processed, traced)
|
||||
if logging_obj is None:
|
||||
return result
|
||||
return await _fire_mcp_tool_call_logging(
|
||||
logging_obj,
|
||||
result,
|
||||
started,
|
||||
datetime.now(), # noqa: DTZ005 # MCP logging uses naive timestamps
|
||||
caller,
|
||||
data,
|
||||
)
|
||||
except Exception as exc:
|
||||
await fire_mcp_tool_call_failure_logging(logging_obj, exc, started, caller, data)
|
||||
raise
|
||||
|
||||
|
||||
async def _execute_script(arguments: Mapping[str, object], context: OperationContext) -> CallToolResult:
|
||||
from litellm.proxy.mcp_server.operations import _dispatch_virtual_mcp_tool, _list_mcp_tools, find_mcp_proxy_tool
|
||||
from litellm.proxy.mcp_server.tool_search import (
|
||||
MCP_PROXY_TOOL_NAMES,
|
||||
VIRTUAL_TOOL_NAMES,
|
||||
mcp_proxy_tool_id,
|
||||
)
|
||||
|
||||
if not settings().enabled:
|
||||
return error_result("MCP code mode is disabled")
|
||||
caller: Final = context.user_api_key_auth
|
||||
if caller is None:
|
||||
return error_result("MCP code execution requires an authenticated caller")
|
||||
from litellm.proxy.mcp_server.code_runtime import execute_code
|
||||
|
||||
request: Final = CodeInput.model_validate(arguments or {})
|
||||
auth, header, servers, server_headers, oauth_headers, raw_headers, client_ip = context.legacy_auth()
|
||||
forbidden: Final = {*MCP_PROXY_TOOL_NAMES, *VIRTUAL_TOOL_NAMES, CODE_TOOL_NAME}
|
||||
|
||||
async def find_tool(tool_id: str) -> Tool | None:
|
||||
if re.fullmatch(r"[0-9a-f]{32}", tool_id):
|
||||
return await find_mcp_proxy_tool(
|
||||
tool_id,
|
||||
user_api_key_auth=caller,
|
||||
mcp_servers=servers,
|
||||
client_ip=client_ip,
|
||||
mcp_auth_header=header,
|
||||
mcp_server_auth_headers=server_headers,
|
||||
oauth2_headers=oauth_headers,
|
||||
raw_headers=raw_headers,
|
||||
)
|
||||
listing: Final = await _list_mcp_tools(
|
||||
user_api_key_auth=auth,
|
||||
mcp_servers=servers,
|
||||
client_ip=client_ip,
|
||||
mcp_auth_header=header,
|
||||
mcp_server_auth_headers=server_headers,
|
||||
oauth2_headers=oauth_headers,
|
||||
raw_headers=raw_headers,
|
||||
mcp_proxy_mode=True,
|
||||
)
|
||||
matches: Final = tuple(tool for tool in listing.tools if tool_id in tool_names(tool))
|
||||
return matches[0] if len(matches) == 1 else None
|
||||
|
||||
async def call_tool(
|
||||
tool_id: str,
|
||||
tool_arguments: dict[str, JsonValue], # mutable-ok: runtime JSON callback contract
|
||||
) -> dict[str, JsonValue]: # mutable-ok: runtime JSON callback contract
|
||||
if tool_id in forbidden:
|
||||
return serialize_tool_result(error_result("Nested orchestration tools are unsupported"))
|
||||
selected: Final = await find_tool(tool_id)
|
||||
if selected is None:
|
||||
return serialize_tool_result(error_result("Unknown, ambiguous or unauthorized tool"))
|
||||
support: Final = compatibility(selected)
|
||||
if support["status"] == "unsupported":
|
||||
return serialize_tool_result(error_result(f"Tool is unsupported in code mode: {support['reason']}"))
|
||||
result: Final = await _dispatch_virtual_mcp_tool(
|
||||
name="call_tool",
|
||||
arguments={"tool_id": mcp_proxy_tool_id(selected), "arguments": tool_arguments},
|
||||
user_api_key_auth=auth,
|
||||
client_ip=client_ip,
|
||||
mcp_servers=servers,
|
||||
mcp_auth_header=header,
|
||||
mcp_server_auth_headers=server_headers,
|
||||
oauth2_headers=oauth_headers,
|
||||
raw_headers=raw_headers,
|
||||
mcp_proxy_mode=True,
|
||||
incoming_bearer_token=context.incoming_bearer_token,
|
||||
)
|
||||
assert result is not None
|
||||
from litellm.proxy.mcp_server.result_conversion import INPUT_REQUIRED_UNSUPPORTED_MESSAGE
|
||||
|
||||
if any(
|
||||
isinstance(block, TextContent) and block.text == INPUT_REQUIRED_UNSUPPORTED_MESSAGE
|
||||
for block in result.content
|
||||
):
|
||||
return serialize_tool_result(
|
||||
error_result(
|
||||
"This tool requires an interactive continuation, which is unsupported in code mode. "
|
||||
"Call it directly instead. Earlier calls in this script may already have completed."
|
||||
)
|
||||
)
|
||||
return serialize_tool_result(result)
|
||||
|
||||
result: Final = await execute_code(request.code, call_tool)
|
||||
return CallToolResult(
|
||||
content=[
|
||||
TextContent(
|
||||
type="text",
|
||||
text=json.dumps(
|
||||
{"output": result.output, "error": result.error, "calls_started": result.calls_started}
|
||||
),
|
||||
)
|
||||
],
|
||||
is_error=result.error is not None,
|
||||
)
|
||||
181
litellm/proxy/mcp_server/code_runtime.py
Normal file
181
litellm/proxy/mcp_server/code_runtime.py
Normal file
|
|
@ -0,0 +1,181 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import AsyncIterator, Awaitable, Callable
|
||||
from importlib import metadata
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from typing import Final, TypeAlias
|
||||
|
||||
import anyio
|
||||
from anyio.abc import Process
|
||||
from anyio.lowlevel import RunVar
|
||||
from anyio.streams.buffered import BufferedByteReceiveStream
|
||||
from pydantic import JsonValue
|
||||
|
||||
from litellm.proxy.mcp_server.code_worker import (
|
||||
WORKER_MESSAGE,
|
||||
Call,
|
||||
CodeExecutionLimits,
|
||||
CodeExecutionResult,
|
||||
Done,
|
||||
Reply,
|
||||
Start,
|
||||
encode_message,
|
||||
load_json,
|
||||
)
|
||||
|
||||
ToolCallback: TypeAlias = Callable[[str, dict[str, JsonValue]], Awaitable[dict[str, JsonValue]]]
|
||||
_WORKERS: Final = RunVar[anyio.CapacityLimiter]("mcp_code_workers")
|
||||
_DEFAULT_LIMITS: Final = CodeExecutionLimits()
|
||||
|
||||
|
||||
class _Calls:
|
||||
def __init__(self) -> None:
|
||||
self.started: frozenset[int] = frozenset()
|
||||
self.active: frozenset[int] = frozenset()
|
||||
|
||||
def start(self, call_id: int) -> None:
|
||||
self.started = self.started | {call_id}
|
||||
self.active = self.active | {call_id}
|
||||
|
||||
def finish(self, call_id: int) -> None:
|
||||
self.active = self.active - {call_id}
|
||||
|
||||
|
||||
def check_runtime() -> None:
|
||||
try:
|
||||
metadata.distribution("quickjs-ng")
|
||||
except metadata.PackageNotFoundError as exc:
|
||||
raise ValueError("MCP code mode requires quickjs-ng; install it on a supported platform") from exc
|
||||
try:
|
||||
metadata.distribution("quickjs")
|
||||
except metadata.PackageNotFoundError:
|
||||
pass
|
||||
else:
|
||||
raise ValueError("MCP code mode requires removing the conflicting quickjs package and reinstalling quickjs-ng")
|
||||
if importlib.util.find_spec("_quickjs") is None:
|
||||
raise ValueError("MCP code mode requires quickjs-ng; install it on a supported platform")
|
||||
|
||||
|
||||
async def _build_reply(call: Call, callback: ToolCallback, maximum: int) -> bytes:
|
||||
try:
|
||||
value: Final = await callback(call.tool_id, dict(call.arguments))
|
||||
reply: Final = Reply(id=call.id, value=value)
|
||||
try:
|
||||
return encode_message(reply, maximum)
|
||||
except (ValueError, RecursionError):
|
||||
return encode_message(
|
||||
Reply(id=call.id, error="Tool result is too large or contains unsupported JSON numbers"), maximum
|
||||
)
|
||||
except Exception: # noqa: BLE001 - Untrusted upstream failures must not disclose secrets to scripts.
|
||||
return encode_message(Reply(id=call.id, error="Tool execution failed"), maximum)
|
||||
|
||||
|
||||
async def _exchange(
|
||||
process: Process,
|
||||
code: str,
|
||||
callback: ToolCallback,
|
||||
settings: CodeExecutionLimits,
|
||||
calls: _Calls,
|
||||
) -> CodeExecutionResult:
|
||||
assert process.stdin is not None and process.stdout is not None
|
||||
output: Final = BufferedByteReceiveStream(process.stdout)
|
||||
writer: Final = anyio.Lock()
|
||||
|
||||
async def dispatch(call: Call) -> None:
|
||||
data: Final = await _build_reply(call, callback, settings.message_bytes)
|
||||
async with writer:
|
||||
assert process.stdin is not None
|
||||
calls.finish(call.id)
|
||||
await process.stdin.send(data)
|
||||
|
||||
await process.stdin.send(encode_message(Start(code=code, limits=settings), settings.message_bytes))
|
||||
async with anyio.create_task_group() as group:
|
||||
try:
|
||||
async for message in _messages(output, settings.message_bytes):
|
||||
match message:
|
||||
case Call():
|
||||
if message.id in calls.started or len(calls.started) >= settings.max_calls:
|
||||
return CodeExecutionResult(
|
||||
error="Code execution tool-call limit exceeded", calls_started=len(calls.started)
|
||||
)
|
||||
if len(calls.active) >= settings.max_concurrent_calls:
|
||||
return CodeExecutionResult(
|
||||
error="Code execution concurrent-call limit exceeded", calls_started=len(calls.started)
|
||||
)
|
||||
calls.start(message.id)
|
||||
group.start_soon(dispatch, message)
|
||||
case Done():
|
||||
if len(json.dumps(message.output, ensure_ascii=False).encode("utf-8")) > settings.output_bytes:
|
||||
return CodeExecutionResult(
|
||||
error="Code output exceeds its byte limit", calls_started=len(calls.started)
|
||||
)
|
||||
return CodeExecutionResult(
|
||||
output=message.output,
|
||||
error="Script returned with unawaited tool calls" if calls.active else message.error,
|
||||
calls_started=len(calls.started),
|
||||
)
|
||||
finally:
|
||||
group.cancel_scope.cancel()
|
||||
return CodeExecutionResult(error="Code execution worker closed without a result", calls_started=len(calls.started))
|
||||
|
||||
|
||||
async def _messages(stream: BufferedByteReceiveStream, maximum: int) -> AsyncIterator[Call | Done]:
|
||||
while True:
|
||||
yield WORKER_MESSAGE.validate_python(load_json((await stream.receive_until(b"\n", maximum)).decode("utf-8")))
|
||||
|
||||
|
||||
async def execute_code(
|
||||
code: str, callback: ToolCallback, settings: CodeExecutionLimits = _DEFAULT_LIMITS
|
||||
) -> CodeExecutionResult:
|
||||
try:
|
||||
size: Final = len(code.encode("utf-8"))
|
||||
except UnicodeError:
|
||||
return CodeExecutionResult(error="Code must contain valid Unicode")
|
||||
if size > settings.script_bytes:
|
||||
return CodeExecutionResult(error="Code exceeds its byte limit")
|
||||
check_runtime()
|
||||
existing: Final = _WORKERS.get(None)
|
||||
limiter: Final = existing if existing is not None else anyio.CapacityLimiter(settings.max_workers)
|
||||
if existing is None:
|
||||
_WORKERS.set(limiter)
|
||||
try:
|
||||
limiter.acquire_nowait()
|
||||
except anyio.WouldBlock:
|
||||
return CodeExecutionResult(error="Code execution is busy; retry when a worker is available")
|
||||
calls: Final = _Calls()
|
||||
try:
|
||||
with TemporaryDirectory(prefix="litellm-code-") as directory:
|
||||
environment: Final = {"SYSTEMROOT": os.environ["SYSTEMROOT"]} if "SYSTEMROOT" in os.environ else {}
|
||||
with anyio.fail_after(settings.timeout_seconds):
|
||||
async with await anyio.open_process(
|
||||
[sys.executable, "-I", "-u", str(Path(__file__).with_name("code_worker.py"))],
|
||||
cwd=directory,
|
||||
env=environment,
|
||||
stderr=subprocess.DEVNULL,
|
||||
) as process:
|
||||
try:
|
||||
return await _exchange(process, code, callback, settings, calls)
|
||||
finally:
|
||||
with anyio.CancelScope(shield=True):
|
||||
if process.returncode is None:
|
||||
try:
|
||||
process.kill()
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
await process.wait()
|
||||
except TimeoutError:
|
||||
return CodeExecutionResult(
|
||||
error="Code execution timed out; upstream actions may have completed", calls_started=len(calls.started)
|
||||
)
|
||||
except Exception: # noqa: BLE001 - Process/protocol failures are isolated; cancellation still propagates.
|
||||
return CodeExecutionResult(
|
||||
error="Code execution worker failed; upstream actions may have completed", calls_started=len(calls.started)
|
||||
)
|
||||
finally:
|
||||
limiter.release()
|
||||
261
litellm/proxy/mcp_server/code_worker.py
Normal file
261
litellm/proxy/mcp_server/code_worker.py
Normal file
|
|
@ -0,0 +1,261 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import json
|
||||
import math
|
||||
import sys
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Annotated, Final, Literal, Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue, TypeAdapter
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CodeExecutionLimits:
|
||||
timeout_seconds: float = 120
|
||||
memory_bytes: int = 64 * 1024 * 1024
|
||||
script_bytes: int = 128 * 1024
|
||||
message_bytes: int = 2 * 1024 * 1024
|
||||
output_bytes: int = 256 * 1024
|
||||
max_calls: int = 100
|
||||
max_concurrent_calls: int = 8
|
||||
max_workers: int = 4
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CodeExecutionResult:
|
||||
output: tuple[JsonValue, ...] = ()
|
||||
error: str | None = None
|
||||
calls_started: int = 0
|
||||
|
||||
|
||||
class WireModel(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
|
||||
class Start(WireModel):
|
||||
code: str
|
||||
limits: CodeExecutionLimits
|
||||
|
||||
|
||||
class Call(WireModel):
|
||||
kind: Literal["call"] = "call"
|
||||
id: int = Field(ge=0)
|
||||
tool_id: str
|
||||
arguments: Mapping[str, JsonValue]
|
||||
|
||||
|
||||
class Reply(WireModel):
|
||||
id: int = Field(ge=0)
|
||||
value: Mapping[str, JsonValue] | None = None
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class Done(WireModel):
|
||||
kind: Literal["done"] = "done"
|
||||
output: tuple[JsonValue, ...] = ()
|
||||
error: str | None = None
|
||||
|
||||
|
||||
WORKER_MESSAGE: Final[TypeAdapter[Call | Done]] = TypeAdapter(Annotated[Call | Done, Field(discriminator="kind")])
|
||||
JSON_VALUE: Final[TypeAdapter[JsonValue]] = TypeAdapter(JsonValue)
|
||||
TEXT: Final[TypeAdapter[str]] = TypeAdapter(str)
|
||||
MAX_SAFE_INTEGER: Final = 2**53 - 1
|
||||
|
||||
|
||||
def _integer(value: str) -> int:
|
||||
result: Final = int(value)
|
||||
if abs(result) > MAX_SAFE_INTEGER:
|
||||
raise ValueError("JSON integers must be JavaScript-safe; represent larger integers as strings")
|
||||
return result
|
||||
|
||||
|
||||
def _float(value: str) -> float:
|
||||
result: Final = float(value)
|
||||
if not math.isfinite(result) or (result.is_integer() and abs(result) > MAX_SAFE_INTEGER):
|
||||
raise ValueError("JSON numbers must be finite and integers JavaScript-safe")
|
||||
return result
|
||||
|
||||
|
||||
def load_json(value: str) -> JsonValue:
|
||||
return JSON_VALUE.validate_python(json.loads(value, parse_int=_integer, parse_float=_float, parse_constant=_float))
|
||||
|
||||
|
||||
def encode_message(value: WireModel, maximum: int) -> bytes:
|
||||
text: Final = json.dumps(value.model_dump(), allow_nan=False, ensure_ascii=False)
|
||||
load_json(text)
|
||||
encoded: Final = (text + "\n").encode("utf-8")
|
||||
if len(encoded) > maximum:
|
||||
raise ValueError("Code execution message exceeds its byte limit")
|
||||
return encoded
|
||||
|
||||
|
||||
BOOTSTRAP: Final = r"""
|
||||
(rpc, code, limits) => {
|
||||
const stringify = JSON.stringify.bind(JSON), parse = JSON.parse.bind(JSON);
|
||||
const isSafe = Number.isSafeInteger.bind(Number), isFinite = Number.isFinite.bind(Number);
|
||||
const isInteger = Number.isInteger.bind(Number);
|
||||
const AsyncFunction = Object.getPrototypeOf(async function(){}).constructor;
|
||||
const ErrorType = Error;
|
||||
const StringType = String, slice = Function.prototype.call.bind(String.prototype.slice);
|
||||
const output = [];
|
||||
let outputSize = 0, calls = 0, active = 0, fatal = null;
|
||||
const fail = message => { fatal = message; throw new ErrorType(message); };
|
||||
const encode = value => {
|
||||
const result = stringify(value, (key, item) => {
|
||||
if (typeof item === 'number' && (!isFinite(item) || (isInteger(item) && !isSafe(item))))
|
||||
fail('JSON numbers must be finite and integers JavaScript-safe; use strings for larger integers');
|
||||
if (['undefined', 'bigint', 'function', 'symbol'].includes(typeof item))
|
||||
fail('Output and tool arguments must be JSON values');
|
||||
return item;
|
||||
});
|
||||
if (typeof result !== 'string') fail('Output and tool arguments must be JSON values');
|
||||
return result;
|
||||
};
|
||||
const text = value => {
|
||||
const encoded = encode(value);
|
||||
outputSize += encoded.length;
|
||||
if (outputSize > limits.output_bytes) fail('Code output exceeds its byte limit');
|
||||
output.push(parse(encoded));
|
||||
};
|
||||
const tools = Object.freeze({call_tool: async request => {
|
||||
if (++calls > limits.max_calls) fail('Code execution tool-call limit exceeded');
|
||||
if (++active > limits.max_concurrent_calls) fail('Code execution concurrent-call limit exceeded');
|
||||
try {
|
||||
const encoded = encode(request);
|
||||
if (encoded.length > limits.message_bytes) fail('Tool arguments exceed their byte limit');
|
||||
const reply = parse(await rpc(encoded));
|
||||
if (reply.error) fail(reply.error);
|
||||
return reply.value;
|
||||
} finally { active--; }
|
||||
}});
|
||||
return (async () => {
|
||||
try {
|
||||
await new AsyncFunction('tools', 'text', code)(tools, text);
|
||||
if (active) fail('Script returned with unawaited tool calls');
|
||||
return stringify({kind:'done', output, error:fatal});
|
||||
} catch (error) {
|
||||
let detail = 'JavaScript execution failed';
|
||||
try { detail = slice(StringType(error), 0, 1024); } catch (_) {}
|
||||
return stringify({kind:'done', output, error:fatal || detail});
|
||||
}
|
||||
})();
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
_BRIDGE: Final = (
|
||||
r"""
|
||||
(runner => {
|
||||
const send = globalThis.__litellm_send;
|
||||
delete globalThis.__litellm_send;
|
||||
const parse = JSON.parse.bind(JSON), stringify = JSON.stringify.bind(JSON);
|
||||
const get = Function.prototype.call.bind(Map.prototype.get);
|
||||
const set = Function.prototype.call.bind(Map.prototype.set);
|
||||
const remove = Function.prototype.call.bind(Map.prototype.delete);
|
||||
const then = Function.prototype.call.bind(Promise.prototype.then);
|
||||
const PromiseType = Promise, ErrorType = Error, pending = new Map();
|
||||
let sequence = 0;
|
||||
const rpc = raw => new PromiseType(resolve => {
|
||||
const id = sequence++;
|
||||
set(pending, id, resolve);
|
||||
send(stringify({...parse(raw), kind:'call', id}));
|
||||
});
|
||||
return (kind, raw) => {
|
||||
const value = parse(raw);
|
||||
if (kind === 'start') {
|
||||
then(runner(rpc, value.code, value.limits), send);
|
||||
} else {
|
||||
const resolve = get(pending, value.id);
|
||||
if (!resolve) throw new ErrorType('Invalid parent reply');
|
||||
remove(pending, value.id);
|
||||
resolve(stringify(value.error ? {error:value.error} : {value:value.value}));
|
||||
}
|
||||
};
|
||||
})("""
|
||||
+ BOOTSTRAP
|
||||
+ ")"
|
||||
)
|
||||
|
||||
|
||||
class _Runtime(Protocol):
|
||||
def set_memory_limit(self, limit: int) -> None: ...
|
||||
def set_max_stack_size(self, limit: int) -> None: ...
|
||||
def add_callable(self, name: str, callback: Callable[[str], None]) -> None: ...
|
||||
def eval(self, code: str) -> object: ...
|
||||
def execute_pending_job(self) -> bool: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class _RuntimeModule(Protocol):
|
||||
Context: Callable[[], _Runtime]
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class _Receiver(Protocol):
|
||||
def __call__(self, kind: str, value: str) -> None: ...
|
||||
|
||||
|
||||
class _Output:
|
||||
def __init__(self, maximum: int) -> None:
|
||||
self.maximum = maximum
|
||||
self.result: Done | None = None
|
||||
|
||||
def send(self, raw: str) -> None:
|
||||
if len(raw.encode("utf-8")) > self.maximum:
|
||||
raise ValueError("Code execution message exceeds its byte limit")
|
||||
message: Final = WORKER_MESSAGE.validate_python(load_json(raw))
|
||||
if isinstance(message, Done):
|
||||
self.result = message
|
||||
else:
|
||||
sys.stdout.buffer.write(encode_message(message, self.maximum))
|
||||
sys.stdout.buffer.flush()
|
||||
|
||||
|
||||
def _read_reply(maximum: int) -> Reply:
|
||||
line: Final = sys.stdin.buffer.readline(maximum + 1)
|
||||
if not line or len(line) > maximum:
|
||||
raise ValueError("Invalid parent message")
|
||||
return Reply.model_validate(load_json(line.decode("utf-8")))
|
||||
|
||||
|
||||
def _run(start: Start) -> Done:
|
||||
module: Final = importlib.import_module("_quickjs")
|
||||
if not isinstance(module, _RuntimeModule):
|
||||
raise TypeError("Unsupported quickjs-ng runtime")
|
||||
runtime: Final = module.Context()
|
||||
runtime.set_memory_limit(start.limits.memory_bytes)
|
||||
runtime.set_max_stack_size(512 * 1024)
|
||||
output: Final = _Output(start.limits.message_bytes)
|
||||
runtime.add_callable("__litellm_send", output.send)
|
||||
receive: Final = runtime.eval(_BRIDGE)
|
||||
if not isinstance(receive, _Receiver):
|
||||
raise TypeError("Invalid JavaScript bridge")
|
||||
receive("start", start.model_dump_json())
|
||||
while output.result is None:
|
||||
while runtime.execute_pending_job():
|
||||
if output.result is not None:
|
||||
return output.result
|
||||
if output.result is None:
|
||||
receive("reply", _read_reply(start.limits.message_bytes).model_dump_json())
|
||||
return output.result
|
||||
|
||||
|
||||
def main() -> None:
|
||||
try:
|
||||
maximum: Final = CodeExecutionLimits().message_bytes
|
||||
line: Final = sys.stdin.buffer.readline(maximum + 1)
|
||||
if len(line) > maximum:
|
||||
raise ValueError("Code execution start message exceeds its byte limit")
|
||||
start: Final = Start.model_validate(load_json(line.decode("utf-8")))
|
||||
result: Final = _run(start)
|
||||
sys.stdout.buffer.write(encode_message(result, start.limits.message_bytes))
|
||||
sys.stdout.buffer.flush()
|
||||
except Exception: # noqa: BLE001 - Worker boundary emits only a fixed safe error before parent reaps it.
|
||||
sys.stdout.buffer.write(b'{"kind":"done","error":"JavaScript worker failed","output":[]}\n')
|
||||
sys.stdout.buffer.flush()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -313,14 +313,15 @@ async def _mcp_server_rate_limit_rejection(
|
|||
return admission.rejected_servers[0][1] if admission.rejected_servers else None
|
||||
|
||||
|
||||
async def _build_virtual_call_logging_obj(
|
||||
async def prepare_virtual_mcp_call(
|
||||
name: str,
|
||||
arguments: dict[str, object],
|
||||
user_api_key_auth: UserAPIKeyAuth,
|
||||
logging_auth: UserAPIKeyAuth | None = None,
|
||||
raw_headers: Mapping[str, str] | None = None,
|
||||
client_ip: str | None = None,
|
||||
) -> LiteLLMLoggingObj | None:
|
||||
guardrails_only: bool = False,
|
||||
) -> tuple[Mapping[str, object], LiteLLMLoggingObj | None]:
|
||||
"""Run the pre-call pipeline (guardrails + logging setup) for a virtual
|
||||
mcp_tool_call so the SSE path spend-logs like the REST path."""
|
||||
from litellm.proxy.common_request_processing import (
|
||||
|
|
@ -347,13 +348,28 @@ async def _build_virtual_call_logging_obj(
|
|||
proxy_logging_obj=proxy_logging_obj,
|
||||
general_settings=general_settings,
|
||||
logging_auth=logging_auth,
|
||||
guardrails_only=guardrails_only,
|
||||
)
|
||||
_, logging_candidate = TypeAdapter(tuple[dict[str, object], object]).validate_python(pre_call_result)
|
||||
processed_data, logging_candidate = TypeAdapter(tuple[dict[str, object], object]).validate_python(pre_call_result)
|
||||
if not isinstance(logging_candidate, LiteLLMLoggingObj):
|
||||
verbose_logger.warning("MCP request processing did not return a logging object")
|
||||
return None
|
||||
return processed_data, None
|
||||
virtual_logging_obj: Final = logging_candidate
|
||||
return virtual_logging_obj
|
||||
return processed_data, virtual_logging_obj
|
||||
|
||||
|
||||
async def _build_virtual_call_logging_obj(
|
||||
name: str,
|
||||
arguments: Mapping[str, object],
|
||||
user_api_key_auth: UserAPIKeyAuth,
|
||||
logging_auth: UserAPIKeyAuth | None = None,
|
||||
raw_headers: Mapping[str, str] | None = None,
|
||||
client_ip: str | None = None,
|
||||
) -> LiteLLMLoggingObj | None:
|
||||
_, logging_obj = await prepare_virtual_mcp_call(
|
||||
name, dict(arguments), user_api_key_auth, logging_auth, raw_headers, client_ip
|
||||
)
|
||||
return logging_obj
|
||||
|
||||
|
||||
async def _log_virtual_mcp_tool_failure(
|
||||
|
|
@ -426,6 +442,8 @@ async def _dispatch_virtual_mcp_tool(
|
|||
the caller falls through to normal tool routing.
|
||||
"""
|
||||
from litellm.llms.litellm_proxy.skills.skill_search import DEFAULT_SKILL_SEARCH_TOP_K
|
||||
from litellm.proxy.mcp_server.code_execution import CODE_TOOL_NAME, run_code_tool
|
||||
from litellm.proxy.mcp_server.code_execution import settings as code_mode_settings
|
||||
from litellm.proxy.mcp_server.tool_search import (
|
||||
AGENT_SEARCH_TOOL_NAME,
|
||||
DEFAULT_AGENT_SEARCH_TOP_K,
|
||||
|
|
@ -442,6 +460,22 @@ async def _dispatch_virtual_mcp_tool(
|
|||
handle_skill_search,
|
||||
)
|
||||
|
||||
if name == CODE_TOOL_NAME and code_mode_settings().enabled:
|
||||
return await run_code_tool(
|
||||
arguments,
|
||||
OperationContext(
|
||||
_caller=user_api_key_auth,
|
||||
mcp_auth_header=mcp_auth_header,
|
||||
mcp_servers=tuple(mcp_servers) if mcp_servers is not None else None,
|
||||
mcp_server_auth_headers=mcp_server_auth_headers,
|
||||
oauth2_headers=oauth2_headers,
|
||||
raw_headers=raw_headers,
|
||||
client_ip=client_ip,
|
||||
mcp_proxy_mode=mcp_proxy_mode,
|
||||
incoming_bearer_token=incoming_bearer_token,
|
||||
),
|
||||
)
|
||||
|
||||
if mcp_proxy_mode and name not in MCP_PROXY_TOOL_NAMES:
|
||||
return CallToolResult(
|
||||
content=[TextContent(type="text", text=f"Tool {name} is unavailable on /mcp/proxy")],
|
||||
|
|
@ -3259,6 +3293,7 @@ async def _execute_handle_list_tools(
|
|||
)
|
||||
from mcp.types import Tool
|
||||
|
||||
from litellm.proxy.mcp_server.code_execution import with_code_tool
|
||||
from litellm.proxy.mcp_server.tool_search import (
|
||||
get_mcp_proxy_tool_definitions,
|
||||
get_virtual_tool_definitions,
|
||||
|
|
@ -3267,7 +3302,9 @@ async def _execute_handle_list_tools(
|
|||
if context.mcp_proxy_mode:
|
||||
if params.cursor is not None:
|
||||
raise MCPError(code=INVALID_PARAMS, message="Invalid pagination cursor; start a fresh listing")
|
||||
return ListToolsResult(tools=[Tool.model_validate(d) for d in get_mcp_proxy_tool_definitions()])
|
||||
return ListToolsResult(
|
||||
tools=with_code_tool([Tool.model_validate(d) for d in get_mcp_proxy_tool_definitions()])
|
||||
)
|
||||
if getattr(
|
||||
getattr(user_api_key_auth, "object_permission", None),
|
||||
"mcp_tool_search_enabled",
|
||||
|
|
@ -3275,7 +3312,9 @@ async def _execute_handle_list_tools(
|
|||
):
|
||||
if params.cursor is not None:
|
||||
raise MCPError(code=INVALID_PARAMS, message="Invalid pagination cursor; start a fresh listing")
|
||||
return ListToolsResult(tools=[Tool.model_validate(d) for d in get_virtual_tool_definitions()])
|
||||
return ListToolsResult(
|
||||
tools=with_code_tool([Tool.model_validate(d) for d in get_virtual_tool_definitions()])
|
||||
)
|
||||
|
||||
from litellm.proxy.mcp_server.catalog import list_gateway_catalog
|
||||
from litellm.proxy.mcp_server.server_resolution import MCPServerTargetCatalog
|
||||
|
|
@ -3286,7 +3325,7 @@ async def _execute_handle_list_tools(
|
|||
)
|
||||
result: Final = await catalog.list(context, ListToolsRequest(params=params))
|
||||
assert isinstance(result, ListToolsResult)
|
||||
return result
|
||||
return result.model_copy(update={"tools": with_code_tool(result.tools, first_page=params.cursor is None)})
|
||||
except MCPError:
|
||||
raise
|
||||
except HTTPException as e:
|
||||
|
|
|
|||
|
|
@ -233,6 +233,7 @@ if MCP_AVAILABLE:
|
|||
)
|
||||
from litellm.mcp_client.client import MCPClient, as_mcp_read_timeout
|
||||
from litellm.proxy.mcp_server.auth.user_api_key_auth_mcp import MCPRequestHandler
|
||||
from litellm.proxy.mcp_server.code_execution import with_code_tool
|
||||
from litellm.proxy.mcp_server.mcp_server_manager import ( # noqa: F401 # legacy module exports
|
||||
_UPSTREAM_OAUTH_DISCOVERY_AUTH_TYPES, # pyright: ignore[reportPrivateUsage,reportUnusedImport] # backwards-compatible package export
|
||||
UPSTREAM_OAUTH_DISCOVERY_AUTH_TYPES,
|
||||
|
|
@ -819,7 +820,15 @@ if MCP_AVAILABLE:
|
|||
listed_generation,
|
||||
)
|
||||
|
||||
return _create_tool_response_objects(served_tools, server)
|
||||
from litellm.proxy.mcp_server.code_execution import settings as code_mode_settings
|
||||
from litellm.proxy.mcp_server.tool_search import with_mcp_proxy_identity
|
||||
|
||||
code_tools: Final = (
|
||||
[with_mcp_proxy_identity(tool, server.server_id) for tool in served_tools]
|
||||
if code_mode_settings().enabled
|
||||
else served_tools
|
||||
)
|
||||
return _create_tool_response_objects(code_tools, server)
|
||||
|
||||
async def fetch_pinnable_tool_catalog(
|
||||
server: MCPServer, request: Request, user_api_key_dict: UserAPIKeyAuth
|
||||
|
|
@ -968,7 +977,7 @@ if MCP_AVAILABLE:
|
|||
"message": f"Failed to get tools from server {server.name}: {e}",
|
||||
}
|
||||
return {
|
||||
"tools": list_tools_result,
|
||||
"tools": with_code_tool(list_tools_result),
|
||||
"error": None,
|
||||
"message": "Successfully retrieved tools",
|
||||
}
|
||||
|
|
@ -1067,7 +1076,12 @@ if MCP_AVAILABLE:
|
|||
)
|
||||
|
||||
return {
|
||||
"tools": get_virtual_tool_definitions(),
|
||||
"tools": tuple(
|
||||
tool.model_dump(by_alias=True, exclude_none=True)
|
||||
for tool in with_code_tool(
|
||||
[MCPTool.model_validate(item) for item in get_virtual_tool_definitions()]
|
||||
)
|
||||
),
|
||||
"error": None,
|
||||
"message": "Successfully retrieved tools",
|
||||
}
|
||||
|
|
@ -1184,7 +1198,7 @@ if MCP_AVAILABLE:
|
|||
else None
|
||||
)
|
||||
return {
|
||||
"tools": list_tools_result,
|
||||
"tools": with_code_tool(list_tools_result),
|
||||
"error": "partial_failure" if error_message else None,
|
||||
"message": (error_message if error_message else "Successfully retrieved tools"),
|
||||
"server_outcomes": {key: outcome_wire_value(outcome) for key, outcome in server_outcomes.items()},
|
||||
|
|
@ -1344,6 +1358,32 @@ if MCP_AVAILABLE:
|
|||
tool_name: Final[str | None] = data.get("name")
|
||||
tool_arguments: Final[dict[str, object]] = data.get("arguments") or {}
|
||||
|
||||
from litellm.proxy.mcp_server.code_execution import CODE_TOOL_NAME, run_code_tool
|
||||
from litellm.proxy.mcp_server.code_execution import settings as code_mode_settings
|
||||
from litellm.proxy.mcp_server.contracts import OperationContext
|
||||
|
||||
if tool_name == CODE_TOOL_NAME and code_mode_settings().enabled:
|
||||
code_data: Final = TypeAdapter(dict[str, object]).validate_python(data)
|
||||
code_toolset: Final = TypeAdapter(str | None).validate_python(code_data.get("toolset_name"))
|
||||
code_auth: Final = await _resolve_acting_auth(code_toolset, user_api_key_dict)
|
||||
code_header, code_server_headers, code_raw_headers, code_oauth = _extract_mcp_headers_from_request(
|
||||
request, code_auth
|
||||
)
|
||||
code_server: Final = TypeAdapter(str | None).validate_python(code_data.get("server_id"))
|
||||
return await run_code_tool(
|
||||
tool_arguments,
|
||||
OperationContext(
|
||||
_caller=code_auth,
|
||||
mcp_auth_header=code_header,
|
||||
mcp_servers=(code_server,) if code_server is not None else None,
|
||||
mcp_server_auth_headers=code_server_headers,
|
||||
raw_headers=code_raw_headers,
|
||||
oauth2_headers=code_oauth,
|
||||
incoming_bearer_token=MCPRequestHandler.get_guardrail_bearer_token(request.headers, code_auth),
|
||||
client_ip=IPAddressUtils.get_mcp_client_ip(request),
|
||||
),
|
||||
)
|
||||
|
||||
from litellm.proxy.mcp_server.tool_search import VIRTUAL_TOOL_NAMES
|
||||
|
||||
if tool_name in VIRTUAL_TOOL_NAMES:
|
||||
|
|
|
|||
|
|
@ -139,6 +139,7 @@ def coerce_top_k(value: Any, default: int = 5) -> int:
|
|||
|
||||
|
||||
class ToolSearchResult(TypedDict, total=False):
|
||||
code_mode: ReadOnly[Mapping[str, JsonValue]]
|
||||
name: Required[ReadOnly[str]]
|
||||
description: Required[ReadOnly[str]]
|
||||
inputSchema: Required[ReadOnly[Mapping[str, object]]]
|
||||
|
|
@ -146,6 +147,7 @@ class ToolSearchResult(TypedDict, total=False):
|
|||
|
||||
|
||||
class MCPProxySearchResult(TypedDict, total=False):
|
||||
code_mode: ReadOnly[Mapping[str, JsonValue]]
|
||||
tool_id: Required[ReadOnly[str]]
|
||||
name: Required[ReadOnly[str]]
|
||||
description: Required[ReadOnly[str]]
|
||||
|
|
@ -186,20 +188,18 @@ def mcp_tool_search_settings() -> MCPToolSearchSettings | ValidationError:
|
|||
|
||||
|
||||
def _tool_result(tool: Tool) -> ToolSearchResult:
|
||||
return {
|
||||
from litellm.proxy.mcp_server.code_execution import compatibility, settings
|
||||
|
||||
base: Final[ToolSearchResult] = {
|
||||
"name": tool.name,
|
||||
"description": tool.description or "",
|
||||
"inputSchema": tool.input_schema,
|
||||
}
|
||||
return {**base, "code_mode": compatibility(tool)} if settings().enabled else base
|
||||
|
||||
|
||||
def _scored_result(tool: Tool, score: float) -> ToolSearchResult:
|
||||
return {
|
||||
"name": tool.name,
|
||||
"description": tool.description or "",
|
||||
"inputSchema": tool.input_schema,
|
||||
"score": score,
|
||||
}
|
||||
return {**_tool_result(tool), "score": score}
|
||||
|
||||
|
||||
_MCP_PROXY_IDENTITY_META_KEY: Final[str] = "litellm.ai/proxy_tool_identity"
|
||||
|
|
@ -228,24 +228,32 @@ def mcp_proxy_tool_id(tool: Tool) -> str:
|
|||
|
||||
|
||||
def _proxy_search_result(hit: MCPToolSearchHit) -> MCPProxySearchResult:
|
||||
from litellm.proxy.mcp_server.code_execution import compatibility, settings
|
||||
|
||||
base: Final[MCPProxySearchResult] = {
|
||||
"tool_id": mcp_proxy_tool_id(hit.tool),
|
||||
"name": hit.tool.name,
|
||||
"description": hit.tool.description or "",
|
||||
}
|
||||
return {**base, "score": hit.score} if hit.score is not None else base
|
||||
annotated: Final[MCPProxySearchResult] = (
|
||||
{**base, "code_mode": compatibility(hit.tool)} if settings().enabled else base
|
||||
)
|
||||
return {**annotated, "score": hit.score} if hit.score is not None else annotated
|
||||
|
||||
|
||||
def _proxy_schema_result(tool: Tool) -> MCPProxySchemaResult:
|
||||
from litellm.proxy.mcp_server.code_execution import compatibility, settings
|
||||
|
||||
base: Final[MCPProxySchemaResult] = {
|
||||
"tool_id": mcp_proxy_tool_id(tool),
|
||||
"name": tool.name,
|
||||
"description": tool.description or "",
|
||||
"inputSchema": tool.input_schema,
|
||||
}
|
||||
annotated: Final[MCPProxySchemaResult] = {**base, "code_mode": compatibility(tool)} if settings().enabled else base
|
||||
if tool.output_schema is None:
|
||||
return base
|
||||
return {**base, "outputSchema": tool.output_schema}
|
||||
return annotated
|
||||
return {**annotated, "outputSchema": tool.output_schema}
|
||||
|
||||
|
||||
def _tool_text(tool: Tool) -> str:
|
||||
|
|
|
|||
|
|
@ -6950,6 +6950,10 @@ class ProxyConfig:
|
|||
reset_color_code,
|
||||
)
|
||||
setattr(litellm, key, value)
|
||||
elif key == "mcp_code_mode":
|
||||
from litellm.proxy.mcp_server.code_execution import validate_settings
|
||||
|
||||
litellm.mcp_code_mode = validate_settings(value).model_dump()
|
||||
elif key == "upperbound_key_generate_params":
|
||||
if value is not None and isinstance(value, dict):
|
||||
for _k, _v in value.items():
|
||||
|
|
|
|||
|
|
@ -132,6 +132,13 @@ class MCPAllowedClient(LiteLLMBaseModel):
|
|||
)
|
||||
|
||||
|
||||
class MCPCodeModeSettings(LiteLLMBaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
enabled: bool = False
|
||||
excluded_tools: Mapping[str, str] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class MCPToolSearchSettings(LiteLLMBaseModel):
|
||||
"""`litellm_settings.mcp_tool_search`: how the native `mcp_tool_search` virtual tool ranks the caller's tools."""
|
||||
|
||||
|
|
|
|||
|
|
@ -82,6 +82,7 @@ proxy = [
|
|||
"azure-identity>=1.25.2,<2.0",
|
||||
"azure-storage-blob>=12.28.0,<13.0",
|
||||
"mcp>=2.3.0,<3",
|
||||
"quickjs-ng>=0.17.0.2,<0.18; (sys_platform == 'linux' and (platform_machine == 'x86_64' or platform_machine == 'aarch64' or platform_machine == 'i686')) or (sys_platform == 'darwin' and platform_machine == 'arm64') or (sys_platform == 'win32' and (platform_machine == 'AMD64' or platform_machine == 'x86'))",
|
||||
"httpx2>=2.10.0,<3",
|
||||
"pydantic>=2.12.0,<3",
|
||||
"litellm-proxy-extras==0.4.112",
|
||||
|
|
@ -249,6 +250,7 @@ e2e-dev = [
|
|||
"anthropic==0.84.0",
|
||||
"psutil==7.2.2",
|
||||
"mcp>=2.3.0,<3",
|
||||
"quickjs-ng>=0.17.0.2,<0.18; (sys_platform == 'linux' and (platform_machine == 'x86_64' or platform_machine == 'aarch64' or platform_machine == 'i686')) or (sys_platform == 'darwin' and platform_machine == 'arm64') or (sys_platform == 'win32' and (platform_machine == 'AMD64' or platform_machine == 'x86'))",
|
||||
]
|
||||
admin-mcp = [
|
||||
"litellm-admin-mcp @ https://github.com/BerriAI/liteadmin-mcp/archive/31fb096904ae05684c0779110a93b135b5601182.tar.gz#sha256=5803a20a00a18177f4591706956f31be311879543049b8b38143ad90bb1cb0a6 ; python_version >= '3.12'",
|
||||
|
|
|
|||
200
tests/integration/mcp/test_mcp_code_runtime.py
Normal file
200
tests/integration/mcp/test_mcp_code_runtime.py
Normal file
|
|
@ -0,0 +1,200 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
from typing import Final
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
from pydantic import JsonValue
|
||||
|
||||
from litellm.proxy.mcp_server.code_runtime import CodeExecutionLimits, execute_code
|
||||
|
||||
|
||||
async def _echo(tool_id: str, arguments: dict[str, JsonValue]) -> dict[str, JsonValue]:
|
||||
return {"structuredContent": arguments, "content": [], "isError": tool_id == "error"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_roundtrips_full_results_and_chains_calls() -> None:
|
||||
result = await execute_code(
|
||||
"""
|
||||
const first = await tools.call_tool({tool_id: 'echo', arguments: {id: '9007199254740993'}});
|
||||
const results = await Promise.all(['echo', 'error'].map(tool_id =>
|
||||
tools.call_tool({tool_id, arguments: first.structuredContent})));
|
||||
text(results);
|
||||
""",
|
||||
_echo,
|
||||
)
|
||||
assert result.error is None
|
||||
assert result.calls_started == 3
|
||||
assert result.output == (
|
||||
[
|
||||
{"structuredContent": {"id": "9007199254740993"}, "content": [], "isError": False},
|
||||
{"structuredContent": {"id": "9007199254740993"}, "content": [], "isError": True},
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_script_near_byte_limit_survives_json_escaping() -> None:
|
||||
source = "/*" + '\\"' * ((CodeExecutionLimits().script_bytes - 20) // 2) + "*/text(42)"
|
||||
result = await execute_code(source, _echo)
|
||||
assert result.error is None
|
||||
assert result.output == (42,)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
("code", "message"),
|
||||
[
|
||||
("syntax error !", "SyntaxError"),
|
||||
("text(missing)", "ReferenceError"),
|
||||
("throw new Error('repair me')", "repair me"),
|
||||
],
|
||||
)
|
||||
async def test_script_errors_help_the_caller_repair_the_code(code: str, message: str) -> None:
|
||||
result = await execute_code(code, _echo)
|
||||
assert result.error is not None and message in result.error
|
||||
assert result.calls_started == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("code", ["while (true) {}", "await new Promise(() => {})"])
|
||||
async def test_code_deadline_releases_worker_for_next_request(code: str) -> None:
|
||||
limits = replace(CodeExecutionLimits(), timeout_seconds=1)
|
||||
result = await execute_code(code, _echo, limits)
|
||||
assert result.error is not None
|
||||
next_result = await execute_code("text(42)", _echo)
|
||||
assert next_result.output == (42,)
|
||||
assert next_result.error is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"code",
|
||||
[
|
||||
"text(9007199254740993)",
|
||||
"text(NaN)",
|
||||
"text(undefined)",
|
||||
"text(1n)",
|
||||
"const a = {}; a.a = a; text(a)",
|
||||
"text('x'.repeat(10000))",
|
||||
"syntax error !",
|
||||
],
|
||||
)
|
||||
async def test_code_rejects_unserializable_or_oversized_output(code: str) -> None:
|
||||
result = await execute_code(code, _echo, replace(CodeExecutionLimits(), output_bytes=1024))
|
||||
assert result.error is not None
|
||||
assert result.output == ()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_rejects_unsafe_upstream_integer_without_silent_rounding() -> None:
|
||||
async def large_integer(tool_id: str, arguments: dict[str, JsonValue]) -> dict[str, JsonValue]:
|
||||
return {"value": 9007199254740993}
|
||||
|
||||
result = await execute_code("text(await tools.call_tool({tool_id:'echo', arguments:{}}))", large_integer)
|
||||
assert result.error is not None
|
||||
assert "JSON numbers" in result.error
|
||||
assert result.calls_started == 1
|
||||
assert result.output == ()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_bounds_calls_even_when_javascript_catches_errors() -> None:
|
||||
result = await execute_code(
|
||||
"for(let i=0;i<5;i++) { try { await tools.call_tool({tool_id:'echo',arguments:{}}); } catch (_) {} }",
|
||||
_echo,
|
||||
replace(CodeExecutionLimits(), max_calls=2),
|
||||
)
|
||||
assert result.calls_started == 2
|
||||
assert result.error is not None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_cancellation_cleans_pending_callbacks_and_releases_capacity() -> None:
|
||||
started: Final = anyio.Event()
|
||||
stopped: Final = anyio.Event()
|
||||
scope: Final = anyio.CancelScope()
|
||||
|
||||
async def blocking(tool_id: str, arguments: dict[str, JsonValue]) -> dict[str, JsonValue]:
|
||||
started.set()
|
||||
try:
|
||||
await anyio.sleep_forever()
|
||||
finally:
|
||||
stopped.set()
|
||||
return {}
|
||||
|
||||
async def run() -> None:
|
||||
with scope:
|
||||
await execute_code("await tools.call_tool({tool_id:'echo', arguments:{}})", blocking)
|
||||
|
||||
async with anyio.create_task_group() as group:
|
||||
group.start_soon(run)
|
||||
with anyio.fail_after(10):
|
||||
await started.wait()
|
||||
scope.cancel()
|
||||
await stopped.wait()
|
||||
result = await execute_code("text('ok')", _echo)
|
||||
assert result.error is None
|
||||
assert result.output == ("ok",)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_has_no_system_apis_and_does_not_keep_globals() -> None:
|
||||
first = await execute_code(
|
||||
"globalThis.secret='temporary'; text([typeof process, typeof require, typeof fetch]);", _echo
|
||||
)
|
||||
second = await execute_code("text(typeof globalThis.secret)", _echo)
|
||||
assert first.output == (["undefined", "undefined", "undefined"],)
|
||||
assert second.output == ("undefined",)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_does_not_retry_a_completed_action_after_failure() -> None:
|
||||
seen: Final[list[str]] = []
|
||||
|
||||
async def action(tool_id: str, arguments: dict[str, JsonValue]) -> dict[str, JsonValue]:
|
||||
seen.append(tool_id)
|
||||
return {}
|
||||
|
||||
result = await execute_code(
|
||||
"await tools.call_tool({tool_id:'write', arguments:{}}); throw new Error('failed');", action
|
||||
)
|
||||
assert result.error is not None
|
||||
assert result.calls_started == 1
|
||||
assert seen == ["write"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_memory_failure_leaves_next_execution_healthy() -> None:
|
||||
result = await execute_code(
|
||||
"const data = []; while(true) data.push(new Array(65536).fill('value'));",
|
||||
_echo,
|
||||
replace(CodeExecutionLimits(), memory_bytes=16 * 1024 * 1024, timeout_seconds=5),
|
||||
)
|
||||
assert result.error is not None
|
||||
next_result = await execute_code("text('healthy')", _echo)
|
||||
assert next_result.output == ("healthy",)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_unawaited_callbacks_are_cancelled() -> None:
|
||||
async def blocking(tool_id: str, arguments: dict[str, JsonValue]) -> dict[str, JsonValue]:
|
||||
await anyio.sleep_forever()
|
||||
return {}
|
||||
|
||||
result = await execute_code("tools.call_tool({tool_id:'echo', arguments:{}})", blocking)
|
||||
assert result.error is not None
|
||||
assert "unawaited" in result.error
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_code_timeout_retains_completed_call_count() -> None:
|
||||
result = await execute_code(
|
||||
"await tools.call_tool({tool_id:'echo',arguments:{}}); await new Promise(()=>{});",
|
||||
_echo,
|
||||
replace(CodeExecutionLimits(), timeout_seconds=1),
|
||||
)
|
||||
assert result.error is not None
|
||||
assert result.calls_started == 1
|
||||
219
tests/unit/proxy/mcp_server/test_code_execution.py
Normal file
219
tests/unit/proxy/mcp_server/test_code_execution.py
Normal file
|
|
@ -0,0 +1,219 @@
|
|||
import pytest
|
||||
from mcp.shared.exceptions import MCPError
|
||||
from mcp.types import TextContent, Tool, ToolExecution
|
||||
from pydantic import ValidationError
|
||||
|
||||
import litellm
|
||||
from litellm.proxy.mcp_server.code_execution import (
|
||||
CODE_MODE_META,
|
||||
CODE_TOOL_NAME,
|
||||
CodeInput,
|
||||
compatibility,
|
||||
run_code_tool,
|
||||
settings,
|
||||
with_code_tool,
|
||||
with_code_trace,
|
||||
)
|
||||
from litellm.proxy.mcp_server.contracts import OperationContext
|
||||
from litellm.types.mcp import MCPCodeModeSettings
|
||||
|
||||
|
||||
def test_disabled_preserves_existing_tools(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", False)
|
||||
tool = Tool.model_validate({"name": "orders_list", "inputSchema": {"type": "object"}, "_meta": {"vendor": "kept"}})
|
||||
listed = with_code_tool([tool])
|
||||
assert listed == [tool]
|
||||
assert listed[0] is tool
|
||||
|
||||
|
||||
def test_enabled_annotates_without_losing_metadata_and_adds_only_on_first_page(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
tool = Tool.model_validate({"name": "orders_list", "inputSchema": {"type": "object"}, "_meta": {"vendor": "kept"}})
|
||||
first = with_code_tool([tool])
|
||||
later = with_code_tool([tool], first_page=False)
|
||||
assert [entry.name for entry in first] == ["orders_list", CODE_TOOL_NAME]
|
||||
assert [entry.name for entry in later] == ["orders_list"]
|
||||
assert first[0].meta is not None
|
||||
assert first[0].meta["vendor"] == "kept"
|
||||
assert first[0].meta[CODE_MODE_META]["status"] == "available"
|
||||
assert tool.meta == {"vendor": "kept"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", [CODE_TOOL_NAME, "call_tool", "search_tools", "mcp_tool_call"])
|
||||
def test_orchestration_is_marked_unsupported(monkeypatch: pytest.MonkeyPatch, name: str) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
assert compatibility(Tool(name=name, input_schema={"type": "object"}))["status"] == "unsupported"
|
||||
|
||||
|
||||
def test_explicit_exclusion_keeps_direct_tool_and_explains_reason(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(
|
||||
litellm, "mcp_code_mode", {"enabled": True, "excluded_tools": {"orders_approve": "Requires human confirmation"}}
|
||||
)
|
||||
tool = Tool(name="orders_approve", input_schema={"type": "object"})
|
||||
result = with_code_tool([tool])
|
||||
assert result[0].name == tool.name
|
||||
assert result[0].meta is not None
|
||||
assert result[0].meta[CODE_MODE_META] == {"status": "unsupported", "reason": "Requires human confirmation"}
|
||||
|
||||
|
||||
def test_required_task_is_marked_unsupported(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
tool = Tool(name="long_job", input_schema={"type": "object"}, execution=ToolExecution(task_support="required"))
|
||||
assert compatibility(tool)["status"] == "unsupported"
|
||||
|
||||
|
||||
def test_reserved_name_collision_is_explicit(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
with pytest.raises(MCPError, match="reserved"):
|
||||
with_code_tool([Tool(name=CODE_TOOL_NAME, input_schema={"type": "object"})])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cached_code_tool_rejected_when_disabled(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", False)
|
||||
result = await run_code_tool({"code": "text(1)"}, OperationContext(_caller=None))
|
||||
assert result.is_error
|
||||
assert isinstance(result.content[0], TextContent)
|
||||
assert "disabled" in result.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unauthenticated_code_tool_rejected(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
result = await run_code_tool({"code": "text(1)"}, OperationContext(_caller=None))
|
||||
assert result.is_error
|
||||
assert isinstance(result.content[0], TextContent)
|
||||
assert "authenticated" in result.content[0].text
|
||||
|
||||
|
||||
def test_config_and_script_reject_unrecognized_fields(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
MCPCodeModeSettings.model_validate({"enabled": True, "allow_network": True})
|
||||
with pytest.raises(ValidationError):
|
||||
CodeInput.model_validate({"code": "text(1)", "credentials": "example"})
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
assert settings().enabled
|
||||
|
||||
|
||||
def test_code_trace_preserves_existing_session_and_isolates_new_runs() -> None:
|
||||
existing = OperationContext(_caller=None, raw_headers={"X-LiteLLM-Session-ID": "original-session"})
|
||||
assert with_code_trace(existing) is existing
|
||||
original = OperationContext(_caller=None)
|
||||
first = with_code_trace(original)
|
||||
second = with_code_trace(original)
|
||||
assert original.raw_headers is None
|
||||
assert first.raw_headers is not None
|
||||
assert second.raw_headers is not None
|
||||
assert first.raw_headers["x-litellm-trace-id"] != second.raw_headers["x-litellm-trace-id"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("proxy_mode", [True, False])
|
||||
async def test_enabled_code_tool_appears_on_existing_discovery_surfaces(
|
||||
monkeypatch: pytest.MonkeyPatch, proxy_mode: bool
|
||||
) -> None:
|
||||
from mcp.types import ListToolsRequest, ListToolsResult
|
||||
|
||||
from litellm.proxy._types import LiteLLM_ObjectPermissionTable, UserAPIKeyAuth
|
||||
from litellm.proxy.mcp_server.operations import GatewayOperations
|
||||
|
||||
auth = UserAPIKeyAuth(
|
||||
object_permission=LiteLLM_ObjectPermissionTable(
|
||||
object_permission_id="code-mode-test",
|
||||
mcp_tool_search_enabled=True,
|
||||
)
|
||||
)
|
||||
context = OperationContext(_caller=auth, mcp_proxy_mode=proxy_mode)
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", False)
|
||||
before = await GatewayOperations().execute(ListToolsRequest(), context)
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
after = await GatewayOperations().execute(ListToolsRequest(), context)
|
||||
assert isinstance(before, ListToolsResult)
|
||||
assert isinstance(after, ListToolsResult)
|
||||
assert [tool.name for tool in after.tools] == [*[tool.name for tool in before.tools], CODE_TOOL_NAME]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("canonical", ["echo", CODE_TOOL_NAME, "call_tool"])
|
||||
def test_identity_covers_display_alias_exclusion_and_nested_orchestration(
|
||||
monkeypatch: pytest.MonkeyPatch, canonical: str
|
||||
) -> None:
|
||||
from litellm.proxy.mcp_server.code_execution import tool_names
|
||||
from litellm.proxy.mcp_server.mcp_server_manager import global_mcp_server_manager
|
||||
from litellm.proxy.mcp_server.tool_search import with_mcp_proxy_identity
|
||||
from litellm.types.mcp import MCPTransport
|
||||
from litellm.types.mcp_server.mcp_server_manager import MCPServer
|
||||
|
||||
server = MCPServer(
|
||||
server_id="code-mode-identity-test",
|
||||
name="gateway",
|
||||
alias="gateway",
|
||||
transport=MCPTransport.http,
|
||||
url="https://example.com/mcp",
|
||||
tool_name_to_display_name={canonical: "Friendly tool"},
|
||||
)
|
||||
registry = global_mcp_server_manager.registry
|
||||
previous = registry.get(server.server_id)
|
||||
registry[server.server_id] = server
|
||||
try:
|
||||
proxy_tool = with_mcp_proxy_identity(Tool(name=f"gateway-{canonical}", input_schema={}), server.server_id)
|
||||
displayed = proxy_tool.model_copy(update={"name": "Friendly tool"})
|
||||
assert {canonical, f"gateway-{canonical}", "Friendly tool"} <= tool_names(proxy_tool)
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", True)
|
||||
expected = "available" if canonical == "echo" else "unsupported"
|
||||
assert compatibility(proxy_tool)["status"] == expected
|
||||
assert compatibility(displayed)["status"] == expected
|
||||
monkeypatch.setattr(
|
||||
litellm, "mcp_code_mode", {"enabled": True, "excluded_tools": {"Friendly tool": "Needs confirmation"}}
|
||||
)
|
||||
assert compatibility(proxy_tool)["status"] == "unsupported"
|
||||
assert compatibility(displayed)["status"] == "unsupported"
|
||||
finally:
|
||||
if previous is None:
|
||||
del registry[server.server_id]
|
||||
else:
|
||||
registry[server.server_id] = previous
|
||||
|
||||
|
||||
def test_worker_serialization_preserves_resource_uris_and_full_mcp_envelope() -> None:
|
||||
from mcp.types import CallToolResult, EmbeddedResource, ResourceLink, TextResourceContents
|
||||
from pydantic import AnyUrl
|
||||
|
||||
from litellm.proxy.mcp_server.code_execution import serialize_tool_result
|
||||
|
||||
result = CallToolResult(
|
||||
content=[
|
||||
ResourceLink(type="resource_link", name="report", uri="https://example.com/report"),
|
||||
EmbeddedResource(
|
||||
type="resource",
|
||||
resource=TextResourceContents(uri="file:///report.txt", text="Report contents"),
|
||||
),
|
||||
],
|
||||
structured_content={"total": 42},
|
||||
is_error=False,
|
||||
)
|
||||
with_metadata = result.model_copy(update={"meta": {"source": AnyUrl("https://example.com/source")}})
|
||||
serialized = serialize_tool_result(with_metadata)
|
||||
assert serialized == {
|
||||
"content": [
|
||||
{"type": "resource_link", "name": "report", "uri": "https://example.com/report"},
|
||||
{"type": "resource", "resource": {"uri": "file:///report.txt", "text": "Report contents"}},
|
||||
],
|
||||
"structuredContent": {"total": 42},
|
||||
"_meta": {"source": "https://example.com/source"},
|
||||
"resultType": "complete",
|
||||
"isError": False,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_disabled_code_name_falls_through_to_existing_tool_routing(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
from litellm.proxy.mcp_server.operations import _dispatch_virtual_mcp_tool
|
||||
|
||||
monkeypatch.setattr(litellm, "mcp_code_mode", False)
|
||||
result = await _dispatch_virtual_mcp_tool(
|
||||
name=CODE_TOOL_NAME,
|
||||
arguments={"message": "ordinary upstream arguments"},
|
||||
user_api_key_auth=None,
|
||||
client_ip=None,
|
||||
)
|
||||
assert result is None
|
||||
59
tests/unit/proxy/mcp_server/test_code_runtime.py
Normal file
59
tests/unit/proxy/mcp_server/test_code_runtime.py
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
import pytest
|
||||
|
||||
from litellm.proxy.mcp_server.code_worker import Reply, encode_message, load_json
|
||||
|
||||
|
||||
def test_code_mode_rejects_archived_quickjs_package(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
from litellm.proxy.mcp_server import code_runtime
|
||||
|
||||
monkeypatch.setattr(code_runtime.metadata, "distribution", lambda name: None)
|
||||
with pytest.raises(ValueError, match="conflicting quickjs package"):
|
||||
code_runtime.check_runtime()
|
||||
|
||||
|
||||
def test_code_mode_reports_missing_runtime_dependency(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
from litellm.proxy.mcp_server import code_runtime
|
||||
|
||||
def missing(name: str) -> None:
|
||||
raise code_runtime.metadata.PackageNotFoundError(name)
|
||||
|
||||
monkeypatch.setattr(code_runtime.metadata, "distribution", missing)
|
||||
with pytest.raises(ValueError, match="requires quickjs-ng"):
|
||||
code_runtime.check_runtime()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [2**53, -(2**53), float(2**53), math.inf, math.nan])
|
||||
def test_tool_result_numbers_are_never_silently_changed(value: float) -> None:
|
||||
with pytest.raises(ValueError, match="JSON"):
|
||||
encode_message(Reply(id=0, value={"id": value}), 4096)
|
||||
|
||||
|
||||
def test_large_ids_as_strings_and_exact_safe_integers_survive() -> None:
|
||||
value = {"large": str(2**63), "safe": 2**53 - 1}
|
||||
result = Reply.model_validate(load_json(encode_message(Reply(id=7, value=value), 4096).decode()))
|
||||
assert result.value == value
|
||||
assert result.id == 7
|
||||
|
||||
|
||||
def test_pipe_limit_counts_unicode_bytes() -> None:
|
||||
with pytest.raises(ValueError, match="byte limit"):
|
||||
encode_message(Reply(id=0, value={"value": "猫" * 100}), 200)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("code", ["\ud800", " " * (128 * 1024 + 1)])
|
||||
async def test_invalid_script_fails_before_starting_a_worker(code: str) -> None:
|
||||
from pydantic import JsonValue
|
||||
|
||||
from litellm.proxy.mcp_server.code_runtime import execute_code
|
||||
|
||||
async def unused(tool_id: str, arguments: dict[str, JsonValue]) -> dict[str, JsonValue]:
|
||||
pytest.fail("Invalid code must never execute tools")
|
||||
|
||||
result = await execute_code(code, unused)
|
||||
assert result.error is not None
|
||||
assert result.calls_started == 0
|
||||
31
uv.lock
generated
31
uv.lock
generated
|
|
@ -5050,6 +5050,7 @@ proxy = [
|
|||
{ name = "pyroscope-io", marker = "sys_platform != 'win32'" },
|
||||
{ name = "python-multipart" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "quickjs-ng", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'i686' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux') or (platform_machine == 'AMD64' and sys_platform == 'win32') or (platform_machine == 'x86' and sys_platform == 'win32')" },
|
||||
{ name = "restrictedpython" },
|
||||
{ name = "rich" },
|
||||
{ name = "rq" },
|
||||
|
|
@ -5192,6 +5193,7 @@ e2e-dev = [
|
|||
{ name = "mcp" },
|
||||
{ name = "playwright" },
|
||||
{ name = "psutil" },
|
||||
{ name = "quickjs-ng", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'i686' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux') or (platform_machine == 'AMD64' and sys_platform == 'win32') or (platform_machine == 'x86' and sys_platform == 'win32')" },
|
||||
{ name = "websockets" },
|
||||
]
|
||||
healthcheck = [
|
||||
|
|
@ -5308,6 +5310,7 @@ requires-dist = [
|
|||
{ name = "pyyaml", specifier = ">=6.0.3,<7.0" },
|
||||
{ name = "pyyaml", marker = "extra == 'cli'", specifier = ">=6.0.3,<7.0" },
|
||||
{ name = "pyyaml", marker = "extra == 'proxy'", specifier = ">=6.0.3,<7.0" },
|
||||
{ name = "quickjs-ng", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin' and extra == 'proxy') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'proxy') or (platform_machine == 'i686' and sys_platform == 'linux' and extra == 'proxy') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'proxy') or (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'proxy') or (platform_machine == 'x86' and sys_platform == 'win32' and extra == 'proxy')", specifier = ">=0.17.0.2,<0.18" },
|
||||
{ name = "redisvl", marker = "extra == 'extra-proxy'", specifier = ">=0.4.1,<1.0" },
|
||||
{ name = "requests", marker = "extra == 'cli'", specifier = ">=2.32.0,<3.0" },
|
||||
{ name = "resend", marker = "extra == 'extra-proxy'", specifier = ">=2.23.0,<3.0" },
|
||||
|
|
@ -5420,6 +5423,7 @@ e2e-dev = [
|
|||
{ name = "mcp", specifier = ">=2.3.0,<3" },
|
||||
{ name = "playwright", specifier = "==1.61.0" },
|
||||
{ name = "psutil", specifier = "==7.2.2" },
|
||||
{ name = "quickjs-ng", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (platform_machine == 'i686' and sys_platform == 'linux') or (platform_machine == 'x86_64' and sys_platform == 'linux') or (platform_machine == 'AMD64' and sys_platform == 'win32') or (platform_machine == 'x86' and sys_platform == 'win32')", specifier = ">=0.17.0.2,<0.18" },
|
||||
{ name = "websockets", specifier = ">=15.0.1,<16.0" },
|
||||
]
|
||||
healthcheck = [
|
||||
|
|
@ -9321,6 +9325,33 @@ wheels = [
|
|||
{ url = "https://files.pythonhosted.org/packages/01/1b/5dbe84eefc86f48473947e2f41711aded97eecef1231f4558f1f02713c12/pyzmq-27.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:c9f7f6e13dff2e44a6afeaf2cf54cee5929ad64afaf4d40b50f93c58fc687355", size = 544862, upload-time = "2025-09-08T23:09:56.509Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "quickjs-ng"
|
||||
version = "0.17.0.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d6/a4/611b1c2e3a505dd66cc03fb6e5963f0550a09ef2bd7c51243a9237d64db8/quickjs_ng-0.17.0.2.tar.gz", hash = "sha256:43d2e48272208721a7aaea8e3f02d4b9318f12ef20cc4f5421b59d9a470706ef", size = 548468, upload-time = "2026-10-22T23:10:26.261Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/da/36/7e91b954af2229a3bd257bd2d380c0c740622a8ac49094f9c81ace850d8c/quickjs_ng-0.17.0.2-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:80dd1d2d4b83eef6bdf9ce85de020b99b8a641e52c809ba4b2193535a68dbd13", size = 533474, upload-time = "2026-10-22T23:09:52.662Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/54/1e/f53ee49cb88c1a75f9abea0a1148d024bea1cce09cdf46a9c537ef39c79f/quickjs_ng-0.17.0.2-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3f87be3bdadc5fae7bd1f1d711838b35ef850fdbd6e55bed2a7d3ebc76c1456d", size = 2470446, upload-time = "2026-10-22T23:09:54.82Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bf/25/c4ae89980638fc9e81cfeab0ed876e8ce08f2d821a915c02409660c7010d/quickjs_ng-0.17.0.2-cp310-abi3-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:88be7657cb5983e99a2ae3736ee145f00604cff69ccac45e4e1bd21dfb46cff5", size = 2415266, upload-time = "2026-10-22T23:09:57.179Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0f/ad/8afe9aba7dbca9d6da0e92599f6f31bd5acb11d8c8c8c5a476292c2992ef/quickjs_ng-0.17.0.2-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:28cf98a33bf4c58f9d59957d0d71d8f7c72d9b85d80bdf63d0229b176f4353cd", size = 2550993, upload-time = "2026-10-22T23:09:59.61Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/ab/a9447dc667d2c173973e07afb317ac19dc53e50e2ddbaac5d4a3673d5acb/quickjs_ng-0.17.0.2-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:718350f3ed2dee04ee21e8d22abde8490196c6746826c8c3b91c5664948238b7", size = 2454139, upload-time = "2026-10-22T23:10:01.295Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/43/c1/7ca4834725d292db566de8583e5f6c8240aec706e2fce5e5a8bc7496c802/quickjs_ng-0.17.0.2-cp310-abi3-musllinux_1_2_i686.whl", hash = "sha256:77bcbb35340142a2ad9c9caa40bdb6908ec44d12644ca27209f566c513fd3aef", size = 2444620, upload-time = "2026-10-22T23:10:03.349Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d3/b0/4b051ba1f738575fb062bbeb9144413159b833cc98061d7b102f97111c25/quickjs_ng-0.17.0.2-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:bc3c4f2b2542ff624c74520670f4785a55f061cd7aaaaedef06ab6d4e18a7fed", size = 2539126, upload-time = "2026-10-22T23:10:04.998Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/8d/6b72f2b02c5eebaad3f34ea2e39f5b871ae2b3e5f1c342cfdc474dc8c1c8/quickjs_ng-0.17.0.2-cp310-abi3-win32.whl", hash = "sha256:2303c84092a0ed62f4068f2d4c1a273fd6833509ceb53da72266f96ced7b4dfe", size = 395466, upload-time = "2026-10-22T23:10:06.52Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d0/0e/3923bd3bc864e107219ce1e6f1a156a48743f4c73b008109440889bbb727/quickjs_ng-0.17.0.2-cp310-abi3-win_amd64.whl", hash = "sha256:3052711482264fd98563a94a55444280659b955155506f333169d9dfd19db892", size = 519428, upload-time = "2026-10-22T23:10:08.132Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fc/55/cd1c92343a3229d9f898eea39078f44b4b121f979b3165eaecb2fd6bb217/quickjs_ng-0.17.0.2-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:d6d68cbf31cb61202c724864a5a88a18f1852281504ab458fc7dfc51b18277b6", size = 477630, upload-time = "2026-10-22T23:10:09.706Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e8/d7/c79a878c11b616a44242ed86d19c2d2974608b3a372eb03a664f789dbb3d/quickjs_ng-0.17.0.2-pp310-pypy310_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c5eea68660c01b5f96a2e793597c029fc78636aa979146b7843a57214d5baba7", size = 524419, upload-time = "2026-10-22T23:10:11.271Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3f/70/6d1edf61300be1d4c4991b6ba8797de5b084df5b32029bc1631657d2ceb7/quickjs_ng-0.17.0.2-pp310-pypy310_pp73-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:e034627c4edeb01de2db1d228636b9bf0b2992b9bd3a5e3b3d7d5a0a38f0e182", size = 597468, upload-time = "2026-10-22T23:10:12.73Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4b/f1/fae8b69cdefd0943cef4efd6bb2486624a8c8b033e413c8760422d80df3d/quickjs_ng-0.17.0.2-pp310-pypy310_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1893c1a6326d9ca4fb18aceaa8abf588ec14634a2bb23d6b54abdf86bcf4d4d5", size = 556861, upload-time = "2026-10-22T23:10:15.157Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c8/24/280439244966c753c312c8a043df76b5303d0316e8ee76b77b23f08a7302/quickjs_ng-0.17.0.2-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:652616953bca04b9b77a9d474d0e7ef24ea42c7f44422bf55b2741f60052da75", size = 519542, upload-time = "2026-10-22T23:10:16.513Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/19/f53ecf341e92f23bdfd525ca288a0192b83bb6ee09a13f8aa2f6fc937ed0/quickjs_ng-0.17.0.2-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:6e11fd97f9c5032f83e7d636f252756c0011a984d7d3c1aa3cfca1a49d800b4d", size = 477550, upload-time = "2026-10-22T23:10:18.197Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/89/0d/7139242a89576787f306e24c9eb8c3aaf17de122867a8810e6bdf9eec480/quickjs_ng-0.17.0.2-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:52f28a20c92e62b4282e808ef96d6cbbc4899fc8faa4a6754b3fa35b2fb4d4f9", size = 524419, upload-time = "2026-10-22T23:10:19.771Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/56/210bbb549e11d149a924e693a6cb0a7b8b690b76b69cd8d301395675adb4/quickjs_ng-0.17.0.2-pp311-pypy311_pp73-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:530f1d07be7a72bcb1241d48eb21f4b68ef1c79613913eae4f86b1aede73b04c", size = 597467, upload-time = "2026-10-22T23:10:21.269Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b9/95/f4ab8a88807a2a9ee50034cce275dd50fa7898ebeb7d0e62bbe8fd3aef52/quickjs_ng-0.17.0.2-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4539c777fe30833dd7b405e965079bee35c614733f45a8f21ee64f55444c35cd", size = 556862, upload-time = "2026-10-22T23:10:22.832Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9b/f0/169e12d85c832314dc8facf04e3e3ccc2636678991d43b11eb7ef170c9f0/quickjs_ng-0.17.0.2-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:1920f007f00f16da4d8ab110110b1b2c9e19817775e837b45079d8f07b5ef96d", size = 519546, upload-time = "2026-10-22T23:10:24.813Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redis"
|
||||
version = "5.3.1"
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue