mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
feat(otel): emit a tools/list CLIENT span for MCP discovery under otel_v2 (#31525)
* feat(otel): emit a tools/list CLIENT span for MCP discovery under otel_v2
Under otel_v2 an MCP tools/call already produced a dedicated CLIENT span, but tools/list produced none. The discovery call surfaced only as the bare POST /{mcp_server_name}/mcp server span with no MCP attributes, indistinguishable from initialize and impossible to query by method
The list success event already reaches the v2 logger with call_type list_mcp_tools, but _emit_mcp_tool_call only matched call_mcp_tool, so listing fell through to the LLM-call path and emitted nothing. This adds a dedicated MCP_LIST_TOOLS span role with its own MCPListToolsSpanData, emitted from a sibling _emit_mcp_list_tools branch that mirrors the tools/call path
Per the OTel GenAI MCP semantic conventions the span is named tools/list (the method name alone, since there is no low-cardinality target), is a CLIENT span parented to the request span, and carries mcp.method.name plus the call id. It deliberately omits gen_ai.operation.name and gen_ai.tool.name, which the convention reserves for tool executions, since listing runs no tool
* fix(otel): anchor MCP spans to params._meta trace context, not the transport span
MCP streamable-HTTP multiplexes many JSON-RPC messages over one session, so the request-root anchor captured on initialize persisted and every later message's span (tools/call, tools/list) nested under it. A tools/list run 44s after the initialize rendered 44s to the right of its parent with a clock-skew warning, because the MCP message and the HTTP transport are independent lifecycles
Following the OTel GenAI MCP semantic conventions, an MCP span now parents to the W3C trace context the client propagated in the request's params._meta (a remote parent, per SEP-414), records the transport/session span as a span link rather than the parent, and starts its own root trace when nothing was propagated. The MCP gateway captures traceparent/tracestate/baggage from each message's params._meta into a per-message contextvar that the otel_v2 emitter reads; opentelemetry stays an optional dependency via guarded lazy imports
This applies to tools/call as well as the new tools/list span, since both shared the same transport-anchoring bug
* fix(otel): drop client baggage from MCP params._meta to prevent identity spoofing
The MCP trace propagation added a W3CBaggagePropagator, so resolve_mcp_span_context
extracted the client's W3C Baggage from params._meta into the span's parent context.
The LiteLLMBaggageSpanProcessor then stamps allowlisted baggage keys onto the span,
and the list-tools/tool-call mappers don't set those identity keys, so nothing
overwrites them. A malicious MCP client could send
params._meta.baggage: litellm.team.id=...,litellm.metadata.user_api_key_user_id=...
and have those identity attributes attributed to its spans.
Extract trace context only (traceparent/tracestate) in the propagator, and stop
collecting the baggage key at the source in _mcp_meta_trace_carrier. Parenting to the
client's trace context, the actual goal, needs only trace context; remote baggage had
no legitimate consumer here. Regression tests at both layers assert a spoofed
params._meta.baggage never lands as a span identity attribute.
* style(mcp): clear ruff strict-budget breach in otel trace-carrier helpers
The otel MCP trace-carrier helpers added in this branch pushed the BLE001 and
UP006 strict-rule totals past their ceilings. Use PEP 585 `dict[str, str]` instead
of `Dict`, and narrow the optional-import guards to `except ImportError` (the only
failure these can hit, matching the "when otel_v2 is unavailable" intent) instead of
a blind `except Exception`.
* fix(otel): stamp authenticated identity baggage onto MCP spans
Parenting MCP spans to the client's params._meta trace context over an empty
Context() meant the tool-call and tools/list spans carried no team/key/metadata
identity at all, so they couldn't be attributed or filtered by team in a traces
backend. The LLM-call span already re-seeds identity from the parsed, authenticated
StandardLoggingPayload rather than trusting ambient/remote context; extract that into
a shared _seed_identity_baggage helper and run both MCP emitters through it.
Identity comes only from the authenticated payload, never the client carrier, so this
keeps the earlier spoofing fix intact while restoring attribution. Regression tests
assert the authenticated team lands on both MCP spans and that a spoofed
params._meta.baggage value can't override it.
* refactor(otel): model MCP spans as roots that link the transport in SPAN_REGISTRY
This commit is contained in:
ryan-crabbe-berri
GitHub
parent
2e575d39f2
commit
468d11f71d
12 changed files with 547 additions and 31 deletions
|
|
@ -32,11 +32,13 @@ from litellm.integrations.otel.model.payloads import (
|
|||
LLMCallSpanData,
|
||||
LLMRequestParams,
|
||||
LLMUsage,
|
||||
MCPListToolsSpanData,
|
||||
MCPToolCallSpanData,
|
||||
ProxyRequestSpanData,
|
||||
ServerInfo,
|
||||
ServiceSpanData,
|
||||
SpanError,
|
||||
is_mcp_list_tools,
|
||||
is_mcp_tool_call,
|
||||
)
|
||||
from litellm.integrations.otel.model.semconv import (
|
||||
|
|
@ -106,6 +108,7 @@ __all__ = [
|
|||
"LLMCallSpanData",
|
||||
"LLMRequestParams",
|
||||
"LLMUsage",
|
||||
"MCPListToolsSpanData",
|
||||
"MCPToolCallSpanData",
|
||||
"ProxyRequestSpanData",
|
||||
"RequestContext",
|
||||
|
|
@ -113,6 +116,7 @@ __all__ = [
|
|||
"ServerInfo",
|
||||
"ServiceSpanData",
|
||||
"SpanError",
|
||||
"is_mcp_list_tools",
|
||||
"is_mcp_tool_call",
|
||||
"promoted_baggage",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ from collections import OrderedDict
|
|||
from typing import Callable, Sequence
|
||||
|
||||
from opentelemetry.context import Context
|
||||
from opentelemetry.trace import Span, Tracer
|
||||
from opentelemetry.trace import Link, Span, Tracer
|
||||
from opentelemetry.trace.status import Status, StatusCode
|
||||
|
||||
from litellm.integrations.otel.model.config import OpenTelemetryV2Config
|
||||
|
|
@ -13,6 +13,7 @@ from litellm.integrations.otel.mappers.base import AttributeMapper, SpanData
|
|||
from litellm.integrations.otel.model.payloads import (
|
||||
GuardrailSpanData,
|
||||
LLMCallSpanData,
|
||||
MCPListToolsSpanData,
|
||||
MCPToolCallSpanData,
|
||||
ServiceSpanData,
|
||||
)
|
||||
|
|
@ -23,6 +24,7 @@ from litellm.integrations.otel.model.spans import (
|
|||
SpanRole,
|
||||
guardrail_span_name,
|
||||
llm_call_span_name,
|
||||
mcp_list_tools_span_name,
|
||||
mcp_tool_call_span_name,
|
||||
service_span_name,
|
||||
)
|
||||
|
|
@ -33,6 +35,7 @@ from litellm.integrations.otel.model.spans import (
|
|||
_NAME_BUILDERS: dict[SpanRole, Callable[..., str]] = {
|
||||
SpanRole.LLM_CALL: llm_call_span_name,
|
||||
SpanRole.MCP_TOOL_CALL: mcp_tool_call_span_name,
|
||||
SpanRole.MCP_LIST_TOOLS: mcp_list_tools_span_name,
|
||||
SpanRole.GUARDRAIL: guardrail_span_name,
|
||||
# DB_CALL and SERVICE are both built from ServiceSpanData; they differ only in
|
||||
# span kind (CLIENT vs INTERNAL) and attribute vocabulary, not in naming.
|
||||
|
|
@ -74,18 +77,21 @@ class SpanEmitter:
|
|||
start_time_ns: int | None = None,
|
||||
*,
|
||||
tracer: Tracer | None = None,
|
||||
links: Sequence[Link] | None = None,
|
||||
) -> Span:
|
||||
"""Start a span for ``role`` without dedup or attribute mapping.
|
||||
|
||||
For callers that own and manage their own span lifecycle. ``tracer``
|
||||
overrides the bound tracer for this span only, used for per-request
|
||||
multi-tenant credential routing.
|
||||
multi-tenant credential routing. ``links`` records related-but-not-parent
|
||||
spans (e.g. the transport span of an MCP message, per MCP semconv).
|
||||
"""
|
||||
return (tracer or self._tracer).start_span(
|
||||
name,
|
||||
context=parent_context,
|
||||
kind=to_otel_span_kind(SPAN_REGISTRY[role].kind),
|
||||
start_time=start_time_ns,
|
||||
links=list(links) if links else None,
|
||||
)
|
||||
|
||||
def _seen(self, dedup_key: str | None, role: SpanRole) -> bool:
|
||||
|
|
@ -116,16 +122,23 @@ class SpanEmitter:
|
|||
start_time_ns: int | None = None,
|
||||
end_time_ns: int | None = None,
|
||||
tracer: Tracer | None = None,
|
||||
links: Sequence[Link] | None = None,
|
||||
) -> Span | None:
|
||||
"""Emit one complete span: dedup, start, map attributes, status, end.
|
||||
|
||||
Return the span, or ``None`` if it was deduplicated away. ``tracer``
|
||||
overrides the bound tracer for this span, used for per-request routing.
|
||||
``links`` records related-but-not-parent spans (the transport span of an
|
||||
MCP message).
|
||||
"""
|
||||
# LLM-call and MCP tool-call spans carry a dedup key (their request's
|
||||
# call id), so a sync+async double-firing coalesces. ``isinstance`` narrows
|
||||
# the type for mypy and keeps the engine free of duck-typed attribute reads.
|
||||
dedup_key = data.identity.call_id if isinstance(data, (LLMCallSpanData, MCPToolCallSpanData)) else None
|
||||
dedup_key = (
|
||||
data.identity.call_id
|
||||
if isinstance(data, (LLMCallSpanData, MCPToolCallSpanData, MCPListToolsSpanData))
|
||||
else None
|
||||
)
|
||||
if self._seen(dedup_key, role):
|
||||
return None
|
||||
span = self.start_span(
|
||||
|
|
@ -134,6 +147,7 @@ class SpanEmitter:
|
|||
parent_context=parent_context,
|
||||
start_time_ns=start_time_ns,
|
||||
tracer=tracer,
|
||||
links=links,
|
||||
)
|
||||
self.finish_span(role, span, data, end_time_ns=end_time_ns)
|
||||
return span
|
||||
|
|
@ -166,6 +180,7 @@ class SpanEmitter:
|
|||
(
|
||||
LLMCallSpanData,
|
||||
MCPToolCallSpanData,
|
||||
MCPListToolsSpanData,
|
||||
ServiceSpanData,
|
||||
GuardrailSpanData,
|
||||
),
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ from contextlib import contextmanager
|
|||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING, Any, Callable, Iterator, Mapping, Sequence, cast
|
||||
|
||||
from opentelemetry.context import attach, get_current
|
||||
from opentelemetry.context import Context, attach, get_current
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.trace import Span, Tracer, get_current_span, use_span
|
||||
|
||||
|
|
@ -17,6 +17,7 @@ from litellm.integrations.otel.model.config import OpenTelemetryV2Config
|
|||
from litellm.integrations.otel.plumbing.context import (
|
||||
is_recordable_span,
|
||||
request_root_span,
|
||||
resolve_mcp_span_context,
|
||||
resolve_parent_context,
|
||||
resolve_request_span_context,
|
||||
set_request_baggage,
|
||||
|
|
@ -32,9 +33,11 @@ from litellm.integrations.otel.model.metadata import (
|
|||
from litellm.integrations.otel.model.payloads import (
|
||||
GuardrailSpanData,
|
||||
LLMCallSpanData,
|
||||
MCPListToolsSpanData,
|
||||
MCPToolCallSpanData,
|
||||
ServiceSpanData,
|
||||
SpanError,
|
||||
is_mcp_list_tools,
|
||||
is_mcp_tool_call,
|
||||
)
|
||||
from litellm.integrations.otel.plumbing.metrics import (
|
||||
|
|
@ -218,6 +221,8 @@ class OpenTelemetryV2(CustomLogger):
|
|||
async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
|
||||
if self._emit_mcp_tool_call(kwargs, start_time, end_time):
|
||||
return
|
||||
if self._emit_mcp_list_tools(kwargs, start_time, end_time):
|
||||
return
|
||||
self._close_llm_call(kwargs, start_time, end_time)
|
||||
self._record_metrics(kwargs, response_obj, start_time, end_time)
|
||||
|
||||
|
|
@ -242,8 +247,24 @@ class OpenTelemetryV2(CustomLogger):
|
|||
async def async_log_failure_event(self, kwargs, response_obj, start_time, end_time):
|
||||
if self._emit_mcp_tool_call(kwargs, start_time, end_time):
|
||||
return
|
||||
if self._emit_mcp_list_tools(kwargs, start_time, end_time):
|
||||
return
|
||||
self._close_llm_call(kwargs, start_time, end_time)
|
||||
|
||||
def _seed_identity_baggage(self, identity: RequestIdentity, model: str | None, context: Context) -> Context:
|
||||
"""Seed authenticated request-identity Baggage onto ``context`` so the Baggage
|
||||
processor stamps team/key/metadata onto the span. Identity is read from the
|
||||
parsed payload, never the client's ``params._meta`` carrier, so it can't be
|
||||
spoofed."""
|
||||
bag = promoted_baggage(
|
||||
identity,
|
||||
model,
|
||||
promoted_keys=tuple(self.config.baggage_promoted_keys),
|
||||
metadata_keys=tuple(self.config.baggage_metadata_keys),
|
||||
team_metadata_keys=tuple(self.config.baggage_team_metadata_keys),
|
||||
)
|
||||
return set_request_baggage(bag, context=context) if bag else context
|
||||
|
||||
def _emit_mcp_tool_call(
|
||||
self,
|
||||
kwargs: Mapping[str, Any],
|
||||
|
|
@ -254,10 +275,12 @@ class OpenTelemetryV2(CustomLogger):
|
|||
|
||||
MCP tool calls reach the success/failure callbacks like any other request
|
||||
(with ``call_type`` ``call_mcp_tool``), but they are not LLM calls and have
|
||||
no ``pre_call`` carrier — so they get their own CLIENT span here, parented
|
||||
to the request's server span. Returns whether it handled the event, so the
|
||||
caller skips the LLM-call path. The whole span is emitted at once (there is
|
||||
no boundary to open it at), deduped on the call id by the emitter.
|
||||
no ``pre_call`` carrier — so they get their own CLIENT span here. Per the MCP
|
||||
semconv it parents to the trace context the client propagated in
|
||||
``params._meta`` (or starts a new root) and links the transport span, rather
|
||||
than nesting under the HTTP/session span. Returns whether it handled the
|
||||
event, so the caller skips the LLM-call path. The whole span is emitted at
|
||||
once (there is no boundary to open it at), deduped on the call id.
|
||||
"""
|
||||
raw_payload = kwargs.get("standard_logging_object")
|
||||
if not raw_payload or not is_mcp_tool_call(cast(Mapping[str, object], raw_payload)):
|
||||
|
|
@ -271,12 +294,51 @@ class OpenTelemetryV2(CustomLogger):
|
|||
# as a phantom LLM span.
|
||||
if data.identity.call_id:
|
||||
self._open_llm_calls.pop(data.identity.call_id, None)
|
||||
parent_context, links = resolve_mcp_span_context()
|
||||
parent_context = self._seed_identity_baggage(data.identity, None, parent_context)
|
||||
self._emitter.emit(
|
||||
SpanRole.MCP_TOOL_CALL,
|
||||
data,
|
||||
parent_context=resolve_request_span_context(),
|
||||
parent_context=parent_context,
|
||||
start_time_ns=to_ns(start_time),
|
||||
end_time_ns=to_ns(end_time),
|
||||
links=links,
|
||||
)
|
||||
return True
|
||||
|
||||
def _emit_mcp_list_tools(
|
||||
self,
|
||||
kwargs: Mapping[str, object],
|
||||
start_time: datetime | float | None,
|
||||
end_time: datetime | float | None,
|
||||
) -> bool:
|
||||
"""Emit an MCP ``tools/list`` span when the closed request was a discovery call.
|
||||
|
||||
Like a tool call, listing reaches the success/failure callbacks (here with
|
||||
``call_type`` ``list_mcp_tools``) with no ``pre_call`` carrier, so it gets its
|
||||
own CLIENT span. Per the MCP semconv it parents to the ``params._meta`` trace
|
||||
context (or starts a new root) and links the transport span, rather than
|
||||
nesting under the HTTP/session span. Returns whether it handled the event so
|
||||
the caller skips the LLM-call path.
|
||||
"""
|
||||
raw_payload = kwargs.get("standard_logging_object")
|
||||
if not raw_payload or not is_mcp_list_tools(cast(Mapping[str, object], raw_payload)):
|
||||
return False
|
||||
payload = cast("StandardLoggingPayload", raw_payload)
|
||||
data = MCPListToolsSpanData.from_standard_logging_payload(
|
||||
payload, capture_content=self.config.capture_span_content
|
||||
)
|
||||
if data.identity.call_id:
|
||||
self._open_llm_calls.pop(data.identity.call_id, None)
|
||||
parent_context, links = resolve_mcp_span_context()
|
||||
parent_context = self._seed_identity_baggage(data.identity, None, parent_context)
|
||||
self._emitter.emit(
|
||||
SpanRole.MCP_LIST_TOOLS,
|
||||
data,
|
||||
parent_context=parent_context,
|
||||
start_time_ns=to_ns(start_time),
|
||||
end_time_ns=to_ns(end_time),
|
||||
links=links,
|
||||
)
|
||||
return True
|
||||
|
||||
|
|
@ -319,16 +381,7 @@ class OpenTelemetryV2(CustomLogger):
|
|||
# root span — parent to it (ambient fallback on the SDK path). Seed identity
|
||||
# Baggage so the span — and the SDK path, which has none — is labeled
|
||||
# consistently.
|
||||
parent_ctx = resolve_request_span_context()
|
||||
bag = promoted_baggage(
|
||||
data.identity,
|
||||
data.request_model,
|
||||
promoted_keys=tuple(self.config.baggage_promoted_keys),
|
||||
metadata_keys=tuple(self.config.baggage_metadata_keys),
|
||||
team_metadata_keys=tuple(self.config.baggage_team_metadata_keys),
|
||||
)
|
||||
if bag:
|
||||
parent_ctx = set_request_baggage(bag, context=parent_ctx)
|
||||
parent_ctx = self._seed_identity_baggage(data.identity, data.request_model, resolve_request_span_context())
|
||||
return self._emitter.emit(
|
||||
SpanRole.LLM_CALL,
|
||||
data,
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ from typing_extensions import Protocol, runtime_checkable
|
|||
from litellm.integrations.otel.model.payloads import (
|
||||
GuardrailSpanData,
|
||||
LLMCallSpanData,
|
||||
MCPListToolsSpanData,
|
||||
MCPToolCallSpanData,
|
||||
ServiceSpanData,
|
||||
)
|
||||
|
|
@ -20,7 +21,7 @@ AttributeMap = dict[str, AttrValue]
|
|||
# The closed set of span-data types the engine routes through the mapper chain.
|
||||
# Server spans (PROXY_REQUEST + management routes) belong to the mounted FastAPI
|
||||
# instrumentor, not the mapper chain.
|
||||
SpanData = LLMCallSpanData | MCPToolCallSpanData | GuardrailSpanData | ServiceSpanData
|
||||
SpanData = LLMCallSpanData | MCPToolCallSpanData | MCPListToolsSpanData | GuardrailSpanData | ServiceSpanData
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ from litellm.integrations.otel.mappers.utils import (
|
|||
from litellm.integrations.otel.model.payloads import (
|
||||
GuardrailSpanData,
|
||||
LLMCallSpanData,
|
||||
MCPListToolsSpanData,
|
||||
MCPToolCallSpanData,
|
||||
ServiceSpanData,
|
||||
ToolDefinition,
|
||||
|
|
@ -100,6 +101,15 @@ class GenAIMapper:
|
|||
f"{LiteLLM.COST_PREFIX}total": lambda d: d.response_cost,
|
||||
}
|
||||
|
||||
# A tools/list discovery span: the method and session only. Per semconv it must
|
||||
# NOT carry gen_ai.operation.name (execute_tool) or gen_ai.tool.name — those are
|
||||
# for tool calls, and listing executes no tool.
|
||||
_MCP_LIST_ATTRS: dict[str, Callable[[MCPListToolsSpanData], AttrValue | None]] = {
|
||||
MCP.METHOD_NAME: lambda d: d.method,
|
||||
MCP.SESSION_ID: lambda d: d.session_id,
|
||||
LiteLLM.CALL_ID: lambda d: d.identity.call_id or None,
|
||||
}
|
||||
|
||||
_GUARDRAIL_ATTRS: dict[str, Callable[[GuardrailSpanData], AttrValue | None]] = {
|
||||
LiteLLM.GUARDRAIL_NAME: lambda d: d.guardrail_name,
|
||||
LiteLLM.GUARDRAIL_MODE: lambda d: d.mode,
|
||||
|
|
@ -130,6 +140,8 @@ class GenAIMapper:
|
|||
return self._llm_call(data)
|
||||
case MCPToolCallSpanData():
|
||||
return collect(self._MCP_ATTRS, data)
|
||||
case MCPListToolsSpanData():
|
||||
return collect(self._MCP_LIST_ATTRS, data)
|
||||
case GuardrailSpanData():
|
||||
return self._guardrail(data)
|
||||
case ServiceSpanData():
|
||||
|
|
|
|||
|
|
@ -37,12 +37,14 @@ __all__ = [
|
|||
"LLMCost",
|
||||
"LLMRequestParams",
|
||||
"LLMUsage",
|
||||
"MCPListToolsSpanData",
|
||||
"MCPToolCallSpanData",
|
||||
"ProxyRequestSpanData",
|
||||
"ServerInfo",
|
||||
"ServiceSpanData",
|
||||
"SpanError",
|
||||
"ToolDefinition",
|
||||
"is_mcp_list_tools",
|
||||
"is_mcp_tool_call",
|
||||
]
|
||||
|
||||
|
|
@ -415,6 +417,42 @@ def is_mcp_tool_call(payload: Mapping[str, object]) -> bool:
|
|||
return bool(_mcp_tool_call_metadata(payload)) or (payload.get("call_type") == "call_mcp_tool")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MCPListToolsSpanData:
|
||||
"""One MCP ``tools/list`` discovery call, parsed from a closed request's payload.
|
||||
|
||||
The proxy is an MCP *client* enumerating an upstream server's tools, so this is
|
||||
a CLIENT span. It carries neither ``gen_ai.operation.name`` nor ``gen_ai.tool.name``:
|
||||
the GenAI semconv sets ``execute_tool`` (and the tool name) only for tool *calls*,
|
||||
and listing executes no tool.
|
||||
"""
|
||||
|
||||
method: str
|
||||
session_id: str | None
|
||||
error: SpanError | None
|
||||
identity: RequestIdentity
|
||||
|
||||
@classmethod
|
||||
def from_standard_logging_payload(
|
||||
cls, payload: StandardLoggingPayload, capture_content: bool = False
|
||||
) -> MCPListToolsSpanData:
|
||||
# The list-tools logging path does not thread an MCP session id into the
|
||||
# payload (only the tool-call path stamps ``mcp_tool_call_metadata``), so
|
||||
# there is none to read here; ``mcp.session.id`` is simply omitted.
|
||||
return cls(
|
||||
method=MCPMethod.TOOLS_LIST.value,
|
||||
session_id=None,
|
||||
error=_parse_error(payload),
|
||||
identity=RequestContext.from_standard_logging_payload(payload).identity,
|
||||
)
|
||||
|
||||
|
||||
def is_mcp_list_tools(payload: Mapping[str, object]) -> bool:
|
||||
"""Whether a closed request's payload is an MCP ``tools/list`` discovery call
|
||||
rather than a tool call or an LLM call — true when the call type says so."""
|
||||
return payload.get("call_type") == "list_mcp_tools"
|
||||
|
||||
|
||||
# --- service event_metadata sanitization ------------------------------------ #
|
||||
|
||||
# Substrings (case-insensitive) of keys that must never reach a span: secrets,
|
||||
|
|
|
|||
|
|
@ -18,6 +18,13 @@ before the LLM call even starts), so a guardrail is a sibling of the LLM call,
|
|||
not a child of it. The emitter parents every span to the ambient OTel context
|
||||
(the active server span), which matches this.
|
||||
|
||||
MCP spans (``MCP_TOOL_CALL``, ``MCP_LIST_TOOLS``) are intentionally NOT in this
|
||||
tree. Per the OTel GenAI MCP semconv, MCP and the HTTP transport are independent
|
||||
contexts, so an MCP span parents to the trace context the client propagated in
|
||||
``params._meta`` (or starts its own root when none is propagated) and records the
|
||||
``PROXY_REQUEST`` transport span as a span *link*, never a parent. The registry
|
||||
encodes this as ``parent=None, links=PROXY_REQUEST``.
|
||||
|
||||
Not every service call becomes a span — :func:`span_role_for_service` decides:
|
||||
|
||||
- ``DB_CALL`` (CLIENT) — outbound datastores (redis, postgres,
|
||||
|
|
@ -46,6 +53,7 @@ if TYPE_CHECKING:
|
|||
from litellm.integrations.otel.model.payloads import (
|
||||
GuardrailSpanData,
|
||||
LLMCallSpanData,
|
||||
MCPListToolsSpanData,
|
||||
MCPToolCallSpanData,
|
||||
ProxyRequestSpanData,
|
||||
ServiceSpanData,
|
||||
|
|
@ -56,6 +64,7 @@ class SpanRole(str, Enum):
|
|||
PROXY_REQUEST = "proxy_request"
|
||||
LLM_CALL = "llm_call"
|
||||
MCP_TOOL_CALL = "mcp_tool_call"
|
||||
MCP_LIST_TOOLS = "mcp_list_tools"
|
||||
GUARDRAIL = "guardrail"
|
||||
DB_CALL = "db_call"
|
||||
SERVICE = "service"
|
||||
|
|
@ -74,14 +83,24 @@ class SpanSpec:
|
|||
role: SpanRole
|
||||
kind: LiteLLMSpanKind
|
||||
parent: SpanRole | None
|
||||
links: SpanRole | None = None
|
||||
|
||||
|
||||
SPAN_REGISTRY: dict[SpanRole, SpanSpec] = {
|
||||
SpanRole.PROXY_REQUEST: SpanSpec(SpanRole.PROXY_REQUEST, LiteLLMSpanKind.SERVER, parent=None),
|
||||
SpanRole.LLM_CALL: SpanSpec(SpanRole.LLM_CALL, LiteLLMSpanKind.CLIENT, parent=SpanRole.PROXY_REQUEST),
|
||||
# The proxy is an MCP client to the upstream server it dispatches the tool
|
||||
# call to, so this is a CLIENT span, sibling of the LLM call under the request.
|
||||
SpanRole.MCP_TOOL_CALL: SpanSpec(SpanRole.MCP_TOOL_CALL, LiteLLMSpanKind.CLIENT, parent=SpanRole.PROXY_REQUEST),
|
||||
# MCP and the HTTP transport are independent contexts (OTel GenAI MCP semconv),
|
||||
# so an MCP span does not nest under the transport span. The proxy is an MCP
|
||||
# client to the upstream server, so it's a CLIENT span; it parents to the trace
|
||||
# context the client propagated in ``params._meta`` (or starts its own root when
|
||||
# none is propagated) and records the PROXY_REQUEST transport span as a span
|
||||
# *link*, never a parent — hence ``parent=None, links=PROXY_REQUEST``.
|
||||
SpanRole.MCP_TOOL_CALL: SpanSpec(
|
||||
SpanRole.MCP_TOOL_CALL, LiteLLMSpanKind.CLIENT, parent=None, links=SpanRole.PROXY_REQUEST
|
||||
),
|
||||
SpanRole.MCP_LIST_TOOLS: SpanSpec(
|
||||
SpanRole.MCP_LIST_TOOLS, LiteLLMSpanKind.CLIENT, parent=None, links=SpanRole.PROXY_REQUEST
|
||||
),
|
||||
SpanRole.GUARDRAIL: SpanSpec(SpanRole.GUARDRAIL, LiteLLMSpanKind.INTERNAL, parent=SpanRole.PROXY_REQUEST),
|
||||
SpanRole.DB_CALL: SpanSpec(SpanRole.DB_CALL, LiteLLMSpanKind.CLIENT, parent=SpanRole.PROXY_REQUEST),
|
||||
SpanRole.SERVICE: SpanSpec(SpanRole.SERVICE, LiteLLMSpanKind.INTERNAL, parent=SpanRole.PROXY_REQUEST),
|
||||
|
|
@ -163,6 +182,12 @@ def mcp_tool_call_span_name(data: "MCPToolCallSpanData") -> str:
|
|||
return f"{data.method} {data.tool_name}".strip()
|
||||
|
||||
|
||||
def mcp_list_tools_span_name(data: "MCPListToolsSpanData") -> str:
|
||||
"""``"{mcp.method.name}"`` i.e. ``"tools/list"`` — no low-cardinality target, so
|
||||
the method name alone names the span (MCP semconv)."""
|
||||
return data.method
|
||||
|
||||
|
||||
def proxy_request_span_name(data: "ProxyRequestSpanData") -> str:
|
||||
"""``"{method} {route}"`` (HTTP semconv)."""
|
||||
return f"{data.http_method} {data.route}".strip()
|
||||
|
|
@ -179,7 +204,8 @@ def service_span_name(data: "ServiceSpanData") -> str:
|
|||
|
||||
|
||||
def root_roles() -> list[SpanRole]:
|
||||
"""Roles that start a new trace (no in-process parent)."""
|
||||
"""Roles with no in-process parent. They start a new trace unless they adopt a
|
||||
remote parent (e.g. an MCP span joining the client's propagated context)."""
|
||||
return [role for role, spec in SPAN_REGISTRY.items() if spec.parent is None]
|
||||
|
||||
|
||||
|
|
@ -196,6 +222,8 @@ def validate_registry(
|
|||
raise ValueError(f"SPAN_REGISTRY[{role}] has mismatched role {spec.role}")
|
||||
if spec.parent is not None and spec.parent not in reg:
|
||||
raise ValueError(f"span role {role} declares unknown parent {spec.parent}")
|
||||
if spec.links is not None and spec.links not in reg:
|
||||
raise ValueError(f"span role {role} declares unknown link target {spec.links}")
|
||||
missing = [role for role in SpanRole if role not in reg]
|
||||
if missing:
|
||||
raise ValueError(f"SPAN_REGISTRY is missing roles: {missing}")
|
||||
|
|
|
|||
|
|
@ -1,11 +1,11 @@
|
|||
"""Trace-context + Baggage helpers."""
|
||||
|
||||
from contextvars import ContextVar
|
||||
from contextvars import ContextVar, Token
|
||||
from typing import Mapping
|
||||
|
||||
from opentelemetry import baggage
|
||||
from opentelemetry.context import Context, get_current
|
||||
from opentelemetry.trace import Span, get_current_span, set_span_in_context
|
||||
from opentelemetry.trace import Link, Span, get_current_span, set_span_in_context
|
||||
from opentelemetry.trace.propagation.tracecontext import (
|
||||
TraceContextTextMapPropagator,
|
||||
)
|
||||
|
|
@ -47,6 +47,31 @@ def request_root_span() -> "Span | None":
|
|||
return span if is_recordable_span(span) else None
|
||||
|
||||
|
||||
# The W3C trace-context carrier (``traceparent``/``tracestate``/``baggage``) the
|
||||
# MCP client propagated in the current request's ``params._meta``. The MCP gateway
|
||||
# sets it per message so the MCP span can parent to the client's span rather than
|
||||
# to the transport. A ``ContextVar`` because, like the root-span anchor, it must
|
||||
# ride the request task and be readable by the inline success-logging callback.
|
||||
_mcp_message_trace_carrier: "ContextVar[Mapping[str, str] | None]" = ContextVar(
|
||||
"litellm_otel_mcp_message_trace_carrier", default=None
|
||||
)
|
||||
|
||||
|
||||
def set_mcp_message_trace_carrier(
|
||||
carrier: "Mapping[str, str] | None",
|
||||
) -> "Token[Mapping[str, str] | None]":
|
||||
"""Stash the current MCP message's propagated trace-context carrier.
|
||||
|
||||
Returns the reset token; the caller must reset it once the message is handled
|
||||
so the carrier never leaks to the next message on the same session task.
|
||||
"""
|
||||
return _mcp_message_trace_carrier.set(carrier)
|
||||
|
||||
|
||||
def reset_mcp_message_trace_carrier(token: "Token[Mapping[str, str] | None]") -> None:
|
||||
_mcp_message_trace_carrier.reset(token)
|
||||
|
||||
|
||||
def set_request_baggage(values: Mapping[str, str], context: Context | None = None) -> Context:
|
||||
"""Return a context with ``values`` written into Baggage."""
|
||||
ctx = context
|
||||
|
|
@ -104,6 +129,38 @@ def resolve_request_span_context() -> Context:
|
|||
return get_current()
|
||||
|
||||
|
||||
def resolve_mcp_span_context(
|
||||
carrier: "Mapping[str, str] | None" = None,
|
||||
) -> "tuple[Context, tuple[Link, ...]]":
|
||||
"""Parent context + links for an MCP message span, per the OTel GenAI MCP semconv.
|
||||
|
||||
MCP and the underlying transport (HTTP) are independent lifecycles — one
|
||||
streamable-HTTP session multiplexes many messages, so nesting the message span
|
||||
under the HTTP/session span is wrong (it renders the message at the session's
|
||||
start, skewed by however long the session has been open). Instead:
|
||||
|
||||
* parent to the trace context the client propagated in the request's
|
||||
``params._meta`` (a *remote* parent), and
|
||||
* record the transport/session span as a *link*, never the parent.
|
||||
|
||||
Only trace context (``traceparent``/``tracestate``) is extracted, never the
|
||||
client's W3C Baggage: ``params._meta`` is caller-controlled, and the otel
|
||||
baggage processor stamps allowlisted baggage keys (``litellm.team.id``,
|
||||
``litellm.metadata.*``, ...) onto the span as attributes, so honoring remote
|
||||
baggage would let a client spoof a span's identity attribution.
|
||||
|
||||
With no propagated context the returned context carries no span, so the span
|
||||
starts its own root trace (still linked to the transport). The base context is
|
||||
explicitly empty so an absent ``traceparent`` can never fall through to the
|
||||
ambient (stale session) span.
|
||||
"""
|
||||
source = carrier if carrier is not None else _mcp_message_trace_carrier.get()
|
||||
parent = _PROPAGATOR.extract(dict(source or {}), context=Context())
|
||||
transport = request_root_span()
|
||||
links = (Link(transport.get_span_context()),) if transport is not None else ()
|
||||
return parent, links
|
||||
|
||||
|
||||
def is_recordable_span(obj: object) -> bool:
|
||||
"""True if ``obj`` is a live span with a valid context (safe to parent under)."""
|
||||
if not isinstance(obj, Span):
|
||||
|
|
|
|||
|
|
@ -229,6 +229,56 @@ def _jsonrpc_text_has_top_level_method(text: str) -> bool:
|
|||
return False
|
||||
|
||||
|
||||
def _mcp_meta_trace_carrier(req_ctx: object) -> Optional[dict[str, str]]:
|
||||
"""The W3C trace context (``traceparent``/``tracestate``) the MCP client
|
||||
propagated in the request's ``params._meta`` (SEP-414), or ``None``.
|
||||
|
||||
Per the OTel MCP semconv the MCP span parents to this propagated context rather
|
||||
than to the HTTP/session transport (which is recorded as a link instead), so a
|
||||
streamable-HTTP session that multiplexes many messages does not glue every
|
||||
message under the session's first request. The client's W3C Baggage is
|
||||
deliberately excluded: it is caller-controlled, and the otel baggage processor
|
||||
stamps allowlisted baggage keys (``litellm.team.id``, ``litellm.metadata.*``,
|
||||
...) onto the span, so honoring remote baggage would let a client spoof a
|
||||
span's identity attribution.
|
||||
"""
|
||||
meta = getattr(req_ctx, "meta", None)
|
||||
extra = getattr(meta, "model_extra", None)
|
||||
if not isinstance(extra, dict):
|
||||
return None
|
||||
carrier = {key: extra[key] for key in ("traceparent", "tracestate") if isinstance(extra.get(key), str)}
|
||||
return carrier or None
|
||||
|
||||
|
||||
def _otel_set_mcp_trace_carrier(carrier: Optional[dict[str, str]]) -> object:
|
||||
"""Stash ``carrier`` for the otel_v2 MCP span and return a reset token, or
|
||||
``None`` when otel_v2 is unavailable. Lazily imported so opentelemetry stays an
|
||||
optional dependency."""
|
||||
try:
|
||||
from litellm.integrations.otel.plumbing.context import (
|
||||
set_mcp_message_trace_carrier,
|
||||
)
|
||||
|
||||
return set_mcp_message_trace_carrier(carrier)
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
|
||||
def _otel_reset_mcp_trace_carrier(token: object) -> None:
|
||||
"""Clear the per-message trace carrier so it never leaks to the next message on
|
||||
the same session task. Paired with ``_otel_set_mcp_trace_carrier``."""
|
||||
if token is None:
|
||||
return
|
||||
try:
|
||||
from litellm.integrations.otel.plumbing.context import (
|
||||
reset_mcp_message_trace_carrier,
|
||||
)
|
||||
|
||||
reset_mcp_message_trace_carrier(token)
|
||||
except ImportError:
|
||||
return
|
||||
|
||||
|
||||
def _proxy_exception_to_http_exception(exc: ProxyException) -> HTTPException:
|
||||
"""Map a ``ProxyException`` to an ``HTTPException`` that preserves its real
|
||||
status code and headers.
|
||||
|
|
@ -595,8 +645,10 @@ if MCP_AVAILABLE:
|
|||
_session_reset_token = None
|
||||
if req_ctx:
|
||||
_session_reset_token = active_mcp_session_var.set(req_ctx.session)
|
||||
_trace_token = None
|
||||
|
||||
try:
|
||||
_trace_token = _otel_set_mcp_trace_carrier(_mcp_meta_trace_carrier(req_ctx))
|
||||
# Get user authentication from context variable
|
||||
(
|
||||
user_api_key_auth,
|
||||
|
|
@ -632,6 +684,7 @@ if MCP_AVAILABLE:
|
|||
# This prevents the HTTP stream from failing and allows the client to get a response
|
||||
return []
|
||||
finally:
|
||||
_otel_reset_mcp_trace_carrier(_trace_token)
|
||||
if _session_reset_token is not None:
|
||||
active_mcp_session_var.reset(_session_reset_token)
|
||||
|
||||
|
|
@ -658,8 +711,10 @@ if MCP_AVAILABLE:
|
|||
_session_reset_token = None
|
||||
if req_ctx:
|
||||
_session_reset_token = active_mcp_session_var.set(req_ctx.session)
|
||||
_trace_token = None
|
||||
|
||||
try:
|
||||
_trace_token = _otel_set_mcp_trace_carrier(_mcp_meta_trace_carrier(req_ctx))
|
||||
# Validate arguments
|
||||
(
|
||||
user_api_key_auth,
|
||||
|
|
@ -778,6 +833,7 @@ if MCP_AVAILABLE:
|
|||
|
||||
return response
|
||||
finally:
|
||||
_otel_reset_mcp_trace_carrier(_trace_token)
|
||||
if _session_reset_token is not None:
|
||||
active_mcp_session_var.reset(_session_reset_token)
|
||||
|
||||
|
|
|
|||
|
|
@ -27,9 +27,11 @@ from litellm.integrations.otel import ( # noqa: E402
|
|||
OpenTelemetryV2Config,
|
||||
)
|
||||
from litellm.integrations.otel.plumbing import providers # noqa: E402
|
||||
from litellm.integrations.otel.plumbing.context import (
|
||||
from litellm.integrations.otel.plumbing.context import ( # noqa: E402
|
||||
reset_mcp_message_trace_carrier,
|
||||
set_mcp_message_trace_carrier,
|
||||
set_request_root_span,
|
||||
) # noqa: E402
|
||||
)
|
||||
from litellm.integrations.otel.logger import OpenTelemetryV2 # noqa: E402
|
||||
from litellm.integrations.otel.model.spans import ( # noqa: E402
|
||||
LITELLM_PROXY_REQUEST_SPAN_NAME,
|
||||
|
|
@ -53,8 +55,10 @@ def _reset_request_root_span():
|
|||
from litellm.integrations.otel.plumbing import context as _otel_context
|
||||
|
||||
_otel_context._request_root_span.set(None)
|
||||
_otel_context._mcp_message_trace_carrier.set(None)
|
||||
yield
|
||||
_otel_context._request_root_span.set(None)
|
||||
_otel_context._mcp_message_trace_carrier.set(None)
|
||||
|
||||
|
||||
def _payload(**overrides):
|
||||
|
|
@ -387,6 +391,190 @@ def test_mcp_tool_call_metadata_read_from_nested_metadata_not_top_level():
|
|||
assert LiteLLM.MCP_SERVER_NAME not in span.attributes
|
||||
|
||||
|
||||
def _mcp_list_payload(**overrides):
|
||||
payload = {
|
||||
"call_type": "list_mcp_tools",
|
||||
"status": "success",
|
||||
"litellm_call_id": "mcp_list_1",
|
||||
"metadata": {
|
||||
"user_api_key_team_id": "t1",
|
||||
"spend_logs_metadata": {"mcp_operation": "list_tools"},
|
||||
},
|
||||
"hidden_params": {},
|
||||
}
|
||||
payload.update(overrides)
|
||||
return payload
|
||||
|
||||
|
||||
def test_mcp_list_tools_emits_client_span():
|
||||
"""An MCP ``tools/list`` discovery call becomes a CLIENT span named ``tools/list``,
|
||||
carrying only the MCP method and the call id. Per the GenAI MCP semconv the list
|
||||
span omits ``gen_ai.operation.name`` and ``gen_ai.tool.name`` (tool-call-only) and
|
||||
``mcp.session.id`` (the list path threads no session id), so a naive reuse of the
|
||||
tool-call mapper would wrongly stamp them, and the pre-fix code emitted no span at
|
||||
all for a ``list_mcp_tools`` payload."""
|
||||
logger, exporter = _logger()
|
||||
kwargs = {"standard_logging_object": _mcp_list_payload()}
|
||||
asyncio.run(logger.async_log_success_event(kwargs, None, None, None))
|
||||
(span,) = exporter.get_finished_spans()
|
||||
assert span.name == "tools/list"
|
||||
assert span.kind is SpanKind.CLIENT
|
||||
assert span.attributes["mcp.method.name"] == "tools/list"
|
||||
assert span.attributes[LiteLLM.CALL_ID] == "mcp_list_1"
|
||||
assert span.status.status_code is StatusCode.UNSET
|
||||
# Bug-killers: no span pre-fix (empty exporter -> the unpack above raises), and a
|
||||
# tool-call-shaped fix would leak execute_tool / tool name / session id here.
|
||||
assert GenAI.OPERATION_NAME not in span.attributes
|
||||
assert "gen_ai.tool.name" not in span.attributes
|
||||
assert "mcp.session.id" not in span.attributes
|
||||
|
||||
|
||||
_MCP_SPAN_CASES = [
|
||||
(_mcp_payload, "tools/call get_weather"),
|
||||
(_mcp_list_payload, "tools/list"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("make_payload, span_name", _MCP_SPAN_CASES)
|
||||
def test_mcp_span_roots_and_links_transport_without_propagated_context(
|
||||
make_payload, span_name
|
||||
):
|
||||
"""MCP and the HTTP transport are independent lifecycles (one streamable-HTTP
|
||||
session multiplexes many messages), so per the MCP semconv the message span
|
||||
must NOT nest under the session/transport span — that is what made it render
|
||||
skewed at the session's start. With no propagated ``params._meta`` context it
|
||||
starts its own root trace and records the transport span as a *link*, never
|
||||
the parent."""
|
||||
logger, exporter = _logger()
|
||||
transport = logger._emitter.start_span(
|
||||
SpanRole.PROXY_REQUEST, LITELLM_PROXY_REQUEST_SPAN_NAME
|
||||
)
|
||||
set_request_root_span(transport)
|
||||
asyncio.run(
|
||||
logger.async_log_success_event(
|
||||
{"standard_logging_object": make_payload()}, None, None, None
|
||||
)
|
||||
)
|
||||
transport.end()
|
||||
span = next(s for s in exporter.get_finished_spans() if s.name == span_name)
|
||||
assert span.parent is None
|
||||
assert span.context.trace_id != transport.get_span_context().trace_id
|
||||
assert [link.context.span_id for link in span.links] == [
|
||||
transport.get_span_context().span_id
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("make_payload, span_name", _MCP_SPAN_CASES)
|
||||
def test_mcp_span_parents_to_propagated_meta_trace_context(make_payload, span_name):
|
||||
"""When the client propagates W3C trace context in the request's
|
||||
``params._meta`` (SEP-414), the MCP span parents to it (one distributed trace)
|
||||
and still links the transport span — never falling through to the
|
||||
ambient/session span."""
|
||||
logger, exporter = _logger()
|
||||
transport = logger._emitter.start_span(
|
||||
SpanRole.PROXY_REQUEST, LITELLM_PROXY_REQUEST_SPAN_NAME
|
||||
)
|
||||
set_request_root_span(transport)
|
||||
token = set_mcp_message_trace_carrier(
|
||||
{"traceparent": "00-11111111111111111111111111111111-2222222222222222-01"}
|
||||
)
|
||||
try:
|
||||
asyncio.run(
|
||||
logger.async_log_success_event(
|
||||
{"standard_logging_object": make_payload()}, None, None, None
|
||||
)
|
||||
)
|
||||
finally:
|
||||
reset_mcp_message_trace_carrier(token)
|
||||
transport.end()
|
||||
span = next(s for s in exporter.get_finished_spans() if s.name == span_name)
|
||||
assert span.context.trace_id == 0x11111111111111111111111111111111
|
||||
assert span.parent is not None
|
||||
assert span.parent.span_id == 0x2222222222222222
|
||||
assert [link.context.span_id for link in span.links] == [
|
||||
transport.get_span_context().span_id
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("make_payload, span_name", _MCP_SPAN_CASES)
|
||||
def test_mcp_span_ignores_client_supplied_baggage(make_payload, span_name):
|
||||
"""The MCP span must NOT honor W3C Baggage from the client's ``params._meta``.
|
||||
|
||||
``params._meta`` is caller-controlled and the baggage processor stamps
|
||||
allowlisted baggage keys onto every span, so extracting remote baggage would
|
||||
let a client spoof a span's identity (e.g. ``litellm.team.id``). The propagator
|
||||
extracts trace context only, so the spoofed keys never reach the span while the
|
||||
legitimate traceparent parenting still works."""
|
||||
logger, exporter = _logger()
|
||||
transport = logger._emitter.start_span(
|
||||
SpanRole.PROXY_REQUEST, LITELLM_PROXY_REQUEST_SPAN_NAME
|
||||
)
|
||||
set_request_root_span(transport)
|
||||
token = set_mcp_message_trace_carrier(
|
||||
{
|
||||
"traceparent": "00-11111111111111111111111111111111-2222222222222222-01",
|
||||
"baggage": "litellm.team.id=spoofed-team,litellm.metadata.user_api_key_user_id=attacker",
|
||||
}
|
||||
)
|
||||
try:
|
||||
asyncio.run(
|
||||
logger.async_log_success_event(
|
||||
{"standard_logging_object": make_payload()}, None, None, None
|
||||
)
|
||||
)
|
||||
finally:
|
||||
reset_mcp_message_trace_carrier(token)
|
||||
transport.end()
|
||||
span = next(s for s in exporter.get_finished_spans() if s.name == span_name)
|
||||
# Trace context still honored: proves the carrier was processed, not dropped wholesale.
|
||||
assert span.parent is not None and span.parent.span_id == 0x2222222222222222
|
||||
# Identity is the authenticated payload's team, never the client's spoofed value.
|
||||
assert span.attributes[LiteLLM.TEAM_ID] == "t1"
|
||||
assert "litellm.metadata.user_api_key_user_id" not in span.attributes
|
||||
|
||||
|
||||
@pytest.mark.parametrize("make_payload, span_name", _MCP_SPAN_CASES)
|
||||
def test_mcp_span_carries_authenticated_identity(make_payload, span_name):
|
||||
"""An MCP span is labeled with the authenticated request's identity (team/key),
|
||||
seeded from the parsed payload like the LLM-call span. Without this seeding the
|
||||
span — parented to an empty remote context — would carry no team/key attribute at
|
||||
all, so it couldn't be attributed or filtered by team in the traces backend."""
|
||||
logger, exporter = _logger()
|
||||
asyncio.run(
|
||||
logger.async_log_success_event(
|
||||
{"standard_logging_object": make_payload()}, None, None, None
|
||||
)
|
||||
)
|
||||
span = next(s for s in exporter.get_finished_spans() if s.name == span_name)
|
||||
assert span.attributes[LiteLLM.TEAM_ID] == "t1"
|
||||
|
||||
|
||||
def test_mcp_span_malformed_traceparent_starts_root():
|
||||
"""A malformed traceparent in ``params._meta`` must not crash or parent to a
|
||||
bogus span: the propagator ignores it, so the span starts its own root trace and
|
||||
still links the transport span."""
|
||||
logger, exporter = _logger()
|
||||
transport = logger._emitter.start_span(
|
||||
SpanRole.PROXY_REQUEST, LITELLM_PROXY_REQUEST_SPAN_NAME
|
||||
)
|
||||
set_request_root_span(transport)
|
||||
token = set_mcp_message_trace_carrier({"traceparent": "not-a-valid-traceparent"})
|
||||
try:
|
||||
asyncio.run(
|
||||
logger.async_log_success_event(
|
||||
{"standard_logging_object": _mcp_list_payload()}, None, None, None
|
||||
)
|
||||
)
|
||||
finally:
|
||||
reset_mcp_message_trace_carrier(token)
|
||||
transport.end()
|
||||
span = next(s for s in exporter.get_finished_spans() if s.name == "tools/list")
|
||||
assert span.parent is None
|
||||
assert [link.context.span_id for link in span.links] == [
|
||||
transport.get_span_context().span_id
|
||||
]
|
||||
|
||||
|
||||
def test_pre_call_idempotent_keeps_first_span():
|
||||
"""A retried call may re-enter ``pre_call`` with the same call id; the first
|
||||
span (with the true start time) is kept, not replaced."""
|
||||
|
|
|
|||
|
|
@ -94,19 +94,32 @@ def test_registry_parent_integrity_no_orphans():
|
|||
|
||||
|
||||
def test_registry_hierarchy_shape():
|
||||
assert set(root_roles()) == {SpanRole.PROXY_REQUEST}
|
||||
# MCP roles have no in-process parent: per the MCP semconv they root (or adopt
|
||||
# the client's propagated _meta context), so they sit alongside PROXY_REQUEST.
|
||||
assert set(root_roles()) == {
|
||||
SpanRole.PROXY_REQUEST,
|
||||
SpanRole.MCP_TOOL_CALL,
|
||||
SpanRole.MCP_LIST_TOOLS,
|
||||
}
|
||||
# Guardrails parent to the request span, not the LLM call: a pre-call
|
||||
# guardrail runs before the LLM call exists, so it's a sibling of it.
|
||||
assert set(child_roles(SpanRole.PROXY_REQUEST)) == {
|
||||
SpanRole.LLM_CALL,
|
||||
SpanRole.MCP_TOOL_CALL,
|
||||
SpanRole.GUARDRAIL,
|
||||
SpanRole.DB_CALL,
|
||||
SpanRole.SERVICE,
|
||||
}
|
||||
assert SPAN_REGISTRY[SpanRole.LLM_CALL].kind is LiteLLMSpanKind.CLIENT
|
||||
# The proxy is an MCP client to the upstream tool server: CLIENT span.
|
||||
# The proxy is an MCP client to the upstream tool server: CLIENT span. Listing
|
||||
# tools is the same client relationship, so it's a CLIENT span too.
|
||||
assert SPAN_REGISTRY[SpanRole.MCP_TOOL_CALL].kind is LiteLLMSpanKind.CLIENT
|
||||
assert SPAN_REGISTRY[SpanRole.MCP_LIST_TOOLS].kind is LiteLLMSpanKind.CLIENT
|
||||
# MCP spans don't nest under the transport: they link the PROXY_REQUEST span
|
||||
# instead of parenting to it (OTel GenAI MCP semconv).
|
||||
assert SPAN_REGISTRY[SpanRole.MCP_TOOL_CALL].parent is None
|
||||
assert SPAN_REGISTRY[SpanRole.MCP_LIST_TOOLS].parent is None
|
||||
assert SPAN_REGISTRY[SpanRole.MCP_TOOL_CALL].links is SpanRole.PROXY_REQUEST
|
||||
assert SPAN_REGISTRY[SpanRole.MCP_LIST_TOOLS].links is SpanRole.PROXY_REQUEST
|
||||
assert SPAN_REGISTRY[SpanRole.PROXY_REQUEST].kind is LiteLLMSpanKind.SERVER
|
||||
assert SPAN_REGISTRY[SpanRole.GUARDRAIL].parent is SpanRole.PROXY_REQUEST
|
||||
# An outbound datastore call is a CLIENT span; an internal service is INTERNAL.
|
||||
|
|
|
|||
|
|
@ -6447,3 +6447,54 @@ class TestStreamableHttpAuthErrorMapping:
|
|||
m.get("type") == "http.response.start" and m.get("status") == 500
|
||||
for m in sent
|
||||
)
|
||||
|
||||
|
||||
class TestMCPMetaTraceCarrier:
|
||||
"""`_mcp_meta_trace_carrier` extracts the W3C trace context the MCP client
|
||||
propagated in the request's params._meta (SEP-414) so the otel_v2 MCP span can
|
||||
parent to the client's span. Exercises the real MCP SDK `RequestParams.Meta`
|
||||
shape (extra='allow' preserves the unprefixed keys), not just an injected
|
||||
carrier."""
|
||||
|
||||
def test_extracts_trace_context_and_excludes_baggage_and_other_meta(self):
|
||||
"""Only traceparent/tracestate are carried. The client's W3C ``baggage`` is
|
||||
deliberately dropped even though it rides in params._meta: it is
|
||||
caller-controlled, and the otel baggage processor stamps allowlisted baggage
|
||||
keys onto the span, so honoring it would let a client spoof a span's identity
|
||||
(e.g. ``litellm.team.id``). Dropping it at the source is the regression guard."""
|
||||
from types import SimpleNamespace
|
||||
|
||||
from mcp.types import RequestParams
|
||||
|
||||
from litellm.proxy._experimental.mcp_server.server import (
|
||||
_mcp_meta_trace_carrier,
|
||||
)
|
||||
|
||||
meta = RequestParams.Meta.model_validate(
|
||||
{
|
||||
"traceparent": "00-11111111111111111111111111111111-2222222222222222-01",
|
||||
"tracestate": "rojo=1",
|
||||
"baggage": "litellm.team.id=spoofed-team,litellm.metadata.user_api_key_user_id=attacker",
|
||||
"progressToken": "p1",
|
||||
}
|
||||
)
|
||||
carrier = _mcp_meta_trace_carrier(SimpleNamespace(meta=meta))
|
||||
assert carrier == {
|
||||
"traceparent": "00-11111111111111111111111111111111-2222222222222222-01",
|
||||
"tracestate": "rojo=1",
|
||||
}
|
||||
assert "baggage" not in carrier
|
||||
|
||||
def test_none_when_no_trace_context(self):
|
||||
from types import SimpleNamespace
|
||||
|
||||
from mcp.types import RequestParams
|
||||
|
||||
from litellm.proxy._experimental.mcp_server.server import (
|
||||
_mcp_meta_trace_carrier,
|
||||
)
|
||||
|
||||
assert _mcp_meta_trace_carrier(None) is None
|
||||
assert _mcp_meta_trace_carrier(SimpleNamespace(meta=None)) is None
|
||||
only_progress = RequestParams.Meta.model_validate({"progressToken": "p1"})
|
||||
assert _mcp_meta_trace_carrier(SimpleNamespace(meta=only_progress)) is None
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue