mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-09 22:31:41 +00:00
7 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
mateo-berri
|
fe567bd846 | fix(a2a): speak the 0.3 dialect to servers with mis-cased protocol bindings | ||
|
milan
|
0697188be4 |
refactor(a2a): use direct typed access in protocol binding normalization
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
milan
|
d6f9bce4bb |
fix(a2a): normalize agent card protocolBinding casing before transport match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
ae1d1cb05e
|
fix(http): stop pooled clients persisting cookies on the aiohttp jar too (#36149)
#35978 stopped the pooled A2A client replaying one upstream's Set-Cookie to another by installing a blocking policy on that client's httpx cookie jar. That covers only one of the two jars on the request path. AiohttpTransport is the default transport unless it is explicitly disabled, and the aiohttp ClientSession behind it keeps its own cookie jar which no httpx-level assertion can observe, so the leak is still live on the default path: a live proxy on that commit still delivers agent-alpha's session cookie to agent-beta's card fetch and JSON-RPC call. The reason it looked fixed is that aiohttp's default CookieJar is built with unsafe=False and refuses to store cookies for IP hosts, so a proof addressed to 127.0.0.1 comes back clean whether or not that jar is blocked. Cookie persistence is now blocked where the clients are built rather than at one call site: blocked_cookie_jar() gives every httpx client, async and sync, a jar whose DefaultCookiePolicy(allowed_domains=()) rejects every domain in both directions, and both ClientSession constructions litellm owns, the transport's session factory and the proxy's shared startup session, get a DummyCookieJar. LiteLLM reads a response cookie nowhere, and an explicitly supplied Cookie header still goes out, so passthrough forwarding and an agent's extra_headers are unaffected. The A2A-scoped policy #35978 added is removed, since it is now dead. The two suites that drive the aiohttp session factory synchronously mock ClientSession because a real one needs a running event loop; DummyCookieJar has the same requirement, so they mock it for the same reason. |
||
|
Yassin Kortam
|
2b38991df9
|
fix(a2a): stop writing per-caller state onto the shared cached httpx client (#35978)
create_a2a_client took the raw client off a process-wide cached handler and called headers.update() on it, then leaned on folding the header set into the cache key (through the unrelated disable_aiohttp_transport field) to keep one caller's credentials away from the next. Per-caller headers now ride with each request through the a2a SDK's call context, and the agent card fetch gets them through resolver_http_kwargs, so the shared client is never written to and its cache key no longer varies by header set. Since the proxy puts a fresh trace id in every request's headers, that key previously changed on every call, giving each request its own httpx client and flushing the 200-entry client cache that every other provider shares. All A2A callers on one timeout now reuse a single pooled client. Sharing that client also means sharing its httpx cookie jar, which httpx fills from every Set-Cookie and replays on any later request to a matching domain, so one agent's session cookie would arrive at another agent on the same host. The pooled client now carries a cookie policy that stores and sends nothing, which neither litellm nor the a2a SDK relies on: the SDK's auth interceptor skips cookie-borne API keys outright. |
||
|
Yassin Kortam
|
abd239f903
|
fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name (#35151)
* fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name The GenAI metric attribute builder mapped only chat, text completion, embedding, responses and MCP tool calls to an operation name, so vector-store searches and A2A agent sends fell through to the "chat" default. Their duration and cost then landed in the same series a Grafana GenAI dashboard reads chat latency off, with no way to tell them apart. Both now map to the operation names the convention defines for them, retrieval and invoke_agent, and an unmapped call type says so at debug instead of silently becoming chat. The provider label used gen_ai.system, which the convention deprecated in favor of gen_ai.provider.name; the dashboards built on that vocabulary find nothing under the old key. Metrics now carry gen_ai.provider.name with the semconv provider value (bedrock -> aws.bedrock) via the resolve_provider helper the span path already uses, and keep dual-emitting gen_ai.system with its raw value so a dashboard already querying it keeps matching. A request litellm cannot attribute to a provider gets no provider label at all rather than a placeholder "Unknown" that minted a permanent series nobody can act on. Resolves LIT-4954 Resolves LIT-4959 * fix(otel): map the rest of the vector-store call types off the chat default Mapping only the search left the store lifecycle (create, retrieve, list, update, delete) and the file operations (create, list, retrieve, content, update, delete) falling through to chat, so vector-store admin traffic kept polluting the same series a dashboard reads chat latency off. A live run confirmed it: all 20 metric datapoints from a create, retrieve, list, file-list and delete came out labelled chat. The convention names no operation for vector-store management, so these take vendor values under the litellm. prefix, litellm.vector_store_management and litellm.vector_store_file_management, one per REST resource. Its note on gen_ai.operation.name directs instrumentation to use a system-specific name when no predefined value applies, which is the same allowance resolve_provider already relies on for unmapped providers. Excluding them from the GenAI metrics altogether was the alternative; it deletes series an operator may be watching today and is far harder to reverse than a rename, so it stays available as a follow-up rather than being decided here. Mapping them onto the semconv memory store family was rejected: litellm vector stores hold documents, not agent memory records, and borrowing those names would put document admin calls into whatever charts agent-memory operations, which is the bug this fixes. /rag/query reaches the same recorder and is the same operation as a vector-store search, so query and aquery map to retrieval too; leaving them would have left the defect alive on a second retrieval surface. /rag/ingest is a write with no semconv equivalent and no retrieval or agent confusion, so it is left for the RAG owners to name. Resolves LIT-4954 * fix(otel): give the streaming A2A path a call type so it labels as invoke_agent The streaming logging object is built by hand and never runs through update_environment_variables, the only place call_type reaches model_call_details, so every streamed agent turn arrived at the recorder with no call type and fell back to chat. Stamp it, and map the streaming spelling alongside the non-streaming ones. |
||
|
Sameer Kankute
|
8e30cfbeb1
|
feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950)
* feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents Bump a2a-sdk to 1.x and wire send/stream through compat conversions so the proxy accepts A2A 1.0 JSON-RPC while preserving 0.3 wire clients. Co-authored-by: Cursor <cursoragent@cursor.com> * Add user controlled protocol version in agents * Fix exeception mapping * Fix a2a base url * Add e2e test for a2a * Fix lint * Fix lint * fix(a2a): harden card version detection and header isolation coverage Use protocolVersion when inferring agent card wire format, assert distinct httpx cache keys in the header-isolation test, and suppress targeted basedpyright errors for optional SDK imports. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): suppress reportArgumentType for SDK compat types and fix streaming trace ID - Add pyright: ignore[reportArgumentType] to SendMessageSuccessResponse id= and result= args in _send_message, and SendStreamingMessageResponse root= in _stream_messages, where a2a-sdk compat types diverge from basedpyright's inferred signature, reducing the reportArgumentType count back within budget. - Fix streaming trace ID in astream_a2a_message to use str(request.id) when available instead of always generating a new uuid4(), restoring JSON-RPC request-ID correlation for observability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style(a2a): expand SendStreamingMessageResponse for black formatting Move pyright: ignore comment to the root= argument line so Black accepts the expanded multi-line form. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(a2a): fix 2 reportArgumentType errors without suppression - main.py: narrow logging_obj from object|None to Optional[Logging] via isinstance check before A2AStreamingIterator call, fixing the "Logging | object" argument type mismatch at line 699. - a2a_endpoints.py: extract response_dict with explicit isinstance(dict) guard before passing to normalize_jsonrpc_response, fixing the "LLMResponseTypes | dict[str, Any]" type mismatch at line 835. - Remove spurious pyright: ignore comments added in previous commits that were not suppressing the actual errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(a2a): rewrite upstream URL for 1.0 agent cards in getAuthenticatedExtendedCard 1.0 upstream agent cards store the endpoint URL in supportedInterfaces[0].url rather than a top-level url field. The previous guard only rewrote url when it existed at the top level, so after normalize_agent_card lowered a 1.0 card to 0.3 the upstream internal address leaked into the url field of the 0.3 response. Fix: rewrite both url and supportedInterfaces[0].url to the proxy address before calling normalize_agent_card, ensuring the upstream address is never visible to downstream clients regardless of the upstream card's wire format. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: extend _served_version to all PascalCase methods; add direct httpx-client isolation proof - _served_version now checks `_PASCAL_TO_WIRE` membership instead of two hardcoded names, so GetTask/CancelTask/etc. are promoted to 1.0 wire format alongside SendMessage — prevents mixed wire formats mid-session - test_create_a2a_client_uses_fresh_httpx_client now asserts a2a_client_a._litellm_httpx_client is not a2a_client_b._litellm_httpx_client (direct proof that header bleed cannot occur), in addition to the cache-key inequality check Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: id:0 silently dropped in version_convert; explicit continue in stream retry - version_convert.py: replace `request_id or ""` with `str(request_id) if request_id is not None else ""` in both _send_result_to and _stream_result_to; id=0 is valid JSON-RPC and must not be coerced to "" which breaks response correlation - main.py: add explicit `continue` after the A2ALocalhostURLError retry in _execute_a2a_stream_with_retry so the control flow (retry → next iteration → stream_succeeded guard) is unambiguous Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: preserve a2a retry and discovery card urls * Fix black * Fix test * fix(a2a): avoid KeyError in discovery log after 0.3→1.0 card normalization When a 0.3-style agent card is normalized to 1.0, the top-level url key is replaced by supportedInterfaces; log the already-computed proxy_url instead. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): preserve taskId when lowering push notification config set params Flatten 1.x create envelope fields before parsing into TaskPushNotificationConfig so 1.0 clients forwarding to 0.3 upstream keep taskId and config. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): ignore unknown fields in message/send proto fallback ParseDict in _build_message_send_params now matches other inbound paths so 1.0 clients with extra proto fields are not rejected with -32602. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): normalize tasks/list params and response across protocol versions Convert list task entries on the response path and lower ListTasksRequest params including status filters when forwarding 1.0 clients to 0.3 upstream. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): avoid reportArgumentType in _lower_list_tasks_params; use local var instead of _parse return Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(a2a): drop private SDK symbol in tasks/list status lowering _lower_list_tasks_params imported _CORE_TO_COMPAT_TASK_STATE, a private a2a-sdk symbol that could disappear on a patch release and silently break status-filter lowering. Derive the 0.3 wire string from the public protobuf enum name instead (TASK_STATE_<NAME> maps to the 0.3 value once the prefix is dropped and underscores become dashes) and validate the result against the 0.3 TaskState enum's own values via a fully-typed pure helper. Behavior is unchanged for every state; unspecified or unrecognized states still drop the filter. Adds parametrized regression tests covering dashed wire values (input-required, auth-required) and the unspecified drop. * fix(a2a): drop redundant push-notification envelope key; unify MessageToDict import _flatten_create_push_notification_params used `config or pushNotificationConfig`, which short-circuits so a co-present pushNotificationConfig key was never popped and leaked into the flattened params. Pop both keys unconditionally and prefer config when present. Adds a regression test on the helper that fails on the old leak. Also import MessageToDict from a2a.compat.v0_3.conversions in _lower_list_tasks_params to match every other conversion helper in the module instead of pulling it straight from google.protobuf.json_format. * fix(a2a): reject invalid message/stream params early with -32602 _handle_stream_message built MessageSendParams lazily inside the stream_response() generator, so malformed 1.0 params surfaced as a generic -32603 after the 200 status line was already committed. The non-streaming path validates up front and returns -32602 (Invalid params). Validate eagerly before returning the StreamingResponse and emit -32602 on failure so both paths reject malformed params identically. Adds a regression test asserting the streamed error code is -32602. * fix(a2a): raise clear error when non-streaming send ends on an update event _send_message fed the SDK iterator's last event straight into SendMessageSuccessResponse, whose result only accepts Message or Task. A non-standard upstream whose final event is a TaskStatusUpdateEvent or TaskArtifactUpdateEvent made the response construction raise an opaque pydantic ValidationError. Guard the converted result and raise a clear RuntimeError instead, consistent with the no-response guard above it. Adds regression tests for the Message happy path and the update-event rejection via an injected fake client. * test(a2a): lock in clean merged agent-card URL without PROXY_BASE_URL Regression coverage proving _build_merged_agent_card produces no double slash in supportedInterfaces[0].url when PROXY_BASE_URL is unset and request.base_url carries a trailing slash. get_custom_url routes through join_paths, which rstrips the base, so the f-string join stays clean. * style(a2a): modernize type annotations to satisfy strict ruff budget After merging the black->ruff-format migration from base, the A2A files owned by this PR still used Optional[X]/quoted annotations that pushed UP037/UP045 over their lowered ceilings. Convert to X | None, drop the now-unnecessary quoted local annotation in _send_message, and remove the imports left unused by the rewrite. Type semantics are unchanged. * style(a2a): type a2a_endpoints dict params as dict[str, Any] The merge with the formatter-migration baseline tightened the reportUnknownArgumentType ceiling; bare dict annotations made every value Unknown and pushed the codebase total over cap. Annotate the JSON-RPC params, body, metadata, and litellm_params dicts as dict[str, Any] so their values are typed, dropping the unknown-argument count back under the ceiling. No behavior change. * fix(a2a): guard localhost retry against a missing agent card handle_a2a_localhost_retry rewrote the card URL and called create_client with whatever agent_card it received. The caller resolves the card from the SDK client (Optional), so a None card reached set_agent_card_url and create_client, surfacing an opaque SDK error instead of a clear one. Add an early RuntimeError guard mirroring the httpx-client check, drop the now always-true card None-check on the stash line, and cover it with a regression test. * style(a2a): disable reportUnknownArgumentType in a2a-sdk boundary modules The lint env type-checks without the optional a2a-sdk/protobuf installed, so every call into the protobuf-generated compat conversions counts as an Unknown-typed argument and the new A2A code pushed the codebase reportUnknownArgumentType total over its ceiling. These three modules are the A2A SDK boundary; turn the rule off file-wide with a documented reason instead of scattering dozens of per-line ignores across every SDK call. * fix(a2a): tolerate unknown fields when lowering 1.0->0.3; align streaming trace id Two issues greptile flagged: version_convert: the 1.0->0.3 lowering paths (_send_result_to, _task_to, _stream_result_to) called ParseDict without ignore_unknown_fields=True, so a 1.0 upstream response carrying vendor extensions raised and best-effort fell back to passing the un-lowered 1.0 shape to a 0.3 client. Set the flag to match the agent-card path and every inbound path; unknown fields are now dropped and the result is correctly lowered. main.py: asend_message_streaming derived X-LiteLLM-Trace-Id from the JSON-RPC request id, unlike asend_message which uses the logging object's litellm_trace_id. Prefer the logging trace id (then request id, then a uuid) so streamed and non-streamed calls correlate under the same trace. Adds regression tests for both, including the stream-event lowering path. * style(a2a): apply ruff format to a2a protocol and proxy modules Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |