mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
19 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Yassin Kortam
|
ae1d1cb05e
|
fix(http): stop pooled clients persisting cookies on the aiohttp jar too (#36149)
#35978 stopped the pooled A2A client replaying one upstream's Set-Cookie to another by installing a blocking policy on that client's httpx cookie jar. That covers only one of the two jars on the request path. AiohttpTransport is the default transport unless it is explicitly disabled, and the aiohttp ClientSession behind it keeps its own cookie jar which no httpx-level assertion can observe, so the leak is still live on the default path: a live proxy on that commit still delivers agent-alpha's session cookie to agent-beta's card fetch and JSON-RPC call. The reason it looked fixed is that aiohttp's default CookieJar is built with unsafe=False and refuses to store cookies for IP hosts, so a proof addressed to 127.0.0.1 comes back clean whether or not that jar is blocked. Cookie persistence is now blocked where the clients are built rather than at one call site: blocked_cookie_jar() gives every httpx client, async and sync, a jar whose DefaultCookiePolicy(allowed_domains=()) rejects every domain in both directions, and both ClientSession constructions litellm owns, the transport's session factory and the proxy's shared startup session, get a DummyCookieJar. LiteLLM reads a response cookie nowhere, and an explicitly supplied Cookie header still goes out, so passthrough forwarding and an agent's extra_headers are unaffected. The A2A-scoped policy #35978 added is removed, since it is now dead. The two suites that drive the aiohttp session factory synchronously mock ClientSession because a real one needs a running event loop; DummyCookieJar has the same requirement, so they mock it for the same reason. |
||
|
Yassin Kortam
|
2b38991df9
|
fix(a2a): stop writing per-caller state onto the shared cached httpx client (#35978)
create_a2a_client took the raw client off a process-wide cached handler and called headers.update() on it, then leaned on folding the header set into the cache key (through the unrelated disable_aiohttp_transport field) to keep one caller's credentials away from the next. Per-caller headers now ride with each request through the a2a SDK's call context, and the agent card fetch gets them through resolver_http_kwargs, so the shared client is never written to and its cache key no longer varies by header set. Since the proxy puts a fresh trace id in every request's headers, that key previously changed on every call, giving each request its own httpx client and flushing the 200-entry client cache that every other provider shares. All A2A callers on one timeout now reuse a single pooled client. Sharing that client also means sharing its httpx cookie jar, which httpx fills from every Set-Cookie and replays on any later request to a matching domain, so one agent's session cookie would arrive at another agent on the same host. The pooled client now carries a cookie policy that stores and sends nothing, which neither litellm nor the a2a SDK relies on: the SDK's auth interceptor skips cookie-borne API keys outright. |
||
|
Yassin Kortam
|
abd239f903
|
fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name (#35151)
* fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name The GenAI metric attribute builder mapped only chat, text completion, embedding, responses and MCP tool calls to an operation name, so vector-store searches and A2A agent sends fell through to the "chat" default. Their duration and cost then landed in the same series a Grafana GenAI dashboard reads chat latency off, with no way to tell them apart. Both now map to the operation names the convention defines for them, retrieval and invoke_agent, and an unmapped call type says so at debug instead of silently becoming chat. The provider label used gen_ai.system, which the convention deprecated in favor of gen_ai.provider.name; the dashboards built on that vocabulary find nothing under the old key. Metrics now carry gen_ai.provider.name with the semconv provider value (bedrock -> aws.bedrock) via the resolve_provider helper the span path already uses, and keep dual-emitting gen_ai.system with its raw value so a dashboard already querying it keeps matching. A request litellm cannot attribute to a provider gets no provider label at all rather than a placeholder "Unknown" that minted a permanent series nobody can act on. Resolves LIT-4954 Resolves LIT-4959 * fix(otel): map the rest of the vector-store call types off the chat default Mapping only the search left the store lifecycle (create, retrieve, list, update, delete) and the file operations (create, list, retrieve, content, update, delete) falling through to chat, so vector-store admin traffic kept polluting the same series a dashboard reads chat latency off. A live run confirmed it: all 20 metric datapoints from a create, retrieve, list, file-list and delete came out labelled chat. The convention names no operation for vector-store management, so these take vendor values under the litellm. prefix, litellm.vector_store_management and litellm.vector_store_file_management, one per REST resource. Its note on gen_ai.operation.name directs instrumentation to use a system-specific name when no predefined value applies, which is the same allowance resolve_provider already relies on for unmapped providers. Excluding them from the GenAI metrics altogether was the alternative; it deletes series an operator may be watching today and is far harder to reverse than a rename, so it stays available as a follow-up rather than being decided here. Mapping them onto the semconv memory store family was rejected: litellm vector stores hold documents, not agent memory records, and borrowing those names would put document admin calls into whatever charts agent-memory operations, which is the bug this fixes. /rag/query reaches the same recorder and is the same operation as a vector-store search, so query and aquery map to retrieval too; leaving them would have left the defect alive on a second retrieval surface. /rag/ingest is a write with no semconv equivalent and no retrieval or agent confusion, so it is left for the RAG owners to name. Resolves LIT-4954 * fix(otel): give the streaming A2A path a call type so it labels as invoke_agent The streaming logging object is built by hand and never runs through update_environment_variables, the only place call_type reaches model_call_details, so every streamed agent turn arrived at the recorder with no call type and fell back to chat. Stamp it, and map the streaming spelling alongside the non-streaming ones. |
||
|
Yassin Kortam
|
765fd0762e
|
fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) | ||
|
michelligabriele
|
30ddef78d9
|
fix(a2a): record agent cost_per_query and input tokens on native send path (#31979)
* fix(a2a): record agent cost_per_query and input tokens on native send path * test(a2a): add __init__.py to avoid test_utils.py module collision |
||
|
Sameer Kankute
|
8e30cfbeb1
|
feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950)
* feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents Bump a2a-sdk to 1.x and wire send/stream through compat conversions so the proxy accepts A2A 1.0 JSON-RPC while preserving 0.3 wire clients. Co-authored-by: Cursor <cursoragent@cursor.com> * Add user controlled protocol version in agents * Fix exeception mapping * Fix a2a base url * Add e2e test for a2a * Fix lint * Fix lint * fix(a2a): harden card version detection and header isolation coverage Use protocolVersion when inferring agent card wire format, assert distinct httpx cache keys in the header-isolation test, and suppress targeted basedpyright errors for optional SDK imports. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): suppress reportArgumentType for SDK compat types and fix streaming trace ID - Add pyright: ignore[reportArgumentType] to SendMessageSuccessResponse id= and result= args in _send_message, and SendStreamingMessageResponse root= in _stream_messages, where a2a-sdk compat types diverge from basedpyright's inferred signature, reducing the reportArgumentType count back within budget. - Fix streaming trace ID in astream_a2a_message to use str(request.id) when available instead of always generating a new uuid4(), restoring JSON-RPC request-ID correlation for observability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style(a2a): expand SendStreamingMessageResponse for black formatting Move pyright: ignore comment to the root= argument line so Black accepts the expanded multi-line form. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(a2a): fix 2 reportArgumentType errors without suppression - main.py: narrow logging_obj from object|None to Optional[Logging] via isinstance check before A2AStreamingIterator call, fixing the "Logging | object" argument type mismatch at line 699. - a2a_endpoints.py: extract response_dict with explicit isinstance(dict) guard before passing to normalize_jsonrpc_response, fixing the "LLMResponseTypes | dict[str, Any]" type mismatch at line 835. - Remove spurious pyright: ignore comments added in previous commits that were not suppressing the actual errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(a2a): rewrite upstream URL for 1.0 agent cards in getAuthenticatedExtendedCard 1.0 upstream agent cards store the endpoint URL in supportedInterfaces[0].url rather than a top-level url field. The previous guard only rewrote url when it existed at the top level, so after normalize_agent_card lowered a 1.0 card to 0.3 the upstream internal address leaked into the url field of the 0.3 response. Fix: rewrite both url and supportedInterfaces[0].url to the proxy address before calling normalize_agent_card, ensuring the upstream address is never visible to downstream clients regardless of the upstream card's wire format. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: extend _served_version to all PascalCase methods; add direct httpx-client isolation proof - _served_version now checks `_PASCAL_TO_WIRE` membership instead of two hardcoded names, so GetTask/CancelTask/etc. are promoted to 1.0 wire format alongside SendMessage — prevents mixed wire formats mid-session - test_create_a2a_client_uses_fresh_httpx_client now asserts a2a_client_a._litellm_httpx_client is not a2a_client_b._litellm_httpx_client (direct proof that header bleed cannot occur), in addition to the cache-key inequality check Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: id:0 silently dropped in version_convert; explicit continue in stream retry - version_convert.py: replace `request_id or ""` with `str(request_id) if request_id is not None else ""` in both _send_result_to and _stream_result_to; id=0 is valid JSON-RPC and must not be coerced to "" which breaks response correlation - main.py: add explicit `continue` after the A2ALocalhostURLError retry in _execute_a2a_stream_with_retry so the control flow (retry → next iteration → stream_succeeded guard) is unambiguous Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: preserve a2a retry and discovery card urls * Fix black * Fix test * fix(a2a): avoid KeyError in discovery log after 0.3→1.0 card normalization When a 0.3-style agent card is normalized to 1.0, the top-level url key is replaced by supportedInterfaces; log the already-computed proxy_url instead. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): preserve taskId when lowering push notification config set params Flatten 1.x create envelope fields before parsing into TaskPushNotificationConfig so 1.0 clients forwarding to 0.3 upstream keep taskId and config. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): ignore unknown fields in message/send proto fallback ParseDict in _build_message_send_params now matches other inbound paths so 1.0 clients with extra proto fields are not rejected with -32602. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): normalize tasks/list params and response across protocol versions Convert list task entries on the response path and lower ListTasksRequest params including status filters when forwarding 1.0 clients to 0.3 upstream. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): avoid reportArgumentType in _lower_list_tasks_params; use local var instead of _parse return Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(a2a): drop private SDK symbol in tasks/list status lowering _lower_list_tasks_params imported _CORE_TO_COMPAT_TASK_STATE, a private a2a-sdk symbol that could disappear on a patch release and silently break status-filter lowering. Derive the 0.3 wire string from the public protobuf enum name instead (TASK_STATE_<NAME> maps to the 0.3 value once the prefix is dropped and underscores become dashes) and validate the result against the 0.3 TaskState enum's own values via a fully-typed pure helper. Behavior is unchanged for every state; unspecified or unrecognized states still drop the filter. Adds parametrized regression tests covering dashed wire values (input-required, auth-required) and the unspecified drop. * fix(a2a): drop redundant push-notification envelope key; unify MessageToDict import _flatten_create_push_notification_params used `config or pushNotificationConfig`, which short-circuits so a co-present pushNotificationConfig key was never popped and leaked into the flattened params. Pop both keys unconditionally and prefer config when present. Adds a regression test on the helper that fails on the old leak. Also import MessageToDict from a2a.compat.v0_3.conversions in _lower_list_tasks_params to match every other conversion helper in the module instead of pulling it straight from google.protobuf.json_format. * fix(a2a): reject invalid message/stream params early with -32602 _handle_stream_message built MessageSendParams lazily inside the stream_response() generator, so malformed 1.0 params surfaced as a generic -32603 after the 200 status line was already committed. The non-streaming path validates up front and returns -32602 (Invalid params). Validate eagerly before returning the StreamingResponse and emit -32602 on failure so both paths reject malformed params identically. Adds a regression test asserting the streamed error code is -32602. * fix(a2a): raise clear error when non-streaming send ends on an update event _send_message fed the SDK iterator's last event straight into SendMessageSuccessResponse, whose result only accepts Message or Task. A non-standard upstream whose final event is a TaskStatusUpdateEvent or TaskArtifactUpdateEvent made the response construction raise an opaque pydantic ValidationError. Guard the converted result and raise a clear RuntimeError instead, consistent with the no-response guard above it. Adds regression tests for the Message happy path and the update-event rejection via an injected fake client. * test(a2a): lock in clean merged agent-card URL without PROXY_BASE_URL Regression coverage proving _build_merged_agent_card produces no double slash in supportedInterfaces[0].url when PROXY_BASE_URL is unset and request.base_url carries a trailing slash. get_custom_url routes through join_paths, which rstrips the base, so the f-string join stays clean. * style(a2a): modernize type annotations to satisfy strict ruff budget After merging the black->ruff-format migration from base, the A2A files owned by this PR still used Optional[X]/quoted annotations that pushed UP037/UP045 over their lowered ceilings. Convert to X | None, drop the now-unnecessary quoted local annotation in _send_message, and remove the imports left unused by the rewrite. Type semantics are unchanged. * style(a2a): type a2a_endpoints dict params as dict[str, Any] The merge with the formatter-migration baseline tightened the reportUnknownArgumentType ceiling; bare dict annotations made every value Unknown and pushed the codebase total over cap. Annotate the JSON-RPC params, body, metadata, and litellm_params dicts as dict[str, Any] so their values are typed, dropping the unknown-argument count back under the ceiling. No behavior change. * fix(a2a): guard localhost retry against a missing agent card handle_a2a_localhost_retry rewrote the card URL and called create_client with whatever agent_card it received. The caller resolves the card from the SDK client (Optional), so a None card reached set_agent_card_url and create_client, surfacing an opaque SDK error instead of a clear one. Add an early RuntimeError guard mirroring the httpx-client check, drop the now always-true card None-check on the stash line, and cover it with a regression test. * style(a2a): disable reportUnknownArgumentType in a2a-sdk boundary modules The lint env type-checks without the optional a2a-sdk/protobuf installed, so every call into the protobuf-generated compat conversions counts as an Unknown-typed argument and the new A2A code pushed the codebase reportUnknownArgumentType total over its ceiling. These three modules are the A2A SDK boundary; turn the rule off file-wide with a documented reason instead of scattering dozens of per-line ignores across every SDK call. * fix(a2a): tolerate unknown fields when lowering 1.0->0.3; align streaming trace id Two issues greptile flagged: version_convert: the 1.0->0.3 lowering paths (_send_result_to, _task_to, _stream_result_to) called ParseDict without ignore_unknown_fields=True, so a 1.0 upstream response carrying vendor extensions raised and best-effort fell back to passing the un-lowered 1.0 shape to a 0.3 client. Set the flag to match the agent-card path and every inbound path; unknown fields are now dropped and the result is correctly lowered. main.py: asend_message_streaming derived X-LiteLLM-Trace-Id from the JSON-RPC request id, unlike asend_message which uses the logging object's litellm_trace_id. Prefer the logging trace id (then request id, then a uuid) so streamed and non-streamed calls correlate under the same trace. Adds regression tests for both, including the stream-event lowering path. * style(a2a): apply ruff format to a2a protocol and proxy modules Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Mateo Wang
|
250d8d2a96
|
fix(a2a): forward agent_extra_headers through completion bridge (#28277)
* fix(a2a): forward agent_extra_headers through completion bridge
A2A agents backed by a custom_llm_provider (e.g. langgraph,
bedrock_agentcore) silently dropped any per-request headers rewritten
from the inbound `x-a2a-{agent}-*` convention or admin-configured
`extra_headers`. The headers were correctly extracted in
`a2a_endpoints.py` but never passed into
`_send_message_via_completion_bridge` or the bridge handler, so the
upstream HTTP request reached the agent backend without them.
Thread `agent_extra_headers` through:
- asend_message / asend_message_streaming -> bridge call sites
- _send_message_via_completion_bridge
- A2ACompletionBridgeHandler.handle_non_streaming / handle_streaming
- Inject as `extra_headers` into the underlying litellm.acompletion()
call, and forward to provider configs via kwargs (their **kwargs
signature absorbs it harmlessly today).
* fix(a2a): forward agent_extra_headers through bridge convenience wrappers
Address greptile review on PR #28277:
- handle_a2a_completion / handle_a2a_completion_streaming (the public,
exported convenience wrappers) now accept agent_extra_headers and
forward it to the underlying class methods. Without this, callers
going through the public API would still silently drop per-request
headers — the exact regression this PR fixes for the class-method
path.
- Add the missing agent_extra_headers entry to the handle_streaming
docstring for parity with handle_non_streaming.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(a2a/bedrock): forward agent_extra_headers to AgentCore HTTP request
Address greptile follow-up on PR #28277:
BedrockAgentCoreA2AConfig was absorbing agent_extra_headers via **kwargs
but never propagating to the underlying HTTP POST, so x-a2a-{agent}-*
rewrites and admin extra_headers were silently dropped on the
bedrock_agentcore path that bypasses the completion bridge.
Thread the parameter through the full Bedrock AgentCore stack:
- config.handle_non_streaming / handle_streaming pull
agent_extra_headers from kwargs and pass to the handler.
- handler.handle_non_streaming / handle_streaming accept it and forward
to the transformation layer.
- transformation.get_url_and_signed_request merges agent_extra_headers
into the headers dict BEFORE signing, so SigV4 covers them in the
signature. JWT/Bearer path: AgentCore signer always overwrites
Authorization with api_key, so use api_key (not agent_extra_headers)
to override the bearer token.
Also fix a pre-existing test assertion that was already broken by the
parent commit ab70ff6 (test_provider_config_receives_litellm_params
didn't include agent_extra_headers in the expected call).
Tests:
- TestTransformation::test_agent_extra_headers_merged_into_signed_headers_jwt
- TestTransformation::test_agent_extra_headers_signed_for_sigv4
- TestNonStreaming::test_agent_extra_headers_forwarded_on_outbound_post
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(a2a/bedrock): drop reserved AWS headers from agent_extra_headers
Per veria-ai security review on PR #28277:
agent_extra_headers carries values rewritten from the client-controlled
x-a2a-{agent}-* convention, so the unconditional 'headers.update(agent_extra_headers)'
in BedrockAgentCoreA2ATransformation.get_url_and_signed_request let any
caller with access to an agent overwrite headers the proxy sets from
trusted server-side config -- most notably
X-Amzn-Bedrock-AgentCore-Runtime-User-Id, which AWS treats as the runtime
identity. Because the merge happened before SigV4 signing, the spoofed
value would also be bound into a valid signature.
Strip reserved AWS/AgentCore headers (authorization, host,
x-amzn-bedrock-agentcore-runtime-*, x-amz-*) from agent_extra_headers
before merging and log a warning when any are dropped. Legitimate
per-request headers (e.g. x-mcp-token, x-tenant) still pass through.
Adds two tests covering both the JWT path (verifies the spoof does not
land on the outbound headers) and the SigV4 path (verifies the signer
never sees the spoofed values).
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(a2a): annotate completion_params dict for mypy
The dict literal initializing completion_params had heterogeneous value
types (str, list, bool), so mypy inferred the value type as a narrow
union that did not accept dict[str, str] when assigning extra_headers.
Annotate completion_params as Dict[str, Any] in both the non-streaming
and streaming bridge handlers so the agent_extra_headers merge
type-checks cleanly.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(a2a/pydantic_ai): forward agent_extra_headers to upstream HTTP request
* fix(a2a/bridge): admin litellm_params.extra_headers win over caller-rewritten headers
agent_extra_headers contains both admin static_headers and caller-derived
dynamic headers (from the x-a2a-{agent}-* rewrite). Merging it last would
let a caller replace headers that the proxy was configured to send upstream
via litellm_params.extra_headers. Flip the merge order so admin-configured
headers take precedence on conflict.
* fix(a2a/headers): merge_agent_headers compares case-insensitively
HTTP header names are case-insensitive, but the previous merge was a
case-sensitive dict update. That meant an admin-configured
static_headers['Authorization'] (capital A) did not strip a
caller-rewritten x-a2a-{agent}-authorization (lowercase, from the
inbound header normalization in a2a_endpoints) - both ended up on the
outbound request to pydantic_ai / langgraph / etc.
Restore the documented 'static wins on conflict' invariant by comparing
case-insensitively when overlaying static_headers. Static side's casing
is preserved on the output.
* fix(a2a/bridge): merge configured extra_headers case-insensitively over caller headers
A caller-rewritten lowercase header (e.g. authorization from the
x-a2a-{agent}-* convention) could ride alongside an admin-configured
case-variant key in litellm_params.extra_headers, sending duplicate
Authorization headers upstream. The bridge now reuses
merge_agent_headers so configured headers win case-insensitively, in
both the non-streaming and streaming paths. merge_agent_headers moved
to litellm.interactions.agents.utils (re-exported from the proxy utils)
so the SDK-level bridge does not import from litellm.proxy.
https://claude.ai/code/session_017cBvda8Y4CLo8wspB2kfSV
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
|
||
|
Sameer Kankute
|
48c9fabb26
|
Fix : a2a bugs 030626 (#29566)
* Fix error code and context id injection bug * Add support for all A2A methods * Add logging * address greptile review: relay upstream JSON-RPC errors, move _PASCAL_TO_WIRE to module level, add error path tests * fix(a2a): run pre_call_hook for tasks/resubscribe SSE path to enforce guardrails tasks/resubscribe was returning the raw SSE stream without calling proxy_logging_obj.pre_call_hook, silently bypassing any guardrails configured on the agent. This patch calls pre_call_hook before streaming begins and wires post_call_failure_hook into the SSE generator so errors are logged. Adds a regression test verifying the hook is called. * fix(a2a): use get_async_httpx_client instead of creating httpx clients per request Creating httpx.AsyncClient instances per-request adds ~500ms latency. Switch _forward_jsonrpc and _forward_jsonrpc_sse to use the shared client from get_async_httpx_client(httpxSpecialProvider.A2A). * fix(a2a): forward caller identity headers on task ops; validate push notification URL Two security fixes for task management methods: 1. All task operations (tasks/get, tasks/list, tasks/cancel, tasks/resubscribe, push notification config methods) now forward X-LiteLLM-User-Id and X-LiteLLM-Team-Id headers to the upstream agent, so the agent can scope task access to the authenticated caller. 2. tasks/pushNotificationConfig/set validates the callback URL before forwarding: requires HTTPS and rejects private/loopback/reserved IP ranges and localhost hostnames to prevent SSRF. * Fix A2A task hook and push URL handling * fix(a2a): fix mypy type errors for request_id and header_name dict key types * Fix A2A request id and params forwarding * Forward trace IDs for A2A task calls * fix(a2a): strip client-forwarded X-LiteLLM-* headers before applying authenticated identity A client could send x-a2a-<agent>-x-litellm-user-id in their request and have it forwarded to the upstream agent as an authenticated identity header. Fix: sanitize any X-LiteLLM-* headers from agent_extra_headers before merging, then apply the authenticated identity headers last so they always override client-supplied values. * Fix A2A SSE fallback JSON-RPC error code * Fix A2A SSE error id backfill * fix(a2a): validate both push notification url fields to close SSRF bypass * fix(a2a): widen request_id annotation to match JSON-RPC id call sites * fix(a2a): run post-call streaming hook for tasks/resubscribe so agent guardrails apply tasks/resubscribe returned the raw upstream SSE stream without routing events through the post-call streaming hook, so output guardrails configured on the agent were silently skipped for streaming task subscriptions while every other task method and message/stream applied them. Parse upstream JSON-RPC SSE events and feed them through async_streaming_data_generator, matching message/stream, so guardrails inspect the streamed task content. Adds a regression test that fails when the streamed events bypass the guardrail hook. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Sameer Kankute
|
c8bcfbb20c
|
feat(a2a): watsonx Orchestrate agent provider (#29410)
* feat(a2a): add watsonx Orchestrate agent provider Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth), with dashboard agent type metadata and unit tests for transformations. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): use shared httpx client and cache WXO auth tokens Route WXO streaming through get_async_httpx_client (TLS verification enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via a dedicated helper instead of hard-coded JSON paths. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): treat CP4D token expiration as absolute Unix time CP4D /authorize returns expiration as epoch seconds, not TTL. Compute remaining lifetime against wall clock so cached tokens refresh before expiry. Co-authored-by: Cursor <cursoragent@cursor.com> * style(a2a): black-format watsonx orchestrate handler for CI py312 Co-authored-by: Cursor <cursoragent@cursor.com> * Fix watsonx orchestrate edge cases * Fix WXO streaming fallback error handling * Fix watsonx orchestrate run completion handling * fix(a2a): make WXO username optional in agent create UI Username is only required for cp4d auth; ibm_cloud uses api_key alone. Backend still validates username when auth_mode is cp4d. Co-authored-by: Cursor <cursoragent@cursor.com> * test(a2a): align WXO dashboard field test with optional username Username is not required in agent_create_fields.json; backend validates for cp4d auth_mode only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): send Accept header on WXO streaming run request * fix(a2a/wxo): scope streaming transport fallback to initial POST only Narrow the httpx.TransportError fallback in handle_streaming so it only covers the initial POST /runs/stream. Errors during polling or SSE consumption now propagate instead of triggering handle_non_streaming, which would have submitted a duplicate WXO run for the same request. * refactor(a2a/wxo): type run-param extraction and use text response_type Return a typed WXORequestParams NamedTuple from _extract_litellm_params instead of a positional tuple so call sites read params by name, and send the user message with response_type 'text' so the run body is valid across all WXO agent configurations rather than the search-specific type. * fix(a2a/wxo): evict expired token cache entries and raise asyncio.TimeoutError on poll timeout --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Sameer Kankute
|
af17400c38
|
feat(a2a): well-known agent-card discovery + LangGraph Platform mode (#28860)
* feat(a2a): well-known agent-card discovery + LangGraph Platform mode Adds a registration-time discovery flow so admins can paste an upstream agent URL, see its skills/capabilities, pick what to expose, and have the proxy front it with a LiteLLM-shaped agent card. Backend (new litellm/proxy/a2a/ module): - fetch_well_known_card walks /.well-known/agent-card.json, /.well-known/agent.json, /agent.json by default. langgraph_platform mode hits the canonical path with ?assistant_id=<id> (LangGraph serves one shared endpoint per deployment). - merge_agent_card overlays LiteLLM overrides on the upstream card: drops upstream url, forces protocolVersion=1.0, replaces securitySchemes with LiteLLMKey bearer, emits supportedInterfaces pointing at the proxy, filters capabilities to a small allowlist, strips non-v1.0 fields. - POST /v1/a2a/discover returns the raw upstream card (admin-only) so the UI can render skills/capabilities for selection. - create/update/patch agent endpoints pre-generate the agent_id and run merge_agent_card before storing, so DB.agent_card_params already embeds the proxy-fronted URL. UI (ui/litellm-dashboard): - New AgentCardDiscovery component with a parent-driven plan: discovery_mode + params + display URL. For LangGraph the parent composes (api_base, assistant_id); for pure A2A it uses the url field. Component hides the manual URL input when the parent drives. - add_agent_form wires discovery for every non-custom agent type and overlays the user's selections onto agent_card_params at submit, fixing the bug where dynamic agent forms ignored discovery picks. Completion-bridge fixes (paired): - Add kind: "message" to A2A response messages and unwrap result so it's a Message directly per spec (matches a2a SDK SendMessageResponse validation). - Forward A2A metadata to LangGraph runs via extra_body.metadata. * fix(a2a): preserve agent url, fix streaming chunk envelope, and protect forwarded metadata - Streaming chunk: move final out of the message object into the result envelope per the A2A spec. - Agent card merge: keep upstream url on the stored card so the runtime invocation path can locate the upstream backend; the public well-known endpoint already rewrites this field to the proxy URL before exposing it to clients. - Completion bridge: apply A2A forward metadata after merging litellm_params so an agent-configured extra_body cannot overwrite the forwarded metadata. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): fix legacy streaming chunk, agent card test, and metadata merge - providers/litellm_completion: move 'final' out of the message object into the result envelope per the A2A spec (matches the bridge fix). - agent endpoints test: the runtime invocation path now preserves the top-level 'url' on the stored card, so update the assertion to match. - completion bridge metadata: when forwarding A2A metadata via extra_body.metadata, merge into any existing extra_body.metadata instead of replacing it, so an agent-configured metadata block is preserved (forward metadata still wins on key conflicts). Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): remove dead duplicate transformation dir; drop SSRF-prone headers field from /v1/a2a/discover Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): revert accidental html→index.html rename from |
||
|
Ishaan Jaffer
|
e8461b5b97
|
style: run black formatter on files from main merge | ||
|
michelligabriele
|
a292add9bd
|
fix(a2a): preserve JSON-RPC envelope for AgentCore A2A-native agents (#25092) | ||
|
Ishaan Jaff
|
b78f4c924c
|
[Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604)
* v1 card resolver fix * fix: is_localhost_or_internal_url * fix code * test_fix_agent_card_url_replaces_localhost * test restruct * test_a2a_non_streaming * test agnts * add exception handling * init errors * add localhost retry * add agent_testing * test_a2a_non_streaming * _build_streaming_logging_obj * code qa fixes * test_card_resolver_fallback_from_new_to_old_path * fix linting |
||
|
Ishaan Jaff
|
3ef475b70e
|
[Fix] A2a Gateway - Allow supporting old A2a card formats (#19949)
* fix: LiteLLMA2ACardResolver * fix: LiteLLMA2ACardResolver * feat: .well-known/agent.json * test_card_resolver_fallback_from_new_to_old_path |
||
|
Ishaan Jaff
|
3054b6ea60
|
[Feat] A2A Gateway - allow adding Azure Foundry Agents on UI (#17909)
* add CostConfigFields * add CostConfigFields * add output_cost_per_token * refactor table * add agent cost view * add azure foundry fields * add foundry logo * fix: clean error * fix utils * fix agent edi * add easter egg * fix order * test_handle_streaming_forwards_api_key * fix forward api key down * fix a2a send msg * add A2a comparison on compare playground * fix chat ui * fix bedrock agentcore stream |
||
|
Ishaan Jaff
|
4a7437ba5f
|
[Feat] Agent Gateway - allow adding langgraph, bedrock agent core agents (#17802)
* fix: langgraph bridge streaming * add public/agents/fields * test_a2a_completion_bridge_non_streaming * TestA2AStreamingTransformation * AgentCredentialFieldMetadata * add new logo * refactor add agent * fix add dynamic fields * feat allow adding langgraph agent * add langgraph provider * stash * add AgentCreateInfo * agent_create_fields * fix fields * test_a2a_completion_bridge_bedrock_agentcore * test_a2a_completion_bridge_bedrock_agentcore * add public endpoints * fix a2a endpoints * fix dynamic fields |
||
|
Ishaan Jaff
|
059fedbed5
|
[Feat] Agent Gateway - Track agent_id in SpendLogs (#17795)
* add agent_id in metadata in spend logs * add agent_id in SpendLogsPayload * add agent_id in SpendLogsPayload * add _set_agent_id_on_logging_obj * add agent id tracking in SpendLogs * add agent id in spend logs * fix create_a2a_client * test_asend_message_passes_agent_id_to_callback * test_get_logging_payload_includes_agent_id_from_kwargs * test_asend_message_streaming_triggers_callbacks * fix asend_message_streaming * asend_message_streaming * A2AStreamingIterator * _handle_stream_message * test_asend_message_streaming_propagates_metadata |
||
|
Ishaan Jaff
|
49b91c4a35
|
[Feat] A2a gateway - Add cost per token pricing (#17780)
* fix calculate_a2a_cost * add cost_per_query * add test_asend_message_uses_cost_per_query * fix: _initialize_slack_alerting_jobs * feat: add token tracking for agents invoke * add A2ARequestUtils * add _set_usage_on_logging_obj * test_asend_message_token_tracking * add _handle_a2a_response_logging * test_asend_message_streaming_token_tracking * add A2AStreamingIterator * add cost calculator for agents * test_asend_message_uses_input_output_cost_per_token * docs gix |
||
|
Ishaan Jaff
|
7a33579af6
|
[Feat] Agent Gateway - Add cost per query for agent invocations (#17774)
* fix calculate_a2a_cost * add cost_per_query * add test_asend_message_uses_cost_per_query * fix: _initialize_slack_alerting_jobs |