create_a2a_client took the raw client off a process-wide cached handler and
called headers.update() on it, then leaned on folding the header set into the
cache key (through the unrelated disable_aiohttp_transport field) to keep one
caller's credentials away from the next.
Per-caller headers now ride with each request through the a2a SDK's call
context, and the agent card fetch gets them through resolver_http_kwargs, so
the shared client is never written to and its cache key no longer varies by
header set. Since the proxy puts a fresh trace id in every request's headers,
that key previously changed on every call, giving each request its own httpx
client and flushing the 200-entry client cache that every other provider
shares. All A2A callers on one timeout now reuse a single pooled client.
Sharing that client also means sharing its httpx cookie jar, which httpx fills
from every Set-Cookie and replays on any later request to a matching domain, so
one agent's session cookie would arrive at another agent on the same host. The
pooled client now carries a cookie policy that stores and sends nothing, which
neither litellm nor the a2a SDK relies on: the SDK's auth interceptor skips
cookie-borne API keys outright.
Closes GHSA-6hr6-w5qg-qmwg (CVSS 5.3), the second finding from the same
osv-scan run as the js-yaml bump. Bundled here so the scan goes green in
one merge instead of two PRs that each stay red on the other's finding.
Re-derived with `uv lock --upgrade-package h2` rather than taking the
Dependabot lock wholesale: that keeps the diff to the two packages that
actually move (h2, plus hpack 4.2.0 which h2 4.4.1 requires) and leaves
the `exclude-newer` snapshot a real timestamp.
h2 4.4.1 published 2026-08-03, hpack 4.2.0 on 2026-06-23 — both clear of
the 3-day exclude-newer window.
The constraint was 1.0, so anything above that was silently clamped down.
SCX accepts [0.0, 2.0), verified live against both GLM-5.2 and Qwen3.8
Max: 1.5, 1.99 and 1.999 all return 200, while 2.0 returns 400 with
"Temperature should be in [0.0, 2.0)"
Since the clamp is an inclusive min(), 2.0 cannot be the ceiling or it
would pass through a value the endpoint rejects. 1.99 is the practical
maximum
The clamp test now pins both ends: 2.5 comes back as 1.99, and 1.7 rides
through untouched where it used to be flattened to 1.0
Replaces the five launch models with the two that SCX.ai now leads on.
Both are live on api.scx.ai and both were verified against it for tool
calling, json_object and json_schema output, reasoning, prompt caching,
and, for Qwen3.8 Max, image input
Pricing follows SCX's published USD rates. GLM-5.2 lands at $0.55/M input
and $1.9255/M output, tracking the recent GLM-5.2 market repricing;
Qwen3.8 Max at $1.815/M and $5.4461/M sits under the only other seller of
that model, and is the first Qwen3.8 Max entry in the catalog
Also corrects a metadata bug the removed entries carried: they set
max_tokens equal to max_input_tokens, conflating the context window with
the output cap. Both new entries declare a max_output_tokens of 131072,
which is what the endpoint's own validator enforces
The Add Model placeholder moves to scx-ai/GLM-5.2 now that MiniMax-M2.7
is no longer in the catalog
Closes GHSA-5p4m-2wfm-xmqj (CVSS 7.5), flagged by osv-scan against
ui/litellm-dashboard/package-lock.json. js-yaml is pinned by an exact
npm override, so the override and the lock move together.
Dev-only dependency: js-yaml reaches the tree through eslintrc, knip
and @redocly/openapi-core, none of which ship in the built dashboard.
4.3.1 published 2026-07-31, clear of the 3-day min-release-age cooldown.
* fix(proxy): re-assert the authenticated identity on passthrough requests
The passthrough merges the client's litellm_metadata into the request metadata
and then re-asserts only user_api_key and the parent span. Every other identity
field the spend and budget pipeline reads stays whatever the request body set,
so a body carrying user_api_key_user_id, user_api_key_team_id,
user_api_key_org_id or user_api_key_end_user_id charges that user, team, org or
end user instead of the caller.
Re-assert the whole sanitized identity after the merge, so the client's copy of
any of those fields is overwritten by the authenticated key's own values.
* test(passthrough): assert no authenticated identity field is client settable
The existing regression names seven fields; the re-assertion covers every field
get_sanitized_user_information_from_key returns, which is twenty today. Derive
the set from the helper so a field added to StandardLoggingUserAPIKeyMetadata is
covered without touching the test.
Two of the twenty were not covered before, including user_api_key_hash, which is
distinct from user_api_key and was client settable.
get_litellm_params returned metadata=None whenever only litellm_metadata was
supplied, which overwrote the fallback function_setup had already applied and
left litellm_params["metadata"] empty. On the /v1/responses
completion-transformation bridge, used by every provider without a native
Responses API config, and on /v1/messages, that discarded the caller's trace
fields a second time after the proxy had promoted them.
Resolve metadata to a copy of litellm_metadata when metadata is empty, guarding
on isinstance because the proxy leaves an unparseable litellm_metadata string in
place and a null metadata would otherwise suppress the backfill and break the
merge. update_from_kwargs copies rather than aliases for the same reason: on
these routes it is handed the caller's provider-bound dict and would otherwise
write user_api_key_auth into it.
The shared `timeout` guardrail param already parsed into LitellmParams, but
the Zscaler initializer never forwarded it and _send_request hardcoded a 5
second constant, so a configured value was silently ignored and slow scans
failed with `Timeout passed=5` regardless of config.
Forward litellm_params.timeout through to the HTTP call, keep 5 seconds as
the default, fall back to it for non-positive values, and declare the field
on the config model so the dashboard renders it.
* fix(proxy): promote caller metadata trace fields into litellm_metadata
Routes in LITELLM_METADATA_ROUTES keep the caller's metadata as a provider
passthrough field and track proxy state in litellm_metadata, which is the dict
the logging integrations read. The caller's trace_id, session_id, trace_user_id
and trace_metadata therefore never reached any callback on /v1/responses,
/v1/messages, /v1/batches or /v1/files, and mask_input / mask_output were
dropped with them so a caller asking for redaction had their prompt logged in
full.
Promote an explicit allow-list of those fields from the requester_metadata
snapshot into litellm_metadata, never overwriting a value already set so
header-derived ids keep precedence. Trace-mutation controls (existing_trace_id,
update_trace_keys) and trace_public are deliberately excluded: langfuse applies
them to an arbitrary caller-chosen trace with no ownership check. tags is
excluded because per-tag budget enforcement runs earlier, at auth time.
This covers providers with a native Responses API config. Providers reaching
/v1/responses through the chat-completions bridge need the companion change to
get_litellm_params.
* ci: retrigger workflows
* warn at startup when a proxy-wide budget is set but no DB is connected
litellm.max_budget is only enforced via DB-loaded global spend, so a DB-less proxy silently ignores it. Log a one-time startup warning.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): inject max_budget into DB-less budget warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover DB-less budget warning startup call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin DB-less budget warning call site
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stabilize budget warning call-site pin
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Cancel in-flight model fetches when the key or source changes so an older
response cannot overwrite modelInfo. Drop the inverted endpoint-filter
assertion; filtering coverage lands in the next stack PR
Replace ChatUI cards, inputs, dialogs, popovers, MCP selects, uploads,
tooltips, and icons with shadcn/Base UI and Lucide. Update ChatUI tests
to drive searchable combobox controls instead of Ant Design selectors
Continue the Playground Chat Ant Design/Tremor migration: shared MultiSelect,
upload validation with semantic file inputs, collapsible message widgets, and
AdditionalModelSettings on Base UI controls
Bare fireworks_ai/<slug> only resolved to accounts/fireworks/models/<slug>,
so Fireworks routers (served at accounts/fireworks/routers/<id>, e.g.
glm-latest and firerouter) could not be reached without passing the full
resource id. Add a shared resolve_fireworks_resource_name helper that maps an
explicit routers/<id> or models/<id> segment to the right resource path, keeps
the existing -fast router heuristic, and defaults bare slugs to models/ for
backward compatibility. Wire it into both the chat and text-completion
transforms, which had drifted (completion lacked router handling entirely)
PR #35956 added the max_input_chars passthrough to the AutoRouter
constructor but left this mock assertion in tests/router_unit_tests
unchanged, so test_init_auto_router_deployment_success has been failing
on litellm_internal_staging ever since.
The passthrough itself is intentional and its behaviour is already
covered by TestAutoRouterMaxInputCharsWiring in tests/test_litellm, so
only the stale expected kwargs need updating. Assert the shared constant
rather than the literal 2000 so tuning the default does not break this
test again.
Add test_abort_upstream_logs_warning_when_aclose_raises: verifies that _abort_upstream swallows and logs any exception raised by the upstream's aclose() method instead of propagating it.
Add test_enqueue_for_client_returns_false_when_already_detached: verifies that _enqueue_for_client returns False immediately without touching the queue when client_detached is already set before the call.
Add test_enqueue_for_
Strip net-new inline # blocks from streaming_iterator.py, the unit test file,
and the live-proxy regression test to comply with the no-new-comments rule.
Add test_async_sse_wrapper_aborts_upstream_when_detached_drain_cap_reached:
verifies that when the detached-drain cap is already full, the pump calls
aclose() on the upstream so the provider stops generating and billing
instead of continuing to stream while we record only the partial prefix.
Also fixes LIT001 (bare dict in AsyncIterator union) by replacing dict
with Mapping[str, object] across all three stream-type annotations, and
adds the required LIT003 reason strings to the three noqa: BLE001 directives.
Add ANTHROPIC_MESSAGES_MAX_DETACHED_STREAM_DRAINS and ANTHROPIC_MESSAGES_STREAM_RELAY_QUEUE_MAXSIZE to the excluded set. These are advanced internal infrastructure parameters for streaming/queue management with sensible defaults that most users should not modify.
The relay queue was unbounded, so a client reading a long stream more slowly
than Bedrock produced it let the pump accumulate every pending SSE chunk in
memory, and detached post-disconnect drains had no concurrency bound, so an
authenticated client could open many large streams and read slowly to pin
unbounded worker state.
Bound the relay queue and make the pump apply backpressure while the client is
connected (it blocks on a full queue, racing the disconnect signal), so a slow
reader throttles the upstream read exactly as the old direct yield did. Cap how
many detached drains run at once; over the cap a disconnected pump bills what it
collected instead of draining further. Detached-drain lifetime is otherwise
bounded by the upstream stream/read timeout. Both limits are tunable via env.
The detached pump previously caught every upstream exception (Bedrock read,
decode, provider-response, or chunk-conversion error) and terminated the
client stream normally, masking the original provider exception and its
status so downstream failure handling never ran.
Now, when the upstream fails while the client is still connected, forward the
original exception through the queue so the client-facing generator re-raises
it and the proxy's failure handling (status code, post_call_failure_hook)
runs unchanged. Only when the client has already disconnected, where there is
no one to propagate to and no failure hook will fire, fall back to salvaging
partial spend from the collected chunks.
On the /v1/messages -> bedrock/ invoke streaming path a client disconnect
raises CancelledError inside the httpx socket read, which unwinds the whole
upstream generator chain before any finally can drain it. Bedrock keeps
generating and billing the full response, so spend tracking logged only the
truncated partial the client drained (output tokens ~1-15 vs the real count)
and undercounted against AWS invocation logs.
Move the upstream read into a detached background task that fully drains the
provider stream to its terminal message_delta/message_stop and bills there.
The client-facing generator only relays chunks off a queue, so a disconnect
tears down the relay but not the pump. A client_detached event stops
enqueueing after disconnect so the queue can't grow unbounded.
The provider's release gate in project-releaser publishes only when the
topmost released heading in terraform/provider/CHANGELOG.md moves past the
tag the mirror already carries. That heading has been 0.2.2 since
2026-05-13, so every stable release since has correctly decided there was
nothing to publish and the registry has gone stale.
Two things were blocking a release:
1. The mirror shipped 0.3.0 out-of-band on 2026-07-13 (pricing_base_model,
BerriAI/terraform-provider-litellm#47) after the source move, so that
code exists only in the mirror. The publish rsyncs monorepo -> mirror
with --delete, so publishing without this port would have deleted a
released feature from the registry.
2. Nothing here declared a new version.
Port #47 verbatim (resource_model.go and resource_model_crud.go are now
byte-identical to the mirror's released files), backfill the 0.3.0
changelog entry it shipped under, and cut 0.4.0 covering the changes made
here since the source move. 0.3.0 is not reusable as the next version --
the mirror holds that tag and the publish workflow's tag guard rejects it.
