An aggregate gateway DCR authorize whose RFC 8707 resource resolves to exactly one
gateway-managed oauth2 server sealed that server into the flow and then sent the browser to
the generic connect grid anyway, so the user had to find the server the client had already
named and click Connect.
The connect URL now carries only the flow handle. GET /authorize/flow classifies the sealed
flow as unscoped, interactive, M2M, or stale, and returns the matching state to the page.
Interactive flows require a live per-user vendor credential before minting and do not burn the
flow on an early submit. M2M flows use the gateway's configured service credential and finish
without an interactive OAuth trip. Stale flows fail closed instead of becoming unscoped.
The existing explicit Finish action and a new Cancel path preserve deliberate user intent.
Preserve deployment identity through savings calculation, with canonical model fallback only when either ID is absent. Cover negotiated rates, unchanged deployments, alias/base-model cache accounting and missing IDs.
Fixes#38811. Based on the deployment-identity approach proposed by @QuantumBreakz in #38834.
Co-authored-by: Claude Code <noreply@anthropic.com>
post was the only method the mock-transport tests reached, leaving the anchor
in async delete and in sync patch/put/delete uncovered in the tree Codecov
measures. Parametrizing also means a method added later is covered here rather
than being the one that forgets to anchor.
A streamed Responses API relay handed the success handler a bare ResponsesAPIResponse, which the streaming assembly step drops, so the relay never reached the spend callbacks. Hand it the terminal response.completed event instead, which the assembly step already converts, and cover the whole flush path with a regression test that fails on the previous tip.
tests/local_testing runs on CircleCI, which does not feed the patch-coverage
check, so the anchor read as half-uncovered. These three drive it over a mock
transport, which is enough for the lifetime contract even though it cannot
reproduce the teardown the loopback tests exist for.
litellm's azure_ai config rejects reasoning_effort for gpt-chat-latest and Azure documents a fixed reasoning level for it, so the entry no longer advertises reasoning_effort_levels. The catalog metadata tests compare cost_per_token and the whisper transcription cost with the entry the calculator read instead of with list-price literals, the pattern #40195 removed
The merge base read the variable by truthiness, so any non-empty value
turned the global flag on. Parsing it as a flag made a value such as
temperature or enabled silently turn it off, and the only docs for the
variable describe it as a list of parameter names, so keep those values
on and log a warning that asks for true or false. A blank value stays off
without a warning
A key allowed to call a Bedrock model could delete any object under the
deployment's buckets through DELETE /bedrock/v1/files/{s3 id}?model=...
because the managed-file ownership check only runs for unified ids. Raw
cloud storage ids now answer 403 on every delete route unless the caller
is a proxy admin; managed ids and require_managed_files are unchanged
_handler_may_close_client withholds the finalizer's close from a client that
someone else references. A streaming response is not one of those referrers: it
holds the connection it reads from, never the client, so the refcount says
"sole referrer" for exactly the client that is busiest, and __del__ tears the
pool down mid-body. Both handlers, both transports, and it is the shape #24929
reports: a provider's streaming call returns the response and drops the
handler, and get_async_httpx_client lets a cached handler go after an hour.
A streaming send now anchors the handler to the response it returns, so the
handler is finalized once the caller is done with the body and the ordinary
close applies with nothing in flight. The anchor is a weakref.finalize, which
holds the handler in its own registry rather than on the response. That keeps
the handler out of the response's reference cycle, so it is finalized by
refcount and can still schedule an async close, rather than inside a cyclic
collection that reaps its aiohttp session in the same pass; and a handler
serving several streams is released only once all of them are done, because
each anchor holds it separately.
Only a streaming send anchors. A non-streaming response has been read in full
by the time the method returns, so pinning the handler to it would delay every
client close behind whatever the caller does with the response.
The alternative was to ask the connection pool whether a request was in flight,
reusing EvictedClientCloser's check. It reads client._transport, so it answers
"idle" for any client with a proxy configured, where httpx routes through
client._mounts; and having found the client busy it can only poll, which never
terminates for a response the caller abandons unread, since httpx leaves that
connection checked out. The response's own lifetime is the condition both were
approximating.
Nothing calls it since the streaming pipeline detects rewrites at run time
through the stream observer, so the base method and the content filter's
override were dead code with dead tests
A policy that both adds a guardrail and steps it in a post_call pipeline
used to drop the guardrail from the request's guardrail list outright, so
its pre_call stage never ran. The per-hook loops already skip guardrails
by pipeline mode, so the mode-agnostic subtraction only lost coverage
CustomGuardrail.__init_subclass__ wrapped _StreamRewriteObserver.apply_guardrail in log_guardrail_information, so every streaming step recorded a second standard_logging_guardrail_information entry and span next to the inner guardrail's own. The observer's method now carries the marker that skips the wrapper. The step also adds the guardrail to the applied guardrails header the way the non-streaming unified path does, so streamed spend rows name the guardrail that scanned them
The streaming loop skipped every guardrail stepped by a post_call pipeline, even when the pipeline was dropped from the stream for lacking the unified apply_guardrail interface, so a default_on guardrail that only implements async_post_call_streaming_iterator_hook stopped governing streams it governed on the merge base. The skip set now comes from the pipelines that will gate the stream
Direct litellm.cost_per_token callers that name a Model Router deployment as
the model get the routing fee again, as they did before this branch, and the
fee is still charged exactly once on every completion_cost path. The
grok-4-20 entries bill cached prompt tokens at the input rate, since Azure has
no cached-input meter for them, and the model_router twin carries the same
limits and retirement date as model-router. The catalog test now exercises
the cost calculator and map relations instead of pinning map fields.
http_handler.py resolved to main. This branch deleted both __del__ methods;
main has since kept them and guarded them (#35981 refcount + ownership, #36670
loop-aware aiohttp session disposal), and added tests that assert a collected
handler does close an exclusively owned client. Taking this branch's deletion
would remove all of that, so the deletion is dropped here and the case main's
guard still misses -- a response in flight -- is fixed in the next commit.
Restores the public keyword removed at 415bdbfd8f. A direct caller that
names the Model Router as the request model gets the routing fee folded
into the prompt cost once; completion_cost never passes it and charges
the fee through the additional-costs hook as before
A deployment drop_params string that is not a flag value (a typo like ture) stayed silently off. The router now logs one warning per deployment. LITELLM_DROP_PARAMS and litellm_settings.drop_params share the same helper, so a non-flag value there warns as well instead of flipping silently from on to off
GET /v1/files through a provider config now returns the OpenAI page shape
(object list, data, first_id, last_id, has_more) instead of a bare array, and
DELETE /v1/files/{id} on a managed id answers the OpenAI FileDeleted shape with
deleted true instead of an empty body
Bedrock listing asks S3 for max-keys=0 when the purpose is one Bedrock never
stores under LiteLLM's prefixes, and batch_output listing no longer requires an
input bucket when only s3_output_bucket_name is configured. The mock request
behind the 400 for a foreign file id uses the same https://litellm.ai URL the
exception module uses
The router fee was folded into azure_ai.cost_per_token and then added again
by the additional_costs hook, so every routed request paid it twice. The hook
now owns the fee, the entry named by the deployment supplies the price, and a
response priced as the router entry itself is not charged again
model-router, gpt-chat-latest and cohere-command-a carry the limits from the
Foundry models page, and model-router and grok-4-20-* carry their retirement
dates. The router tests now run at the completion_cost level with a Logging
object, which is the path the proxy takes, and fail at the merge base
The failure hook's usage estimate and the project ITPM reservation both
called litellm.token_counter inline on the event loop, so a large request
that failed or hit the limiter stalled the gateway the same way the
count_tokens endpoints did. Both now run through asyncify. The loop-lag
probe the existing tests used moves into a shared helper that warms the
tokenizer first, and two new tests fail when either count runs inline
get_litellm_params normalizes drop_params once, so a client-body string and
router_settings.default_litellm_params reach the anthropic, bedrock, and
azure_ai gates as a bool. LITELLM_DROP_PARAMS=false now means off. A value
that is neither a flag nor a string logs one warning and counts as unset,
both in the deployment validator and in litellm_settings.
Marengo 3.0 requests now get a 400 naming any textTruncate, lengthSec,
useFixedLengthSec, or minClipSec parameter, and any video or audio option
sent with a text, image, text_image, or multi_input request, instead of
silently dropping them. drop_params (global, per deployment, or per
request) drops them instead. Pydantic validation errors name the field
and the reason, and the 3.0 marker is the exact "marengo-embed-3-" model
id segment.
The totals, the per-token-type lines and the reported rates each resolved off-peak pricing on their own clock read, so a quote taken as a window opened could bill on one side of the boundary and report rates from the other. /cost/estimate now pins a billing moment for the whole quote, and every rate lookup answers for the pinned moment instead of reading the clock again
Claude-Session: https://claude.ai/code/session_011Tn3657NkV6ojLqewL64Kb
The new metadata test sat at the top of tests/test_litellm. The azure_ai metadata tests live in tests/test_litellm/llms/azure_ai next to the cost calculator test, so this moves it there and bumps its repo-root lookup by the two extra directory levels. No test changes.
