The endpoint answers a probe that exceeds HEALTH_CHECK_TIMEOUT_SECONDS with
HTTP 200 and an in-body "Timeout exceeded", which the harness's status-code
rerun policy cannot see. The suite's parallel Bedrock load can push a Mantle
probe past that cap transiently, so only that exact error is retried, three
bounded attempts with visible prints; any other error verdict still fails
immediately.
The edit form's Authorize & Fetch Token button built its temporary OAuth
session payload without issuer, authorization_url, token_url, or
registration_url, unlike the create form's equivalent payload builder. The
backend's temporary-session endpoint builds its ephemeral server purely from
that payload, so any admin-configured OAuth endpoints on an existing server
were silently dropped, endpoint discovery fell back to (and failed against)
the plain server url, and Authorize & Fetch Token 400'd with "authorization
url is not configured" even though the saved server had those fields filled
in. Add the four missing fields to the edit form's temporary payload builder,
mirroring the create form.
LoggingWorker now carries still-queued coroutines onto the next event loop (12a34a10d8). Under xdist,
a success-logging coroutine queued by test_acompletion_mcp_respects_manual_approval ran nine seconds
later inside test_mcp_tool_call_hook on the same worker, resolved litellm.callbacks at run time and
overwrote that test's captured payload with a gpt-4o-mini completion (assert 1.35e-05 == 1.42).
Run clear_queue() in the suite's autouse teardown so every coroutine a test enqueues finishes before the
next test registers its callbacks, and add a subprocess regression test that runs the real conftest
against a stopped worker with work still queued.
The model Azure Model Router served was recovered by checking whether the text
"model_router" or "model-router" appeared in a model string. Spend logs applied that
check to the litellm model path, where the route prefix guarantees a match, but the
proxy applied it to the client's model group alias, which carries no prefix. A model
group named anything else therefore lost the selected model in both the response and
the spend row.
AzureModelRouterConfig now stamps the served model onto _hidden_params, and the spend
log payload and the proxy's response restamping read that stamp. The name heuristic
survives as a fallback for callers with no response in hand, routed through
get_azure_ai_route so it lives in one place.
Router._add_deployment called get_llm_provider without the deployment's api_base, so a config entry with a bare model plus a known OpenAI-compatible endpoint failed startup validation with LLM Provider NOT provided and the proxy returned 400 no healthy deployments for that model group. acompletion had the same gap at request time: it forwarded only base_url into its get_llm_provider call, dropping the api_base kwarg the router passes. Both now forward api_base so endpoint matching resolves the provider the same way sync completion already does
Together AI moved its canonical API host from api.together.xyz to
api.together.ai. Default the provider api_base and the rerank handler to
the new host, make rerank honor api_base and TOGETHER_AI_API_BASE like
chat already does, map both hosts to together_ai when passed as
api_base, and delete the dead models/info fetch in factory.py.
Adds live e2e coverage for the Bedrock combinations behind recent customer
incidents: llm_provider-* response-header forwarding on /chat/completions
(nonstream and stream), regional us.anthropic.* inference-profile ids over
the invoke route, and the Admin UI Test Connection probe for a
responses-mode Bedrock Mantle deployment. Registers the matching cells in
the coverage registry and publishes the provider x feature matrix table in
its README.
Backfill 21 serverless chat models, the multilingual-e5 embedding model, and
Llama-Guard-4-12B from the live Together catalog with per-token pricing and
capability flags. Mark 25 delisted together_ai entries with their documented
deprecation_date and point superseded models at a live successor via metadata.
Reprice Llama-3.3-70B-Instruct-Turbo to Together's current rate.
Mantle 400s ("Invalid 'input': value did not match any expected variant")
on the Codex history item types agent_message, context_compaction, and
local_shell_call, killing every Codex multi-agent session on the first
sub-agent turn. Rewrite agent_message into an assistant output_text message
(preserving encrypted_content slot payloads, which carry the plaintext task
through Mantle), context_compaction into Mantle's supported compaction
spelling, and local_shell_call into the function_call its recorded
function_call_output already pairs with.
* fix(proxy): reset a stuck team member's budget
A per-team-member budget check reads a cross-pod spend counter that
nothing ever invalidates. Once a member exceeds their per-member
budget, resetting the key's spend, raising the user's or the team's
own budget, or issuing a new key all leave the member stuck, because
none of them touch this counter or its cached membership object.
Add POST /team/{team_id}/member/{user_id}/reset_spend to reset a
member's tracked spend, and invalidate the same cached state from
/team/member_update when it raises a member's own budget, so that
path also takes effect immediately instead of waiting on the
membership cache's TTL. Name the entity in the check's error message
so a stuck member is diagnosable from the 429 body alone.
* fix(proxy): close reset-vs-floor-read race and surface double Redis write failure on member spend reset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast spend reset as a SET so the handler's self-delivered message cannot erase the reset guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): omit null fields from the invalidation message so plain evictions keep the old wire format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Read strict from the caller's output_format/output_config.format instead
of hardcoding true, defaulting to false to match OpenAI's API default.
Explicit true/false values are preserved and output_format still takes
precedence over output_config.format.
* fix(http_handler): dispose aiohttp session when finalized without a running loop
AsyncHTTPHandler.__del__ can only schedule an async close when a running
event loop exists at finalization time; in any other context (worker
threads whose loop has closed, sync contexts, interpreter shutdown) the
RuntimeError from get_running_loop() is swallowed and the underlying
aiohttp ClientSession is abandoned to GC, emitting 'Unclosed client
session' / 'Unclosed connector' warnings.
This is the disposal gap left after the recycle-time fix: clients created
for short-lived event loops (the loop-id-keyed LLM client cache mints one
handler per loop) are never recycled - they live and die with their loop,
and their finalization is precisely the loop-less case.
Fix:
- no running loop: fall back to the connector's synchronous teardown via
LiteLLMAiohttpTransport._mark_connector_closed - the same finalizer-safe
path used for dead-loop recycles - honoring _owns_session so a shared
session is never closed.
- running loop: keep the async close, but hold a strong reference to the
scheduled task until it completes (a bare create_task() result may be
collected before running), mirroring _background_close_tasks.
Tests: loop-less finalization closes a dead-loop session; running-loop
finalization registers and drains the close task; the sync fallback
respects session ownership. All three fail without the fix.
* lint: conform new finalizer code to the type-discipline budget
Final on the five never-rebound locals (LIT010); the class-level task
registry keeps its mutable set with the sanctioned mutable-ok reason,
mirroring the aiohttp transport's registry (LIT001).
* lint: reasoned pyright ignore on the cross-class teardown call
The handler deliberately reuses the transport's finalizer-safe connector
teardown; no public seam exists and an async close can never run at
loop-less finalization. Clears the net-new reportPrivateUsage the
basedpyright budget gate flagged once the LIT stage passed.
* fix(http_handler): retrieve exceptions from finalizer close tasks
A bare discard done-callback dropped the task without consuming its
exception, so a failing aclose() emitted "Task exception was never
retrieved" at GC, the same noise class this path exists to remove.
Mirror the transport's _on_close_task_done: discard, early-return on
cancellation, retrieve and debug-log the exception.
* fix(http_handler): dispose foreign-loop sessions instead of scheduling aclose on the live loop
GC on a live loop (e.g. the app's) of a handler whose session belongs to
another, possibly dead, loop scheduled aclose() on the current loop, the
cross-loop path the transport refuses. Route both that case and the
loop-less case through the transport's lifecycle-aware
_close_recycled_session, which picks async close on the session's own
loop, threadsafe handoff, or the synchronous connector teardown.
Regression test: a dead-loop session collected while another loop runs
is disposed without scheduling anything on that loop.
* chore: retrigger CI (test_mcp_logging payload-order flake, also failed on litellm_spendlogs_fallback_metadata minutes earlier)
* test(mcp): select the MCP tool-call payload instead of the last-delivered one
TestMCPLogger kept a single last-writer slot; an async success event from
another call (a mocked acompletion whose log task lands late) races the
MCP event for it, so the cost assertions intermittently read the wrong
payload. This PR's finalizer change shifts task interleaving on the loop
and tips that latent race over (also seen on an unrelated PR minutes
earlier). Collect call_type=call_mcp_tool payloads in their own list and
assert on those.
* test(mcp): MCPLoggerHook inherits the order-independent payload capture
It duplicated TestMCPLogger's init and success handler verbatim; the
hook test reads the same MCP payload selection, so subclass instead.
