Every lite login inserted a row that was never removed, so the registry grew without bound as logins accumulated. Registration now deletes rows whose expires_at has passed, bounding the table by the number of live sessions.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The two new CLI-session test files were added before #37802 swept the
suite's sys.path.insert(0, os.path.abspath(...)) calls, so rebasing onto
staging put them over the TQ003 test-quality-budget ceiling and failed lint.
`lite login` mints a self-contained encrypted UserAPIKeyAuth blob rather than a
virtual key, so nothing about the session was stored server side and the auth
path authenticated it by decrypting it. An admin had no way to see who held a
live CLI credential and no way to cut one off before it expired.
Every login now registers a row in LiteLLM_CLISessionTable, keyed by the sha256
of the session token, and the auth path refuses a session whose row is revoked.
The lookup is cached in the shared auth DualCache for DEFAULT_IN_MEMORY_TTL, so
a session costs one read per cache interval per replica and a revoke converges
on every replica within that same interval. A CLI Sessions page in the dashboard
lists the sessions with their user, issue time and expiry, and revokes one.
Registration is fail-closed: a session that cannot be recorded is never handed
to the CLI, since it could never be revoked. Sessions minted before this change
have no row and keep working until they expire. A lookup that cannot reach the
database follows the proxy-wide allow_requests_on_db_unavailable posture rather
than inventing its own, so the credential's availability during an outage is
whatever the operator already chose for every other DB-backed auth read.
* fix(dashboard): don't show a stale provider prompt-cache chip on a response-cache hit
The playground's non-streaming chat completion and responses paths replayed a cache hit's original usage payload verbatim, so ResponseMetrics kept rendering the provider's prompt-cache-write/read chips using token counts from the original request. Detect the hit via the x-litellm-cache-key response header and render a Response Cache indicator instead.
* fix(dashboard): expose x-litellm-cache-key through CORS for the playground cache-hit indicator
* fix(proxy): stop expected 4xx responses from saturating worker CPU on failure logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep threaded sync failure handler so CustomLogger sync callbacks still run
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: remove redundant comments per review
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The edit form's Authorize & Fetch Token button built its temporary OAuth
session payload without issuer, authorization_url, token_url, or
registration_url, unlike the create form's equivalent payload builder. The
backend's temporary-session endpoint builds its ephemeral server purely from
that payload, so any admin-configured OAuth endpoints on an existing server
were silently dropped, endpoint discovery fell back to (and failed against)
the plain server url, and Authorize & Fetch Token 400'd with "authorization
url is not configured" even though the saved server had those fields filled
in. Add the four missing fields to the edit form's temporary payload builder,
mirroring the create form.
LoggingWorker now carries still-queued coroutines onto the next event loop (12a34a10d8). Under xdist,
a success-logging coroutine queued by test_acompletion_mcp_respects_manual_approval ran nine seconds
later inside test_mcp_tool_call_hook on the same worker, resolved litellm.callbacks at run time and
overwrote that test's captured payload with a gpt-4o-mini completion (assert 1.35e-05 == 1.42).
Run clear_queue() in the suite's autouse teardown so every coroutine a test enqueues finishes before the
next test registers its callbacks, and add a subprocess regression test that runs the real conftest
against a stopped worker with work still queued.
The model Azure Model Router served was recovered by checking whether the text
"model_router" or "model-router" appeared in a model string. Spend logs applied that
check to the litellm model path, where the route prefix guarantees a match, but the
proxy applied it to the client's model group alias, which carries no prefix. A model
group named anything else therefore lost the selected model in both the response and
the spend row.
AzureModelRouterConfig now stamps the served model onto _hidden_params, and the spend
log payload and the proxy's response restamping read that stamp. The name heuristic
survives as a fallback for callers with no response in hand, routed through
get_azure_ai_route so it lives in one place.
Router._add_deployment called get_llm_provider without the deployment's api_base, so a config entry with a bare model plus a known OpenAI-compatible endpoint failed startup validation with LLM Provider NOT provided and the proxy returned 400 no healthy deployments for that model group. acompletion had the same gap at request time: it forwarded only base_url into its get_llm_provider call, dropping the api_base kwarg the router passes. Both now forward api_base so endpoint matching resolves the provider the same way sync completion already does
Together AI moved its canonical API host from api.together.xyz to
api.together.ai. Default the provider api_base and the rerank handler to
the new host, make rerank honor api_base and TOGETHER_AI_API_BASE like
chat already does, map both hosts to together_ai when passed as
api_base, and delete the dead models/info fetch in factory.py.
Mantle 400s ("Invalid 'input': value did not match any expected variant")
on the Codex history item types agent_message, context_compaction, and
local_shell_call, killing every Codex multi-agent session on the first
sub-agent turn. Rewrite agent_message into an assistant output_text message
(preserving encrypted_content slot payloads, which carry the plaintext task
through Mantle), context_compaction into Mantle's supported compaction
spelling, and local_shell_call into the function_call its recorded
function_call_output already pairs with.
* fix(proxy): reset a stuck team member's budget
A per-team-member budget check reads a cross-pod spend counter that
nothing ever invalidates. Once a member exceeds their per-member
budget, resetting the key's spend, raising the user's or the team's
own budget, or issuing a new key all leave the member stuck, because
none of them touch this counter or its cached membership object.
Add POST /team/{team_id}/member/{user_id}/reset_spend to reset a
member's tracked spend, and invalidate the same cached state from
/team/member_update when it raises a member's own budget, so that
path also takes effect immediately instead of waiting on the
membership cache's TTL. Name the entity in the check's error message
so a stuck member is diagnosable from the 429 body alone.
* fix(proxy): close reset-vs-floor-read race and surface double Redis write failure on member spend reset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast spend reset as a SET so the handler's self-delivered message cannot erase the reset guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): omit null fields from the invalidation message so plain evictions keep the old wire format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
