test_google_login_only_threads_user_code_when_enabled cleared the whole
process environment for the duration of the call. mutmut's trampoline reads
os.environ['MUTANT_UNDER_TEST'] with a bare subscript, so the first
trampolined callee inside the block, _get_cli_sso_flow_or_raise, raised
KeyError. The bare `except Exception: pass` swallowed it and the assertion
then read call_args on a mock that was never called, which is where
"'NoneType' object has no attribute 'kwargs'" came from.
The test only needs the SSO provider variables unset, so it now preserves the
rest of the environment instead of clearing everything. google_login does not
raise here, so the try/except is gone and any future exception propagates; the
added assert turns a silent early return into a readable failure instead of an
AttributeError.
Root cause measured in a trampolined copy of the mutated folder with
MUTANT_UNDER_TEST=stats: the old test fails there with
KeyError: 'MUTANT_UNDER_TEST' inside _mutmut_trampoline, the new one passes.
That was the only test the mutation run could not execute, so the --deselect
comes back out and it counts toward the score again.
Two of the new lines tripped the ratcheting gate.
TQ005 flagged restoring litellm.model_cost by assignment. Dropped the
save/restore pair for monkeypatch.setitem, which adds the one model the
test needs and takes it back out at teardown, so the module global is
never reassigned.
TQ008 flagged patching litellm.proxy.proxy_server.llm_router. The
endpoint imports the router from that module inside the function body,
so there is no seam to inject through without changing the endpoint.
Suppressed with the reason already used elsewhere in the suite for the
same module global, on the single helper the new tests share.
The endpoint already had tests for deployments that set both an input and
an output price, and for litellm_params winning over model_info. Nothing
covered a deployment that prices only one of the two sides, the daily and
monthly totals, or the price and provider read from the public cost map.
Found by changing one line of cost_tracking_settings.py at a time and
running the mapped test file against each change. Nine of eleven one-line
changes went unnoticed: dropping custom pricing entirely when only one
side is priced, billing the unpriced side at something other than zero,
skipping the model lookup so the reported price and provider go empty,
turning zero requests a day into a cost of zero rather than no estimate,
and scaling a period total by one request instead of the real count.
The seven tests added here kill all eleven. The cost math is real; only
the router is faked, matching the fixtures already in this file.
PR #38114 dropped whichever header user_api_key_auth would read the caller's
key from, by precedence. Under custom_auth, JWT auth, or no master key that
header is the caller's own Google token, so the bring-your-own-credentials
Vertex branch answered 401 to every valid request.
A header value is now dropped only when it is the master key or when its
hash is the api_key that authenticated the request, so a Google token that
auth never consumed keeps flowing while a LiteLLM key still never reaches
Google.
test_passthrough_post_call_guardrails.py no longer plants a MagicMock
proxy_server module in sys.modules at import, which poisoned sibling tests
that read module globals at call time.
Keys minted by /key/generate get no LiteLLM_UserTable row, so /v2/model/info?user_models_only=true for such a user hit the new None guard and returned 400 where the merge base returned the user's own models. Skip the team-model merge for a missing row instead of raising, since a user with no row belongs to no team
The router's pre-content ping filter dropped AgenticAnthropicStreamingIterator's
hold-back keepalive, so a held-back turn sent the client nothing until the buffer
settled. A ping that no lifecycle frame precedes is now forwarded live, since a
fallback's message_start can still follow it without overlapping lifecycles
The proxy's cancel-refund guard checked isinstance against the iterator, but the
proxy only ever sees it behind FallbackAwareAnthropicMessagesStream and
AnthropicMessagesStreamingResponse, so a disconnect during hold-back refunded the
budget reservation anyway. Both wrappers now forward a duck-typed
has_buffered_provider_output flag, and the router wrapper follows a fallback
source so the flag tracks the stream actually being consumed
prisma-client-py delete() returns None when the row is already gone, so the
handler reported deleted: true for a row this caller never removed. Surface
the same 404 the read path uses instead
* fix(rerank): emit latency and cost headers on /rerank
Thread the logging object into the rerank httpx calls and pass hidden_params through to get_custom_headers, so x-litellm-overhead-duration-ms, x-litellm-response-duration-ms, x-litellm-response-cost, x-litellm-call-id and the LITELLM_DETAILED_TIMING x-litellm-timing-* headers show up on rerank like they do on chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rerank): keep zero response cost in the /rerank cost header
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: assign the new rerank endpoint tests to the proxy-endpoints shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: suppress TQ008 on the rerank header tests with reasons
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
* fix(proxy): make /team/member_delete's four cleanups atomic
The team roster update, the user.teams update, the team membership
delete, and the team-scoped verification token delete ran as four
sequential writes with no transaction around them, so a failure
between any two left the removal half applied. Thread a single
prisma transaction through all four writes, following the same
tx.<table> pattern /team/member_add and /team/member_update already
use, so either all four land or none do.
* fix(team): serialize member_add, member_delete, and delete under the team's advisory lock
/team/member_add validated a team exists and then wrote the user's teams array and
a membership row without holding anything across that gap, so a /team/delete could
commit its reference sweeps in between and leave a member pointing at a team id that
no longer exists. The write path already re-read members_with_roles under a row lock
before this change, but SELECT ... FOR UPDATE can deadlock with the access-group
endpoints, which lock an access group and then a team.
member_add now takes pg_advisory_xact_lock(hashtext(team_id)) before re-reading the
team and only writes if it is still there, so a delete that already committed is
visible before any write happens. delete_team takes the same lock around its own
row delete and reference sweep, so the two requests can never interleave: whichever
acquires the lock first runs to completion before the other's read can proceed.
Dropping the row lock from member_add's read also dropped the incidental protection
it gave against a concurrent member_delete, which still wrote from the snapshot it
validated against, unlocked, and could silently overwrite whatever member_add had
just committed. member_delete now takes the same advisory lock and re-reads the
roster under it before computing its own write, so it can never resurrect a member
by overwriting from stale data.
Resolves LIT-5544
* fix(team): run member writes on the advisory lock's transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): keep member writes on the lock holder's connection after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): keep the transactional member create an upsert on user_id
The transaction path was creating the email-identified user row outright, where the
regular client path upserts on user_id. Share one upsert helper between both member
paths so the create stays idempotent on the lock holder's connection.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): read member_delete's user and key rows on the lock-holding transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(scim): return user_id as Group members[].value on transformed group responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: format scim transformation tests per ruff
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop redundant assertion comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: retrigger ci
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): don't show a stale provider prompt-cache chip on a response-cache hit
The playground's non-streaming chat completion and responses paths replayed a cache hit's original usage payload verbatim, so ResponseMetrics kept rendering the provider's prompt-cache-write/read chips using token counts from the original request. Detect the hit via the x-litellm-cache-key response header and render a Response Cache indicator instead.
* fix(dashboard): expose x-litellm-cache-key through CORS for the playground cache-hit indicator
* fix(proxy): stop expected 4xx responses from saturating worker CPU on failure logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep threaded sync failure handler so CustomLogger sync callbacks still run
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: remove redundant comments per review
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The model Azure Model Router served was recovered by checking whether the text
"model_router" or "model-router" appeared in a model string. Spend logs applied that
check to the litellm model path, where the route prefix guarantees a match, but the
proxy applied it to the client's model group alias, which carries no prefix. A model
group named anything else therefore lost the selected model in both the response and
the spend row.
AzureModelRouterConfig now stamps the served model onto _hidden_params, and the spend
log payload and the proxy's response restamping read that stamp. The name heuristic
survives as a fallback for callers with no response in hand, routed through
get_azure_ai_route so it lives in one place.
* fix(proxy): reset a stuck team member's budget
A per-team-member budget check reads a cross-pod spend counter that
nothing ever invalidates. Once a member exceeds their per-member
budget, resetting the key's spend, raising the user's or the team's
own budget, or issuing a new key all leave the member stuck, because
none of them touch this counter or its cached membership object.
Add POST /team/{team_id}/member/{user_id}/reset_spend to reset a
member's tracked spend, and invalidate the same cached state from
/team/member_update when it raises a member's own budget, so that
path also takes effect immediately instead of waiting on the
membership cache's TTL. Name the entity in the check's error message
so a stuck member is diagnosable from the 429 body alone.
* fix(proxy): close reset-vs-floor-read race and surface double Redis write failure on member spend reset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast spend reset as a SET so the handler's self-delivered message cannot erase the reset guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): omit null fields from the invalidation message so plain evictions keep the old wire format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Every repository handed its `.table` back untyped, so a dozen modules had
each grown a private `_PrismaTableActions` Protocol to paper over it. They
had drifted: some declared `update` as returning the row, others the row or
None, and none agreed on whether `find_many` was covariant
Replace all of them with a single `TableActions[RowT_co]` in
`litellm/repositories/prisma_protocols.py`, keyed to the prisma row each
repository is bound to. Query inputs stay `Mapping[str, object]` so callers
keep passing plain dicts, and `find_many` returns `Sequence` so the row type
stays covariant
Typing the nullable returns honestly surfaced paths that were already
crashing. A team admin could never edit or delete a memory entry owned by
their team: the write-auth check fed a raw prisma row to a helper that
expects the domain model, so `members_with_roles` arrived as plain dicts and
the request died as a 500 instead of applying the edit. Non-admin members hit
the same 500 in place of the 403 they were owed, so refusal and breakage were
indistinguishable. `/v2/model/info?user_models_only=true` dereferenced a
missing user row rather than returning the 400 the route already had, three
team routes dereferenced a team deleted between the read and the write, and
the agent registry dereferenced a missing agent instead of naming it
basedpyright drops 2,132 errors, 1,454 of them reportAny and 73
reportExplicitAny. The dashboard's generated types pick up `string[]` where
they had `unknown[]` for a team's members, admins and models
/global/activity/cache_hits now returns an error_breakdown: failed spend
logs bucketed per call_type by error code and error class, read from
metadata->error_information. Clicking a red failed-requests segment on
the cache activity chart opens a per-code bar chart; hovering a bar
lists the error classes behind that code.
The video edit endpoint parsed the multipart body but dropped the uploaded
source video, only normalizing it to an id. When a raw file is uploaded it now
flows through videos.main -> the http handler -> the provider transform, which
emits multipart/form-data with the source video as a file part, matching the
official OpenAI SDK's videos.edit wire format. Edit-by-id still egresses JSON.
On mapped pass-through routes, of which /vertex_ai is one,
user_api_key_auth accepts the caller key from a header literally named
litellm_user_api_key and applies it last, so it overrides every other source.
The credential-less filter neither dropped it nor resolved the caller key from
it, so a virtual key there reached Google past a real x-goog-api-key, and a
bring-your-own Authorization could be stripped when auth actually came from that
header. Drop it by name and resolve it at highest precedence.
