Two of the new lines tripped the ratcheting gate.
TQ005 flagged restoring litellm.model_cost by assignment. Dropped the
save/restore pair for monkeypatch.setitem, which adds the one model the
test needs and takes it back out at teardown, so the module global is
never reassigned.
TQ008 flagged patching litellm.proxy.proxy_server.llm_router. The
endpoint imports the router from that module inside the function body,
so there is no seam to inject through without changing the endpoint.
Suppressed with the reason already used elsewhere in the suite for the
same module global, on the single helper the new tests share.
The endpoint already had tests for deployments that set both an input and
an output price, and for litellm_params winning over model_info. Nothing
covered a deployment that prices only one of the two sides, the daily and
monthly totals, or the price and provider read from the public cost map.
Found by changing one line of cost_tracking_settings.py at a time and
running the mapped test file against each change. Nine of eleven one-line
changes went unnoticed: dropping custom pricing entirely when only one
side is priced, billing the unpriced side at something other than zero,
skipping the model lookup so the reported price and provider go empty,
turning zero requests a day into a cost of zero rather than no estimate,
and scaling a period total by one request instead of the real count.
The seven tests added here kill all eleven. The cost math is real; only
the router is faked, matching the fixtures already in this file.
Adds `general_settings.model_list_healthy_only`, which makes `/models`,
`/v1/models/{id}` and `/model/info` hide models whose backing deployments are
all marked unhealthy by background health checks, for every caller, without
each client having to pass `healthy_only=true`. `/model/info` also gains the
per-request `healthy_only` parameter that `/v1/models` already had.
Everything here is opt-in. With the setting absent, the endpoints take the same
code path they do today and no health lookup runs at all.
The listing filter reads the deployment health cache, which until now was only
populated when `enable_health_check_routing` was on, so `healthy_only=true`
silently did nothing in a plain `background_health_checks` setup. The setting
now also keeps that cache filled. That is a pure write: every routing-time
reader is itself gated on `enable_health_check_routing`, and the cooldown and
failure bookkeeping stays behind that flag, so routing is untouched.
Filtering stays presentation-only and fails open. A hidden model is still
callable, and missing, stale or empty health state hides nothing.
Now that /v1/messages routes provider failures through exception_type, an
Anthropic permission_error fell through the anthropic branch to the generic
APIConnectionError and reached the client as a 500 where the raw exception
used to answer 403. Map 403 to PermissionDeniedError so the status survives
on every route.
Bugbot Autofix pushed e2e16d7e2d to split 403 out of the shared 401/403 branch in _map_openai_like_exception. That premise was the BaseLLMException fallback, which d2e4e74685 already removed, and remapping 403 for every openai-like provider is a separate contract change, so this merge resolves both files back to the base branch versions
The step-timeout comment claimed mutmut streams each mutant's result into
mutants/mutmut-stats.json. It does not. That file holds the pre-run test
timings and coverage map (tests_by_mangled_function_name, duration_by_test,
stats_time) written once by save_stats() before mutation starts.
Per-mutant results live in mutants/<source path>.meta. Verified against
mutmut 3.5.0: SourceFileMutationData.register_result() calls save() after
every single result, and export-cicd-stats walks those .meta files to build
mutmut-cicd-stats.json. So the reason the step deadline exists is still
right, an interrupted run keeps the mutants it already scored, but the
comment pointed at the wrong file.
Also upload the .meta files, since they are the partial results the comment
relies on and the artifact could not otherwise show them.
The fallback mapped every unbranched provider error by status code on every route, which changed the exception class and HTTP status for those providers and failed four provider test suites in CI. The /v1/messages handler change alone covers the ticket, since the anthropic branch already maps its errors
The fixture was matched by host alone, so it answered any method and path and
the tests would have stayed green if the request went somewhere else. It now
matches POST on the Foundry route, and asserts the route was called.
Worth pinning on its own: the real path carries a /models prefix, which the
first attempt at this got wrong, so the match now also holds the routing in
place rather than only the retry.
The source drops the rejected field in place, so a payload shared across
tests could in principle be consumed by whichever case ran first. It does
not happen today, because the request is copied before the transform runs,
and the cases pass in reverse and async-first order alike. Building the
payload per call costs nothing and keeps that true if the copy ever goes.
Azure AI is the only provider that retries a 422 inside the translation
layer: when the endpoint rejects a field, litellm drops that field and sends
the request again, up to twice. That is the difference between a customer's
tool call working and coming back as a hard 400, and none of it was covered.
The retry loop in llm_http_handler.py is 13,419 lines of source against a
0.20 test-to-source ratio, and nothing exercised this path at all.
Drives real litellm.completion and litellm.acompletion calls against a
recorded Azure AI endpoint, so the assertions read the bytes that actually
went over the wire rather than a mock's call list. Nothing internal is
patched: respx fakes the HTTP boundary and the provider config, retry loop
and serialization are all the real ones.
Pins:
- a tool field the endpoint rejects is dropped and the call retried, and the
caller gets a normal completion
- the retry changes only the field the provider named
- a provider that keeps rejecting stops after exactly two attempts
- a rejection the provider cannot fix is not retried at all
- an extra input outside a tool is retried only when drop_params was asked for
Mutating the source confirms these bite: raising the retry cap from 2 to 3,
and making the tool-level field check always return False, each turn the
suite red.
The async cases pin the transport to httpx, because the aiohttp default
carries its own transport that an httpx-level fake cannot intercept. Without
that the two async tests reached the real Azure endpoint and failed on a 401.
proxy/_types.py is 4,965 lines holding 202 request and auth models with 27
validators, and its mapped test file was 32 lines covering one of them. The
validators decide what a caller is allowed to send, so a silent change here
reaches customers as a request that should have been refused and wasn't, or
the reverse.
Pins the contracts that carry real consequence:
- the server-only MCP markers and via_virtual_key are stripped from any
caller-supplied input, so they cannot be forged through the constructor or
model_validate, while the server can still set them by assignment
- a virtual key is hashed out of the auth object, and Bearer-prefixed and
bare keys hash alike
- a JWT issuer must name an audience or opt out of one, never both and never
neither
- a boolean spend reset is refused rather than silently read as 1.0 or 0.0
- a key or user update must say which key or user it updates
- a key lookup naming nothing is refused rather than matching everything
- an organization member cannot be given a role that lives outside an
organization
- an audit log stores the key it recorded a change to only masked, and keeps
the non-secret fields intact
Every case asserts the observed value rather than that a call happened, and
nothing is patched. Verified by mutating the source: dropping the marker
strip, flipping the audience rule's and to or, letting booleans through the
spend reset, treating an empty key list as naming a key, and disabling the
role check each turn the suite red.
Moves the file to the path that mirrors litellm/proxy/_types.py, which the
old file's own first line already said it should have been at, and carries
its two tests over.
`_delete_cache_key_object` awaited the Redis delete unguarded, so any cache
backend error surfaced as a failure on an operation that had already been
committed. A Redis ACL that denies DEL on LiteLLM's unprefixed token-hash keys
turned a persisted `/key/update` into `400 Authentication Error, No permissions
to access a key`, and `/key/block` and `/key/regenerate` into 500s
Make the helper best-effort, the way `delete_cache_team_object` and
`delete_cache_key_objects` on either side of it already are: log the failure and
carry on. Nothing ends up staler for it, since the in-memory entry is dropped
before the Redis round trip and the write has already committed, so raising only
misreported a success
Virtual Keys, Budgets, Projects, Access Groups, Guardrails Monitor and Cost
Optimization all move onto the shared PageHeader, matching the Teams page.
That empties LegacyPageHeader, so it and its test are deleted.
Each page now uses its own sidebar icon, so the nav and the page agree:
Virtual Keys keeps KeyRound and Budgets keeps Wallet, Projects picks up
Folder and Access Groups picks up Boxes, and Guardrails Monitor swaps the
indigo Shield for the sidebar's HeartPulse. Cost Optimization keeps
PiggyBank but drops its hardcoded size and stroke, which PageHeader owns.
Control rows follow the spec instead of each page inventing one. Virtual
Keys had its create button rendered as a sibling below the header, Budgets
hand-rolled a row with a bottom border that closed the header off, and
Projects and Access Groups sat their button next to the title. All four now
pass primaryAction. Guardrails Monitor's date picker moves out of the parent
and joins Export Data in utilities. Cost Optimization's tabs move into the
tabs slot with the standard 22px spacing.
Page insets go to p-8 with a 24px gap to content, replacing p-6 px-12,
p-6, mx-4 and py-2.
Every page test now asserts its heading, subtext and sidebar icon. Swapping
any of the six icons fails its suite.
mutmut's gather_coverage() looks each source file's covered lines up by
absolute path, but [tool.coverage.run] sets relative_files = true, so every
lookup misses. With mutate_only_covered_lines = true that leaves no line
eligible for mutation, and the run ends on "Stopping early, because we could
not find any test case for any mutant" after spending 26 minutes collecting
coverage. The last four dispatches all died that way.
Point COVERAGE_RCFILE at a small rc file for mutation runs only, so the
coverage instance mutmut builds stores absolute paths. Scoped to one module
locally this takes the run from 0 mutants to 8 generated and 8 killed.
Also give the mutmut step a deadline inside the job's own. mutmut records
each mutant's verdict to mutants/mutmut-stats.json as it finishes, so a run
that outlasts its budget still scores what it got through, but a cancelled
job skips the report and upload steps and publishes nothing. That is how the
two runs before these four ended.
Ignore mutants/ and .venv-mutmut, which a local run leaves behind untracked.
* feat(prometheus): configure deployment caller identity
* test(prometheus): satisfy strict caller identity lint
* fix(prometheus): align caller identity on latency metrics
* fix(prometheus): validate caller identity mode before collectors register
Fail config load on an invalid prometheus_deployment_and_latency_caller_identity
value (including null) and on include_labels entries the selected mode removes
from a target metric, instead of booting green with an empty /metrics.
Validate the mode at the top of PrometheusLogger.__init__ so an invalid value
raises before any collector lands in the process-global registry, keeping
retries free of duplicated-timeseries errors. Label-validation errors now name
the mode setting alongside the rejected label.
---------
Co-authored-by: Mark Philipp <mphilipp622@gmail.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
These four primitives still wrapped their body in React.forwardRef, which
the dashboard has not needed since it moved to React 19: a function
component receives ref as an ordinary prop and the existing {...props}
spread already hands it to the DOM node.
Re-pulling each from base-vega drops the wrapper and its displayName.
These four were picked because the ref plumbing is their only divergence
from current upstream, so the class strings, data-slot values and exports
are untouched and nothing renders differently. The other seven primitives
that still carry forwardRef have also drifted on their class strings, so
re-pulling them would ship a visual change alongside the cleanup and they
are left alone here.
Textarea is the one with real ref call sites, roughly seventeen of them
through react-hook-form's field.ref, and ref-forwarding.test.tsx did not
cover it. Add that case next to the Label, Separator and Skeleton ones
already there.
components/shared/Alert.tsx was base-vega's own alert.tsx copied in by
hand, carrying the same four exports and the same class strings, so
npx shadcn add could never reach it and it would drift from every
upstream fix silently. It also still wrapped each part in forwardRef,
which React 19 no longer needs.
Install the primitive into components/ui/ where the CLI can update it,
and reduce the shared file to a wrapper that adds the four status
variants (info, success, warning, error) the dashboard actually uses on
top of upstream's default and destructive.
Rendered output is unchanged: every variant produces byte-identical
classes, role and data-variant, so all 45 call sites look the same.
PR #38114 dropped whichever header user_api_key_auth would read the caller's
key from, by precedence. Under custom_auth, JWT auth, or no master key that
header is the caller's own Google token, so the bring-your-own-credentials
Vertex branch answered 401 to every valid request.
A header value is now dropped only when it is the master key or when its
hash is the api_key that authenticated the request, so a Google token that
auth never consumed keeps flowing while a LiteLLM key still never reaches
Google.
test_passthrough_post_call_guardrails.py no longer plants a MagicMock
proxy_server module in sys.modules at import, which poisoned sibling tests
that read module globals at call time.
The auth handler stamps the requested model's provider onto BudgetExceededError before logging it, which made a key-over-budget 429 look provider-originated and regain its traceback (and an OTel stack_trace) after the provider 4xx carve-out. Any exception whose unified rate-limit category names litellm's own limiter is now a proxy rejection, matching the HTTPException rule.
ProxyRateLimitError derives from HTTPException but carries an llm_provider,
so it read as provider-originated and regained its traceback. Any
HTTPException is a proxy rejection regardless of llm_provider.
The /v1/messages route logs the provider's raw BaseLLMException, which carries
no llm_provider, so its 4xx still counted as an expected client error and lost
its traceback. Treat BaseLLMException as provider-originated as well.
is_expected_client_error treated every HTTP 4xx as a rejection the proxy
issued itself, so a 401 or 429 the provider returned lost its traceback in
the standard logging payload and the OTel error span dropped
litellm.provider.error.stack_trace. An exception carrying llm_provider is
an upstream or deployment problem and keeps its traceback; the proxy's own
pre-call rejections still skip it
Document the intentional internal seams used by the DCR bridge admission tests and normalize import ordering.\n\nGenerated with AI\n\nCo-Authored-By: Codex
Preserve standard Authorization key validation while preventing client MCP credentials from receiving anonymous bridge admission.
Generated with AI
Co-Authored-By: Codex
* feat(logging): add async_post_call_failure_deployment_hook
CustomLogger already has async_pre_call_deployment_hook and
async_post_call_success_deployment_hook, both firing once per real
deployment attempt from wrapper_async since the router re-enters that
wrapper fresh on every retry and fallback step. There was no failure-side
counterpart; the only failure signal, async_log_failure_event, fires once
per logical client request behind a dedup gate, so fallback chain attempts
2+ were invisible to callbacks needing per-deployment-attempt granularity.
Adds async_post_call_failure_deployment_hook(request_data, exception,
call_type) to CustomLogger and a matching dispatcher in utils.py, called
from wrapper_async's except block. It needs no dedup coordination since
each real attempt naturally re-enters the wrapper once. Unlike its two
siblings, the dispatcher wraps each callback call in its own try/except
since it runs on the wrapper's own exception path and a broken callback
must never mask the exception about to be re-raised to the caller.
* feat(logging): pass fallback_depth through to async_post_call_failure_deployment_hook
Router already tracks fallback_depth internally on each fallback hop
(litellm/router_utils/fallback_event_handlers.py), incrementing it once per
target tried, but nothing surfaced it to CustomLogger callbacks. Reads it
off request_data in the dispatcher and passes it through as a best-effort
int | None keyword: None on the first, pre-fallback attempt or a bare SDK
call with no router, 1 on the first fallback hop, 2 on the second, and so
on. Verified live against a real multi-hop Router fallback chain before
adding the regression tests.
* fix(logging): fire async_post_call_failure_deployment_hook on internal calls too
The failure hook was gated behind the same not _is_litellm_internal_call
check as the request-level dedup-gated failure logging, so a failed
internal sub-call (e.g. an emulated file-search step) never reached it,
even though its async_pre_call_deployment_hook and
async_post_call_success_deployment_hook siblings already fire
unconditionally for such calls.
* chore: retrigger CI (lint job hit a transient GitHub Actions infra outage on the prior push)
* chore: retrigger CI (lint job hit the same GitHub Actions infra outage again)
* fix(logging): scope async_post_call_failure_deployment_hook to the actual model call
The hook was dispatched from the wrapper's broad outer except, which also
catches BudgetExceededError (raised before any deployment attempt),
errors from async_pre_call_deployment_hook, and errors raised after a
successful model call (post_call_processing, async_post_call_success_deployment_hook,
caching). None of those are a deployment attempt failing, so the hook
misreported them as one.
Scoped the hook to a try/except around the model call itself, so it only
fires when that specific call raises, matching its own documented contract.
* test: assert the callback actually ran in the failure-hook error-isolation test
An upstream test-quality gate (TQ001) flagged this test for asserting
nothing, so it could only fail by raising. Track whether the exploding
callback actually ran and assert on it, so the test would catch a
dispatcher that silently skipped every callback instead of isolating a
raising one.
* fix(logging): harden async_post_call_failure_deployment_hook against 5 maintainer-verified issues
A maintainer's live-proxy A/B review against base found five real
problems with the failure hook, all reproduced and fixed:
- The dispatcher called overrides with fallback_depth as a required
keyword, so an override matching this PR's own earlier 3-arg
proof-of-fix example raised TypeError, swallowed at debug level, on
every call. Now checks the override's signature once per class and
omits the keyword when unsupported.
- A callback mutating the exception it receives (e.g. status_code)
changed what the real caller got back, since it was the same live
object about to be re-raised. Callbacks now receive a same-class
snapshot instead.
- request_data exposed attempted_targets, the router's own live
fallback-walk bookkeeping shared by reference across every hop, so a
callback calling .record() on it could make the router skip a
deployment it never actually tried. Now excluded from what the hook
receives.
- The hook's own await sat directly in the model-call except block, so
a caller-side cancellation landing mid-await (e.g. asyncio.wait_for)
replaced the real deployment exception with CancelledError/
TimeoutError. Now isolated so hook dispatch can never mask the real
failure.
- The timestamp used for the reported failure duration was captured
after the hook ran, so a slow callback inflated
async_log_failure_event's duration. Now captured before the hook
dispatches.
* fix(logging): preserve traceback/cause/context on the failure-hook exception snapshot
Bugbot found a real gap in the previous round's exception-mutation fix:
_snapshot_exception_for_hook only copied __dict__ and args, so a
callback formatting or inspecting the failure chain saw an empty
traceback and lost chained-exception context, even though the live
exception still has them. __traceback__/__cause__/__context__ aren't
stored in __dict__, so they need copying explicitly.
* fix(logging): preserve __suppress_context__ on the failure-hook exception snapshot
Setting __cause__ has a documented CPython side effect of implicitly
forcing __suppress_context__ to True. Since the previous round's
traceback fix set __cause__ before __suppress_context__, a normal
implicit-chaining exception (no `raise ... from`, __suppress_context__
naturally False) got its context wrongly suppressed on the snapshot.
Now __suppress_context__ is set explicitly, after __cause__, so it
always reflects the real exception.
* fix(logging): use MappingProxyType for the failure-hook's sanitized request_data
A LIT002 budget check (surfaced by rebasing onto a moved base) flagged
the dict comprehension building safe_request_data as mutable
construction. MappingProxyType is also a strictly better fit here: a
genuinely read-only view, not just an immutable-looking dict, matching
the intent that callbacks should never be able to mutate what they're
handed.
---------
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
