redis-py awaits a redis_connect_func that is a coroutine function, so
dropping every connect func the async paths cannot convert took away an
auth path that worked.
The AUTH exchange it runs is the blocking client API, so on an async
connection send_command and read_response hand back coroutines nobody
awaits and the connect fails outright.
REDIS_URL-based async clients and every async connection pool dropped the
managed-identity credential the caller configured, so they connected
unauthenticated against an auth-enforcing Redis. The conversion from
redis_connect_func to a CredentialProvider now happens once, before any
branch, and covers the url, sentinel, cluster, and pool paths alike.
Also adds credential_provider to the cluster kwargs allowlist, which
silently filtered it out.
* fix(ui): surface the paginated fallback on Cost Optimization
The page streamed its fallback silently: useDailyActivityRange dropped
the hook's progress and cancel fields and CacheLeakageCard only showed
a loading state while empty. Extract the Usage page's fetch banner into
a shared PaginationStatusAlerts component, render it above the tabs,
and note on the cache leakage tables when pages are still arriving.
* fix(ui): gate the cache leakage streaming note on isFetchingMore only
loading also covers a fresh aggregated request over the previous
range's rows, where pagination copy mislabels stale data. Drop the
redundant component comment flagged against the repo comment policy.
* fix(ptu): refuse an incomplete config.yaml reservation the way the endpoints do
POST /model/new answers 400 when PTU fields are set without a team_id, a
ptu_effective_from, or the count and rate together. config.yaml ran none of
those checks, so the same deployment loaded and served, billing per token
while accruing no flat cost, with nothing logged.
The rule moves into litellm_core_utils.ptu_pricing so both paths state it
once. Registration refuses such a deployment and names it, and the proxy's
ignore_invalid_deployments keeps that to the one entry. Only enforced while
PTU cost attribution is enabled, so a proxy that never opted in is unchanged.
* refactor(ptu): build the refusal message in the module that owns the rule
router.py raised a message it composed itself, which put proxy-facing
wording on the shared SDK surface. ptu_config_error now takes the
deployment name and returns the whole sentence; the endpoints still ask
without a name and their 400 bodies are unchanged.
JSON-RPC 2.0 types `id` as string, integer or null, but
LiteLLMSendMessageResponse annotated it as a bare required `str`. Pydantic v2
dropped v1's int-to-str coercion, so an upstream agent echoing an integer id was
rejected outright, and a null id, which section 5 requires for an error that
cannot be correlated to a request, was rejected too. Both surfaced as -32603 with
a pydantic ValidationError in the message: five distinct 500s on
/a2a/{agent_id}, across message/send and tasks/get.
Everything around the model already handled the full union: the endpoint reads
the id off the body as Any, its helpers are typed `str | int | None`, the error
builder takes `object`, and the streaming path passes the id through untouched.
The response model was the only narrowing left.
Backfilling an id the agent omitted keeps the caller's type too, since JSON-RPC
requires the response id to equal the request id and a caller that sent 7 cannot
correlate a response carrying "7".
`bool` is excluded from the integer half even though it subclasses `int`, so a
boolean id is stringified rather than relayed as 1 or 0, where it would collide
with a real integer id another in-flight request may be using.
* feat(ci): freeze the conftest save/restore inventory so it can only shrink
* fix(ci): resolve the named constant a conftest save loop iterates
* fix(ci): match the snapshot shape instead of a list of blessed dict names
* feat(ci): fail a branch that clears TQ violations without lowering the ceiling
A limit that only ever falls is not the same as one that falls when it can.
Clearing violations and leaving the ceiling above the new count let the same
violations return later under a limit nobody moved, so the gate now fails on
that and names `make lint-budget-update` as the fix. It needs both head below
base and head below limit, so headroom already in the base is never blamed on
the branch that happens to run next.
Drops the seeded-rule exemption from the ratchet along with it. Its stated
reason was that the base tree predates a rule introduced on this branch, but
base counts are measured with the current checker, so such a rule is counted at
the base too and its grandfathered total was never at risk of reading as fixed.
Removing the exemption is what lets a newly seeded rule ratchet like the six
that came before it.
The base scan is skipped when the branch touches neither the test tree nor the
checker, since neither count can have moved.
* feat(ci): assert .github/workflows holds only workflows, correctly named
* style(tests): annotate the hygiene test module's names with Final
* fix(ci): report a .yaml workflow as a naming finding, not a stray
GitHub reads .yml and .yaml alike, so WF001 telling you to move a valid
.yaml workflow to .github/scripts/ was wrong advice. WF001 now covers only
files that are not workflows at all, and the .yml spelling this directory
keeps moves to WF004, which says to rename rather than relocate.
WF001 also never looked into subdirectories, since GitHub does not read
them either; the message now says so. The directory is injected rather
than read off a module constant, so the cases are testable without
monkeypatching.
* test: merge three stranded twins into the files that shadow them
The second mirror's last four files each share a filename with a live test, so
the previous commit could not move them. Three of the four turn out to be plain
additions: their classes collide with nothing in the live file, so the tests are
extra coverage that has sat unrun rather than a competing version of anything.
Appending them takes the three files from 156 collected tests to 196, and all
196 pass. The 40 recovered are 13 OCI cases covering key normalization,
credential validation, complete-URL building and image-url transformation, 15
management-endpoint cases covering empty-value handling and the premium check,
and 12 DeepSeek thinking-parameter cases.
One assertion had to change. test_map_reasoning_effort_none_does_not_enable_thinking
asserted that reasoning_effort='none' leaves no thinking key, while the handler
maps it to {'type': 'disabled'} on purpose, documented in map_openai_params as
the OpenAI-style way to ask for thinking off. The test's stated intent holds,
since disabled does not enable anything, so it now asserts the disabled mapping
instead of the key's absence. Two imports moved to module scope for the
appended code, and no live test was touched.
test_discoverable_endpoints.py is the one left. Its twin grew from 1268 lines
to 9434, 25 of its assertions fail against today's code, and only 5 of its 19
tests have no counterpart, so deciding what survives that rewrite is a
judgement about the endpoints rather than a merge. The allowlist now holds
exactly that file and that reasoning.
* test(oci): stop the OCI suite reading credentials from the environment
validate_environment falls back to os.environ for every OCI credential and only
defaults the region when OCI_REGION is unset, so on a machine with OCI
configured the missing-credential test finds credentials it never passed and the
default-region test builds a URL for the ambient region. The suite then passes
or fails depending on who runs it.
A fixture drops the seven OCI variables for the four classes this branch added
and for TestOCIChatConfig, which had the same dependency before any of this and
fails the same way: with OCI_USER and friends exported, two of its cases fail on
origin/litellm_internal_staging today.
clean env: 83 passed
ambient OCI env: 83 passed
Same numbers either way, where the pre-existing file gave 68 passed / 2 failed
under the second.
Coverage is the single biggest time lever on the unit shards: the legacy
no-coverage workflow ran the same directory in about 5 minutes against 11 to 13
with coverage on. coverage.py's sys.monitoring backend (PEP 669) is the cheapest
core it ships, and it is not in use here today.
It has to be asked for explicitly. coverage 7.14 only defaults to sysmon from
Python 3.14 (`SYSMON_DEFAULT = CPYTHON and PYVERSION >= (3, 14)`) and these
shards pin 3.12, so without `COVERAGE_CORE` they get the slow tracer.
The audit left open whether sysmon survives turning on branch coverage. It does
not, at this Python. coverage gates branch measurement under sysmon on
`branch_right_left`, which needs newer than 3.14.0a5; on 3.12 it refuses and
falls back to the default core with a `no-sysmon` warning. Verified directly
against Python 3.12.13 with coverage 7.14.0:
$ COVERAGE_CORE=sysmon python -m coverage run --branch --source=. run.py
CoverageWarning: Can't use core=sysmon: sys.monitoring can't measure
branches in this version, using default core (no-sysmon)
So this speedup and `branch = true` are mutually exclusive until the runners
move to 3.14. Nothing here turns branch coverage on, so the two never collide
in this change, but whoever does turn it on is choosing to give this back.
`assert False` inside a `try:` raises AssertionError, which the `except
Exception` right below it catches, so several tests reported green no matter
what the code did. `pytest.fail` raises Failed, a BaseException, and escapes.
A bare `a == b` statement is evaluated and discarded. Nine of those sat in
tests, and one was comparing against a model name the router never produces.
Selects B011, B015, B018, PT015, PLR0133 and PLW0127 in ruff-tests.toml
alongside F821, with all 50 existing violations fixed, so no budget file or
ratchet is needed. CI already runs this config over tests/.
MCP egress prefixed the configured scheme unconditionally, but callers legitimately supply
both a bare token (from a stored credential) and an already-schemed value (passed through
from the caller's x-mcp-auth or Authorization header). The second shape produced
Authorization: Bearer Bearer <jwt>, which upstream servers reject as a malformed token. It
presented intermittently because a resolved stored credential arrives via extra_headers and
overwrites the doubled header, so only users without one always failed.
strip_auth_scheme drops one leading scheme before the header is rebuilt. It matches the
scheme case-insensitively per RFC 7235 and requires a credential behind it, so both a token
that merely begins with the scheme text and a scheme with nothing behind it are left intact.
MCPAuth.authorization stays verbatim because that auth type means the caller owns the whole
header value.
For MCPAuth.basic the normalization has to happen in update_auth_value rather than at
header-build time: to_basic_auth has already encoded the whole "Basic <credentials>" string
by then, so no prefix is left to find. A schemed value whose remainder decodes is already
encoded and is reused; one that does not decode is the bare pair with the scheme written in
front of it, and is encoded rather than forwarded as an invalid header.
The same doubling reached OpenAPI-backed servers through _format_byok_openapi_auth_header. A
non-BYOK server short-circuits _resolve_byok_mcp_auth_header, so that formatter also receives
the deprecated global x-mcp-auth, which is already a complete header value.
store_user_oauth_credential refused to overwrite any existing row that did not
decode as an OAuth2 payload, which conflated two states: a live BYOK secret that
reads back as plaintext, and ciphertext written under a LITELLM_SALT_KEY the proxy
no longer holds. The second is unrecoverable by any caller, so refusing preserved
nothing and instead wedged the user out of the OAuth flow permanently, since
re-authorizing is their only recovery.
The guard now raises only when the existing value is genuinely readable. An
undecryptable row is logged and replaced by the newly authorized token.
Both read paths were equally silent: get_user_oauth_credential and
list_user_oauth_credentials (which backs the bulk prefetch) each dropped an
undecryptable row indistinguishably from "user never authorized", so an operator
saw an upstream 401 and no hint that a credential had failed to decrypt. Both now
warn with the user and server ids, never the stored value.
ruff.toml excludes tests/* from `ruff check`, so nothing has ever checked the
test tree for names that do not exist. That matters more in tests than in
product code: a NameError inside a test whose body is wrapped in
`except Exception: pass` is swallowed, and the test reports green forever.
Adds ruff-tests.toml selecting F821 alone, wired into the lint workflow and
`make lint-ruff`, and clears every existing violation:
- 4 tests interpolated an unbound `e` into a `pytest.fail` message reached only
on the failure path, so the NameError, not the assertion, is what ran.
test_llm_guard_error_raising is the worst: it passes today with content
safety disabled entirely. It now asserts the 400 and its detail body.
- 5 sites construct BaseExceptionGroup, a 3.11 builtin, in a tree that still
supports 3.10. Guarded behind the exceptiongroup backport that anyio already
pulls in below 3.11.
- 9 missing imports (json, openai, Any, Final, HTTPException), including one in
a helper that catches HTTPException by a name it never imported, so the
challenge path it exists to detect raises NameError instead.
- 5 annotations naming types imported inside the function body, hoisted to
module scope or TYPE_CHECKING.
- 2 blocks of dead code: everything after a pytest.fail in
test_claude_agent_sdk, and an unused helper in test_end_users calling a
function defined in a different module.
- 1 error-path f-string in the router-settings doc test that masked the real
FileNotFoundError behind a NameError.
Only F821 for now. Widening the select list means ratcheting thousands of
pre-existing findings, so rules go in one at a time with their violations
already fixed.
The Experimental badge sat in the top bar next to the icon, which read as if the
whole theme control were experimental and cost toolbar width for a caveat that
only applies once. It moves into the menu as a Beta tag on the Dark entry, where
it labels exactly the choice it is about and is visible before the choice is made
rather than only after.
* feat(proxy): redact or drop individual batch records instead of rejecting the file
A single record tripping a guardrail rejected the whole upload, which is unusable for a file
holding thousands of rows. A record a guardrail rewrites is now submitted in its rewritten
form, a record it blocks is left out, and the create response reports every changed record by
both custom_id and line so a caller can reconcile against the file it sent. The same outcome
is written to the proxy log and to request metadata, so it is not visible only to the caller.
A rewritten record goes straight to a spool and only its offset is carried, so a masking
guardrail touching most rows of a large upload does not build a second copy of the file on the
heap, and the rewrite runs off the event loop the way the sibling full-file validation does.
Both proxy-injected metadata keys are captured from the record and restored exactly, including
an explicit null, so a masked row keeps the tags that decide how it is attributed.
A record is dropped only when a guardrail judged its content. `GuardrailRaisedException` now
carries `blocked_content` for that, because half its raise sites in the repo signal an
unreachable or unparseable backend under a fail-closed policy, and treating those as blocks
would turn "refuse this request" into "drop this record and submit the rest". The default is
off, so a raise that does not say what it means aborts the upload instead of silently
shrinking the file.
* fix(proxy): only drop a batch record on a verdict the guardrail actually reached
A guardrail that reports a technical failure as an HTTPException carrying a block status was
read as a content block, so an unreachable backend under a fail-closed policy quietly shrank the
file instead of failing the upload. Two in-tree integrations do exactly that, and one of them
defaults to fail-closed, so the broken configuration was the default one. Such an exception is
raised `from` the underlying error, which is a deliberate statement that something else caused
it, and no content verdict in the repo is raised that way, so the chain now settles it. Implicit
context is left alone, since a block raised inside an unrelated `except` would read as a failure.
Two annotation errors in the same family: the one GuardrailRaisedException subclass in tree never
opted into blocked_content, so a real block took the whole upload down with it, and straiker's
block helper is reached both from its verdict and from its fail-closed handler, so it claimed a
verdict for an outage. The helper now takes the flag from its caller.
A record could also opt itself out of the chain. Guardrail selection reads a body-level
`guardrails` key ahead of the proxy-injected list, and online that key can only add to the key
and team selection, never replace it, so a batch record naming an empty list skipped every
guardrail that was not default_on and was still reported as scanned. Every injected key is now
stripped before dispatch and restored afterwards.
A guardrail that reroutes a record to another model is honoured on the online path by rewriting
the model, which the scan read as a rewrite and submitted in the same file, sending content to
the provider the reroute existed to avoid. Every record of a batch file goes to one provider, so
the upload is refused instead, naming the line.
The scan spool is closed on the paths that never read it back.
* fix(proxy): give the scan the metadata bag guardrails actually read, and close its spools
The narrowed request metadata was installed under `litellm_metadata` only, but a record is
scanned as the chat request it describes, and the guardrails that pick a policy from a request
header read `metadata` instead. Noma choosing an application and Aim choosing a user both look
there, so the header allowlist added for them did not reach either one and a batch record was
still evaluated under the fallback policy. The scan metadata now goes into both bags, which are
both stripped and restored, so neither survives into the record that ships.
The scan spool was closed on the paths that abort, which are exactly the paths where it is
empty, and left open on the one path where it holds the rewritten records. Nothing closed the
rewrite output either, where before this feature the uploaded handle belonged to Starlette. The
upload now owns both and closes them however it exits.
* fix(proxy): register the scan spool before the rewrite can fail
The scan spool was added to the request's cleanup list only after the rewrite returned, so a
rewrite that raised, which for a spilled file can be as ordinary as the disk filling up, jumped
to the handler with the list still empty and left the scan's own handle open. The rewrite also
left its half-written output behind on that path, since nothing owns that handle until it is
returned. Both now close.
completion() strips the litellm routing prefix before it dispatches to the
responses bridge, but responses() runs get_llm_provider() again, so a model id
that itself starts with the provider name lost a second prefix and reached the
provider as a name it does not know. Handing responses() the prefixed model
back makes its own resolve a no-op: across the 3061 cost map entries, 76 reach
the responses bridge and only the four perplexity Agent API models change.
* feat(ui): add a light/dark/system theme toggle
The dashboard already carried a full `.dark` palette, dark-aware surfaces and a
dark logo variant, but nothing ever put the `dark` class on the document, so
none of it could be reached. next-themes now owns that class: it reads the
stored choice, falls back to the OS preference, and stamps the class from an
inline script before first paint so there is no light flash on load.
The toggle is a three-way System / Light / Dark control in the account menu,
in both the sidebar menu and the older navbar dropdown, so it is reachable from
the gateway dashboard, chat and the model hub alike.
useIsDarkMode watched the root element with a MutationObserver purely to answer
a question next-themes now answers directly, so it goes, and useSyntaxTheme
reads resolvedTheme instead. The toaster follows the resolved theme too.
* feat(ui): move the theme control to the top bar and default to light
The toggle now lives in the header toolbar of both shells, the gateway
dashboard's DashboardHeader and the older full-width Navbar, where it replaces
the placeholder comment that had been holding its spot. It reads better there
as a single icon button with a System / Light / Dark menu than as a segmented
row buried in the account popover, so the account menus lose their theme row.
Dark mode is still being rolled out, so an install that has never touched the
control now stays light instead of following the OS. System is still a choice,
just no longer the default. While dark is active the toolbar carries a small
Experimental badge, so nobody mistakes an unstyled surface for a bug.
* fix(ui): serve the dark logo in the legacy navbar too
The sidebar already paired its logo with a dark variant, but the full-width
navbar kept a single light-only image. That did not matter while dark mode was
unreachable; now that the toggle sits in that shell's own top bar, the white
JPEG slab lands on a dark bar. It gets the same two-image swap the sidebar uses,
and a test that pins the pairing so the two shells cannot drift apart again.
* fix(proxy): run pre-call guardrails on batch input file uploads
POST /v1/files with purpose=batch was the only route in files_endpoints that
never reached pre_call_hook, so guardrails did not see batch content at all and
records reached the provider unscanned.
Stream the uploaded JSONL a record at a time and run each record's body through
the existing pre_call_hook dispatch under the call type its url maps to, so
guardrail resolution, key and team config, and the per-endpoint translations are
reused rather than reimplemented. The hook gains a guardrails_only mode for this,
since the same callback loop also drives rate limiters, budget hooks, prompt
templates and hanging-request alerting, none of which should fire once per record.
A guardrail that blocks raises its own exception, which propagates untouched so
its status code survives. A record a guardrail would rewrite, a record that
cannot be parsed, and a record whose url cannot be scanned all reject the upload,
since silently skipping any of them is the bypass this is meant to close.
Per-record redaction lands separately.
The scan only runs when a guardrail that actually runs pre_call, or a guardrail
pipeline, is configured, so deployments without one are byte for byte unchanged.
* fix(proxy): compare the dict a batch guardrail returns, not the one it was given
async_pre_call_hook may return a replacement dict instead of mutating its input, and
process_pre_call_hook_response then makes that replacement the request. The scan only
inspected the dict it passed in, so a guardrail that redacts by returning a copy was
treated as a no-op and its record uploaded unchanged.
* fix(proxy): treat a missing batch body key as different from a null one
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(proxy): document the guardrails_only mode on pre_call_hook
* fix(proxy): resolve a batch record's scan type from its body when the url is unfamiliar
The scanner only accepted five exact urls, but callers write that field by hand and the
provider transformers are far more permissive: bedrock treats any non-empty url as chat
and vertex strips query strings and trailing slashes. Uploads that work today would have
started failing the moment a pre-call guardrail was configured.
Normalize the url before lookup and fall back to the body shape when it is unfamiliar, so
a record we can still read is a record we still scan. Only a body with no messages, prompt
or input is now refused, and the error says so instead of listing urls that were never the
whole set.
Also pins the default side of the guardrails_only gate: the hanging-request alert and
prompt templating are asserted to still fire when the flag is absent.
* refactor(proxy): drop batch guardrail checks the upload validation already makes
check_batch_file_upload now runs first and rejects a line that does not parse, a line that
is not an object, and a line missing custom_id, method, url or body, so the guardrail scan
can rely on all four. Its own parse handling was unreachable through the endpoint and is
gone, along with the tests for it. What is left is the case that validation does not cover,
a body whose value is not an object, since it only checks that the key is present.
* fix(proxy): resolve a batch record's call type from the url path, not the whole url
A record naming its route in full, which is how callers actually write batch files, matched
no known route, so it fell through to the body shape. A Responses record carries `input`,
and that reads as an embedding, so the record was scanned as the wrong call type and any
guardrail scoped to chat or Responses skipped it while the upload was accepted. Chat records
survived only because their body shape happens to map back to the same call type. The url is
now reduced to its path before matching.
Guardrails that pick their policy from a request header, such as noma choosing an application
id, saw no headers at all during the scan and fell back to a default, so a batch record could
be evaluated under a different policy than the same content sent online. The sanitized headers
the proxy already stores in request metadata now travel with the scan.
Also drops the bare `dict` annotation, the unreachable non-dict branch on the guardrail chain's
own return, and the type alias that was missing its `TypeAlias`, which together were failing
the lint gate.
* fix(proxy): give each batch record its own copy of the scan metadata
The narrowed metadata was handed to every record as a shallow copy, so `headers` and `tags`
stayed shared with the upload request and with the other records in the same window. A guardrail
that writes into one of those in place, which several do to record their own bookkeeping, would
have its write show up in every record scanned after it and in the request itself. The narrowing
already removed the values that cannot be copied, so each record now gets a deep copy.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): let admins supply a dark-mode variant of their custom logo
A deployment branded through UI_LOGO_PATH got its light artwork on the
dark sidebar, and there was nothing an admin could set to change that.
Adds UI_LOGO_PATH_DARK, exposed as the logo_url_dark theme setting and a
second field on the UI theme page. /get_image now walks an ordered list
of candidates for the requested theme and serves the first usable one:
the dark logo, then the light logo, then the bundled default.
Falling through rather than failing is the point. An admin who never
sets a dark logo keeps their own light one instead of reverting to
LiteLLM's, and a dark logo that goes missing later degrades to their
light logo rather than dropping their branding entirely.
* fix(ui): recover from a dark logo the browser cannot load
A dark logo given as an http(s) URL is loaded by the browser straight
from the sidebar, so it never passes through the proxy's fallback chain.
A URL that 404s left a broken image where the admin's light logo should
have been, while the same logo given as a local path fell back cleanly.
The sidebar now remembers the dark URL that failed and drops to the light
logo, matching how the proxy resolves an unusable dark logo and how the
provider Logo component already handles a broken image.
Perplexity's Agent API model ids already start with perplexity/, so a litellm
model like perplexity/perplexity/glm-5.2 normalizes to model perplexity/glm-5.2
with custom_llm_provider perplexity. Every existing candidate in the cost map
lookup ladder reads that leading perplexity/ as the litellm prefix and strips
it, so the four new rows never matched and supports_reasoning, get_model_info,
and completion_cost all failed on them.
Add the doubled form as a last-resort candidate, tried after every candidate
tried today and still ahead of the capability generalization rules, so only
lookups that already raised can newly succeed. Sweeping all 3062 cost map keys
across the three _get_potential_model_names branches (8433 probes) shows 7
lookups change, all of them errors that now resolve: the 4 new perplexity rows
plus openrouter/openrouter/auto, /free, and /bodybuilder. Nothing regresses.
The new perplexity/perplexity/glm-5.2 row carried 2.6e-07, which is glm-5.3's
cache read rate. api.perplexity.ai/v1/models publishes 0.14 usd per 1M cached
input tokens for glm-5.2, so the rate is 1.4e-07.
* feat(proxy): add POST /auto_router/validate_config to dry-run the complexity-router write gate
* refactor(proxy): scope the validation endpoint name to the complexity router
* fix(proxy): give the complexity-router validate route the same audience as /model/new
* feat(proxy): gate auto-router dry runs like the write they rehearse
* chore(proxy): dedupe the validate route's self_managed_routes entry
* test(proxy): fold the dry-run route reachability check into the model-new audience parity test
* fix(proxy): scope test_routing's configured check to models the caller can use
* chore(ui): regenerate schema.d.ts for the scoped configured-check description
* test: replace blind sleeps with deadline waits in callback and caching tests
tests/local_testing/test_custom_callback_input.py slept a fixed 1-3s after
every call and then asserted the callback handler recorded no errors. Because
the handler only appends to `states` when a callback actually fires, an assert
of `len(errors) == 0` passes just as happily when nothing fired at all, so the
sleep was buying flakiness in exchange for a vacuous check. The async tests
were worse: `time.sleep` blocks the event loop, so the success/failure tasks
scheduled on it could not run before the assertion.
Adds tests/_wait_helpers.py with `wait_until` / `await_until`, which poll a
predicate against a deadline, and converts all 17 sites to wait on the thing
the test actually cares about (the terminal state landing in `states`, or the
patched log hook being called). The waits assert the callback fired, so these
tests now fail on a dropped callback instead of passing silently.
The three sleeps in test_caching_handler.py sat between `sync_set_cache` and
`_sync_get_cache`, both fully synchronous against a local in-memory cache, so
they are just deleted.
* fix(test): wait on the priming call's own logging in the cache-hit test
The 3s sleep in test_logging_async_cache_hit_sync_call was not waiting for the
cache write, which lands before the stream iterator is exhausted. It was
waiting for the priming call's success callback to drain, so the handler
installed right after it only ever sees the second, cache-hit call. Waiting on
a populated cache_dict let the priming call's still-pending log_success_event
reach the new mock, and the test then read cache_hit off the wrong payload.
Waits on the priming handler's own sync_success state instead.
`lite login --pkce` mints a refresh token that buys a fresh key from the
proxy on demand, so it is the credential just as much as the key is. Moving
the key into the keychain left it behind in ~/.litellm/token.json, where any
process running as the user can read it and renew the login for itself.
It now travels with the key: `save_cli_token` writes both into the keychain
entry, the token file keeps only metadata, and `lite logout` takes it out of
the file whether or not the keychain answers.
Upgrading finds one sign-in split across the two stores, the key already in
the keychain and the refresh token still on disk. That case rejoins the two
halves into a single entry before scrubbing the file, so the write never
replaces a live key with nothing, and a machine that refuses the scrub keeps
what it has rather than having the key rolled back out from under it.
* feat(otel): route Phoenix traces to per-key/team projects under otel v2
The v2 arize_phoenix preset read PHOENIX_PROJECT_NAME once at startup into a
static resource attribute, silently dropping the per-key/team project routing
v1 supported. Route it via Phoenix's x-project-name OTLP/HTTP header instead:
the env var stays the global default, and a phoenix_project_name (or
phoenix_project_name_override) in key/team metadata sends that key's traces
to the named project.
The project comes only from user_api_key_auth_metadata (server-set at auth),
never from client request metadata or StandardCallbackDynamicParams, since
choosing the telemetry destination is a data-exfiltration primitive. The
header is appended to the exporter's static headers rather than replacing
them, so the preset's Authorization survives, and it is gated to OTLP/HTTP
exporters because Phoenix only reads it on /v1/traces.
Also unban the bare phoenix_project_name fields from the request-body gate:
the proxy integrations ignore them (only user_api_key_auth_metadata routes,
and that stays banned), so rejecting them just broke SDK-style callers.
* fix(otel): root project-routed Phoenix spans in their own trace
Phoenix assigns a whole trace to one project by whichever span arrives
first. The request's auth/db/root spans always export through the default
provider without the project header, so a project-routed LLM span parented
into that trace got dragged back into the default project and the header
did nothing (verified against a live Phoenix instance). Detach the routed
span into its own trace with a link back to the request trace, mirroring
how the v1 Phoenix logger exported each request under its own local parent.
* fix(otel): drain in-flight spans before shutting down evicted providers
LRU eviction shut a routed provider down immediately, but an LLM span
opened at pre_call stays open until the later success or failure callback;
with more than 256 overlapping credential/project routes that in-flight
span was silently dropped instead of exported. Refcount open spans per
provider (hold at span open, release when the carrier is removed on close,
carrier-map eviction, or MCP stray-carrier cleanup) and defer a retired
provider's shutdown until its last open span closes.
* fix(otel): take the provider hold inside route_for to close the eviction race
pre_call can run on thread-pool workers, so between route_for returning a
provider and the caller recording its open span, a concurrent request could
overflow the LRU and shut that provider down with a zero span count, dropping
the routed trace. route_for now increments the open-span count in the same
locked critical section as the cache update and hands back an already-held
provider; every caller releases it once its span has landed. The lock also
makes the cache mutations safe under that same thread-pool concurrency.
* fix(otel): skip tenant routing on deferred pre_call
route_for ran before the recordable-parent check, so a thread-pool
pre_call still built or LRU-touched a tenant provider and could evict
an idle one even though the hold was released immediately and close
re-routed. Only route when the span actually opens
* add somethign
* Revert "add somethign"
This reverts commit 2f2cf84c5a.
* fix(otel): cap retired tenant providers draining open spans
* docs(otel): justify the retired-provider cap
The bundled logo is a JPEG, so it carries no alpha and its white
background renders as a bright slab against a dark sidebar. Making it
transparent alone would not be enough either: the wordmark is near-black
and would disappear on dark.
Adds logo_dark.png, derived from the light logo. The sky-blue disc and
train are kept as they are behind a circular alpha mask, and the
wordmark's antialiasing is un-flattened from white into straight alpha
and repainted in the dark theme's own foreground colour. Both files are
1000x257, so swapping between them cannot shift the sidebar header.
/get_image gains a theme query param. The default response is byte for
byte what it was, and a logo configured through UI_LOGO_PATH is served
unchanged in both themes, since custom logos have no dark variant yet.
* feat(complexity_router): add business classification rubric preset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate api schema for business rubric
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): suppress preexisting antd import violations in touched files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): draw one Per Day savings bar per date on Cost Optimization
The page paged /user/daily/activity over raw rows, so a date spanning
pages arrived N times with partial metrics and rendered as N thin bars.
Switch to the single-shot aggregated endpoint, thread
include_current_utc_day through it to keep the live-end extension from
PR #36051, and merge the paginated fallback by date.
* fix(ui): keep aggregated call at four params and mock it in view tests
Trailing userId and includeCurrentUtcDay ride a named rest tuple so the
eslint max-params baseline stays at 23, and the CostOptimizationView
suites mock the new networking export their render now reaches.
* fix(ui): make hardcoded palette surfaces theme-aware
Twenty-one dashboard files painted fills from the raw Tailwind palette with no
dark counterpart, so in dark mode they rendered as near-white islands carrying
dark text: unreadable. The route sweep caught them on teams, access-groups,
policies, users, skills, guardrails-monitor, logs, compliance, playground,
fallbacks and the AI hub.
Where the hue already had a semantic token, the surface moves onto it. Every one
of these lines had a token on its border and a palette class on its fill, so
this finishes a migration that had stalled halfway: bg-blue-50 next to
border-info/20 becomes bg-info, bg-gray-50 becomes bg-muted, DocLink's bg-white
becomes bg-card, and the Alert error variant drops text-red-800 and text-red-600
for the destructive token its sibling variants already use.
Purple, violet and indigo have no token in the system, which is exactly why the
maps in PluginTableColumns, GuardrailsOverview, AccessGroupsTableColumns and
teamTableColumns had migrated every other entry and left those behind. Rather
than mint a brand token here, they take the dark palette step, matching what
TeamGuardrailsTab, add_agent_form, MCPToolsetsTab and mcp_connect already do.
Gradient stops get the same treatment since bg-linear stops have no token form.
Light mode is unchanged apart from the four surfaces that moved onto a token,
and those stay inside the same colour family.
The #1e1e1e code slabs in guardrail_info and CustomCodeModal are deliberately
left alone: they are intentionally dark editors in both themes, and their
gray-200 text stays legible either way.
* fix(ui): give dark surfaces a readable foreground step
The dark fills added for the purple and indigo surfaces left three nested
foregrounds on their original light-palette step, so the text and icon sitting
on those new fills dropped below readable contrast in dark mode.
text-purple-800 on purple-950 measured 1.72:1, text-indigo-600 on indigo-950
2.54:1, and text-purple-600 on the blue-950 gradient stop 2.73:1. Each now
takes the purple-300 / indigo-300 step this PR already uses elsewhere, which
lands them at 8.48:1, 8.02:1 and 8.31:1.
The pricing calculator renders the same cost expression twice, so both copies
move together rather than leaving one half-migrated.
* fix(ui): keep the guardrail chip remove button visible on hover
The chip itself moved to the dark indigo fill, but its remove button still
darkened to indigo-900 on hover, which against indigo-950 measures 1.40:1 and
makes the X vanish under the cursor in dark mode.
Dark mode now brightens to indigo-100 on hover instead, mirroring the light
theme where hover darkens away from the resting colour.
* test: run the 30 test files stranded in the second mirror
tests/litellm sat beside tests/test_litellm, which is the mirror the repo
convention names, and no job collected it. The allowlist called the directory
unresolved and assumed it was a duplicate. It is not: 30 of its 34 files have no
counterpart in the real mirror, so they are tests nobody has run since they were
written, not copies of tests that run elsewhere.
Moving them in is byte-identical, and it is what makes them run. Every one is
now claimed by a shard's test-path rather than by an allowlist entry, and the
216 tests they hold pass. Directories that needed to become packages did, since
several files are named test_transformation.py and pytest cannot import two of
those from non-package directories in one session.
Never running is why three assertions had drifted away from the code:
* nvidia.nemotron-super-3-120b max_output_tokens, 32000 -> 32768
* sambanova/MiniMax-M2.7 max_input_tokens, 204800 -> 196608
* the Vertex text-to-speech handler moved from data= to json=, so the test
reads the decoded body off the json kwarg instead of parsing the data one
The first two follow model_prices_and_context_window.json, which the catalog
sync keeps current; the third follows the handler. In all three the test was the
stale side.
The lint workflow ran test_no_hardcoded_secrets.py by path and now points at the
new one.
Four files stay behind. Each shares a filename with a live test whose contents
are disjoint from it, so landing those means merging test bodies, which is a
content review rather than a move. The allowlist entry now names those four and
records how many tests each would bring, in place of calling the whole
directory unresolved.
* fix(ci): keep the secret scan out of the mirror's conftest
The secret-scan job runs pytest under uv run --no-project, so its environment
holds pytest and nothing else. That worked while the file sat in tests/litellm,
which has no conftest, and broke the moment it moved into tests/test_litellm,
whose conftest imports litellm on collection: ModuleNotFoundError: No module
named 'dotenv', before a single test ran.
The file is a repo-wide static scan that imports only base64, os, re and pytest,
so it belongs with the other repo-wide checks in tests/code_coverage_tests,
which has no conftest, rather than in the package mirror. Installing the full
dependency set into a 15-second job to satisfy a conftest it does not use would
be the wrong trade.
Verified with the job's exact command:
uv run --no-project --with 'pytest==9.0.2' pytest \
tests/code_coverage_tests/test_no_hardcoded_secrets.py -q
1 passed in 0.47s
* feat(ci): catch files a -k expression deselects from every job
The coverage census asks whether some job names a file. It cannot ask what that
job's -k then does with it, and the gap is not hypothetical: tests/local_testing
is globbed by five jobs, two of which carry
-k "... and not router and not assistants and not langfuse and not caching and not cache"
while the other three keep one keyword each. Any file whose path holds an
excluded term is dropped by the first two and matched by none of the rest, so it
runs nowhere while the census counts it as covered. 118 tests across eight
caching files sit in exactly that hole today.
The new mode reads the same CircleCI jobs the census already parses and asks
whether each globbed file survives its job's selector. Two facts about -k make
that decidable without running pytest: it matches an item's own name and its
parents', so a term appearing in the module path deselects the whole file; and
the names it can match are otherwise the classes and functions in the file,
which ast reads. A positive term is therefore satisfied by the path or by a name
inside, which is what keeps a langfuse-named test inside test_logging.py from
being reported.
Where the parser is unsure it stays quiet. An expression with or, parentheses,
or a negated group is left unmodelled and its job is treated as claiming
everything it globs, so an unparsed selector can never raise a false alarm.
Glob translation learned character classes, without which
tests/local_testing/**/test_[a-mA-M]*.py matches nothing and the guard would
report that whole directory. The census and shard counts are unchanged by it,
2423 files and 327 shard children before and after.
Validated against the real thing: collecting tests/local_testing under each
job's own selector leaves 175 of 1577 tests unselected, in exactly the ten files
this check derives statically, no more and no fewer. Two of the ten are named
outright by other jobs, which the check credits, leaving the eight now recorded
in the allowlist as a decision rather than an accident.
Verified red-first: dropping one of those eight from the allowlist reports it,
and adding 'and not embedding' to the two part jobs reports test_embedding.py
and test_get_optional_params_embeddings.py.
* fix(ci): keep the slice guard from pairing one command's -k with another's glob
Two accuracy notes from review, both about the parser's model rather than its
current verdicts.
A job that runs several pytest commands offers no way to tell which glob a -k
belongs to, since both are read out of the same flattened job text. Combining
them could pair one command's exclusion with another command's glob and report a
file that in fact runs. Such a job is now left unmodelled, which means it claims
everything it globs, matching how the parser already treats an expression it
cannot read. Only one job in the config has two globs today and it carries no
-k at all, so no verdict changes.
The second is a deliberate limit, now stated where it lives: an excluded term is
only honoured when it sits in the module path, because that is the case that
takes the whole file with it. A term matching one function inside drops that
test and leaves the file running, and reporting it would be a false alarm.
Answering per-test instead would need a baseline of test ids that churns on
every rename, for a smaller failure than a file going dark.
Both are pinned by tests.
* feat(ci): ratchet tests that skip themselves when a credential is absent
* docs(ci): name the new rule where the gate's rules are listed
* fix(ci): require the condition to test for absence before TQ006 fires
* test: settle three allowlist entries that were open questions
The allowlist is meant to hold decisions, not deferrals, so an entry reading
'needs moving' or 'referenced by no job' is a gap wearing an exemption. These
three each get an answer.
The two prompt-factory tests move into the mirror, which is what their own entry
said they needed. Both were passing the whole time, so the 23 tests they hold
start running and the entry goes away rather than getting reworded.
test_aio_http_image_conversion.py is not a test. It fetches live image URLs,
times aiohttp against httpx, prints the ratio, and asserts nothing, and pytest
cannot collect it because its functions take arguments rather than fixtures.
Running it beside its siblings would buy CI a network dependency and a number
nothing reads, so it stays exempt with that written down.
test_litellm_proxy_extras_utils.py stays exempt with a measured reason. 24 of
its 28 tests pass; the 4 in TestMigrationSQLIdempotency fail because nine
migrations from 2026-04 onward use bare CREATE TABLE, ADD COLUMN and CREATE
INDEX where that file requires guarded forms. The convention eroded quietly
precisely because the test enforcing it has never run. Wiring it up is blocked
on what to do about those migrations, and editing them is not the answer, since
Prisma checksums an applied migration and a changed one breaks migrate deploy
for existing installs.
Allowlist entries 10 -> 9, paths 88 -> 86.
* docs(ci): correct the migration count in the proxy-extras allowlist reason
