Retypes the 54 highest-density reportAny/reportExplicitAny sources with real
types instead of shuffling the ceilings around: typed prisma table Protocols so
the untyped client surface stops at the query, local TypedDicts for JSON and
dict payloads, concrete chunk and logging types on the streaming and callback
surfaces, and 3-argument getattr with a Callable annotation where an SDK object
is genuinely duck-typed
No cast(), no type: ignore, no noqa, no suppression comments, and no new Any
annotations. Whole-tree basedpyright drops 1,941 errors with no rule rising
anywhere, and all three budget files are ratcheted so the cleared headroom
cannot silently grow back
Adds a GDC regression test pinning the named AttributeError that the typed
credential accessor now raises when with_gdch_audience is missing
The UI unit test job narrows a pull request to `vitest related <changed
files>`. `related` maps a file to the tests that import it, so a file no
test imports maps to nothing, and `--passWithNoTests` turns that empty
selection into a green job. package.json, package-lock.json, the Vitest,
Tailwind and TypeScript configs and tests/setupTests.ts are all in that
category even though each of them can change the behaviour of every test
in the suite, so a dashboard dependency bump merged having run no unit
tests at all and only got real coverage later, from the full run on the
push to litellm_internal_staging.
Keep `related` for the common case where a pull request only touches
files under src/, and fall back to the full suite as soon as one changed
file sits outside it. The decision lives in
.github/scripts/select_ui_test_scope.sh so it can be tested on its own,
next to the existing classify_changes.sh gate.
* fix(ptu): hand the prune a plain delete filter the query builder can serialise
The bounded sweep built its predicate as a read-only mapping view, which the query
builder refuses to serialise, so the nightly job raised as soon as a config-declared
deployment was priced. The charges were already written by then, which is why the run
looked like it had produced its rows.
The in-memory table these tests run against accepts any mapping, so only a live run
caught it. A predicate builder now returns a plain dict and is asserted as one, and the
catch-up pass has a test covering a config-declared reservation.
* refactor(ptu): build the prune predicate in one shot
Both filter shapes are known upfront, so the bounded one is constructed
directly rather than by mutating a value already declared Final.
The catch-up test took two independent clock reads, which disagree across
UTC midnight; it now derives both the reservation start and the expected
last charged day from a single read, matching the three sibling tests.
* feat(ptu): accrue flat cost for PTU deployments declared in config.yaml
The flat-cost rollup reads deployments from LiteLLM_ProxyModelTable, and config.yaml
models never reach that table by design, so a PTU deployment declared there accrued no
flat cost at all while still billing its traffic per token. The provider bills the
reservation whichever file declared it.
The rollup now also reads the deployments the router holds that no database row owns,
identified by db_model, skipping the per-request credential clones that carry
original_model_id and reuse their source's PTU config under a fresh id. Registering such
a deployment zeroes its pricing, since reserved capacity already pays for the traffic it
serves, and leaving a rate unset falls back to the public cost map, which makes the double
charge the default rather than an opt-in.
The rules both halves apply now live in one module. The rollup's test for what it will
charge and the router's test for what to zero have to agree, or a deployment one accepts
and the other declines serves its traffic for free. That module also owns the fields the
write endpoints already zero, so the two paths cannot drift: tiered_pricing is emptied
rather than zeroed because its tiers outrank the rates beside them, the search context
table is written zeroed because an absent one means the provider default, and any further
rate the deployment itself declares is zeroed alongside the standing set.
The prune is bounded to the deployments a run scanned, but only for a run that priced a
config-declared deployment. Deciding a row is garbage on staleness alone stays correct
while every run derives its charges from the same table, so a database-only run sweeps
exactly as it did before; once one host's charges come from a file the others cannot read,
a row it never considered is not evidence of anything.
Behaviour change worth calling out: a zeroed deployment sorts ahead of an unpriced sibling
in QualityRouter's cost tiebreak, where an unset rate previously sorted last. Reserved
capacity really is the cheaper choice, but the ordering moves.
* refactor(ptu): drop a Final rebind and two redundant isinstance guards
The basedpyright budget rejected reassigning a Final in the datetime coercion and
two isinstance calls the router entry's own type already guarantees. Filtering the
built records rather than the raw entries removes both guards and leaves
_router_deployment as the single validator.
PR #37550 taught the backend unit-test shards to read the pull request's own
file list, but four required jobs were never wired to that gate and ran in full
on every pull request regardless of what it touched. A UI-only pull request
still paid roughly 17 runner-minutes of Python work it could not have affected,
and a backend-only one still installed and built the dashboard.
Lint and the MCP suite now take the existing backend decision. The dashboard
build and unit tests take a new ui decision, which tracks ui/ rather than
reusing client: client deliberately runs whenever the backend changes, because
it gates CircleCI's end-to-end jobs that drive a real proxy, while the build and
the unit tests cannot see the backend at all. CI config counts as ui-relevant
too, so a pull request that rewrites the dashboard workflows still exercises
them instead of shipping unvalidated.
The gate stays inside the job rather than moving to on.paths or to a job-level
condition on the shard callers. A workflow filtered out by on.paths never starts
and never reports, so a required check waits forever, and a skipped caller job
publishes its own name instead of the nested "<shard> / Run tests" the ruleset
requires. Both were measured before settling on this shape.
Three setup steps in the shard base and in the documentation job also leaked
past the gate, so a skipped shard still spent about twelve seconds installing uv
and restoring its cache. They now carry the same condition, and the documentation
job stops cloning litellm-docs when it has nothing to validate.
The field replaces the standard RPM/TPM checks for batch submissions, so a
key holder or team admin writing it could pick their own batch quota.
Mirrors the output-token-estimate admin gate: change-based, so resending
the stored value stays allowed, and enforced on key generate, update, bulk
team-key update, regenerate, and team new/update.
`ProxyConfig.get_config()` walked the parsed config and replaced every
`os.environ/<KEY>` string with `get_secret(value)` before anything initialized
the secret manager, so a key held only by the manager resolved to `None` and
that `None` was written back into the config. The later fallback in
`load_config` could not recover it, because the key now existed with a `None`
value.
Hoist the initialization into `get_config()`, ahead of the resolution pass, so
every entrypoint gets it: the CLI already did this itself, but the microservice
entrypoints (`gateway/main.py`, `backend/main.py`) uvicorn the app directly and
bypass the CLI. `load_config`'s own call is now redundant and is dropped, so
startup builds the manager once instead of building one and discarding it.
`get_config()` also runs on management-endpoint request paths, so this returns
early once a manager exists rather than rebuilding the client per request.
Also warn when a reference the manager would have been asked for resolves to
`None`. The reporter had no log line at all to work from. `get_secret` only
reaches the manager when reads are enabled and the name is in `hosted_keys`, so
`secret_manager_would_be_consulted` mirrors that gate and keeps the warning off
env-only references, which are expected rather than an error.
A Redis over-limit verdict now survives a failing rollback DECRBY instead of
escaping into the in-memory fallback and granting tokens the counter already
rejected; the unrolled increments expire with the TTL. pop_reservation now
falls through to the local record when the Redis pop succeeds but finds
nothing, so a reservation saved in memory after a transient Redis save
failure still refunds on cancel or completion.
detect-backend-changes diffed the event payload's base.sha against the
checked-out ref. Those are two different points in time: actions/checkout
resolves refs/pull/N/merge, and GitHub recomputes that ref whenever the base
branch advances, so the diff picked up whatever landed on staging between the
event firing and the job starting. On a recent UI-only pull request three
backend commits from staging were attributed to the branch, and every backend
shard ran in full
Ask the API which files the pull request touches instead. That is the same set
the Files changed tab shows, and it is immune to either endpoint moving. The
shell body moves into .github/scripts/detect_backend_changes.sh so it can be
exercised directly, and the fail-open paths now also cover an API failure, a
file list past the API's 3000-entry listing ceiling, and a classifier that
prints something unexpected
Every dynamic tracer-provider build called Resource.create, which scans the entry
points of every installed distribution, roughly 3ms and 200 file opens. The dynamic
providers reach it from the async logging path, which runs on the event loop serving
requests, so past the provider cache bound every request paid it and delayed the
requests in flight alongside it
The value derives only from the logger's config and process environment, so it is
built once per logger and reused. This logger's own init-time providers share it,
which also removes redundant startup builds. ArizeLogger overrides _init_tracing and
still builds its own, so it keeps one extra build
Refs LIT-5437
In-memory grants now record an owner token, and a refund only debits local
counters when the popping worker is the one that granted them, so a terminal
response handled elsewhere can no longer shrink another worker's unrelated
fallback reservations. A Redis-granted refund that fails no longer falls back
to decrementing local counters either: the leaked Redis increments expire
with the TTL and only tighten the allowance.
The proxy pre-creates the logging object before the router picks a deployment,
and the native /v1/messages handler never copied the deployment's
vertex_location into the logging params it updates, so cost resolution fell
back to the environment or the default region and priced every call on this
surface with the regional endpoint uplift. Copy the explicitly configured
location from the request's litellm params, the same source dispatch builds
the request URL from, and register the new regional_endpoint_uplift_multiplier
field in the cost map schema test.
Auth and pass-through failures reach post_call_failure_hook with the raw request body unstripped, so a client-supplied standard_logging_object could feed the new attribution fallback when the logging object carries none. Pop the key before the lift so only the logging object may supply it. Also coalesce a None recovered cost to 0.0 so the lift always overwrites any client-supplied response_cost, matching the merge base's clobber semantics.
Reserve-script failures now roll back the scopes already incremented before
re-raising into the in-memory fallback, so a partial redis outage no longer
leaks counter increments that shrink the shared allowance. Reservations
record which backend granted them, so a refund never debits redis counters
an in-memory grant did not charge. Terminal-status matching is now
case-insensitive because the Bedrock async-invoke retrieve path returns raw
AWS-cased statuses like Completed.
The reasoning override's floor was pinned to tier_boundaries.simple_medium,
so an operator could not restore the unconditional promotion nor raise the bar
independently of the SIMPLE/MEDIUM cut. Setting reasoning_override_min_score
was accepted and echoed back by /model/info, because the config model allows
extra keys, while routing ignored it.
Resolve the floor through one accessor that falls back to simple_medium when
the field is unset, so moving that boundary still moves the floor with it, and
an explicit 0 is a real floor rather than an absent one. Record the resolved
value on the routing decision so a logged row states the floor that applied,
which is also what lets the Admin UI stop hardcoding the copy PR #37500 added.
* fix(auth): resolve bare model names against wildcard deployments in model access groups
* test(e2e): cover model access group permission checks on keys and teams
`lite up` already patches ~/.claude/settings.json, but only for as long as it
runs in the foreground, and it restores the original file on exit. Users
proxying Claude Code through LiteLLM therefore have to re-wire it by hand after
every login.
--config-claude makes that write persistent. It reuses the settings shape
`lite up` writes (env.ANTHROPIC_BASE_URL plus an apiKeyHelper invocation),
preserves every unrelated key, creates the file when missing, and writes it
atomically with owner-only permissions. Plain `lite login` is unchanged.
Reaching the credential through apiKeyHelper rather than copying it into the
file means a later login refreshes it with no further action, and keeps the
short-lived CLI token out of settings.json entirely.
The shared parts of the settings-file handling move from up.py into a new
claude_settings.py, since up.py imports auth.py and so auth.py cannot import
up.py back. That module now also owns the registry of commands that can be
temporarily managing the file, so the persistent write refuses while either
`lite up` or `lite autoroute up` holds a backup it would later restore over
this write.
Because this write has no backup and no `lite down`, it is stricter than
`lite up` about the user's file: it writes through a symlinked settings.json
rather than replacing the link with a regular file, and it refuses rather than
silently discarding a non-object `env` value.
Also fixes the apiKeyHelper command itself: --base-url belongs to the
top-level `lite` group, so `lite auth print-token --base-url X` is rejected by
click with "No such option". Every settings file `lite up` has written carries
that malformed command, which makes the helper return nothing and every Claude
Code request lose its token. The existing tests only string-matched the
generated command, so the new tests parse it through the real CLI instead.
An unknown reasoning split now falls back to reasoning_tokens=0 in the
chat-to-responses usage translation, since the OpenAI SDK requires
output_tokens_details with an int reasoning_tokens, and the streaming
chunk builder caps the tokenized reasoning estimate at completion_tokens
and fills text_tokens with the remainder
