reportAny 16720 -> 15482 and reportExplicitAny 5689 -> 5316 with real types only: no casts, no ignores, no new Any. Whole-tree basedpyright drops 2173 diagnostics with zero per-rule or per-file regressions. Budgets ratcheted: basedpyright -2173, ruff-strict -188, type-discipline -55
* fix(reset_budget_job): advance budget_reset_at atomically with the spend cascade
A postgres timeout mid-cascade previously left LiteLLM_BudgetTable rows
stamped for the next window while team member, enduser, org and tag spend
stayed at cap, so every later tick skipped them until the window rolled
over. All cascade writes and the budget_reset_at advance now share one
prisma batch transaction; a failed run persists nothing and the rows stay
due for the next ~10 minute tick. Cache and counter invalidation runs only
after commit, and the catch-all enduser log line now names the cascade.
* fix(reset_budget_job): elect one runner per tick and chunk the reset scans
Every pod and worker previously ran the reset job every ~10 minutes,
each fetching every expired row with no limit and writing one giant
transaction at the same calendar-aligned boundary; that concurrency is
what piled up postgres lock contention and timeouts. The job now takes
the shared PodLockManager redis lock (no redis keeps the old behavior),
and each phase walks its due rows in 500-row chunks, one transaction per
chunk, stopping when a chunk is short, makes no forward progress, or
hits the per-run cap; leftovers wait for the next tick.
* chore(lint): ratchet budget ceilings down for fixed violations
* fix(reset_budget_job): harden chunk loop, fail open on redis errors, heartbeat the lock
Review fixes on the two prior commits. Reset scans now skip rows with no
budget_duration, so permanently due rows can neither starve a phase nor
have a lifetime cap zeroed every tick. Chunk progress counts rows whose
new budget_reset_at actually cleared the cutoff, so a zero-length
duration cannot burn the per-run chunk cap. A failed lock acquire only
skips the run when another pod verifiably holds the lock; a broken redis
runs unguarded instead of silently disabling resets fleet-wide. Partial
row failures report real progress and fire the failure hook without
killing the phase. The leader re-asserts the lock between phases and
stops if another pod took over, and the budget window advance uses
update_many so a tier deleted mid-chunk cannot abort the transaction.
Lint budget ceilings re-ratcheted for the net-fixed violations.
* fix(reset_budget_job): renew the leader lease and reject non-positive budget durations
Bot review follow-ups. PodLockManager now extends the lock TTL when the
holding pod re-acquires, via an atomic compare-and-expire script with a
plain SET fallback, so a run longer than the TTL keeps its lease instead
of silently sharing the job with another pod. The positive-duration
validation that team member endpoints already had is hoisted to
management common_utils and applied to key, internal user, budget,
customer and team intake, so a tenant can no longer create zero-duration
budgets whose permanently due rows starve other tenants' resets. Such
durations now return 400 at intake; existing rows are untouched.
* refactor(reset_budget_job): defer leader election to a follow-up PR
* fix(reset_budget_job): satisfy strict lint gates
String defaults for the two getenv calls (PLW1508) and the chunk
outcome returns moved to try/else (TRY300).
Whole-tree basedpyright drops from 147,728 to 146,543 errors (reportAny
-670, reportExplicitAny -244) with no rule increasing repo-wide or in
any file. TypedDicts, Mapping/Sequence views, Protocols, and precise
helper return types replace Any; no casts, ignores, or runtime changes.
Budgets ratcheted down by the fixed amounts.
Typing-only pass over the 21 files with the highest reportAny and
reportExplicitAny density among self-contained modules: management
endpoints, guardrails, streaming internals, response transformations,
MCP server, enterprise managed files, and vector store management.
Whole-tree basedpyright drops from 148,648 to 146,984 errors (-1,664),
with reportAny -1,111 and reportExplicitAny -296. No rule increased
repo-wide and no file regressed on any rule. No cast(), type: ignore,
noqa, suppression comments, or new Any annotations anywhere in the diff,
and no runtime behavior changes.
Budgets ratcheted by make lint-budget-update: basedpyright -1,663 across
48 rules, ruff-strict -86, type-discipline -110.
* feat(proxy): return per-group model provenance on /team/info
/team/info now carries access_group_details, one entry per resolved access
group with its id, name, and model list, so the UI can attribute each
inherited model to the group granting it. The batch resolver returns the
access group rows keyed by id instead of a stringly dict of lists, and the
team member budget helper returns a copy instead of mutating its parameter.
Type discipline and basedpyright budgets ratchet down accordingly.
* feat(ui): allow group-only teams and show model provenance on hover
Team create and edit no longer require a model selection: an empty
selection is saved as the no-default-models sentinel, never as a bare
empty list, since an empty team model list means unrestricted access.
The team info Models card now renders every badge with a hover tooltip
naming how the team got that model: directly, via named access groups,
or both, and group-granted badges stay visible when the direct list is
empty or a sentinel.
* refactor(proxy): dedupe access group ids and return copies instead of mutating
Duplicate access_group_ids no longer amplify the /team/info response: ids
collapse order-preserving before provenance is built, pinned by a regression
test. The resolver returns a model_copy rather than mutating its parameter,
and the team create call sends a new object instead of reassigning
formValues.models. Budgets ratchet down further with the mutation removal.
* fix(proxy): deny when agent grants resolve to nothing
`get_allowed_agents` returned a plain list where the empty value meant both
"this caller was never restricted" and "this caller's grants resolved to
nothing". Downstream read either as allow-all, so a key restricted to one
agent inside a team restricted to another reached every agent on the proxy,
and an access group that resolved to no agents did the same.
Replace it with `resolve_agent_access`, returning a tagged
UnrestrictedAgentAccess | RestrictedAgentAccess. Only a caller with no grant
anywhere is unrestricted; an empty restricted set denies. Access group lookup
failures now propagate to the key/team resolvers so a DB error still fails
open exactly as before, while a group that genuinely resolves to nothing
denies.
* style(proxy): drop redundant comments from the agent access match
The base branch ratcheted the same limits in 28a277e9, so the conflicting
files were reset to base and the ratchet re-run against the new merge-base
rather than resolved by hand. Each limit is now the base value minus this
branch's own delta, so both ratchets survive: basedpyright -653 across 48
rules, strict ruff -80, LIT -85.
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree
Every ruff-strict rule that sat above its budget limit (FURB188, RUF022,
SIM118, UP007, UP032, UP037) is now at zero, LIT001 and LIT006 are back
under their ceilings, and the freed headroom is ratcheted out of
ruff-strict-budget.json, type-discipline-budget.json, and
basedpyright-code-budget.json so the gates take the fast path again
The proxy serves POST /openai/v1/responses alongside /responses and
/v1/responses, but only the latter two were in API_ROUTE_TO_CALL_TYPES.
UnifiedLLMGuardrails.async_post_call_success_hook resolves the call type
from request_route, so on the alias it resolved to None and returned the
response unscanned; model output reached the client with post-call
guardrails never running. The key and team tool allowlist was unenforced
on the same alias for the same reason.
Register the alias family in API_ROUTE_TO_CALL_TYPES and in
LiteLLMRoutes.openai_routes, mirroring how the /openai/v1/realtime
aliases are registered, and log a warning at the two points where the
unified guardrail skips post-call scanning so a future unmapped route is
visible instead of silent.
The Responses block of API_ROUTE_TO_CALL_TYPES moves from list to tuple
literals because the LIT002 budget rejects net-new mutable-collection
construction; the map is read-only, so it is now typed as a Mapping of
Sequence and the budgets ratchet down accordingly.
Replace Any-typed payload dicts, record shapes, and provider request/response
seams with TypedDicts, Protocols, and precise annotations in the ten litellm/
files carrying the highest combined basedpyright reportAny + reportExplicitAny
counts. No behavior changes.
Adds a regression test covering the managed-id list path so a prisma client
missing the managed tables keeps returning a fail-closed empty page.
The block event Rubrik receives sourced caller identity from
model_call_details[metadata], where the enriched litellm metadata never
lives; it sits under litellm_params. Every block therefore reported
user_api_key_hash as an empty string, so a security block could not be
traced to a key, user, or team.
Read identity off the authenticated UserAPIKeyAuth the failure hook is
already handed, via the same mapper the success path and the proxy spend
logger use, so a block log and a success log describe their caller with an
identical key set.
Replace Any seams in three proxy modules with real types so the values keep
their shape through the call graph:
- reset_budget_job: Protocols for the Prisma spend-linked tables, the reset
batcher, and each cascade row shape, with the per-table counter/cache key
lambdas promoted to typed module functions so the row type is inferred
- access_group_endpoints: Protocols for the access group record, the team and
key tables, and the transaction handle; record to response conversion now
goes through model_validate on the record dict
- cache_settings_endpoints: the opaque cache settings blobs are Mapping[str,
object] / dict[str, object] instead of Any, keeping Any only on the two
returns that feed the dynamic litellm.Cache kwargs bag
Whole-tree basedpyright: reportAny 19435 -> 19306, reportExplicitAny 6518 ->
6487, total errors 148372 -> 148117, with no rule above its baseline and no
untouched file changed. No behavior changes.
The type-discipline gate flagged 17 new mutable-collection annotations and 31
new mutable-collection constructions added by this branch. Replace raw dict
literals with the OpenAI SDK's TypedDict call forms, annotate read-only params
as Mapping/Sequence, precompute the custom tool call id set as a frozenset,
and accumulate streamed arguments as tuples. The few places where a plain
list/dict is a hard contract (pydantic response fields, fastapi route tags,
parsed request bodies, in-place tool call patching) carry reasoned mutable-ok
suppressions instead. Ratchet the ruff, type-discipline, and basedpyright
budgets down by the violations this branch now fixes on net
Replace Any-typed seams with real types in the files carrying the highest
reportAny/reportExplicitAny density. The dominant source was the repository
layer: BaseRepository.table is declared Any, so every repository read poisoned
its rows and every downstream call. Typed pass-through accessors under a
_PrismaTableActions Protocol pay that crossing once per table, and TypedDicts
and Protocols replace the remaining Any-typed request, row, and tool payloads
across the team, key, SCIM, spend, MCP, guardrail, video, and websearch
surfaces
No casts, no type: ignore, no noqa, no new Any annotations, no behavior
changes. Whole-tree basedpyright: reportAny 20,840 -> 19,397,
reportExplicitAny 7,253 -> 6,518, all rules 151,424 -> 149,066, with no rule
increased in any file. Budgets ratcheted: basedpyright -2,358, ruff-strict
-300, type-discipline -68
Swap `Model(**payload)` for `Model.model_validate(payload)` at the seams where
the payload comes back untyped, so basedpyright stops widening every target
field to Any. None of the models involved override `__init__`, so validation
goes through the same core validator either way.
Also route UserRepository through its own typed helpers (find_many, update,
find_by_id) instead of the raw Prisma table, drop the redundant `_to_model`
override signature, and call generate_key_helper_fn with explicit arguments in
the SSO callback rather than splatting an untyped dict.
Whole-tree basedpyright: reportAny 21481 -> 20834, reportExplicitAny 7258 ->
7252, with every other rule unchanged or lower.
Replace `Model(**untyped_dict)` construction with `Model.model_validate(...)` at
the hot Any seams, and give the repository layer a real record type instead of
`Any`.
reportAny 22710 -> 21448, reportExplicitAny 7283 -> 7269, with every other rule
at or below its baseline repo-wide.
Completed-batch cost tracking parsed the whole output file into a list of
dicts, pretty-printed it into debug strings even with debug logging off, and
walked the list three times (cost, usage, models), so a large batch output
could pin a worker's memory. The output is now folded line by line into small
per-line stats records via _aggregate_batch_cost_usage_models, the eager
json.dumps debug calls are gone, and the raw-vertex path computes cost and
usage in one call instead of two. _get_batch_output_file_content_as_dictionary
becomes _fetch_batch_output_file_content (returns bytes); the superseded
three-pass helpers are deleted and their tests migrated
Convert pydantic table-model construction from Cls(**row.model_dump())
kwargs-unpacking to Cls.model_validate(...) across the management endpoint
hotspot files (team, key, internal user, scim, model management, spend
tracking, auth checks, proxy_server). Unpacking an untyped dict reports one
Any-typed argument per matched model field, so each converted site clears
10-35 diagnostics while running the exact same pydantic validation.
Conversions were limited to models verified to use pydantic's default
__init__; UserAPIKeyAuth and LiteLLM_VerificationTokenView keep their custom
kwargs-rewriting __init__ and are untouched. Two locally-verified helper
params move from Any to object.
Whole-tree basedpyright, measured against the branch point in the same
environment: reportAny 24,431 -> 22,741 (-1,690), reportArgumentType
2,189 -> 2,136 (-53), reportUnknownArgumentType 34,370 -> 34,067 (-303),
reportExplicitAny 7,285 -> 7,283 (-2); total 154,882 -> 152,834 (-2,048)
with no rule increasing anywhere and no per-file increases. No casts, no
suppressions, no behavior changes. Budgets ratcheted: basedpyright -2,048
across 4 rules, ruff ANN401 -2.
Replace Any-typed seams with real types in the files carrying the highest
reportAny/reportExplicitAny density: typed Prisma read helpers in the MCP
db layer and verification token repository, TypedDicts for OAuth credential
payloads and aggregated spend rows, a DailySpendRecord protocol for the
daily activity endpoints, and concrete request/response types in the
volcengine, openai evals, azure batches, azure_ai count_tokens, and ocr
transformation modules. Modernize touched annotations to PEP 604/585 forms.
No casts, no type: ignore, no noqa, no new Any annotations, no behavior
changes. Whole-tree basedpyright: reportAny 27,005 -> 24,427,
reportExplicitAny 7,439 -> 7,280, no rule increased anywhere. Budgets
ratcheted: basedpyright -2,869, ruff-strict -1,505, type-discipline -167.
* feat(messages): route Azure Anthropic /messages through Rust behind rust:true
Adds an opt-in Rust path for non-streaming Azure Anthropic Messages. A
deployment sets rust: true in litellm_params to route litellm.messages()
and the proxy /v1/messages endpoint through the native Rust bridge; a
missing flag or rust: false keeps the existing Python path, and non-Azure
providers, streaming, an unavailable bridge, or a None result all fall
back to Python. Rust-backed responses carry an x-litellm-rust: true
response header so callers can see which path served the request.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(docs): exclude LITELLM_USE_RUST_MESSAGES rollout flag from env-doc check
Mirrors the existing LITELLM_USE_RUST_OCR entry; the flag is an internal
rollout toggle that is intentionally not in the public environment settings
docs yet.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(rust_bridge): isolate OCR enable flag and drop dead messages global toggle
use_litellm_rust only mutates the OCR enabled flag when configuring OCR (or
called with no bridge kwargs, preserving the legacy contract), so configuring
only the messages bridge no longer flips OCR state.
Remove the vestigial global enabled/env state from the messages bridge. Routing
is controlled per deployment by rust:true in the shared handler gate, so the
messages module never consulted the global toggle; drop it rather than leave a
no-op switch.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* refactor(rust/messages): split Anthropic config into its own provider file and type the request/response contract
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* feat(messages): route eligible Azure Anthropic streaming through Rust via buffered fake-stream
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(messages): fold system-role messages for Azure Anthropic and fall back to Python on Rust bridge errors
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(rust_bridge): use Python::attach for amessages after pyo3 bump
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(proxy): mock get_configured_token_limits in model_info tests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* ci: run rust_bridge unit tests in misc shard
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* Revert "ci: run rust_bridge unit tests in misc shard"
This reverts commit c86d861a03.
* test(anthropic): move rust messages bridge tests into misc-shard dir
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(batches): price anthropic passthrough message batches correctly in batch cost job
Anthropic message batches created via the /anthropic passthrough were never
cost tracked. The CheckBatchCost job fetched batch results from the Files API
(POST /v1/files/msgbatch_.../content), which Anthropic rejects with "File id
must have file_ prefix"; the error response was silently wrapped as file
content, parsed as zero successful rows, logged as a $0 aretrieve_batch spend
row, and the job was marked batch_processed=true so the $0 was permanent.
Route msgbatch_ file ids to GET /v1/messages/batches/{id}/results in the
anthropic files transformation, raise on HTTP error status in
retrieve_file_content instead of returning the error body as content, parse
Anthropic's results JSONL shape (result.type == "succeeded",
result.message.usage with cache creation/read tokens) in batch_utils, price
cache creation tokens at cache_creation_input_token_cost in the batch cost
fallback (50% batch discount preserved for base input, cache reads, cache
writes, and output), and leave the managed object row unprocessed when cost
tracking fails so a later poll retries instead of permanently recording $0.
* fix(batches): carry cache token details into aggregated anthropic batch usage
* fix: zero out crash-class basedpyright rules across litellm/
* feat(lint): add LIT009 banning inert type: ignore comments
* docs: require bracketed rule and reason on every suppression
* chore(lint): ratchet budgets down and zero crash-class pyright limits
* fix: narrow auto router routelayer through a local before calling
* test: add regression tests for crash-class fixes
* fix: drop dead AZURE_AD_TOKEN lookups and word-bound the type-ignore regex
* feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix
Upgrade the dashboard from Tailwind v3 to v4 with CSS-first config: the
official upgrade codemod renamed utilities across 151 files, and
tailwind.config.js (plus the dead tailwind.config.ts) is replaced by
@theme tokens, @source globs, and @plugin directives in globals.css. The
Tremor safelist becomes @source inline patterns and the legacy tremor
theme tokens carry over verbatim. ui_colors.json was build-time only and
fed the dying Tremor palette, so its brand values are inlined and the
file removed; runtime theming replaces that path next.
shadcn is initialized with a hand-authored components.json (rsc,
cssVariables, baseColor gray) pointing utils at the existing
lib/cva.config.ts, which now exports cn (cva beta cx + twMerge) instead
of adding class-variance-authority as a second variant library. The two
ad-hoc cn helpers fold into it. Button lands as the canary primitive,
adapted to cva beta and React 18 forwardRef, with tests covering the
variant, twMerge, asChild, and ref seams. --radius is 0.5rem so the
shadcn radius scale reproduces Tailwind defaults and legacy rounded-*
classes render unchanged.
antd v5 emits unlayered CSS-in-JS that would beat every layered v4
utility, so AntdGlobalProvider now wraps the app in StyleProvider layer
and ConfigProvider cssVar, and globals.css declares
@layer theme, base, antd, components, utilities. antd wins over
preflight but yields to utilities, which is what lets migrated shadcn
pages coexist with legacy antd pages. Preflight stays global with the
three v3 behaviors pinned (default border color, button cursor,
placeholder color).
* fix(ui): restore tremor opacity tints removed by tailwind v4
Tailwind v4 removed the *-opacity-* utilities, but the precompiled
@tremor/react dist still composes them with shade-500 palette classes
(bg-opacity-10 over bg-<color>-500 etc.), so Badge, BadgeDelta, Callout,
light Icon and Button, BarList, and ProgressBar lost their tints and
rendered solid 500-shade fills. Adversarial review caught it; the
original smoke pages only exercised antd Tags.
tremor-v3-compat.css restores exactly the pairs tremor emits: for each
of the 22 safelisted colors, bg-opacity-{10,20,40}, hover/group-hover
bg-opacity-{20,30}, and ring-opacity-{20,40} against the -500 shade,
via color-mix into the utilities layer. Tremor's colorPalette maps both
background and iconRing to 500, so the -500 pairing covers every
composition in the dist; dark: variants are inert until dark mode ships.
The shim dies with @tremor/react at the end of the migration.
The upgrade codemod also missed two hand-rolled modal scrims using
bg-black bg-opacity-{30,50} (solid black under v4); now bg-black/30 and
bg-black/50. Removed the docker/build_admin_ui.sh copy of
enterprise_colors.json into the deleted ui_colors.json; that build-time
rebrand path is retired and its runtime replacement lands with the
theming phase.
* fix(ui): pair ring-opacity-40 with shade 300 in tremor compat shim
Tremor's colorPalette maps ring to shade 300, and the only consumer of
ring-opacity-40 (Icon variant outlined) composes it with that shade,
so the shade-500 rows were dead and outlined icon rings would render
at full opacity. Latent today (no dashboard usage of the outlined
variant); caught by adversarial review. ring-opacity-20 stays at 500
(iconRing), matching Badge and BadgeDelta.
* chore(lint): raise basedpyright per-rule slack to 50% of baseline
The per-rule ceilings in basedpyright-code-budget.json sat at roughly 10% slack over baseline, which several in-flight PRs are already bumping into. Raise the slack on every rule to at least 50% of its baseline so there is ample headroom for a long while, while never lowering any rule that already had more generous slack (e.g. reportReturnType stays at 100).
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* refactor(lint): collapse type/lint budgets to a single per-rule limit
The three non-frontend budget files (ruff-strict, type-discipline, basedpyright-code) tracked a per-rule baseline and slack whose sum was the ceiling. Nothing consumed the split beyond that sum, so this replaces both keys with a single limit equal to the old baseline + slack; the original baselines live in git history if anyone needs them.
The gate scripts and the ratchet guard now read limit directly. lint-budget-update no longer re-captures raw counts; it ratchets each rule's limit down by the number of violations this branch cleared since its branch point (the merge-base), so the granted headroom shrinks by exactly what was fixed and a limit never rises. The ratchet guard reads either schema so it still compares correctly across the migration boundary.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* chore(lint): surface staged-vs-working parity for pre-commit and budget-update
make pre-commit selects which checks to run from the staged index but runs the linters over the working tree, so unstaged edits to tracked files and untracked files skew a green/red away from what a commit of only the staged changes would produce. There is no safe in-place way to lint the index, so the script now warns when unstaged or untracked changes are present, and CLAUDE.md documents that you must stage everything first for both make pre-commit and make lint-budget-update to predict CI correctly.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* docs(lint): list type-discipline budget in lint-budget-update instruction
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(cli): mint per-session agent credential on lite login
The `lite login` command was producing a shared UI session token that broke agent use in three ways: a $0.25 budget cap (from max_ui_session_budget) that killed agent sessions in minutes, a fixed identity "cli-jwt-token" shared across every user preventing per-session spend attribution, and auth gated behind EXPERIMENTAL_UI_LOGIN so the token was rejected on default deployments.
This fixes all three. Each login now generates a unique cli-session-{uuid} token with no per-key budget cap (enforced via shared team/user counters instead), and the decrypt path activates for any non-sk- token without requiring EXPERIMENTAL_UI_LOGIN.
* fix(cli): address review feedback on EXPERIMENTAL_UI_LOGIN gate and e2e test
Restore EXPERIMENTAL_UI_LOGIN=false as an explicit opt-out: operators who set it to false keep the old boundary; unset (new default) and true both attempt NaCl decryption, which fails closed for non-blob tokens.
In the e2e test: replace the silent Redis fallback with pytest.skip so a missing Redis instance is explicit rather than silently degrading to a directly-minted token. Write the seeded flow back as JSON (proxy reads it via json.loads on cache fetch) instead of Python repr, and build the updated flow immutably.
* fix(key-management): cap CLI session token delegation budget to team ceiling
A CLI session token intentionally carries max_budget=None to avoid a per-session LLM spend cap. The key-generation delegation check (GHSA-q775-qw9r-2r4g) previously skipped non-admin callers with max_budget=None, treating them as having unlimited delegation authority. This allowed any internal user with a lite login session to mint virtual keys with arbitrary budgets.
Adds is_session_token=True to UserAPIKeyAuth for CLI session tokens and uses the caller's team budget as the delegation ceiling in that case, so the effective limit is min(requested_budget, team.max_budget) rather than unbounded.
* chore: regenerate dashboard OpenAPI types
The is_session_token field added to UserAPIKeyAuth cascades to the
dashboard schema. Regenerate types from the updated OpenAPI spec.
* fix(key-management): block personal key budget delegation from CLI session tokens
When team_table is None (personal key, no team_id in request), the personal key
has no team-budget enforcement at request time. A session token therefore cannot
delegate any explicit max_budget for a personal key -- that would open a budget
bypass path. Block the request with a clear 400 directing the caller to use a
team_id instead.
* test(auth): add unit coverage for non-admin CLI session token production path
* fix(type-check): use model_validate in _return_user_api_key_auth_obj to fix reportArgumentType gate
UserAPIKeyAuth(**user_api_key_kwargs) spread triggers a basedpyright
reportArgumentType error for each named field in UserAPIKeyAuth because
the dict's inferred value type (str | Span | LitellmUserRoles | Unknown)
is not assignable to each field's specific type. Adding is_session_token:
bool introduced +2 more such errors, breaching the gate cap.
model_validate accepts an untyped dict without per-field argument checking,
which eliminates the +2 new errors and also ratchets down the pre-existing
333 errors at those call sites. basedpyright-code-budget.json is updated
to reflect the new lower baseline (1814, down from 1934).
* fix(type-check): ratchet down reportArgumentType baseline only
The previous lint-budget-update captured all baselines from the local
environment, raising many ceilings vs the merge-base and failing the
non-gating budget_ratchet_check. Restore staging's values for every
rule and only lower reportArgumentType (1934 -> 1814) to reflect the
reduction from switching to model_validate in _return_user_api_key_auth_obj.
* fix(auth): set max_budget on CLI session token to enforce max_ui_session_budget
CLI session tokens were missing max_budget, so _virtual_key_max_budget_check
had no per-session ceiling to enforce. Operators relying on max_ui_session_budget
could be bypassed for the full token lifetime. Mirrors the existing UI token path.
* revert(auth): remove max_ui_session_budget from CLI session token
max_ui_session_budget defaults to $0.25 and is sized for the UI chat
pane (10-min sessions). CLI sessions are 24-hour tokens for real work;
capping them at that ceiling would throttle users under their actual
user/team budget. Budget enforcement for CLI sessions is via the shared
user and team counters as originally intended.
* fix(auth): cap CLI session at max_ui_session_budget only when user and team have no budget
When neither the user nor their team has a budget configured, CLI sessions
were fully uncapped. The poll endpoint now looks up the real user and team
objects from DB; if both have no max_budget, it passes litellm.max_ui_session_budget
as the token's per-key ceiling. Users or teams that already have a budget
configured are unaffected and continue to rely on the shared counters.
* fix(auth): fix black formatting and update test mock for cli_poll_key budget lookup
The get_user_object and get_team_object async calls in cli_poll_key were
not mocked in the existing test, causing MagicMock await errors. Patch
both functions at the auth_checks module level. Also apply black formatting
to ui_sso.py which CI rejected.
* fix(auth): skip fallback budget cap when team lookup fails for cli session token
* test(auth): pin cli session budget cap to user/team budget presence
The session_max_budget fallback in cli_poll_key only applied
max_ui_session_budget when neither the user nor the resolved team had a
budget. The existing coverage exercised only the team-lookup-failure
branch. Add two regression tests: a user with a configured budget must
not receive the fallback cap, and a session with no user and no team
budget must fall back to max_ui_session_budget. Mutating either guard
out of the branch now fails these tests.
* fix: remove CLI poll session budget cap
* revert(auth): restore CLI session fallback budget cap
Bugbot autofix (60b81fb8) removed the user/team budget lookup in
cli_poll_key and stopped passing max_budget to the session token,
making CLI sessions fully uncapped whenever neither the user nor the
team has an explicit budget.
That reintroduces the unbounded-spend bypass veria flagged as High
("CLI session budget bypass"): on deployments that rely on
max_ui_session_budget rather than per-user/team budgets, a completed
lite login could run LLM calls with no ceiling for the whole token
lifetime. The fallback only applies when no other budget bounds the
session, so users and teams with a configured budget are unaffected and
keep relying on their shared counters.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(mcp): add v1 bridge + none/api_key resolver arms (unwired)
PR4a of the MCP v2 outbound-credential migration, stacked on the resolver skeleton.
Builds the bridge for the first live modes without wiring it onto the request path:
- resolver.py: the none arm (NoOpAuth) and the api_key shared-key arm (StaticHeaderAuth
from the config); the BYOK source and the other five arms stay not_implemented.
- adapter.py: the v1 <-> v2 edge (to_subject, to_server_spec, raise_public, should_defer).
to_server_spec maps only none + the static-header family and returns None to defer every
other mode to v1. Imports v1, kept out of the package __init__ so the resolver core stays
v1-free.
- MCPClient gains an optional resolved_auth that feeds the factory's auth= slot, taking
precedence over the SigV4 aws_auth; default None keeps current behavior.
Nothing calls these from _create_mcp_client yet, so production behavior is unchanged; the
graft lands in PR4b. Unit tests cover the two arms, the full mapping table, and the auth
plumbing.
* feat(mcp): graft v2 resolver onto _create_mcp_client for migrated modes
Wire the none + api_key static-family resolver arms from PR4a onto v1's
live request path. In _create_mcp_client's HTTP/SSE branch, to_server_spec
decides per mode: a migrated mode resolves through the injected
UpstreamCredentialProvider and feeds the resulting httpx.Auth into the new
resolved_auth slot; every other mode returns None and falls through to the
unchanged v1 construction. resolve_mcp_auth now runs only when the mode
defers, so a migrated server skips the v1 token-exchange / M2M I/O.
stdio is untouched: auth_type/auth_value never reach the upstream on the
stdio path (_get_auth_headers is HTTP/SSE only), so there is nothing to
graft there. No v1 code is deleted yet; resolve_mcp_auth's static return
still backs stdio and the not-yet-migrated modes until later PRs retire it.
* test(mcp): cover the v2-resolver graft in _create_mcp_client
Regression tests for the PR4 graft. Migrated HTTP modes resolve through the
provider into resolved_auth: none -> NoOpAuth, and the static api_key family
emits the right header per scheme (X-API-Key, Bearer, token, raw authorization,
base64 basic). Deferred modes (oauth2) and a missing static token fall back to
v1's auth_value. A stdio server with a migrated auth_type still defers to v1,
since httpx.Auth never reaches the subprocess. A resolver Error is mapped to the
public HTTP contract (401) via an injected provider, exercising the DI seam.
* fix(mcp): defer to v1 when an inbound credential would be overridden
The graft attaches the resolved static credential as an httpx.Auth, whose auth
flow writes its header after extra_headers. That silently overrode an inbound
Authorization: a per-request mcp_auth_header override, or a header supplied via a
guardrail hook / static_headers / forwarded caller header. v1 lets those win, so
the graft had inverted the credential precedence for the migrated static modes.
Mirror the v2 egress credential-isolation invariant: defer the request to v1 when
mcp_auth_header is set, or when the header the resolved credential would write is
already present in extra_headers. none writes no header, so it never defers.
* test(mcp): cover the credential-isolation defer guard
Regression tests for the precedence fix. A per-request mcp_auth_header override and an
Authorization already present in extra_headers (guardrail hook like the JWT signer,
static_headers, or a forwarded caller header) both defer a migrated static server to v1
so the inbound credential wins; none stays on v2 and does not clobber an inbound
Authorization since NoOpAuth writes nothing. The deferred cases assert resolved_auth is
None, which fails if the guard is removed.
* refactor(mcp): resolve inbound-header conflict on v2 instead of deferring
For an Authorization already supplied via extra_headers (a guardrail hook such as the
JWT signer, static_headers, or a forwarded caller header), keep the request on the v2
path and skip resolved_auth rather than deferring to v1. The inbound header still wins
since nothing overwrites it, but hooks no longer pin a v1 fallback, which is what lets
resolve_mcp_auth be retired once the remaining modes migrate.
The mcp_auth_header per-request override still defers to v1, since that value becomes
the upstream credential rather than sitting in extra_headers; that defer falls away
once the per-user modes stop writing mcp_auth_header.
* fix(mcp): clear UP037 lint gate and fix allowed-servers test under the graft
adapter.py uses `from __future__ import annotations`, so the quoted "UserAPIKeyAuth" /
"MCPServer" annotations in to_subject/to_server_spec/_shared_key_spec were unnecessary
and pushed UP037 over the strict-rule budget; drop the quotes.
test_list_tools_only_returns_allowed_servers passed a MagicMock as user_api_key_auth.
The graft now builds a Subject from the principal, and the MagicMock's non-string
org_id/user_id fail Subject validation, so the listing came back empty. Use a real
UserAPIKeyAuth instead (MagicMock for an injected dependency was the anti-pattern here).
* test(mcp): assert config token via resolved_auth, not the headers dict
test_mcp_server_config_auth_value_header_used inspected _get_auth_headers(), but the
graft now carries the static credential on the client's httpx.Auth (resolved_auth) and
writes the header at send time, so that dict is empty. Assert the header the
StaticHeaderAuth emits onto the request instead. Both config keys (authentication_token,
auth_value) stay covered.
* chore(typecheck): set reportMatchNotExhaustive slack to 0
The previous slack of 3 put the ceiling at baseline + slack = 4, so a newly
non-exhaustive match (for instance dropping an Error arm off a Result match)
could land without tripping the gate. Setting slack to 0 pins the ceiling at
the current baseline of 1, so any added non-exhaustive match now fails CI while
the one pre-existing violation in router.py stays within budget
* ci: re-run absolute basedpyright budget gate on push to long-lived branches
The basedpyright budget gate counts codebase-wide errors per rule against a
committed ceiling, but it only ran on pull_request against each PR's own head.
Two PRs that each pass in isolation can together push a per-rule count over its
ceiling once both merge, and nothing re-evaluated the budget on the merge
commit, so the breach only surfaced on the next PR that happened to be checked
out after the count crossed the line.
Add a push trigger on the long-lived branches and a post-merge-budget job that
re-runs the absolute gate on the merged tree, catching the accumulation on the
merge commit itself. The existing pull_request jobs are guarded so their
delta-vs-base gates don't misfire on push, where no PR base SHA exists.
* ci: shallow-fetch the post-merge-budget checkout
The post-merge-budget job only runs basedpyright over the working tree and
the committed budget file; it never inspects git history, unlike the lint
job whose delta-vs-base gates need full history. Drop its checkout from
fetch-depth: 0 to fetch-depth: 1 to avoid cloning the whole repo history.
* ci: scope post-merge-budget push trigger to long-lived branches
On a push event the branches filter matches the branch being pushed to,
not the PR target. The litellm_** glob, correct for the pull_request
filter where it matches the target branch, therefore fired the
post-merge-budget basedpyright job on every short-lived feature branch
carrying the litellm_ prefix (litellm_dev_*, litellm_add_*, and so on),
duplicating the PR lint job and burning ~10 minutes of CI per push.
Restrict the push trigger to the long-lived branches PRs actually merge
into (main, litellm_internal_staging, litellm_oss_branch), where budget
accumulation happens. The pull_request filter keeps litellm_** so PRs
targeting any long-lived branch are still linted.
* ci: make the basedpyright budget gate delta-vs-base
The basedpyright gate counted absolute codebase-wide errors per rule against a
committed ceiling and ran only on each PR's own head. Two PRs that each pass in
isolation could together push a rule past its ceiling once both merged, and
because the gate had no comparison against the base, the next unrelated PR
branched off the now-over-ceiling tree inherited a red it did nothing to cause.
Give it the same shape as the ruff strict gate: a rule fails only when its total
is both over the ceiling and higher than the count on the merge-base it merges
into. Drift already in the base is never blamed on a bystander, while any change
that actually grows a rule past the cap still fails. Head counts come from the
existing stdin pipe; the base count is a second basedpyright pass over a detached
worktree at the merge-base, reusing the head environment so import resolution
matches and no second uv sync is needed.
This obsoletes the push-triggered post-merge-budget job (and its event guards),
which only detected accumulation after the fact; the delta check blocks it on the
PR instead. Slack for reportReturnType and reportUnnecessaryComparison is raised
to give real headroom under the cap.
* refactor(ci): give the base ref its own name in type_check_gate cmd_check
cmd_check took a parameter named base that held a git ref string, then
rebound the same name to the dict of base-tree error counts returned by
base_counts. Rename the parameter to base_ref so the ref and the counts
each keep a single name and type, matching the no-reassignment style used
elsewhere; behavior is unchanged.
---------
Co-authored-by: Claude <noreply@anthropic.com>
