A transient DB error during the spend log flush dropped that batch's guardrail
metrics and usage unit rows for good. Retry only the rows that failed, up to 3
times with 1s/2s/4s backoff, mirroring the daily spend writer, and inject the
sleep so tests stay fast. Lowers the lint budgets the refactor freed up
Every bedrock batch output line went through the Anthropic usage parser, which
reads snake_case input_tokens/output_tokens. Converse-family models (Nova and
friends) report camelCase inputTokens/outputTokens, so their usage came back
0/0/0 and the batch billed $0 despite real token consumption.
Usage is now selected by the shape of the payload: a Converse-shaped block goes
through the same transform the live Converse path uses, so a batch and an
equivalent non-batch call agree on tokens, including cache reads and writes.
Anthropic-shaped bedrock output is unchanged.
A shape neither parser understands (an InvokeModel-native payload from Titan,
Cohere, or Llama, which name their counts differently again) still reads zero,
but now warns with the keys it saw instead of silently billing $0.
Exposes the Converse usage transform as public, since batch parsing is a second
legitimate caller; that also removes the private-member access invoke_handler
was already making.
Replace Any-typed seams with real types in files carrying the highest
remaining reportAny/reportExplicitAny density after #34745: the proxy
server and its utils, the router, the streaming handler and chunk builder,
litellm_logging, the redis cache, the MCP db/tool-registry/spend-writer
layer, the anthropic pass-through adapters and guardrail translation, the
lasso and presidio guardrail hooks, the azure_ai agents handler, the
management endpoints (keys, users, ui_sso, model access groups, config
override, MCP, projects), the responses MCP handlers, response polling
background streaming, and the containers and vector stores mains
No casts, no type: ignore, no noqa, no new suppressions, and no Any
annotations that were not already at base. Whole-tree basedpyright:
reportAny 14,610 -> 14,009, reportExplicitAny 5,100 -> 4,780, total
144,743 -> 143,471, with no rule increasing repo-wide or in any file.
Budgets ratcheted: basedpyright -1,272 across 48 rules, ruff-strict -85,
type-discipline -37
The summary model access and budget checks read team_id, user_id,
project_id, and end_user_id via getattr with a None default so duck-typed
auth objects without those attributes keep working
Types 28 files with Protocols, TypedDicts, and Pydantic validation in place
of Any, cutting basedpyright reportAny by 974 and reportExplicitAny by 261
(1411 errors total across 48 rules), and ratchets the basedpyright, ruff
strict, and type discipline budgets down to match
reportAny 16720 -> 15482 and reportExplicitAny 5689 -> 5316 with real types only: no casts, no ignores, no new Any. Whole-tree basedpyright drops 2173 diagnostics with zero per-rule or per-file regressions. Budgets ratcheted: basedpyright -2173, ruff-strict -188, type-discipline -55
* fix(reset_budget_job): advance budget_reset_at atomically with the spend cascade
A postgres timeout mid-cascade previously left LiteLLM_BudgetTable rows
stamped for the next window while team member, enduser, org and tag spend
stayed at cap, so every later tick skipped them until the window rolled
over. All cascade writes and the budget_reset_at advance now share one
prisma batch transaction; a failed run persists nothing and the rows stay
due for the next ~10 minute tick. Cache and counter invalidation runs only
after commit, and the catch-all enduser log line now names the cascade.
* fix(reset_budget_job): elect one runner per tick and chunk the reset scans
Every pod and worker previously ran the reset job every ~10 minutes,
each fetching every expired row with no limit and writing one giant
transaction at the same calendar-aligned boundary; that concurrency is
what piled up postgres lock contention and timeouts. The job now takes
the shared PodLockManager redis lock (no redis keeps the old behavior),
and each phase walks its due rows in 500-row chunks, one transaction per
chunk, stopping when a chunk is short, makes no forward progress, or
hits the per-run cap; leftovers wait for the next tick.
* chore(lint): ratchet budget ceilings down for fixed violations
* fix(reset_budget_job): harden chunk loop, fail open on redis errors, heartbeat the lock
Review fixes on the two prior commits. Reset scans now skip rows with no
budget_duration, so permanently due rows can neither starve a phase nor
have a lifetime cap zeroed every tick. Chunk progress counts rows whose
new budget_reset_at actually cleared the cutoff, so a zero-length
duration cannot burn the per-run chunk cap. A failed lock acquire only
skips the run when another pod verifiably holds the lock; a broken redis
runs unguarded instead of silently disabling resets fleet-wide. Partial
row failures report real progress and fire the failure hook without
killing the phase. The leader re-asserts the lock between phases and
stops if another pod took over, and the budget window advance uses
update_many so a tier deleted mid-chunk cannot abort the transaction.
Lint budget ceilings re-ratcheted for the net-fixed violations.
* fix(reset_budget_job): renew the leader lease and reject non-positive budget durations
Bot review follow-ups. PodLockManager now extends the lock TTL when the
holding pod re-acquires, via an atomic compare-and-expire script with a
plain SET fallback, so a run longer than the TTL keeps its lease instead
of silently sharing the job with another pod. The positive-duration
validation that team member endpoints already had is hoisted to
management common_utils and applied to key, internal user, budget,
customer and team intake, so a tenant can no longer create zero-duration
budgets whose permanently due rows starve other tenants' resets. Such
durations now return 400 at intake; existing rows are untouched.
* refactor(reset_budget_job): defer leader election to a follow-up PR
* fix(reset_budget_job): satisfy strict lint gates
String defaults for the two getenv calls (PLW1508) and the chunk
outcome returns moved to try/else (TRY300).
Whole-tree basedpyright drops from 147,728 to 146,543 errors (reportAny
-670, reportExplicitAny -244) with no rule increasing repo-wide or in
any file. TypedDicts, Mapping/Sequence views, Protocols, and precise
helper return types replace Any; no casts, ignores, or runtime changes.
Budgets ratcheted down by the fixed amounts.
Typing-only pass over the 21 files with the highest reportAny and
reportExplicitAny density among self-contained modules: management
endpoints, guardrails, streaming internals, response transformations,
MCP server, enterprise managed files, and vector store management.
Whole-tree basedpyright drops from 148,648 to 146,984 errors (-1,664),
with reportAny -1,111 and reportExplicitAny -296. No rule increased
repo-wide and no file regressed on any rule. No cast(), type: ignore,
noqa, suppression comments, or new Any annotations anywhere in the diff,
and no runtime behavior changes.
Budgets ratcheted by make lint-budget-update: basedpyright -1,663 across
48 rules, ruff-strict -86, type-discipline -110.
* feat(proxy): return per-group model provenance on /team/info
/team/info now carries access_group_details, one entry per resolved access
group with its id, name, and model list, so the UI can attribute each
inherited model to the group granting it. The batch resolver returns the
access group rows keyed by id instead of a stringly dict of lists, and the
team member budget helper returns a copy instead of mutating its parameter.
Type discipline and basedpyright budgets ratchet down accordingly.
* feat(ui): allow group-only teams and show model provenance on hover
Team create and edit no longer require a model selection: an empty
selection is saved as the no-default-models sentinel, never as a bare
empty list, since an empty team model list means unrestricted access.
The team info Models card now renders every badge with a hover tooltip
naming how the team got that model: directly, via named access groups,
or both, and group-granted badges stay visible when the direct list is
empty or a sentinel.
* refactor(proxy): dedupe access group ids and return copies instead of mutating
Duplicate access_group_ids no longer amplify the /team/info response: ids
collapse order-preserving before provenance is built, pinned by a regression
test. The resolver returns a model_copy rather than mutating its parameter,
and the team create call sends a new object instead of reassigning
formValues.models. Budgets ratchet down further with the mutation removal.
* fix(proxy): deny when agent grants resolve to nothing
`get_allowed_agents` returned a plain list where the empty value meant both
"this caller was never restricted" and "this caller's grants resolved to
nothing". Downstream read either as allow-all, so a key restricted to one
agent inside a team restricted to another reached every agent on the proxy,
and an access group that resolved to no agents did the same.
Replace it with `resolve_agent_access`, returning a tagged
UnrestrictedAgentAccess | RestrictedAgentAccess. Only a caller with no grant
anywhere is unrestricted; an empty restricted set denies. Access group lookup
failures now propagate to the key/team resolvers so a DB error still fails
open exactly as before, while a group that genuinely resolves to nothing
denies.
* style(proxy): drop redundant comments from the agent access match
The base branch ratcheted the same limits in 28a277e9, so the conflicting
files were reset to base and the ratchet re-run against the new merge-base
rather than resolved by hand. Each limit is now the base value minus this
branch's own delta, so both ratchets survive: basedpyright -653 across 48
rules, strict ruff -80, LIT -85.
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree
Every ruff-strict rule that sat above its budget limit (FURB188, RUF022,
SIM118, UP007, UP032, UP037) is now at zero, LIT001 and LIT006 are back
under their ceilings, and the freed headroom is ratcheted out of
ruff-strict-budget.json, type-discipline-budget.json, and
basedpyright-code-budget.json so the gates take the fast path again
The proxy serves POST /openai/v1/responses alongside /responses and
/v1/responses, but only the latter two were in API_ROUTE_TO_CALL_TYPES.
UnifiedLLMGuardrails.async_post_call_success_hook resolves the call type
from request_route, so on the alias it resolved to None and returned the
response unscanned; model output reached the client with post-call
guardrails never running. The key and team tool allowlist was unenforced
on the same alias for the same reason.
Register the alias family in API_ROUTE_TO_CALL_TYPES and in
LiteLLMRoutes.openai_routes, mirroring how the /openai/v1/realtime
aliases are registered, and log a warning at the two points where the
unified guardrail skips post-call scanning so a future unmapped route is
visible instead of silent.
The Responses block of API_ROUTE_TO_CALL_TYPES moves from list to tuple
literals because the LIT002 budget rejects net-new mutable-collection
construction; the map is read-only, so it is now typed as a Mapping of
Sequence and the budgets ratchet down accordingly.
Replace Any-typed payload dicts, record shapes, and provider request/response
seams with TypedDicts, Protocols, and precise annotations in the ten litellm/
files carrying the highest combined basedpyright reportAny + reportExplicitAny
counts. No behavior changes.
Adds a regression test covering the managed-id list path so a prisma client
missing the managed tables keeps returning a fail-closed empty page.
The block event Rubrik receives sourced caller identity from
model_call_details[metadata], where the enriched litellm metadata never
lives; it sits under litellm_params. Every block therefore reported
user_api_key_hash as an empty string, so a security block could not be
traced to a key, user, or team.
Read identity off the authenticated UserAPIKeyAuth the failure hook is
already handed, via the same mapper the success path and the proxy spend
logger use, so a block log and a success log describe their caller with an
identical key set.
Replace Any seams in three proxy modules with real types so the values keep
their shape through the call graph:
- reset_budget_job: Protocols for the Prisma spend-linked tables, the reset
batcher, and each cascade row shape, with the per-table counter/cache key
lambdas promoted to typed module functions so the row type is inferred
- access_group_endpoints: Protocols for the access group record, the team and
key tables, and the transaction handle; record to response conversion now
goes through model_validate on the record dict
- cache_settings_endpoints: the opaque cache settings blobs are Mapping[str,
object] / dict[str, object] instead of Any, keeping Any only on the two
returns that feed the dynamic litellm.Cache kwargs bag
Whole-tree basedpyright: reportAny 19435 -> 19306, reportExplicitAny 6518 ->
6487, total errors 148372 -> 148117, with no rule above its baseline and no
untouched file changed. No behavior changes.
The type-discipline gate flagged 17 new mutable-collection annotations and 31
new mutable-collection constructions added by this branch. Replace raw dict
literals with the OpenAI SDK's TypedDict call forms, annotate read-only params
as Mapping/Sequence, precompute the custom tool call id set as a frozenset,
and accumulate streamed arguments as tuples. The few places where a plain
list/dict is a hard contract (pydantic response fields, fastapi route tags,
parsed request bodies, in-place tool call patching) carry reasoned mutable-ok
suppressions instead. Ratchet the ruff, type-discipline, and basedpyright
budgets down by the violations this branch now fixes on net
Replace Any-typed seams with real types in the files carrying the highest
reportAny/reportExplicitAny density. The dominant source was the repository
layer: BaseRepository.table is declared Any, so every repository read poisoned
its rows and every downstream call. Typed pass-through accessors under a
_PrismaTableActions Protocol pay that crossing once per table, and TypedDicts
and Protocols replace the remaining Any-typed request, row, and tool payloads
across the team, key, SCIM, spend, MCP, guardrail, video, and websearch
surfaces
No casts, no type: ignore, no noqa, no new Any annotations, no behavior
changes. Whole-tree basedpyright: reportAny 20,840 -> 19,397,
reportExplicitAny 7,253 -> 6,518, all rules 151,424 -> 149,066, with no rule
increased in any file. Budgets ratcheted: basedpyright -2,358, ruff-strict
-300, type-discipline -68
Swap `Model(**payload)` for `Model.model_validate(payload)` at the seams where
the payload comes back untyped, so basedpyright stops widening every target
field to Any. None of the models involved override `__init__`, so validation
goes through the same core validator either way.
Also route UserRepository through its own typed helpers (find_many, update,
find_by_id) instead of the raw Prisma table, drop the redundant `_to_model`
override signature, and call generate_key_helper_fn with explicit arguments in
the SSO callback rather than splatting an untyped dict.
Whole-tree basedpyright: reportAny 21481 -> 20834, reportExplicitAny 7258 ->
7252, with every other rule unchanged or lower.
Replace `Model(**untyped_dict)` construction with `Model.model_validate(...)` at
the hot Any seams, and give the repository layer a real record type instead of
`Any`.
reportAny 22710 -> 21448, reportExplicitAny 7283 -> 7269, with every other rule
at or below its baseline repo-wide.
Completed-batch cost tracking parsed the whole output file into a list of
dicts, pretty-printed it into debug strings even with debug logging off, and
walked the list three times (cost, usage, models), so a large batch output
could pin a worker's memory. The output is now folded line by line into small
per-line stats records via _aggregate_batch_cost_usage_models, the eager
json.dumps debug calls are gone, and the raw-vertex path computes cost and
usage in one call instead of two. _get_batch_output_file_content_as_dictionary
becomes _fetch_batch_output_file_content (returns bytes); the superseded
three-pass helpers are deleted and their tests migrated
Convert pydantic table-model construction from Cls(**row.model_dump())
kwargs-unpacking to Cls.model_validate(...) across the management endpoint
hotspot files (team, key, internal user, scim, model management, spend
tracking, auth checks, proxy_server). Unpacking an untyped dict reports one
Any-typed argument per matched model field, so each converted site clears
10-35 diagnostics while running the exact same pydantic validation.
Conversions were limited to models verified to use pydantic's default
__init__; UserAPIKeyAuth and LiteLLM_VerificationTokenView keep their custom
kwargs-rewriting __init__ and are untouched. Two locally-verified helper
params move from Any to object.
Whole-tree basedpyright, measured against the branch point in the same
environment: reportAny 24,431 -> 22,741 (-1,690), reportArgumentType
2,189 -> 2,136 (-53), reportUnknownArgumentType 34,370 -> 34,067 (-303),
reportExplicitAny 7,285 -> 7,283 (-2); total 154,882 -> 152,834 (-2,048)
with no rule increasing anywhere and no per-file increases. No casts, no
suppressions, no behavior changes. Budgets ratcheted: basedpyright -2,048
across 4 rules, ruff ANN401 -2.
Replace Any-typed seams with real types in the files carrying the highest
reportAny/reportExplicitAny density: typed Prisma read helpers in the MCP
db layer and verification token repository, TypedDicts for OAuth credential
payloads and aggregated spend rows, a DailySpendRecord protocol for the
daily activity endpoints, and concrete request/response types in the
volcengine, openai evals, azure batches, azure_ai count_tokens, and ocr
transformation modules. Modernize touched annotations to PEP 604/585 forms.
No casts, no type: ignore, no noqa, no new Any annotations, no behavior
changes. Whole-tree basedpyright: reportAny 27,005 -> 24,427,
reportExplicitAny 7,439 -> 7,280, no rule increased anywhere. Budgets
ratcheted: basedpyright -2,869, ruff-strict -1,505, type-discipline -167.
