Cache-hit success callbacks in short-lived SDK scripts enqueue
Logging.async_success_handler onto the global logging worker; the worker
loop dequeues the task and asyncio.run cancels the processing task before
it ever starts, so the coroutine leaves the queue unawaited and the atexit
flush finds an empty queue and rescues nothing. Track dequeued-but-unfinished
tasks with strong refs and have the atexit flush run any whose coroutine
never started
Together AI moved its canonical API host from api.together.xyz to
api.together.ai. Default the provider api_base and the rerank handler to
the new host, make rerank honor api_base and TOGETHER_AI_API_BASE like
chat already does, map both hosts to together_ai when passed as
api_base, and delete the dead models/info fetch in factory.py.
Every repository handed its `.table` back untyped, so a dozen modules had
each grown a private `_PrismaTableActions` Protocol to paper over it. They
had drifted: some declared `update` as returning the row, others the row or
None, and none agreed on whether `find_many` was covariant
Replace all of them with a single `TableActions[RowT_co]` in
`litellm/repositories/prisma_protocols.py`, keyed to the prisma row each
repository is bound to. Query inputs stay `Mapping[str, object]` so callers
keep passing plain dicts, and `find_many` returns `Sequence` so the row type
stays covariant
Typing the nullable returns honestly surfaced paths that were already
crashing. A team admin could never edit or delete a memory entry owned by
their team: the write-auth check fed a raw prisma row to a helper that
expects the domain model, so `members_with_roles` arrived as plain dicts and
the request died as a 500 instead of applying the edit. Non-admin members hit
the same 500 in place of the 403 they were owed, so refusal and breakage were
indistinguishable. `/v2/model/info?user_models_only=true` dereferenced a
missing user row rather than returning the 400 the route already had, three
team routes dereferenced a team deleted between the read and the write, and
the agent registry dereferenced a missing agent instead of naming it
basedpyright drops 2,132 errors, 1,454 of them reportAny and 73
reportExplicitAny. The dashboard's generated types pick up `string[]` where
they had `unknown[]` for a team's members, admins and models
type the strategy-router health check params instead of a bare dict, annotate
the new interactions usage locals Final, drop a reportUnnecessaryIsInstance
suppression by narrowing the grounding tool list before iterating it, and delete
the duplicated file-id decode comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Follow-up to #38130. The function has no callers in the repo or the docs and is
not exported from `litellm/__init__.py`, and `token_counter` already does the same
job better, so keeping a second entry point only preserves a trap.
That trap is real: Greptile flagged on #38130 that `token_counter` picks the claude
tokenizer only for bare ids. `claude-sonnet-4-5` resolves to huggingface_tokenizer,
while `claude-3-opus-20240229` and `anthropic/claude-sonnet-4-5` fall back to the
OpenAI one, 24 tokens against 27 on the same string. Deleting the wrapper removes
the surface rather than papering over it; the selection gap in `token_counter`
itself is worth its own fix.
BREAKING CHANGE: `from litellm.utils import prompt_token_calculator` no longer
resolves. Use `litellm.token_counter(model=..., text=...)`.
The claude branch called the anthropic SDK's `Anthropic().count_tokens`, which the
SDK removed, so every claude call raised AttributeError. Counting now goes through
litellm's own token_counter, which handles anthropic models offline and drops the
SDK dependency entirely.
Hiding that was a swallowed error: `except Exception: Exception("Anthropic import
failed please run `pip install anthropic`")` built the exception without raising
it, so an environment missing the SDK fell through to the unguarded
`from anthropic import ...` on the next line and got a bare ModuleNotFoundError
instead of the install hint.
That was the codebase's last PLW0133, so the rule graduates from the ratcheted
budget into ruff.toml where it hard-fails, and editors get the diagnostic inline.
Type the annotations that landed in the last 24 hours and ratchet the lint budgets down accordingly. No behavior change.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Agent registry CRUD (/v1/agents*) sat in agent_routes, which feeds
llm_api_routes, so DISABLE_LLM_API_ENDPOINTS returned "LLM API routes are
disabled for this instance." for every Admin UI Agents tab call. Split the
group the same way MCP is split: agent_inference_routes stays on the data
plane, agent_management_routes joins management_routes, and agent_routes
remains their union for keys configured with allowed_routes=["agent_routes"].
Non-admin callers reached agent CRUD through llm_api_routes before, so the
management paths also join self_managed_routes and the llm_api_routes virtual
key carve-out; the handlers already scope reads by role and 403 non-admin
writes.
Both new groups are tuples, so check_route_access now takes a Sequence and
matches wildcards through a generator instead of materializing an
intermediate list on every call.
Azure Database for PostgreSQL Flexible Server takes a Microsoft Entra ID access
token as the connection password, and those tokens last about an hour, so a
proxy pointed at one dies shortly after boot unless something keeps minting
fresh ones
Set AZURE_POSTGRESQL_AUTH=True (or pass --azure_postgresql_auth) alongside
DATABASE_HOST, DATABASE_USER, and DATABASE_NAME, and the proxy mints a token at
startup, assembles the connection URL around it, and refreshes it in the
background for as long as the process runs. That is the same shape
IAM_TOKEN_DB_AUTH already had for AWS RDS, so the two now share one code path:
a tagged union picks the minting strategy once, and the wrapper, the read
replica, and the refresh loop all read the choice off it instead of each
guessing from the environment. Setting both toggles is a startup error, in the
chart as well as in Python
The helm chart gets database.writer.useAzureEntraAuth and the matching reader
knob next to the existing useIAMAuth
Fixes#29661
Co-authored-by: David Balatoni <balcsida@gmail.com>
Every dynamic tracer-provider build called Resource.create, which scans the entry
points of every installed distribution, roughly 3ms and 200 file opens. The dynamic
providers reach it from the async logging path, which runs on the event loop serving
requests, so past the provider cache bound every request paid it and delayed the
requests in flight alongside it
The value derives only from the logger's config and process environment, so it is
built once per logger and reused. This logger's own init-time providers share it,
which also removes redundant startup builds. ArizeLogger overrides _init_tracing and
still builds its own, so it keeps one extra build
Refs LIT-5437
Resync registry misses with single-row DB fetches (guardrail by unique
name, agent by unique id or name, model by name then id) instead of
full-table loads, and bound them with a global budget of 20 resyncs per
5s window per registry that fails closed without negative-caching the
key.
Access group create/update now trust the reconcile outcome snapshot
captured under the reload lock instead of a post-lock router read, so a
concurrent reconcile can no longer surface a false degraded-serving 500.
Router.upsert_deployment restores the previously served deployment when
the replacement add fails under ignore_invalid_deployments, so a bad
update no longer silently drops a healthy deployment from serving.
* feat(proxy): add /team/daily/activity/aggregated and use it in the Usage UI
The Team Usage tab drained row-paginated pages client side, which painted
newest days first and drew duplicate bars when a day's rows straddled a
page boundary. Serve the whole range in one SQL GROUPING SETS pass instead:
the aggregated query gains optional per-entity rollup levels (entity as the
most-significant GROUPING bit) so breakdown.entities keeps per-team spend,
aliases, and per-key splits. The endpoint shares the paginated route's
scoping via _resolve_team_daily_activity_scope, accepts the timezone the UI
already sends, and the api_key filter now takes a list so non-admin member
scoping works. The dashboard tries the aggregated endpoint first and falls
back to page draining on failure.
* chore: ratchet B008 budget down by the endpoint converted to Annotated Depends
* chore: keep mutable-ok suppressions on their annotation lines after formatting
* fix(proxy): reject malformed or over-wide ranges on team aggregated activity
The aggregated endpoint has no pagination bounding its work, so validate
start_date and end_date as real dates and cap the span at 400 days. The
dashboard's widest presets fit well inside the cap, and an over-cap range
falls back to the paginated flow. Also trim implementation comments that
restated the grouping-set code.
* fix(proxy): parse aggregated range bounds as UTC to satisfy DTZ007
* refactor(proxy): fetch entity rollups with a companion query instead of extending the main one
The entity-as-extra-GROUPING-bit approach made the bitmask layout
mode-dependent: the same constant meant (date) for normal rows and
(date, entity) for entity rows, disambiguated by masking. Split it out:
the shared WHERE builder feeds both the untouched main query and a small
per-entity rollup query keyed by GROUPING(api_key), run concurrently, and
a fold writes breakdown.entities onto the built response.
* refactor(proxy): share the daily-activity error and entity-metadata shapes
The type-discipline ceiling for LIT002 ratcheted down on staging, so the new
aggregated endpoint had to stop hand-rolling collections the codebase already
builds elsewhere. Funnel the `{"error": ...}` detail through one construction
site, turn the range validator into an error-as-value, reuse a single
entity-metadata lookup for both breakdown paths, and widen
get_api_key_metadata to any set so callers stop copying frozensets.
A transient DB error during the spend log flush dropped that batch's guardrail
metrics and usage unit rows for good. Retry only the rows that failed, up to 3
times with 1s/2s/4s backoff, mirroring the daily spend writer, and inject the
sleep so tests stay fast. Lowers the lint budgets the refactor freed up
Replace Any-typed seams with real types in files carrying the highest
remaining reportAny/reportExplicitAny density after #34745: the proxy
server and its utils, the router, the streaming handler and chunk builder,
litellm_logging, the redis cache, the MCP db/tool-registry/spend-writer
layer, the anthropic pass-through adapters and guardrail translation, the
lasso and presidio guardrail hooks, the azure_ai agents handler, the
management endpoints (keys, users, ui_sso, model access groups, config
override, MCP, projects), the responses MCP handlers, response polling
background streaming, and the containers and vector stores mains
No casts, no type: ignore, no noqa, no new suppressions, and no Any
annotations that were not already at base. Whole-tree basedpyright:
reportAny 14,610 -> 14,009, reportExplicitAny 5,100 -> 4,780, total
144,743 -> 143,471, with no rule increasing repo-wide or in any file.
Budgets ratcheted: basedpyright -1,272 across 48 rules, ruff-strict -85,
type-discipline -37
Types 28 files with Protocols, TypedDicts, and Pydantic validation in place
of Any, cutting basedpyright reportAny by 974 and reportExplicitAny by 261
(1411 errors total across 48 rules), and ratchets the basedpyright, ruff
strict, and type discipline budgets down to match
* feat(router): add required-AND (&) tag prefix and allow_fail_open flag
Tag routing supported inclusion-OR and independent "!" negation, but had no way
to express a hard "must match all of these" constraint per request, and no way
for a model group to opt into degrading gracefully instead of raising when a
constraint eliminates every deployment.
Adds a "&tag" prefix for required-AND inclusion, composing with existing plain
(OR) and "!" (negate) tags: negation still applies first, then required tags
narrow the survivors, then plain tags apply today's OR/AND preference logic
unchanged. Adds model_info.allow_fail_open (default false) so a chain can opt
into falling back to the default-tagged pool instead of raising
no_deployments_with_tag_routing when "!" or "&" empties the candidate set;
existing chains without the flag keep today's fail-closed behavior exactly.
* fix(router): gate mixed negation on allow_fail_open and stop diluting required-only requests
Two gaps in the initial required-AND/allow_fail_open change: a "!" exclusion
combined with a plain positive tag that emptied the candidate set raised
unconditionally, bypassing allow_fail_open entirely, since the fail-open check
only looked at required-AND exhaustion. And a request using only "&" tags
could get narrowed down to just the deployment matching an incidental
tag_regex/User-Agent preference, silently dropping other deployments that
satisfied the required tags but had no tag_regex at all.
Fixes both: the fail-open check now fires whenever either "!" or "&" leaves
the candidate set empty, not just "&". And regex/header preference no longer
counts as a positive filter when a required-AND ask is present, so a
required-only request returns every deployment satisfying the required tags
regardless of regex/header matching.
Also regenerates ui/litellm-dashboard/src/lib/http/schema.d.ts for the new
model_info.allow_fail_open field, and removes source comments explaining
the router logic per repository convention.
* fix(router): let allow_fail_open cover a non-empty !/& survivor set that fails the plain-tag preference
The unconditional raise inside the has_positive_filter loop was the one
remaining path a chain could hit despite setting allow_fail_open: when "!"
or "&" leaves a non-empty candidate set but none of the survivors match the
request's plain preference tag or carry "default", the request still raised
instead of degrading. Routes that raise through the same allow_fail_open
check used everywhere else, so it now falls back to the default-tagged pool
for opted-in chains and keeps raising unconditionally for everyone else.
This also let the now-redundant pre-loop empty-candidates shortcut be
removed, since the loop reaches the same outcome on its own.
* fix(router): deny allow_fail_open when an unrecognized required tag is masking a satisfiable answer
A caller could add a single "&" tag no deployment in the group has ever
carried to force an empty required-AND set on demand. On a chain with
allow_fail_open, that emptied set fell back to the default-tagged pool
unconditionally, discarding every other constraint merged into the same
request, including ones inherited from key/team policy, even when the rest
of those constraints were still individually satisfiable.
Before falling back, drop any required tag not carried by any deployment in
the group and recompute: if a specific, non-empty answer exists using only
the recognized tags, the unrecognized tag was the actual cause of the
exhaustion, and fail-open must not paper over it. If every required tag is
already recognized, or none are, there's nothing hidden behind an invented
tag, and fail-open proceeds exactly as before; this keeps a single opted-in
deployment's legitimate catch-all behavior working when a caller's tag
simply doesn't exist anywhere in that group.
Ratchets ANN401 and LIT001 budgets down to reflect fixes already earned in
this branch.
* test(router): cover required-AND, allow_fail_open, and unknown-tag denial across fallback chains and model groups
Extends coverage beyond single-hop scenarios: & exhausting a primary group
falls through to a fallback group exactly like ! already does; !, &, and
allow_fail_open composed together across three chained model groups each
raise or fall back independently per-hop; and the unknown-tag denial from
the previous commit is evaluated fresh per hop rather than leaking state
across groups in a fallback chain.
* feat(router): add model_info.enable_tag_filtering per-model-group override
enable_tag_filtering was router-wide only: an operator turning it on for one
model group that needs tag-driven routing exposed every other model group on
the same proxy to the same tag evaluation, even ones that never use tags.
Adds model_info.enable_tag_filtering, checked against any deployment sharing
a model_name, so a chain can flip the router-wide default in either
direction for itself alone: opt a specific group into filtering while the
rest of the proxy stays off, or opt a group out (e.g. an incident-response
catch-all) while the rest of the proxy enforces it.
Precedence, low to high: router-wide default, then the chain override if
set, then the existing request-level escalation (from key/team settings),
which still only ever turns filtering on, never off, over whatever the
router and chain already decided.
Also regenerates ui/litellm-dashboard/src/lib/http/schema.d.ts for the new
field.
* fix(router): gate plain-tag exhaustion on allow_fail_open when the tag is known to the group
A model group where every deployment is tagged "default" (a legitimate
cross-cutting safety-net pattern) never has an empty default_deployments
list, so the existing exhaustion check (len(new)==0 and len(default)==0)
never fired for a plain positive tag that matched nothing among the
currently healthy candidates. The request silently fell through to whatever
"default"-tagged deployment happened to survive, even when allow_fail_open
was never set and the caller's intent (e.g. quality:high) was never honored.
Adds a check for whether the requested tag is part of the group's real
vocabulary at all: if some deployment configured under this model_name
(regardless of current health) genuinely carries the tag, and nothing
healthy currently matches it, the request now raises by default or falls
back per allow_fail_open, through the same _resolve_or_fail_open gate every
other exhaustion path already uses. A tag that's foreign to the group
entirely (e.g. one meant for an unrelated mechanism sharing the same
request-tags list) keeps falling back to the default pool unconditionally,
unchanged, since there's nothing this group's own routing intent could be
violating.
* fix(router): preserve inherited tag constraints when allow_fail_open discards a caller-caused exhaustion
Adds metadata.caller_tags in litellm_pre_call_utils.py, populated only from
what the request itself supplied (header, body tags, body metadata.tags),
never from key/team metadata merged into the same metadata.tags list.
get_deployments_for_tag now uses it to compute a trusted-only pool before
falling open: a required/excluded tag attributable to the caller can be
discarded on fail-open, one inherited from key/team policy cannot. If the
trusted-only pool is itself empty, allow_fail_open raises instead of
silently routing around an unsatisfiable inherited constraint. When
caller_tags carries no information at all (direct SDK Router usage,
bypassing the proxy layer), behavior is unchanged: unconditional fall-open
to the default pool, exactly as before this fix.
* feat(router): add opt-in tag_routing_prefix for collision-proof tag disambiguation
router_settings.tag_routing_prefix lets a caller explicitly mark which
x-litellm-tags/metadata.tags values are routing directives, exempting
them from the known-tag-vocabulary heuristic used to guard fail-open
against caller-invented "&"/"!" tags. Unprefixed tags keep going
through today's existing handling unchanged (hybrid, no migration
required); default "" is a full no-op.
Fixes a bug caught during live-proxy verification: the prefix-stripped
"confirmed" set kept the "&"/"!" marker character, so it never matched
required_set/excluded_set (which _split_tags always strips bare) -- the
entire trusted-required/excluded-tag mechanism silently no-opped for
its primary use case. Adds regression tests for the bare-value mismatch
and updates existing _chain_allows_fail_open/_tag_known_to_group/
_caller_constraint_sets call sites for the new routing_confirmed/
routing_prefix parameters.
* fix(router): resolve model_info.enable_tag_filtering override from the full model group, not just healthy deployments
Cooldown filtering runs before get_deployments_for_tag, so
_chain_tag_filtering_override only saw the survivors of that filter.
A model group whose only enable_tag_filtering-carrying deployment goes
into cooldown lost the override entirely, silently falling back to
the router-wide default and letting any !/&/tag constraint on that
chain be bypassed by driving the one overriding deployment into
cooldown. Resolve the override from every deployment configured for
the model instead, mirroring _tag_known_to_group's existing pattern.
Verified live: with a bad-key deployment carrying the override forced
into real cooldown via allowed_fails=1, an explicit "!provider:openai"
ban on the remaining deployment reproducibly returned 200 via OpenAI
before this fix and 401 (tag filtering still enforced) after it.
* fix(router): avoid Final-reassignment lint error and a MagicMock router fixture gap from tag_routing_prefix
_chain_tag_filtering_override's try/except reassigned a Final-annotated
name across branches, which basedpyright flags as illegal; extracted
the lookup-with-fallback into its own helper so the binding is assigned
once. Also sets tag_routing_prefix on the bare MagicMock router used by
test_router_tag_regex_routing.py's fixture, which otherwise returns an
auto-generated MagicMock (truthy, non-string) for the new attribute and
crashes _strip_routing_prefix's removeprefix() call.
* fix(router): key inherited-tag protection off provenance, not value subtraction
allow_fail_open's trusted-only pool computed "not caller-attributable"
as required_set - caller_required_set. A caller who resubmits the
exact value of an inherited "&"/"!" tag (e.g. an inherited "®ion:eu"
alongside a caller-supplied "®ion:eu" plus a conflicting
"!region:eu") collapses both origins to the same set value, so the
subtraction zeroes out the inherited requirement's protection too,
letting fail-open route outside a key/team-enforced constraint.
Adds metadata.inherited_tags in litellm_pre_call_utils.py: a snapshot
of "tags" taken after key/team/project policy is merged in but before
this request's own caller-supplied tags are merged on top. A required
or excluded tag is now protected from fail-open discard if it has ANY
inherited backing (set intersection with inherited_tags), regardless
of whether the caller also happens to submit the identical value --
this is what set membership alone could never tell apart under the
old subtraction-based approach. caller_tags is kept (documented as the
complementary record) but no longer consulted for this decision.
Verified live: a virtual key with metadata.tags=["®ion:eu"] hit
with header x-litellm-tags: ®ion:eu,!region:eu (the exact
value-collision attack) reproducibly routed to the OpenAI/us
deployment before this fix and stayed on the Anthropic/eu deployment
after it.
* fix(lint): re-ratchet budget ceilings after rebasing onto litellm_internal_staging
Regenerated ruff-strict-budget.json and type-discipline-budget.json
via make lint-ruff-budget-update / lint-type-discipline-budget-update
against the post-rebase merge-base.
* fix(proxy): compute inherited_tags from key/team/project sources directly, not a tags-list snapshot
apply_client_tag_policy_pre_auth (run from user_api_key_auth, for
_tag_max_budget_check) merges the caller's x-litellm-tags header into
the same metadata.tags list before add_litellm_data_to_request ever
runs. The previous inherited_tags snapshot ("whatever's in tags before
this function's own caller-tag merge") therefore misattributed that
caller-controlled value as policy-backed whenever a request arrived
with the header set -- Greptile flagged this as a P1 security finding.
inherited_tags is now built directly from key_metadata/team_metadata/
project_metadata's own "tags" fields, independent of the shared,
pipeline-position-dependent "tags" list's mutation history. Verified
with a direct reproduction mirroring the real pipeline (calling
apply_client_tag_policy_pre_auth on the same data dict before
add_litellm_data_to_request, as user_api_key_auth actually does): the
caller's header tag no longer appears in inherited_tags. Added a
regression test exercising that same call order; confirmed it fails
against the pre-fix snapshot approach and passes against this fix.
* fix(router): make tag_routing_prefix configurable through update_settings/get_settings and UpdateRouterConfig
router_settings.tag_routing_prefix was only ever applied via the
Router() constructor. Router.update_settings's _allowed_settings
(used directly by proxy_server.py's _add_router_settings_from_db_config
for the DB-backed router_settings path) and get_settings's
vars_to_include both omitted it, so an operator relying on that path
had the value silently ignored -- flagged by veria-ai. Also adds it to
UpdateRouterConfig (the pydantic schema behind POST /config/update),
the same bug shape LIT-3152 previously fixed for retry_policy: a field
missing from that schema gets silently dropped by
model_dump(exclude_none=True) before update_settings is ever called.
* chore(ui): regenerate schema.d.ts for UpdateRouterConfig.tag_routing_prefix
Adding tag_routing_prefix to UpdateRouterConfig changed the proxy's
OpenAPI spec; regenerate the dashboard's generated API types to match.
* fix(lint): re-ratchet budget ceilings after rebasing onto litellm_internal_staging
Regenerated ruff-strict-budget.json and type-discipline-budget.json
against the post-rebase merge-base. LIT002/LIT011 ceilings reflect
this branch's true current counts (confirmed unchanged across the
rebase by diffing against the pre-rebase commit); the base's own
counts moved independently.
* fix(lint): replace mutable-collection fallbacks with immutable ones in inherited_tags computation
key_metadata/team_metadata/project_metadata's "tags" fallbacks used
`or {}` / `or []` literals, each a LIT002 mutable-collection-construction
violation that pushed the branch 4 over its ratchet ceiling relative to
a moved base. Swapped to MappingProxyType({}) / () to match the
immutable idiom the rest of tag_based_routing.py already uses; no
behavior change, since both are falsy and only ever read via .get()/
unpacking. Tightens type-discipline-budget.json's LIT002 ceiling back
down to match, fully closing that gap (LIT011 keeps a genuine 1-count
gap from pre-existing, untouched lines in this file, non-gating).
* fix(lint): suppress LIT011 on the two new data[...] mutation sites
Both new lines follow this file's established data[...] mutation
idiom for add_litellm_data_to_request, matching the existing
suppression already on the inherited_tags line.
* test(router): lock in fallback + tag-filtering interaction
Cover the router-level fallbacks mechanism composing with tag-based
routing: a plain negation exhausting a group correctly advances to
the fallback group, the same exclusion tag exhausting every hop
correctly raises, and allow_fail_open resolving locally must not
spuriously trigger an unrelated external fallback.
* chore: retrigger CI now that litellm-docs#814 is merged
---------
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
reportAny 16720 -> 15482 and reportExplicitAny 5689 -> 5316 with real types only: no casts, no ignores, no new Any. Whole-tree basedpyright drops 2173 diagnostics with zero per-rule or per-file regressions. Budgets ratcheted: basedpyright -2173, ruff-strict -188, type-discipline -55
* fix(reset_budget_job): advance budget_reset_at atomically with the spend cascade
A postgres timeout mid-cascade previously left LiteLLM_BudgetTable rows
stamped for the next window while team member, enduser, org and tag spend
stayed at cap, so every later tick skipped them until the window rolled
over. All cascade writes and the budget_reset_at advance now share one
prisma batch transaction; a failed run persists nothing and the rows stay
due for the next ~10 minute tick. Cache and counter invalidation runs only
after commit, and the catch-all enduser log line now names the cascade.
* fix(reset_budget_job): elect one runner per tick and chunk the reset scans
Every pod and worker previously ran the reset job every ~10 minutes,
each fetching every expired row with no limit and writing one giant
transaction at the same calendar-aligned boundary; that concurrency is
what piled up postgres lock contention and timeouts. The job now takes
the shared PodLockManager redis lock (no redis keeps the old behavior),
and each phase walks its due rows in 500-row chunks, one transaction per
chunk, stopping when a chunk is short, makes no forward progress, or
hits the per-run cap; leftovers wait for the next tick.
* chore(lint): ratchet budget ceilings down for fixed violations
* fix(reset_budget_job): harden chunk loop, fail open on redis errors, heartbeat the lock
Review fixes on the two prior commits. Reset scans now skip rows with no
budget_duration, so permanently due rows can neither starve a phase nor
have a lifetime cap zeroed every tick. Chunk progress counts rows whose
new budget_reset_at actually cleared the cutoff, so a zero-length
duration cannot burn the per-run chunk cap. A failed lock acquire only
skips the run when another pod verifiably holds the lock; a broken redis
runs unguarded instead of silently disabling resets fleet-wide. Partial
row failures report real progress and fire the failure hook without
killing the phase. The leader re-asserts the lock between phases and
stops if another pod took over, and the budget window advance uses
update_many so a tier deleted mid-chunk cannot abort the transaction.
Lint budget ceilings re-ratcheted for the net-fixed violations.
* fix(reset_budget_job): renew the leader lease and reject non-positive budget durations
Bot review follow-ups. PodLockManager now extends the lock TTL when the
holding pod re-acquires, via an atomic compare-and-expire script with a
plain SET fallback, so a run longer than the TTL keeps its lease instead
of silently sharing the job with another pod. The positive-duration
validation that team member endpoints already had is hoisted to
management common_utils and applied to key, internal user, budget,
customer and team intake, so a tenant can no longer create zero-duration
budgets whose permanently due rows starve other tenants' resets. Such
durations now return 400 at intake; existing rows are untouched.
* refactor(reset_budget_job): defer leader election to a follow-up PR
* fix(reset_budget_job): satisfy strict lint gates
String defaults for the two getenv calls (PLW1508) and the chunk
outcome returns moved to try/else (TRY300).
Whole-tree basedpyright drops from 147,728 to 146,543 errors (reportAny
-670, reportExplicitAny -244) with no rule increasing repo-wide or in
any file. TypedDicts, Mapping/Sequence views, Protocols, and precise
helper return types replace Any; no casts, ignores, or runtime changes.
Budgets ratcheted down by the fixed amounts.
Typing-only pass over the 21 files with the highest reportAny and
reportExplicitAny density among self-contained modules: management
endpoints, guardrails, streaming internals, response transformations,
MCP server, enterprise managed files, and vector store management.
Whole-tree basedpyright drops from 148,648 to 146,984 errors (-1,664),
with reportAny -1,111 and reportExplicitAny -296. No rule increased
repo-wide and no file regressed on any rule. No cast(), type: ignore,
noqa, suppression comments, or new Any annotations anywhere in the diff,
and no runtime behavior changes.
Budgets ratcheted by make lint-budget-update: basedpyright -1,663 across
48 rules, ruff-strict -86, type-discipline -110.
Every strict-gate rule whose budget ceiling was already 0 moves into the base
config's lint.extend-select, so editors and ruff check --fix surface the
diagnostics directly and the budget file shrinks to rules with real debt.
Graduates stay in ruff-strict.toml's select so the strict RUF100 pass keeps
policing their stale noqa directives, and base external entries they made
redundant (FURB, I001, RUF010, RUF022, RUF023, RUF051) are dropped so base
RUF100 polices those directly. UP037 had two violations hidden behind a star
import; importing Literal explicitly fixes them so UP037 can graduate too.
New drift tests pin the invariants: every strict-selected rule is budgeted or
hard-failed by base, every base-owned rule stays visible to exactly one
RUF100 pass, and graduated rules fail the normal ruff run.
* fix(proxy): deny when agent grants resolve to nothing
`get_allowed_agents` returned a plain list where the empty value meant both
"this caller was never restricted" and "this caller's grants resolved to
nothing". Downstream read either as allow-all, so a key restricted to one
agent inside a team restricted to another reached every agent on the proxy,
and an access group that resolved to no agents did the same.
Replace it with `resolve_agent_access`, returning a tagged
UnrestrictedAgentAccess | RestrictedAgentAccess. Only a caller with no grant
anywhere is unrestricted; an empty restricted set denies. Access group lookup
failures now propagate to the key/team resolvers so a DB error still fails
open exactly as before, while a group that genuinely resolves to nothing
denies.
* style(proxy): drop redundant comments from the agent access match
* fix(proxy): derive config agent ids from agent_name so grants survive secret rotation
Config-defined A2A agents were identified by a sha256 of the whole resolved
config entry, secrets included, so rotating an os.environ secret re-minted the
agent_id on restart and orphaned every object_permission.agents grant while
grant-less keys kept access (LIT-5144). The id now hashes only agent_name, and
the old full-entry hash is kept as a legacy alias: permission checks,
GET /v1/agents filtering, spend and key attachment, and public_agent_groups all
normalize legacy ids so pre-upgrade grants keep working
* fix(proxy): persist stable agent ids into stored grants at startup
The runtime alias only translates a legacy grant while the current config
still hashes to it, so a secret rotation after upgrading would orphan the
grant, and an orphaned grant intersecting a stable team grant collapses to
an empty list that downstream reads as allow-all. Rewriting the stored ids
once at boot removes both. This cannot be a SQL migration because only the
running proxy can recompute the legacy hash from resolved config secrets
* fix(proxy): make the grant id migration a compare-and-swap
A grant edited between the migration's read and write kept the stale
snapshot. The update now predicates on the agents array read at scan time
via update_many, so a concurrently modified row is skipped and the runtime
alias covers it until the next boot retries
* fix(proxy): retry the grant id migration and stay within the LIT002 ceiling
The one-shot startup task now retries up to three times with a short delay
so a transient DB error at boot cannot leave a legacy grant unmigrated
until an operator's next restart is the rotation itself. The new list
constructions in the migration and the alias-expanded agent id lookups are
tuples now, keeping the branch under the mutable-collection budget
* fix(proxy): count compare-and-swap misses in the grant id migration
migrate_legacy_grant_ids now returns rewritten and missed counts from the
update_many results instead of reporting scanned rows as migrated, and the
startup task retries while any rows remain unmigrated, not just on errors
* fix(lint): clear basedpyright budget breaches in agent id aliasing
The base branch ratcheted the same limits in 28a277e9, so the conflicting
files were reset to base and the ratchet re-run against the new merge-base
rather than resolved by hand. Each limit is now the base value minus this
branch's own delta, so both ratchets survive: basedpyright -653 across 48
rules, strict ruff -80, LIT -85.
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree
