* ci: warn on SQL IN lists with no written bound
Postgres caps a prepared statement at 32,767 bind parameters and a
membership filter binds one per value, so an IN list built from table
data breaks once the table outgrows the cap. That is how the budget reset
job froze every due budget (LIT-7535, #40564).
check_unbounded_in_lists.py reports every Prisma "in" / "not_in" filter
whose value has no fixed size and every raw SQL literal that splices a
list in after "IN (", unless the line carries "# bounded-ok: <reason>".
It only warns for now: the output is the inventory for RCA action item
AI-1, and it exits 0.
* ci: decide a constant IN list by its module binding, not its casing
An ALL_CAPS name imported or filled at runtime is as unbounded as any
other, so a name now passes only when the module binds it once to a
value of fixed size. Adds Final to the locals a loop does not forbid.
* ci: only a frozen module value makes an IN list constant
A module list bound once could still grow through append or extend, so
a name now counts as fixed only when it is bound to a tuple, frozenset
or constant. Trims the module docstring to what a reader needs.
* ci: chunk Prisma IN lists with a shared helper and fail on new unbounded ones
Add litellm.repositories.bounded_in: find_many_in, count_in, update_many_in
and delete_many_in split a deduplicated value list into 5,000-value chunks,
AND each chunk with the caller's where, run them in order (a transaction
handle works) and combine the results. Writes take a required atomicity
argument, and a where that already filters the chunked field is refused.
check_unbounded_in_lists.py now fails CI on any finding missing from
unbounded_in_baseline.txt and on any stale baseline entry, so the baseline
only shrinks. Entries are keyed by path, enclosing scope, kind, field and
occurrence, not line numbers. The helper module is exempt, a constant
spread into a frozen tuple counts as fixed, and messages point at the
helper for "in" and at an array parameter for "not_in" and raw SQL.
A real-Postgres integration test shows a raw 40,000-value filter rejected
for too many bind variables while the helpers handle it.
* refactor: rename bounded_in to chunked_in and let callers pick a chunk size
The helper module is litellm.repositories.chunked_in, and its unit and
integration tests, the checker's exemption path and its finding messages
follow the new name. The `# bounded-ok` marker is unchanged.
find_many_in, count_in, update_many_in and delete_many_in take a
keyword-only chunk_size, defaulting to IN_LIST_CHUNK_SIZE (5,000). A value
below 1 or above MAX_IN_LIST_CHUNK_SIZE (30,000) raises ValueError before
any query, which leaves the rest of the filter headroom under Postgres's
32,767 bind-parameter cap.
* refactor: flatten chunked_in's stacked comprehensions with chain.from_iterable
LIT014 (#42650) caps a comprehension at one for and one if clause. The four nested walks in the helper now chain their iterables instead, with the same order and results.
* refactor: recover user details with find_many_in, sending chunks as lists
_details_for_user_ids reads users through find_many_in instead of a raw
"in" filter, so its lookup stays under the bind-parameter cap for any
number of recovered keys. Up to 5,000 ids it still sends one find_many
with the same where dict, and a PrismaError from any chunk is still
logged and treated as no details.
The helper now sends each chunk as a list, so a chunked filter equals
the dict a hand-written call would send and a migrated call site's
existing assertions keep passing.
The site's baseline entry is gone.
* ci: skip functional TypedDict field maps in the unbounded IN list check
The dict passed as the field map of TypedDict("Name", {...}), or as its fields= keyword, names fields: an "in" or "notIn" key there is a type, not a filter. Only that dict is skipped, for TypedDict, typing.TypedDict and typing_extensions.TypedDict; a filter nested in a field value or passed to any other call is still reported. The two types/proxy/management_endpoints/team_endpoints.py entries leave the baseline, which is now 156.
* fix: refuse an update_many_in whose data writes the chunked field
Chunks run one after another, so an update that sets the chunked field can move a row into a later chunk, which updates it again and counts it twice: values ["old", "new"] with chunk_size=1 and data={"id": "new"} does exactly that. update_many_in now raises ChunkedFieldWriteError before any query when data has the chunked field as a top-level key, in any form, including Prisma operators such as {"set": ...}.
* docs: cut the unbounded IN list checker's docstring to what it flags and how to clear it
It now says what is reported, the three ways to clear a finding, and how the baseline and --update-baseline work, in 11 lines. The per-shape detail lives in the tests.
* ci: key an unbounded IN list finding by its filtered expression too
A baseline key of path, scope, kind, field and occurrence let a PR delete
a baselined filter and add a different unbounded one on the same field in
the same function, and the new one took over the old key. The key now
also carries the filtered expression's source, whitespace-normalized
(the Prisma value, or a raw-SQL `IN (...)` slot), so that swap reads as
one new and one stale entry and fails the run. The same expression
re-added in the same function is still the same finding.
Every baseline entry is rewritten in the new form; the 156 findings are
unchanged, and only occurrence indexes renumber where one field had
several different expressions.
* test(integration): add read-replica routing harness
* refactor(integration): hoist the maintenance url imports
* fix(integration): keep per-test databases and the witness sequence readable under replica roles
* fix(integration): opt bespoke database and pool tests out of the injected read replica
* test(integration): commit recorded replica routing expectations
* fix(integration): judge routing by role containment so shrinking role sets do not fail
* fix(integration): run the pool-limit shutdown choreography on the superuser database url
* ci(integration): add the mcp group to the replica matrix
* fix(integration): judge routing by exact role sets with a named either-role allowlist
* test(integration): drop containment-era routing expectations for re-recording
* chore(integration): drop docstrings from the replica harness scripts
* docs(integration): describe exact routing matching and the either-role list
* test(integration): record exact replica routing expectations
* test(integration): allow the SELECT 1 health probe on either role
* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run
* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test
* test(integration): make the sorted-role serialization test deterministic in-process
* test(integration): swap all product code in parity runs and pin role gains
* ci(integration): force tracked-file removal before parity checkout
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test(integration): streamed Bedrock Messages usage cost equals the recorded spend (Pylon #6667)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): echoed cost-map model info is not persisted as deployment overrides (Pylon #6844)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): reset sweep runs on one pod per tick while replicas share the lease (Pylon #6521)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock post-call guardrail scans streamed Anthropic Messages tool use without 500 (Pylon #6503)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): realtime cached audio tokens bill at the audio cache-read rate (Pylon #6704)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): legacy GET /spend/logs returns at most the 10000 most recent rows and flags truncation (Pylon #6752)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): itemize Responses API cache write tokens as cache creation cost (Pylon #6454)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): migration entrypoint deploys pending migrations before proxy startup (Pylon #6649)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): opted-in team keys stop at the owner's personal budget (Pylon #6641)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): guardrail information stays in the spend log when the caller sends metadata on /v1/messages (Pylon #6614)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): key model allowlist is enforced on Bedrock passthrough routes (Pylon #6419)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): JWT mapped key backfills a null user email from token claims (Pylon #6266)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): scheduled budget reset recovers from a transient DB transport failure (Pylon #6582)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): team key lists models granted through a team access group (Pylon #6044)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): JWT subject without team claim lands in the configured default team (Pylon #5895)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): end-user spend lands for a key without user_id when the auth cache is Redis (Pylon #6021)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stale-low redis counter still blocks team member over budget (Pylon #5824)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prompt-carrying spend rows are written in byte-bounded statements (Pylon #6083)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): logs UI session_total_spend sums every round of a multi-round session (Pylon #5928)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): config.yaml guardrails are served by the guardrail usage detail and overview (Pylon #5813)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): plain chat request skips the object permission lookup (Pylon #5965)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): register july accounting regression contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): isolate cost map override clear on owned proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): make reset lease claim and db relay refusal deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): poll pg_stat settle, bound unbanned relay refusals, clear reset lease on teardown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bound relay refusals so the budget sweep can reconnect
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
