Commit graph

7 commits

Author SHA1 Message Date
devin-ai-integration[bot]
3fb6f8740b
test(integration): add read-replica routing harness to the CircleCI integration suite (#42692)
* test(integration): add read-replica routing harness

* refactor(integration): hoist the maintenance url imports

* fix(integration): keep per-test databases and the witness sequence readable under replica roles

* fix(integration): opt bespoke database and pool tests out of the injected read replica

* test(integration): commit recorded replica routing expectations

* fix(integration): judge routing by role containment so shrinking role sets do not fail

* fix(integration): run the pool-limit shutdown choreography on the superuser database url

* ci(integration): add the mcp group to the replica matrix

* fix(integration): judge routing by exact role sets with a named either-role allowlist

* test(integration): drop containment-era routing expectations for re-recording

* chore(integration): drop docstrings from the replica harness scripts

* docs(integration): describe exact routing matching and the either-role list

* test(integration): record exact replica routing expectations

* test(integration): allow the SELECT 1 health probe on either role

* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run

* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test

* test(integration): make the sorted-role serialization test deterministic in-process

* test(integration): swap all product code in parity runs and pin role gains

* ci(integration): force tracked-file removal before parity checkout

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-09-24 00:25:03 -07:00
devin-ai-integration[bot]
514bc181d6
test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694)
* test(integration): streamed Bedrock Messages usage cost equals the recorded spend (Pylon #6667)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): echoed cost-map model info is not persisted as deployment overrides (Pylon #6844)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): reset sweep runs on one pod per tick while replicas share the lease (Pylon #6521)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bedrock post-call guardrail scans streamed Anthropic Messages tool use without 500 (Pylon #6503)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): realtime cached audio tokens bill at the audio cache-read rate (Pylon #6704)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): legacy GET /spend/logs returns at most the 10000 most recent rows and flags truncation (Pylon #6752)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): itemize Responses API cache write tokens as cache creation cost (Pylon #6454)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): migration entrypoint deploys pending migrations before proxy startup (Pylon #6649)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): opted-in team keys stop at the owner's personal budget (Pylon #6641)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): guardrail information stays in the spend log when the caller sends metadata on /v1/messages (Pylon #6614)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): key model allowlist is enforced on Bedrock passthrough routes (Pylon #6419)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): JWT mapped key backfills a null user email from token claims (Pylon #6266)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): scheduled budget reset recovers from a transient DB transport failure (Pylon #6582)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): team key lists models granted through a team access group (Pylon #6044)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): JWT subject without team claim lands in the configured default team (Pylon #5895)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): end-user spend lands for a key without user_id when the auth cache is Redis (Pylon #6021)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): stale-low redis counter still blocks team member over budget (Pylon #5824)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prompt-carrying spend rows are written in byte-bounded statements (Pylon #6083)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): logs UI session_total_spend sums every round of a multi-round session (Pylon #5928)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): config.yaml guardrails are served by the guardrail usage detail and overview (Pylon #5813)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): plain chat request skips the object permission lookup (Pylon #5965)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): register july accounting regression contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): isolate cost map override clear on owned proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): make reset lease claim and db relay refusal deterministic

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): poll pg_stat settle, bound unbanned relay refusals, clear reset lease on teardown

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bound relay refusals so the budget sweep can reconnect

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 09:51:56 -07:00
Yuneng Jiang
c00f1b4a5c
test: add extension and browser integration contracts
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
2026-09-16 13:15:53 -07:00
Yuneng Jiang
dfac4e0a9f
Format database and accounting integration tests
Some checks failed
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
ai-gateway image / ai-gateway release image (push) Has been cancelled
2026-09-14 21:24:20 -07:00
Yuneng Jiang
94f2a4702b
test: remove redundant integration harness commentary 2026-09-14 09:40:21 -07:00
Yuneng Jiang
57825775b8
test: tighten reload coverage and avoid an artificial lock timing cutoff 2026-09-14 05:08:14 -07:00
Yuneng Jiang
682296ad68
test: verify database transactions and persisted accounting contracts 2026-09-14 05:00:32 -07:00