Commit graph

15 commits

Author SHA1 Message Date
devin-ai-integration[bot]
3fb6f8740b
test(integration): add read-replica routing harness to the CircleCI integration suite (#42692)
* test(integration): add read-replica routing harness

* refactor(integration): hoist the maintenance url imports

* fix(integration): keep per-test databases and the witness sequence readable under replica roles

* fix(integration): opt bespoke database and pool tests out of the injected read replica

* test(integration): commit recorded replica routing expectations

* fix(integration): judge routing by role containment so shrinking role sets do not fail

* fix(integration): run the pool-limit shutdown choreography on the superuser database url

* ci(integration): add the mcp group to the replica matrix

* fix(integration): judge routing by exact role sets with a named either-role allowlist

* test(integration): drop containment-era routing expectations for re-recording

* chore(integration): drop docstrings from the replica harness scripts

* docs(integration): describe exact routing matching and the either-role list

* test(integration): record exact replica routing expectations

* test(integration): allow the SELECT 1 health probe on either role

* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run

* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test

* test(integration): make the sorted-role serialization test deterministic in-process

* test(integration): swap all product code in parity runs and pin role gains

* ci(integration): force tracked-file removal before parity checkout

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-09-24 00:25:03 -07:00
devin-ai-integration[bot]
e26a6450c8
test(integration): add MCP gateway coverage wave 1 with a dedicated mcp shard and proxy coverage artifact (#42711)
* test(integration): drop the contracts.json manifest and the covers requirement

Groups live as a GROUPS literal in run.py, the browser expectations move next to the
browser tests, and the runner fails only on pytest failure, collection errors or a
selected file that collects zero tests. The covers marker stays registered for the
existing tests but is no longer checked. The mcp directory gets its own group

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(integration): run mcp as its own shard with xdist and a peer proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(integration): INTEGRATION_COVERAGE=1 runs the proxy under coverage for the MCP modules

The mcp shard sets it. The proxy and its peer start under coverage run in parallel mode,
get SIGTERM after the tests so coverage flushes, and the combined text and HTML reports
land in the suite results that CircleCI already stores as artifacts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(integration): let the test proxy flush coverage when uvicorn re-raises SIGTERM

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add SSE, stdio, scripted, OpenAPI and OAuth 2.1 MCP peer doubles

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add MCP transport and access-control matrices

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add MCP credential and OAuth flow coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add MCP LLM endpoint, accounting, guardrail, resilience and lifecycle coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): stop the same-URL grant test from counting a late initialize as a leaked call and satisfy the test-tree lint

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert the REST denied-server listing is refused or empty

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin the REST denied-server listing to 403 access_denied

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 07:48:46 -07:00
devin-ai-integration[bot]
24a05b29a6
test(integration): allow skipped nodes and drop the shard cap (#42687)
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 23:08:18 -07:00
devin-ai-integration[bot]
f44052d87b
fix(vector_stores): keep config-defined vector stores listed and read-only (#42574)
* fix(vector_stores): keep config-defined vector stores listed and read-only

Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show vector store source and read-only state for config-defined stores

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show a visible read-only hint in the config vector store actions menu

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 04:02:16 +00:00
kerry
110d4c2ad1 test(integration): add passthrough cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 22:05:57 +00:00
kerry-berri
6718fd67fb
Merge pull request #41328 from BerriAI/litellm_e2e_cost_calculation_scripted_provider
test(integration): literal request/response cost tracking suite on a test-owned cost map
2026-09-19 09:40:04 -07:00
kerry
e52eea84e6 test(integration): serve scripted wires from the shared upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:50:26 +00:00
kerry
f836bb481d test(integration): keep cost diagnostics and widen shard timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:19:11 +00:00
kerry
69f9106759 test(integration): move scripted-provider cost suite into cost shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:16:12 +00:00
mateo-berri
7cfec730d5 fix(proxy): refuse config-owned keys on POST /config/update
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.

The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.

The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
2026-09-18 12:42:08 -07:00
Yuneng Jiang
c00f1b4a5c
test: add extension and browser integration contracts
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
2026-09-16 13:15:53 -07:00
Yuneng Jiang
cfda6dea3e
test: cover persisted updates and warmed authorization policies 2026-09-14 04:20:06 -07:00
Yuneng Jiang
16d7e695c0
test: stop proxy before cleaning up its query engine 2026-09-14 04:04:21 -07:00
Yuneng Jiang
d7c13625e9
test: qualify integration cleanup and replay invocation 2026-09-14 03:48:23 -07:00
Yuneng Jiang
92e0b72e2d
test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00