* ci: cut CircleCI wall time without loosening test isolation
* fix(ci): parse integration split files that follow --results
The CircleCI machine image ships Python 3.12.2, whose argparse leaves the
files positional empty when it follows an option and another positional, so
every extensions node exited with 'unrecognized arguments'. Reproduced on
3.12.2; parse_intermixed_args selects the files on 3.12.2, 3.12.13 and 3.13
* test(ci): resolve command references in the Rust toolchain guard
The Windows rustup install moved into the install_windows_toolchain command,
which the guard only recognized for install_rust. It now accepts any command
that installs a pinned rustup and reads the Windows toolchain pin from it
* ci: cache the Windows release cargo build from main
windows_release_wheel rebuilt every dependency with fat LTO on each run. It now
restores the release target and cargo registry saved by main's scheduled run,
drops the workspace crates' fingerprints so they always rebuild from the
checked-out source, and still runs the full LTO link
* ci: run the Windows release wheel build on windows.xlarge
The fat-LTO release build is the slowest job in the pipeline; more cores
speed up the dependency compile ahead of the final link
* ci: skip the Windows fingerprint cleanup when the cargo cache missed
On a cold cache the release fingerprint directory does not exist, and the
CircleCI PowerShell wrapper failed the step on the suppressed not-found error
* fix(ci): stop five stale or flaky CI reds and retry CyberArk policy-load conflicts
The Langfuse redaction unit test exports to a local OTLP capture instead of
polling Langfuse Cloud through a recorded lookup. The passthrough worker-kill
test only requires spend rows for requests the surviving worker served. The
spend-routes sweep treats the intentional /spend/capture_rate 503 as expected.
CyberArk retries a 409 policy load in Python, Rust and the e2e Conjur helper
instead of reading it as "variable exists". The integration egress guard now
matches the script's own cgroup, so it no longer blocks the CircleCI agent,
which runs as the same user.
* fix(ci): keep the policy-load backoff typed as float
* fix(ci): retry CyberArk policy loads without blocking the event loop and tighten the worker-kill and Langfuse tests
* fix(secrets): load CyberArk policy one request at a time per manager
* test(secrets): pin that non-conflict CyberArk policy failures are not retried
* test(unit): run tests/unit with only an allowlisted host environment
CircleCI's unit job inherits every project env var, so real provider keys,
REDIS_HOST, DATABASE_URL and AWS or Azure credentials reached tests that
assume none are set. Locally, litellm's import-time load_dotenv did the same
from any .env up the tree. The unit conftest now drops every variable outside
a small allowlist and disables dotenv before litellm is imported.
* test(e2e): name a failed search and the stuck batch status instead of misattributing them
The websearch session test read an empty web_search_tool_result_error block as a
successful search, so a failing search tool surfaced as a session billing bug.
The batch cancellation timeout now reports the last status the proxy returned.
* fix(ci): scrub the host environment per unit test instead of for the whole pytest process
GHA shards run tests/unit next to other suites in one process, so the import-time
scrub deleted MCP_TEST_PEER_PYTHON before tests/mcp_tests read it and the MCP
upstream fell back to the SDK2 interpreter. The two websearch tests that called
OpenAI and Perplexity live are removed: tests/unit no longer sees their keys.
* fix(ci): scrub only the host variables present before litellm is imported
The per-test scrub also deleted TIKTOKEN_CACHE_DIR, which litellm sets at import to
its bundled encodings, so tokenizer paths tried to download them and hit the
socket guard. The prisma setup test now passes its own database URL instead of
reading one another test leaked into the process environment.
* fix(ci): stop the order-dependent unit reds and settle logging tasks on their own queue
LoggingWorker marked a task done on whichever queue was current when the callback
finished, so a callback that outlived an event-loop change raised "task_done()
called too many times" or undercounted the new loop's queue. It now settles the
queue the task came from.
The rest are test isolation fixes for failures that only appeared when another
file ran first on the same xdist worker: a replaced user_api_key_cache, breaker
metrics unregistered by prometheus tests, semantic_router's health-check filter on
uvicorn.access, logging tasks carried over from bedrock tests, a Router-written
model_cost entry, and a stray post captured by the langflow test. The token
counter check now asserts bounded chunking instead of wall-clock time.
* test(e2e/ui): wait for the logout redirect before visiting a protected page
Logout revokes the session server-side before clearing cookies and navigating, so an immediate page.goto either ran with the cookie still set or was aborted by the logout redirect (net::ERR_ABORTED).
* test(unit): restore the prometheus metrics config per test and settle logs carried from earlier tests in the a2a cost tests
* test(router): pin the router clock in the usage counter tests so a minute rollover cannot empty the read
* test(e2e/ui): wait for logout to clear the token cookie instead of for a login redirect
* test(integration/mcp): answer the model-info probe another test's proxy sends to the model double
* test(integration): add read-replica routing harness
* refactor(integration): hoist the maintenance url imports
* fix(integration): keep per-test databases and the witness sequence readable under replica roles
* fix(integration): opt bespoke database and pool tests out of the injected read replica
* test(integration): commit recorded replica routing expectations
* fix(integration): judge routing by role containment so shrinking role sets do not fail
* fix(integration): run the pool-limit shutdown choreography on the superuser database url
* ci(integration): add the mcp group to the replica matrix
* fix(integration): judge routing by exact role sets with a named either-role allowlist
* test(integration): drop containment-era routing expectations for re-recording
* chore(integration): drop docstrings from the replica harness scripts
* docs(integration): describe exact routing matching and the either-role list
* test(integration): record exact replica routing expectations
* test(integration): allow the SELECT 1 health probe on either role
* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run
* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test
* test(integration): make the sorted-role serialization test deterministic in-process
* test(integration): swap all product code in parity runs and pin role gains
* ci(integration): force tracked-file removal before parity checkout
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test(integration): drop the contracts.json manifest and the covers requirement
Groups live as a GROUPS literal in run.py, the browser expectations move next to the
browser tests, and the runner fails only on pytest failure, collection errors or a
selected file that collects zero tests. The covers marker stays registered for the
existing tests but is no longer checked. The mcp directory gets its own group
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): run mcp as its own shard with xdist and a peer proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): INTEGRATION_COVERAGE=1 runs the proxy under coverage for the MCP modules
The mcp shard sets it. The proxy and its peer start under coverage run in parallel mode,
get SIGTERM after the tests so coverage flushes, and the combined text and HTML reports
land in the suite results that CircleCI already stores as artifacts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): let the test proxy flush coverage when uvicorn re-raises SIGTERM
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add SSE, stdio, scripted, OpenAPI and OAuth 2.1 MCP peer doubles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add MCP transport and access-control matrices
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add MCP credential and OAuth flow coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add MCP LLM endpoint, accounting, guardrail, resilience and lifecycle coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stop the same-URL grant test from counting a late initialize as a leaked call and satisfy the test-tree lint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the REST denied-server listing is refused or empty
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin the REST denied-server listing to 403 access_denied
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vector_stores): keep config-defined vector stores listed and read-only
Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show vector store source and read-only state for config-defined stores
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show a visible read-only hint in the config vector store actions menu
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.
The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.
The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
