* perf(ci): distribute unit cases and reuse security sweep connections
* revert(ci): reject first speed trial after new failures
* perf(ci): avoid repeated test discovery and parsing
* perf(ci): isolate parallel security workers
* revert(ci): restore measured security execution isolation
* perf(tests): reuse source scans and deduplicate cache cleanup
* perf(ci): restore parallel security with safe log batching
* test(ci): isolate fixture state and schema comparisons
* test(ci): make schema and scheduler fixtures deterministic
* test(ci): drain queued logging before fixture teardown
* test(ci): preserve logging delivery without idle worker restarts
* test(ci): keep guardrail sinks available during parallel bursts
* test(ci): isolate full proxy lifecycle from gateway imports
* test(ci): preserve first failure responses and sweep evidence
* test(ci): restore documentation URL settings between cases
* test(ci): isolate open MCP tools in private gateways
* test(ci): group private MCP cases and handle exited workers
* test(ci): retain proxy exits and MCP failure details
* test(ci): repair merge interactions with main
Key the schema shard's full clone on artifact-name, since main removed the
unit-flag input. Give each owned Redis its own log file again. Hand the
realtime transcription tests main added a logging object whose dispatch
handlers are awaitable, so the per-test logging drain no longer waits on a
MagicMock task forever
* perf(ci): shard owned integration cases and close lifecycle leaks
* fix(ci): qualify shard execution and harden interrupted cleanup
* fix(ci): bound worker supervision and own fault traffic
* perf(ci): reuse unparametrized fixtures across bounded case groups
* fix(ci): preserve boot fault allowances and complete exit records
* test(ci): tolerate startup retries and retain readiness details
* perf(ci): balance integration shards by recorded duration
* fix(ci): load migration database helpers only for workers
* test: move routing, caching and callback tests out of local_testing
Moves the 37 routing, caching and callback files in tests/local_testing into mirrored tests/unit paths and tests/integration sdk and security groups per the migration manifest, and deletes the 8 rows it marks for deletion
* test: cover least-busy routing and tighten migrated router, cache and pass-through tests
Add router-level least-busy tests for the three legacy rows that had no
destination, move completion fallback and model alias tests to their mirror
files, move router batch tests next to the router, and strengthen tests
whose mutants survived (specific deployment, batch pair grouping, custom
cache key, router cache ttl). Add a unit test for configured pass-through
Authorization forwarding.
* test: tighten migrated callback, proxy config and cache ttl tests
Assert the full StandardLoggingPayload key set and the wire user on callback
tests, add an async embedding failure callback test, drive the prometheus
redis failure through a real RedisCache, use the real salt key in proxy
config tests, pin the redis ttl value and rename fallback test helpers
* test: fix semantic cache assertions and order-dependent router and callback tests
Count semantic cache embeddings on the route, drain async cache writes before
the second lookup, echo the requested model from the callback provider mock so
response_cost does not depend on model_cost entries other tests register, and
move the two-key authorization assertion to the auth failure test it belongs to
* test: cover model_list fan-out and the usage-based routing tpm limit lost in the move
* test: wait for the streamed cache entry in redis and drop deleted files from the hosted mock guard
* test: restore sync callback, audio redaction and langfuse export coverage, keep redis-compat filter on the moved file
* ci: run the remaining router-keyword legacy tests on one node
* test: refuse the redis write in-process instead of resolving an invalid host
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(ci): keep the security sweep off the observed ROI GitHub route and give Lens integration tests a release identity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ci): expect the credential JWKS export to 404 for non-federation credentials in the security sweep
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ci): repair stale request fakes, spend-log golden, auto-router labels, and Interactions spec lookups
Request fakes now carry the scope a real Starlette request has, the GCS pub/sub
spend-log golden gains the agent identity keys from #43722, the auto-router
session tests follow the baseline_models contract from #43348, and the
Interactions spec checks resolve the create body and resource paths from the
live spec instead of hardcoded names
* test(ci): move retired OpenAI text-completion fixtures to live vehicles
OpenAI still serves native /v1/completions on the gpt-5.4 family, so the
single-prompt cases move to text-completion-openai/gpt-5.4-nano. Multi-prompt
batches and echo with logprobs now 500 on every OpenAI model, so those cases
keep the same text-completion-openai transport pointed at Fireworks, which
documents both. The optional-params test asserts the request body actually
sent instead of a success callback whose assertions were swallowed
* test(ci): use a serverless Fireworks model for the text-completion batch and echo cases
gpt-oss-20b is on-demand only on Fireworks, so the CI key got 404 model not
deployed; glm-5p3-flash is listed as serverless
* test(ci): skip the ROI calculator repository listing in the security route sweep
GET /roi-calculator/repositories (#43669) lists repositories from the configured
GitHub API, api.github.com by default, so the S2 sweep's GET of every route made
the owned proxy reach an external host and failed the egress check in 31
integration-security tests. It joins /get/latest_release_info in the deny list
* test(integration): credential canary suite harness
Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix.
* test(integration): widen canary route sweep and harden the rig
Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy.
* test(integration): descend into any decoded value that can still hold an encoded canary
* test(integration): bound canary decoding by depth and decoded bytes
* test(integration): scope log-table and spend-log reads to the scenario window
* test(integration): sweep spend-log rows in the scenario date window
* test(integration): keep spend-log date window summarized
* test(integration): sweep proxy logs, metrics, a gzip Datadog intake and the Logs drawer for credential canaries
* test(e2e): treat an unset prompt-storage setting as unset and restore it
* test(integration): name the Datadog sink slot G1d
* test(e2e): search the Logs page for base64 forms of the deployment key
* test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot
* test(integration): pass the resolved deployment id to the Datadog route sweep
* test(integration): expect 404 from the caller-scoped team membership route
* test(integration): use the rig's own master key and expect 404 from submission lookups
* test(integration): check the overridden rig key without assuming the default key is unknown
* test(integration): credential canary suite harness
Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix.
* test(integration): widen canary route sweep and harden the rig
Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy.
* test(integration): descend into any decoded value that can still hold an encoded canary
* test(integration): bound canary decoding by depth and decoded bytes
* test(integration): scope log-table and spend-log reads to the scenario window
* test(integration): sweep spend-log rows in the scenario date window
* test(integration): keep spend-log date window summarized
* test(integration): request-path credential canary slots D1-D4
* test(integration): read the Logs drawer and spend-log filter for failed request rows
* test(integration): check the marker in each failed row's spend-log filter; run header slots on chat-family routes
* test(integration): run D2 on embeddings again; only the client-header slot runs on chat-family routes
* test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot
* test(integration): pass the slot deployment's model_info id to the route sweep
* test(integration): expect 404 from the caller-scoped team membership route
* test(integration): use the rig's own master key and expect 404 from submission lookups
* test(integration): check the overridden rig key without assuming the default key is unknown
* test(integration): credential canary suite harness
Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix.
* test(integration): widen canary route sweep and harden the rig
Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy.
* test(integration): descend into any decoded value that can still hold an encoded canary
* test(integration): bound canary decoding by depth and decoded bytes
* test(integration): scope log-table and spend-log reads to the scenario window
* test(integration): sweep spend-log rows in the scenario date window
* test(integration): keep spend-log date window summarized
* test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot
* test(integration): expect 404 from the caller-scoped team membership route
* test(integration): use the rig's own master key and expect 404 from submission lookups
* test(integration): check the overridden rig key without assuming the default key is unknown
* test(integration): callback credential canary slots C1-C3 and D5
Team callback, team callback_settings, config default_team_settings and key metadata.logging Langfuse secrets, a team Datadog dd_api_key, and request-body Langfuse keys (allow_client_side_credentials) must reach only their sink. Each scenario checks its sink received the canary as auth and that the marker is visible at the stored body, the Logs drawer route and the sink. Adds a unit test that the stored request body snapshot carries no callback parameter.
* test(integration): give the callback sink waits a wider bound
* test(integration): sweep provider requests for callback credentials
* test(integration): credential canary suite harness
Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix.
* test(integration): widen canary route sweep and harden the rig
Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy.
* test(integration): descend into any decoded value that can still hold an encoded canary
* test(integration): bound canary decoding by depth and decoded bytes
* test(integration): scope log-table and spend-log reads to the scenario window
* test(integration): sweep spend-log rows in the scenario date window
* test(integration): keep spend-log date window summarized
* test(integration): stored-config credential canary slots
Add canary slots for credentials the proxy holds in its env, config or
database: virtual key raw value, master key, deployment api_key via
/model/new, named credentials, AWS secret key, Vertex service-account
JSON and its minted token, team model_config credential overrides,
config guardrail api_key, and sink credentials from env.
* test(integration): resolve the config guardrail id and require detail routes to return 200
* test(integration): drop repeated timeout comments
* test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot
* test(integration): expect 404 from the caller-scoped team membership route
* test(integration): use the rig's own master key and expect 404 from submission lookups
* test(integration): check the overridden rig key without assuming the default key is unknown
* test(integration): sweep config-deployment routes with the real model id and use the rig admin for the master-key slot
* test(integration): mark the configure-hook config edits as intended
* test(integration): drop suppression markers that suppress nothing
* Use claude-haiku-4-5 for the Bedrock stored-credential test model
* test(integration): credential canary suite harness
Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix.
* test(integration): widen canary route sweep and harden the rig
Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy.
* test(integration): descend into any decoded value that can still hold an encoded canary
* test(integration): bound canary decoding by depth and decoded bytes
* test(integration): scope log-table and spend-log reads to the scenario window
* test(integration): sweep spend-log rows in the scenario date window
* test(integration): keep spend-log date window summarized
* test(integration): credential canary slots for MCP and pass-through credentials
Adds slots F1 (MCP static auth), F2 (per-user MCP OAuth token), F2E (per-user
MCP env var), F3 (x-mcp client auth header), H1 (pass-through credential header),
H2 (vector store api_key) and H2S (search tool api_key) to the credential canary
suite. The OAuth double gains an optional mint hook so a test can choose the
issued access token.
* test(integration): wait for MCP spend rows by call type
* test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot
* test(integration): canary MCP and pass-through slots pass resolved ids
* test(integration): expect 404 from the caller-scoped team membership route
* test(integration): use the rig's own master key and expect 404 from submission lookups
* test(integration): check the overridden rig key without assuming the default key is unknown
* test(integration): credential canary suite harness
Adds tests/integration/security with canary generation and search, sweeps over the database, GET routes, client responses, sink doubles and Redis, an owned proxy rig, a sweep sensitivity self-test and the config deployment api_key slot. Registers the security group in run.py, the manifest and the CircleCI integration matrix.
* test(integration): widen canary route sweep and harden the rig
Enumerate lazily registered feature routers, call parameterized routes with placeholder ids, fail on routes that return no response, skip provider pass-through routes, add an explicit admin-only route allowance, let the sink double use a configurable token, inflate gzip members anywhere in a blob, sweep Redis before the route walk, and trap outbound connections from the owned proxy.
* test(integration): descend into any decoded value that can still hold an encoded canary
* test(integration): bound canary decoding by depth and decoded bytes
* test(integration): scope log-table and spend-log reads to the scenario window
* test(integration): sweep spend-log rows in the scenario date window
* test(integration): keep spend-log date window summarized
* test(integration): resolve deployment ids, scope paginated log lists, key allowances by slot
* test(integration): expect 404 from the caller-scoped team membership route
* test(integration): use the rig's own master key and expect 404 from submission lookups
* test(integration): check the overridden rig key without assuming the default key is unknown
