* fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e
PUT /v1/mcp/toolset dumped its payload with exclude_none, so a field sent as
null looked exactly like one the caller left out and the stored value
survived. An admin could not clear a toolset's description: the save reported
success and the old text came straight back. It now dumps with exclude_unset,
so absent keeps and null clears, which is what PUT /v1/mcp/server already did.
A null tools list clears the selection to empty, and a null toolset_name is
ignored because a toolset always has a name.
Adds create, read, partial-update, clear and delete e2e coverage for MCP
servers and toolsets, with every read-back polled on every replica so an edit
that lands on one replica and not another fails the test, plus an enforcement
test proving a key granted a toolset lists exactly that toolset's tools
against the real Datadog upstream.
* fix(e2e): refuse a read-back that no replica serves
A read-back over an empty replica mapping satisfied every predicate and
returned as if it had converged, so it would have asserted nothing and
passed. No wiring can produce that today, since the replica list always
falls back to at least one URL, but a helper whose whole job is proving a
write reached every replica should not have a shape that passes vacuously.
* fix(mcp): keep a null tools list a no-op on toolset update
Treating a null tools list as a clear meant an existing client that sends
tools=null during a partial update, meaning "leave the selection alone",
silently lost every tool the toolset grants. That is a permission surface,
so the quiet version of it is the worst version.
A toolset always has a tool list, the same way it always has a name, so a
null on either is now a no-op. Emptying the selection is an explicit [],
which cannot be confused with a field the caller left out, and which is
what the dashboard already sends.
* fix(e2e): keep MCP admin routes on the data plane
/v1/mcp/* is a lazily mounted feature, so a gateway registers it on the first
matching request, which happens after the startup route trim that drops
management endpoints. Routing it to the control plane therefore sent every MCP
call to the one backend process: the new lifecycle read-backs proved a single
process rather than every replica, and mcp_client's await_registered barrier
waited on a registry that does not serve the tools/list call it guards, so the
existing MCP suites polled a gateway that had not synced yet until poll_timeout
Verified against a two-gateway split stack (backend on 4001, gateways on 4010
and 4011, one postgres): both gateways answer /v1/mcp/server and /v1/mcp/toolset,
and each served 6 server reads and 7 toolset reads over the run
* fix(e2e): grant the toolset by the tool's own name, not the wire name
tools/list serves a tool as <prefix><tool_name>, but a toolset grants by the
tool's own name: resolve_toolset_permissions reads toolset.tools[].tool_name
straight through, and the prefix is added on the way out. The test built the
toolset from the names tools/list reported, so the grant matched nothing, the
scoped key listed no tools, and await_tools ran out its whole poll_timeout
before failing
Measure the prefix off search_datadog_logs, whose own name is known, rather than
guessing it from the alias, since the proxy can be configured to prefix with a
short server id instead. The expectation compared against tools/list stays in
wire names; only what the toolset stores crosses back
* test(mcp): build immutable lifecycle updates and replica results
* test: validate opaque stream IDs and hide log-reader credentials
* test: isolate auto-router scenarios and clean partial setup
* test: honor Datadog search rate-limit reset headers
* test: share the Datadog read-back deadline across retries
* test: preserve captured MCP toolset update fields
* test(e2e/ui): cover key budget window, non-admin model scope edit, and key blocking
Three Playwright specs for the Virtual Keys flows customers hit most, each
reading its result back through /key/info and /v1/chat/completions rather
than trusting the toast:
- a monthly spend cap and reset window set through Edit Settings, surviving
a reload, with clearing the window leaving the cap in place
- a team member narrowing their own team key's models, and the proxy
refusing the model they dropped
- blocking a key from its detail page, then unblocking it
Each test owns the key it edits and deletes it on teardown, so retries and
--repeat-each never run out of fixtures.
* test(e2e/ui): tighten virtual key specs from review feedback
Replace the mutable suite-level key state with a Playwright fixture, so the
alias and token are never reassigned and cleanup stays tied to the test.
Assert /key/delete succeeded instead of discarding the response, so a failed
cleanup surfaces rather than leaving rows behind.
Drop the explanatory JSDoc the repo's comment policy disallows, keeping only
the one line explaining why Date.now() alone is not unique enough.
Type the master-key POST helper against a real guard instead of casting to
Record<string, any>.
Assert the unblocked key is served with a 200, not just the response text,
and that clearing the reset window also clears budget_reset_at.
* test(ui): assert the team response through Playwright
* test(e2e/ui): cover team-scoped model visibility, re-editing litellm params, and model health checks
Three Models and Endpoints flows had no end-to-end coverage, and all three
keep coming back as bug reports.
modelsByTeam walks an internal user through the Current team control and
asserts the table lists exactly what each team grants. It creates one
deployment that belongs to no team, proves that deployment is visible under
Personal, then proves it is absent under both seeded teams, so an empty
table cannot pass the same assertions.
editLitellmParams adds a temperature and a custom pair to a deployment,
saves, then re-edits the temperature and drops the custom pair. It checks
both update request bodies, polls the stored deployment until the new
temperature is there, reloads the page to confirm the second save is what
renders, and sends one chat completion to prove the deployment still serves.
modelHealthStatus runs the health check on a reachable deployment and on one
pointed at a dead port, asserts the healthy and unhealthy cells and the two
detail dialogs, and reloads to confirm both statuses are stored.
Every deployment these specs create carries a unique name and is deleted in
afterEach, including on the failure path.
* test(e2e/ui): find health rows across every page of the health table
The health table pages server-side at 50 rows with no search box, so on a
proxy carrying more deployments than that the two deployments the spec
creates can land on a later page and the lookup finds nothing.
Row lookups now walk the pages, using the table's own page indicator to
know when to advance and when to wrap back to the first page.
* test(e2e/ui): build the created deployment ids without mutating the array
* test(ui): scope model deployments to Playwright fixtures
The streaming pipeline step only takes a post-call hook on routes whose
translation assembles the streamed response (chat completions, Responses,
Messages). On /v1/completions, the Gemini streamGenerateContent route, and
A2A streams the pipeline is skipped with the merge-base warning and the hook
runs on its own afterwards, instead of getting a None response while the
header says the guardrail ran. A guardrail that overrides
async_post_call_streaming_iterator_hook next to its post-call hook keeps its
native per-chunk path rather than running buffered through the adapter
Reasoning items the Responses API returned for a /v1/messages turn were rebuilt
from their summary text on every replay, so the prompt the model saw changed
between turns and the prompt cache never matched. The bridge now asks for
reasoning.encrypted_content, carries it in the thinking signature (or as a
redacted_thinking block when there is no summary), and replays it verbatim as
the reasoning item's encrypted_content. Anthropic replay paths drop those
tagged blocks so a cross-model resume never forwards OpenAI bytes to Anthropic
Replace the thread semaphore around HuggingFace encodes with an anyio CapacityLimiter
applied at every offloaded count site through offload_token_count, so waiting counts no
longer hold slots in the shared 40-thread pool and inline counts on the event loop never
block on the bound. Rename TOKEN_COUNTER_MAX_CONCURRENT_HF_ENCODES to
TOKEN_COUNTER_MAX_CONCURRENT_COUNTS
get_optional_params_image_gen only forwarded the per-call drop_params flag
to provider configs, so litellm_settings drop_params: true never dropped
the n the MAI generations endpoint ignores. The MAI edits config also
advertised and forwarded size, which that endpoint ignores. Non-numeric
and non-positive n now surface as a 400 instead of a 500 or a pass-through.
Every test this PR adds now annotates its fixture and parametrize
parameters. The submit-time warning for a tag-matched deferred policy
names only the policies, since a wildcard attachment pattern would let
caller-provided tag text reach the log.
A guardrail that removes or adds a tool call on an ended stream used to be
silently ignored: every handler substitutes the original list on a count
mismatch and the executor skipped its observer once the translation could
deliver rewrites. The executor now tracks the count change on the observer
and releases the original chunks with the discard warning on every
translation, matching what the merge base did for any tool call rewrite
* test(mcp): exercise /mcp/proxy authorization against the real registry instead of patched manager methods
* fix(mcp): preserve proxy logging and authorization coverage
* test(mcp): respect the proxy FastAPI import boundary
The bridges derived prompt_cache_key as the first 64 chars of metadata.user_id.
Claude Code packs a JSON object into that field whose prefix is the per-install
device_id, so every session and subagent on one machine shared a single key,
and a plain end-user id pinned all of that user's conversations to one slot.
Parse the JSON and use session_id; send no key otherwise so the provider falls
back to its own prompt-prefix hashing. An explicit prompt_cache_key still wins.
Fixes#39145
* fix(otel v2): restore the Datadog auth span and the last-wins callback merge
Move @tracer.wrap() back onto user_api_key_auth so USE_DDTRACE=true emits the
auth span again, and let a failure entry's callback_vars take part in the
destination merge so the resolver picks the same account the runtime parser does
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel v2): drop docstrings from the two regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rerun proxy-infra after the flaky test_check_migration process-tree test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
