Commit graph

4 commits

Author SHA1 Message Date
yuneng-jiang
44d99475a5
test(integration): stop GET /__observations from draining the upstream log (#47256)
* test(integration): stop GET /__observations from draining the upstream log

GET /__observations used to empty the queue, so a second read in a test
saw nothing and a "never reached upstream" check could pass whether or not
the request was forwarded. GET now returns every request since the last
reset and DELETE /__observations is the reset. An autouse fixture resets
the worker upstream before each test, which also covers tests that read
without the gateway fixture.

Discarded reads used as resets become DELETEs, helpers that accumulated
drained reads now read one snapshot, and tests with several phases reset
between them. The three vacuous negative checks (rag allowlist, guardrail
opt-out chaos, xAI batch output) now assert the absence on the same read
that shows a request which did reach the upstream.

* style(integration): space the observations unit test like its neighbours
2026-10-18 09:58:05 +00:00
devin-ai-integration[bot]
859529ccbc
fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939)
* fix(realtime): keep turn_detection.type when a transcript guardrail disables auto-response

* test(realtime): type the GA session.update helper and mark its locals Final

* test(realtime): expect typed turn_detection on the voice guardrail session updates

* test(realtime): accept either gate path for the client's first voice session.update

* fix(gemini): drop VAD tuning when the guardrail turns auto-response off

* fix(gemini): check create_response without a redundant isinstance

* fix(vertex_ai): keep the client's VAD tuning on Vertex when auto-response is off

* fix(realtime): keep the transcript-guardrail gate typeless on xAI so it does not auto-answer

* fix(realtime): replace any client turn_detection with the untyped gate on xAI

xAI answers on its own under a typed server_vad even with create_response off,
so a client's typed turn_detection must not reach xAI while the transcript
guardrail gates the turn.

* fix(realtime): keep a client's early turn_detection and hide the guardrail cancel's error

A session.update sent before session.created reached OpenAI ahead of the proxy's own gate, so the gate replaced the client's turn_detection. The proxy now re-sends the client's gated turn_detection after the gate when the two differ.

The guardrail's response.cancel now carries a litellm_guardrail_ event_id, and the error OpenAI returns for it (response_cancel_not_active when nothing is answering) no longer reaches the client.

* fix(realtime): send the guardrail gate before a client's early session.update and let Muse and Chirp fall back to server_vad under drop_params

* fix(realtime): hide only the cancel errors that echo the guardrail's own response.cancel ids

Track the event ids of the guardrail cancels this session sent and drop a backend error only when its
error.event_id is one of them, so a client's own event id with the same prefix still gets its error.
Cover the scripted response.cancel reply in the integration upstream, update the stale list-mode test
to the typed gate, and mark the xAI cancel error cell as an expected failure, since xAI replies with
its own event id.

* test(integration support): deliver the disconnect a real server sends after the client goes away

The scripted upstream now keeps reading frames after a reply hits a closed socket, so the unit test's
fake ASGI receive has to end with the websocket.disconnect message a real server delivers.

* fix(realtime): delete the blocked transcript item before the guardrail's refusal turn

OpenAI GA answered the blocked question in some refusal turns because the
blocked user item stayed in the conversation. The raw OpenAI-protocol path
now sends conversation.item.delete for it right after the cancel. Provider
config backends have no such event, so that path no longer passes an item id.

Also bound the guardrail integration file by the shared graceful-stop
timeout and make the xAI cancel-error xfail require an AssertionError.

* fix(realtime): keep the blocked item on xAI, which cannot delete an audio input item

* test(realtime): assert the xAI voice block sends no conversation.item.delete upstream

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-16 15:24:40 +00:00
joshua-berri
440744f967
feat(mcp): enable verified stateless protocol capabilities (#46264)
* feat(mcp): advertise verified stateless capabilities

* fix(mcp): honor rate limits during capability discovery

* test(mcp): cover protocol interactions and scoped enablement rollback

* fix(mcp): preserve legacy capabilities and bound discovery probes

* fix(mcp): keep discovery results single-assignment

* ci(mcp): distribute acceptance cases across existing workers

* ci: balance MCP integration cases across idle workers

* ci: isolate MCP interaction acceptance workload

* test(mcp): acknowledge fixture notifications after dispatch

* test(mcp): drain session dispatch before fixture deletion

* test(mcp): preserve deletion of streaming fixture requests

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-10-15 23:18:18 +00:00
devin-ai-integration[bot]
3090f4b23e
fix(ci): isolate integration workers and repair test fixtures (#45863)
* test(integration): wait for customer spend rows and use owned OpenRouter cost

* ci: isolate integration workers and clean up resources after crashes

* docs: describe isolated extensions workers and shard census

* fix(ci): restore integration service and fixture guarantees

* fix(integration): preserve replay IDs and native route fixtures

* fix(tests): scope integration harness imports to each unit case

* fix(ci): preserve fixtures and wait for owned workers

* fix(ci): isolate nested runs and telemetry cases

* fix(ci): make worker readiness and fault tests deterministic

* chore(ci): drop MCP fixture repair already covered by main

* chore(ci): keep the local catalog fix from main

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-11 01:56:02 -07:00