* fix(responses): decode JSON-string tool schemas before sending to the provider
A caller that hands a tool schema over already JSON-encoded reached the
Responses API with a string `parameters`, and the provider rejected the
request with a 400 naming the routed model instead of the offending tool.
Decode it at the one place every Responses request converges, and refuse
anything that is neither an object nor a string encoding one.
Collapses the duplicated input/tool sanitization block shared by the
request and compact-request builders into a single owner, so the decode
cannot be wired into one path and not the other.
* test(responses): pin null tool schemas as accepted, and type the parametrized cases
The Responses API serves `parameters: null` and an omitted schema alike, so
neither may raise. Pin both against a future tightening, annotate the
parametrized inputs, and trim the docstrings back to what the code does not
already say.
* fix(guardrails): skip streaming guardrail rounds that re-scan cleared output
Streaming guardrails scanned the finished answer twice at end of stream
whenever the chunk count landed on a multiple of the sampling rate, ran
sampled rounds whose payload was identical to the previous one, and on
/v1/messages could scan an empty text before the first content chunk.
Every redundant round is a paid guardrail provider call.
Each endpoint handler now exposes a scan key describing what a round
would hand to apply_guardrail (the text so far, plus tool calls once the
stream has ended), and the unified streaming hook skips a sampled or
end-of-stream round whose key equals the last scanned one or carries
nothing to scan yet. Rounds that carry tool calls are never skipped.
* test(guardrails): expect one end-of-stream scan when the terminal chunk is sampled
Update sampled cadence expectations and use tuple-backed scan state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sort the chat-tool keys once and number duplicates with groupby instead of
rescanning every preceding key per position, so the guardrail merge stays
O(n log n) on client-supplied tool lists. Drop the comment that restated the
unsupported-tool warning in the Responses-to-chat transformation.
Any pre_call guardrail on /v1/responses flattened Codex namespace tools
into ns__member functions and wrote the flattened list back to the
request, so the model called mcp__server__tool with no namespace and
Codex rejected the call as unsupported.
The handler now keeps the client's original tools, hands the guardrail a
deep copy of the flattened ones, and rebuilds data["tools"] by matching
the guardrail's output to the originals by type and name. Unchanged
tools go back as the original objects, a dropped or edited namespace
member changes only that member, and tools the guardrail injects are
still appended.
Fixes#39183
The chat-completions bridge emitted Responses output items whose item ids
were raw Anthropic tool ids (toolu_/srvtoolu_), which OpenAI rejects on
replay with "Expected an ID that begins with 'fc'", breaking router
fallback conversations from gpt-5 to claude models.
Four fixes, composable and independently useful:
- emission: bridge output items get fc_/ctc_-prefixed item ids while
call_id stays raw so tool_result pairing keeps working (streaming and
non-streaming share the same helpers)
- openai replay: request transformation drops tool call item ids that do
not match OpenAI's own shapes instead of forwarding them, gated to
OpenAI and Azure, since the API accepts the items with no id at all
- anthropic replay: a replayed srvtoolu_ call whose paired server tool
result is unavailable degrades to a plain client tool_use instead of a
dangling server_tool_use that 400s the client's tool_result
- tool-only turns no longer emit a message output item with output_text
text null, matching native OpenAI output
A guardrail modify_response verdict on a streaming request only produced a
proper replacement on /v1/messages: the chat completions and Responses API
translations had no build_block_sse_chunks, so the ModifyResponseException
re-raised and surfaced as an in-stream 500 error frame (or a whole-request
500 in buffered mode) instead of the documented 200 replacement.
Implement build_block_sse_chunks for both OpenAI translations: chat emits a
content delta plus a finish_reason content_filter chunk with real usage;
Responses emits the typed event sequence (standalone via
build_synthetic_response_events pre-stream, or an output-item continuation
under the in-progress response id mid-stream) ending in response.completed.
The Responses-to-chat transform dropped the filename OpenAI requires next to
file_data, so a request carrying an inline PDF counted 13 tokens instead of 36
and a real completion through the chat bridge got a 400.
Assistant list content was forwarded to /v1/responses/input_tokens as chat
`text` blocks, which the Responses API rejects (it accepts only output_text
and refusal inside an assistant turn). The 400 sent the whole request to the
local tokenizer, so any conversation with an assistant turn silently lost
provider-exact counting, including the image counting added in 73ab647b1c.
Assistant content now collapses to the plain string the Responses API counts
identically, and image parts are kept to user turns where they are legal.
The chat-to-Responses reverse transform kept only text blocks, so an image
input was dropped before the count went to OpenAI. A 256x256 image request
counted 13 tokens instead of 268.
_input_item_provenance converted every input prefix, so an n-item request paid
for n+1 full conversions. It now converts each item once, glues consecutive
function_call items (plus their trailing-assistant context) into units so the
transform's tool_call merging is reproduced inside the unit conversion, and
verifies the unit concatenation against one full conversion, bailing to the
full-conversion fallback on any mismatch. Messages from multi-item units are
tainted, which keeps parallel tool calls patchable exactly like the old prefix
pass while unpredicted merges fall back safely.
A guardrail handing back a non-list structured_messages payload (the
HiddenLayer v2 evaluation dict) previously fell through the length-mismatch
fallback and 500ed converting the dict's keys as messages. The write-back is
now skipped for non-list payloads, restoring the previous no-write-back
behavior on the Responses surface.
Also refreshes the compresr texts-mirror docstring, which still claimed the
Responses translation cannot round-trip structured_messages.
- _write_back_structured_messages now patches only the rewritten rows back
into the original input items, so reasoning items (encrypted_content),
function_call ids, and web_search_call items survive a guardrail rewrite
verbatim; rewrites that cannot be row-mapped fall back to the previous
full conversion
- the responses bridge stream snapshot restores usage hidden in
_hidden_params when stream_options is unset, so converted fake streams
report real input_tokens instead of 0
GPT-5 and later accept a top-level anyOf natively and call tools better with it intact, so the flattening now runs only for the gpt-4, gpt-3.5, chatgpt-4o, o1, o3, and o4 families. Non-dict tool entries pass through untouched, a typeless root that carries properties counts as an object, and the bounded $ref walker is listed in the recursion detector allowlist.
Message-rewriting guardrails such as Headroom return their rewrite in
structured_messages and leave texts untouched. The responses guardrail
translation only mapped texts back, so compression never reached the
upstream request on /v1/responses while the retrieve tool still got
injected. Convert the returned messages back to Responses input (plus
instructions) the way the chat and Anthropic handlers already do, and
keep developer messages as input_text in the chat-to-responses bridge.
Resolves LIT-6494
OpenAI's function-calling validator rejects tool parameters carrying
oneOf/anyOf/allOf/enum/const/not at the top level, while the ChatGPT
backend Codex talks to natively accepts them, so an MCP tool declaring a
top-level union 400s through the proxy. Merge the branches into the
object schema for OpenAI itself only, walking the namespace-nested tools
current Codex builds send, on both /v1/responses and /v1/responses/compact
