mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-08-28 05:27:41 +00:00
e4a85679bf
## What Fixes the `openai_twin_*` parity-matrix failures that have been on `main` since #449: every multi-turn scenario whose scripted response includes text fails on its second turn with 400 `"message input items require supported content"`. ## Root cause Two twin behaviors collided (bisected: passes at #447, fails at #449): 1. **The twin's streaming `response.output_item.done` for message items omitted the `content` array** (`test/twin/openai/src/sse.rs`) — it sent only `id`/`type`/`status`/`role`, where the real API sends the completed item in full. The openai adapter preserves message output items verbatim (`ContentPart::Other { kind: OPENAI_MESSAGE }`) and replays them as assistant history on the next turn — required so reasoning items keep their "required following item" in Responses round-trips. So the replay arrived content-less. 2. **#449 tightened the twin's input validation** to also validate explicit `type: "message"` items (previously only type-less items were validated as messages; anything with an explicit type was accepted unchecked). The twin started rejecting its own round-tripped output. The new validation caught a real infidelity in the emitter — the emit side is what's wrong. Nobody noticed because **CI never runs the twin e2e suites**: `rust.yml` runs `--profile ci` without `--run-ignored`, so the parity matrix only runs when someone invokes the e2e profile locally. ## Fix - The streamed message `output_item.done` now carries its `output_text` content, matching the real API and the twin's own non-streaming `responses_json()`. - The input validator accepts `output_text` parts on **assistant** message items (the real API allows these; the twin's non-streaming responses already require it for faithful replay). Non-assistant `output_text` parts get a dedicated rejection message. ## Tests - New contract test `responses_stream_message_item_done_round_trips_as_input`: streams a response, asserts the completed message item carries its `output_text` content, and replays the item verbatim as assistant-history input, asserting the twin accepts its own output. - `cargo nextest run -p twin-openai` — 56 passed - `cargo nextest run -p fabro-agent -E 'test(parity)' --run-ignored only` — **91/91 passed** (was 7 failing) - `cargo nextest run -p fabro-llm --run-ignored only` — 10 passed - `cargo nextest run --workspace` — green apart from two pre-existing env-dependent `fabro-workflow` failures that reproduce on clean `main` in shells with provider API keys exported (unrelated; CI is green on them because it has no such keys) - clippy `-D warnings` / fmt — clean Found while reviewing #481 (whose parity runs surfaced this); #481 itself is unaffected — it doesn't touch the openai adapter or the twin, and the failures exist on its merge-base. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## CI (separate commit, drop if unwanted) `ci: run twin-mode e2e suites on Linux` adds a step to the existing Linux test job running the ignored twin-mode suites for the packages that are fully green today (`fabro-agent`, `fabro-llm`, `twin-openai`) — 104 tests, ~1s on a warm build, no secrets needed (live-only tests self-skip in twin mode). This is what would have caught the #449 regression. The remaining ignored suites (fabro-cli twin tests, Docker/Daytona sandbox tests, fabro-spa asset test) need their own fixes before joining; widen the `-E` filter as they're cleaned up. Note the step deliberately avoids the `e2e` nextest profile, since `NEXTEST_PROFILE=e2e` implies strict mode, which fails on missing secrets. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| nightly.yml | ||
| release.yml | ||
| rust.yml | ||
| typescript.yml | ||