fabro/.github/workflows/rust.yml
Scott Werner e4a85679bf
fix(test): make twin-openai streamed message items round-trip as input (#484)
## What

Fixes the `openai_twin_*` parity-matrix failures that have been on
`main` since #449: every multi-turn scenario whose scripted response
includes text fails on its second turn with 400 `"message input items
require supported content"`.

## Root cause

Two twin behaviors collided (bisected: passes at #447, fails at #449):

1. **The twin's streaming `response.output_item.done` for message items
omitted the `content` array** (`test/twin/openai/src/sse.rs`) — it sent
only `id`/`type`/`status`/`role`, where the real API sends the completed
item in full. The openai adapter preserves message output items verbatim
(`ContentPart::Other { kind: OPENAI_MESSAGE }`) and replays them as
assistant history on the next turn — required so reasoning items keep
their "required following item" in Responses round-trips. So the replay
arrived content-less.
2. **#449 tightened the twin's input validation** to also validate
explicit `type: "message"` items (previously only type-less items were
validated as messages; anything with an explicit type was accepted
unchecked). The twin started rejecting its own round-tripped output.

The new validation caught a real infidelity in the emitter — the emit
side is what's wrong.

Nobody noticed because **CI never runs the twin e2e suites**: `rust.yml`
runs `--profile ci` without `--run-ignored`, so the parity matrix only
runs when someone invokes the e2e profile locally.

## Fix

- The streamed message `output_item.done` now carries its `output_text`
content, matching the real API and the twin's own non-streaming
`responses_json()`.
- The input validator accepts `output_text` parts on **assistant**
message items (the real API allows these; the twin's non-streaming
responses already require it for faithful replay). Non-assistant
`output_text` parts get a dedicated rejection message.

## Tests

- New contract test
`responses_stream_message_item_done_round_trips_as_input`: streams a
response, asserts the completed message item carries its `output_text`
content, and replays the item verbatim as assistant-history input,
asserting the twin accepts its own output.
- `cargo nextest run -p twin-openai` — 56 passed
- `cargo nextest run -p fabro-agent -E 'test(parity)' --run-ignored
only` — **91/91 passed** (was 7 failing)
- `cargo nextest run -p fabro-llm --run-ignored only` — 10 passed
- `cargo nextest run --workspace` — green apart from two pre-existing
env-dependent `fabro-workflow` failures that reproduce on clean `main`
in shells with provider API keys exported (unrelated; CI is green on
them because it has no such keys)
- clippy `-D warnings` / fmt — clean

Found while reviewing #481 (whose parity runs surfaced this); #481
itself is unaffected — it doesn't touch the openai adapter or the twin,
and the failures exist on its merge-base.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

## CI (separate commit, drop if unwanted)

`ci: run twin-mode e2e suites on Linux` adds a step to the existing
Linux test job running the ignored twin-mode suites for the packages
that are fully green today (`fabro-agent`, `fabro-llm`, `twin-openai`) —
104 tests, ~1s on a warm build, no secrets needed (live-only tests
self-skip in twin mode). This is what would have caught the #449
regression. The remaining ignored suites (fabro-cli twin tests,
Docker/Daytona sandbox tests, fabro-spa asset test) need their own fixes
before joining; widen the `-E` filter as they're cleaned up. Note the
step deliberately avoids the `e2e` nextest profile, since
`NEXTEST_PROFILE=e2e` implies strict mode, which fails on missing
secrets.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:56:20 -04:00

134 lines
4.7 KiB
YAML

name: Rust
on:
push:
branches: [main]
paths:
- "lib/crates/**"
- "test/**"
- "Cargo.toml"
- "Cargo.lock"
- ".cargo/**"
- ".config/**"
- "bin/dev/**"
- "docs/public/reference/cli.mdx"
- "docs/public/reference/user-configuration.mdx"
- "openapi/**"
- ".github/workflows/rust.yml"
pull_request:
branches: [main]
paths:
- "lib/crates/**"
- "test/**"
- "Cargo.toml"
- "Cargo.lock"
- ".cargo/**"
- ".config/**"
- "bin/dev/**"
- "docs/public/reference/cli.mdx"
- "docs/public/reference/user-configuration.mdx"
- "openapi/**"
- ".github/workflows/rust.yml"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: {}
env:
CARGO_TERM_COLOR: always
jobs:
fmt:
name: Format
runs-on: ubuntu-24.04-x86-32-cores
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
with:
toolchain: nightly-2026-04-14
components: rustfmt
- run: cargo +nightly-2026-04-14 fmt --check --all
clippy:
name: Clippy
runs-on: ubuntu-24.04-x86-32-cores
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
with:
toolchain: nightly-2026-04-14
components: clippy
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
cache-on-failure: true
- name: Verify legacy auth identity removal
run: |
! rg -n 'AuthMode::Disabled|RunAuthMethod|RunSubjectProvenance|\bActorRef\b|\bActorKind\b|AuthenticatedSubject|AuthenticatedService|AuthorizeRunScoped|AuthorizeRunBlob|AuthorizeStageArtifact|AuthorizeCommandLog|auth_method\s*==\s*"disabled"' \
lib/crates apps lib/packages docs/public/api-reference/fabro-api.yaml
- run: cargo +nightly-2026-04-14 clippy --locked --workspace --all-targets -- -D warnings
generated-docs:
name: Generated Docs
runs-on: ubuntu-24.04-x86-32-cores
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
cache-on-failure: true
- run: cargo --locked dev docs check
test:
name: Test (Linux)
runs-on: ubuntu-24.04-x86-32-cores
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
cache-on-failure: true
- uses: taiki-e/install-action@773334c0e05d7e699e4d78234494308223f3a2cf # nextest
- run: cargo nextest run --locked --workspace --status-level slow --profile ci
# Twin-mode e2e suites. These are hermetic (in-process twin provider, no
# secrets): FABRO_TEST_MODE defaults to twin, so live-only tests
# self-skip. Scoped to the packages whose ignored tests are fully green
# in twin mode; widen as the remaining suites are fixed up for CI.
# Must not use the e2e nextest profile here: NEXTEST_PROFILE=e2e implies
# strict mode, which fails (rather than skips) live tests without keys.
- run: cargo nextest run --locked --workspace --status-level slow --profile ci --run-ignored only -E 'package(fabro-agent) + package(fabro-llm) + package(twin-openai)'
test-macos:
name: Test (macOS)
if: github.event_name == 'workflow_dispatch'
runs-on: macos-15
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
cache-on-failure: true
- uses: taiki-e/install-action@773334c0e05d7e699e4d78234494308223f3a2cf # nextest
- run: cargo nextest run --locked --workspace --status-level slow --profile ci