mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-09-14 23:22:51 +00:00
## What Fixes the `openai_twin_*` parity-matrix failures that have been on `main` since #449: every multi-turn scenario whose scripted response includes text fails on its second turn with 400 `"message input items require supported content"`. ## Root cause Two twin behaviors collided (bisected: passes at #447, fails at #449): 1. **The twin's streaming `response.output_item.done` for message items omitted the `content` array** (`test/twin/openai/src/sse.rs`) — it sent only `id`/`type`/`status`/`role`, where the real API sends the completed item in full. The openai adapter preserves message output items verbatim (`ContentPart::Other { kind: OPENAI_MESSAGE }`) and replays them as assistant history on the next turn — required so reasoning items keep their "required following item" in Responses round-trips. So the replay arrived content-less. 2. **#449 tightened the twin's input validation** to also validate explicit `type: "message"` items (previously only type-less items were validated as messages; anything with an explicit type was accepted unchecked). The twin started rejecting its own round-tripped output. The new validation caught a real infidelity in the emitter — the emit side is what's wrong. Nobody noticed because **CI never runs the twin e2e suites**: `rust.yml` runs `--profile ci` without `--run-ignored`, so the parity matrix only runs when someone invokes the e2e profile locally. ## Fix - The streamed message `output_item.done` now carries its `output_text` content, matching the real API and the twin's own non-streaming `responses_json()`. - The input validator accepts `output_text` parts on **assistant** message items (the real API allows these; the twin's non-streaming responses already require it for faithful replay). Non-assistant `output_text` parts get a dedicated rejection message. ## Tests - New contract test `responses_stream_message_item_done_round_trips_as_input`: streams a response, asserts the completed message item carries its `output_text` content, and replays the item verbatim as assistant-history input, asserting the twin accepts its own output. - `cargo nextest run -p twin-openai` — 56 passed - `cargo nextest run -p fabro-agent -E 'test(parity)' --run-ignored only` — **91/91 passed** (was 7 failing) - `cargo nextest run -p fabro-llm --run-ignored only` — 10 passed - `cargo nextest run --workspace` — green apart from two pre-existing env-dependent `fabro-workflow` failures that reproduce on clean `main` in shells with provider API keys exported (unrelated; CI is green on them because it has no such keys) - clippy `-D warnings` / fmt — clean Found while reviewing #481 (whose parity runs surfaced this); #481 itself is unaffected — it doesn't touch the openai adapter or the twin, and the failures exist on its merge-base. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## CI (separate commit, drop if unwanted) `ci: run twin-mode e2e suites on Linux` adds a step to the existing Linux test job running the ignored twin-mode suites for the packages that are fully green today (`fabro-agent`, `fabro-llm`, `twin-openai`) — 104 tests, ~1s on a warm build, no secrets needed (live-only tests self-skip in twin mode). This is what would have caught the #449 regression. The remaining ignored suites (fabro-cli twin tests, Docker/Daytona sandbox tests, fabro-spa asset test) need their own fixes before joining; widen the `-E` filter as they're cleaned up. Note the step deliberately avoids the `e2e` nextest profile, since `NEXTEST_PROFILE=e2e` implies strict mode, which fails on missing secrets. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
134 lines
4.7 KiB
YAML
134 lines
4.7 KiB
YAML
name: Rust
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- "lib/crates/**"
|
|
- "test/**"
|
|
- "Cargo.toml"
|
|
- "Cargo.lock"
|
|
- ".cargo/**"
|
|
- ".config/**"
|
|
- "bin/dev/**"
|
|
- "docs/public/reference/cli.mdx"
|
|
- "docs/public/reference/user-configuration.mdx"
|
|
- "openapi/**"
|
|
- ".github/workflows/rust.yml"
|
|
pull_request:
|
|
branches: [main]
|
|
paths:
|
|
- "lib/crates/**"
|
|
- "test/**"
|
|
- "Cargo.toml"
|
|
- "Cargo.lock"
|
|
- ".cargo/**"
|
|
- ".config/**"
|
|
- "bin/dev/**"
|
|
- "docs/public/reference/cli.mdx"
|
|
- "docs/public/reference/user-configuration.mdx"
|
|
- "openapi/**"
|
|
- ".github/workflows/rust.yml"
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
permissions: {}
|
|
|
|
env:
|
|
CARGO_TERM_COLOR: always
|
|
|
|
jobs:
|
|
fmt:
|
|
name: Format
|
|
runs-on: ubuntu-24.04-x86-32-cores
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
|
|
with:
|
|
toolchain: nightly-2026-04-14
|
|
components: rustfmt
|
|
- run: cargo +nightly-2026-04-14 fmt --check --all
|
|
|
|
clippy:
|
|
name: Clippy
|
|
runs-on: ubuntu-24.04-x86-32-cores
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
|
|
with:
|
|
toolchain: nightly-2026-04-14
|
|
components: clippy
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
|
|
with:
|
|
cache-on-failure: true
|
|
- name: Verify legacy auth identity removal
|
|
run: |
|
|
! rg -n 'AuthMode::Disabled|RunAuthMethod|RunSubjectProvenance|\bActorRef\b|\bActorKind\b|AuthenticatedSubject|AuthenticatedService|AuthorizeRunScoped|AuthorizeRunBlob|AuthorizeStageArtifact|AuthorizeCommandLog|auth_method\s*==\s*"disabled"' \
|
|
lib/crates apps lib/packages docs/public/api-reference/fabro-api.yaml
|
|
- run: cargo +nightly-2026-04-14 clippy --locked --workspace --all-targets -- -D warnings
|
|
|
|
generated-docs:
|
|
name: Generated Docs
|
|
runs-on: ubuntu-24.04-x86-32-cores
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
|
|
with:
|
|
cache-on-failure: true
|
|
- run: cargo --locked dev docs check
|
|
|
|
test:
|
|
name: Test (Linux)
|
|
runs-on: ubuntu-24.04-x86-32-cores
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
|
|
with:
|
|
cache-on-failure: true
|
|
- uses: taiki-e/install-action@773334c0e05d7e699e4d78234494308223f3a2cf # nextest
|
|
- run: cargo nextest run --locked --workspace --status-level slow --profile ci
|
|
# Twin-mode e2e suites. These are hermetic (in-process twin provider, no
|
|
# secrets): FABRO_TEST_MODE defaults to twin, so live-only tests
|
|
# self-skip. Scoped to the packages whose ignored tests are fully green
|
|
# in twin mode; widen as the remaining suites are fixed up for CI.
|
|
# Must not use the e2e nextest profile here: NEXTEST_PROFILE=e2e implies
|
|
# strict mode, which fails (rather than skips) live tests without keys.
|
|
- run: cargo nextest run --locked --workspace --status-level slow --profile ci --run-ignored only -E 'package(fabro-agent) + package(fabro-llm) + package(twin-openai)'
|
|
|
|
test-macos:
|
|
name: Test (macOS)
|
|
if: github.event_name == 'workflow_dispatch'
|
|
runs-on: macos-15
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
persist-credentials: false
|
|
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
|
|
with:
|
|
cache-on-failure: true
|
|
- uses: taiki-e/install-action@773334c0e05d7e699e4d78234494308223f3a2cf # nextest
|
|
- run: cargo nextest run --locked --workspace --status-level slow --profile ci
|