Find a file
Scott Werner e4a85679bf
fix(test): make twin-openai streamed message items round-trip as input (#484)
## What

Fixes the `openai_twin_*` parity-matrix failures that have been on
`main` since #449: every multi-turn scenario whose scripted response
includes text fails on its second turn with 400 `"message input items
require supported content"`.

## Root cause

Two twin behaviors collided (bisected: passes at #447, fails at #449):

1. **The twin's streaming `response.output_item.done` for message items
omitted the `content` array** (`test/twin/openai/src/sse.rs`) — it sent
only `id`/`type`/`status`/`role`, where the real API sends the completed
item in full. The openai adapter preserves message output items verbatim
(`ContentPart::Other { kind: OPENAI_MESSAGE }`) and replays them as
assistant history on the next turn — required so reasoning items keep
their "required following item" in Responses round-trips. So the replay
arrived content-less.
2. **#449 tightened the twin's input validation** to also validate
explicit `type: "message"` items (previously only type-less items were
validated as messages; anything with an explicit type was accepted
unchecked). The twin started rejecting its own round-tripped output.

The new validation caught a real infidelity in the emitter — the emit
side is what's wrong.

Nobody noticed because **CI never runs the twin e2e suites**: `rust.yml`
runs `--profile ci` without `--run-ignored`, so the parity matrix only
runs when someone invokes the e2e profile locally.

## Fix

- The streamed message `output_item.done` now carries its `output_text`
content, matching the real API and the twin's own non-streaming
`responses_json()`.
- The input validator accepts `output_text` parts on **assistant**
message items (the real API allows these; the twin's non-streaming
responses already require it for faithful replay). Non-assistant
`output_text` parts get a dedicated rejection message.

## Tests

- New contract test
`responses_stream_message_item_done_round_trips_as_input`: streams a
response, asserts the completed message item carries its `output_text`
content, and replays the item verbatim as assistant-history input,
asserting the twin accepts its own output.
- `cargo nextest run -p twin-openai` — 56 passed
- `cargo nextest run -p fabro-agent -E 'test(parity)' --run-ignored
only` — **91/91 passed** (was 7 failing)
- `cargo nextest run -p fabro-llm --run-ignored only` — 10 passed
- `cargo nextest run --workspace` — green apart from two pre-existing
env-dependent `fabro-workflow` failures that reproduce on clean `main`
in shells with provider API keys exported (unrelated; CI is green on
them because it has no such keys)
- clippy `-D warnings` / fmt — clean

Found while reviewing #481 (whose parity runs surfaced this); #481
itself is unaffected — it doesn't touch the openai adapter or the twin,
and the failures exist on its merge-base.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

## CI (separate commit, drop if unwanted)

`ci: run twin-mode e2e suites on Linux` adds a step to the existing
Linux test job running the ignored twin-mode suites for the packages
that are fully green today (`fabro-agent`, `fabro-llm`, `twin-openai`) —
104 tests, ~1s on a warm build, no secrets needed (live-only tests
self-skip in twin mode). This is what would have caught the #449
regression. The remaining ignored suites (fabro-cli twin tests,
Docker/Daytona sandbox tests, fabro-spa asset test) need their own fixes
before joining; widen the `-E` filter as they're cleaned up. Note the
step deliberately avoids the `e2e` nextest profile, since
`NEXTEST_PROFILE=e2e` implies strict mode, which fails on missing
secrets.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:56:20 -04:00
.ai/prompts Unify fabro run foreground to use create + start + attach (#141) 2026-03-22 22:48:09 -04:00
.cargo refactor(dev): decouple CLI reference generation 2026-05-06 12:31:02 -04:00
.claude refactor: remove devcontainer support (#433) 2026-05-27 12:48:40 -04:00
.config refactor: remove devcontainer support (#433) 2026-05-27 12:48:40 -04:00
.fabro chore: add demo workflows 2026-06-04 18:54:23 -04:00
.github fix(test): make twin-openai streamed message items round-trip as input (#484) 2026-06-10 12:56:20 -04:00
apps feat: chat-driven workflow builder at /playground (#450) 2026-06-09 11:24:56 -04:00
bin/agent chore: remove legacy FABRO_JWT_* key generation script 2026-04-25 12:33:03 -04:00
docker fix(server): allow required CSP handoffs 2026-06-01 18:24:45 -04:00
docs feat: chat-driven workflow builder at /playground (#450) 2026-06-09 11:24:56 -04:00
evals/swe-bench refactor(workflow): remove retro stage (#230) 2026-05-09 10:18:20 -04:00
installer refactor(release): simplify pre-release channel to nightly only 2026-04-17 11:39:19 -04:00
lib Interpolation foundation (InterpString v2) (#472) 2026-06-10 12:51:08 -04:00
test fix(test): make twin-openai streamed message items round-trip as input (#484) 2026-06-10 12:56:20 -04:00
.dockerignore chore: move docker-context/ staging dir under tmp/ 2026-04-25 12:37:45 -04:00
.env.example chore: remove legacy FABRO_JWT_* key generation script 2026-04-25 12:33:03 -04:00
.gitattributes chore: mark generated fabro-api-client as linguist-generated 2026-05-16 17:17:42 -04:00
.gitignore fix(server): make --watch-web honest and fast 2026-05-06 10:36:01 -04:00
AGENTS.md refactor: remove devcontainer support (#433) 2026-05-27 12:48:40 -04:00
bun.lock feat: chat-driven workflow builder at /playground (#450) 2026-06-09 11:24:56 -04:00
Cargo.lock Bump version to 0.260.0-nightly.0 2026-06-10 10:40:28 +00:00
Cargo.toml Bump version to 0.260.0-nightly.0 2026-06-10 10:40:28 +00:00
CLAUDE.md Move CLAUDE.md to AGENTS.md with symlink for compatibility 2026-03-09 13:02:06 -04:00
clippy.toml Interpolation foundation (InterpString v2) (#472) 2026-06-10 12:51:08 -04:00
CONTRIBUTING.md docs: offer issue-based contribution path alongside PRs 2026-05-16 13:40:23 -04:00
docker-compose.local.yaml chore(compose): drop debug log override for local docker 2026-04-30 08:25:50 -04:00
docker-compose.prod.yaml chore: move Caddyfile into docker/ 2026-04-27 11:23:02 -07:00
docker-compose.split-web.yaml Add split web Docker Compose PoC (#445) 2026-05-28 00:03:00 -04:00
docker-compose.yaml Improve Docker Compose deployment defaults 2026-05-09 18:59:25 -04:00
Dockerfile feat(server): support stdout log destination 2026-04-26 14:52:15 -04:00
install.md Move install files to apps/marketing/public, symlink from repo root 2026-03-16 13:23:58 -04:00
install.sh Move install files to apps/marketing/public, symlink from repo root 2026-03-16 13:23:58 -04:00
LICENSE.md Add README.md and MIT LICENSE 2026-03-10 14:48:58 -04:00
package.json chore: remove stale SQLite references after retirement 2026-04-05 21:29:24 -04:00
README.md docs: accept outside pull requests 2026-05-13 07:27:26 -04:00
rustfmt.toml fmt 2026-04-11 11:27:46 -04:00

Fabro

The open source dark software factory for expert engineers

AI coding agents are powerful but unpredictable. You either babysit every step or review a 50-file diff you don't trust. Fabro gives you a middle path: define the process as a graph, let agents execute it, and intervene only where it matters. Why Fabro?

Rust License: MIT docs Discord

# With Claude Code
curl -fsSL https://fabro.sh/install.md | claude

# With Codex
codex "$(curl -fsSL https://fabro.sh/install.md)"

# With Homebrew
brew install fabro-sh/tap/fabro-nightly

# With Bash
curl -fsSL https://fabro.sh/install.sh | bash

Then run fabro server start to finish setup in your browser. The server opens a web wizard, exits when the wizard completes, and starts in configured mode the next time you run it.

Fabro Runs board showing workflows across Working, Pending, Verify, and Merge stages

Use Cases

  • Extend disengagement time — Stop babysitting an agent REPL. Define a workflow with verification gates and walk away — Fabro keeps the process on track without you.
  • Leverage ensemble intelligence — Seamlessly combine models from different vendors. Use one model to implement, another to cross-critique, and a third to summarize — all in a single workflow.
  • Share best practices across your team — Collaborate on version-controlled workflows that encode your software processes as code. Review, iterate, and reuse them like any other source file.
  • Reduce token bills — Route cheap tasks to fast, inexpensive models and reserve frontier models for the steps that need them. CSS-like stylesheets make this a one-line change.
  • Improve agent security — Run agents in cloud sandboxes with full network and filesystem isolation. Keep untrusted code off your laptop and out of your production environment.
  • Run agents 24/7 — Fabro's API server queues and executes runs continuously. Close your laptop — workflows keep running and results are waiting when you return.
  • Scale infinitely — Move execution off your laptop and into cloud sandboxes. Run as many concurrent workflows as your infrastructure allows.
  • Guarantee code quality — Layer deterministic verifications — test suites, linters, type checkers, LLM-as-judge — into your workflow graph. Failures trigger fix loops automatically.
  • Inspect every run — Query durable event streams, checkpoints, conclusions, and stage outputs to understand what happened and improve the workflow.
  • Specify in natural language — Define requirements as natural-language specs and let Fabro generate — and regenerate — implementations that conform to them.

Key Features

Feature Description
🔀 Deterministic workflow graphs Define pipelines in Graphviz DOT with branching, loops, parallelism, and human gates. Diffable, reviewable, version-controlled
🙋 Human-in-the-loop Approval gates pause for human decisions. Steer running agents mid-turn. Interview steps collect structured input
🎨 Multi-model routing CSS-like stylesheets route each node to the right model and provider, with automatic fallback chains
☁️ Cloud sandboxes Run agents in isolated Daytona cloud VMs with snapshot-based setup, network controls, and automatic cleanup
🔌 SSH access and preview links Shell into running sandboxes with fabro sandbox ssh and expose ports with fabro sandbox preview for live debugging
🌲 Git checkpointing Every stage commits code changes and execution metadata to Git branches. Resume, revert, or trace any change
📊 Run observability Durable events, checkpoints, conclusions, and stage outputs make every run inspectable and exportable
⚡ Comprehensive API REST API with SSE event streaming and a React web UI. Run workflows programmatically or as a service
🦀 Single binary, no runtime One compiled Rust executable with zero dependencies. No Python, no Node, no Docker required
⚖️ Open source (MIT) Full source code, no vendor lock-in. Self-host, fork, or extend to fit your workflow

Example Workflow

A plan-approve-implement workflow where a human reviews the plan before the agent writes code:

Plan-Implement workflow graph showing Start → Plan → Approve Plan → Implement → Simplify → Exit with a Revise loop
digraph PlanImplement {
    graph [
        goal="Plan, approve, implement, and simplify a change"
        model_stylesheet="
            *        { model: claude-haiku-4-5; reasoning_effort: low; }
            .coding  { model: claude-sonnet-4-5; reasoning_effort: high; }
        "
    ]

    start [shape=Mdiamond, label="Start"]
    exit  [shape=Msquare, label="Exit"]

    plan      [label="Plan", prompt="Analyze the goal and codebase. Write a step-by-step plan.", reasoning_effort="high"]
    approve   [shape=hexagon, label="Approve Plan"]
    implement [label="Implement", class="coding", prompt="Read plan.md and implement every step."]
    simplify  [label="Simplify", class="coding", prompt="Review the changes for clarity and correctness."]

    start -> plan -> approve
    approve -> implement [label="[A] Approve"]
    approve -> plan      [label="[R] Revise"]
    implement -> simplify -> exit
}

Agents run as multi-turn LLM sessions with tool access. Human gates (hexagon) pause for approval. The stylesheet routes planning to a cheap model and coding to a frontier model. See the Graphviz DOT language reference for the full syntax.


📖 Documentation

Fabro ships with comprehensive documentation covering every feature in depth:

  • Getting Started -- Installation, first workflow, and why Fabro exists
  • Defining Workflows -- Node types, transitions, variables, stylesheets, and human gates
  • Executing Workflows -- Run configuration, sandboxes, checkpoints, observability, and failure handling
  • Tutorials -- Step-by-step guides from hello world to parallel multi-model ensembles
  • API Reference -- Full OpenAPI spec with authentication, SSE events, and client SDKs

Quick Start

Install

# With Claude Code
curl -fsSL https://fabro.sh/install.md | claude

# With Codex
codex "$(curl -fsSL https://fabro.sh/install.md)"

# With Homebrew
brew install fabro-sh/tap/fabro-nightly

# With Bash
curl -fsSL https://fabro.sh/install.sh | bash

Release binaries and the multi-arch Docker image ship with SLSA Build Provenance attestations. See Verifying Releases to check an artifact was built by our GitHub Actions workflow.

Then finish setup in your browser and initialize Fabro in your project:

fabro server start     # opens a web install wizard in your browser
                       # (server exits when the wizard finishes — start it again to run Fabro)

cd my-project
fabro repo init        # per project

For headless or scripted environments, fabro install runs the same setup as a CLI-only wizard.


Running Fabro

Fabro runs as a server. You choose where it runs:

  • On your laptop — install the CLI (above) and run fabro server start. Workflows pause when your laptop sleeps.
  • On a host (self-hosted) — deploy the Docker image with docker compose or any cloud container service (ECS, Cloud Run, Kubernetes). See Self-host with Docker.

One-click managed alternative for the same Docker image:

Deploy on Railway

See the deployment overview for the full picture.


Contributing to Fabro

Outside contributions are welcome! Whether it's a bug fix, a new feature, documentation, or a typo -- we'd love your help making Fabro better.

  • Bug fixes and small improvements -- Send a pull request directly.
  • Larger features or changes -- Open a GitHub Issue or start a Discussion first so we can align on the approach.
  • Questions -- Open a Discussion or email bryan@qlty.sh.

See CONTRIBUTING.md for build instructions and development workflow.


Help or Feedback


License

Fabro is licensed under the MIT License.