## What Fixes the `openai_twin_*` parity-matrix failures that have been on `main` since #449: every multi-turn scenario whose scripted response includes text fails on its second turn with 400 `"message input items require supported content"`. ## Root cause Two twin behaviors collided (bisected: passes at #447, fails at #449): 1. **The twin's streaming `response.output_item.done` for message items omitted the `content` array** (`test/twin/openai/src/sse.rs`) — it sent only `id`/`type`/`status`/`role`, where the real API sends the completed item in full. The openai adapter preserves message output items verbatim (`ContentPart::Other { kind: OPENAI_MESSAGE }`) and replays them as assistant history on the next turn — required so reasoning items keep their "required following item" in Responses round-trips. So the replay arrived content-less. 2. **#449 tightened the twin's input validation** to also validate explicit `type: "message"` items (previously only type-less items were validated as messages; anything with an explicit type was accepted unchecked). The twin started rejecting its own round-tripped output. The new validation caught a real infidelity in the emitter — the emit side is what's wrong. Nobody noticed because **CI never runs the twin e2e suites**: `rust.yml` runs `--profile ci` without `--run-ignored`, so the parity matrix only runs when someone invokes the e2e profile locally. ## Fix - The streamed message `output_item.done` now carries its `output_text` content, matching the real API and the twin's own non-streaming `responses_json()`. - The input validator accepts `output_text` parts on **assistant** message items (the real API allows these; the twin's non-streaming responses already require it for faithful replay). Non-assistant `output_text` parts get a dedicated rejection message. ## Tests - New contract test `responses_stream_message_item_done_round_trips_as_input`: streams a response, asserts the completed message item carries its `output_text` content, and replays the item verbatim as assistant-history input, asserting the twin accepts its own output. - `cargo nextest run -p twin-openai` — 56 passed - `cargo nextest run -p fabro-agent -E 'test(parity)' --run-ignored only` — **91/91 passed** (was 7 failing) - `cargo nextest run -p fabro-llm --run-ignored only` — 10 passed - `cargo nextest run --workspace` — green apart from two pre-existing env-dependent `fabro-workflow` failures that reproduce on clean `main` in shells with provider API keys exported (unrelated; CI is green on them because it has no such keys) - clippy `-D warnings` / fmt — clean Found while reviewing #481 (whose parity runs surfaced this); #481 itself is unaffected — it doesn't touch the openai adapter or the twin, and the failures exist on its merge-base. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## CI (separate commit, drop if unwanted) `ci: run twin-mode e2e suites on Linux` adds a step to the existing Linux test job running the ignored twin-mode suites for the packages that are fully green today (`fabro-agent`, `fabro-llm`, `twin-openai`) — 104 tests, ~1s on a warm build, no secrets needed (live-only tests self-skip in twin mode). This is what would have caught the #449 regression. The remaining ignored suites (fabro-cli twin tests, Docker/Daytona sandbox tests, fabro-spa asset test) need their own fixes before joining; widen the `-E` filter as they're cleaned up. Note the step deliberately avoids the `e2e` nextest profile, since `NEXTEST_PROFILE=e2e` implies strict mode, which fails on missing secrets. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .ai/prompts | ||
| .cargo | ||
| .claude | ||
| .config | ||
| .fabro | ||
| .github | ||
| apps | ||
| bin/agent | ||
| docker | ||
| docs | ||
| evals/swe-bench | ||
| installer | ||
| lib | ||
| test | ||
| .dockerignore | ||
| .env.example | ||
| .gitattributes | ||
| .gitignore | ||
| AGENTS.md | ||
| bun.lock | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CLAUDE.md | ||
| clippy.toml | ||
| CONTRIBUTING.md | ||
| docker-compose.local.yaml | ||
| docker-compose.prod.yaml | ||
| docker-compose.split-web.yaml | ||
| docker-compose.yaml | ||
| Dockerfile | ||
| install.md | ||
| install.sh | ||
| LICENSE.md | ||
| package.json | ||
| README.md | ||
| rustfmt.toml | ||
The open source dark software factory for expert engineers
AI coding agents are powerful but unpredictable. You either babysit every step or review a 50-file diff you don't trust. Fabro gives you a middle path: define the process as a graph, let agents execute it, and intervene only where it matters. Why Fabro?
# With Claude Code
curl -fsSL https://fabro.sh/install.md | claude
# With Codex
codex "$(curl -fsSL https://fabro.sh/install.md)"
# With Homebrew
brew install fabro-sh/tap/fabro-nightly
# With Bash
curl -fsSL https://fabro.sh/install.sh | bash
Then run fabro server start to finish setup in your browser. The server opens a web wizard, exits when the wizard completes, and starts in configured mode the next time you run it.
Use Cases
- Extend disengagement time — Stop babysitting an agent REPL. Define a workflow with verification gates and walk away — Fabro keeps the process on track without you.
- Leverage ensemble intelligence — Seamlessly combine models from different vendors. Use one model to implement, another to cross-critique, and a third to summarize — all in a single workflow.
- Share best practices across your team — Collaborate on version-controlled workflows that encode your software processes as code. Review, iterate, and reuse them like any other source file.
- Reduce token bills — Route cheap tasks to fast, inexpensive models and reserve frontier models for the steps that need them. CSS-like stylesheets make this a one-line change.
- Improve agent security — Run agents in cloud sandboxes with full network and filesystem isolation. Keep untrusted code off your laptop and out of your production environment.
- Run agents 24/7 — Fabro's API server queues and executes runs continuously. Close your laptop — workflows keep running and results are waiting when you return.
- Scale infinitely — Move execution off your laptop and into cloud sandboxes. Run as many concurrent workflows as your infrastructure allows.
- Guarantee code quality — Layer deterministic verifications — test suites, linters, type checkers, LLM-as-judge — into your workflow graph. Failures trigger fix loops automatically.
- Inspect every run — Query durable event streams, checkpoints, conclusions, and stage outputs to understand what happened and improve the workflow.
- Specify in natural language — Define requirements as natural-language specs and let Fabro generate — and regenerate — implementations that conform to them.
Key Features
| Feature | Description | |
|---|---|---|
| 🔀 | Deterministic workflow graphs | Define pipelines in Graphviz DOT with branching, loops, parallelism, and human gates. Diffable, reviewable, version-controlled |
| 🙋 | Human-in-the-loop | Approval gates pause for human decisions. Steer running agents mid-turn. Interview steps collect structured input |
| 🎨 | Multi-model routing | CSS-like stylesheets route each node to the right model and provider, with automatic fallback chains |
| ☁️ | Cloud sandboxes | Run agents in isolated Daytona cloud VMs with snapshot-based setup, network controls, and automatic cleanup |
| 🔌 | SSH access and preview links | Shell into running sandboxes with fabro sandbox ssh and expose ports with fabro sandbox preview for live debugging |
| 🌲 | Git checkpointing | Every stage commits code changes and execution metadata to Git branches. Resume, revert, or trace any change |
| 📊 | Run observability | Durable events, checkpoints, conclusions, and stage outputs make every run inspectable and exportable |
| ⚡ | Comprehensive API | REST API with SSE event streaming and a React web UI. Run workflows programmatically or as a service |
| 🦀 | Single binary, no runtime | One compiled Rust executable with zero dependencies. No Python, no Node, no Docker required |
| ⚖️ | Open source (MIT) | Full source code, no vendor lock-in. Self-host, fork, or extend to fit your workflow |
Example Workflow
A plan-approve-implement workflow where a human reviews the plan before the agent writes code:
digraph PlanImplement {
graph [
goal="Plan, approve, implement, and simplify a change"
model_stylesheet="
* { model: claude-haiku-4-5; reasoning_effort: low; }
.coding { model: claude-sonnet-4-5; reasoning_effort: high; }
"
]
start [shape=Mdiamond, label="Start"]
exit [shape=Msquare, label="Exit"]
plan [label="Plan", prompt="Analyze the goal and codebase. Write a step-by-step plan.", reasoning_effort="high"]
approve [shape=hexagon, label="Approve Plan"]
implement [label="Implement", class="coding", prompt="Read plan.md and implement every step."]
simplify [label="Simplify", class="coding", prompt="Review the changes for clarity and correctness."]
start -> plan -> approve
approve -> implement [label="[A] Approve"]
approve -> plan [label="[R] Revise"]
implement -> simplify -> exit
}
Agents run as multi-turn LLM sessions with tool access. Human gates (hexagon) pause for approval. The stylesheet routes planning to a cheap model and coding to a frontier model. See the Graphviz DOT language reference for the full syntax.
📖 Documentation
Fabro ships with comprehensive documentation covering every feature in depth:
- Getting Started -- Installation, first workflow, and why Fabro exists
- Defining Workflows -- Node types, transitions, variables, stylesheets, and human gates
- Executing Workflows -- Run configuration, sandboxes, checkpoints, observability, and failure handling
- Tutorials -- Step-by-step guides from hello world to parallel multi-model ensembles
- API Reference -- Full OpenAPI spec with authentication, SSE events, and client SDKs
Quick Start
Install
# With Claude Code
curl -fsSL https://fabro.sh/install.md | claude
# With Codex
codex "$(curl -fsSL https://fabro.sh/install.md)"
# With Homebrew
brew install fabro-sh/tap/fabro-nightly
# With Bash
curl -fsSL https://fabro.sh/install.sh | bash
Release binaries and the multi-arch Docker image ship with SLSA Build Provenance attestations. See Verifying Releases to check an artifact was built by our GitHub Actions workflow.
Then finish setup in your browser and initialize Fabro in your project:
fabro server start # opens a web install wizard in your browser
# (server exits when the wizard finishes — start it again to run Fabro)
cd my-project
fabro repo init # per project
For headless or scripted environments, fabro install runs the same setup as a CLI-only wizard.
Running Fabro
Fabro runs as a server. You choose where it runs:
- On your laptop — install the CLI (above) and run
fabro server start. Workflows pause when your laptop sleeps. - On a host (self-hosted) — deploy the Docker image with
docker composeor any cloud container service (ECS, Cloud Run, Kubernetes). See Self-host with Docker.
One-click managed alternative for the same Docker image:
See the deployment overview for the full picture.
Contributing to Fabro
Outside contributions are welcome! Whether it's a bug fix, a new feature, documentation, or a typo -- we'd love your help making Fabro better.
- Bug fixes and small improvements -- Send a pull request directly.
- Larger features or changes -- Open a GitHub Issue or start a Discussion first so we can align on the approach.
- Questions -- Open a Discussion or email bryan@qlty.sh.
See CONTRIBUTING.md for build instructions and development workflow.
Help or Feedback
- Bug reports via GitHub Issues
- Feature requests via GitHub Discussions
- Email bryan@qlty.sh for questions
- See CONTRIBUTING.md for build instructions and development workflow
License
Fabro is licensed under the MIT License.