diff --git a/docs/internal/parallel-strategy.md b/docs/internal/parallel-strategy.md new file mode 100644 index 000000000..351764ade --- /dev/null +++ b/docs/internal/parallel-strategy.md @@ -0,0 +1,318 @@ +# Parallel fan-out / fan-in strategy + +Status: proposed (not yet implemented). Line numbers reference the tree at the +time of writing and will drift. + +This document specifies the intended product behavior of parallel fan-out +(`shape=component`) and fan-in (`shape=tripleoctagon`). Appendix A catalogs +pre-existing bugs this design resolves. Appendix B lists behavior changes +relative to today's implementation. + +## 1. Introduction: goals and current weaknesses + +The goal of this design is a parallel execution model that is **simple**, +**coherent**, and **correct**: + +- **Simple** — a user should be able to predict what a fan-out/fan-in does + from the graph alone. One mental model ("branches produce candidates; fan-in + picks a commit; downstream sees all responses"), no new syntax for + synthesis, no incantations (special fidelity settings, escape-hatch + attributes) to make the flagship patterns work. +- **Coherent** — the same rules apply regardless of node type or channel. + What holds for a sequential command node's output should hold for a branch + command node's output; the workspace (git) facet and the text (context) + facet should follow parallel logic; selection should live in exactly one + place. +- **Correct** — the engine must do what the graph, the docs, and the recorded + run state say it does. Nodes drawn in the graph must run; documented outputs + must exist; a judge asked to pick the best candidate must be shown the + candidates. + +Today's implementation misses all three. Issue #490 is the visible symptom: +a synthesis node after fan-in never sees branch responses — only +`{id, status, head_sha}` metadata — while two tutorials promise the opposite. +Investigation showed a broader incoherence: + +- **Inherited spec gap.** The Attractor spec (fabro's ancestor) deliberately + isolates branch context and never defines a channel for branch output text. + Its fan-in pseudocode judges candidates it cannot see (`llm_evaluate` + receives only statuses) and sorts by a `score` field nothing sets. Fabro + inherited this gap faithfully. +- **Missing compensation.** Kilroy (a sibling Attractor implementation) + compensates with a file/git handoff: the post-merge node's prompt is + injected with each branch's `worktree_dir`, `logs_root`, and `head_sha` plus + instructions to read/merge them. Fabro has no equivalent, but its tutorials + promise kilroy-like behavior ("the synth node receives all four perspectives + in its preamble"). +- **Accidental semantics.** Several adjacent behaviors are unprincipled + accidents rather than decisions: branches execute a single node and silently + skip chained nodes; two handlers fast-forward to two different definitions + of "winner"; branch nodes run with a stale preamble built for the fan-out + node; selection is vacuous in both modes; nested fan-outs silently lose git + isolation. Appendix A catalogs these. + +## 2. Core model: candidates + +A parallel branch is an isolated unit of execution that produces a +**candidate**. A candidate has exactly three facets: + +| Facet | Content | Carried by | +|---|---|---| +| Commit | Workspace state produced by the branch | Per-branch git commit (`head_sha`) | +| Response | Text output (LLM response, command output) | Branch stage records + context keys | +| Verdict | Terminal status plus optional numeric `score` | `parallel.results` entries | + +Fan-out produces N candidates in isolation. Fan-in selects **one commit** to +continue on. Downstream nodes (synthesis) get **all responses**. Selection and +synthesis are distinct concerns: selection needs a node (`tripleoctagon`); +synthesis is any ordinary downstream node, because responses propagate. + +The post-fan-in contract, in one sentence: **after fan-in, the run looks as if +every branch had run sequentially, and the winner ran last.** + +## 3. Topology: branches are subgraphs + +Each outgoing edge of a fan-out node starts a branch. A branch executes as a +subgraph walk: the engine traverses nodes and edges from the branch entry node +until it reaches the join node (the fan-in). Multi-node chains +(`fork -> plan_a; plan_a -> impl_a; impl_a -> merge`) run every node in the +chain. This matches the Attractor spec (`execute_subgraph`) and kilroy +(`runSubgraphUntil`); today fabro executes exactly one node per branch and +silently skips the rest (Appendix A.2). + +Structural validation (lint): + +- Every path from a branch entry must converge on the run's join node. + A branch path that escapes the join (reaches exit, or a node outside the + fan-out region) is a validation error. +- A nested `component` node inside a branch is rejected until + worktree-from-worktree isolation is implemented (today it silently runs with + no git isolation; Appendix A.6). +- `fidelity="full"` on a fan-in's outgoing edge gets a lint warning: branches + run on different threads, so full fidelity can never carry branch outputs + (it drops the preamble entirely). + +## 4. Node types in branches + +No type-based restrictions. Restriction is structural (§3), not by allowlist: + +- **Agent / prompt nodes** — the primary case. +- **Command / script / tool nodes** — first-class. Deterministic fan-out + (test matrices, benchmark bake-offs across worktrees) is a supported pattern + with no LLM anywhere: branches emit `score` via status fields, heuristic + selection picks the winner by measurement. +- **Conditionals** — meaningful under subgraph branches: they route *within* + the branch. +- **Human gates** — allowed; each branch may pause independently. +- **Nested parallel** — rejected by lint until isolation composes (§3). + +## 5. Git isolation + +Unchanged mechanics, with ownership fixed: + +1. Before fan-out, checkpoint the sandbox to produce `base_sha`. +2. Each branch gets a worktree on a branch ref + (`fabro/run/parallel///pass/`), rooted at `base_sha`. + The branch's `internal.work_dir` points at the worktree. +3. After a branch completes, `git add -A` + commit (`--allow-empty`) yields the + candidate's `head_sha`. +4. Worktrees are removed after the join. Loser branch refs are **kept** so + downstream nodes and humans can `git show`/`git diff` any candidate. +5. **Fan-in exclusively owns the fast-forward.** The parallel handler performs + no merge. After selection, fan-in fast-forwards the primary workspace to the + winner's `head_sha`. (Today both handlers fast-forward, to potentially + different winners; Appendix A.3.) + +Degradation without git (no repo, or git isolation disabled): branches share +the primary sandbox with no workspace isolation, `head_sha` is absent from +candidates, and fan-in performs no merge. Response and verdict facets work +unchanged — prompt-only ensembles do not require git. + +## 6. Execution and stage recording + +Branch nodes execute as **real stages**, recorded through the normal +`ExecutionState::record` path and namespaced under the fan-out +(e.g. stage `a@1` within `fork@1`). Consequences (all fixes to current +behavior): + +- Branch prompts/responses appear in events, `fabro dump`, and the web UI as + ordinary stages. +- Each branch node gets a **freshly built preamble** for its own position, via + the standard lifecycle, instead of reusing the fan-out node's stale preamble. +- Branch stages participate in the standard retry policy per node. + +## 7. Context merge-back at fan-in + +When fan-in completes, branch context updates are applied to the parent +context with a collision rule: + +- **Per-node keys** (`response.`, structured-output fields + namespaced by node) apply for **all** branches. Branch node IDs are unique, + so no collisions. +- **Singleton keys** (`last_stage`, `last_response`, `command.output`, + un-namespaced status fields) are taken from the **winner only**. +- Failed branches' `response.` values are still applied (a synthesis node + analyzing disagreement wants to see the failure text). Their singletons are + never applied. + +Fan-in additionally writes (as today): + +- `parallel.results` — one entry per candidate: `{id, status, head_sha?, + score?}`. +- `parallel.branch_count`, `parallel.fan_in.best_id`, + `parallel.fan_in.best_outcome`, `parallel.fan_in.best_head_sha`. + +No file is materialized into the run workspace. `parallel.results` reaches LLM +consumers through the preamble's context section, and agents can `git show` +any candidate via its `head_sha`. (The current docs claim +`parallel_results.json` is available to downstream nodes; that claim is false +today and should be corrected rather than implemented — see open question 4 +for the one consumer this leaves unserved.) + +## 8. Selection + +Fan-in selects the winning candidate. Two modes, as today, but with real +signal: + +- **Heuristic** (no prompt on the fan-in node): rank by status + (succeeded < partially_succeeded < failed), then `score` descending, then + lexical id. `score` becomes settable: branches emit it via structured-output + / status fields, which now survive into `parallel.results` (§7). +- **LLM judge** (fan-in node has a `prompt` and a backend is configured): the + judge prompt includes, per candidate: id, status, score, a bounded response + excerpt, and `git diff --stat` vs. `base_sha` when git isolation is active. + Today the judge sees only `{id, status, head_sha}` and cannot possibly + discriminate (Appendix A.4). + +Selection determines the commit facet only. It does not suppress loser +responses (§7) or loser refs (§5). + +## 9. Downstream visibility (preamble) + +Prompt templates render once at manifest build time with `{goal, inputs}` +only; the preamble is the sole channel for runtime context into a +fresh-session node. Therefore: + +- At **`compact`** (default) fidelity, branch stage summaries render their + responses **inline**, bounded, with a `See: ` reference when + truncated — the same treatment `command.output` already receives at compact. + Rationale: post-fan-in branch responses are unrecoverable through any + fidelity setting (different threads), exactly like command output. +- The per-branch response budget is larger than command output's 25-line tail + (ensemble analyses front-load their substance; a small tail amputates it). + Exact budget TBD at implementation; must remain bounded so N long branches + cannot blow the downstream context. Agent-type synthesis nodes can read the + full text from the blob/artifact reference. +- `summary:high` renders the same with its larger budget. `truncate` carries + goal only (explicit opt-out). `full` remains the degenerate case and lints + (§3). + +## 10. Join policies + +- `wait_all` (default): all branches run to completion; join proceeds when all + are terminal. Succeeds if no branch failed, else partially succeeds (fan-in + fails only when *all* candidates failed). +- `first_success`: join proceeds at the first successful branch. Remaining + branches are cancelled; cancelled branches record a terminal cancelled stage + (their partial responses are not merged back). The sole successful branch is + the winner. +- `k_of_n` / `quorum` (kilroy has them): deliberately **not** added now. The + surface stays minimal until a concrete need appears. + +## Open questions + +1. Implementation phasing: subgraph branches (§3) are the largest lift. + Response propagation (§6–§9) fixes #490 and both tutorials on its own and + can ship first. +2. `first_success` cancellation semantics for in-flight agent sessions + (graceful stop vs. abort; what the cancelled stage records). +3. Exact preamble budget per branch response (§9). +4. Context access for deterministic post-fan-in consumers. Command/script + nodes have no channel to context (no preamble, no template rendering in + `script`), so a deterministic aggregator after fan-in cannot learn + candidate `head_sha`s. Candidate mechanisms: a results file under a + checkpoint-excluded workspace path, or an env var (e.g. + `FABRO_PARALLEL_RESULTS`) pointing at a file outside the workspace. A bare + workspace file is ruled out: checkpoint commits `git add -A`, so it would + leak into run history and PRs. Design alongside the deterministic fan-out + pattern (§4). + +--- + +## Appendix A: pre-existing bugs + +Cataloged against the current tree; line numbers will drift. + +1. **Branch outputs dropped (#490).** The fan-out task reads only + `outcome.status` and `head_sha` from each branch; branch `context_updates` + (including `response.`) are discarded with the forked context + (`fabro-workflow/src/handler/parallel.rs:389-397,465-470`). No channel + carries branch text to downstream nodes. Confirmed by live repro on + fabro-testing (run `01KX148ZAMMJRAADHK1HBF7PC3`, server 0.287.0-nightly.0). +2. **Chained branch nodes silently skipped.** Branches execute exactly one + node; the engine then jumps to the join (`parallel.rs:388-397,612,628`; + `fabro-core/src/executor.rs:421`). In `fork -> a; a -> a2; a2 -> merge`, + `a2` never runs and nothing warns. +3. **Double fast-forward with two different winner definitions.** The parallel + handler fast-forwards the *lexically first* successful branch + (`parallel.rs:511-538`); fan-in then fast-forwards *its* selected winner + (`fan_in.rs:120-131`). If selection ever picks a non-lexical-first branch, + the second `--ff-only` merge cannot succeed (sibling commits diverge). + Masked today only because selection is vacuous (A.4). +4. **Selection is vacuous.** Heuristic tie-breaks on a `score` field nothing + can set (scores would arrive via branch context updates, which are dropped + per A.1). The LLM judge prompt is `serde_json::to_string_pretty` of + `parallel.results` — id/status/head_sha only (`fan_in.rs:247-250`). Both + modes reduce to "first successful branch, alphabetically." +5. **Branch preambles are stale.** Branches run via `dispatch_handler`, + bypassing the lifecycle's per-node preamble rebuild; each branch node + inherits `current.preamble` as computed for the fan-out node itself. +6. **Nested parallel silently loses git isolation.** Branch `EngineServices` + are built with `git_state: RwLock::new(None)` (`parallel.rs:380`), so a + `component` node inside a branch runs its own branches with no worktrees + and no warning. +7. **Docs contradict the engine.** `tutorials/ensemble.mdx:85` and + `tutorials/parallel-review.mdx:81` claim the post-merge node receives all + branch perspectives in its preamble (false, per A.1). + `workflows/stages-and-nodes.mdx:195` claims merged results are available to + downstream nodes as `parallel_results.json` (the file exists only under + `stages/@1/` in dumps, not in any node's working directory). +8. **`fidelity="full"` across a fan-in is a trap.** It drops the preamble + (metadata included) and cannot attach to any branch thread; raising + fidelity strictly reduces what the downstream node sees. No lint warns. + +## Appendix B: behavior changes vs. today + +Changes a user could observe if this spec is implemented as written. + +1. **Chained branch nodes execute.** Graphs that (unknowingly) relied on + single-node branch semantics will now run the full chain (fixes A.2; may + lengthen existing runs). +2. **New validation errors.** Branch paths that don't converge on the join, + and nested `component` nodes inside branches, become lint failures for + graphs that previously ran (with wrong or silently degraded semantics). +3. **Fan-in owns the fast-forward.** The workspace after fan-in may land on a + different commit than today whenever selection (scores, LLM judge) + disagrees with lexical-first order. The parallel handler no longer merges. +4. **Post-fan-in context is richer.** `response.` for every branch, + winner-sourced singletons (`last_stage`, `last_response`, + `command.output`), and `score` in `parallel.results`. Today those + singletons retain their pre-fork values; workflows with edge conditions + over them could route differently. +5. **Preambles after fan-in grow.** Branch responses render inline at + `compact` fidelity (bounded). Downstream nodes see more tokens per run; + snapshot tests over preambles will churn. +6. **Branch executions become visible stages.** Events, dumps, the web UI, and + the stage list gain per-branch stages (`a@1` under `fork@1`). Consumers of + `events.jsonl` / the API will see new stage records. +7. **`parallel_results.json` docs claim is corrected, not implemented.** + `workflows/stages-and-nodes.mdx:195` is updated to describe the real + channels (context key + preamble); no file appears in the workspace + (open question 4 covers deterministic consumers). +8. **Loser branch refs are documented as retained** and become part of the + product contract instead of an accident of not deleting them. +9. **`first_success` cancels losers explicitly** and records cancelled stages; + today's exact cancellation behavior is unspecified. +10. **LLM judge prompts change shape.** Fan-in nodes with prompts now send + candidate excerpts and diff stats to the judge — more tokens, different + (better) selections than today's id-only prompt.