diff --git a/run.json b/run.json index 40e9d1d6c..84578bce3 100644 --- a/run.json +++ b/run.json @@ -505,11 +505,12 @@ "status_updated_at": "2026-05-04T17:51:36.132690Z", "pending_control": null, "checkpoint": { - "timestamp": "2026-05-04T17:51:39.550911Z", - "current_node": "toolchain", + "timestamp": "2026-05-04T17:53:51.795767Z", + "current_node": "preflight_compile", "completed_nodes": [ "start", - "toolchain" + "toolchain", + "preflight_compile" ], "node_retries": {}, "context_values": { @@ -518,15 +519,17 @@ "internal.retry_count.start": 0, "graph.goal": "# Stage URLs encode visit (`node@visit`)\n\n## Context\n\nToday, in the Fabro web UI, stages that re-run (e.g. `verify` in a loop) all\ncollapse to the same URL: `/runs/{id}/stages/verify`. The sidebar lists them\nmultiple times but every link/selection points at the first visit.\n\nA **Stage** is a Node + a Visit (1-indexed; bumped each time the workflow\nre-enters that node). The data model already knows this:\n`StageId(node_id, visit)` exists in `lib/crates/fabro-types/src/stage_id.rs`,\nand the OpenAPI `StageId` path parameter\n(`docs/public/api-reference/fabro-api.yaml:2925`) already documents the\n`node_id@visit` form. The bug is that `GET /api/v1/runs/{id}/stages` returns\n`RunStage.id = node_id` (no visit suffix), and the events fallback in the UI\nfilters by `node_id` instead of the full `stage_id`.\n\nNote: \"visit\" is deliberate. There is a separate retry-attempt counter\ninside a single visit (`StageStartedProps.attempt` in\n`lib/crates/fabro-types/src/run_event/stage.rs:13`) — that's not what we're\nmodeling here. URLs and the new field both refer to **visits**.\n\nOutcome: each stage gets a distinct URL (e.g. `verify@1`, `verify@2`) that\nloads only that visit's turns/logs, with a `(N)` indicator in the sidebar\nwhen `N > 1`.\n\n## Approach\n\n**Server**: rebuild the stage list from `RunProjection::iter_stages()`, which\nis already keyed by full `StageId` (`HashMap` in\n`lib/crates/fabro-types/src/run_projection.rs:32`). This deletes the\n`checkpoint.completed_nodes` walk (which loses visit info — it's a\n`Vec` of node_ids only) and the `next_node_id` branch entirely.\n\n**Status derivation is event-driven, not completion-driven.**\n`StageProjection.completion` is set by `StageFailed` *even when* the workflow\nis about to retry (`run_state.rs:329` — `StageRetrying` does not clear it),\nso reading completion alone would show `failed` for a stage that's\nretrying. For each stage, scan its events (filtered by exact `stage_id`)\nand take the **latest** lifecycle event:\n- `stage.retrying` → `StageState::Retrying`\n- `stage.failed(props)` with `props.will_retry == true` → `StageState::Retrying`\n- `stage.failed(props)` with `props.will_retry == false` → `StageState::Failed`\n- `stage.completed` → `StageState::from(StageCompletedProps.status)`\n- `stage.started` (no later completed/failed/retrying) → `StageState::Running`\n\nUse the projection's `completion` only as a tiebreaker when no lifecycle\nevents for that stage_id exist (defensive case). The\n`StageState::from(StageOutcome)` impl is at\n`lib/crates/fabro-types/src/outcome.rs:136`.\n\n**API contract**: on `RunStage`, add a required `visit: integer` field, and\n**rename `dot_id` → `node_id`** (required) for consistency with\n`StageId::node_id()`, `EventEnvelope.node_id`, and the rest of the type\nvocabulary. Tighten the `id` description to call out the `node_id@visit`\nform. This is a breaking field rename; per project policy\n(\"simplest change possible, we don't care about migration\"), we do it now\nrather than carrying both names.\n\n**Frontend**: links and selection already use `stage.id`, so they propagate\nnaturally once the API returns `verify@1`/`verify@2`. The events-fallback\nfilter switches from `e.node_id === stageId` to `e.stage_id === stageId`.\nSidebar/header append `(N)` only when `visit > 1`. The\n`isVisibleStage(stage.id)` filter switches to `isVisibleStage(stage.node_id)`\nso that `start@1`/`exit@1` are still hidden.\n\nOne fixture run with two visits of the same node is added to demo data so\nthis code path stays under test.\n\n## Files to change (in order)\n\n### 1. OpenAPI — `docs/public/api-reference/fabro-api.yaml`\n\n`RunStage` schema (line 6315):\n- `id`: clarify description: `StageId in \"node_id@visit\" form, e.g. verify@2`. Update example to `verify@2`.\n- Add `visit: { type: integer, format: uint32, minimum: 1, description: \"1-based visit count; bumped each time the workflow re-enters this node\" }`. Mark required. (`format: uint32` + `minimum: 1` codegens to `NonZeroU32`, matching `StageProjection.first_event_seq` at line 5286–5288 of the spec.)\n- **Rename `dot_id` → `node_id`** and mark required. Description: \"Node id in the workflow graph; multiple stages with different visits share the same node_id.\" Example: `verify`.\n\n### 2. Generated code\n\n- `cargo build -p fabro-api` — regenerates Rust types via `progenitor`.\n- `cd lib/packages/fabro-api-client && bun run generate` — regenerates TS client.\n\n### 3. Workflow lib — `lib/crates/fabro-workflow/src/lib.rs`\n\nAdd a sibling to `extract_stage_durations_from_events` (line 89). Leave the\nexisting function alone — `finalize.rs:81,506` and `retro.rs:56` operate on a\nsingle visit per node and shouldn't change. New function:\n\n```rust\npub fn extract_stage_durations_by_stage_id(\n events: &[EventEnvelope],\n) -> HashMap\n```\n\nFilters `stage.completed`/`stage.failed`, keys by `envelope.stage_id`.\n\n### 4. Server handler — `lib/crates/fabro-server/src/server/handler/billing.rs`\n\nRewrite `list_run_stages` (lines 38–126):\n\n- Replace `checkpoint.completed_nodes` iteration with\n `projection.iter_stages()`, collected and sorted by `first_event_seq`.\n- Per stage, build `RunStage`:\n - `id = stage_id.to_string()`\n - `node_id = stage_id.node_id().to_string()`\n - `name = stage_id.node_id().to_string()` (UI adds the suffix)\n - `visit = NonZeroU32::new(stage_id.visit()).expect(\"StageId.visit is 1-based\")`\n (generated type is `NonZeroU32` because of `format: uint32` + `minimum: 1`)\n - `status = stage_status_from_events(events, &stage_id, &projection)`\n (see Status derivation below)\n - `duration_secs`: from the new `extract_stage_durations_by_stage_id`.\n- **Status derivation** — replace `active_stage_state_from_events` (line 19)\n with `stage_status_from_events(events: &[EventEnvelope], stage_id: &StageId,\n projection: &RunProjection) -> StageState`. Implementation:\n 1. Filter events to those with `envelope.event.stage_id == Some(stage_id)`.\n 2. Find the **latest** lifecycle event among `stage.started`,\n `stage.retrying`, `stage.completed`, `stage.failed` for that stage_id.\n 3. Map:\n - `stage.started` → `Running`\n - `stage.retrying` → `Retrying`\n - `stage.failed(props)` with `props.will_retry == true` → `Retrying`\n (a will-retry failure is conceptually mid-retry, even before the\n `stage.retrying` envelope lands; field defined at\n `lib/crates/fabro-types/src/run_event/stage.rs:57`)\n - `stage.failed(props)` with `props.will_retry == false` → `Failed`\n - `stage.completed` → `StageState::from(StageCompletedProps.status)`\n (using the existing `From for StageState` impl)\n 4. Fallback: if no lifecycle events, use\n `StageState::from(completion.outcome)` from the projection if present,\n else `Pending`.\n- Drop the `next_node_id` branch (lines 113–123) entirely — the projection\n now carries the in-flight stage.\n\n### 5. Demo fixtures — `lib/crates/fabro-server/src/demo/mod.rs:1156`\n\nSuffix existing four IDs with `@1` and set `visit: 1`. Add a 5th entry to\nmodel a re-run:\n\n```rust\nfn visit(n: u32) -> NonZeroU32 { NonZeroU32::new(n).expect(\"visit is 1-based\") }\n\nRunStage { id: \"apply-changes@1\".into(), name: \"apply-changes\".into(),\n status: Succeeded, duration_secs: Some(118.0),\n node_id: \"apply\".into(), visit: visit(1) },\nRunStage { id: \"apply-changes@2\".into(), name: \"apply-changes\".into(),\n status: Running, duration_secs: None,\n node_id: \"apply\".into(), visit: visit(2) },\n```\n\n`visit` codegens to `NonZeroU32` (see Section 1) — direct `1`/`2` literals\nwon't compile. Use the helper above (or inline\n`NonZeroU32::new(n).unwrap()`).\n\nBoth share `node_id: \"apply\"` so the graph node lights up regardless of\nselection.\n\n### 6. Frontend mapping — `apps/fabro-web/app/lib/stage-sidebar.ts:13`\n\n- Pass through `visit` and `node_id` (renamed from `dot_id`) to the sidebar\n `Stage` shape.\n- Change filter to `isVisibleStage(stage.node_id)` (line 17) so suffixed IDs\n still hide `start`/`exit`.\n- Drop the `?? stage.id` fallback (line 21) — `node_id` is now required.\n\n### 7. Sidebar component — `apps/fabro-web/app/components/stage-sidebar.tsx`\n\n- Rename the `dotId` field on `Stage` to `nodeId` (line 21).\n- Add `visit: number` to the `Stage` interface (line 16).\n- Render display label as `${stage.name}` when `visit <= 1`, otherwise\n `${stage.name} (${visit})` in the `` at line 103.\n- Update any callers reading `stage.dotId` (graph highlighting) to\n `stage.nodeId`.\n\n### 8. SSE cache invalidation — `apps/fabro-web/app/lib/run-events.ts`\n\nTwo fixes here:\n\n1. **Suffixed StageId routing** (`:132`): `stageIdFromPayload` currently\n returns `payload.node_id`. Once\n `queryKeys.runs.stageTurns(runId, stageId)` is keyed by `verify@1` (lines\n 84, 95 of the same file), invalidations passing `verify` won't match.\n - Add `stage_id?: string` to `RunEventPayload` (line 14).\n - In `stageIdFromPayload`: prefer `payload.stage_id`; fall back to\n `payload.node_id` for events that don't carry the full StageId (e.g.\n pre-stage envelopes).\n2. **Add `stage.retrying` to `STAGE_EVENTS`** (line 35). Today the set is\n `[\"stage.started\", \"stage.completed\", \"stage.failed\"]`. The new\n server-side status logic relies on `stage.retrying`, and the workflow\n already emits it (`lib/crates/fabro-workflow/src/lifecycle/event.rs:232`).\n Without this, a selected stage stays visually `failed` until another\n invalidating event arrives — defeats the P1 fix above.\n\nTests:\n- An envelope with `stage_id: \"verify@2\"` and `event: \"stage.retrying\"`\n invalidates `stages`, `events`, `graph`, run `detail`, and\n `stageTurns(runId, \"verify@2\")`.\n\n### 9. Run-stages route — `apps/fabro-web/app/routes/run-stages.tsx`\n\n- Line 72: change `events.filter((e) => e.node_id === stageId)` to\n `events.filter((e) => e.stage_id === stageId)`. The filter narrows the\n scope so `stageId` (the function parameter) is the authoritative StageId\n inside the loop.\n- Lines 117 and 126: drop the `` ?? `${stageId}@1` `` fallback. Note:\n `EventEnvelope.stage_id` is generated as `string | null | undefined`\n (see `lib/packages/fabro-api-client/src/models/run-event.ts:34`), so\n assigning `stageId: e.stage_id` directly fails typecheck. Use the\n function parameter instead — after the filter, all surviving events have\n `stage_id === stageId` by construction:\n ```ts\n pendingCommand = { stageId, script, language };\n ...\n turns.push({ kind: \"command\", stageId, ... });\n ```\n- Header (line ~640): when `selectedStage.visit > 1`, render\n `${selectedStage.name} (${selectedStage.visit})`.\n\n### 10. Graph aggregation policy — `apps/fabro-web/app/routes/run-overview.tsx:66-77`\n\nToday the graph code maps `Map`; with two visits sharing a\nnode_id, the second entry silently overwrites the first, and the status\nsets union all visits. Make the policy explicit:\n\n- **Click target**: open the **latest** visit for that node_id (highest\n `visit`). Build the map deterministically — `Map.set(nodeId, latestStage.id)`\n after sorting visits ascending.\n- **Status policy**: *latest visit wins for terminal states; active states\n win globally.* That is — for a given node, if any visit is `running` or\n `retrying`, the node renders that active state. Otherwise the node renders\n the **latest visit's** terminal state. So:\n - `(failed, running)` → `running` (active wins)\n - `(failed, succeeded)` → `succeeded` (latest visit wins; failure-then-fix\n should look healed, not failed)\n - `(succeeded, failed)` → `failed` (latest visit wins)\n - `(running, retrying)` → `retrying` (active; pick the latest)\n- The current if/else cascade in run-overview.tsx orders running before\n failed unconditionally — switch it to a two-step compute: pick the\n display status per node by the rule above, *then* render once.\n- **Frontend tests**: `(failed, running)` → running color, click → `verify@2`.\n `(failed, succeeded)` → succeeded color, click → `verify@2`.\n `(succeeded, failed)` → failed color, click → `verify@2`.\n\n### 11. Tests\n\n- **`lib/crates/fabro-server/src/server/tests.rs`** (alongside existing\n `list_run_stages_projects_retrying_until_completion` at line 2126): add\n `list_run_stages_distinguishes_visits` — build a run with two visits of\n the same node, hit `GET /runs/{id}/stages`, assert two `RunStage`\n entries with distinct `id`/`visit` and the same `node_id`.\n- **`lib/crates/fabro-server/src/server/tests.rs`**:\n `list_run_stages_shows_retrying_after_failed_event` — a stage where the\n latest event is `stage.failed` followed by `stage.retrying` renders as\n `Retrying`, not `Failed`.\n- **`lib/crates/fabro-server/src/server/tests.rs`**:\n `list_run_stages_shows_retrying_when_failed_will_retry` — a stage whose\n *only* lifecycle event so far is `stage.failed { will_retry: true }`\n (no `stage.retrying` envelope yet) still renders as `Retrying`. Narrower\n guard for the will_retry branch.\n- **`apps/fabro-web/app/routes/run-stages.test.ts`**: `turnsFromEvents`\n filters correctly on `stage_id` (verify@1 events vs verify@2 events do\n not cross-contaminate).\n- **`apps/fabro-web/app/lib/stage-sidebar.test.ts`** (new or existing): map\n fixture with two `apply-changes` visits → two distinct sidebar entries,\n display labels `apply-changes` and `apply-changes (2)`.\n- **`apps/fabro-web/app/lib/run-events.test.tsx`**: SSE envelope with\n `stage_id: \"verify@2\"` triggers invalidation of\n `stageTurns(runId, \"verify@2\")`.\n- **Graph test** (`apps/fabro-web/app/routes/run-overview.test.tsx` or\n similar): two visits of the same node — graph status follows the\n cascade, click target is the latest visit.\n\n## Out of scope\n\n- Wiring up the production `/runs/{id}/stages/{stageId}/turns` handler\n (`lib/crates/fabro-server/src/server/handler/mod.rs:116` is\n `not_implemented`). The events fallback is doing the work today and will\n keep doing it; the per-stage filter fix is what unblocks multi-visit\n display.\n- Per-visit billing breakdown in `get_run_billing` — that path still uses\n the existing per-node duration map.\n\n## Critical files\n\n- `docs/public/api-reference/fabro-api.yaml` — schema source of truth\n- `lib/crates/fabro-server/src/server/handler/billing.rs` — `list_run_stages`\n- `lib/crates/fabro-types/src/run_projection.rs` — `iter_stages` data source\n- `lib/crates/fabro-workflow/src/lib.rs` — new duration extractor\n- `lib/crates/fabro-server/src/demo/mod.rs` — fixture\n- `apps/fabro-web/app/routes/run-stages.tsx` — events filter + header\n- `apps/fabro-web/app/components/stage-sidebar.tsx` — display label\n- `apps/fabro-web/app/lib/stage-sidebar.ts` — visibility filter + mapping\n- `apps/fabro-web/app/lib/run-events.ts` — SSE cache invalidation\n- `apps/fabro-web/app/routes/run-overview.tsx` — graph aggregation policy\n- `lib/crates/fabro-types/src/outcome.rs` — `From for StageState` (already exists; reuse)\n\n## Verification\n\nBuild:\n- `cargo build -p fabro-api` — regenerates types from updated YAML\n- `cd lib/packages/fabro-api-client && bun run generate`\n- `cargo build --workspace`\n\nTests:\n- `cargo nextest run -p fabro-server` — conformance + new tests in\n `lib/crates/fabro-server/src/server/tests.rs` (distinguish_visits,\n shows_retrying_after_failed_event, shows_retrying_when_failed_will_retry)\n- `cd apps/fabro-web && bun test && bun run typecheck`\n\nEnd-to-end (single-visit regression):\n- `fabro server start` → open the demo URL → confirm\n `detect-drift`/`propose-changes`/`review-changes` show no `(N)` suffix.\n URLs are `.../stages/detect-drift@1` etc. Graph still highlights correctly.\n\nEnd-to-end (the fix):\n- Demo: two `apply-changes` rows. Sidebar shows `apply-changes` and\n `apply-changes (2)`. URLs `.../stages/apply-changes@1` vs\n `.../stages/apply-changes@2` are distinct and load distinct content. Graph\n lights up the same `apply` node either way.\n- Real loop run: trigger a workflow that loops `verify` (fail → fix → pass).\n Confirm two distinct entries with distinct statuses, durations, turns, and\n command logs (`/stages/verify@1/logs/stdout` vs\n `/stages/verify@2/logs/stdout`).\n\nAPI contract:\n- `curl /api/v1/runs/{id}/stages | jq` — `id` contains `@`; `visit` is\n present and ≥ 1; `node_id` is the bare node id with no `@`. The old\n `dot_id` field is gone.\n\nNegative checks:\n- Terminal run: no trailing in-flight row.\n- Parallel fanout: still one row per group (parallel branches don't promote\n to separate `RunStage` entries).\n- Empty checkpoint: empty list, no panic.\n- **Retry mid-flight**: trigger a stage that fails and then retries; sidebar\n shows `Retrying`, not `Failed`. Confirms P1 regression guard.\n- **SSE liveness**: while a run is active and a stage emits events, the\n selected stage's turn list updates without a manual refresh — confirms\n cache invalidation works against suffixed keys.\n", "command.stderr": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", - "command.output": "blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c", + "internal.retry_count.preflight_compile": 0, + "command.output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", "internal.retry_count.toolchain": 0, + "thread.toolchain.current_node": "preflight_compile", "failure_class": "", "graph.model_stylesheet": "\n * { model: claude-opus-4-7; }\n ", "internal.node_visit_count": 1, "graph.rankdir": "LR", "failure_signature": "", - "internal.thread_id": "start", - "current_node": "toolchain", + "internal.thread_id": "toolchain", + "current_node": "preflight_compile", "thread.start.current_node": "toolchain", "internal.run_id": "01KQT1VDVXGWN9P6MFK4R5E44D", "outcome": "succeeded" @@ -536,6 +539,15 @@ "status": "succeeded", "usage": null }, + "preflight_compile": { + "status": "succeeded", + "context_updates": { + "command.output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", + "command.stderr": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126" + }, + "notes": "Script completed: cargo check -q --workspace 2>&1", + "usage": null + }, "toolchain": { "status": "succeeded", "context_updates": { @@ -546,10 +558,11 @@ "usage": null } }, - "next_node_id": "preflight_compile", + "next_node_id": "preflight_lint", "node_visits": { "start": 1, - "toolchain": 1 + "toolchain": 1, + "preflight_compile": 1 } }, "checkpoints": [ @@ -588,6 +601,58 @@ "start": 1 } } + ], + [ + 26, + { + "timestamp": "2026-05-04T17:51:43.542987Z", + "current_node": "toolchain", + "completed_nodes": [ + "start", + "toolchain" + ], + "node_retries": {}, + "context_values": { + "failure_signature": "", + "graph.model_stylesheet": "\n * { model: claude-opus-4-7; }\n ", + "current_node": "toolchain", + "internal.node_visit_count": 1, + "command.stderr": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", + "internal.work_dir": "/home/daytona/workspace", + "command.output": "blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c", + "thread.start.current_node": "toolchain", + "failure_class": "", + "internal.fidelity": "compact", + "internal.run_id": "01KQT1VDVXGWN9P6MFK4R5E44D", + "internal.retry_count.start": 0, + "graph.goal": "# Stage URLs encode visit (`node@visit`)\n\n## Context\n\nToday, in the Fabro web UI, stages that re-run (e.g. `verify` in a loop) all\ncollapse to the same URL: `/runs/{id}/stages/verify`. The sidebar lists them\nmultiple times but every link/selection points at the first visit.\n\nA **Stage** is a Node + a Visit (1-indexed; bumped each time the workflow\nre-enters that node). The data model already knows this:\n`StageId(node_id, visit)` exists in `lib/crates/fabro-types/src/stage_id.rs`,\nand the OpenAPI `StageId` path parameter\n(`docs/public/api-reference/fabro-api.yaml:2925`) already documents the\n`node_id@visit` form. The bug is that `GET /api/v1/runs/{id}/stages` returns\n`RunStage.id = node_id` (no visit suffix), and the events fallback in the UI\nfilters by `node_id` instead of the full `stage_id`.\n\nNote: \"visit\" is deliberate. There is a separate retry-attempt counter\ninside a single visit (`StageStartedProps.attempt` in\n`lib/crates/fabro-types/src/run_event/stage.rs:13`) — that's not what we're\nmodeling here. URLs and the new field both refer to **visits**.\n\nOutcome: each stage gets a distinct URL (e.g. `verify@1`, `verify@2`) that\nloads only that visit's turns/logs, with a `(N)` indicator in the sidebar\nwhen `N > 1`.\n\n## Approach\n\n**Server**: rebuild the stage list from `RunProjection::iter_stages()`, which\nis already keyed by full `StageId` (`HashMap` in\n`lib/crates/fabro-types/src/run_projection.rs:32`). This deletes the\n`checkpoint.completed_nodes` walk (which loses visit info — it's a\n`Vec` of node_ids only) and the `next_node_id` branch entirely.\n\n**Status derivation is event-driven, not completion-driven.**\n`StageProjection.completion` is set by `StageFailed` *even when* the workflow\nis about to retry (`run_state.rs:329` — `StageRetrying` does not clear it),\nso reading completion alone would show `failed` for a stage that's\nretrying. For each stage, scan its events (filtered by exact `stage_id`)\nand take the **latest** lifecycle event:\n- `stage.retrying` → `StageState::Retrying`\n- `stage.failed(props)` with `props.will_retry == true` → `StageState::Retrying`\n- `stage.failed(props)` with `props.will_retry == false` → `StageState::Failed`\n- `stage.completed` → `StageState::from(StageCompletedProps.status)`\n- `stage.started` (no later completed/failed/retrying) → `StageState::Running`\n\nUse the projection's `completion` only as a tiebreaker when no lifecycle\nevents for that stage_id exist (defensive case). The\n`StageState::from(StageOutcome)` impl is at\n`lib/crates/fabro-types/src/outcome.rs:136`.\n\n**API contract**: on `RunStage`, add a required `visit: integer` field, and\n**rename `dot_id` → `node_id`** (required) for consistency with\n`StageId::node_id()`, `EventEnvelope.node_id`, and the rest of the type\nvocabulary. Tighten the `id` description to call out the `node_id@visit`\nform. This is a breaking field rename; per project policy\n(\"simplest change possible, we don't care about migration\"), we do it now\nrather than carrying both names.\n\n**Frontend**: links and selection already use `stage.id`, so they propagate\nnaturally once the API returns `verify@1`/`verify@2`. The events-fallback\nfilter switches from `e.node_id === stageId` to `e.stage_id === stageId`.\nSidebar/header append `(N)` only when `visit > 1`. The\n`isVisibleStage(stage.id)` filter switches to `isVisibleStage(stage.node_id)`\nso that `start@1`/`exit@1` are still hidden.\n\nOne fixture run with two visits of the same node is added to demo data so\nthis code path stays under test.\n\n## Files to change (in order)\n\n### 1. OpenAPI — `docs/public/api-reference/fabro-api.yaml`\n\n`RunStage` schema (line 6315):\n- `id`: clarify description: `StageId in \"node_id@visit\" form, e.g. verify@2`. Update example to `verify@2`.\n- Add `visit: { type: integer, format: uint32, minimum: 1, description: \"1-based visit count; bumped each time the workflow re-enters this node\" }`. Mark required. (`format: uint32` + `minimum: 1` codegens to `NonZeroU32`, matching `StageProjection.first_event_seq` at line 5286–5288 of the spec.)\n- **Rename `dot_id` → `node_id`** and mark required. Description: \"Node id in the workflow graph; multiple stages with different visits share the same node_id.\" Example: `verify`.\n\n### 2. Generated code\n\n- `cargo build -p fabro-api` — regenerates Rust types via `progenitor`.\n- `cd lib/packages/fabro-api-client && bun run generate` — regenerates TS client.\n\n### 3. Workflow lib — `lib/crates/fabro-workflow/src/lib.rs`\n\nAdd a sibling to `extract_stage_durations_from_events` (line 89). Leave the\nexisting function alone — `finalize.rs:81,506` and `retro.rs:56` operate on a\nsingle visit per node and shouldn't change. New function:\n\n```rust\npub fn extract_stage_durations_by_stage_id(\n events: &[EventEnvelope],\n) -> HashMap\n```\n\nFilters `stage.completed`/`stage.failed`, keys by `envelope.stage_id`.\n\n### 4. Server handler — `lib/crates/fabro-server/src/server/handler/billing.rs`\n\nRewrite `list_run_stages` (lines 38–126):\n\n- Replace `checkpoint.completed_nodes` iteration with\n `projection.iter_stages()`, collected and sorted by `first_event_seq`.\n- Per stage, build `RunStage`:\n - `id = stage_id.to_string()`\n - `node_id = stage_id.node_id().to_string()`\n - `name = stage_id.node_id().to_string()` (UI adds the suffix)\n - `visit = NonZeroU32::new(stage_id.visit()).expect(\"StageId.visit is 1-based\")`\n (generated type is `NonZeroU32` because of `format: uint32` + `minimum: 1`)\n - `status = stage_status_from_events(events, &stage_id, &projection)`\n (see Status derivation below)\n - `duration_secs`: from the new `extract_stage_durations_by_stage_id`.\n- **Status derivation** — replace `active_stage_state_from_events` (line 19)\n with `stage_status_from_events(events: &[EventEnvelope], stage_id: &StageId,\n projection: &RunProjection) -> StageState`. Implementation:\n 1. Filter events to those with `envelope.event.stage_id == Some(stage_id)`.\n 2. Find the **latest** lifecycle event among `stage.started`,\n `stage.retrying`, `stage.completed`, `stage.failed` for that stage_id.\n 3. Map:\n - `stage.started` → `Running`\n - `stage.retrying` → `Retrying`\n - `stage.failed(props)` with `props.will_retry == true` → `Retrying`\n (a will-retry failure is conceptually mid-retry, even before the\n `stage.retrying` envelope lands; field defined at\n `lib/crates/fabro-types/src/run_event/stage.rs:57`)\n - `stage.failed(props)` with `props.will_retry == false` → `Failed`\n - `stage.completed` → `StageState::from(StageCompletedProps.status)`\n (using the existing `From for StageState` impl)\n 4. Fallback: if no lifecycle events, use\n `StageState::from(completion.outcome)` from the projection if present,\n else `Pending`.\n- Drop the `next_node_id` branch (lines 113–123) entirely — the projection\n now carries the in-flight stage.\n\n### 5. Demo fixtures — `lib/crates/fabro-server/src/demo/mod.rs:1156`\n\nSuffix existing four IDs with `@1` and set `visit: 1`. Add a 5th entry to\nmodel a re-run:\n\n```rust\nfn visit(n: u32) -> NonZeroU32 { NonZeroU32::new(n).expect(\"visit is 1-based\") }\n\nRunStage { id: \"apply-changes@1\".into(), name: \"apply-changes\".into(),\n status: Succeeded, duration_secs: Some(118.0),\n node_id: \"apply\".into(), visit: visit(1) },\nRunStage { id: \"apply-changes@2\".into(), name: \"apply-changes\".into(),\n status: Running, duration_secs: None,\n node_id: \"apply\".into(), visit: visit(2) },\n```\n\n`visit` codegens to `NonZeroU32` (see Section 1) — direct `1`/`2` literals\nwon't compile. Use the helper above (or inline\n`NonZeroU32::new(n).unwrap()`).\n\nBoth share `node_id: \"apply\"` so the graph node lights up regardless of\nselection.\n\n### 6. Frontend mapping — `apps/fabro-web/app/lib/stage-sidebar.ts:13`\n\n- Pass through `visit` and `node_id` (renamed from `dot_id`) to the sidebar\n `Stage` shape.\n- Change filter to `isVisibleStage(stage.node_id)` (line 17) so suffixed IDs\n still hide `start`/`exit`.\n- Drop the `?? stage.id` fallback (line 21) — `node_id` is now required.\n\n### 7. Sidebar component — `apps/fabro-web/app/components/stage-sidebar.tsx`\n\n- Rename the `dotId` field on `Stage` to `nodeId` (line 21).\n- Add `visit: number` to the `Stage` interface (line 16).\n- Render display label as `${stage.name}` when `visit <= 1`, otherwise\n `${stage.name} (${visit})` in the `` at line 103.\n- Update any callers reading `stage.dotId` (graph highlighting) to\n `stage.nodeId`.\n\n### 8. SSE cache invalidation — `apps/fabro-web/app/lib/run-events.ts`\n\nTwo fixes here:\n\n1. **Suffixed StageId routing** (`:132`): `stageIdFromPayload` currently\n returns `payload.node_id`. Once\n `queryKeys.runs.stageTurns(runId, stageId)` is keyed by `verify@1` (lines\n 84, 95 of the same file), invalidations passing `verify` won't match.\n - Add `stage_id?: string` to `RunEventPayload` (line 14).\n - In `stageIdFromPayload`: prefer `payload.stage_id`; fall back to\n `payload.node_id` for events that don't carry the full StageId (e.g.\n pre-stage envelopes).\n2. **Add `stage.retrying` to `STAGE_EVENTS`** (line 35). Today the set is\n `[\"stage.started\", \"stage.completed\", \"stage.failed\"]`. The new\n server-side status logic relies on `stage.retrying`, and the workflow\n already emits it (`lib/crates/fabro-workflow/src/lifecycle/event.rs:232`).\n Without this, a selected stage stays visually `failed` until another\n invalidating event arrives — defeats the P1 fix above.\n\nTests:\n- An envelope with `stage_id: \"verify@2\"` and `event: \"stage.retrying\"`\n invalidates `stages`, `events`, `graph`, run `detail`, and\n `stageTurns(runId, \"verify@2\")`.\n\n### 9. Run-stages route — `apps/fabro-web/app/routes/run-stages.tsx`\n\n- Line 72: change `events.filter((e) => e.node_id === stageId)` to\n `events.filter((e) => e.stage_id === stageId)`. The filter narrows the\n scope so `stageId` (the function parameter) is the authoritative StageId\n inside the loop.\n- Lines 117 and 126: drop the `` ?? `${stageId}@1` `` fallback. Note:\n `EventEnvelope.stage_id` is generated as `string | null | undefined`\n (see `lib/packages/fabro-api-client/src/models/run-event.ts:34`), so\n assigning `stageId: e.stage_id` directly fails typecheck. Use the\n function parameter instead — after the filter, all surviving events have\n `stage_id === stageId` by construction:\n ```ts\n pendingCommand = { stageId, script, language };\n ...\n turns.push({ kind: \"command\", stageId, ... });\n ```\n- Header (line ~640): when `selectedStage.visit > 1`, render\n `${selectedStage.name} (${selectedStage.visit})`.\n\n### 10. Graph aggregation policy — `apps/fabro-web/app/routes/run-overview.tsx:66-77`\n\nToday the graph code maps `Map`; with two visits sharing a\nnode_id, the second entry silently overwrites the first, and the status\nsets union all visits. Make the policy explicit:\n\n- **Click target**: open the **latest** visit for that node_id (highest\n `visit`). Build the map deterministically — `Map.set(nodeId, latestStage.id)`\n after sorting visits ascending.\n- **Status policy**: *latest visit wins for terminal states; active states\n win globally.* That is — for a given node, if any visit is `running` or\n `retrying`, the node renders that active state. Otherwise the node renders\n the **latest visit's** terminal state. So:\n - `(failed, running)` → `running` (active wins)\n - `(failed, succeeded)` → `succeeded` (latest visit wins; failure-then-fix\n should look healed, not failed)\n - `(succeeded, failed)` → `failed` (latest visit wins)\n - `(running, retrying)` → `retrying` (active; pick the latest)\n- The current if/else cascade in run-overview.tsx orders running before\n failed unconditionally — switch it to a two-step compute: pick the\n display status per node by the rule above, *then* render once.\n- **Frontend tests**: `(failed, running)` → running color, click → `verify@2`.\n `(failed, succeeded)` → succeeded color, click → `verify@2`.\n `(succeeded, failed)` → failed color, click → `verify@2`.\n\n### 11. Tests\n\n- **`lib/crates/fabro-server/src/server/tests.rs`** (alongside existing\n `list_run_stages_projects_retrying_until_completion` at line 2126): add\n `list_run_stages_distinguishes_visits` — build a run with two visits of\n the same node, hit `GET /runs/{id}/stages`, assert two `RunStage`\n entries with distinct `id`/`visit` and the same `node_id`.\n- **`lib/crates/fabro-server/src/server/tests.rs`**:\n `list_run_stages_shows_retrying_after_failed_event` — a stage where the\n latest event is `stage.failed` followed by `stage.retrying` renders as\n `Retrying`, not `Failed`.\n- **`lib/crates/fabro-server/src/server/tests.rs`**:\n `list_run_stages_shows_retrying_when_failed_will_retry` — a stage whose\n *only* lifecycle event so far is `stage.failed { will_retry: true }`\n (no `stage.retrying` envelope yet) still renders as `Retrying`. Narrower\n guard for the will_retry branch.\n- **`apps/fabro-web/app/routes/run-stages.test.ts`**: `turnsFromEvents`\n filters correctly on `stage_id` (verify@1 events vs verify@2 events do\n not cross-contaminate).\n- **`apps/fabro-web/app/lib/stage-sidebar.test.ts`** (new or existing): map\n fixture with two `apply-changes` visits → two distinct sidebar entries,\n display labels `apply-changes` and `apply-changes (2)`.\n- **`apps/fabro-web/app/lib/run-events.test.tsx`**: SSE envelope with\n `stage_id: \"verify@2\"` triggers invalidation of\n `stageTurns(runId, \"verify@2\")`.\n- **Graph test** (`apps/fabro-web/app/routes/run-overview.test.tsx` or\n similar): two visits of the same node — graph status follows the\n cascade, click target is the latest visit.\n\n## Out of scope\n\n- Wiring up the production `/runs/{id}/stages/{stageId}/turns` handler\n (`lib/crates/fabro-server/src/server/handler/mod.rs:116` is\n `not_implemented`). The events fallback is doing the work today and will\n keep doing it; the per-stage filter fix is what unblocks multi-visit\n display.\n- Per-visit billing breakdown in `get_run_billing` — that path still uses\n the existing per-node duration map.\n\n## Critical files\n\n- `docs/public/api-reference/fabro-api.yaml` — schema source of truth\n- `lib/crates/fabro-server/src/server/handler/billing.rs` — `list_run_stages`\n- `lib/crates/fabro-types/src/run_projection.rs` — `iter_stages` data source\n- `lib/crates/fabro-workflow/src/lib.rs` — new duration extractor\n- `lib/crates/fabro-server/src/demo/mod.rs` — fixture\n- `apps/fabro-web/app/routes/run-stages.tsx` — events filter + header\n- `apps/fabro-web/app/components/stage-sidebar.tsx` — display label\n- `apps/fabro-web/app/lib/stage-sidebar.ts` — visibility filter + mapping\n- `apps/fabro-web/app/lib/run-events.ts` — SSE cache invalidation\n- `apps/fabro-web/app/routes/run-overview.tsx` — graph aggregation policy\n- `lib/crates/fabro-types/src/outcome.rs` — `From for StageState` (already exists; reuse)\n\n## Verification\n\nBuild:\n- `cargo build -p fabro-api` — regenerates types from updated YAML\n- `cd lib/packages/fabro-api-client && bun run generate`\n- `cargo build --workspace`\n\nTests:\n- `cargo nextest run -p fabro-server` — conformance + new tests in\n `lib/crates/fabro-server/src/server/tests.rs` (distinguish_visits,\n shows_retrying_after_failed_event, shows_retrying_when_failed_will_retry)\n- `cd apps/fabro-web && bun test && bun run typecheck`\n\nEnd-to-end (single-visit regression):\n- `fabro server start` → open the demo URL → confirm\n `detect-drift`/`propose-changes`/`review-changes` show no `(N)` suffix.\n URLs are `.../stages/detect-drift@1` etc. Graph still highlights correctly.\n\nEnd-to-end (the fix):\n- Demo: two `apply-changes` rows. Sidebar shows `apply-changes` and\n `apply-changes (2)`. URLs `.../stages/apply-changes@1` vs\n `.../stages/apply-changes@2` are distinct and load distinct content. Graph\n lights up the same `apply` node either way.\n- Real loop run: trigger a workflow that loops `verify` (fail → fix → pass).\n Confirm two distinct entries with distinct statuses, durations, turns, and\n command logs (`/stages/verify@1/logs/stdout` vs\n `/stages/verify@2/logs/stdout`).\n\nAPI contract:\n- `curl /api/v1/runs/{id}/stages | jq` — `id` contains `@`; `visit` is\n present and ≥ 1; `node_id` is the bare node id with no `@`. The old\n `dot_id` field is gone.\n\nNegative checks:\n- Terminal run: no trailing in-flight row.\n- Parallel fanout: still one row per group (parallel branches don't promote\n to separate `RunStage` entries).\n- Empty checkpoint: empty list, no panic.\n- **Retry mid-flight**: trigger a stage that fails and then retries; sidebar\n shows `Retrying`, not `Failed`. Confirms P1 regression guard.\n- **SSE liveness**: while a run is active and a stage emits events, the\n selected stage's turn list updates without a manual refresh — confirms\n cache invalidation works against suffixed keys.\n", + "internal.thread_id": "start", + "internal.retry_count.toolchain": 0, + "outcome": "succeeded", + "graph.rankdir": "LR" + }, + "node_outcomes": { + "start": { + "status": "succeeded", + "usage": null + }, + "toolchain": { + "status": "succeeded", + "context_updates": { + "command.stderr": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", + "command.output": "blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c" + }, + "notes": "Script completed: command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", + "usage": null + } + }, + "next_node_id": "preflight_compile", + "git_commit_sha": "4e6b965637a41ed8a305cb45c0df31bfaa96c301", + "node_visits": { + "start": 1, + "toolchain": 1 + } + } ] ], "conclusion": null, @@ -607,11 +672,33 @@ "superseded_by": null, "pending_interviews": {}, "stages": { + "preflight_compile@1": { + "first_event_seq": 29, + "prompt": null, + "response": null, + "completion": null, + "provider_used": null, + "diff": null, + "script_invocation": { + "script": "cargo check -q --workspace 2>&1", + "command": "cargo check -q --workspace 2>&1", + "language": "shell" + }, + "script_timing": null, + "parallel_results": null, + "stdout": null, + "stderr": null + }, "toolchain@1": { "first_event_seq": 19, "prompt": null, "response": null, - "completion": null, + "completion": { + "outcome": "succeeded", + "notes": "Script completed: command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", + "failure_reason": null, + "timestamp": "2026-05-04T17:51:39.549748Z" + }, "provider_used": null, "diff": null, "script_invocation": { @@ -619,10 +706,25 @@ "command": "command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", "language": "shell" }, - "script_timing": null, + "script_timing": { + "stdout": "blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c", + "stderr": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", + "exit_code": 0, + "duration_ms": 1351, + "termination": "exited", + "stdout_bytes": 36, + "stderr_bytes": 0, + "streams_separated": true, + "live_streaming": true + }, "parallel_results": null, "stdout": null, - "stderr": null + "stderr": null, + "stdout_bytes": 36, + "stderr_bytes": 0, + "streams_separated": true, + "live_streaming": true, + "termination": "exited" }, "start@1": { "first_event_seq": 15, diff --git a/stages/002-toolchain@1/script_timing.json b/stages/002-toolchain@1/script_timing.json new file mode 100644 index 000000000..9b763ae26 --- /dev/null +++ b/stages/002-toolchain@1/script_timing.json @@ -0,0 +1,11 @@ +{ + "stdout": "blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c", + "stderr": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", + "exit_code": 0, + "duration_ms": 1351, + "termination": "exited", + "stdout_bytes": 36, + "stderr_bytes": 0, + "streams_separated": true, + "live_streaming": true +} \ No newline at end of file diff --git a/stages/002-toolchain@1/status.json b/stages/002-toolchain@1/status.json new file mode 100644 index 000000000..d4ddd6e09 --- /dev/null +++ b/stages/002-toolchain@1/status.json @@ -0,0 +1,6 @@ +{ + "outcome": "succeeded", + "notes": "Script completed: command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1", + "failure_reason": null, + "timestamp": "2026-05-04T17:51:39.549748Z" +} \ No newline at end of file diff --git a/stages/002-toolchain@1/stderr.log b/stages/002-toolchain@1/stderr.log new file mode 100644 index 000000000..d87ba9545 --- /dev/null +++ b/stages/002-toolchain@1/stderr.log @@ -0,0 +1 @@ +blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126 \ No newline at end of file diff --git a/stages/002-toolchain@1/stdout.log b/stages/002-toolchain@1/stdout.log new file mode 100644 index 000000000..4e86d161d --- /dev/null +++ b/stages/002-toolchain@1/stdout.log @@ -0,0 +1 @@ +blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c \ No newline at end of file diff --git a/stages/003-preflight_compile@1/script_invocation.json b/stages/003-preflight_compile@1/script_invocation.json new file mode 100644 index 000000000..16acaaf06 --- /dev/null +++ b/stages/003-preflight_compile@1/script_invocation.json @@ -0,0 +1,5 @@ +{ + "script": "cargo check -q --workspace 2>&1", + "command": "cargo check -q --workspace 2>&1", + "language": "shell" +} \ No newline at end of file