Commit graph

7 commits

Author SHA1 Message Date
Bryan Helmkamp
8d14b54994
Bound for_each fan-out memory
Addresses a Copilot review comment on #653.

The source array is runtime data, usually produced by a model, so its
length is not something a workflow author reviewed. Two changes, so an
over-long array degrades into a clear error rather than memory pressure.

Cap the item count at 1000. Above that the stage fails deterministically
before `parallel.started`, alongside the other for_each contract
violations, and the message says how to reduce the array.

Fork the parent context inside the branch task, after it acquires a
`max_parallel` slot, instead of at dispatch time. Live context copies now
track `max_parallel` rather than item count. Only the branch's own
preamble entry is moved into the task, so the shared stash is not cloned
per branch either.

The reviewer also suggested replacing spawn-all with `max_parallel`
workers pulling from a queue. Not done here: with the fork deferred, a
pending task holds little beyond its item, and reshaping the dispatch
loop would change cancellation and scope-reservation ordering, which
deserves its own review.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:16:20 -04:00
Bryan Helmkamp
4efaf5c3d7
Let --dry-run walk a for_each fan-out
build_branch_plan read the runtime array before run_branches looked at
`simulated`, so every for_each workflow failed under --dry-run with
"for_each source '...' was not found in workflow context". Nothing had
populated the key yet: upstream LLM nodes take Handler::simulate, which
returns no context updates.

A dry run now stands in one placeholder item when the source is absent or
unusable, and simulates the template target once. Graph-shape mistakes
still fail, since catching those is the point of a dry run.

Also from review:
- ITEM_FENCE_PREFIX replaces the bare "untrusted-" literal that
  render_item_data and its test each spelled out.
- ItemRecordingHandler no longer guesses an item label by substring
  search. Nothing asserted it, and the third item's label "2" matched
  stray hex from the random fence tag about two thirds of the time.
- The twin test reads keys::PARALLEL_RESULTS instead of the raw string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:49:41 -04:00
Bryan Helmkamp
b53045a1ac
Add runtime for_each item injection 2026-07-27 11:55:55 -04:00
Bryan Helmkamp
ac62585eb7
fix parallel result artifact handling 2026-07-24 09:26:22 -04:00
Bryan Helmkamp
7216c49e44
docs: align parallel strategy with typed StageOutcome results
Review follow-ups: the ParallelBranchResult snippet showed status as
String (it is StageOutcome), and the cancellation section implied a
cancelled branch status that the type does not have — cancelled-while-
waiting branches record a failed outcome (reason "branch cancelled")
and the handler returns Error::Cancelled to the run executor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 07:00:44 -04:00
Bryan Helmkamp
0a39ba9e06
Shared-checkout parallel execution (recovered from run 01KY7YH7RYCJ1BDVTTP96ZA4HV)
Cumulative implement + simplify_fable diff recovered from the run's meta
branch (fabro/meta/01KY7YH7RYCJ1BDVTTP96ZA4HV, stage 006 diff.patch).
The run validated this tree clean: cargo nextest (7,007 passed), clippy,
fmt, TS client regen + typecheck, web tests (679 passed), docs check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 06:19:11 -04:00
Bryan Helmkamp
736e302636 Add internal parallelization strategy doc
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 13:37:31 -04:00