From 3b3fe80ae1924060ad455ac3342f67cfc2f33b51 Mon Sep 17 00:00:00 2001 From: davincidreams Date: Sat, 25 Apr 2026 02:34:31 -0400 Subject: [PATCH] docs: add PRD for swarm / true multi-agent execution Captures the full architecture plan for moving Moo Code from sequential task delegation to genuine concurrent in-process swarm, modelled on the claude-code teammate/swarm implementation. Phases: P1 - Replace clineStack with concurrent tasks: Map, remove single-open invariant P2 - SwarmRegistry, AgentIdentity, round-robin color assignment P3 - InMemoryMailbox, idle worker loop, task_assignment/idle_notification P4 - LeaderPermissionBridge, worker badge in tool-ask UI P5 - FileMailbox with lockfile for cross-process workers P6 - CliWorkerBackend, moo-worker entry point, spawn_swarm tool Includes data model changes, new event names, updated API surface for spawn_parallel_tasks (concurrent flag) and new spawn_swarm tool spec, UI changes, phased delivery estimates, and risk/mitigation table. Co-Authored-By: Claude Sonnet 4.6 --- docs/prd-swarm-multi-agent.md | 487 ++++++++++++++++++++++++++++++++++ 1 file changed, 487 insertions(+) create mode 100644 docs/prd-swarm-multi-agent.md diff --git a/docs/prd-swarm-multi-agent.md b/docs/prd-swarm-multi-agent.md new file mode 100644 index 0000000000..90f7a43d20 --- /dev/null +++ b/docs/prd-swarm-multi-agent.md @@ -0,0 +1,487 @@ +# PRD: Swarm / True Multi-Agent Execution in Moo Code + +**Status:** Draft +**Author:** davincidreams +**Branch:** `feat/swarm-prd` +**Created:** 2026-04-25 + +--- + +## 1. Problem Statement + +Moo Code today runs tasks **strictly sequentially**. When `spawn_parallel_tasks` is used, +children execute one at a time via a queued delegation chain (`parallelQueue` on `HistoryItem`). +The root cause is the `clineStack: Task[]` LIFO invariant in `ClineProvider`, which enforces +"only one task open at a time" and disposes the parent before the child starts. + +This means: + +- A 4-task "parallel" job takes 4× the wall-clock time of the longest subtask. +- Workers cannot communicate with each other or the leader while running. +- There is no concept of idle workers waiting for the next task from a shared queue. +- Spawning an agent in an isolated environment (separate process / window) is impossible. + +The goal of this PRD is to close the gap with claude-code's **swarm/teammate** architecture and +deliver genuine multi-instance, multi-agent execution inside Moo Code. + +--- + +## 2. Goals + +| # | Goal | Success metric | +| --- | ---------------------------- | --------------------------------------------------------- | +| G1 | True in-process concurrency | N tasks run simultaneously, wall time ≈ max(subtask time) | +| G2 | Swarm identity model | Each agent has stable ID, name, color; visible in UI | +| G3 | Agent-to-agent communication | Leader can send tasks/messages to named workers | +| G4 | Permission delegation | Worker tool-use approvals surfaced in leader's VS Code UI | +| G5 | External process backend | Spawn headless CLI workers or new VS Code windows | +| G6 | Fault isolation | One failing worker does not kill the swarm | + +**Non-goals (this version):** + +- Distributed execution across machines +- Custom agent personas / system-prompt authoring UI +- Persistent cross-session team identities (teams dissolve when the session ends) + +--- + +## 3. Background: How claude-code Solves This + +Reference implementation: `C:\Users\lisam\laud code\claude-code\src\utils\swarm\` + +### 3.1 Concurrent task map + +claude-code stores all running agents in a flat `AppState.tasks: Record` map. +There is no stack, no single-open invariant. Any number of `InProcessTeammateTask` objects can +be `status: "running"` simultaneously. + +### 3.2 AsyncLocalStorage context isolation + +Each in-process teammate runs inside `runWithTeammateContext(context, fn)` which uses Node.js +`AsyncLocalStorage`. Every async operation on any call stack descending from that invocation +sees the same `TeammateContext` — no cross-agent state bleed. + +```typescript +// teammateContext.ts +const teammateContextStorage = new AsyncLocalStorage() + +export function runWithTeammateContext(ctx: TeammateContext, fn: () => T): T { + return teammateContextStorage.run(ctx, fn) +} +``` + +Note: Moo Code `Task` instances are already class-isolated (each holds its own messages, API +client, history). `AsyncLocalStorage` is needed primarily for utility functions that look up +"current context" without being passed an explicit reference. + +### 3.3 Two-level abort + +Each teammate has: + +- `abortController` — kills the whole agent lifecycle +- `currentWorkAbortController` — aborts only the current LLM turn (like pressing Escape), + leaving the agent alive and idle + +### 3.4 Idle loop + mailbox polling + +After finishing a turn, a worker does **not** complete. It enters an idle loop that polls a +file-based mailbox (`~/.claude/teams/{team}/inboxes/{agent}.json`) every 500 ms for: + +- A new task message from the leader +- A shutdown request +- A permission response + +### 3.5 File-based mailboxes with locking + +``` +~/.claude/teams//inboxes/.json +``` + +Writes use `proper-lockfile` with retry logic to prevent concurrent corruption. Message types: +`idle_notification`, `shutdown_request`, `task_assignment`, `permission_request`, +`permission_response`. + +### 3.6 Permission bridge + +A module-level singleton (`leaderPermissionBridge.ts`) lets in-process workers inject a +`WorkerPermissionRequest` directly into the leader's `ToolUseConfirm` queue — the same dialog +the human uses — with a colored worker badge showing which agent is asking. + +### 3.7 Backend registry + +``` +detectAndGetBackend(): + priority 1: already inside tmux → TmuxBackend (native splits) + priority 2: iTerm2 with it2 CLI → ITermBackend (native splits) + priority 3: any tmux available → TmuxBackend (external session) + priority 4: headless / no terminal → InProcessBackend +``` + +--- + +## 4. Proposed Architecture + +### 4.1 Phase 1 — Concurrent task map (foundation) + +**The single highest-impact change.** Remove the `clineStack: Task[]` LIFO invariant. + +```typescript +// ClineProvider.ts — replace line 140 +// FROM: +private clineStack: Task[] = [] + +// TO: +private tasks: Map = new Map() +private focusedTaskId?: string // which task the VS Code UI is showing +private leaderTaskId?: string // root leader of the current swarm session +``` + +Changes cascade through: + +- `getCurrentTask()` → returns `tasks.get(focusedTaskId)` (UI focus) +- `addClineToStack()` → `registerTask(task)` — adds to map, emits `TaskFocused` +- `removeClineFromStack()` → `unregisterTask(taskId)` — removes from map; if + leader is unregistered, tears down swarm +- `delegateParentAndOpenChild()` — **stops disposing parent**; instead sets + `focusedTaskId` to child, parent remains `status: "delegated"` but alive in map +- `createTask()` — no longer enforces single-open invariant for swarm spawns + +Estimated scope: ~400 LOC in `ClineProvider.ts` + updates to `webviewMessageHandler.ts`, +event emitters, and any call site that assumes `clineStack.length <= 1`. + +### 4.2 Phase 2 — Swarm identity and registry + +New file: `src/core/swarm/SwarmRegistry.ts` + +```typescript +export type AgentColorName = "red" | "blue" | "green" | "yellow" | "purple" | "orange" | "pink" | "cyan" + +export interface AgentIdentity { + agentId: string // "@" + agentName: string + teamName: string + color: AgentColorName + isLeader: boolean + taskId: string // linked Moo Code task ID +} + +export interface SwarmSession { + sessionId: string + leaderTaskId: string + teammates: Map // keyed by agentId + taskList: string[] // unclaimed tasks (worker-pull model) +} + +export class SwarmRegistry { + private sessions: Map = new Map() + private colorIndex = 0 + private readonly COLORS: AgentColorName[] = [ + "red","blue","green","yellow","purple","orange","pink","cyan" + ] + + createSession(leaderTaskId: string): SwarmSession { … } + assignColor(agentId: string): AgentColorName { … } // round-robin + registerTeammate(sessionId: string, identity: AgentIdentity): void { … } + unregisterTeammate(sessionId: string, agentId: string): void { … } + getSession(sessionId: string): SwarmSession | undefined { … } + destroySession(sessionId: string): void { … } +} +``` + +The registry lives on `ClineProvider` (one per workspace). The `AgentIdentity` is stored in +`HistoryItem` (extend schema) so it survives process restart. + +### 4.3 Phase 3 — Mailbox communication + +New file: `src/core/swarm/MailboxService.ts` + +Two implementations behind one interface: + +```typescript +export interface IMailboxService { + send(to: string, msg: TeammateMessage): Promise + read(agentName: string): Promise + markRead(agentName: string, idx: number): Promise +} +``` + +- **`InMemoryMailbox`** — for in-process workers; uses a `Map` + with async mutex instead of file locks. Zero latency. +- **`FileMailbox`** — for cross-process/cross-window workers; stores at + `~/.roo/teams//inboxes/.json` with `proper-lockfile`. + +Message types to implement first: + +| Type | Direction | Purpose | +| --------------------- | --------------- | ---------------------------- | +| `task_assignment` | leader → worker | Assign a new task | +| `idle_notification` | worker → leader | "I finished, ready for more" | +| `shutdown_request` | leader → worker | "Stop after current turn" | +| `permission_request` | worker → leader | Request tool-use approval | +| `permission_response` | leader → worker | Approval/rejection result | + +### 4.4 Phase 4 — Idle worker loop + +Extend `Task.ts` with a `runSwarmWorkerLoop()` method. After each LLM turn completes, instead of +marking the task done, the worker: + +1. Sends `idle_notification` to leader with summary. +2. Polls mailbox every 500 ms waiting for `task_assignment` or `shutdown_request`. +3. On `task_assignment`: adds user message to conversation, re-enters + `recursivelyMakeClineRequests`. +4. On `shutdown_request`: marks task completed after current turn. +5. On leader abort signal: hard-stops immediately. + +This mirrors `waitForNextPromptOrShutdown` in `inProcessRunner.ts:689`. + +Two-level abort (mirrors claude-code): + +```typescript +class Task { + readonly lifecycleAbortController = new AbortController() // kills whole worker + // currentRequestAbortController already exists — rename to turnAbortController + // (aborts only the ongoing LLM stream, worker stays alive) +} +``` + +### 4.5 Phase 5 — Permission bridge + +New file: `src/core/swarm/LeaderPermissionBridge.ts` + +```typescript +export type WorkerPermissionRequest = { + requestId: string + workerTaskId: string + agentName: string + color: AgentColorName + toolName: string + toolUseId: string + input: Record + description: string + onAllow(updatedInput?: Record): void + onReject(reason?: string): void +} + +// Module-level singleton — workers inject, leader UI reads +let pendingRequests: WorkerPermissionRequest[] = [] +let notifyLeader: (() => void) | null = null + +export function registerLeaderNotifier(fn: () => void): void { + notifyLeader = fn +} +export function submitPermissionRequest(req: WorkerPermissionRequest): void { + pendingRequests.push(req) + notifyLeader?.() +} +export function consumePendingRequests(): WorkerPermissionRequest[] { + const copy = [...pendingRequests] + pendingRequests = [] + return copy +} +``` + +The VS Code UI renders pending worker permission requests as either: + +- An inline badge in the existing tool-approval ask UI (preferred), or +- A VS Code `showInformationMessage` with Allow/Reject buttons (fallback) + +### 4.6 Phase 6 — External process backend + +For full OS-level isolation (separate heaps, separate git worktrees): + +**Option A — Headless CLI worker** (lighter, recommended first): + +```bash +moo-worker \ + --agent-id researcher@my-team \ + --team-name my-team \ + --parent-session $SESSION_ID \ + --worktree ~/.roo/worktrees/project-a1b2c3d4 \ + --mailbox-dir ~/.roo/teams/my-team/inboxes \ + --model claude-sonnet-4-6 +``` + +The worker process reads its first task from its mailbox file and uses `FileMailbox` for +bidirectional communication. Mirrors claude-code's `PaneBackendExecutor.spawn()`. + +**Option B — New VS Code window** (full extension, heavier): + +```typescript +vscode.commands.executeCommand("vscode.openFolder", worktreeUri, { + forceNewWindow: true, +}) +// + write initial task to mailbox before window opens +``` + +**Backend registry** (mirrors claude-code `registry.ts`): + +```typescript +export interface IWorkerBackend { + spawn(config: WorkerSpawnConfig): Promise + terminate(workerId: string): Promise + isActive(workerId: string): Promise +} + +// Priority order: +// 1. InProcessBackend — default; no extra dependencies +// 2. CliWorkerBackend — opt-in; requires moo-worker binary +// 3. VsCodeWindowBackend — opt-in; spawns new VS Code window +``` + +--- + +## 5. Data Model Changes + +### 5.1 HistoryItem extensions + +```typescript +// packages/types/src/history.ts — additions +agentId?: string // "@" if this task is a swarm worker +agentName?: string +agentColor?: string +teamName?: string +swarmSessionId?: string +isSwarmLeader?: boolean +isIdle?: boolean // worker is alive but waiting for next task +``` + +### 5.2 New types package exports + +```typescript +// packages/types/src/swarm.ts (new file) +export type AgentColorName = "red" | "blue" | "green" | "yellow" | "purple" | "orange" | "pink" | "cyan" + +export interface AgentIdentity { … } +export interface SwarmSession { … } +export interface TeammateMessage { … } +export type TeammateMessageType = + | "task_assignment" + | "idle_notification" + | "shutdown_request" + | "permission_request" + | "permission_response" +``` + +### 5.3 New event names + +```typescript +// packages/types/src/events.ts — additions to RooCodeEventName +SwarmSessionStarted = "swarmSessionStarted" +SwarmSessionEnded = "swarmSessionEnded" +WorkerSpawned = "workerSpawned" +WorkerIdle = "workerIdle" +WorkerShutdown = "workerShutdown" +PermissionRequested = "permissionRequested" // worker→leader +PermissionResolved = "permissionResolved" // leader→worker +``` + +--- + +## 6. API surface (spawn_parallel_tasks evolution) + +Today `spawn_parallel_tasks` runs tasks sequentially. Once Phase 1+2 land, it gains a +`concurrent: true` flag: + +```json +{ + "tool": "spawn_parallel_tasks", + "tasks": [ + { "mode": "code", "message": "implement auth service", "worktree": "auto" }, + { "mode": "code", "message": "implement payment service", "worktree": "auto" }, + { "mode": "code", "message": "implement notification service", "worktree": "auto" } + ], + "concurrent": true, + "abortOnChildFailure": false +} +``` + +When `concurrent: true`, all tasks start immediately against the task map (Phase 1). The parent +registers `onIdle` callbacks on each child and resumes only when all have called +`idle_notification` with a completed status. + +A new `spawn_swarm` tool (Phase 3+) allows richer worker control: + +```json +{ + "tool": "spawn_swarm", + "teamName": "feature-v2", + "workers": [ + { "name": "researcher", "mode": "architect", "color": "blue" }, + { "name": "coder-1", "mode": "code", "color": "green" }, + { "name": "reviewer", "mode": "code", "color": "yellow" } + ], + "taskList": [ + "Research existing auth implementations", + "Implement JWT refresh token flow", + "Review and test the JWT implementation" + ] +} +``` + +Workers pull tasks from the `taskList` atomically; the leader monitors idle notifications and +can push new tasks dynamically. + +--- + +## 7. UI Changes + +### 7.1 Active workers panel + +A new collapsible section in the Chat sidebar shows all running workers with their color dot, +current status (running / idle), and latest tool in use. Mirrors how SubtaskRow renders +child tasks today. + +### 7.2 Worker permission badge + +When a worker requests tool approval, the existing tool-ask dialog renders a colored badge +(`[researcher · blue]`) above the file diff / bash command block. + +### 7.3 Swarm activity log + +The Output Channel gains a `Moo Code Swarm` channel that logs all cross-agent messages, +idle transitions, and permission events in real time. + +--- + +## 8. Phased Delivery + +| Phase | Name | Key deliverables | Estimated effort | +| ----- | ------------------- | ------------------------------------------------------------------------------------------------------------------ | ----------------- | +| P1 | Concurrent task map | Replace `clineStack` with `tasks: Map`; remove single-open invariant; `concurrent: true` on `spawn_parallel_tasks` | Large (1–2 weeks) | +| P2 | Swarm identity | `SwarmRegistry`, `AgentIdentity`, color assignment; UI color dots | Medium (3–5 days) | +| P3 | In-process mailbox | `InMemoryMailbox`, idle loop in `Task`, `idle_notification` / `task_assignment` messages | Medium (3–5 days) | +| P4 | Permission bridge | `LeaderPermissionBridge`, worker badge in tool-ask UI | Medium (3–5 days) | +| P5 | File mailbox | `FileMailbox` with lockfile; cross-process swarm works | Small (2–3 days) | +| P6 | External backends | `CliWorkerBackend`; `moo-worker` entry point; `spawn_swarm` tool | Large (1–2 weeks) | + +--- + +## 9. Risks and Mitigations + +| Risk | Impact | Mitigation | +| ------------------------------------------------------------------------ | ------ | -------------------------------------------------------------------------------------------------------------- | +| Removing single-open invariant breaks webview state assumptions | High | Audit all `getCurrentTask()` call sites; add `getFocusedTask()` as new name to make call sites obvious | +| Concurrent tasks racing on `taskHistoryStore` writes | Medium | Serialize writes via async mutex (one per taskId); reads are safe (immutable snapshots) | +| VS Code extension host is single-threaded — "concurrency" is cooperative | Medium | Tasks yield naturally at `await` points; long CPU-bound turns block others. Acceptable for I/O-bound LLM tasks | +| File mailbox lock contention with many workers | Low | In-process workers use `InMemoryMailbox`; file mailbox only for cross-process | +| External CLI worker binary distribution | Medium | Ship as optional; document manual install; in-process is always available | + +--- + +## 10. Prior Art / References + +- **claude-code swarm source:** `C:\Users\lisam\laud code\claude-code\src\utils\swarm\` + - `inProcessRunner.ts` — idle loop, permission bridge + - `teammateContext.ts` — AsyncLocalStorage pattern + - `backends/registry.ts` — pluggable backend selection + - `teammateMailbox.ts` — file-based mailbox with lockfile + - `leaderPermissionBridge.ts` — module-level permission injection +- **Current Moo Code parallel implementation:** + - `src/core/webview/ClineProvider.ts` — `delegateParentAndOpenChild`, `reopenParentFromDelegation` + - `src/core/tools/SpawnParallelTasksTool.ts` + - `packages/types/src/history.ts` — `parallelQueue`, `parallelResults` +- **Previous improvements (merged in `moo-code-standalone`):** + - Worktree orphan detection + - `abortOnChildFailure` flag + - `getParallelTaskStatus()` introspection API + - Telemetry for worktree/parallel events