mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-08-28 05:27:41 +00:00
* arc(01KK7524KNGTPS4090QMF87FJN): implement (success) Arc-Run: 01KK7524KNGTPS4090QMF87FJN Arc-Completed: 2 Arc-Checkpoint: 1ff03c704805dfe8e7c37b37f97bd06dfa0e5dc5 * Fix: restore trailing newlines stripped by previous commit * arc(01KK7524KNGTPS4090QMF87FJN): simplify (success) Arc-Run: 01KK7524KNGTPS4090QMF87FJN Arc-Completed: 3 Arc-Checkpoint: 21771adfd26283a1d1e6b8a123a83a0c4277db48 --------- Co-authored-by: arc <arc@local> Co-authored-by: Arc Assistant <assistant@arc.dev> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
123 lines
5.8 KiB
Text
123 lines
5.8 KiB
Text
---
|
||
title: "Retros"
|
||
description: "Automatic retrospectives that analyze every workflow run"
|
||
---
|
||
|
||
After every workflow run, Arc generates a **retro** — a structured retrospective that captures what happened, what went well, and what didn't. Retros combine deterministic metrics extracted from the run's checkpoint with a qualitative narrative produced by an LLM agent that analyzes the full event stream.
|
||
|
||
The goal is continuous improvement. Retros give you a searchable history of how your workflows perform over time, surface friction patterns that would otherwise go unnoticed, and identify follow-up work before it falls through the cracks.
|
||
|
||
## What's in a retro
|
||
|
||
A retro has two layers: **quantitative stats** derived cheaply from checkpoint data, and an **agent-generated narrative** that interprets the run holistically.
|
||
|
||
### Quantitative layer
|
||
|
||
The quantitative layer is extracted directly from the [checkpoint](/execution/checkpoints) and event stream — no LLM calls required:
|
||
|
||
| Field | Description |
|
||
|---|---|
|
||
| **Per-stage breakdown** | Duration, retry count, cost, files touched, status, and failure reason for each stage |
|
||
| **Aggregate stats** | Total duration, total cost, total retries, all files touched, stages completed vs. failed |
|
||
|
||
### Narrative layer
|
||
|
||
An LLM agent reads the run's `progress.jsonl` event stream and produces a structured analysis:
|
||
|
||
| Field | Description |
|
||
|---|---|
|
||
| **Smoothness** | Overall rating on a 5-point scale (see below) |
|
||
| **Intent** | What the run was trying to accomplish |
|
||
| **Outcome** | What actually happened |
|
||
| **Learnings** | What was discovered about the repo, code, workflow, or tools |
|
||
| **Friction points** | Where things got stuck and why |
|
||
| **Open items** | Follow-up work, tech debt, test gaps, or investigations identified |
|
||
|
||
The agent has tool access to grep and read the event stream, so it can inspect actual tool call patterns, error messages, and approach pivots — not just pass/fail signals.
|
||
|
||
## Smoothness ratings
|
||
|
||
Every retro includes a smoothness rating that grades the overall quality of the run's execution:
|
||
|
||
| Rating | Meaning |
|
||
|---|---|
|
||
| **Effortless** | Goal achieved on the first try. No retries, no wrong approaches. Agent moved efficiently from start to finish. |
|
||
| **Smooth** | Goal achieved with minor hiccups — 1–2 retries or a brief wrong approach quickly corrected. No human intervention needed. |
|
||
| **Bumpy** | Goal achieved but with notable friction: multiple retries, at least one significant wrong approach, or substantial time on dead ends. |
|
||
| **Struggled** | Goal achieved only with difficulty: many retries, major approach changes, human intervention, or partial failures requiring recovery. |
|
||
| **Failed** | Run did not achieve its stated goal. Some stages may have completed, but the overall intent was not fulfilled. |
|
||
|
||
The rating considers the full context visible in agent events — tool call patterns, error recovery sequences, approach pivots — not just stage pass/fail counts.
|
||
|
||
## Learnings, friction points, and open items
|
||
|
||
### Learnings
|
||
|
||
Learnings capture what was discovered during the run, categorized by type:
|
||
|
||
| Category | Examples |
|
||
|---|---|
|
||
| `repo` | Repository structure, build system quirks, CI configuration |
|
||
| `code` | Bug root causes, module boundaries, API contracts |
|
||
| `workflow` | Node ordering issues, missing stages, prompt improvements |
|
||
| `tool` | Tool limitations, MCP server behavior, command output parsing |
|
||
|
||
### Friction points
|
||
|
||
Friction points identify where the run got stuck and what caused the slowdown:
|
||
|
||
| Kind | Description |
|
||
|---|---|
|
||
| `retry` | A stage needed multiple attempts |
|
||
| `timeout` | A stage or tool call hit a time limit |
|
||
| `wrong_approach` | The agent pursued a dead end before pivoting |
|
||
| `tool_failure` | A tool or command failed unexpectedly |
|
||
| `ambiguity` | Unclear requirements or conflicting signals caused confusion |
|
||
|
||
Each friction point can optionally reference the `stage_id` where it occurred.
|
||
|
||
### Open items
|
||
|
||
Open items capture follow-up work identified during the run:
|
||
|
||
| Kind | Description |
|
||
|---|---|
|
||
| `tech_debt` | Code quality issues worth addressing later |
|
||
| `follow_up` | Work that's related but out of scope for this run |
|
||
| `investigation` | Unknowns that need further research |
|
||
| `test_gap` | Missing test coverage discovered during the run |
|
||
|
||
## How retros are generated
|
||
|
||
Retro generation happens in two phases after a run completes:
|
||
|
||
1. **Derive** — Arc extracts stage durations from `progress.jsonl` and builds a retro from the checkpoint data. This is deterministic, fast, and produces the quantitative layer. The retro is saved immediately as `retro.json` in the run's logs directory.
|
||
|
||
2. **Narrate** — An LLM agent session analyzes the run data. The agent has read access to `progress.jsonl`, `checkpoint.json`, and `manifest.json`. It uses grep and read tools to find interesting signals — failures, retries, errors, approach changes — then calls a `submit_retro` tool with its structured analysis. The narrative fields are merged into the existing retro and saved.
|
||
|
||
Both phases run automatically at the end of every CLI run. The API server derives the quantitative layer but does not currently run the narrative agent.
|
||
|
||
## Accessing retros
|
||
|
||
### CLI
|
||
|
||
Retros are saved to `{logs_dir}/retro.json` after every run. The path is printed at the end of the run output:
|
||
|
||
```
|
||
Retro: smooth — Successfully implemented the feature
|
||
Retro saved to ~/arc-logs/01JKXYZ.../retro.json
|
||
```
|
||
|
||
To skip retro generation, pass `--no-retro`:
|
||
|
||
```bash
|
||
arc run workflow.dot --no-retro
|
||
```
|
||
|
||
### API
|
||
|
||
Retros are also available via the REST API. See the [Retros API reference](/api-reference/retros) for endpoints to retrieve a single run's retro or list retros across all runs.
|
||
|
||
## Storage
|
||
|
||
Retros are stored as `retro.json` in the run's logs directory alongside `checkpoint.json` and `progress.jsonl`. They are plain JSON files — easy to parse, query, or pipe into other tools.
|