Commit graph

16 commits

Author SHA1 Message Date
Bryan Helmkamp
1e3726b7f3 Add missing data to agent events and cost tracking in pipeline logs
Enrich three AgentEvent variants with payload data (UserInput.text,
TurnLimitReached.max_turns, SteeringInjected.text) and forward
UserInput/SteeringInjected to the pipeline event stream. Add cost
field to StageUsage (computed from model pricing catalog) and
total_cost to PipelineCompleted (summed from stage outcomes).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: adb2363c9ec4
2026-02-25 17:17:13 -05:00
Bryan Helmkamp
74a6b219cc Fix spec: FAIL does not trigger retries, only RETRY does
The test criteria incorrectly said FAIL outcomes should be retried.
The pseudocode (Section 3.5) and the C reference implementation both
return immediately on FAIL. Only RETRY status triggers the retry loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 16:19:10 -05:00
Bryan Helmkamp
94326f5438 docs/specs 2026-02-23 10:17:40 -05:00
Bryan Helmkamp
d3e396753c Remove stale review docs, add spec DoD pipeline definitions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 09:47:50 -05:00
Bryan Helmkamp
30cf3a6787 Rename coding-agent-loop crate to agent
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 09:17:48 -05:00
Bryan Helmkamp
5932b500e7 Implement GET /pipelines/{id}/graph endpoint for SVG rendering
Stores DOT source in ManagedPipeline and pipes it through `dot -Tsvg`
on request. Returns image/svg+xml on success, 502 if graphviz is
unavailable, 404 if pipeline not found. Resolves spec gap #1.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 09:13:36 -05:00
Bryan Helmkamp
54d0c887dc Add end-to-end test for subgraph class derivation, update spec gap analysis
Closes the subgraph class derivation test gap by adding a parse() pipeline
test that verifies DOT subgraph labels produce correct CSS-like classes on
contained nodes. Updates gap analysis to remove resolved items, add spec
contradictions section, and narrow remaining gaps to SVG rendering and
retry predicate customization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 08:39:42 -05:00
Bryan Helmkamp
a917a8d615 Implement 6 spec gaps: preamble synthesis, thread_id plumbing, cancellation, recording replay, retry presets, inform()
- Gap 1: Real preamble synthesis per fidelity mode (truncate, compact,
  summary:low/medium/high) in new preamble.rs module, replacing placeholder
- Gap 2: Pass thread_id to CodergenBackend.run() so backends can reuse
  LLM sessions across nodes sharing the same thread
- Gap 7: Engine cancellation via AtomicBool token checked between nodes,
  wired to server cancel endpoint, new Cancelled error variant
- Gap 8: RecordingInterviewer serialization (to_json/from_json, file I/O)
  and new ReplayInterviewer for replaying recorded Q&A sessions
- Gap 12: Preset retry policies selectable from DOT via retry_policy attr
  (none, standard, aggressive, linear, patient)
- Gap 15: Engine calls Interviewer.inform() at pipeline start, stage
  start, and stage complete lifecycle points

37 new tests (450 unit + 49 integration, all passing).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 08:23:27 -05:00
Bryan Helmkamp
a2f9e87b7c Close spec compliance gaps: retry context, thread resolution, fan-in logging
Fix 3 confirmed gaps from spec compliance review (85 items, 91.8% aligned):

- Write internal.retry_count.<node_id> to PipelineContext after retries
  so handlers and conditions can access retry counts (spec 5.1)
- Add graph-level default_thread (step 3) to 5-step thread ID resolution,
  pass graph param to resolve_thread_id (spec 5.4)
- Write prompt.md/response.md in fan_in LLM evaluation path (spec 5.6)

Also includes pre-existing improvements: checkpoint stores node_outcomes
and next_node_id for correct resume, engine timeout enforcement,
auto_status support, fidelity degradation on resume, preamble injection,
is_retryable error classification, stylesheet specificity correction,
full stylesheet parse validation, direction_valid lint rule, pre-hook
returns Skipped not Fail, fan-in score-based sorting and all-fail
detection, manager_loop child autostart and steer cooldown.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 10:49:21 -04:00
Bryan Helmkamp
34836200e5 Implement attractor crate: DOT-based pipeline runner with full spec compliance
Adds the attractor crate implementing all 11 sections of the attractor spec:
- DOT parser (lexer, grammar, semantic analysis) for strict DOT subset
- Pipeline execution engine with edge selection, goal gates, retry logic,
  failure routing, checkpoint save/resume, and loop_restart
- 9 node handlers: start, exit, codergen, wait_human, conditional, parallel
  (concurrent with join/error policies), fan_in (with LLM eval), tool, manager_loop
- State management: PipelineContext, Outcome, Artifact store, fidelity resolution
- Human-in-the-loop: Interviewer trait with auto_approve, callback, queue,
  recording, and console implementations, plus timeout enforcement
- Validation: 14 built-in lint rules with custom rule registration API
- Model stylesheet with universal/shape/class/ID selectors and specificity
- Transforms: variable expansion, stylesheet application, preamble; plus
  PipelineBuilder with register_transform and prepare_pipeline
- Condition expression language with =, !=, bare-key truthiness, && combinator
- Event system with all 16 event types emitted by engine and handlers
- Tool call hooks (pre/post) for CodergenHandler
- Run directory with manifest.json and per-node status.json

370 tests (354 unit + 16 integration) covering all spec sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 12:17:32 -04:00
Bryan Helmkamp
db32f00cd0 Simplify coding-agent-loop: deduplicate mocks, narrow traits, type events
- Extract shared test infrastructure (MockExecutionEnvironment, TestProfile,
  MockLlmProvider) replacing 11 duplicate mock implementations across tests
- Deduplicate tool execution logic between sequential and parallel paths
- Narrow ProviderProfile trait from 14 to 7 required methods via
  ProfileCapabilities struct and default implementations
- Replace stringly-typed HashMap event data with typed EventData enum
- Extract shared assemble_system_prompt helper and register_subagent_tools
  default method, eliminating copy-paste across all 3 profiles
- Replace fragile shell-based glob with glob crate, fix rg detection
- Add delete_file to ExecutionEnvironment, wire git context into env block
- Remove dead code (AgentError::Io, count_turns, trivial derived-trait tests)
- Use match-based lookups in truncation instead of per-call HashMap allocation

Net reduction: -1,401 lines across 20 files. All 180 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:34:29 -04:00
Bryan Helmkamp
1ffc954a0f Add knowledge_cutoff to ProviderProfile, fix subagent default max_turns
- Add knowledge_cutoff() method to ProviderProfile trait so session can
  populate EnvContext from the profile instead of leaving it empty
- Set subagent default max_turns to 50 per spec (was using session
  factory default which could be 0/unlimited)
- Add corrected spec compliance review after manual verification found
  the initial 5-agent review was largely false positives

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:02:20 -04:00
Bryan Helmkamp
82572678c0 Fix model alias resolution and invalid gpt-5.2-mini model ID
resolve_model() was passing raw alias strings (e.g. "gpt5") directly to
APIs instead of resolving them to actual model IDs (e.g. "gpt-5.2").
Also rename gpt-5.2-mini to gpt-5-mini, which is the correct OpenAI
model ID.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 14:01:30 -04:00
Bryan Helmkamp
978a477d1e Fix 8 spec compliance gaps in unified-llm
- Enforce stream_read timeout (30s default) in all 4 providers' streaming code
- Add with_timeout() builder method to all adapter constructors
- Fix ResponseFormatType::JsonObject to serialize as "json" per spec
- Add STEP_FINISH to StreamEventType enum in spec doc
- Add UnsupportedToolChoice error and enforce in all adapters via validate_tool_choice()
- Fix error classification to check status code before message content
- Add stop_sequences support to OpenAI Responses API adapter
- Handle Gemini thought parts (thought: true) in both complete and streaming paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 12:41:41 -04:00
Bryan Helmkamp
ed07d43335 Implement spec gaps: rate limit headers, error classification, total timeout, metadata, stream_object
- Parse x-ratelimit-* headers into RateLimitInfo for Anthropic, OpenAI, and
  OpenAI-compatible providers (previously hardcoded to None)
- Add "not found"/"does not exist" and "unauthorized"/"invalid key" error
  message classification patterns for ambiguous HTTP status codes
- Apply TimeoutConfig.total to wrap the entire multi-step generate() loop
  (previously only per_step was used)
- Add metadata field to GenerateParams with builder method, pass through to
  Request instead of hardcoding None
- Implement stream_object() for streaming structured output with incremental
  JSON parsing via new ObjectStreamEvent type (Partial/Delta/Complete variants)
- Add OpenAI-compatible Chat Completions adapter for third-party endpoints

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 11:05:10 -04:00
Bryan Helmkamp
40be740949 specs 2026-02-19 13:41:55 -04:00