web_fetch now converts HTML responses to clean markdown using the htmd
crate (stripping script/style tags), and supports an optional prompt
parameter that makes a secondary LLM call to answer questions about the
fetched content. Each provider profile picks a cheap/fast summarizer
model (Haiku, gpt-4o-mini, gemini-2.0-flash).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Let the agent autonomously load skill templates when it recognizes a
matching task, instead of requiring users to type /skill-name. The
system prompt now instructs the agent to call `use_skill` and skill
names use backtick formatting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enables machine-readable output from the agent CLI by streaming
SessionEvent objects as newline-delimited JSON to stdout. This
unlocks scripting, integration testing, and UI integration.
- Make SessionEvent serializable with ISO-8601 timestamps via chrono
- Add OutputFormat enum (text/json) and --output-format CLI flag
- JSON mode: each event is one JSON line to stdout, flushed per line
- JSON mode: skip print_output/print_summary (all info in event stream)
- Text mode (default): behavior unchanged
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add gpt-5.3-codex to catalog (API model, 1047576 context, 128K output)
- Move "codex" alias from gpt-5.2-codex to gpt-5.3-codex
- Add e2e integration test for gpt-5.3-codex via OpenAI API
- Remove gpt-5.3-codex-spark (not yet available)
- Empty CLI_ONLY_MODELS list and update related tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Test both tools across all three providers (Anthropic, OpenAI, Gemini).
web_fetch fetches example.com and asserts content is written to a file.
web_search searches for "Rust programming language" and asserts results
are saved.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the placeholder web_fetch tool with a working implementation
that executes curl within the execution environment, respecting
Docker/Daytona network sandboxing.
- Build curl command with shell-escaped URL, follow redirects, custom
user agent, and configurable timeout (default 30s, max 60s)
- Validate URL scheme (http/https only) to prevent misuse
- Truncate responses exceeding 100KB to protect context window
- Register web_fetch in Anthropic and OpenAI profiles (was Gemini-only)
- Add web_fetch guidance to all three profile system prompts
- Add captured_command to MockExecutionEnvironment for test assertions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SubAgentSpawned/Completed/Failed/Closed/Event variants to AgentEvent
with a deferred callback mechanism on SubAgentManager. Child session
events are subscribed to at spawn time and forwarded as wrapped
SubAgentEvent (skipping streaming noise). Wired in both the standalone
CLI and the pipeline AgentBackend.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The rg availability probe (rg --version / which rg) was running on every
grep() call. Cache the result in a OnceLock/OnceCell per environment so
the probe runs at most once. Also add grep -rn fallback to Daytona env,
matching the pattern already used by local and Docker envs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Moves build_system_prompt() from run_single_input() to initialize(),
storing the result as a Session field. This makes the prompt's
static-ness explicit and guarantees Anthropic cache breakpoint 1
always hits. Internal methods (build_request, check_context_usage,
compact_context, estimate_token_count) no longer take a system_prompt
parameter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the placeholder web_search tool with a real implementation backed
by the Brave Search API. Register it in all three profiles (Anthropic,
OpenAI, Gemini) so every provider has web search capability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ToolResult::success()/error() constructors, replacing 16 manual
construction sites across tools.rs, session.rs, history.rs, generate.rs
- Collapse ContentPart::RedactedThinking into Thinking (use redacted field)
- Extract HttpApi base struct shared by all 4 provider adapters
- Extract LineReader into common.rs for shared SSE byte buffering and
timeout handling; convert Anthropic/OpenAI from BoxStream to Response
- Replace manual delegation methods on GenerateResult/StepResult with
Deref<Target=Response>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Merges two small utility crates into a single `util` crate to reduce
workspace clutter. No behavioral changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused items that have no callers outside their own tests:
middleware wrap_stream_with_middleware/process_stream_event,
common ApiMessage/send_and_read_body, types ProviderEvent variant,
lib CancellationToken re-export, tools execute_all_tools, and
catalog get_latest_model.
Add --schema/-S flag to `ullm prompt` so generate_object() and
stream_object() are exercisable end-to-end. Includes unit test for
invalid JSON rejection and two ignored integration tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register spawn_agent, send_input, wait, and close_agent tools in both
AgentBackend::create_session and the agent CLI run() so subagents are
available outside of tests. Child sessions inherit the parent's
tool_approval callback but omit subagent tools to prevent recursive
spawning.
Also classifies subagent tools as auto-approved at all permission levels
and removes unused id/depth fields from SubAgent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace eager Regex compilation in GitleaksEngine::build() with
OnceLock-based lazy compilation. Individual regexes (rule patterns,
allowlist patterns, global allowlist patterns) are now compiled on
first use rather than at startup. This drops dry-run time from ~1.4s
to ~0.5s since the common no-match path never compiles any regexes.
Also removes the warm_up() pre-loading function and its callers since
lazy compilation makes it unnecessary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a pipeline revisits a node (goal gate retries, loops), each visit
now gets a distinct stage directory instead of silently overwriting the
previous one. First visit keeps the clean `nodes/{id}/` path; visit 2+
produces `nodes/{id}-attempt_{n}/`.
The engine always tracks visit counts and sets
`internal.node_visit_count` in context. Handlers read the count via
`visit_from_context()` and pass it to the updated `node_dir()`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Runs the pipeline engine with a live Daytona sandbox: handler produces
150KB output, engine offloads to local artifact store, syncs to the
sandbox, and the checkpoint pointer references the remote path. Verifies
the file is readable in the sandbox.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update daytona-sdk-rust to 5d370099 which fixes multipart file upload
(upload_file was sending an empty form). Add E2E test that verifies
artifact pointers are rewritten and files uploaded to a live Daytona
sandbox.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After offloading large context values to local disk, check whether each
artifact file is accessible in the execution environment. When it isn't
(Docker/Daytona), read the local file and upload it via write_file,
rewriting the pointer to the remote path. Local envs skip the sync
since file_exists returns true for local paths.
Also switches the pointer prefix from artifact:// to file://.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Node stage directories ({logs_root}/{node_id}/) could collide with
{logs_root}/artifacts/ if a node has id="artifacts". Fix by nesting
all node directories under a nodes/ subdirectory so the layout becomes:
{logs_root}/manifest.json
{logs_root}/checkpoint.json
{logs_root}/nodes/{node_id}/status.json, prompt.md, response.md, ...
{logs_root}/artifacts/{key}.json
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After a handler returns, the engine automatically offloads context values
exceeding 100KB into the ArtifactStore, replacing them with artifact://
pointer strings. This prevents bloating Context, Checkpoints, and
status.json files. The preamble renders "See: {path}" for artifact
pointers instead of inlining large content.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Drop the `tool.output` context key (unused alias from tool→script rename)
and rewrite the failure branch to include stdout in both `failure_reason`
and `script.output` context, so build/test output isn't lost on failure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
summary:medium now uses compact handler-specific rendering (script
command/stdout/stderr, codergen model/files) instead of dumping raw
context_updates. summary:low adds minimal handler info (type, script
command, or model name) to each stage line.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use ? operator instead of if-let-err pattern in backend.rs
- Collapse nested if blocks in preamble.rs summary:medium path
- Allow too_many_arguments on CodergenBackend trait and llm_evaluate fn
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Compact mode now renders nested-bullet summaries with handler-specific
sub-items (script: command/stdout/stderr; codergen: model/tokens/files)
under a ## Completed stages heading. Summary:high renders per-stage
## Stage sections with full detail and a ## Current context table.
Also captures script.stderr in context_updates for both success and
failure branches, and improves context filtering across all modes to
exclude engine keys (graph.*, thread.*, response.*, last_stage, etc.).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CodergenHandler now reads `current.preamble` from context and prepends it
to the prompt before dispatching to backends. For `full` fidelity the engine
sets preamble to empty, so the prepend is a no-op. This ensures downstream
LLM nodes see prior node outputs (e.g. script stdout) without any trait
changes.
AgentBackend gains a `sessions` cache keyed by thread ID. When fidelity is
`full` and a thread_id is present, sessions are reused across nodes so the
LLM sees the full conversation history. Usage aggregation only counts turns
added after the reuse point to prevent double-counting. Failed sessions are
dropped rather than cached.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused set_event_callback on MockExecutionEnvironment and
replace always-true u64 >= 0 comparisons with is_u64() checks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Exercises the old tool_command attribute through validate and
dry-run to catch regressions in the tool → script compatibility.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register "tool" as an alias for ScriptHandler, fall back to
tool_command attribute, dual-write tool.output context key, and
add "tool" to known handler types for validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename the entire "tool" concept to "script": ToolHandler → ScriptHandler,
tool_command attribute → script, tool.output → script.output, and all
related artifact filenames. Add a language attribute (shell | python,
default shell) so script nodes can run Python via python3 -c natively.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SSH git URLs (git@github.com:...) cause "invalid auth method" errors
since Daytona uses HTTPS token auth. Added ssh_url_to_https() to
convert them before cloning. Also removed the static sandbox name
config field and auto-generate unique names with timestamps to prevent
"already exists" errors on repeated runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces ExecutionEnvEvent enum (15 variants covering lifecycle, Docker
image pull, Daytona snapshot, and git clone operations) with callback
injection on all execution environment implementations. Adds
PipelineEvent::ExecutionEnv wrapper and Setup* variants for setup command
tracking. Wires callbacks in run.rs so exec env and setup events flow
through the pipeline event emitter to stderr and progress.ndjson.
Also changes PipelineEngine::new to accept Arc<EventEmitter> so the
emitter can be shared with exec env callbacks before engine construction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of creating a separate Docker/Daytona sandbox per stage inside
AgentBackend::run(), create the execution environment once in
run_command(), initialize it, run setup commands, and pass it through
EngineServices to all handlers. This lets multi-stage pipelines share
file changes across nodes and avoids redundant container/sandbox creation.
Key changes:
- Add execution_env field to EngineServices
- Add execution_env param to CodergenBackend::run() trait method
- Remove env creation, setup_commands, daytona_config from AgentBackend
- Lift env lifecycle (create, initialize, setup, cleanup) to run_command()
- Use scopeguard for cleanup on error/panic paths
- Update PipelineEngine constructors to accept execution_env
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 92cbe0d3c133
Allow specifying execution environment (local/docker/daytona) in TOML
task configs so pipelines are self-contained. CLI flag takes precedence
over TOML, which takes precedence over the default (local).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 600b7720bbda
Replace the `--docker` boolean flag with `--execution-env <local|docker|daytona>`
to support three execution environments. The new `DaytonaExecutionEnvironment`
uses the Daytona cloud sandbox SDK to run agent tools remotely, auto-cloning the
current git repo into the sandbox via `gh auth token` credentials.
Setup commands from TOML task configs now run inside the execution environment
(via `exec_command` after `initialize()`) rather than locally, so they work
correctly for both Docker and Daytona sandboxes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 306da8ff6de1
Allows users to define variables in their TOML task config that get
expanded as $name placeholders in the DOT file before parsing. This
enables parameterized pipelines without duplicating DOT files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 293af13a0e54
Support `attractor run task.toml` as an alternative to `.dot` files.
Auto-detects format by file extension. TOML bundles pipeline config
(graph path, model, setup commands, working directory) into a single
file with precedence: CLI flag > TOML > DOT graph attrs > defaults.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 8744d57d6053
Adds 8 log file outputs to reach parity with Kilroy's per-node and
run-level logging:
- panic.txt: written on handler panic with the panic message
- graph.dot: DOT source saved to logs dir at run start
- run.pid: process ID saved to logs dir at run start
- final.json: run outcome (status, duration, failure_reason) written
after engine completes, before error propagation
- parallel_results.json: branch results written per parallel node
- provider_used.json: mode/provider/model written by AgentBackend
- api_request.json: serialized LLM request in one_shot mode
- api_response.json: serialized LLM response in one_shot mode
Adds stage_dir parameter to CodergenBackend trait methods so backends
can write artifacts to the correct per-node log directory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: a3934db0f826
ToolHandler now writes 4 files to {logs_root}/{node_id}/:
- tool_invocation.json (command, timeout_ms) before execution
- stdout.log and stderr.log after command completes
- tool_timing.json (duration_ms, exit_code, timed_out) after execution
The timeout branch also writes tool_timing.json with timed_out: true
and exit_code: null. Inlined process_output since stdout/stderr are
now needed in the surrounding scope for file writes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 6c9fde8bdc72
Enables simple generation nodes (summarization, classification, routing)
without the overhead of a full agent session. Nodes default to agent_loop
when the attribute is absent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 1574218eca96
Enrich three AgentEvent variants with payload data (UserInput.text,
TurnLimitReached.max_turns, SteeringInjected.text) and forward
UserInput/SteeringInjected to the pipeline event stream. Add cost
field to StageUsage (computed from model pricing catalog) and
total_cost to PipelineCompleted (summed from stage outcomes).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: adb2363c9ec4
Apply redact::redact_jsonl_line at all three serialization sites so
secrets (AWS keys, GitHub PATs, private keys, etc.) are scrubbed
before reaching disk or the network.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: e23c92750cd9
Cyclic graphs (e.g. consensus_task.dot, semport.dot) loop infinitely
under --dry-run because the mock backend always returns outcome=success,
which never matches conditional edges. Add a configurable max_node_visits
limit (graph attr, default 0=disabled) that defaults to 10 in dry-run
mode, terminating execution when any node is visited more than the limit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 633b59435252
