When an LLM provider returns a transient error (rate limit, server error,
quota exceeded, timeout, network, or stream failure), Arc now automatically
retries on fallback providers using the closest matching model from the
catalog based on capability filters and cost proximity.
Key changes:
- closest_model() and build_fallback_chain() in arc-llm catalog
- failover_eligible() on SdkError to classify transient vs deterministic errors
- fallbacks config field on LlmConfig with task-wins-over-defaults merging
- WorkflowRunEvent::Failover variant for observability
- Failover logic in both one_shot and agent session (run) code paths
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reuse expand_vars() from run_config to scan for $identifier patterns
in codergen prompt expansion. Unknown variables like $gaol now produce
an ArcError::Validation at runtime instead of silently passing through.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces 10 identical copies of the pagination block with a single
shared function. Also eliminates a redundant second collect() by
using truncate() instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Apply the same page[limit]/page[offset] pagination pattern from
GET /runs to: workflows, workflow runs, retros, sessions, projects,
branches, saved queries, query history, and stage turns.
Each endpoint now returns { data, meta: { has_more } } instead of
a bare array. Includes OpenAPI spec updates, demo handler changes,
regenerated TS client, updated frontend consumers, and a new
pagination conformance test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
load_server_config() now accepts an optional explicit path. When
provided, it reads from that path (erroring if missing) instead of
the default ~/.arc/server.toml. The --config flag is wired through
ServeArgs and the hot-reload polling loop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- AppState.server_config was stored but never read; remove it and revert
create_app_state_with_options back to 5 parameters
- Config polling now compares under a read lock first, only acquiring
the write lock when a change is detected
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Poll ~/.arc/server.toml every 5s and swap run_defaults/git config for new
runs without restarting. CLI overrides (--model, --provider) always win.
On parse error, log a warning and keep the previous config.
Also add a `default` field to the model catalog so default model resolution
uses catalog data instead of hardcoded model names in Rust code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Runs now go through a state machine (queued → starting → running →
completed/failed/cancelled) instead of spawning immediately. A background
scheduler promotes queued runs when capacity is available, defaulting to
4 concurrent runs. Configurable via --max-concurrent-runs CLI flag or
max_concurrent_runs in server.toml.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce page[limit]/page[offset] query params and { data, meta: { hasMore } }
response wrapper for the /runs endpoint, establishing the pagination pattern
for all future list endpoints.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace opaque (i64, i64, i64, f64) tuple with ModelUsageTotals struct
for readable field access. Remove redundant top-level token/cost fields
that duplicated by_model sums — derive them at read time instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tracks total runs, input/output tokens, cost, and runtime in-memory
(resets on server restart). Accumulates from checkpoint node_outcomes
when runs complete. Includes per-model breakdown.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All API errors now return a consistent JSON shape:
{"errors": [{"status": "4xx", "title": "...", "detail": "..."}]}
Introduces ApiError type with IntoResponse impl, replaces bare
StatusCode returns and ad-hoc {"error": "..."} responses in all
handlers and auth extractors. Updates OpenAPI spec and regenerates
TypeScript client.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Maps the Graphviz "insulator" shape to a new wait.timer handler that
reads a duration attribute and sleeps before proceeding with success.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Google's most cost-efficient model ($0.25/M input, $1.50/M output) with
1M context, 65K output, and full tool/vision/reasoning support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add discovery, health check, OpenAPI spec, and current user endpoints.
The first three are public; /user requires authentication and returns
the login extracted from JWT sub claim, mTLS CN, or "demo" in demo mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move CheckReport, CheckResult, CheckDetail, and CheckStatus types from
arc-cli doctor into arc-util so they can be reused. Refactor the workflow
run_preflight to render a styled check report instead of ad-hoc key=value
output and separate stderr errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Install ring CryptoProvider at CLI startup to prevent rustls panic
- Skip TLS in demo mode so the server uses plain HTTP
- Add ARC_DEMO=1 env var to web app config to bypass GitHub OAuth
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Anthropic API returns HTTP 529 for overload conditions, but this
status code was falling through to the catch-all which classified it as
InvalidRequest (non-retryable). This meant neither LLM-level nor
node-level retries would trigger, causing workflow nodes to fail
immediately on transient overload errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Checks llm_model and llm_provider values in model_stylesheet against the
built-in catalog during `arc validate`. Unknown models or providers emit
warnings (not errors) since the catalog is advisory, not restrictive.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Multi-line arguments (e.g. spawn_agent task descriptions) were breaking
the single-line progress output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
LLM responses were printing raw Markdown (bold markers, table pipes,
heading hashes). Use termimad to render with proper ANSI formatting
when color is enabled, falling back to plain text when NO_COLOR is set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both verbose and normal modes now use the same indicatif-based ProgressUI
renderer with the same visual hierarchy (indentation, glyphs, colors).
In verbose mode: tool calls persist after stage completion, no 5-call cap,
stage completion shows stats (turns, tool calls, tokens), and additional
events are rendered (edge transitions, loop restarts, setup commands,
retries, context warnings, compaction, subagents).
Remove the old format_event_summary function and its ~70 tests, the
verbose stderr printing from AgentApiBackend, and the verbose/styles
fields that are no longer needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New demo 11-ensemble.dot fans out to Opus (Anthropic), Gemini (Google),
Codex (OpenAI), and Mercury (Inception) in parallel, then synthesizes
results. Also generates PNG and SVG renderings for all existing demo
dot files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a `--preserve-sandbox` CLI flag and `sandbox.preserve` TOML config
option that skips sandbox cleanup after a run. Cascade order:
CLI flag > run TOML > server.toml defaults > false.
When preserving, prints sandbox identity (container ID or Daytona
sandbox name) so users can reconnect for debugging.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Simplify create_ssh_access to return String directly
- Add jump_to_node field to Outcome for parallel handler fan-in
- Find convergence node from branch outgoing edges
- Use ..Outcome::success() spread in tests and error paths
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Return just the SSH command string instead of the full SshAccessDto,
and hardcode the 60-minute expiration internally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Show smoothness rating, outcome, friction/open item counts, and file
path after retro completes instead of just the file path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Creates SSH access after sandbox init and displays the connection
command in the progress output, aligned under the sandbox detail line.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Handle ParallelBranchStarted/Completed events as nested entries under
the fork stage, using the same indented spinner styling as tool calls.
Branch bars persist after the stage completes so they remain visible
in the final output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use insert_after instead of add so the detail bar renders directly
below the sandbox bar, not at the bottom of the multi-progress.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Show sandbox name, CPU, and memory on a second indented detail line
beneath the "Sandbox: provider" line. Hyperlink the provider name to
its dashboard URL using OSC 8 terminal escape sequences (TTY only).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sandbox cleanup was spawned fire-and-forget in a scopeguard, causing DNS
resolution to be cancelled when the tokio runtime shut down before the
HTTP request completed. Now cleanup is awaited explicitly before
returning, with the scopeguard kept only as a safety net for panics.
Also: add blank line before the warning and color-code it yellow to
match other warnings. Update daytona-sdk to include full error source
chain in error messages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
apply_defaults() previously did whole-struct replacement for llm, setup,
and sandbox sections. If run.toml specified only [llm] model, the entire
server.toml [llm] block (including provider) was lost. Now each field
merges individually: task values win per-field, with map-merge for labels
(like vars) and whole-struct semantics for snapshot.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a shared floor_char_boundary() utility that rounds byte indices down
to the nearest char boundary, preventing panics on multi-byte UTF-8
characters like emoji. Applied to all 9 unsafe slice sites across
arc-agent and arc-workflows.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Display the LLM model ID (dim, bracketed) next to stage names as soon as
the first AssistantMessage arrives, and show the script command (dim,
truncated to 60 chars) for parallelogram/script stages from stage start.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reads the checkpoint to find the last stage with a response and
prints it under an "=== Output ===" header so users can see what
the workflow produced without digging into log files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Show tool args in dim parenthetical: read_file(CLAUDE.md)
- Shorten CWD-relative paths in tool call display
- Show elapsed time on completed tool calls
- Emit StageStarted/StageCompleted for terminal (exit) nodes
- Add --no-retro flag to skip retro generation
- Add demo/02-tool-use.dot, renumber existing demos 03-07
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Shorten paths with ~ instead of full home directory
- Show retro duration right-aligned like stage lines
- Add "Running retro..." indicator before retro agent runs
- Dim the Cost/Tokens summary line
- Add blank line after Goal before progress section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Swap cost and duration order so cost appears first and duration
stays consistently on the far right edge.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Insert a separator bar through indicatif before hiding progress bars
for interactive prompts, ensuring a visual gap between the spinner
and the dialoguer prompt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
