Commit graph

295 commits

Author SHA1 Message Date
Bryan Helmkamp
96f70a9aea
fix(timing): simplify live timing projections 2026-07-28 14:20:59 -04:00
Bryan Helmkamp
010c8d50c1
Add structured review targets to human gates 2026-07-28 13:24:12 -04:00
Bryan Helmkamp
33b94d850e
Support provider-qualified fallback selectors 2026-07-28 11:48:20 -04:00
Bryan Helmkamp
f0a7423b51
refactor(config): stop resolving {{ env.* }} in interpolated config
The process environment is no longer a configuration source. `{{ vars.NAME }}`
(non-sensitive, server-stored) and `{{ secrets.NAME }}` (vault-backed) cover
both cases, and reading the worker's ambient environment made a run's inputs
depend on how its process happened to be launched.

`Namespace::Env` is kept but wired to nothing, so `{{ env.NAME }}` still
parses and fails with a message naming its replacement rather than reaching
a consumer as literal text. `ResolveCtx::with_env` is gone, so no call site
can opt back in.

Two long-standing warts were env-only and go with it:

- `InterpString::resolve_or_source`, the "fall back to the raw template
  source on failure" path, which let an unresolved token reach a sandbox or
  the GitHub API as literal `{{ ... }}` text. Its own comment noted it was
  slated for hard-error semantics.
- `RunEnvironmentSettings::resolve_env`'s matching source fallback for
  env-only values.

Both carried `#[expect(clippy::disallowed_methods)]` escape hatches. Every
run-boundary resolver — sandbox env, prepare steps, MCP transports, GitHub
permissions, Slack channels, run goal files, provider extra_headers — now
fails closed instead.

Hooks lose their `allowed_env_vars` allowlist, `resolve_header`, and
`HeaderResolveError` along with the `E: Env` generic threaded through the
executor. They keep `{{ vars.* }}`, which `RunSettings::substitute_variables`
already substitutes server-side at run creation.

`allowed_env_vars` is removed from the OpenAPI spec and the generated
TypeScript client. The docs example showing `{{ env.* }}` in
`[server.slatedb.s3].bucket` was already wrong — that field is a plain
String and never interpolated — and is now a literal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 21:09:35 -04:00
Release Repro
c812274db8
Show live status for parallel branches 2026-07-27 17:08:08 -04:00
Bryan Helmkamp
7841a77f2c
feat(web): show stage tokens and cost in the model popover
The model indicator on a stage page hovered to provider, model, and
reasoning effort only. Seeing what a stage actually spent meant leaving
for the Billing tab, which reports per node rather than per visit.

The stage list had no token data to show, so add a per-visit `billing`
block to `GET /runs/{id}/stages`. The Billing tab's pricing rule (a
provider-reported cost wins, otherwise the server catalog prices the
tokens) was private to `billing_rollup`; move it to
`StageProjection::billed_usage` and drive both call sites from it so the
two views cannot drift.

The popover's buckets use the Billing tab's labels verbatim. It stays
scoped to one visit, so a looped node's row on the Billing tab is the sum
of what each of its visits shows here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 16:06:19 -04:00
Bryan Helmkamp
b53045a1ac
Add runtime for_each item injection 2026-07-27 11:55:55 -04:00
Bryan Helmkamp
1c82bd9008
fix(workflow): make publish failures terminal 2026-07-27 11:25:18 -04:00
Bryan Helmkamp
dd9f75fb05
fix(timing): harden live active projections 2026-07-25 23:43:43 -04:00
Bryan Helmkamp
c4971b93d3
fix(timing): accumulate active time for in-flight stages
`active_time_ms` was only ever computed from terminal stage events, so a
stage still running contributed zero to the run rollup. A run parked in one
long agent stage reported 2m 8s of active time against 16m 53s of wall
clock — the two finished stages — while the running stage had been doing
continuous inference and tool work for over 14 minutes.

`live_run_timing` summed `filter_map(|stage| stage.timing)`, and
`stage.timing` is only written at finalization. Wall time ticked live off
`start_time`; active time did not tick at all.

Stage projections now accumulate brackets from the event log:

- Closing an inference bracket folds its span into `live_inference_ms`
  instead of discarding it, including across retries, matching the
  in-process stopwatch.
- Tool calls open a batch on the first outstanding call and close it when
  the last one drains, so tools running concurrently within a turn count
  once — the same span `execute_tool_calls` is bracketed by. Summing
  per-call durations would over-count parallel tool use. Subagent tool
  events are excluded; they run inside the root call's span already.
- `StageProjection::live_timing(now)` composes accumulators with any open
  bracket, per handler: agent stages use the brackets, prompt and command
  stages count elapsed time as inference and tool respectively, and
  handlers that wait on a human, timer, condition, or child branches
  report zero.

Active is clamped to wall per stage. A worker killed mid-turn leaves its
bracket open forever, and without the clamp it would tick up unbounded.
The clamp does not need to detect the dead worker: a stage cannot have been
active longer than it has existed. `watchdog.timeout` remains the authority
on whether a run is stuck. The clamp is deliberately not applied at run
level, where concurrent branches can legitimately sum past run wall time.

Timing is derived from events rather than emitted by the worker, so this
needs no event-schema change and applies to runs already stored.
`StageProjection.timing` keeps its terminal-only meaning, and the
authoritative breakdown still replaces the live estimate at terminal
events.

The billing endpoint had the same hole behind its `wall_only` fallback:
running stages reported zero inference/tool/active. Not visible in the
product, which renders only `wall_time_ms`, but wrong for any other
consumer of `GET /runs/{id}/billing`.

Parallel branch stages lose their breakdown permanently, even after
completion, because `parallel.branch.completed` carries only `duration_ms`.
That is a separate data-loss bug, tracked in #644.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:54:38 -04:00
Bryan Helmkamp
c81ea69c73
Merge origin/main into feat/inference-observability 2026-07-24 22:55:15 -04:00
Bryan Helmkamp
d4f619bc2a
fix: clean up inference observability 2026-07-24 22:50:00 -04:00
Bryan Helmkamp
eddee10b35
fix(agent): harden Kimi profile tool contracts 2026-07-24 22:05:48 -04:00
Bryan Helmkamp
6659ae768a
feat(events): make inference in-flight state observable
During a long LLM turn the durable event stream was silent: between
`agent.tool.completed` and the next `agent.message` nothing was emitted,
so "the model is generating" and "the worker is wedged" were
indistinguishable from the run store, SSE, or the UI.

The signal already existed. `AssistantTextStart` fired at exactly the
right point — after `build_request()`, after compaction, immediately
before the stream opens — then was classified as streaming noise and
thrown away. This promotes it rather than inventing a new one.

Two events, each asserting only what is provable when it is emitted:

- `agent.llm.started` carries the *requested* provider/model. No usage,
  no cost, no context window: none of it exists yet, and failover can
  re-target, so `agent.message` stays authoritative for what answered.
- `agent.llm.first_output` is edge-triggered on the first output of an
  attempt and names what arrived. `ToolCall` is required, not optional:
  a turn that opens with a tool call produces no text or reasoning
  delta, so a latch keyed on those two would stay silent for exactly
  the tool-heavy rounds where liveness matters most.

`agent.llm.retry` now also fires on the one previously invisible
mid-turn path — a stream that ends without a finish event, which
replays the turn and discards its output with nothing to show for it.
Its `attempt` field was already fed by two independent counters, so an
optional `phase` (open | consume) names which loop it counts.

`StageProjection.inference` projects the open bracket. `Some` means
"the event log contains an unclosed inference bracket", not "the model
is computing now" — a SIGKILLed worker leaves it open, which is the
truthful statement of what we know, and `watchdog.timeout` remains the
authority on actually-stuck.

The close is the subtle part. Terminal cancel and wall-clock timeout
tear the session down through `discard_session` without emitting a
message, error, or interrupt, so a session-lifecycle backstop is
required. It has to be `agent.session.ended`, not
`agent.session.deactivated`: deactivation is emitted by `lease.release()`
*before* the forwarder drains queued agent events, so a queued
`agent.llm.started` can arrive after it and re-open the bracket. But
`agent.session.ended` carries no stage identity, so the close takes
ordering from the event and identity from the projection, scanning for
brackets the ending session opened. A normal stage lookup there finds
no target and silently no-ops.

Presentation states what the log proves and nothing more: no progress
bar or ETA (no completion estimate exists), "reasoning" only when the
provider sent reasoning output, elapsed counted since the request
opened, and no live animation once the run is terminal.

Scope is session-backed agent stages. One-shot completions call
`client.complete` directly and never build a session; covering them
means moving the emit point into `fabro-llm`, filed as a follow-up.

`agent.output.start` was never persisted — it existed in a name map,
an `unreachable!` arm, and docs — so the rename carries no migration
risk. Corrects `events.md`, which documented it as a real emitted
event, and the v2 proposal, which mapped it to `message.part.started`
despite it firing before the request opens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 21:47:51 -04:00
Release Repro
512ab50f9c
fix(reasoning): align stream and client invariants 2026-07-24 17:42:44 -04:00
Release Repro
4d3de5f564
fix(reasoning): tighten capture normalization 2026-07-24 17:31:14 -04:00
Bryan Helmkamp
e7740b4acb
feat(reasoning): capture provider reasoning in agent.message
Normalize the readable reasoning providers already return into a
canonical `ReasoningOutput` and carry it through the `agent.message`
run event to storage, SSE, and JSONL.

The shape is derived from the final response's canonical message
content rather than stored a second time, so there is no duplicate
source of truth and retried or replaced streaming buffers never
become durable reasoning. OpenAI-compatible `reasoning_details` are
now preserved verbatim as an opaque content part; only known readable
members are normalized out of them, leaving encrypted entries for a
later provider-aware replay phase.

This phase is passive: no request parameters change, no capability
guessing, and no newly observed provider field is replayed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 14:21:15 -04:00
Bryan Helmkamp
c9900b2bfa
Merge branch 'main' into feat/backward-event-pagination 2026-07-24 10:14:57 -04:00
Bryan Helmkamp
eb83539a18
Merge origin/main into feat/stage-execution-identity-on-resume
Resolves conflicts with the shared-checkout parallel rewrite (#607) and the
cached-run/billing dedup (de60eb900):

- handler/parallel.rs: rebuilt on main's shared-checkout version. Branch
  ordinals are still reserved inside the branch task right before
  ParallelBranchStarted (with graph_visit/resumed_from_stage_id), and the
  reserved StageScope is shared with post-await error paths via a OnceLock
  slot instead of main's dispatch-time visit=1 scope, so completion events
  are never emitted under a guessed ordinal.
- billing.rs: keep this branch's run_stage_from_projection (RunStage grew
  graph_visit/resumed_from_stage_id and a typed id), adopt main's
  state.cached_run() and drop the removed run_stage_from_stage_id import.
- run_projection.rs: adopt main's typed parallel_results
  (Option<Vec<ParallelBranchResult>>).
- run_event/misc.rs: union of both sides' imports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 10:07:35 -04:00
Bryan Helmkamp
ec100fca2b
Merge remote-tracking branch 'origin/main' into feat/backward-event-pagination
Resolved conflicts against main's shared-projection-cache rework:
- projection_cache.rs: kept main's projection_snapshot and dropped this
  branch's last_seq accessor, which it subsumes; latest_event_seq now
  reads the sequence from projection_snapshot.
- run_store.rs: kept main's EventScan cursor and added a seek_before
  constructor so the backward-pagination range scan bounds its end key
  through the same abstraction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 09:57:15 -04:00
Bryan Helmkamp
78fea736e3
fix: harden stage execution identity on resume 2026-07-24 09:37:05 -04:00
Bryan Helmkamp
396f75578a
Merge remote-tracking branch 'origin/main' into feat/backward-event-pagination
Resolved conflict in run_store.rs tests: kept both the new
list_events_before_with_limit tests from this branch and the
append_event_rejects_sequences_beyond_key_order_limit test from main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 09:35:34 -04:00
Bryan Helmkamp
415e9e5cd8
Merge pull request #617 from fabro-sh/feat/expose-model-controls
Expose model reasoning effort controls
2026-07-24 09:34:55 -04:00
Bryan Helmkamp
3c6a26e8c2
Merge pull request #607 from fabro-sh/feat/shared-checkout-parallel
Shared-checkout parallel execution
2026-07-24 09:33:03 -04:00
Bryan Helmkamp
3beaddc224
Merge remote-tracking branch 'origin/main' into feat/expose-model-controls
# Conflicts:
#	lib/foundation/fabro-api/src/lib.rs
2026-07-24 09:28:01 -04:00
Bryan Helmkamp
f2a8b4e989
Merge pull request #616 from fabro-sh/codex/expose-completion-token-usage
Expose detailed completion token usage
2026-07-24 09:01:21 -04:00
Bryan Helmkamp
cd706646c6
feat: treat resumed in-flight nodes as new stage executions
A node cancelled (or lost to a crash) mid-flight and then resumed now
starts a new stage execution with the next StageId ordinal (work@2)
instead of reusing and clearing the cancelled execution's projection.
The old execution stays immutable with its own events, session, output,
timing, billing, and termination state.

Engine:
- Add a run-scoped StageExecutionTracker on RunServices with per-node
  high-water marks. Ordinals are reserved after the StageStart hook
  passes on the first attempt (retries reuse the reservation), ensured
  at the composite checkpoint pre-step for hook-skips, and reserved in
  on_terminal_reached for terminal nodes' synthetic events.
- Keep three concepts distinct: graph visit (max_visits/checkpoints,
  unchanged), stage execution ordinal (the @N in StageId), and handler
  attempt. The tracker is not checkpointed; the append-only stage event
  history is its durable source of truth.
- resume() seeds the allocator from the run projection and computes a
  node -> StageId provenance map of executions observed after the
  selected checkpoint, threaded through execute_persisted_run,
  RunSession, and InitOptions.

Events and projections:
- stage.started, parallel.branch.started, and checkpoint.completed
  carry optional graph_visit and resumed_from_stage_id; StageProjection
  stores both. Old events deserialize with None and legacy duplicate
  stage.started replays keep last-attempt behavior.
- The CheckpointCompleted reducer is envelope-first: diffs and
  skipped-stage synthesis attach to the exact execution StageId, an
  existing Retrying projection finalizes as Skipped without losing
  identity, and historical node_outcomes no longer create or collide
  with newer ordinals (node_visits remains a legacy fallback).

Handlers:
- Parallel fan-out reserves child ordinals through the shared tracker,
  derives worktree pass{N} from the parent's execution ordinal, and
  seeds branch contexts with explicit child stage scopes so branch
  lifecycle and nested handler events agree.
- Artifact capture and manager-loop child logs follow the ordinal.

API and UI:
- RunStage documents visit as the execution ordinal and adds optional
  graph_visit and resumed_from_stage_id; Rust and TypeScript clients
  regenerated.
- The web sidebar lists both executions chronologically; resumed stages
  show a "Resumed from" link in the stage detail header and hover
  popover, with the graph visit surfaced when it diverges.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 09:00:37 -04:00
Bryan Helmkamp
4bd9753217
Expose model reasoning effort controls 2026-07-24 07:44:40 -04:00
Bryan Helmkamp
142862f342
Expose detailed completion token usage 2026-07-24 07:36:56 -04:00
Bryan Helmkamp
bb1afae363
feat(events): add backward cursor pagination 2026-07-24 07:23:23 -04:00
Bryan Helmkamp
673a7064fe
Validate completion reasoning effort 2026-07-24 07:04:31 -04:00
Bryan Helmkamp
4621149b6e
Merge remote-tracking branch 'origin/main' into feat/shared-checkout-parallel 2026-07-24 06:54:32 -04:00
Bryan Helmkamp
85f3286c66
Merge branch 'main' into feat/shared-checkout-parallel 2026-07-24 06:29:57 -04:00
Bryan Helmkamp
0a39ba9e06
Shared-checkout parallel execution (recovered from run 01KY7YH7RYCJ1BDVTTP96ZA4HV)
Cumulative implement + simplify_fable diff recovered from the run's meta
branch (fabro/meta/01KY7YH7RYCJ1BDVTTP96ZA4HV, stage 006 diff.patch).
The run validated this tree clean: cargo nextest (7,007 passed), clippy,
fmt, TS client regen + typecheck, web tests (679 passed), docs check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 06:19:11 -04:00
Bryan Helmkamp
84c5468722
Merge remote-tracking branch 'origin/main' into fix/cancellation-interrupt-lifecycle
# Conflicts:
#	lib/components/fabro-agent/src/subagent.rs
#	lib/components/fabro-agent/tests/it/parity_matrix.rs
2026-07-23 20:55:25 -04:00
Bryan Helmkamp
5cf1c7d183
Harden cancellation and interrupt lifecycles 2026-07-23 20:40:22 -04:00
Bryan Helmkamp
67644c7c3c
Merge pull request #602 from fabro-sh/fix/openrouter-anthropic-prompt-caching
fix(llm): send cache_control breakpoints for Claude via OpenRouter
2026-07-23 20:06:10 -04:00
Bryan Helmkamp
78718c101a
Merge remote-tracking branch 'origin/main' into fix/root-agent-todo-projection 2026-07-23 19:30:35 -04:00
Bryan Helmkamp
3c33e0fb69
Merge origin/main (three-layer crate layout) into fix/openrouter-anthropic-prompt-caching
The crate reorganization renamed lib/crates/ to lib/apps|components|foundation/.
Git followed all modified files across the rename; the only conflict was the
newly added codec/cache.rs, now placed at lib/components/fabro-llm/src/codec/cache.rs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:30:20 -04:00
Bryan Helmkamp
d8d1c14116
Clarify stage TODO ownership and plan guidance 2026-07-23 18:23:37 -04:00
Scott Werner
47bc772f7b refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
Bryan Helmkamp
e5f0290229
fix(llm): send cache_control breakpoints for Claude via OpenRouter
Anthropic prompt caching is opt-in per request: without explicit
ephemeral cache_control breakpoints in the body, no cache writes or
reads ever happen. The OpenAI-compatible codec never emitted them, so
every run on openrouter Claude models billed the full conversation at
the uncached input rate on every turn (0 cache tokens on the billing
page, confirmed by OpenRouter's activity portal).

- Add a `cache_control_breakpoints` model feature declaring that a
  route only caches when the request marks the cacheable prefix; set it
  on the builtin OpenRouter Claude rows. Catalog build rejects the flag
  without `prompt_cache`.
- Teach the Chat Completions wire shape a parts-form content variant so
  a message can carry the annotation; unmarked messages keep the
  plain-string form for compatibility with strict servers.
- Mark the last system message (covers tools + system upstream) and the
  second-to-last user turn, counting tool results as user turns —
  mirroring the anthropic codec's placement so agent loops get
  incremental cache hits.
- Extract the shared placement/opt-out policy into codec::cache and
  refactor the anthropic codec onto it; anthropic wire snapshots are
  unchanged.
- Honor `provider_options.<name>.auto_cache = false` as an opt-out and
  consume the control key instead of merging it into the body.
- Mirror the new feature through settings (fabro-config), the OpenAPI
  schema, and the generated TypeScript client.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:55:41 -04:00
Bryan Helmkamp
65cdf52061
feat: make model aliases provider-aware 2026-07-23 10:12:25 -04:00
Scott Werner
0244736b05
Resolve run.prepare.steps env and interpolation at the run boundary (#530)
## What

Per-step environment in `run.prepare.steps[].env` was parsed and then
**dropped** before it reached the resolved run settings, so prepare
steps could never see their declared env. This PR carries that env all
the way through to the executor, resolves prepare-step interpolation at
the run boundary, and fixes an argv-quoting bug.

Three things:

1. **Per-step env is carried through.** `RunPrepareSettings` now holds
`steps: Vec<PreparedStep>` (command plus per-step `env`) instead of a
flat `commands: Vec<String>`. The per-step env reaches `exec_command`,
which already accepts per-command env vars, and is merged on top of the
base sandbox environment.
2. **Interpolation resolves at the run boundary.** Prepare-step
`script`/`command` and per-step `env` values are carried in source form
out of the portable config resolve layer (so `fabro validate` stays
portable and never requires env to be set). Their `{{ env.* }}` tokens
resolve in the process that actually runs the steps, via
`RunPrepareSettings::resolve_step_env` — mirroring the existing MCP
transport env resolution. A missing env var is a **hard error**
(fail-closed); there is no fallback to the unresolved literal.
3. **Argv is shell-quoted.** Argv-style prepare steps were assembled
with `join(" ")`, so an argument containing spaces or quotes was
re-split by the shell. They are now shell-quoted per element with the
shared `shell_quote()` helper. `script` steps stay verbatim because they
are raw shell snippets.

## How

- `RunPrepareSettings.commands: Vec<String>` becomes
`RunPrepareSettings.steps: Vec<PreparedStep>` where `PreparedStep {
command, env }`. The server-side `{{ vars.* }}` substitution pass now
walks each step's command and env.
- New `RunPrepareSettings::resolve_step_env(env_lookup)` resolves `{{
env.* }}` in each step's command and env values, returning a hard error
on a missing var (and a loud `Unavailable` error for reserved
`secrets`/`inputs` tokens).
- The run boundary (`fabro_workflow::operations::start`) gains
`runtime_setup_commands`, the prepare-step counterpart to
`runtime_mcp_server`. `LifecycleOptions` now carries `Vec<SetupCommand>`
(command + env), and the initialize phase passes each step's env to
`exec_command`.
- `resolve_prepare` shell-quotes each argv element and carries per-step
env in source form. The stale lint suppression on the resolved fields is
rewritten to describe the deliberate source preservation that now
resolves at the run boundary.
- The shell-quoting helper moves to a shared `fabro_util::shell` module
(backed by `shlex`); `fabro_sandbox::shell_quote` delegates to it so the
config resolve layer and sandbox code share one audited implementation.
- The OpenAPI `RunPrepareSettings` schema and the generated TypeScript
client are updated to the new `steps`/`PreparedStep` shape.

## Testing

- `cargo build --workspace`
- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D
warnings`
- `cargo nextest run` for `fabro-util`, `fabro-types`, `fabro-config`,
`fabro-sandbox`, `fabro-api`, `fabro-workflow`, `fabro-server`,
`fabro-cli` (provider keys stripped) — all green.
- `cd lib/packages/fabro-api-client && bun run typecheck` — clean.

New tests cover: per-step env carried through resolution; script/command
+ env resolved at the run boundary; a missing env var is a hard error
(in both the command and a per-step env value); reserved `secrets`
tokens surface as `Unavailable`; argv elements are shell-quoted (an arg
with spaces/quotes is correctly quoted) while a `script` stays verbatim;
and an end-to-end check that per-step env reaches the executed setup
command (with a negative control proving the success is attributable to
the per-step env).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 10:31:26 -04:00
Scott Werner
287afd7928
Hooks: typed end-to-end interpolation, narrow header tokens, fail-closed resolution (#528)
## What

Makes hook interpolation typed end-to-end and fail-closed, and removes
the bespoke template engine on HTTP-hook headers.

- **Typed end-to-end.** Hook `command`, `url`, header values, `prompt`,
and `model` are now carried as a typed `InterpString` from the config
resolve layer all the way to the executor. The executor resolves each
segment at hook fire time from the typed value instead of collapsing it
to a `String` and re-parsing it. This mirrors the MCP transport env
resolution boundary (`resolve_transport_env` / `runtime_mcp_server`).
- **Narrow header tokens.** HTTP-hook headers previously ran through
MiniJinja with an env allowlist
(`TemplateContext::with_env_lookup_allowed`). They now resolve through
the same narrow `{{ ns.NAME }}` token resolver as every other hook field
— no template engine, no allowlist.
- **Fail-closed everywhere.** A missing or out-of-scope `{{ env.* }}` /
`{{ secrets.* }}` token in a command, URL, header, prompt, or model is
now a hard error that blocks the hook rather than firing it with a
half-resolved or empty value. Previously command hooks failed closed but
http/prompt/agent hooks failed open (warned and proceeded), which could
dispatch an HTTP request with an empty credential header or run an LLM
call against a half-rendered prompt. Transport-level outcomes (non-2xx
responses, connection errors, unparseable bodies) stay fail-open.

A follow-up cleanup commit removes the template engine's `env` namespace
(`with_env_lookup` / `with_env_lookup_allowed` / the `EnvLookup`
object), which the header path was the last consumer of.

## How

- `fabro-types` and `fabro-hooks` `HookType` / `HookDefinition` now type
the interpolatable fields as `InterpString`. `InterpString` serializes
as its raw source, so persisted run specs and checkpoints round-trip
unchanged.
- The `fabro-config` resolve layer clones the typed `InterpString`
through instead of calling `as_source()`, so the fields no longer leak
unresolved template text — the old "source preservation" `#[expect]`
annotations on the hook resolvers are gone.
- The executor's single `resolve_interp` helper resolves a typed
`InterpString` and is shared by the command, http, prompt, and agent
paths; resolution failure maps to `HookDecision::Block`, which the
runner already reports loudly (error for blocking hooks, warn for
non-blocking).

## Testing

- New unit tests: fire-time resolution from the typed value (no
re-parse), narrow-token header resolution, and fail-closed behavior for
HTTP url, HTTP header, and prompt hooks on a missing variable (the hook
does not fire and the resolution error surfaces).
- Existing hook tests updated and kept green.
- Gates: `cargo build --workspace`, `cargo +nightly-2026-04-14 fmt
--check --all`, `cargo +nightly-2026-04-14 clippy --workspace
--all-targets -- -D warnings`, and `cargo nextest run` for the touched
crates (`fabro-hooks`, `fabro-types`, `fabro-config`, `fabro-template`,
`fabro-workflow`, `fabro-server`, and the `fabro-cli` hook/config
tests), all green.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:07:11 -04:00
Scott Werner
173968a780
feat(server): mcp-servers HTTP API — handlers + AppState wiring (#532)
Some checks are pending
Rust / Format (push) Waiting to run
Rust / Clippy (push) Waiting to run
Rust / Generated Docs (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Typecheck (push) Waiting to run
TypeScript / Test (push) Waiting to run
TypeScript / Build (push) Waiting to run
## What

Adds the **mcp-servers HTTP API**: `GET/POST /api/v1/mcp-servers` and
`GET/PUT/DELETE /api/v1/mcp-servers/{id}` on top of the merged
`fabro-mcp-store` foundation and OpenAPI spec.

This includes the AppState wiring needed for the catalog to work end to
end: `McpServerStore` construction from `{active-config-dir}/mcps/`, an
`AppState` accessor, the `fabro-server` dependency, and route
registration for list/create/get/replace/delete handlers.

The API mirrors the automations concurrency pattern with ETags on
read/write responses and required `If-Match` headers for replace/delete.

## Resolved before merge

- **Credential-omitting read model:** read responses now return
`McpServerView` / `McpTransportView`, so stored env/header values are
not exposed by GET/list/create/replace responses. Responses include only
`env_keys` / `header_keys`; persisted values remain available to runtime
execution.
- **Manifest catalog references:** run manifest validation, graph
rendering, preflight, and run creation now resolve server-managed MCP
catalog references such as `[run.agent.mcps.<name>] id = "..."`.
- **Schema strictness:** unknown MCP transport fields are rejected,
aligning the reused Rust domain type with the OpenAPI
`additionalProperties: false` contract.
- **Create response headers:** the `POST /mcp-servers` 201 response now
documents its `ETag` header in OpenAPI.

## Follow-up intentionally left out

Credential-literal validation remains structural only: create/replace
currently accept literal env/header values and persist them for runtime
use. The warn-vs-hard-reject UX is a separate follow-up for the settings
UI; it is not a response-omission issue.

## Testing

Current PR checks are green:

- Rust: format, clippy, generated docs, Linux tests
- TypeScript: build, test, typecheck

Local checks run during the simplify/CI-fix pass:

- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy --locked --workspace --all-targets
-- -D warnings`
- `cargo nextest run -p fabro-config run_agent_mcps`
- `cargo nextest run -p fabro-mcp-store`
- `cargo nextest run -p fabro-api --test mcp_server_round_trip`
- `cargo build -p fabro-api`
- `cargo nextest run -p fabro-server --features test-support
system_sandbox_provider`
- `cargo nextest run -p fabro-server --features test-support --test it
mcp_servers`

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 15:06:24 -04:00
Bryan Helmkamp
2307468bc6
fix(cli): use server catalog for provider login (#529)
Some checks are pending
Rust / Format (push) Waiting to run
Rust / Clippy (push) Waiting to run
Rust / Generated Docs (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Build (push) Waiting to run
TypeScript / Typecheck (push) Waiting to run
TypeScript / Test (push) Waiting to run
## Summary

`fabro provider login --server ... --provider openrouter` now asks the
selected Fabro server for provider metadata before reading, validating,
and storing API keys, so server-enabled providers are accepted even when
the local CLI catalog does not know them.

This adds a server-side credential test endpoint that validates
submitted API keys against the server's effective catalog without
persisting them, then keeps saving the resulting secret to the selected
target server. OpenAI Codex device login remains client-side for the
browser/device flow, with the resulting OAuth credential stored on the
selected server.

The OpenRouter docs and model docs are updated to use the current
`--provider openrouter` login syntax and clarify that remote deployments
need the server host settings updated.

## Testing

- `cargo nextest run -p fabro-client -p fabro-server -p fabro-cli
provider`
- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy -p fabro-client -p fabro-server -p
fabro-cli --all-targets -- -D warnings`
- `rg -n "provider login openrouter|fabro provider login [a-z]"
docs/public lib/crates/fabro-cli/tests lib/crates/fabro-cli/src -g
'*.md' -g '*.mdx' -g '*.rs'`

---

[![Compound
Engineering](https://img.shields.io/badge/Compound_Engineering-6366f1)](https://github.com/EveryInc/compound-engineering-plugin)
🤖 Generated with GPT-5 (context compacted, extended thinking) via
[Codex](https://openai.com/codex)
2026-06-26 08:46:38 -04:00
Scott Werner
c311b6c67f
feat(api): add /api/v1/mcp-servers OpenAPI endpoints (#522)
## What

Adds the HTTP contract for managing server-defined MCP servers. The
handler implementation follows in a later change.

- New `/api/v1/mcp-servers` paths: `list`, `create`, `retrieve`,
`replace`, `delete`, with ETag / `If-Match` optimistic concurrency
mirroring the automations conventions.
- New schemas: `McpServer`, `CreateMcpServerRequest`,
`ReplaceMcpServerRequest`, `McpServerListResponse`.
- **Collapsed a duplicate `McpTransport` schema** into the single
canonical one and gave it a proper `discriminator` plus the
previously-missing optional `protocol` field (`streamable_http` |
`sse`). This also fixes a latent gap in the existing run-config
projection and is non-breaking (`protocol` is `#[serde(default)]`).

## Testing

- `cargo build -p fabro-api` is green — progenitor generates the client
methods and types cleanly from the new spec.

## Notes / follow-ups for the handler change

- Recommended `with_replacement` mapping (reuse, no parallel DTOs):
`McpServer` → `McpServerDefinition`, create/replace →
`McpServerDraft`/`McpServerReplace`, transport → existing
`fabro_types::McpTransport`/`McpHttpProtocol`; list envelopes become
small DTOs.
- Parity caveat: progenitor emits `i64` for the `u64` timeouts and `i32`
for the `u16 port`; harmless under `with_replacement`, but the handler
change must add identity/JSON-parity tests and not skip
`with_replacement` for those types.
- `createMcpServer` returns ETag on 201 (Environments convention) so the
UI gets the fresh revision.
- The "warn vs hard-reject credential-looking literal values" question
is recorded in the request-schema descriptions and intentionally not
enforced.
- Part of a short series adding server-managed MCP servers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 15:50:13 -04:00
Adrian Muraru
a769336c39
feat(workflow): support overriding cwd for local sandbox provider (#467)
Some checks failed
Rust / Format (push) Has been cancelled
Rust / Clippy (push) Has been cancelled
Rust / Generated Docs (push) Has been cancelled
TypeScript / Build (push) Has been cancelled
Rust / Test (Linux) (push) Has been cancelled
Rust / Test (macOS) (push) Has been cancelled
TypeScript / Typecheck (push) Has been cancelled
TypeScript / Test (push) Has been cancelled
## Problem

The `local` sandbox uses the run's `source_directory` (the CLI's cwd at
invocation time) as its working directory and `create_dir_all`s it on
the server (`LocalSandbox::initialize` in `fabro-sandbox`). That is
correct when the CLI and the server share a host — the agent operates
directly on the user's project tree.

When the server is **remote** from the CLI — e.g. `fabro serve` running
in a container in Kubernetes, driven over HTTP with the `local` sandbox
— the client's cwd (e.g. `/Users/alice/project`) does not exist on the
server. The sandbox then tries to create that path as the (often
unprivileged) server user and fails at init:

```
sandbox.failed provider="local" error="Failed to create working directory" causes=["Permission denied (os error 13)"]
```

and the run dies with `workflow_error` before the agent starts.

## Fix

When `source_directory` is absent or does not exist on the server, fall
back to a server-writable `workspace` directory under the run's scratch
dir instead of recreating the client path. **Same-host behavior is
unchanged**: an existing `source_directory` is still used as-is.

The selection is extracted into a small pure helper,
`local_working_directory(source_directory, run_dir)`, so it can be
unit-tested directly.

## Testing

- `cargo test -p fabro-workflow local_working_directory` — 3 new tests
(existing source dir → used; absent → fallback;
present-but-missing-on-server → fallback)
- `cargo check -p fabro-workflow`

🤖 Generated with [Claude Code](https://claude.com/claude-code)


Thanks for fabro @brynary!

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 12:32:32 -04:00
Bryan Helmkamp
bc0bda73a6
feat(web): add server-managed Environments CRUD settings UI (#462)
Some checks are pending
Rust / Clippy (push) Waiting to run
Rust / Format (push) Waiting to run
TypeScript / Build (push) Waiting to run
Rust / Generated Docs (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Typecheck (push) Waiting to run
TypeScript / Test (push) Waiting to run
## What

Adds a CRUD interface for **server-managed Environments** at
`/settings/environments`, driven by the `/api/v1/environments` REST API
(list / create / retrieve / replace / delete), and reshapes how built-in
environments are provisioned and protected.

The page lives in the **Workflows** settings nav section (also
introduced in this branch), positioned before Variables.

## Why

The Environments REST API shipped (#453) but had no UI — environments
could only be managed via the API/CLI. This gives operators a web UI
alongside Variables and Secrets, and along the way tightens the model:
environments are seeded at install time (not silently re-created on
every boot), and the `default` fallback is an ordinary, deletable
environment.

## Web UI

**Pages & component**
- `settings-environments.tsx` — list view: provider badge,
image/resource summary, row actions (Edit/Delete). **"New environment"
is a dropdown** of the enabled sandbox providers; the chosen provider is
fixed for the environment's lifetime.
- `settings-environments-new.tsx` / `settings-environments-edit.tsx` —
create/edit flows; create reads the provider from a query param.
- `environment-form.tsx` — shared form, reorganized:
- **General** panel (merged identity + image): id, and an **image-source
selector** (Image reference *vs* inline Dockerfile) that shows,
requires, and sends only the selected, mutually-exclusive source.
- **Resources**: CPU / memory / disk as **range sliders** (CPU 1–8,
memory 1–16 GB, disk 1–20 GB), each always writing a concrete value.
  - **Environment variables** key/value editor.
- **Advanced** progressive-disclosure section holding **Network** (a
single "Block all network access" toggle — allow-all vs block) and
**Lifecycle** (preserve / stop-on-terminal / auto-stop). Opens by
default when any advanced value is non-default.
- The in-form **provider control and the Labels editor were removed** —
labels remain API-managed and are round-tripped untouched so UI edits
never clear them.

**Data layer**: `environmentsApi` client, `queryKeys.environments`,
`useEnvironments` / `useEnvironment` SWR hooks.

**Nav & routing**: "Environments" item in the Workflows section before
Variables; routes registered in `router.tsx`.

## Backend: seed at install, deletable `default`

- **Seeding moved to install time.** The server no longer seeds
built-ins on startup; `EnvironmentStore::load_or_seed` → `load`
(load-only). A new public `seed_environments(dir)` (idempotent,
preserves operator edits) is called by both the web installer and the
CLI installer. An uninstalled instance therefore has no managed
environments, and a run selecting an absent environment fails explicitly
(`unknown environment: default`) rather than resurrecting a built-in.
- **`default` is no longer protected.** The delete guard and the
`Protected` error variant are gone; deleting `default` succeeds (204)
and removes the run fallback on purpose — forcing an explicit choice.
`local` is unchanged (reserved, in-memory).
- **`volumes` removed** from environment settings across the OpenAPI
spec, generated Rust + TS clients, config layers,
sandbox/server/workflow plumbing, docs, and tests.

## API contract details honored
- Edit sends the environment `revision` as `If-Match`; 409 conflicts
surface a "changed since you opened it" message.
- The REST API accepts inline Dockerfiles only — the form never sends a
Dockerfile path.

## Verification
- Rust: `cargo build` (touched crates) ✅, `cargo nextest -p
fabro-environment` 21/21 ✅, server env unit + `tests/it` integration 2/2
+ 15/15 ✅, `clippy` (nightly, touched crates, all targets) clean ✅, `fmt
--check` clean ✅. Full `--workspace` suite not run here — worth a CI
pass.
- Web: `bun run typecheck` ✅, `bun run build` ✅,
`environment-form.test.ts` 5/5 ✅. Web suite: 512 pass / 1 unrelated
pre-existing `RunDetail` failure.
- **Not visually verified in-browser** — the local app is login-gated
and automated loads redirect to `/login`; rendering of the form, the
New-environment dropdown, and `default` delete should be confirmed in a
logged-in session.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: fabro-sh-0530[bot] <281434857+fabro-sh-0530[bot]@users.noreply.github.com>
Co-authored-by: Fabro <noreply@fabro.sh>
Co-authored-by: Release Repro <release-repro@example.com>
2026-06-13 08:44:38 -04:00