A failed node with an effective `succeed` policy and no explicit recovery
route now finishes as `succeeded` and follows normal success routing. The
original failure stays on the outcome so the stage.completed event and the
checkpoint keep the diagnostic, and the outcome notes record which scope
promoted it.
- OnFailure gains a Succeed variant; Node::on_failure resolves the
deprecated auto_status=true attribute as an alias, with an explicit
on_failure winning
- The core executor applies the policy before the lifecycle observes the
result, so the recorded outcome, context keys, goal gates, events, and
routing all see the effective outcome; this replaces AutoStatusLifecycle
- Explicit routes take priority: a matching condition, preferred label,
suggested next node, or handler jump keeps the outcome failed. A failed
outcome takes an unconditional edge only under route, so under succeed
any edge selection is an explicit route
- succeed applies only to failed, matching exit; the auto_status alias no
longer promotes partially_succeeded
- Parallel branches promote after their retry loop, so a failed succeed
branch counts as succeeded in the parent aggregate
- Validation accepts succeed and adds an auto_status_deprecated warning
that suggests on_failure="succeed"
- Document the policy table, semantics, and deprecation; add a changelog
entry
Closes#807
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Use /tmp/fabro/runtime for both Docker and Daytona instead of
provider-specific roots. A writable /tmp inside the sandbox is already
a dependency (commit-message files, exec stop-files), it needs no
root-level mkdir for non-root container users, and it makes the two
providers uniform.
The trailing runtime path component stays load-bearing: materialized
blobs at runtime/blobs/{hash}.json are recognized as managed blob
references and normalized back to blob:// in durable context.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Kmn5jyrdpyCdvcfvmEDvA
Remote prompt-value materialization wrote demoted values to
{working_directory}/.fabro/blobs inside the repository checkout, so a
later checkpoint could commit them and leak them into the run pull
request.
Give each sandbox a run-scoped runtime directory outside the source
checkout as part of the Sandbox contract:
- Sandbox::runtime_directory() names the directory; host-local
sandboxes return None because the engine owns a host-side runtime
directory (RunScratch) for those runs.
- Docker creates /fabro/runtime at initialize with umask 077 and
uploads runtime files with mode 0600.
- Daytona creates /home/daytona/fabro/runtime with mode 0700.
- Both remote materialization paths in fabro-workflow share one
materialization-path helper built on the new contract. The paths keep
the runtime/blobs suffix, so durable context still normalizes to
blob://sha256/... references.
- Local materialization now writes owner-private directories and files
on Unix.
Regression coverage: an integration test runs remote-style prompt
demotion against a real git checkout, then a real checkpoint commit,
and asserts the checkout stays clean, the agent-facing file is
readable, and a deleted materialized file is recreated from the
durable blob store. A real-Docker test verifies the runtime directory
and blob file permissions inside a container.
Fixes#798
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Kmn5jyrdpyCdvcfvmEDvA
Blob activation cleanups:
- Reuse fabro-db's append_to_path, remove_file_if_exists, and
set_private_permissions instead of local duplicates.
- Return the store directly from activate_blob_storage; the report
wrapper existed only to be logged internally and then discarded.
- Collapse compute_disk_preflight to return the required free bytes
instead of echoing its inputs back through a struct.
- Deduplicate the "exactly one ok row" PRAGMA integrity_check protocol
into one executor-generic helper used by the backup and live checks.
- Skip re-validating a freshly published backup; the staging copy was
validated immediately before the atomic rename, so only a
concurrently published file needs its own validation.
- Replace the manual anyhow wrapping plus duplicate error log in
serve.rs with a plain .context(), matching other startup errors.
- Extract the disk-candidate enumeration in resource_sampler.rs that
available_space_for_path had copy-pasted from sample_disk_resources.
Test fixture cleanups:
- Route all hand-assembled Database::new(..., test_blob_store()) test
fixtures (32 sites) through fabro_store::test_support::test_database,
and make that helper infallible instead of returning an unconditional
Ok.
- Install the test blob schema from fabro_db::BLOBS_MIGRATION_SQL via a
test-support-gated optional dependency instead of a four-level
relative include_str! into fabro-db's migrations directory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Apply the cleanup findings from a four-angle review (reuse,
simplification, efficiency, altitude) of the demotion change:
- Share one size gate: serialized_if_over now backs both offload_value
and demote_value_for_prompt, restoring the cheap short-string and
scalar pre-checks so per-node demotion no longer serializes every
small value just to measure it.
- Stop re-writing blobs every node: materialize_value_bytes writes the
sandbox file directly from the in-hand bytes and short-circuits on the
content-addressed file's existence, so an already-demoted value costs
one existence probe instead of a store round-trip per node visit. The
local file write is shared with materialize_blob_ref.
- Demote over the resolved snapshot map instead of re-snapshotting a
Context copy, making the context and outcome loops symmetric and
saving a full deep clone per node; the fidelity lifecycle builds the
Context after the pass.
- Skip the pass entirely for Full and Truncate fidelities (nothing
renders context values), except parallel nodes whose branch stash may
render at a richer fidelity.
- Build is_preamble_hidden_key on is_engine_internal_key instead of
restating its prefixes, and call it directly from the preamble
renderer rather than through a wrapper.
- Document that outcome updates are demoted wholesale and that
BranchWorkItem.item carries the prompt-ready (possibly demoted) item;
drop the item rebinding and redundant test assertions; restore the
local integration test's confinement assertion and make the remote
one non-vacuous.
Skipped by choice: unifying the crate's several truncation helpers and
rendering the marker through the "See:" pointer family (cross-module
coupling out of proportion to the preview cosmetics), per-branch
demotion inside parallel.results (wholesale demotion is what bounds the
total), and cross-node demotion memoization (the file-existence
short-circuit already reduces repeats to a stat).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FH8Jj9Y4E4Tu5g1jwDtHAb
Compact and summary preambles render workflow context values and stage
outputs with no per-value size limit. A late-run node inherits everything
the run has accumulated, and one oversized value (a join result, a jobs
list, a single-line command emit) can push the composed prompt past the
model's context window. A security-review run failed exactly this way:
its dedupe stage assembled a ~1.8M-token prompt against a 1M-token model
limit, made almost entirely of accumulated context the agent never
needed inline.
Reuse the existing blob machinery at the last mile. Before the preamble
builders run, any resolved context or outcome value whose serialized
JSON exceeds 8KB is persisted as a content-addressed blob, materialized
as a real file in the sandbox, and replaced with a small marker holding
a preview, the byte count, and the file path. The agent reads the file
if it needs the data. for_each items get the same treatment at fan-out
with a more generous 64KB budget, since the item is the branch's work
assignment; branch labels still come from the full item. Keys the
preamble never renders are left alone, and a value that fails to demote
stays inline and is logged: demotion bounds prompt size, it does not
gate execution.
The two downstream-resolution integration tests asserted that resolving
text values writes no files; demotion now legitimately materializes the
oversized response for preamble use, so they instead pin that resolution
returned the full inline text and that nothing is written outside the
sandbox blob directory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FH8Jj9Y4E4Tu5g1jwDtHAb
Brave stays the default. Shops that already vault VENICE_API_KEY
can drop BRAVE_SEARCH_API_KEY by setting
[server.integrations.search] provider = "venice".
Co-authored-by: Cursor <cursoragent@cursor.com>
Carry the resolved GitHub integration (permissions plus declared
additional repositories) as one value from run materialization into
workflow startup, and make the sandbox environment reach every declared
repository through the single managed GITHUB_TOKEN.
- `StartServices.github_permissions` becomes
`github_integration: ResolvedGithubIntegration`; CLI and server
workers build it with `resolve_integration()` after interpolation and
pass it through `SandboxEnvSpec` as one unit.
- `build_sandbox_env` constructs the validated
`GitHubRepositoryAccess` and scopes the App token source to the whole
effective set. Missing credentials or a missing origin are hard
initialization errors when additional repositories are declared;
legacy permissions-only configuration keeps its best-effort behavior.
- When additional repositories are declared, initialization eagerly
resolves each repository's App installation (naming any repository
the App cannot see) and the token itself, so an inaccessible declared
repository fails before the first workflow stage.
- A new `git_bridge` module injects secret-free `GIT_CONFIG_*` entries
into the stage environment: a github.com credential helper that reads
`$GITHUB_TOKEN` at invocation time, per-repository SSH-to-HTTPS
`insteadOf` rewrites, and `GIT_TERMINAL_PROMPT=0`. Entries append
after a valid user-provided Git config overlay and fail clearly on a
malformed one. Contract tests drive the installed git binary against
local fixtures for the rewrite, credential, prefix-collision, and
overlay-preservation behaviors.
- The long-running ACP notice now says all declared repository access
expires together.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every push previously re-minted a fresh GitHub App installation token and
embedded it in the origin URL, so pushes routinely landed inside GitHub's
token-replication lag window (run 01M0DH033P2XSTHAGVBHG6922F failed
terminally on four consecutive fresh-token 404s). Reusing mature tokens
removes the failure trigger and saves two GitHub API calls plus one sandbox
exec per push.
- New fabro_github::token_source::InstallationTokenSource: one cached,
single-flight source per origin repo. Static credentials pass through
(generation 0); App credentials mint through the cache and reuse tokens
until REFRESH_MARGIN (10 min) before expiry. Every resolve returns a
non-secret TokenSnapshot (generation + Minted/Reused/Static provenance),
and the source logs mints at INFO and reuses at DEBUG.
- Docker and Daytona share the source through PushCredentialState: an embed
mutex serializes compare -> set-url -> record, a matching generation skips
the set-url exec, and the generation is recorded only after a successful
exec. The clone still mints its own token, but now seeds the source cache
(generation 1) and the last-embedded state, so a refresh mint failure
falls back to the known embedded token instead of believing nothing was
ever embedded.
- RefreshOutcome now reports the remote action (embedded/unchanged/none)
separately from the token snapshot; git_push_via_exec logs token age and
provenance with each push, and refresh failures log the last embedded
generation.
- The run-metadata writer resolves through the sandbox's shared source
instead of minting per snapshot (with its own cached source on resume).
- The ACP refresh-ahead loop reschedules from the embedded token's
expires_at minus the margin instead of a fixed 45-minute interval, which
a cached source would have broken for long turns; static credentials stop
the loop.
Plan: .ai/plans/git-push-token-resilience.md (PR 1: items 3 and 6).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Finish the blob-hash vocabulary unification at the defining signatures:
RunStoreBackend::read_blob, RunStoreHandle, LocalRunStoreBackend, the
HTTP backend impl, RunDatabase::read_blob, and BlobStore::read/exists
all said `id`, which kept re-teaching the old vocabulary at every impl
site and inlay hint.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the static-reference vocabulary out of fabro-workflow so every
consumer shares one definition: ReferenceKind, AttributeScope, and
reference_kind_for_attribute land in fabro-types::graph, and
validate_static_reference plus a new visit_graph_references walker land
in fabro-template. The manifest bundler drops its ad-hoc graph scan and
walks references through the shared walker.
Unifying the walkers forces three semantic alignments, each matching
what the engine actually executes rather than what the old scanners
happened to match:
- stack.child_dotfile is no longer classified as a child-workflow
reference; the engine never resolved it as one.
- import and stack.child_workflow only count at node scope; graph- and
edge-level occurrences were scanned but never executed.
- @@-escaped goals flow through the shared walker's escape handling
instead of the bundler's own prefix stripping.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two root-cause fixes for the sandbox failure where an inline Dockerfile
came back from the store as `ARG REDACTED` and the Daytona snapshot
build died on the unset variable.
Entropy redaction measures values, not assignment pairs. The detector
matched `NAME=value` as one token, so an uppercase name merged its
charset into a pure-hex value (which alone can never exceed 4.0 bits)
and pushed the pair over the 4.5-bit threshold — then replaced the
whole pair, destroying the name. `find_entropy_regions` now strips an
identifier-shaped `NAME=` prefix before measuring and redacts only the
value, matching the gitleaks layer's `key=REDACTED` shape.
Execution no longer reads redacted content. Every stored event passes
through the redaction sink, and `load_from_store` rehydrated the
worker's RunSpec from the projection folded from those events — so a
redactor false positive silently rewrote the spec the sandbox builds
from (and changed its snapshot identity). The creation path now writes
the exact spec bytes to the content-addressed blob store and records
`spec_blob` on run.created; `load_from_store` loads the spec from the
blob, keeping the event stream authoritative for run identity,
provenance, and event-recorded blob ids. Retry and fork carry the
source run's `spec_blob` forward, so derived runs stop inheriting the
redacted copy. Runs created before the blob existed fall back to the
folded spec.
The projection and every API surface keep serving the redacted fold;
blobs were already stored unredacted (the workflow bundle carries the
same bytes), so this adds no new exposure at rest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One conflict, in StructuredOutputError::repair_message. main (#709)
added a `previous_error` parameter and richer validation-error
rendering; this branch had replaced the inline expectation match with
OutputSchemaKind::expectation().
Resolved by keeping both: main's new signature and section assembly,
calling schema.expectation() for the expectation text. The method
already supersedes main's inline match and carries this branch's intent
of embedding the resolved JSON Schema instead of naming it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both contract helpers dispatched on OutputSchemaKind, so they belong on
the type. Moves expectation() and agent_prompt() into an impl block and
drops the free functions.
Splits the combined agent test: assertions no longer run inside the
backend's run(), where a failure surfaces as a panic from execute().
Adds coverage for the Routing branch of the contract, which was
previously untested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolve the model catalog table conflict in docs/public/core-concepts/models.mdx
by keeping both changes: this branch's `kimi` -> `moonshot` provider rename for
the Kimi rows, and main's new DeepSeek V4 rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Addresses Copilot review feedback on the repeated-failure check.
serde_json runs with preserve_order, and jsonschema builds the
additionalProperties `unexpected` list by walking the instance in
document order. So the same leftover keys emitted in a different order
produced a different Vec and compared as a different problem, which
suppressed the "unchanged from your previous repair" nudge.
Sorting at capture also makes the MAX_UNEXPECTED_PROPERTIES truncation
pick the same subset every time instead of an order-dependent one, and
stabilizes the rendered message.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up review of the repair-error work. Behavior is the same or better;
the machinery is smaller.
Fixes a false "unchanged from your previous repair" nudge. same_problem_as
fell through to `_ => true`, so any two non-Required issues at the same
instance path, schema path and keyword compared equal. A model that removed
one unexpected property and added another was told it had changed nothing.
SchemaValidationIssue already derives PartialEq, so the 17-line comparison
is now `previous.contains(issue)`.
Drops the hand-written Type and Enum rendering. jsonschema already renders
both, and its messages name the offending value, which the hand-written
ones did not. Also switches masked() back to to_string(): masking replaced
the bad value with a placeholder, working against the goal of an actionable
message, and buys no privacy since the full response is already in the
prompt.
Resolves the schema fragment when the issue is captured rather than
threading Option<&OutputSchemaKind> through rendering. That reverts the
command.rs change and drops the test-only messages() shim. The fragment is
now attached only to Other, where it adds information; for required, type,
enum and additionalProperties it just repeated the prose.
Also: caps the model-controlled unexpected-property list so a wide object
cannot turn the repair prompt into megabytes; drops evaluation_path, which
was dead except under $ref, where it printed a pointer that does not
resolve; drops the keyword field, already named by the schema path; and
records the previous error only after the agent session accepted the
repair, since failover rebuilds the session from the original prompt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot review flagged two backward-compatibility breaks with events
persisted by pre-model-keyed releases; both are stored data that can
never be rewritten, so accept the old shapes on read:
- FailoverProps: original_provider/original_model/attempt are Option
again with serde defaults. New events always set them; failover events
recorded before model-keyed fallbacks lack them. Restores the
historical-event test.
- RunModelSettings: temporary custom deserializer accepts the legacy
flat-array fallbacks shape inside stored run.created events, keying
the chain under the requested model name when one is set. Remove once
pre-0.311 run logs are out of the support window.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot review flagged that a failover event's from route can be a
candidate that failed during activation and never served traffic. That
is intentional — events chain (one event's to is the next one's from)
so the stream records every candidate tried, with the error explaining
why each was abandoned. Document it at the emit site.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Consolidation pass over the fallback feature, no intended behavior
changes beyond noted validation and event-shape cleanups:
- Unify the two parallel notice types: FallbackPlanNotice is gone;
ModelFallbackNotice now owns the runtime NoNearbyReasoningLevel case
and the shared ChainEmpty wording. Notices emit through a new
Emitter::notice_scoped with their own level, and each distinct notice
is emitted once per run instead of on every LLM call.
- Move canonical_model_id onto Catalog so chain keys are written and
read through one function; reject provider-qualified fallback keys,
which could never match at dispatch and were silently dead config.
- Type FallbackTarget as ProviderId/ModelId, removing repeated
ProviderId::new re-wrapping at every use site.
- Derive FallbackPlan's current route from a position index instead of
storing current/requested_controls copies; advance() no longer has
unreachable None branches.
- Bundle the agent invocation's live state (session, bridge, lease,
forwarder, accounting) into LiveAgentInvocation; failover_agent_session
drops from 21 parameters to 7 and the six copies of the
abort/discard/classify teardown collapse into two methods.
- Share one route_request builder between one_shot and its failover
loop; complete_one_shot_request takes the request by value instead of
deep-cloning the message payload per call.
- Event::Failover carries FailoverProps directly; the props' original
route and attempt fields are now required, and reasoning efforts are
typed ReasoningEffort instead of strings.
- Reuse RunModelSettings/RunModelControls in fabro-api via
with_replacement, add the missing controls property to the OpenAPI
schema, regenerate the TS client, and add the type-identity/JSON
parity test.
- Smaller cleanups: ReasoningEffort::closest_supported uses enum
discriminants; ModelFallbackPolicy gains len(); resolve_model_fallbacks
takes a provider slice; duplicate-target filtering lives only in the
resolver.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 10 MiB cap on resolved stdin_source values is tight for wide
fan-in: a context.parallel.results batch from a large for_each round
carries tens of structured agent outputs, and a merge step that feeds
them to a deterministic command hits the ceiling as a hard
deterministic failure. Raise the ceiling to 30 MiB; it still bounds
peak memory and remote uploads, just with headroom matched to the
fan-out sizes for_each already allows.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Workflow agent sessions run at PermissionLevel::Full with the whole
tool registry exposed, so blocking pre_tool_use hooks are the only
policy boundary they have. The child-session factory built for
spawn_agent dropped tool_hooks from the child's SessionOptions, so a
subagent's tool calls never reached the run's hooks: any agent that
could spawn a subagent got an unguarded read-write-shell escape from
every hook-enforced policy.
Clone the parent's tool_hooks into the factory, the same way the
permission level is already carried, so child sessions inherit the
parent's hook boundary. The new end-to-end test drives the real
create_session path against a scripted mock provider: the parent
spawns a child, the child executes read_file, and the hooks must see
both the parent's spawn_agent and the child's read_file.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
