Commit graph

244 commits

Author SHA1 Message Date
Scott Werner
2d292c28f8
Merge pull request #783 from fabro-sh/codex/sqlite-blob-startup-activation
Activate verified SQLite blob storage at server startup
2026-08-24 16:32:59 -04:00
Bryan Helmkamp
c042b9abdc
Merge remote-tracking branch 'origin/main' into feature/bounded-agent-tool-output
# Conflicts:
#	lib/components/fabro-sandbox/src/clone_source.rs
2026-08-24 14:08:04 -04:00
Scott Werner
776e719383 Harden SQLite blob activation safety
Keep VACUUM snapshots private until permissions and durability are established. Refuse to recreate a missing rollback backup after import has begun, and preserve secondary cleanup failures in startup logs.
2026-08-24 14:02:35 -04:00
Scott Werner
5629dcd7d0 Make snapshot and backup publication durable across power loss
Neither the pre-activation backup nor the pre-migration snapshot fsynced
the staged file contents or the parent directory around the publishing
rename. A crash after the import committed could lose the retained
'.pre-blob-activation.bak' (whose directory entry was never made
durable), and the next activation would then write a new backup that
already contains the imported blobs, silently breaking the documented
pre-activation rollback boundary; a torn staging file could likewise
wedge later boots in backup validation.

write_snapshot_to_staging now syncs the staged file before handing it to
the caller, and both publishers sync the destination's parent directory
after their rename (fabro-db on a blocking task, activation inside its
existing blocking publication task).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
e067de9382 Share one SQLite snapshot-staging helper between fabro-db and activation
create_backup re-implemented the staging half of fabro-db's
pre-migration snapshot (remove stale staging file, UTF-8 check,
VACUUM INTO, private permissions), and remove_file_if_exists and
set_private_permissions had been made pub precisely to hand-copy that
sequence. Any future hardening of snapshot staging would have had to
land in two crates and could drift.

fabro-db now exposes write_snapshot_to_staging with a typed
SnapshotStagingError; both the pre-migration snapshot and the
pre-activation backup stage through it, and the hand-copied helpers are
private again. The publish halves stay separate on purpose: migrations
overwrite their snapshot, activation publishes with persist_noclobber
plus integrity validation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
fafb1ed7cc Expose shared SQLite file helpers from fabro-db
Make append_to_path, remove_file_if_exists, and set_private_permissions
public so callers stop keeping verbatim private copies, and export the
blobs migration SQL so fixtures in other crates can install the blob
schema without a relative filesystem path into this crate's source tree.
set_private_permissions now returns io::Result so each caller owns its
own error context.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:34 -04:00
Bryan Helmkamp
1e284c625e
Simplify bounded tool output capture
Apply cleanups from a reuse/simplification/efficiency review of the
bounded-tool-output changes:

- Share one MAX_RUN_EVENT_BODY_BYTES constant in fabro-types; the server
  body limit, the agent's serialized-output reservation, and the event
  headroom test all derive from it.
- Rework truncation.rs around one split_head_tail helper: drop the
  hand-rolled ceil_char_boundary (std's is stable), the duplicate
  truncate_plain_output splitter and its dead Tail arm, and the
  head_bytes field with its sentinel values.
- Return Cow from preview_tool_output and take retain_tool_output's
  input by value, so untruncated output crosses the pipeline without
  full copies. Measure serialized JSON size with a counting writer
  instead of materializing the payload.
- Reuse fabro-llm's byte-token estimate (now public) instead of a third
  copy of the 4-bytes-per-token heuristic.
- Take retain_tool_result's ToolResult by value and mutate content in
  place; extract the triplicated error retain-emit-truncate block into
  finish_error_result.
- Share the shell retain-and-record sequence between the native and
  kimi shell tools as retain_shell_output.
- Move OutputCaptureBuffer::into_parts to reuse the head allocation,
  skip the buffer round-trip in replay_exec_result when output fits,
  and replace daytona's byte-iterator suffix matching with contiguous
  slice comparisons behind one retained_slices accessor.
- Make SessionBoundEmitter's fields private.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TK3QTWQHiXhRbFwTr57LzX
2026-08-24 13:56:16 -04:00
Scott Werner
2c7f46ab64 Validate Git-target branches on the bare branch name
The selector grammar ran against a heads/-prefixed string, so its
leading-character rules saw the prefix instead of the branch: names git
itself rejects, like -foo or HEAD, passed admission and only failed
later at sandbox clone time. Check the bare branch name and reject a
literal HEAD explicitly.

Also build the Git projection's origin URL through
GitHubRepositorySlug::https_url so the URL grammar keeps one owner.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 13:02:32 -04:00
Bryan Helmkamp
401acb6cdf
Record tool output byte counts 2026-08-24 12:46:27 -04:00
Scott Werner
d978a6c89d Route both create-run client methods through one submission helper
create_run_from_manifest and create_run_from_intent were byte-identical
apart from the body type; fold the shared request/retry plumbing into a
private submit_create_run(CreateRunRequest) so the two public entry
points stay thin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 11:55:44 -04:00
Scott Werner
18d98794ae Move Git-target validation onto RunTarget in fabro-types
The Git-target grammar (slug, branch, and SHA rules plus the derived
origin URL) was implemented twice with no shared code path: once in
server admission and again in sandbox start, so the two could drift and
disagree about which persisted targets are valid.

Own it once as RunTarget::validate() in fabro-types, next to the
primitives it uses, returning the canonical target together with its
derived GitContext projection. Admission consumes it directly, and the
start path re-derives the expected clone source from the same rules
before checking the persisted projection against it. The start path now
also moves the derived strings into the sandbox spec instead of cloning
them.

While reordering admission around the shared validator, run the pure,
in-memory checks (target grammar, environment id) before the blob-store
closure fetch and lowering so malformed requests no longer pay for
version-store I/O.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 11:55:44 -04:00
Scott Werner
040bc6c043 Add version-backed run intent creation 2026-08-24 11:55:43 -04:00
Scott Werner
87b49a8527 Merge main into feat/refresh-tokens-sqlite
Preserve the SQLite auth-session release notes alongside main's July 26 fixes and retain all current changelog navigation entries. Make the refresh-token rotation timestamp assertion deterministic after the merged suite exposed its wall-clock race.
2026-08-24 11:04:10 -04:00
Bryan Helmkamp
3217a05aad
Merge origin/main into feat/venice-search-provider 2026-08-21 21:04:28 -04:00
Bryan Helmkamp
88ed2ac9a3
refactor(search): select backend from available credentials 2026-08-21 20:09:52 -04:00
Bryan Helmkamp
ab900485b6
Merge pull request #770 from fabro-sh/feat/release-push-rescue
Rescue release pushes when origin/main moves mid-release
2026-08-21 19:43:52 -04:00
Bryan Helmkamp
cfa8ae92c0
style: apply pinned rustfmt 2026-08-21 19:34:38 -04:00
Bryan Helmkamp
0d028f9b1e
Harden release push recovery 2026-08-21 19:28:55 -04:00
Bryan Helmkamp
45e06d2a6e
Merge pull request #775 from fabro-sh/claude/additional-github-repositories
Additional GitHub repository access
2026-08-21 18:55:31 -04:00
Bryan Helmkamp
09f5bb0f84
Simplify clone depth plumbing
Make RunCloneSettings::DEFAULT_DEPTH the single owner of the default
depth, and interpret the "0 = full history" sentinel in one place via
RunCloneSettings::depth_limit(). Docker's clone_depth becomes
Option<usize> to match Daytona's encoding, with a shared
depth_argument() helper for both git command builders. Drop the
unreachable Option on the resolved depth field, the hand-written
DaytonaSettings::Default, and the pure-forwarding
daytona_git_clone_options helper. The blob-import test helper reuses
the pool's own connect options instead of rebuilding a partial copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019bgXj5J218RXfiT72qhbLV
2026-08-21 18:37:39 -04:00
Bryan Helmkamp
6179470eb2
feat: default sandbox clone depth to 100 2026-08-21 17:25:58 -04:00
Bryan Helmkamp
4c467cd6ba
refactor(github): simplify repository access checks 2026-08-21 17:12:27 -04:00
Bryan Helmkamp
438bab29f0
feat: support shallow sandbox clones 2026-08-21 17:11:24 -04:00
Bryan Helmkamp
47954f731e
refactor(github): deduplicate additional-repository access plumbing
Consolidate the copies that review found across the feature:

- One GITHUB_CREDENTIAL_HELPER / GITHUB_CREDENTIAL_HELPER_KEY pair in
  fabro-github, with apply_probe_git_env() for probe commands; the runtime
  git bridge, server preflight probe, and live contract test all consume it
  so the probes exercise exactly what the bridge configures.
- GitHubRepositoryAccess::resolve_verified_token() owns the
  resolve-installations-then-mint choreography shared by server preflight,
  workflow initialization, and the live test.
- A shared lookup_installation() helper backs both the shared-installation
  resolution and the mint's installation lookup.
- The contents = read|write rule lives once as
  RunIntegrationsGithubSettings::contents_permission_allows_repository_access.
- The preflight probe paces retries with fabro-sandbox's exported
  replication_backoff() (3s/9s) instead of a contradicting 1s/2s loop, and
  shares one run_ls_remote() runner with the existing remote-ref check.

Also: collapse the dead Ok(None) arm and repeated error blocks in the
preflight token check, drop the derivable bridge_entry_count(), privatize
resolve_permissions() behind resolve_integration(), make
GitHubRepositorySlug ordering/hashing allocation-free, and use EnvVars
constants for env names.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 16:31:48 -04:00
Jesse Proudman
53efde3930 feat(search): add Venice backend for web_search
Brave stays the default. Shops that already vault VENICE_API_KEY
can drop BRAVE_SEARCH_API_KEY by setting
[server.integrations.search] provider = "venice".

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-21 12:31:03 -07:00
Bryan Helmkamp
84b75f29f1
docs(api): document additional github repository access
- Add `additional_repositories` to the RunIntegrationsGithubSettings
  OpenAPI schema and reuse the canonical Rust settings types through
  `with_replacement`, with type-identity witnesses and JSON parity
  tests for populated and empty repository sets.
- Regenerate the TypeScript API client.
- Document the feature in the GitHub integration and run-configuration
  guides: exact layer replacement rules, single-token scope, gh/API
  support, App-versus-PAT scope, the same-owner/same-installation
  requirement, validation errors, supported Git URL forms, hard-failure
  semantics for declared repositories, GH_TOKEN precedence, and the
  security boundary (no second server-side repository intersection;
  contents = "write" lets any stage push to any declared repository).
  Correct the earlier claim that injecting GITHUB_TOKEN alone makes
  arbitrary additional private clones work.
- Add a dated changelog entry and an opt-in live GitHub App e2e test
  that verifies a scoped multi-repository token reads every declared
  repository (and that a primary-only token cannot), with repositories
  supplied through the test environment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 15:12:23 -04:00
Bryan Helmkamp
f2047ad9a9
feat(config): add validated additional github repositories
Add `additional_repositories` to `[run.integrations.github]`: a list of
full `owner/repository` slugs, beyond the implicit run origin, that the
minted GITHUB_TOKEN must cover.

- `GitHubRepositorySlug` gains FromStr, Display, string serde, and
  case-insensitive Eq/Ord/Hash identity while preserving the submitted
  spelling for display and serialization.
- The config layer keeps raw strings; the higher-precedence list
  replaces the lower one wholesale, with `[]` as an explicit clear,
  resolving independently from the `permissions` map.
- Resolution validates each entry with indexed error paths: slug
  grammar, case-insensitive duplicates, one shared owner, the
  499-repository cap, and a required `contents = "read"|"write"`
  permission (templated values are re-checked at the runtime boundary).
- `RunIntegrationsGithubSettings` resolves permissions and repositories
  together through `resolve_integration()` so consumers cannot pick up
  one without the other; the field is omitted from serialization when
  empty, keeping single-repository settings byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 14:15:46 -04:00
Scott Werner
a08dff1ce8 Merge main into feat/refresh-tokens-sqlite 2026-08-21 13:16:14 -04:00
Scott Werner
9d3aa7a4d4 Consolidate workflow-version lineage test coverage
The lineage field's `skip_serializing_if` behavior was asserted five times
across three crates. Keep the two assertions in fabro-types, which owns the
attribute, and drop the duplicates:

- Delete `run_created_omits_absent_workflow_version_id` from event/convert.rs,
  a copy of the test above it that re-checked another crate's serde attribute.
  convert.rs's own responsibility is covered by the existing field assertion.
- Delete `legacy_create_input_persists_without_workflow_version_id`, which ran
  the full create() pipeline to prove a hardcoded `None` literal is `None`.
  `CreateRunInput` has no such field, so no input could change the result.
- Fold `run_spec_omits_absent_workflow_version_id` into the adjacent legacy-spec
  test, which already holds an all-`None` record.
- Drop the off-topic spec re-serialization from run_state.rs's retried_from test.

Add `test_support::test_workflow_version_id()` alongside `test_run_provenance()`
and use it everywhere, replacing eight copies of the same magic seed across five
crates plus two assertion sites that recomputed the hash inline. This also
subsumes retry.rs's private helper of the same shape.

Revert the `run_spec_json` parameterization in the projection round-trip test:
`RunProjection` is a `with_replacement` alias for the canonical type, so the
`Some` and `None` call sites exercise identical code.

Have the two run.created literals that mirror a `RunSpec` read the spec's
lineage field instead of hardcoding `None`, so the mirrors stay accurate once a
producer populates it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 12:39:02 -04:00
Scott Werner
27fd48c603 Persist workflow version lineage on runs 2026-08-21 12:39:02 -04:00
Bryan Helmkamp
ded92a215d
Update Venice model catalog 2026-08-21 10:37:04 -04:00
Bryan Helmkamp
a611e00fe6
Rescue release pushes when origin/main moves mid-release
The release push raced any commit that landed on main while the release
smoke ran (~15 minutes): git push was rejected as non-fast-forward and
the whole release failed, as seen on the v0.332.0-nightly.1 attempt.
Worse, the push was not atomic — if the tag ref had been accepted while
the main ref was rejected, the release would have shipped from an
orphan commit and main would never have received the version bump.

Make the push atomic (both refs or neither) and add a bounded rescue
loop: on rejection, drop the bump commit and tag this run created,
fast-forward onto the updated origin/main, recompute the version
against freshly fetched tags, and rebuild the bump commit on the new
tip. The fast-forward uses --ff-only so a genuinely diverged local main
(unpushed commits) fails loudly instead of being reset away.

The retried tag can include commits the smoke did not test; those
commits passed CI to land on main, and the Release workflow re-runs the
full test suite on the tagged commit before publishing anything.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 07:40:13 -04:00
Bryan Helmkamp
7d771e9b96
Merge remote-tracking branch 'origin/main' into pr-764
# Conflicts:
#	lib/components/fabro-sandbox/src/daytona/mod.rs
#	lib/components/fabro-sandbox/src/push_credentials.rs
#	lib/components/fabro-sandbox/src/sandbox.rs
2026-08-20 22:07:20 -04:00
Bryan Helmkamp
78cb0d1348
Clean up git push retry handling 2026-08-20 21:59:47 -04:00
Bryan Helmkamp
ca6d9a46da
Merge remote-tracking branch 'origin/main' into fix/redaction-corrupts-executable-spec
# Conflicts:
#	lib/components/fabro-dump/src/lib.rs
#	lib/components/fabro-store/src/run_state.rs
#	lib/components/fabro-store/tests/serializable_projection.rs
#	lib/components/fabro-workflow/src/billing_rollup.rs
#	lib/components/fabro-workflow/src/run_lookup.rs
#	lib/components/fabro-workflow/src/runtime_store.rs
#	lib/foundation/fabro-api/tests/run_projection_round_trip.rs
#	lib/foundation/fabro-test/src/lib.rs
#	lib/foundation/fabro-types/src/run.rs
#	lib/foundation/fabro-types/src/run_event/run.rs
#	lib/foundation/fabro-types/src/run_projection.rs
#	lib/foundation/fabro-types/tests/run_spec_methods.rs
2026-08-20 20:42:20 -04:00
Bryan Helmkamp
2b095612c8
Address run spec persistence review findings 2026-08-20 20:35:52 -04:00
Release Repro
9aac811953
Merge remote-tracking branch 'origin/fix/max-visits-off-by-one' into fix/max-visits-off-by-one 2026-08-20 19:38:25 -04:00
Release Repro
4eea9b816a
Clarify visit-limit counter comment 2026-08-20 19:38:10 -04:00
Bryan Helmkamp
eb57c6bfd3
Merge branch 'main' into fix/max-visits-off-by-one 2026-08-20 19:37:24 -04:00
Bryan Helmkamp
1688cd5b91
Retry git pushes with a pinned token and record attempt history
Run 01M0DH033P2XSTHAGVBHG6922F completed 2.8 hours of work, then failed
terminally because four consecutive publish pushes hit GitHub's
token-replication lag (404 "Repository not found") — the push path had no
retry, the failure was misclassified as deterministic, and the same
fresh-mint-then-push pattern silently disabled metadata snapshots. This
generalizes the clone retry machinery to pushes and makes attempt detail
durable.

- clone_retry -> git_retry: the classifier's boolean becomes a
  CredentialContext derived from the token snapshot (fresh App tokens retry
  404s as replication lag, mature ones as transient infra, static
  credentials fail fast), and the attempt/backoff limits become a RetryPlan
  with layered optional bounds. Clone behavior is preserved: Docker keeps
  its absolute five-minute deadline, Daytona keeps no deadline.
- Pushes take a scoped CredentialLease before the first attempt: it owns
  the embed mutex for the whole operation, pins the single successful
  resolve, retries only failed resolves, falls back to the last embedded
  token when a mint fails, and force-re-embeds the pinned token once after
  the first auth-shaped failure (drift repair). The margin invariant
  (REFRESH_MARGIN > every push plan's max_elapsed) guarantees the pinned
  token outlives the operation; a unit test asserts it.
- Sandbox::git_push_ref now takes a RetryPlan and returns PushReport /
  PushError with per-attempt records (classification, redacted output tail,
  token generation/provenance/age, credential action, refresh errors).
  Checkpoint pushes use a 90-second budget; the terminal publish push gets
  5 attempts over at most 4 minutes.
- The single durable git.push event per push gains a nested attempts array
  (GitPushAttemptProps, token snapshot flattened to flat fields); stored
  events without it still deserialize. Publish push failures now carry an
  explicit failure category — exhausted transient retries stay
  transient_infra instead of deterministic — plus one bounded cause line
  per attempt and the last successful push time in the message.
- Metadata snapshot degradation records why it degraded: push failures with
  retryable classifications leave the writer eligible to re-probe at each
  later checkpoint, and a successful snapshot clears the degraded state and
  re-arms the warning. Permanent failures keep today's latch.

Plan: .ai/plans/git-push-token-resilience.md (PR 2: items 1, 2, 4, 7 and
the metadata re-probe).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 10:16:10 -04:00
Scott Werner
1898031d74 Make RunSpec example a checked doctest 2026-08-19 17:47:25 -04:00
Scott Werner
19aa5940ea Add a shared RunSpec test fixture and adopt it
`RunSpec` has 13 fields and no `Default`, so every test that needed one
spelled out all 13 even when it cared about one or two. That put 64
hand-rolled `RunSpec { .. }` literals in `lib/`, and made a single
additive field cost a mechanical edit at roughly 30 sites.

Add `test_run_spec()` to `fabro-types`' feature-gated `test_support`
module: fixed `fixtures::RUN_1`, default settings, a minimal `test`
graph, `test_run_provenance()`, and every optional field unset. Tests
now spread it and only spell out what they assert on.

Adopt it at the 13 literals where the spread removes real duplication,
including the crate-local `test_run_spec` helpers in `fabro-store` and
`fabro-workflow`, which are now defined in terms of the shared fixture.
Tests that populate every field on purpose — the exhaustive `RunSpec`
serde round-trip in particular — keep spelling it out.

No production code and no behavior changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 16:50:37 -04:00
Scott Werner
cc16362528 Add SQLite blob store foundation 2026-08-18 17:41:29 -04:00
Scott Werner
2575ab85fc
Merge pull request #747 from fabro-sh/codex/blob-hash-vocabulary
Some checks failed
Rust / Format (push) Waiting to run
Rust / Clippy (push) Waiting to run
Rust / Generated Docs (push) Waiting to run
Rust / Test (Linux) (push) Waiting to run
Rust / Test (macOS) (push) Waiting to run
TypeScript / Typecheck (push) Has been cancelled
TypeScript / Test (push) Has been cancelled
TypeScript / Build (push) Has been cancelled
Unify blob hash vocabulary
2026-08-18 15:55:43 -04:00
Scott Werner
63025bb748 Close workflow goals over version dependencies 2026-08-18 13:01:03 -04:00
Scott Werner
a0845d8346
Merge pull request #756 from fabro-sh/refactor/graph-reference-kind
Classify graph attributes with a graph-only reference kind
2026-08-18 12:57:32 -04:00
Scott Werner
0d6d2392e6
Merge pull request #755 from fabro-sh/fix/template-dependency-discovery
Harden template dependency discovery
2026-08-18 12:28:11 -04:00
Scott Werner
8dfbfb9aa5 Classify graph attributes with a graph-only reference kind
reference_kind_for_attribute returned the full ReferenceKind, which
includes the config-sourced Dockerfile kind the classifier can never
yield, so the shared graph walker carried a silent `continue` and an
`unreachable!` for impossible kinds; each new config-sourced kind widens
those filler arms, and a classifier extension that reuses an existing
kind would be dropped by the walker without validation, visitation, or a
compiler error. Return a GraphReferenceKind subset instead (converting
into ReferenceKind for validation), making the walker's matches total
with every arm meaningful.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 17:58:46 -04:00
Scott Werner
9459ce1d04 Attribute template discovery errors to their source by construction
TemplateDiscoveryError only named a failing source through the Display
strings of its variants: parse and load failures forwarded transparently
to inner errors whose source naming varies (parent for some load
failures, the child path for dynamic dependencies, nothing for I/O
faults), so consumers that need the failing template's path had to
string-round-trip error messages. Carry the parent path on every
variant, exposing a total source_path() accessor, and render parse and
load failures with a parent-naming message above the preserved source
chain.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 17:53:13 -04:00
Scott Werner
679bc6701b Parse template dependencies whose paths collide with discovery roots
Dependency discovery pre-seeded roots into the path-keyed result map
and reused that map as the traversal-dedup set, so a loaded include
target whose path matched a root was recorded but never parsed (an
include chain that reaches the file anchoring a root silently skips its
content), and a second root occurrence at an already-seeded path was
dropped without parsing. Dedup traversal on the full
(path, root, content) occurrence instead, so every distinct authored
occurrence is parsed exactly once and identical duplicates parse once.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 17:51:46 -04:00