Commit graph

123 commits

Author SHA1 Message Date
Scott Werner
f2a2630ad0 Simplify pull request recovery and summary store queries
- Load recovery candidates from the warm projection cache instead of
  replaying each run's full event history, and check dispatch
  eligibility before any I/O
- Extract the shared can_dispatch predicate used by both the recovery
  scan and the worker dispatch loop
- Drop load_durable_run_status, now identical to durable_run_status
- Share parse_stored_run_id across the three stored-id decode sites
- Reuse push_order for the canonical run ordering in list_all and
  list_by_statuses
- Replace the test-only queue clear accessor with the existing drain

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 14:11:53 -04:00
Scott Werner
1a2d8a9056 Move run queries and pull request recovery to SQLite 2026-09-01 14:06:06 -04:00
Scott Werner
563ca26b98
Merge pull request #829 from fabro-sh/codex/session-ownership-index
Index session ownership from creation events
2026-09-01 14:03:25 -04:00
Scott Werner
0fd4714da7
Merge pull request #825 from swerner/codex/automation-workflow-sources
Add independent workflow sources to automations
2026-09-01 12:22:20 -04:00
Scott Werner
bf05b01ba6 Simplify session owner lookup and migration preflight
- Drop the post-decode run_id/session_id/event-body checks in
  find_session_owner: decode_event_row already verifies every stored
  column against the decoded envelope, and the WHERE clause pins
  session_id and event_name to the requested values.
- Build the lookup query from SELECT_EVENT_COLUMNS like the sibling
  event queries instead of duplicating the column list.
- Carry the stored run_id text in the unparseable-id error instead of
  an "<invalid>" placeholder.
- Fetch applied migration versions once per migrate() and share the
  set between the session-owner preflight and the pre-migration
  snapshot; check the applied version first so steady-state startups
  skip the sqlite_master probe. Mark the preflight as removable with
  the run-history compatibility window.
- Restore session_by_id_key as a #[cfg(test)] helper so tests stop
  hand-rolling the legacy reverse-index key shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 14:59:38 -04:00
Scott Werner
3411199103 Index session ownership from creation events 2026-08-31 14:16:58 -04:00
Scott Werner
39c018c430 Harden automation workflow source handling 2026-08-31 13:45:08 -04:00
Scott Werner
c68e67c841 Harden SQLite activation and run history consistency 2026-08-31 12:19:16 -04:00
Scott Werner
0574c818cf Share activation backup helpers and trim run-history residue
Hoist the SQLite backup/integrity helpers duplicated between the blob and
run-history activation migrations into one shared module, drop the
write-only recent-events buffer and other dead state left behind by the
SQLite cutover, reuse existing helpers for projection bootstrap, sequence
allocation, catalog keys, and test pool construction, and collapse the
tombstone and activation-marker plumbing to single statements.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 17:20:05 -04:00
Scott Werner
aa9c468d3e Wait for SQLite writer lock when deleting runs 2026-08-28 15:46:13 -04:00
Scott Werner
f073684227 Activate atomic SQLite run history storage 2026-08-28 15:18:25 -04:00
Scott Werner
4922fec45a Support historical run events during migration 2026-08-28 10:49:43 -04:00
Scott Werner
3591315ba6 Harden legacy run history verification 2026-08-28 10:13:03 -04:00
Scott Werner
a98c0db713 Import and verify legacy run history 2026-08-27 17:27:47 -04:00
Scott Werner
57bbb923c2 Keep the RunSummaryStore name until the SQL cutover
Revert the run summary -> run record rename. The SQLite store is still
the summary read model today; it only grows an inactive events table
here. Renaming it now made the store file show as a delete plus add and
touched nine unrelated files. The final rename happens once, when the
SQL store becomes the run authority.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 15:43:56 -04:00
Scott Werner
ff2aef4564 Simplify SQL run record store write path and test fixtures
Share one bind helper across the runs insert/upsert/update statements,
compute the next event sequence once per append, and decode stored
sequence columns through a single helper. Check the run head before
decoding events, rewrite the first-visit stage listing as a UNION ALL so
each arm uses its partial index, and share the run_events insert SQL
with the test seeder.

Collapse the duplicated in-memory pool fixture, remove two tests that
only asserted Arc sharing, and fold the fabro-db test row helpers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 14:59:34 -04:00
Scott Werner
b62e458289 Add inactive SQL run storage foundation 2026-08-27 13:32:01 -04:00
Scott Werner
dc1f235c48 Keep retired Slate helpers test-only 2026-08-24 17:31:15 -04:00
Scott Werner
f3ff7f27a4 Keep auth code store naming consistent 2026-08-24 17:26:17 -04:00
Scott Werner
68ef8c7e89 Leave old SlateDB authorization-code records in place
Drop the startup retirement of the auth/code keyspace instead of
carrying one-shot cleanup code forever. The records it deleted are
inert: at most a handful exist at cutover, every binary (old or new)
rejects them within 60 seconds of issue via the expiry check, and
nothing reads the keyspace after the move to SQLite. The refresh-token
retirement keeps its original inline shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 17:25:59 -04:00
Scott Werner
a2f0167844 Simplify SQLite authorization-code cutover
Cleanup pass over the pending-CLI-authorization move to SQLite:

- Extract a shared Database::retire_keyspace helper; the refresh-token
  and authorization-code retirements are now one-line wrappers over it.
- Inline the startup retirement call (dropping the single-use wrapper,
  its context-chain test, and the test_close_slate hook it required)
  and run both SlateDB retirement scans concurrently. Error policies
  are unchanged: authorization codes fatal, refresh tokens best-effort.
- Add a shared sqlite_row module with typed identity/timestamp row
  decoding, used by both AuthorizationCodeStore and AuthSessionStore;
  the session store's stringly Error::Other corruption errors become
  the typed InvalidStoredIdentity/InvalidStoredTimestamp variants.
- Delete Repository::gc, which had no production callers left and was
  kept alive by its own test; update the record-layer docs to match.
- Deduplicate the SQLite test-support bootstrap into sqlite_test_pool,
  reuse issue() in the invalid-timestamp test instead of a copied
  INSERT, and fold the new table into the existing existence-check loop
  in the fabro-db schema test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 17:25:38 -04:00
Scott Werner
05999036aa Move pending CLI authorizations to SQLite 2026-08-24 17:25:19 -04:00
Scott Werner
2d292c28f8
Merge pull request #783 from fabro-sh/codex/sqlite-blob-startup-activation
Activate verified SQLite blob storage at server startup
2026-08-24 16:32:59 -04:00
Bryan Helmkamp
c042b9abdc
Merge remote-tracking branch 'origin/main' into feature/bounded-agent-tool-output
# Conflicts:
#	lib/components/fabro-sandbox/src/clone_source.rs
2026-08-24 14:08:04 -04:00
Scott Werner
776e719383 Harden SQLite blob activation safety
Keep VACUUM snapshots private until permissions and durability are established. Refuse to recreate a missing rollback backup after import has begun, and preserve secondary cleanup failures in startup logs.
2026-08-24 14:02:35 -04:00
Scott Werner
bccc5750a4 Size the blob activation disk preflight to the remaining import work
The preflight demanded ~1.5x the full legacy inventory bytes free on
every startup, with no credit for rows already imported. Because the
first activation itself consumes about twice the legacy bytes (the
SQLite copy plus the retained backup) and the legacy keyspace stays in
place for the whole retention window, a successfully activated server
could fall below the requirement and become unable to restart until an
operator freed space the server would never write.

The legacy inventory now checks each row's hash against the SQLite blobs
table and reports pending rows and bytes, and the preflight requires
1.5x only the pending bytes plus the backup reserve and fixed headroom.
A warm restart with nothing left to import needs only the headroom.
Also updates the server operations doc for this and for the
verification pass now running only on boots that import rows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
94e3d46754 Cut redundant blob scans and hashing from server startup
Startup previously scanned the legacy SlateDB keyspace three times and
SHA-256-hashed every value in each pass (inventory, import,
verification), then read and rehashed every row of the live SQLite blobs
table — on every boot, even a warm restart with nothing to import. With
a large object-store-backed legacy keyspace that makes restart time
proportional to total blob bytes for the whole retention window.

The inventory pass now only validates key shapes and sizes the keyspace;
digests are still validated by the import pass before any row persists.
The independent verification sweep now runs only on boots whose import
actually inserted rows: the import pass itself byte-compares every
already-present legacy row each boot, so a no-op restart is already
fully cross-checked without a third scan or a full-table rehash.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
b56aee570c Pass the SQLite pool directly to the legacy blob import and verification
import_legacy_blobs_into and verify_legacy_blobs_in took a &BlobStore and
extracted its pool through sqlite_pool_for_legacy_import, an Option that
was statically always Some in production (the None arm existed only for
the test-only Slate backend). That accessor forced a clippy
unnecessary_wraps suppression and two WrongTargetBackend error variants
no production caller could ever hit, and the activation path round-tripped
a pool it already owned through a BlobStore it had just built.

Both functions now take &SqlitePool, deleting the accessor, the
suppression, both unreachable variants, and their rejection test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
21421dce78 Isolate the shared test blob store between tests
test_blob_store was a process-wide OnceLock singleton over one in-memory
SQLite connection, so content-addressed rows written by one test were
visible to every other test in the same process. nextest's
process-per-test model masked the bleed, but plain cargo test failed
(8/24 in fabro-workflow-version) because negative existence assertions
became order-dependent.

test_blob_store now builds a fresh isolated in-memory store per call,
and test_database gives every database its own blob authority.
Reopen-style tests that model one durable blob authority across several
store handles use the new test_blob_store_at, which keeps the blob table
in a SQLite file beside the store directory, plus
test_database_with_blobs to share it explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
d65785d888 Simplify blob activation and share the test store fixture
Blob activation cleanups:
- Reuse fabro-db's append_to_path, remove_file_if_exists, and
  set_private_permissions instead of local duplicates.
- Return the store directly from activate_blob_storage; the report
  wrapper existed only to be logged internally and then discarded.
- Collapse compute_disk_preflight to return the required free bytes
  instead of echoing its inputs back through a struct.
- Deduplicate the "exactly one ok row" PRAGMA integrity_check protocol
  into one executor-generic helper used by the backup and live checks.
- Skip re-validating a freshly published backup; the staging copy was
  validated immediately before the atomic rename, so only a
  concurrently published file needs its own validation.
- Replace the manual anyhow wrapping plus duplicate error log in
  serve.rs with a plain .context(), matching other startup errors.
- Extract the disk-candidate enumeration in resource_sampler.rs that
  available_space_for_path had copy-pasted from sample_disk_resources.

Test fixture cleanups:
- Route all hand-assembled Database::new(..., test_blob_store()) test
  fixtures (32 sites) through fabro_store::test_support::test_database,
  and make that helper infallible instead of returning an unconditional
  Ok.
- Install the test blob schema from fabro_db::BLOBS_MIGRATION_SQL via a
  test-support-gated optional dependency instead of a four-level
  relative include_str! into fabro-db's migrations directory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
2814c1fd45 Activate verified SQLite blob storage 2026-08-24 14:02:34 -04:00
Bryan Helmkamp
401acb6cdf
Record tool output byte counts 2026-08-24 12:46:27 -04:00
Scott Werner
e2db53011e Fix RunSpec test fixtures 2026-08-24 12:11:49 -04:00
Scott Werner
040bc6c043 Add version-backed run intent creation 2026-08-24 11:55:43 -04:00
Scott Werner
87b49a8527 Merge main into feat/refresh-tokens-sqlite
Preserve the SQLite auth-session release notes alongside main's July 26 fixes and retain all current changelog navigation entries. Make the refresh-token rotation timestamp assertion deterministic after the merged suite exposed its wall-clock race.
2026-08-24 11:04:10 -04:00
Bryan Helmkamp
09f5bb0f84
Simplify clone depth plumbing
Make RunCloneSettings::DEFAULT_DEPTH the single owner of the default
depth, and interpret the "0 = full history" sentinel in one place via
RunCloneSettings::depth_limit(). Docker's clone_depth becomes
Option<usize> to match Daytona's encoding, with a shared
depth_argument() helper for both git command builders. Drop the
unreachable Option on the resolved depth field, the hand-written
DaytonaSettings::Default, and the pure-forwarding
daytona_git_clone_options helper. The blob-import test helper reuses
the pool's own connect options instead of rebuilding a partial copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019bgXj5J218RXfiT72qhbLV
2026-08-21 18:37:39 -04:00
Bryan Helmkamp
fe1d9dc691
test: isolate SQLite checkpoint restoration 2026-08-21 17:57:22 -04:00
Scott Werner
891eb43e9a
Merge pull request #761 from fabro-sh/codex/strict-legacy-blob-import
Import legacy blobs strictly into SQLite
2026-08-21 16:30:45 -04:00
Scott Werner
3872c04430 Derive initial refresh token session state
Replace the public stored-token row with an initial-token input that carries only token-specific facts. Bind the token to the session and initialize it as unused inside AuthSessionStore so callers cannot create mismatched session/token rows.
2026-08-21 14:01:30 -04:00
Scott Werner
b964602b0b Harden legacy blob import cleanup 2026-08-21 13:58:04 -04:00
Scott Werner
7edef76d77 Revoke replayed auth sessions transactionally
Delete the owning auth session inside the refresh-token rotation transaction when a spent token is replayed. Return the replay outcome only after the revocation commits, and propagate database failures without claiming the chain was revoked.
2026-08-21 13:55:41 -04:00
Scott Werner
a08dff1ce8 Merge main into feat/refresh-tokens-sqlite 2026-08-21 13:16:14 -04:00
Scott Werner
9d3aa7a4d4 Consolidate workflow-version lineage test coverage
The lineage field's `skip_serializing_if` behavior was asserted five times
across three crates. Keep the two assertions in fabro-types, which owns the
attribute, and drop the duplicates:

- Delete `run_created_omits_absent_workflow_version_id` from event/convert.rs,
  a copy of the test above it that re-checked another crate's serde attribute.
  convert.rs's own responsibility is covered by the existing field assertion.
- Delete `legacy_create_input_persists_without_workflow_version_id`, which ran
  the full create() pipeline to prove a hardcoded `None` literal is `None`.
  `CreateRunInput` has no such field, so no input could change the result.
- Fold `run_spec_omits_absent_workflow_version_id` into the adjacent legacy-spec
  test, which already holds an all-`None` record.
- Drop the off-topic spec re-serialization from run_state.rs's retried_from test.

Add `test_support::test_workflow_version_id()` alongside `test_run_provenance()`
and use it everywhere, replacing eight copies of the same magic seed across five
crates plus two assertion sites that recomputed the hash inline. This also
subsumes retry.rs's private helper of the same shape.

Revert the `run_spec_json` parameterization in the projection round-trip test:
`RunProjection` is a `with_replacement` alias for the canonical type, so the
`Some` and `None` call sites exercise identical code.

Have the two run.created literals that mirror a `RunSpec` read the spec's
lineage field instead of hardcoding `None`, so the mirrors stay accurate once a
producer populates it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 12:39:02 -04:00
Scott Werner
27fd48c603 Persist workflow version lineage on runs 2026-08-21 12:39:02 -04:00
Bryan Helmkamp
ca6d9a46da
Merge remote-tracking branch 'origin/main' into fix/redaction-corrupts-executable-spec
# Conflicts:
#	lib/components/fabro-dump/src/lib.rs
#	lib/components/fabro-store/src/run_state.rs
#	lib/components/fabro-store/tests/serializable_projection.rs
#	lib/components/fabro-workflow/src/billing_rollup.rs
#	lib/components/fabro-workflow/src/run_lookup.rs
#	lib/components/fabro-workflow/src/runtime_store.rs
#	lib/foundation/fabro-api/tests/run_projection_round_trip.rs
#	lib/foundation/fabro-test/src/lib.rs
#	lib/foundation/fabro-types/src/run.rs
#	lib/foundation/fabro-types/src/run_event/run.rs
#	lib/foundation/fabro-types/src/run_projection.rs
#	lib/foundation/fabro-types/tests/run_spec_methods.rs
2026-08-20 20:42:20 -04:00
Bryan Helmkamp
e688ee59a7
Merge main into feat/live-run-billing-totals 2026-08-20 18:20:10 -04:00
Scott Werner
19aa5940ea Add a shared RunSpec test fixture and adopt it
`RunSpec` has 13 fields and no `Default`, so every test that needed one
spelled out all 13 even when it cared about one or two. That put 64
hand-rolled `RunSpec { .. }` literals in `lib/`, and made a single
additive field cost a mechanical edit at roughly 30 sites.

Add `test_run_spec()` to `fabro-types`' feature-gated `test_support`
module: fixed `fixtures::RUN_1`, default settings, a minimal `test`
graph, `test_run_provenance()`, and every optional field unset. Tests
now spread it and only spell out what they assert on.

Adopt it at the 13 literals where the spread removes real duplication,
including the crate-local `test_run_spec` helpers in `fabro-store` and
`fabro-workflow`, which are now defined in terms of the shared fixture.
Tests that populate every field on purpose — the exhaustive `RunSpec`
serde round-trip in particular — keep spelling it out.

No production code and no behavior changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 16:50:37 -04:00
Scott Werner
a35dacd46c Import legacy blobs strictly into SQLite 2026-08-19 15:45:41 -04:00
Scott Werner
7b47ef2d05 Cover missing SQLite blob reads 2026-08-18 17:43:22 -04:00
Scott Werner
01efe7c883 Document BlobBackend as a transitional enum
Mark the Slate arm as temporary and record that the SQLite arm's
verified-read and hash-conflict semantics are the intended end state,
so the dual-backend enum reads as a rollout vehicle rather than a
permanent abstraction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 17:41:29 -04:00