Commit graph

134 commits

Author SHA1 Message Date
Bryan Helmkamp
619cb44e3c
Import lithos-llm types directly instead of through fabro-types
fabro-types no longer re-exports the lithos catalog and request types
(ProviderId, ModelId, ModelHandle, Message, ContentPart, TokenCounts,
Cost, Speed, ReasoningEffort, ReasoningOutput, and the rest). Every
crate that uses them depends on lithos-llm and names them there, and
the fabro-api progenitor replacements point at the lithos paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 10:03:08 -06:00
Bryan Helmkamp
82bcafcfca
Port fabro-workflow, hooks, validate, and store to the lithos types
Workflow LLM handlers build lithos requests, bill from lithos usage and
cost, and classify failures from lithos `ErrorKind`. Model resolution and
fallback use the fabro-llm selection and catalog helpers. Validation
rules read the lithos catalog, and store fixtures use the new
`BilledModelUsage` shape.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-09 17:26:57 -06:00
Bryan Helmkamp
a7ed03e175
Rebuild conclusion stage summaries from run events 2026-09-05 11:44:23 -04:00
Scott Werner
f52f2a1edb
Merge pull request #838 from fabro-sh/codex/persist-pre-start-worker-failures
Persist pre-start worker failures
2026-09-03 12:48:39 -04:00
Scott Werner
c437dc012d Simplify pre-start worker failure handling
Move the "which failures can happen before Starting" classification onto
FailureReason as an exhaustive predicate and use it for every
Runnable -> Failed transition, replacing the hand-maintained allowlist.
Give the pending-cancel precedence rule a single owner shared by the
worker launch and worker exit paths.

Test cleanups: share the Notify wait loop, server record fixture, and
post-failure assertions; simplify the pre-start test runtime's hold
flag; and parameterize the slate run.failed payload helper.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 11:45:47 -04:00
Scott Werner
54666632c8 Persist pre-start worker failures 2026-09-02 17:29:41 -04:00
Scott Werner
0f1e5e1c9c Centralize run projection replay on ProjectedRun
Move ProjectedRun next to EventProjectionCache in run_state, since the
summary store both produces and consumes it, and give it a replay
constructor that owns the events-to-head derivation. load_projection now
returns RunNotFound directly instead of erasing it to None and having
callers rebuild it; load_run_projection is the single Option translation
point. install_in_memory_state reuses the existing From impl, and the
commit path passes its Arc through instead of unwrapping and
reallocating it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 16:33:14 -04:00
Scott Werner
2360e8046b Move run projection replay into the summary store 2026-09-02 13:32:43 -04:00
Scott Werner
064074233c Narrow pull request recovery to unresolved creation requests
The supervisor's periodic recovery scan asked SQLite for every run that
had ever recorded an explicit pull request creation request and then
replayed each inactive candidate's full history to learn whether the
request was still pending. With projections now loaded on demand that
set grows without bound and was replayed every scan.

The candidate query now mirrors the projection reducer: a run is a
candidate only when its latest creation request has no later request,
created, linked, or unlinked event, and no later failure naming the same
creation id. Callers still replay each candidate to confirm, so the query
only has to avoid omitting a pending run, and the replayed set is bounded
by in-flight requests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 12:48:21 -04:00
Scott Werner
13f3ff867b Simplify on-demand run projection loading
Drop the unreachable active-run mismatch guard that was copied into
load_run_projection: the active-runs map is only ever inserted under the
handle's own run ID, so the check could never fire. Remove it from the two
pre-existing sites too and delete matches_run.

Trim install_in_memory_state to take only the committed projection, since
the event envelope duplicated last_seq and the inner scope only existed to
release the lock before the now-removed shared cache update. Add a From
impl so RunDatabase::build no longer hand-builds EventProjectionCache, and
rename projected_state_locked to match its projection_snapshot sibling.

In fabro-server, have reject_if_archived and ensure_run_exists read the
run summary row instead of replaying the full event history for inactive
runs; the summary is written in the same transaction as the event.

Fold the repeated store-reopen fixtures in fabro-store and fabro-server
tests into helpers, and fix a stale comment about the deleted shared
projection cache.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 11:43:17 -04:00
Scott Werner
c1e9364490 Load inactive run projections on demand 2026-09-01 16:06:54 -04:00
Scott Werner
f2a2630ad0 Simplify pull request recovery and summary store queries
- Load recovery candidates from the warm projection cache instead of
  replaying each run's full event history, and check dispatch
  eligibility before any I/O
- Extract the shared can_dispatch predicate used by both the recovery
  scan and the worker dispatch loop
- Drop load_durable_run_status, now identical to durable_run_status
- Share parse_stored_run_id across the three stored-id decode sites
- Reuse push_order for the canonical run ordering in list_all and
  list_by_statuses
- Replace the test-only queue clear accessor with the existing drain

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 14:11:53 -04:00
Scott Werner
1a2d8a9056 Move run queries and pull request recovery to SQLite 2026-09-01 14:06:06 -04:00
Scott Werner
563ca26b98
Merge pull request #829 from fabro-sh/codex/session-ownership-index
Index session ownership from creation events
2026-09-01 14:03:25 -04:00
Scott Werner
0fd4714da7
Merge pull request #825 from swerner/codex/automation-workflow-sources
Add independent workflow sources to automations
2026-09-01 12:22:20 -04:00
Scott Werner
bf05b01ba6 Simplify session owner lookup and migration preflight
- Drop the post-decode run_id/session_id/event-body checks in
  find_session_owner: decode_event_row already verifies every stored
  column against the decoded envelope, and the WHERE clause pins
  session_id and event_name to the requested values.
- Build the lookup query from SELECT_EVENT_COLUMNS like the sibling
  event queries instead of duplicating the column list.
- Carry the stored run_id text in the unparseable-id error instead of
  an "<invalid>" placeholder.
- Fetch applied migration versions once per migrate() and share the
  set between the session-owner preflight and the pre-migration
  snapshot; check the applied version first so steady-state startups
  skip the sqlite_master probe. Mark the preflight as removable with
  the run-history compatibility window.
- Restore session_by_id_key as a #[cfg(test)] helper so tests stop
  hand-rolling the legacy reverse-index key shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 14:59:38 -04:00
Scott Werner
3411199103 Index session ownership from creation events 2026-08-31 14:16:58 -04:00
Scott Werner
39c018c430 Harden automation workflow source handling 2026-08-31 13:45:08 -04:00
Scott Werner
c68e67c841 Harden SQLite activation and run history consistency 2026-08-31 12:19:16 -04:00
Scott Werner
0574c818cf Share activation backup helpers and trim run-history residue
Hoist the SQLite backup/integrity helpers duplicated between the blob and
run-history activation migrations into one shared module, drop the
write-only recent-events buffer and other dead state left behind by the
SQLite cutover, reuse existing helpers for projection bootstrap, sequence
allocation, catalog keys, and test pool construction, and collapse the
tombstone and activation-marker plumbing to single statements.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 17:20:05 -04:00
Scott Werner
aa9c468d3e Wait for SQLite writer lock when deleting runs 2026-08-28 15:46:13 -04:00
Scott Werner
f073684227 Activate atomic SQLite run history storage 2026-08-28 15:18:25 -04:00
Scott Werner
4922fec45a Support historical run events during migration 2026-08-28 10:49:43 -04:00
Scott Werner
3591315ba6 Harden legacy run history verification 2026-08-28 10:13:03 -04:00
Scott Werner
a98c0db713 Import and verify legacy run history 2026-08-27 17:27:47 -04:00
Scott Werner
57bbb923c2 Keep the RunSummaryStore name until the SQL cutover
Revert the run summary -> run record rename. The SQLite store is still
the summary read model today; it only grows an inactive events table
here. Renaming it now made the store file show as a delete plus add and
touched nine unrelated files. The final rename happens once, when the
SQL store becomes the run authority.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 15:43:56 -04:00
Scott Werner
ff2aef4564 Simplify SQL run record store write path and test fixtures
Share one bind helper across the runs insert/upsert/update statements,
compute the next event sequence once per append, and decode stored
sequence columns through a single helper. Check the run head before
decoding events, rewrite the first-visit stage listing as a UNION ALL so
each arm uses its partial index, and share the run_events insert SQL
with the test seeder.

Collapse the duplicated in-memory pool fixture, remove two tests that
only asserted Arc sharing, and fold the fabro-db test row helpers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 14:59:34 -04:00
Scott Werner
b62e458289 Add inactive SQL run storage foundation 2026-08-27 13:32:01 -04:00
Scott Werner
dc1f235c48 Keep retired Slate helpers test-only 2026-08-24 17:31:15 -04:00
Scott Werner
f3ff7f27a4 Keep auth code store naming consistent 2026-08-24 17:26:17 -04:00
Scott Werner
68ef8c7e89 Leave old SlateDB authorization-code records in place
Drop the startup retirement of the auth/code keyspace instead of
carrying one-shot cleanup code forever. The records it deleted are
inert: at most a handful exist at cutover, every binary (old or new)
rejects them within 60 seconds of issue via the expiry check, and
nothing reads the keyspace after the move to SQLite. The refresh-token
retirement keeps its original inline shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 17:25:59 -04:00
Scott Werner
a2f0167844 Simplify SQLite authorization-code cutover
Cleanup pass over the pending-CLI-authorization move to SQLite:

- Extract a shared Database::retire_keyspace helper; the refresh-token
  and authorization-code retirements are now one-line wrappers over it.
- Inline the startup retirement call (dropping the single-use wrapper,
  its context-chain test, and the test_close_slate hook it required)
  and run both SlateDB retirement scans concurrently. Error policies
  are unchanged: authorization codes fatal, refresh tokens best-effort.
- Add a shared sqlite_row module with typed identity/timestamp row
  decoding, used by both AuthorizationCodeStore and AuthSessionStore;
  the session store's stringly Error::Other corruption errors become
  the typed InvalidStoredIdentity/InvalidStoredTimestamp variants.
- Delete Repository::gc, which had no production callers left and was
  kept alive by its own test; update the record-layer docs to match.
- Deduplicate the SQLite test-support bootstrap into sqlite_test_pool,
  reuse issue() in the invalid-timestamp test instead of a copied
  INSERT, and fold the new table into the existing existence-check loop
  in the fabro-db schema test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 17:25:38 -04:00
Scott Werner
05999036aa Move pending CLI authorizations to SQLite 2026-08-24 17:25:19 -04:00
Scott Werner
2d292c28f8
Merge pull request #783 from fabro-sh/codex/sqlite-blob-startup-activation
Activate verified SQLite blob storage at server startup
2026-08-24 16:32:59 -04:00
Bryan Helmkamp
c042b9abdc
Merge remote-tracking branch 'origin/main' into feature/bounded-agent-tool-output
# Conflicts:
#	lib/components/fabro-sandbox/src/clone_source.rs
2026-08-24 14:08:04 -04:00
Scott Werner
776e719383 Harden SQLite blob activation safety
Keep VACUUM snapshots private until permissions and durability are established. Refuse to recreate a missing rollback backup after import has begun, and preserve secondary cleanup failures in startup logs.
2026-08-24 14:02:35 -04:00
Scott Werner
bccc5750a4 Size the blob activation disk preflight to the remaining import work
The preflight demanded ~1.5x the full legacy inventory bytes free on
every startup, with no credit for rows already imported. Because the
first activation itself consumes about twice the legacy bytes (the
SQLite copy plus the retained backup) and the legacy keyspace stays in
place for the whole retention window, a successfully activated server
could fall below the requirement and become unable to restart until an
operator freed space the server would never write.

The legacy inventory now checks each row's hash against the SQLite blobs
table and reports pending rows and bytes, and the preflight requires
1.5x only the pending bytes plus the backup reserve and fixed headroom.
A warm restart with nothing left to import needs only the headroom.
Also updates the server operations doc for this and for the
verification pass now running only on boots that import rows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
94e3d46754 Cut redundant blob scans and hashing from server startup
Startup previously scanned the legacy SlateDB keyspace three times and
SHA-256-hashed every value in each pass (inventory, import,
verification), then read and rehashed every row of the live SQLite blobs
table — on every boot, even a warm restart with nothing to import. With
a large object-store-backed legacy keyspace that makes restart time
proportional to total blob bytes for the whole retention window.

The inventory pass now only validates key shapes and sizes the keyspace;
digests are still validated by the import pass before any row persists.
The independent verification sweep now runs only on boots whose import
actually inserted rows: the import pass itself byte-compares every
already-present legacy row each boot, so a no-op restart is already
fully cross-checked without a third scan or a full-table rehash.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
b56aee570c Pass the SQLite pool directly to the legacy blob import and verification
import_legacy_blobs_into and verify_legacy_blobs_in took a &BlobStore and
extracted its pool through sqlite_pool_for_legacy_import, an Option that
was statically always Some in production (the None arm existed only for
the test-only Slate backend). That accessor forced a clippy
unnecessary_wraps suppression and two WrongTargetBackend error variants
no production caller could ever hit, and the activation path round-tripped
a pool it already owned through a BlobStore it had just built.

Both functions now take &SqlitePool, deleting the accessor, the
suppression, both unreachable variants, and their rejection test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
21421dce78 Isolate the shared test blob store between tests
test_blob_store was a process-wide OnceLock singleton over one in-memory
SQLite connection, so content-addressed rows written by one test were
visible to every other test in the same process. nextest's
process-per-test model masked the bleed, but plain cargo test failed
(8/24 in fabro-workflow-version) because negative existence assertions
became order-dependent.

test_blob_store now builds a fresh isolated in-memory store per call,
and test_database gives every database its own blob authority.
Reopen-style tests that model one durable blob authority across several
store handles use the new test_blob_store_at, which keeps the blob table
in a SQLite file beside the store directory, plus
test_database_with_blobs to share it explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
d65785d888 Simplify blob activation and share the test store fixture
Blob activation cleanups:
- Reuse fabro-db's append_to_path, remove_file_if_exists, and
  set_private_permissions instead of local duplicates.
- Return the store directly from activate_blob_storage; the report
  wrapper existed only to be logged internally and then discarded.
- Collapse compute_disk_preflight to return the required free bytes
  instead of echoing its inputs back through a struct.
- Deduplicate the "exactly one ok row" PRAGMA integrity_check protocol
  into one executor-generic helper used by the backup and live checks.
- Skip re-validating a freshly published backup; the staging copy was
  validated immediately before the atomic rename, so only a
  concurrently published file needs its own validation.
- Replace the manual anyhow wrapping plus duplicate error log in
  serve.rs with a plain .context(), matching other startup errors.
- Extract the disk-candidate enumeration in resource_sampler.rs that
  available_space_for_path had copy-pasted from sample_disk_resources.

Test fixture cleanups:
- Route all hand-assembled Database::new(..., test_blob_store()) test
  fixtures (32 sites) through fabro_store::test_support::test_database,
  and make that helper infallible instead of returning an unconditional
  Ok.
- Install the test blob schema from fabro_db::BLOBS_MIGRATION_SQL via a
  test-support-gated optional dependency instead of a four-level
  relative include_str! into fabro-db's migrations directory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 14:02:35 -04:00
Scott Werner
2814c1fd45 Activate verified SQLite blob storage 2026-08-24 14:02:34 -04:00
Bryan Helmkamp
401acb6cdf
Record tool output byte counts 2026-08-24 12:46:27 -04:00
Scott Werner
e2db53011e Fix RunSpec test fixtures 2026-08-24 12:11:49 -04:00
Scott Werner
040bc6c043 Add version-backed run intent creation 2026-08-24 11:55:43 -04:00
Scott Werner
87b49a8527 Merge main into feat/refresh-tokens-sqlite
Preserve the SQLite auth-session release notes alongside main's July 26 fixes and retain all current changelog navigation entries. Make the refresh-token rotation timestamp assertion deterministic after the merged suite exposed its wall-clock race.
2026-08-24 11:04:10 -04:00
Bryan Helmkamp
09f5bb0f84
Simplify clone depth plumbing
Make RunCloneSettings::DEFAULT_DEPTH the single owner of the default
depth, and interpret the "0 = full history" sentinel in one place via
RunCloneSettings::depth_limit(). Docker's clone_depth becomes
Option<usize> to match Daytona's encoding, with a shared
depth_argument() helper for both git command builders. Drop the
unreachable Option on the resolved depth field, the hand-written
DaytonaSettings::Default, and the pure-forwarding
daytona_git_clone_options helper. The blob-import test helper reuses
the pool's own connect options instead of rebuilding a partial copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019bgXj5J218RXfiT72qhbLV
2026-08-21 18:37:39 -04:00
Bryan Helmkamp
fe1d9dc691
test: isolate SQLite checkpoint restoration 2026-08-21 17:57:22 -04:00
Scott Werner
891eb43e9a
Merge pull request #761 from fabro-sh/codex/strict-legacy-blob-import
Import legacy blobs strictly into SQLite
2026-08-21 16:30:45 -04:00
Scott Werner
3872c04430 Derive initial refresh token session state
Replace the public stored-token row with an initial-token input that carries only token-specific facts. Bind the token to the session and initialize it as unused inside AuthSessionStore so callers cannot create mismatched session/token rows.
2026-08-21 14:01:30 -04:00