Stabilize the recovery scenario around rebuilt metadata timing and node
ordinals, make in-process run cancellation converge on a cancelled
reason, and keep the label assertion unit test out of the shared
TestContext session lifecycle.
Replace the slow CLI integration tests that waited on worker shutdown
grace periods with focused coverage that still checks the important
behavior. The attach JSON test now finishes the gated run cleanly,
the rm force test uses a mocked server contract, and the Ctrl-C cancel
path is covered at the attach layer instead of through a full live run.
Add a cooperative subprocess cancel control message so cancel and delete
can abort pending interviews without relying only on the 5 second hard
kill fallback.
Replace bin-scoped localhost HTTP tests with command-facing integration
coverage so they run under the intended IT timeout budget without
changing nextest overrides.
Completed runs can briefly retain a stale worker PID after their terminal
state is visible. Using the full 5s worker cancellation grace in that window
made rm and prune pay an avoidable delay.
Keep the existing grace for active runs, but use a short delete grace for
already-terminal runs so completed-run cleanup stays fast.
Remove leftover object-backed terminology from the worker uploader,
rename the remaining scratch-fallback test to match current behavior,
and update the old artifact upload plan to reflect the current
no-fallback model.
Drop compatibility versioning from run-definition blobs, remove the
read-after-write polling added around CAS access, and tighten tests to
assert workflow_bundle.json is never written.
Drop the dead artifact storage capability split from run records,
run.created events, and workflow/server create paths. Worker artifact
upload is now unconditional, and tests/snapshots no longer encode a
legacy object-backed distinction.
Persist submitted run manifests and accepted run definitions as SHA256
blob refs on run events, remove workflow_bundle.json from the runtime
path, and stop deleting shared CAS blobs when removing runs.
Collapse expensive CLI smoke coverage into scenario tests, replace the
slow doctor no-color integration check with a unit-level render test,
and remove duplicate attach coverage. Also fix local Unix-socket
autostart so missing daemons don't spend the full 5s readiness wait
before startup.
The commit includes the measured slow-test report updates for the work
landed here.
Make ArtifactStore the only artifact read path, stop writing
manifest.json into run scratch, and update the CLI summary to
resolve artifact paths from the durable server API.
Stop writing scratch final.patch files now that diffs are projected from
run state, and remove the unused cache/artifacts/values plumbing while
keeping runtime/blobs materialization intact.
Update tests and run-directory docs to match the current scratch contract.
Route subprocess worker stderr directly into server tracing and remove
the scratch-file sink. Update the run-directory docs to reflect that
runtime now only documents blob materialization here.
Drop scratch-only compatibility paths and legacy test scaffolding now that
SlateDB-backed state is authoritative. This removes scratch file fallbacks,
updates docs and UI labels, and moves tests onto durable store-backed helpers.
Guard server worker cleanup against superseded subprocesses so rewind and
resume flows do not append a synthetic failure from an older worker. Update
CLI snapshots for the current interview events and give the shared test
session lock more time to cover daemon startup and shutdown.
Collapse the live answer rendezvous into ControlInterviewer, move pending
question storage onto a shared typed record, and route HTTP and Slack answer
submission through one server-side flow.
Persist pending interviews in run state, deliver accepted answers to workers
through the server-owned control path, and remove the old scratch-file and
WebInterviewer transports.
This also moves Slack onto the canonical server answer flow, adds richer
question metadata to the API and run events, and covers the subprocess
question lifecycle with end-to-end tests.
Share one store-dump export pipeline across the server-backed CLI path
and the local test helper, and add mixed blob-ref plus artifact coverage
for the exported output.
Route `fabro store dump` through the server client for run state, events,
blob hydration, and artifact downloads instead of reopening storage directly
from the CLI process. This fixes blob-backed checkpoint exports and restores
store-dump coverage under the in-memory test server.
Reapply the lint-safe changes that were partially displaced while merging
origin/main, including the billing serialization assertion and the attach
replay/server annotation cleanups. This keeps the merged main branch back to a
clean full-workspace clippy pass before the store-dump debugging continues.
Tighten the worker upload path so object-backed runs only fail when an
artifact upload is actually attempted without a token, and update CLI
snapshots for the new artifact storage metadata.
Fold in the workspace test and clippy fixes needed to verify the final
artifact upload implementation cleanly across Rust and web targets.
Store large context payloads in the global CAS and keep durable
checkpoint state as blob://sha256 refs instead of execution-local file
paths. Resolve and materialize blob refs at execution, output, and export
time so resumed and remote runs can read legacy and new artifacts
consistently.
Add scoped worker upload tokens and HTTP artifact upload clients.
Support manifest-first multipart stage artifact uploads with validation and checksums.
Gate artifact reads by run capability while preserving legacy scratch fallback.
Active runs deleted through rm --force were removed from server state
without signalling the worker process, which could leave detached
workers orphaned after test cleanup. Terminate the tracked worker
process group before deleting run state and cover it with an
integration regression.
Replace the overlapping usage and cost model with canonical billing
primitives centered on ModelRef, ModelHandle, TokenCounts, and
BilledModelUsage. This also renames the public API and web surface from
usage to billing, removes compatibility aliases, and normalizes provider
usage adapters onto the shared billing vocabulary.
Move detached workers onto an HTTP-backed runtime store so the server
remains the only SlateDB owner. This replaces the worker's seeded local
RunDatabase with a canonical server-backed handle for state, events, and
blobs, and updates workflow runtime plumbing to use that abstraction.
Replay persisted run events for attach requests, keep the SSE stream live
only while the run is active, and close on terminal run events instead of
returning 410 for completed runs.
The CLI now treats premature attach EOF as an error, and the affected
integration tests were stabilized around store-backed event ordering and
recovered rewind timelines.
Persist server, client, and subject provenance on run creation so
run state and inspect output can show which Fabro version created a
run, which first-party client submitted it, and how the request was
authenticated.
Replace the attach polling loop with the existing run attach SSE endpoint.
Seed from stored history once, fetch interview questions only when needed,
and keep completed-run replay behavior intact.
