Commit graph

3 commits

Author SHA1 Message Date
Bryan Helmkamp
164d9dcfbc
Move variables to SQLite storage (#537)
## Summary

This moves workflow-visible variables from JSON file storage into
SQLite-backed storage, establishing the first durable SQL table while
preserving the existing variable API behavior.

## What Changed

- Added a `fabro-db` crate with bundled SQLite, an embedded migration
for the `variables` table, and a `Database` owner for `connect()`,
`migrate()`, `health_check()`, and pool access.
- Replaced the `fabro-variable` JSON file store with an async
SQLx-backed `VariableStore` that preserves sorted listing,
case-sensitive names, empty string values, name validation, and
description-preserving upserts.
- Wired server startup to create `<storage>/db/fabro.sqlite3`, run
SQLite migrations, import legacy variables when needed, and pass the
shared pool into server state.
- Grouped live server stores under `AppStores` so runs, variables,
vault, environments, and automations share one state boundary while
artifacts remain separate.
- Updated variable handlers, run creation, validation, and test support
for async SQLite-backed variable access.
- Added schema, store-level, legacy import, and API-level persistence
coverage for variables.

## Legacy JSON Migration

On startup, Fabro looks for `<storage>/variables.json`. If it is
missing, startup is a no-op for legacy variables.

If the file exists, Fabro parses and validates the full file before
mutating SQLite. Valid entries are inserted with `ON CONFLICT(name) DO
NOTHING`, so existing SQLite values remain authoritative and only
missing names are imported from the legacy file.

After a successful import transaction, the source file is renamed to a
timestamped backup such as `variables.json.imported-<timestamp>.bak`. A
later startup naturally skips the import because the original source
path no longer exists. Invalid JSON or invalid variable names leave the
source file in place for operator repair.

Variable values are not logged during import. Logs include only safe
metadata such as source/backup paths, row counts, and variable names.

## Verification

- `cargo nextest run -p fabro-db -p fabro-variable`
- `cargo nextest run -p fabro-server --features test-support variables`
- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D
warnings`

---

[![Compound
Engineering](https://img.shields.io/badge/Compound_Engineering-6366f1)](https://github.com/EveryInc/compound-engineering-plugin)
Generated with GPT-5 via [Codex](https://openai.com/codex)
2026-06-30 12:17:46 -04:00
Scott Werner
a483402092
Interpolate run variables ({{ vars.* }}) in node prompts and goals (#524)
## What

Threads the run's variable store through the workflow transform pipeline
so
node `prompt`s and the graph `goal` can interpolate `{{ vars.* }}`.

Until now `{{ vars.* }}` only resolved in settings-level fields (e.g.
`run.goal`) via the server-side `substitute_variables` pass. Node
prompts are
DOT graph attributes that pass never touched, so `{{ vars.* }}` in a
prompt
rendered as undefined. This closes that gap.

Builds on the earlier template-context slice (adds `vars` to
`TemplateContext`); this PR wires it end to end.

## How

- `TransformOptions` carries a `vars` map, threaded into the import,
  file-inlining, and template transforms — and propagated into imported
subgraphs, so imported prompts interpolate vars too. Every prompt/goal
render
  context gains the variable map.
- The create API accepts `vars` (`CreateRunInput` →
`preprocess_and_validate` →
  `TransformOptions`).
- The server snapshots its `VariableStore` at run creation
  (`VariableStore::value_map()`) and passes it in — the same store the
  settings-goal substitution already reads.

## Scope decisions

- Goal `@file` contents interpolate vars too; **import paths stay
inputs-only**
(structural file resolution, conceptually outside the prompt/goal
scope).
- Offline / CLI / `fabro validate` render with an empty var map, so
`{{ vars.* }}` is undefined there: a warning at validate, a hard error
at
  run-create — identical to how `inputs` behaves offline.

## Testing

- Transform-level: node-prompt and goal interpolation; unknown-var
warning.
- Create-pipeline: vars resolve; an unknown var warns at validate and
promotes
  to a hard error at run-create.
- End-to-end server test: `POST /variables` + `POST /runs`, asserting
the
  rendered prompt in the persisted `run.created` event.

Verified: `cargo +nightly fmt --check`, nightly `clippy -D warnings`
(including
the `test-support`-gated server integration binary), the tests above,
and a
full-workspace `cargo check`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 16:18:09 -04:00
Bryan Helmkamp
b1bd2f522c
feat(server): add variables API (#430)
## Summary

Adds a workflow-visible variables store and HTTP API for managing
non-sensitive run variables, then wires those variables into run config
interpolation before run creation, validation, and preflight.

## What Changed

- Adds `/api/v1/variables` CRUD endpoints backed by a JSON variable
store and generated Rust/TypeScript API types.
- Supports `{{ vars.NAME }}` interpolation alongside existing `{{
env.NAME }}` handling for run-owned config fields, including
environment, MCP, hook, artifact, checkpoint, SCM, and notification
settings.
- Reuses canonical `fabro-types` variable DTOs in `fabro-api` and adds
OpenAPI name patterns so clients see the same env-style variable
contract enforced by the server.
- Keeps variable updates store-owned with `update_existing`, avoiding
duplicated not-found/update semantics in the HTTP handler.
- Shares env-style name validation between variables, interpolation
parsing, and vault token names to avoid grammar drift.

Variables are intentionally non-sensitive: list/get responses include
values, unlike vault secrets.

## Validation

- `cargo +nightly-2026-04-14 fmt --check --all`
- `cargo test -p fabro-types`
- `cargo test -p fabro-variable`
- `cargo test -p fabro-api --test variable_round_trip`
- `cargo test -p fabro-server --features test-support --test it
api::variables`
- `cargo +nightly-2026-04-14 clippy -p fabro-types -p fabro-variable -p
fabro-vault --all-targets -- -D warnings`
- `cargo +nightly-2026-04-14 clippy -p fabro-server --features
test-support --all-targets -- -D warnings`

---

[![Compound
Engineering](https://img.shields.io/badge/Compound_Engineering-6366f1)](https://github.com/EveryInc/compound-engineering-plugin)
🤖 Generated with GPT-5 via [Codex](https://openai.com/codex/)
2026-05-27 11:46:36 -04:00