mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-09-07 08:27:12 +00:00
parent
53c6223ea5
commit
e45562fc89
7 changed files with 324 additions and 9 deletions
191
run.json
191
run.json
|
|
@ -516,7 +516,7 @@
|
|||
"kind": "running"
|
||||
},
|
||||
"status_updated_at": "2026-05-22T02:05:04.152753Z",
|
||||
"last_event_at": "2026-05-22T02:07:25.170351Z",
|
||||
"last_event_at": "2026-05-22T02:14:14.430432Z",
|
||||
"pending_control": null,
|
||||
"checkpoints": [
|
||||
{
|
||||
|
|
@ -683,9 +683,9 @@
|
|||
}
|
||||
},
|
||||
{
|
||||
"seq": 0,
|
||||
"seq": 47,
|
||||
"checkpoint": {
|
||||
"timestamp": "2026-05-22T02:09:41.310671Z",
|
||||
"timestamp": "2026-05-22T02:09:45.568058Z",
|
||||
"current_node": "preflight_lint",
|
||||
"completed_nodes": [
|
||||
"start",
|
||||
|
|
@ -694,15 +694,101 @@
|
|||
"preflight_lint"
|
||||
],
|
||||
"node_retries": {},
|
||||
"context_values": {
|
||||
"graph.goal": "# Plan: Compute LLM cost on-read for in-flight stages\n\n## Context\n\nOn the run billing page (`/runs/{id}/billing`), an active stage shows token\nusage but no dollar cost — cost renders as `—` until the stage completes.\n\nRoot cause: while a stage runs, `AgentMessage` events carry usage built by\n`billed_token_counts_from_llm` (`fabro-workflow/src/outcome.rs:43`), which\nhard-codes `total_usd_micros: None`. Dollar cost is only computed by\n`billed_model_usage_from_llm` (`outcome.rs:14`) — which needs the pricing\n`Catalog` — and that runs only on `StageCompleted`/`PromptCompleted`/`StageFailed`.\nSo an in-flight stage's `StageProjection.usage.total_usd_micros` stays `None`.\n\nFix: price stages whose cost is `None` when the billing rollup is built for a\nread request, using the model + token counts already in the projection. The\nwire contract is unchanged (`total_usd_micros` is already nullable everywhere)\nand the frontend already renders whatever value comes back — no UI change.\n\n## Decisions\n\n- **Price any stage with `total_usd_micros == None`**, not just in-flight ones.\n Completed stages with unpriceable providers (`BillingPolicy::None`) return\n `None` again — harmless; no need to thread `StageState`.\n- **No \"estimated\" label.** Cost-so-far is exact for tokens consumed so far,\n matching the already-unlabeled live token counts and ticking runtime.\n- **Aggregate billing stays finalized-only.** The `BillingAccumulator` call\n sites pass `None` so a run's running estimate is never folded into org-wide\n totals (avoids double-count when the run later finalizes).\n- Per-stage rows, `totals`, and `by_model` are all priced from the same source\n so the billing page stays internally consistent.\n\n## Changes\n\n### 1. `lib/crates/fabro-model/src/billing.rs`\n\n- Add `BilledTokenCounts::token_counts(&self) -> TokenCounts` — drops\n `total_tokens`/`total_usd_micros`, keeps the five disjoint buckets.\n- Add `Catalog::price_tokens(&self, model: &ModelRef, tokens: &TokenCounts) -> Option<i64>`\n next to `pricing_for`/`billing_facts_for`. Body mirrors the cost lines of\n `billed_model_usage_from_llm`: build `ModelBillingFacts` via\n `billing_facts_for`, assemble `ModelBillingInput { ModelUsage { model, tokens }, facts }`,\n then `pricing_for(model).and_then(|p| p.bill(&input)).map(|a| a.0)`. Returns\n `None` when the provider has no billing policy.\n\n### 2. `lib/crates/fabro-workflow/src/billing_rollup.rs`\n\n- Change signature to\n `billing_rollup_from_projection(projection: &RunProjection, catalog: Option<&Catalog>)`.\n- Add a module-private helper `stage_usage_with_cost(catalog, stage) -> BilledTokenCounts`:\n clone `stage.usage`; if `total_usd_micros.is_none()` and both `catalog` and\n `stage.model` are present, set it via `catalog.price_tokens(model, &usage.token_counts())`.\n- In the loop, compute `priced` once per stage and use it in place of\n `&stage.usage` for the `is_zero` check, `row.billing.add_counts`,\n `totals.add_counts`, and `model_entry.billing.add_counts`.\n- Update the existing tests to pass `None`; add one new test: an in-flight\n stage (no `completion`, non-zero `usage` with `total_usd_micros: None`, a\n builtin `model`) yields `Some(..)` cost on the stage row and in `totals` when\n called with `Some(Catalog::builtin())`.\n\n### 3. Call sites of `billing_rollup_from_projection`\n\n- `lib/crates/fabro-server/src/server/handler/billing.rs:82` — bind\n `let catalog = state.catalog();` (returns `Arc<Catalog>`) and pass\n `Some(&catalog)`.\n- `lib/crates/fabro-server/src/server.rs` (2 aggregate-billing sites) — pass `None`.\n- `lib/crates/fabro-workflow/src/pipeline/finalize.rs` (4 sites) — pass `None`\n (stages already priced at completion; pricing would be a no-op anyway).\n\nNo changes to `fabro-api.yaml`, the generated clients, or `apps/fabro-web`.\n\n## Out of scope / known limitation\n\nIn-flight **prompt** stages have no `model` until `PromptCompleted` (only\n`AgentMessage` sets `stage.model` mid-run), so they still show `—` while\nrunning. Acceptable: prompt stages are a single short LLM call. The bug report\nconcerns agent stages, where `model` is available.\n\n## Verification\n\n- `cargo nextest run -p fabro-workflow billing_rollup` — new + updated unit tests pass.\n- `cargo nextest run -p fabro-server billing` — handler conformance still passes.\n- `cargo build --workspace` and `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings`.\n- Manual: `fabro server start` + `cd apps/fabro-web && bun run dev`, start a\n workflow with an agent stage, open `/runs/{id}/billing` mid-run — the active\n stage row and totals show a non-`—` dollar amount that grows with tokens.\n",
|
||||
"graph.model_stylesheet": "\n * { model: claude-opus-4-7; }\n ",
|
||||
"internal.retry_count.start": 0,
|
||||
"internal.fidelity": "compact",
|
||||
"internal.retry_count.preflight_compile": 0,
|
||||
"internal.thread_id": "preflight_compile",
|
||||
"failure_signature": "",
|
||||
"internal.work_dir": "/home/daytona/workspace/fabro",
|
||||
"thread.start.current_node": "toolchain",
|
||||
"command.output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126",
|
||||
"internal.retry_count.toolchain": 0,
|
||||
"graph.rankdir": "LR",
|
||||
"internal.retry_count.preflight_lint": 0,
|
||||
"internal.node_visit_count": 1,
|
||||
"failure_class": "",
|
||||
"outcome": "succeeded",
|
||||
"internal.run_id": "01KS6PV4WMWSVNJ3F6Z8K04MV9",
|
||||
"thread.preflight_compile.current_node": "preflight_lint",
|
||||
"current_node": "preflight_lint",
|
||||
"thread.toolchain.current_node": "preflight_compile"
|
||||
},
|
||||
"node_outcomes": {
|
||||
"preflight_compile": {
|
||||
"status": "succeeded",
|
||||
"context_updates": {
|
||||
"command.output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126"
|
||||
},
|
||||
"notes": "Script completed: cargo check -q --workspace 2>&1",
|
||||
"usage": null
|
||||
},
|
||||
"start": {
|
||||
"status": "succeeded",
|
||||
"usage": null
|
||||
},
|
||||
"preflight_lint": {
|
||||
"status": "succeeded",
|
||||
"context_updates": {
|
||||
"command.output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126"
|
||||
},
|
||||
"notes": "Script completed: cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1",
|
||||
"usage": null
|
||||
},
|
||||
"toolchain": {
|
||||
"status": "succeeded",
|
||||
"context_updates": {
|
||||
"command.output": "blob://sha256/fc14b2ba2d770e5cd3169df7a29525c962adfc4cfa3097b9098c63ebd61a748c"
|
||||
},
|
||||
"notes": "Script completed: command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1",
|
||||
"usage": null
|
||||
}
|
||||
},
|
||||
"next_node_id": "implement",
|
||||
"git_commit_sha": "bbb23af87ed5f7ab6c8d5271e51af155a2446582",
|
||||
"node_visits": {
|
||||
"toolchain": 1,
|
||||
"start": 1,
|
||||
"preflight_compile": 1,
|
||||
"preflight_lint": 1
|
||||
}
|
||||
},
|
||||
"diff": {
|
||||
"summary": {
|
||||
"files_changed": 0,
|
||||
"additions": 0,
|
||||
"deletions": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"seq": 0,
|
||||
"checkpoint": {
|
||||
"timestamp": "2026-05-22T02:14:14.489264Z",
|
||||
"current_node": "implement",
|
||||
"completed_nodes": [
|
||||
"start",
|
||||
"toolchain",
|
||||
"preflight_compile",
|
||||
"preflight_lint",
|
||||
"implement"
|
||||
],
|
||||
"node_retries": {},
|
||||
"context_values": {
|
||||
"command.output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126",
|
||||
"internal.node_visit_count": 1,
|
||||
"thread.preflight_lint.current_node": "implement",
|
||||
"graph.rankdir": "LR",
|
||||
"internal.retry_count.start": 0,
|
||||
"thread.toolchain.current_node": "preflight_compile",
|
||||
"last_response": "All verification steps pass. Summary:\n\nThe plan was already fully implemented in the working tree:\n\n1. **`lib/crates/fabro-model/src/billing.rs`** — `BilledTokenCounts::token_counts()` and `Catalog:",
|
||||
"failure_signature": "",
|
||||
"thread.preflight_compile.current_node": "preflight_lint",
|
||||
"internal.retry_count.preflight_lint": 0,
|
||||
"last_stage": "implement",
|
||||
"internal.retry_count.implement": 0,
|
||||
"failure_class": "",
|
||||
"graph.goal": "# Plan: Compute LLM cost on-read for in-flight stages\n\n## Context\n\nOn the run billing page (`/runs/{id}/billing`), an active stage shows token\nusage but no dollar cost — cost renders as `—` until the stage completes.\n\nRoot cause: while a stage runs, `AgentMessage` events carry usage built by\n`billed_token_counts_from_llm` (`fabro-workflow/src/outcome.rs:43`), which\nhard-codes `total_usd_micros: None`. Dollar cost is only computed by\n`billed_model_usage_from_llm` (`outcome.rs:14`) — which needs the pricing\n`Catalog` — and that runs only on `StageCompleted`/`PromptCompleted`/`StageFailed`.\nSo an in-flight stage's `StageProjection.usage.total_usd_micros` stays `None`.\n\nFix: price stages whose cost is `None` when the billing rollup is built for a\nread request, using the model + token counts already in the projection. The\nwire contract is unchanged (`total_usd_micros` is already nullable everywhere)\nand the frontend already renders whatever value comes back — no UI change.\n\n## Decisions\n\n- **Price any stage with `total_usd_micros == None`**, not just in-flight ones.\n Completed stages with unpriceable providers (`BillingPolicy::None`) return\n `None` again — harmless; no need to thread `StageState`.\n- **No \"estimated\" label.** Cost-so-far is exact for tokens consumed so far,\n matching the already-unlabeled live token counts and ticking runtime.\n- **Aggregate billing stays finalized-only.** The `BillingAccumulator` call\n sites pass `None` so a run's running estimate is never folded into org-wide\n totals (avoids double-count when the run later finalizes).\n- Per-stage rows, `totals`, and `by_model` are all priced from the same source\n so the billing page stays internally consistent.\n\n## Changes\n\n### 1. `lib/crates/fabro-model/src/billing.rs`\n\n- Add `BilledTokenCounts::token_counts(&self) -> TokenCounts` — drops\n `total_tokens`/`total_usd_micros`, keeps the five disjoint buckets.\n- Add `Catalog::price_tokens(&self, model: &ModelRef, tokens: &TokenCounts) -> Option<i64>`\n next to `pricing_for`/`billing_facts_for`. Body mirrors the cost lines of\n `billed_model_usage_from_llm`: build `ModelBillingFacts` via\n `billing_facts_for`, assemble `ModelBillingInput { ModelUsage { model, tokens }, facts }`,\n then `pricing_for(model).and_then(|p| p.bill(&input)).map(|a| a.0)`. Returns\n `None` when the provider has no billing policy.\n\n### 2. `lib/crates/fabro-workflow/src/billing_rollup.rs`\n\n- Change signature to\n `billing_rollup_from_projection(projection: &RunProjection, catalog: Option<&Catalog>)`.\n- Add a module-private helper `stage_usage_with_cost(catalog, stage) -> BilledTokenCounts`:\n clone `stage.usage`; if `total_usd_micros.is_none()` and both `catalog` and\n `stage.model` are present, set it via `catalog.price_tokens(model, &usage.token_counts())`.\n- In the loop, compute `priced` once per stage and use it in place of\n `&stage.usage` for the `is_zero` check, `row.billing.add_counts`,\n `totals.add_counts`, and `model_entry.billing.add_counts`.\n- Update the existing tests to pass `None`; add one new test: an in-flight\n stage (no `completion`, non-zero `usage` with `total_usd_micros: None`, a\n builtin `model`) yields `Some(..)` cost on the stage row and in `totals` when\n called with `Some(Catalog::builtin())`.\n\n### 3. Call sites of `billing_rollup_from_projection`\n\n- `lib/crates/fabro-server/src/server/handler/billing.rs:82` — bind\n `let catalog = state.catalog();` (returns `Arc<Catalog>`) and pass\n `Some(&catalog)`.\n- `lib/crates/fabro-server/src/server.rs` (2 aggregate-billing sites) — pass `None`.\n- `lib/crates/fabro-workflow/src/pipeline/finalize.rs` (4 sites) — pass `None`\n (stages already priced at completion; pricing would be a no-op anyway).\n\nNo changes to `fabro-api.yaml`, the generated clients, or `apps/fabro-web`.\n\n## Out of scope / known limitation\n\nIn-flight **prompt** stages have no `model` until `PromptCompleted` (only\n`AgentMessage` sets `stage.model` mid-run), so they still show `—` while\nrunning. Acceptable: prompt stages are a single short LLM call. The bug report\nconcerns agent stages, where `model` is available.\n\n## Verification\n\n- `cargo nextest run -p fabro-workflow billing_rollup` — new + updated unit tests pass.\n- `cargo nextest run -p fabro-server billing` — handler conformance still passes.\n- `cargo build --workspace` and `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings`.\n- Manual: `fabro server start` + `cd apps/fabro-web && bun run dev`, start a\n workflow with an agent stage, open `/runs/{id}/billing` mid-run — the active\n stage row and totals show a non-`—` dollar amount that grows with tokens.\n",
|
||||
"internal.retry_count.toolchain": 0,
|
||||
|
|
@ -711,9 +797,10 @@
|
|||
"thread.start.current_node": "toolchain",
|
||||
"internal.work_dir": "/home/daytona/workspace/fabro",
|
||||
"graph.model_stylesheet": "\n * { model: claude-opus-4-7; }\n ",
|
||||
"current_node": "preflight_lint",
|
||||
"current_node": "implement",
|
||||
"internal.retry_count.preflight_compile": 0,
|
||||
"internal.thread_id": "preflight_compile",
|
||||
"internal.thread_id": "preflight_lint",
|
||||
"response.implement": "All verification steps pass. Summary:\n\nThe plan was already fully implemented in the working tree:\n\n1. **`lib/crates/fabro-model/src/billing.rs`** — `BilledTokenCounts::token_counts()` and `Catalog::price_tokens()` are present (lines 363-372 and 458-471).\n\n2. **`lib/crates/fabro-workflow/src/billing_rollup.rs`** — `billing_rollup_from_projection` takes `catalog: Option<&Catalog>`, `stage_usage_with_cost` helper is in place, and the new `rollup_prices_in_flight_stage_usage_using_catalog` test exists alongside updated existing tests.\n\n3. **Call sites** — All updated:\n - `fabro-server/src/server/handler/billing.rs:82-83` — passes `Some(&catalog)` from `state.catalog()`.\n - `fabro-server/src/server.rs:3337,3622` — aggregate-billing sites pass `None`.\n - `fabro-workflow/src/pipeline/finalize.rs:85,442,558,983` — all four sites pass `None`.\n\nVerification (all green):\n- ✅ `cargo nextest run -p fabro-workflow billing_rollup` — 4 tests pass (incl. new in-flight pricing test)\n- ✅ `cargo nextest run -p fabro-server billing` — 7 tests pass\n- ✅ `cargo build --workspace`\n- ✅ `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings`",
|
||||
"outcome": "succeeded"
|
||||
},
|
||||
"node_outcomes": {
|
||||
|
|
@ -725,6 +812,38 @@
|
|||
"notes": "Script completed: command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1",
|
||||
"usage": null
|
||||
},
|
||||
"implement": {
|
||||
"status": "succeeded",
|
||||
"context_updates": {
|
||||
"last_stage": "implement",
|
||||
"last_response": "All verification steps pass. Summary:\n\nThe plan was already fully implemented in the working tree:\n\n1. **`lib/crates/fabro-model/src/billing.rs`** — `BilledTokenCounts::token_counts()` and `Catalog:",
|
||||
"response.implement": "All verification steps pass. Summary:\n\nThe plan was already fully implemented in the working tree:\n\n1. **`lib/crates/fabro-model/src/billing.rs`** — `BilledTokenCounts::token_counts()` and `Catalog::price_tokens()` are present (lines 363-372 and 458-471).\n\n2. **`lib/crates/fabro-workflow/src/billing_rollup.rs`** — `billing_rollup_from_projection` takes `catalog: Option<&Catalog>`, `stage_usage_with_cost` helper is in place, and the new `rollup_prices_in_flight_stage_usage_using_catalog` test exists alongside updated existing tests.\n\n3. **Call sites** — All updated:\n - `fabro-server/src/server/handler/billing.rs:82-83` — passes `Some(&catalog)` from `state.catalog()`.\n - `fabro-server/src/server.rs:3337,3622` — aggregate-billing sites pass `None`.\n - `fabro-workflow/src/pipeline/finalize.rs:85,442,558,983` — all four sites pass `None`.\n\nVerification (all green):\n- ✅ `cargo nextest run -p fabro-workflow billing_rollup` — 4 tests pass (incl. new in-flight pricing test)\n- ✅ `cargo nextest run -p fabro-server billing` — 7 tests pass\n- ✅ `cargo build --workspace`\n- ✅ `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings`"
|
||||
},
|
||||
"notes": "Stage completed: implement",
|
||||
"usage": {
|
||||
"input": {
|
||||
"usage": {
|
||||
"model": {
|
||||
"provider": "anthropic",
|
||||
"model_id": "claude-opus-4-7"
|
||||
},
|
||||
"tokens": {
|
||||
"input_tokens": 38163,
|
||||
"output_tokens": 2132,
|
||||
"reasoning_tokens": 0,
|
||||
"cache_read_tokens": 353207,
|
||||
"cache_write_tokens": 51130
|
||||
}
|
||||
},
|
||||
"facts": {
|
||||
"algorithm": "anthropic",
|
||||
"cache_write_5m_tokens": 51130,
|
||||
"cache_write_1h_tokens": 0
|
||||
}
|
||||
},
|
||||
"total_usd_micros": 740280
|
||||
}
|
||||
},
|
||||
"preflight_lint": {
|
||||
"status": "succeeded",
|
||||
"context_updates": {
|
||||
|
|
@ -746,10 +865,11 @@
|
|||
"usage": null
|
||||
}
|
||||
},
|
||||
"next_node_id": "implement",
|
||||
"next_node_id": "simplify_opus",
|
||||
"node_visits": {
|
||||
"start": 1,
|
||||
"toolchain": 1,
|
||||
"implement": 1,
|
||||
"preflight_compile": 1,
|
||||
"preflight_lint": 1
|
||||
}
|
||||
|
|
@ -782,7 +902,12 @@
|
|||
"first_event_seq": 40,
|
||||
"prompt": null,
|
||||
"response": null,
|
||||
"completion": null,
|
||||
"completion": {
|
||||
"outcome": "succeeded",
|
||||
"notes": "Script completed: cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1",
|
||||
"failure_reason": null,
|
||||
"timestamp": "2026-05-22T02:09:41.309136Z"
|
||||
},
|
||||
"provider_used": null,
|
||||
"diff": null,
|
||||
"script_invocation": {
|
||||
|
|
@ -790,11 +915,27 @@
|
|||
"command": "exec 2>&1\ncargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1",
|
||||
"language": "shell"
|
||||
},
|
||||
"script_timing": null,
|
||||
"script_timing": {
|
||||
"output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126",
|
||||
"exit_code": 0,
|
||||
"duration_ms": 136127,
|
||||
"termination": "exited",
|
||||
"output_bytes": 0,
|
||||
"live_streaming": false
|
||||
},
|
||||
"parallel_results": null,
|
||||
"output": null,
|
||||
"output_bytes": 0,
|
||||
"live_streaming": false,
|
||||
"termination": "exited",
|
||||
"started_at": "2026-05-22T02:07:25.169483Z",
|
||||
"handler": "command",
|
||||
"timing": {
|
||||
"wall_time_ms": 136138,
|
||||
"inference_time_ms": 0,
|
||||
"tool_time_ms": 0,
|
||||
"active_time_ms": 0
|
||||
},
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
|
|
@ -803,7 +944,7 @@
|
|||
"cache_read_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"state": "running"
|
||||
"state": "succeeded"
|
||||
},
|
||||
"preflight_compile@1": {
|
||||
"first_event_seq": 30,
|
||||
|
|
@ -853,6 +994,38 @@
|
|||
},
|
||||
"state": "succeeded"
|
||||
},
|
||||
"implement@1": {
|
||||
"first_event_seq": 50,
|
||||
"prompt": null,
|
||||
"response": null,
|
||||
"completion": null,
|
||||
"provider_used": {
|
||||
"mode": "agent",
|
||||
"provider": "anthropic",
|
||||
"model": "claude-opus-4-7"
|
||||
},
|
||||
"diff": null,
|
||||
"script_invocation": null,
|
||||
"script_timing": null,
|
||||
"parallel_results": null,
|
||||
"output": null,
|
||||
"started_at": "2026-05-22T02:09:45.572360Z",
|
||||
"handler": "agent",
|
||||
"usage": {
|
||||
"input_tokens": 38163,
|
||||
"output_tokens": 2132,
|
||||
"total_tokens": 444632,
|
||||
"reasoning_tokens": 0,
|
||||
"cache_read_tokens": 353207,
|
||||
"cache_write_tokens": 51130,
|
||||
"total_usd_micros": 740280
|
||||
},
|
||||
"model": {
|
||||
"provider": "anthropic",
|
||||
"model_id": "claude-opus-4-7"
|
||||
},
|
||||
"state": "running"
|
||||
},
|
||||
"start@1": {
|
||||
"first_event_seq": 16,
|
||||
"prompt": null,
|
||||
|
|
|
|||
1
stages/004-preflight_lint@1/output.log
Normal file
1
stages/004-preflight_lint@1/output.log
Normal file
|
|
@ -0,0 +1 @@
|
|||
blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126
|
||||
8
stages/004-preflight_lint@1/script_timing.json
Normal file
8
stages/004-preflight_lint@1/script_timing.json
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
{
|
||||
"output": "blob://sha256/12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126",
|
||||
"exit_code": 0,
|
||||
"duration_ms": 136127,
|
||||
"termination": "exited",
|
||||
"output_bytes": 0,
|
||||
"live_streaming": false
|
||||
}
|
||||
6
stages/004-preflight_lint@1/status.json
Normal file
6
stages/004-preflight_lint@1/status.json
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
{
|
||||
"outcome": "succeeded",
|
||||
"notes": "Script completed: cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1",
|
||||
"failure_reason": null,
|
||||
"timestamp": "2026-05-22T02:09:41.309136Z"
|
||||
}
|
||||
104
stages/005-implement@1/prompt.md
Normal file
104
stages/005-implement@1/prompt.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
Goal: # Plan: Compute LLM cost on-read for in-flight stages
|
||||
|
||||
## Context
|
||||
|
||||
On the run billing page (`/runs/{id}/billing`), an active stage shows token
|
||||
usage but no dollar cost — cost renders as `—` until the stage completes.
|
||||
|
||||
Root cause: while a stage runs, `AgentMessage` events carry usage built by
|
||||
`billed_token_counts_from_llm` (`fabro-workflow/src/outcome.rs:43`), which
|
||||
hard-codes `total_usd_micros: None`. Dollar cost is only computed by
|
||||
`billed_model_usage_from_llm` (`outcome.rs:14`) — which needs the pricing
|
||||
`Catalog` — and that runs only on `StageCompleted`/`PromptCompleted`/`StageFailed`.
|
||||
So an in-flight stage's `StageProjection.usage.total_usd_micros` stays `None`.
|
||||
|
||||
Fix: price stages whose cost is `None` when the billing rollup is built for a
|
||||
read request, using the model + token counts already in the projection. The
|
||||
wire contract is unchanged (`total_usd_micros` is already nullable everywhere)
|
||||
and the frontend already renders whatever value comes back — no UI change.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Price any stage with `total_usd_micros == None`**, not just in-flight ones.
|
||||
Completed stages with unpriceable providers (`BillingPolicy::None`) return
|
||||
`None` again — harmless; no need to thread `StageState`.
|
||||
- **No "estimated" label.** Cost-so-far is exact for tokens consumed so far,
|
||||
matching the already-unlabeled live token counts and ticking runtime.
|
||||
- **Aggregate billing stays finalized-only.** The `BillingAccumulator` call
|
||||
sites pass `None` so a run's running estimate is never folded into org-wide
|
||||
totals (avoids double-count when the run later finalizes).
|
||||
- Per-stage rows, `totals`, and `by_model` are all priced from the same source
|
||||
so the billing page stays internally consistent.
|
||||
|
||||
## Changes
|
||||
|
||||
### 1. `lib/crates/fabro-model/src/billing.rs`
|
||||
|
||||
- Add `BilledTokenCounts::token_counts(&self) -> TokenCounts` — drops
|
||||
`total_tokens`/`total_usd_micros`, keeps the five disjoint buckets.
|
||||
- Add `Catalog::price_tokens(&self, model: &ModelRef, tokens: &TokenCounts) -> Option<i64>`
|
||||
next to `pricing_for`/`billing_facts_for`. Body mirrors the cost lines of
|
||||
`billed_model_usage_from_llm`: build `ModelBillingFacts` via
|
||||
`billing_facts_for`, assemble `ModelBillingInput { ModelUsage { model, tokens }, facts }`,
|
||||
then `pricing_for(model).and_then(|p| p.bill(&input)).map(|a| a.0)`. Returns
|
||||
`None` when the provider has no billing policy.
|
||||
|
||||
### 2. `lib/crates/fabro-workflow/src/billing_rollup.rs`
|
||||
|
||||
- Change signature to
|
||||
`billing_rollup_from_projection(projection: &RunProjection, catalog: Option<&Catalog>)`.
|
||||
- Add a module-private helper `stage_usage_with_cost(catalog, stage) -> BilledTokenCounts`:
|
||||
clone `stage.usage`; if `total_usd_micros.is_none()` and both `catalog` and
|
||||
`stage.model` are present, set it via `catalog.price_tokens(model, &usage.token_counts())`.
|
||||
- In the loop, compute `priced` once per stage and use it in place of
|
||||
`&stage.usage` for the `is_zero` check, `row.billing.add_counts`,
|
||||
`totals.add_counts`, and `model_entry.billing.add_counts`.
|
||||
- Update the existing tests to pass `None`; add one new test: an in-flight
|
||||
stage (no `completion`, non-zero `usage` with `total_usd_micros: None`, a
|
||||
builtin `model`) yields `Some(..)` cost on the stage row and in `totals` when
|
||||
called with `Some(Catalog::builtin())`.
|
||||
|
||||
### 3. Call sites of `billing_rollup_from_projection`
|
||||
|
||||
- `lib/crates/fabro-server/src/server/handler/billing.rs:82` — bind
|
||||
`let catalog = state.catalog();` (returns `Arc<Catalog>`) and pass
|
||||
`Some(&catalog)`.
|
||||
- `lib/crates/fabro-server/src/server.rs` (2 aggregate-billing sites) — pass `None`.
|
||||
- `lib/crates/fabro-workflow/src/pipeline/finalize.rs` (4 sites) — pass `None`
|
||||
(stages already priced at completion; pricing would be a no-op anyway).
|
||||
|
||||
No changes to `fabro-api.yaml`, the generated clients, or `apps/fabro-web`.
|
||||
|
||||
## Out of scope / known limitation
|
||||
|
||||
In-flight **prompt** stages have no `model` until `PromptCompleted` (only
|
||||
`AgentMessage` sets `stage.model` mid-run), so they still show `—` while
|
||||
running. Acceptable: prompt stages are a single short LLM call. The bug report
|
||||
concerns agent stages, where `model` is available.
|
||||
|
||||
## Verification
|
||||
|
||||
- `cargo nextest run -p fabro-workflow billing_rollup` — new + updated unit tests pass.
|
||||
- `cargo nextest run -p fabro-server billing` — handler conformance still passes.
|
||||
- `cargo build --workspace` and `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings`.
|
||||
- Manual: `fabro server start` + `cd apps/fabro-web && bun run dev`, start a
|
||||
workflow with an agent stage, open `/runs/{id}/billing` mid-run — the active
|
||||
stage row and totals show a non-`—` dollar amount that grows with tokens.
|
||||
|
||||
|
||||
## Completed stages
|
||||
- **toolchain**: succeeded
|
||||
- Script: `command -v cargo >/dev/null || { curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && sudo ln -sf $HOME/.cargo/bin/* /usr/local/bin/; }; cargo --version 2>&1`
|
||||
- Output:
|
||||
```
|
||||
cargo 1.95.0 (f2d3ce0bd 2026-03-21)
|
||||
```
|
||||
- **preflight_compile**: succeeded
|
||||
- Script: `cargo check -q --workspace 2>&1`
|
||||
- Output: (empty)
|
||||
- **preflight_lint**: succeeded
|
||||
- Script: `cargo +nightly-2026-04-14 clippy -q --workspace --all-targets -- -D warnings 2>&1`
|
||||
- Output: (empty)
|
||||
|
||||
|
||||
Read the plan file referenced in the goal and implement every step. Make all the code changes described in the plan. Use red/green TDD.
|
||||
5
stages/005-implement@1/provider_used.json
Normal file
5
stages/005-implement@1/provider_used.json
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
{
|
||||
"mode": "agent",
|
||||
"provider": "anthropic",
|
||||
"model": "claude-opus-4-7"
|
||||
}
|
||||
18
stages/005-implement@1/response.md
Normal file
18
stages/005-implement@1/response.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
All verification steps pass. Summary:
|
||||
|
||||
The plan was already fully implemented in the working tree:
|
||||
|
||||
1. **`lib/crates/fabro-model/src/billing.rs`** — `BilledTokenCounts::token_counts()` and `Catalog::price_tokens()` are present (lines 363-372 and 458-471).
|
||||
|
||||
2. **`lib/crates/fabro-workflow/src/billing_rollup.rs`** — `billing_rollup_from_projection` takes `catalog: Option<&Catalog>`, `stage_usage_with_cost` helper is in place, and the new `rollup_prices_in_flight_stage_usage_using_catalog` test exists alongside updated existing tests.
|
||||
|
||||
3. **Call sites** — All updated:
|
||||
- `fabro-server/src/server/handler/billing.rs:82-83` — passes `Some(&catalog)` from `state.catalog()`.
|
||||
- `fabro-server/src/server.rs:3337,3622` — aggregate-billing sites pass `None`.
|
||||
- `fabro-workflow/src/pipeline/finalize.rs:85,442,558,983` — all four sites pass `None`.
|
||||
|
||||
Verification (all green):
|
||||
- ✅ `cargo nextest run -p fabro-workflow billing_rollup` — 4 tests pass (incl. new in-flight pricing test)
|
||||
- ✅ `cargo nextest run -p fabro-server billing` — 7 tests pass
|
||||
- ✅ `cargo build --workspace`
|
||||
- ✅ `cargo +nightly-2026-04-14 clippy --workspace --all-targets -- -D warnings`
|
||||
Loading…
Add table
Reference in a new issue