fabro/lib/foundation/fabro-api/tests/run_failure_round_trip.rs
Bryan Helmkamp ef86ad278a
Carry usage as lithos-llm's Usage and rename billing to usage
Re-pin lithos-llm to 55add4596b861a0623d00c3a54aa5c147c8d504b and
pebble to c91810fe51aece80359b9cd8efea971af0c46925, where token usage
and cost travel together as Usage { tokens: TokenCounts, cost:
Option<Cost> }. Fabro now carries that one type everywhere it used to
carry BilledTokenCounts, BilledModelUsage, UsdMicros, or a token count
beside a cost_usd_micros.

fabro-types: billing.rs is usage.rs with ModelRef, ModelUsage { model,
usage }, sum_usage, and usage_is_empty; billing_rollup.rs is
usage_rollup.rs with ProjectionUsageStage, ProjectionUsageByModel,
ProjectionUsageRollup, and usage_rollup_from_projection. Every usage
field is named usage: StageProjection.usage and usage_by_model,
Outcome<Option<ModelUsage>>, stage.completed and stage.failed usage and
usage_by_model, prompt.completed usage, run.completed and run.failed
usage (total_usd_micros is gone), Conclusion.usage, StageSummary.usage,
Run.usage. RunSize buckets by Cost.

fabro-workflow: model_usage_from_llm prices tokens from the catalog with
a Catalog cost source, with_reported_cost keeps a provider cost, and the
pebble handler's stage_usage groups pebble's accounts by model and sums
rows with Usage::saturating_add, so a total has a cost only when every
priced part was priced. The store fold's live usage is the agent's
usage plus its descendants'.

API: the OpenAPI spec deletes BilledTokenCounts, BilledModelUsage,
CompletionUsage, CompletionCost, TokenUsage, and RunBillingSummary,
adds TokenCounts, Cost, Usage, and ModelUsage, and renames every
billing schema, property, tag, path, and operation to usage. fabro-api
reuses lithos-llm's and fabro-types' types through with_replacement,
with a round-trip test per replacement.

Old stored runs get no migration: their pebble events in the old shape
read back with zero usage, and their rebuilt projections lose agent
usage.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 12:31:34 -06:00

118 lines
4 KiB
Rust

use std::any::{TypeId, type_name};
use fabro_api::types::{
Conclusion as ApiConclusion, ExecOutputTail as ApiExecOutputTail,
FailureCategory as ApiFailureCategory, FailureDetail as ApiFailureDetail,
FailureSignature as ApiFailureSignature, RunFailure as ApiRunFailure,
};
use fabro_types::{
Conclusion, ExecOutputTail, FailureCategory, FailureDetail, FailureReason, FailureSignature,
RunFailure, StageOutcome,
};
use serde::Serialize;
use serde_json::{Value, json};
#[test]
fn run_failure_family_reuses_domain_types() {
assert_same_type::<ApiConclusion, Conclusion>();
assert_same_type::<ApiRunFailure, RunFailure>();
assert_same_type::<ApiFailureDetail, FailureDetail>();
assert_same_type::<ApiFailureCategory, FailureCategory>();
assert_same_type::<ApiFailureSignature, FailureSignature>();
assert_same_type::<ApiExecOutputTail, ExecOutputTail>();
}
#[test]
fn run_failure_json_matches_openapi_shape() {
assert_json(
RunFailure {
reason: FailureReason::SandboxInitFailed,
detail: {
let mut detail = FailureDetail::new(
"Failed to initialize sandbox",
FailureCategory::TransientInfra,
);
detail.causes = vec!["connection refused".to_string()];
detail.signature =
Some(FailureSignature("init|transient_infra|docker".to_string()));
detail.exec_output_tail = Some(ExecOutputTail {
stdout: None,
stderr: Some("last stderr line".to_string()),
stdout_truncated: false,
stderr_truncated: true,
});
detail
},
},
json!({
"reason": "sandbox_init_failed",
"detail": {
"message": "Failed to initialize sandbox",
"causes": ["connection refused"],
"category": "transient_infra",
"signature": "init|transient_infra|docker",
"exec_output_tail": {
"stderr": "last stderr line",
"stderr_truncated": true
}
}
}),
);
}
#[test]
fn conclusion_json_uses_failure_object() {
assert_json(
Conclusion {
timestamp: chrono::DateTime::parse_from_rfc3339("2026-05-13T12:00:00Z")
.unwrap()
.with_timezone(&chrono::Utc),
status: StageOutcome::Failed {
retry_requested: false,
},
timing: fabro_types::RunTiming::wall_only(42),
failure: Some(RunFailure {
reason: FailureReason::WorkflowError,
detail: FailureDetail::new("boom", FailureCategory::Deterministic),
}),
final_git_commit_sha: None,
stages: Vec::new(),
usage: None,
total_retries: 0,
diff: Default::default(),
},
json!({
"timestamp": "2026-05-13T12:00:00Z",
"status": "failed",
"timing": {
"wall_time_ms": 42,
"inference_time_ms": 0,
"tool_time_ms": 0,
"active_time_ms": 0
},
"failure": {
"reason": "workflow_error",
"detail": {
"message": "boom",
"category": "deterministic"
}
},
"total_retries": 0,
"diff": {}
}),
);
}
fn assert_same_type<T: 'static, U: 'static>() {
assert_eq!(
TypeId::of::<T>(),
TypeId::of::<U>(),
"{} should reuse {}",
type_name::<T>(),
type_name::<U>()
);
}
fn assert_json<T: Serialize>(value: T, expected: Value) {
assert_eq!(serde_json::to_value(value).unwrap(), expected);
}