feat(llm): catalog-estimated completion cost on Response (#494)

Standalone pre-OpenRouter step, pulled forward from the #438 triage (the
gateway-refactor plan's "additive feature PR alongside the redo"):
completion responses carry a USD cost with provenance.

## What's here

**`Response.cost_usd` + `Response.cost_source`** — new optional fields
(`skip_serializing_if` keeps the wire shape byte-identical when unset).
`CostSource` (`authoritative` | `estimated`) lives in fabro-model's
billing vocabulary next to `UsdMicros`/`TokenCounts`, since the API
layer reuses it.

**`fabro-llm/src/cost.rs`** — `estimate_cost_usd`, a thin wrapper over
the existing `Catalog::price_tokens` billing machinery (billing-policy-
and speed-aware), ported from #438's prototype with attribution. One fix
over the prototype: model aliases and provider names are canonicalized
before building the `ModelRef` — `ModelPricing::bill` rejects
non-canonical refs, so the original would silently skip cost on alias
requests (caught by a new test).

**Client-level stamping** — one generic post-decode site instead of
#438's ~8 per-adapter sites (which predate the codec refactor):
`Client::complete` stamps blocking responses and `Client::stream` stamps
`Finish` events, beneath the middleware chain so middleware observes
final responses. Codecs stay wire-translation-only — zero wire-snapshot
churn — and every registered adapter (including custom
`register_provider` ones) gets the same treatment. Stamping never
overwrites an existing cost, so future authoritative in-band costs
(OpenRouter) take precedence by construction.

**API surface** — `cost_usd`/`cost_source` on `CompletionResponse`
(OpenAPI spec + handler + regenerated TS client). The streaming endpoint
already carries cost implicitly since `Finish` events serialize the
`Response` verbatim; this makes the blocking surface match. `CostSource`
reuses the canonical fabro-model type via `with_replacement`, with the
standard round-trip test pinning type identity and JSON parity.

## Deliberately not here (stays with the OpenRouter redo per the plan's
hard rule)

- Authoritative `usage.cost` parsing in the `openai_compatible` codec
wire structs
- Cached-token usage parsing (changes observable usage values)
- Per-model `billing_policy` schema field

## Verification

- `cargo nextest run --workspace --no-fail-fast`: 6701 passed; only the
known 5 pre-existing environment-dependent fabro-workflow failures
(identical on main)
- All fabro-llm wire snapshots unmodified; new pins: cost estimation
unit tests (incl. alias canonicalization), Client stamping tests
(blocking, streaming, beneath middleware, no-catalog), fabro-api
`CostSource` round-trip
- clippy `-D warnings` + pinned-nightly fmt clean; `bun run typecheck`
clean in fabro-web

Independent of the route-vocabulary work in #493 — branches directly off
main. After both land, the OpenRouter redo shrinks to config + typed
codec params + authoritative-cost decode.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Scott Werner 2026-06-11 16:55:47 -04:00 • committed by GitHub
parent d10fcd5e91
commit 23d3644787
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
32 changed files with 674 additions and 25 deletions

View file

@ -7904,6 +7904,21 @@ components:
$ref: "#/components/schemas/CompletionUsage"
output:
description: Parsed structured output when schema was provided.
cost_usd:
type: number
format: double
description: >
USD cost of the completion when known: estimated from catalog
prices unless the provider returned authoritative billing data.
cost_source:
$ref: "#/components/schemas/CostSource"
CostSource:
type: string
description: >
Whether `cost_usd` came from provider billing data (authoritative)
or catalog price estimation (estimated).
enum: [authoritative, estimated]
# ── Playground ────────────────────────────────────────────────────────

View file

@ -1750,6 +1750,8 @@ def farewell(name):
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
},
text_response("Done! Updated greet and farewell functions."),
];
@ -1826,6 +1828,8 @@ def farewell(name):
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
},
text_response("I will correct the patch."),
];

View file

@ -187,6 +187,8 @@ pub fn text_response(text: &str) -> Response {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
}
}
@ -263,6 +265,8 @@ pub fn tool_call_response(
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
}
}
@ -390,5 +394,7 @@ pub fn multi_tool_call_response(calls: Vec<(&str, &str, serde_json::Value)>) ->
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
}
}

View file

@ -647,6 +647,7 @@ fn main() {
("CompletionMessage", "fabro_types::Message", &[]),
("CompletionMessageRole", "fabro_types::Role", &[]),
("CompletionContentPart", "fabro_types::ContentPart", &[]),
("CostSource", "fabro_model::CostSource", &[]),
];
for (name, path, impls) in replacements {
settings.with_replacement(*name, *path, impls.iter().copied());

View file

@ -20,8 +20,8 @@ pub mod types {
};
pub use fabro_environment::Environment;
pub use fabro_model::{
Model, ModelCosts, ModelFeatures, ModelLimits, ModelRef as BillingModelRef, ModelTestMode,
Provider, ReasoningEffort, ReasoningEffortFeature, Speed as BillingSpeed,
CostSource, Model, ModelCosts, ModelFeatures, ModelLimits, ModelRef as BillingModelRef,
ModelTestMode, Provider, ReasoningEffort, ReasoningEffortFeature, Speed as BillingSpeed,
};
pub use fabro_types::run_event::AgentSessionActivatedProps;
pub use fabro_types::settings::ServerNamespace;

View file

@ -0,0 +1,37 @@
use std::any::{TypeId, type_name};
use fabro_api::types::CostSource as ApiCostSource;
use fabro_model::CostSource;
use serde_json::json;
#[test]
fn cost_source_reuses_canonical_type() {
assert_same_type::<ApiCostSource, CostSource>();
}
#[test]
fn cost_source_json_matches_openapi_shape() {
assert_eq!(
serde_json::to_value(CostSource::Authoritative).unwrap(),
json!("authoritative")
);
assert_eq!(
serde_json::to_value(CostSource::Estimated).unwrap(),
json!("estimated")
);
assert_eq!(
serde_json::from_value::<ApiCostSource>(json!("estimated")).unwrap(),
CostSource::Estimated
);
}
fn assert_same_type<T: 'static, U: 'static>() {
assert_eq!(
TypeId::of::<T>(),
TypeId::of::<U>(),
"{} should be the same type as {}",
type_name::<T>(),
type_name::<U>()
);
}

View file

@ -12,7 +12,7 @@ use fabro_llm::error::{
use fabro_llm::provider::{ProviderAdapter, StreamEventStream};
use fabro_llm::providers::common::{LineReader, parse_retry_after};
use fabro_llm::types::{
FinishReason, Message, Request, Response as LlmResponse, StreamEvent, TokenCounts,
CostSource, FinishReason, Message, Request, Response as LlmResponse, StreamEvent, TokenCounts,
};
use fabro_mcp::config::McpServerSettings;
use fabro_model::ProviderId;
@ -52,6 +52,8 @@ struct ServerCompletionResponse {
message: Message,
stop_reason: String,
usage: ServerUsage,
cost_usd: Option<f64>,
cost_source: Option<CostSource>,
}
#[derive(Deserialize)]
@ -227,6 +229,10 @@ impl ProviderAdapter for AuthenticatedFabroServerAdapter {
raw: None,
warnings: vec![],
rate_limit: None,
// Carry the server's cost through; the local client's stamping
// never overwrites an already-set cost.
cost_usd: server_response.cost_usd,
cost_source: server_response.cost_source,
})
}

View file

@ -8,13 +8,14 @@ use tracing::debug;
use crate::adapter_registry::{
AdapterConfig, AdapterKindOptions, OpenAiAdapterOptions, factory_for,
};
use crate::cost;
use crate::error::{Error, ProviderErrorKind};
use crate::middleware::{Middleware, NextFn, NextStreamFn};
use crate::provider::{ProviderAdapter, StreamEventStream};
use crate::token_count::{
InputTokenCount, InputTokenCountMethod, InputTokenCountPreference, estimate_input_tokens,
};
use crate::types::{Request, Response, Speed, Warning};
use crate::types::{Request, Response, Speed, StreamEvent, Warning};
/// The core client that routes requests to provider adapters (Section 2.2, 3).
#[derive(Clone)]
@ -336,18 +337,16 @@ impl Client {
let provider = self.resolve_provider(request)?;
if self.middleware.is_empty() {
provider.validate_request(request)?;
return provider.complete(request).await;
return complete_stamped(&provider, self.catalog.as_deref(), request).await;
}
// Build middleware chain
let provider_clone = provider.clone();
// Build middleware chain. Cost is stamped at the base so middleware
// observes the final response.
let catalog = self.catalog.clone();
let base: NextFn = Arc::new(move |req: Request| {
let p = provider_clone.clone();
Box::pin(async move {
p.validate_request(&req)?;
p.complete(&req).await
})
let provider = provider.clone();
let catalog = catalog.clone();
Box::pin(async move { complete_stamped(&provider, catalog.as_deref(), &req).await })
});
let chain = self.middleware.iter().rev().fold(base, |next, mw| {
@ -374,18 +373,16 @@ impl Client {
let provider = self.resolve_provider(request)?;
if self.middleware.is_empty() {
provider.validate_request(request)?;
return provider.stream(request).await;
return stream_stamped(&provider, self.catalog.clone(), request).await;
}
// Build streaming middleware chain
let provider_clone = provider.clone();
// Build streaming middleware chain. Cost is stamped at the base so
// middleware observes the final Finish events.
let catalog = self.catalog.clone();
let base: NextStreamFn = Arc::new(move |req: Request| {
let p = provider_clone.clone();
Box::pin(async move {
p.validate_request(&req)?;
p.stream(&req).await
})
let provider = provider.clone();
let catalog = catalog.clone();
Box::pin(async move { stream_stamped(&provider, catalog, &req).await })
});
let chain = self.middleware.iter().rev().fold(base, |next, mw| {
@ -492,6 +489,59 @@ impl Client {
}
}
/// Validate, run, and cost-stamp a blocking request. Shared by
/// [`Client::complete`]'s direct path and its middleware-chain base so cost
/// stamping stays single-sited.
async fn complete_stamped(
provider: &Arc<dyn ProviderAdapter>,
catalog: Option<&Catalog>,
request: &Request,
) -> Result<Response, Error> {
provider.validate_request(request)?;
let mut response = provider.complete(request).await?;
cost::apply_estimated_cost(catalog, &request.model, request.speed, &mut response);
Ok(response)
}
/// Validate and run a streaming request, cost-stamping terminal
/// [`StreamEvent::Finish`] responses. Shared by [`Client::stream`]'s direct
/// path and its middleware-chain base so cost stamping stays single-sited.
async fn stream_stamped(
provider: &Arc<dyn ProviderAdapter>,
catalog: Option<Arc<Catalog>>,
request: &Request,
) -> Result<StreamEventStream, Error> {
provider.validate_request(request)?;
let stream = provider.stream(request).await?;
Ok(stamp_stream_costs(
catalog,
request.model.clone(),
request.speed,
stream,
))
}
/// Wrap a provider event stream so terminal [`StreamEvent::Finish`]
/// responses carry a catalog-estimated cost, mirroring what
/// [`Client::complete`] stamps on blocking responses.
fn stamp_stream_costs(
catalog: Option<Arc<Catalog>>,
model: String,
speed: Option<Speed>,
stream: StreamEventStream,
) -> StreamEventStream {
use futures::StreamExt;
Box::pin(stream.map(move |event| {
event.map(|mut event| {
if let StreamEvent::Finish { response, .. } = &mut event {
cost::apply_estimated_cost(catalog.as_deref(), &model, speed, response);
}
event
})
}))
}
fn token_count_fallback_eligible(error: &Error) -> bool {
matches!(
error,
@ -595,6 +645,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -616,6 +668,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
},
)),
];
@ -776,6 +830,130 @@ mod tests {
assert_eq!(response.provider, "test");
}
/// Hermetic catalog pricing `mock-model` under the `test` provider so
/// cost stamping has something to estimate from.
fn priced_mock_catalog() -> Arc<Catalog> {
let settings: LlmCatalogSettings = toml::from_str(
r#"
[providers.test]
display_name = "Test"
adapter = "openai_compatible"
base_url = "https://test.invalid/v1"
[models."mock-model"]
provider = "test"
display_name = "Mock"
family = "mock"
default = true
[models."mock-model".limits]
context_window = 100000
[models."mock-model".features]
tools = false
vision = false
reasoning = false
[models."mock-model".costs]
input_cost_per_mtok = 1.0
output_cost_per_mtok = 2.0
"#,
)
.unwrap();
Arc::new(Catalog::from_settings(&settings).unwrap())
}
#[tokio::test]
async fn complete_stamps_estimated_cost_from_catalog() {
let mut client = Client::new(HashMap::new(), None, vec![]);
client
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
.await
.unwrap();
client.catalog = Some(priced_mock_catalog());
let response = client.complete(&test_request()).await.unwrap();
// 10 input tokens at $1/MTok + 20 output tokens at $2/MTok.
assert_eq!(response.cost_source, Some(CostSource::Estimated));
let cost = response.cost_usd.expect("cost should be stamped");
assert!((cost - 0.000_05).abs() < 1e-12, "got {cost}");
}
#[tokio::test]
async fn complete_leaves_cost_unset_without_catalog() {
let mut client = Client::new(HashMap::new(), None, vec![]);
client
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
.await
.unwrap();
let response = client.complete(&test_request()).await.unwrap();
assert_eq!(response.cost_usd, None);
assert_eq!(response.cost_source, None);
}
#[tokio::test]
async fn complete_stamps_cost_beneath_middleware() {
struct Passthrough;
#[async_trait]
impl Middleware for Passthrough {
async fn handle_complete(
&self,
request: Request,
next: NextFn,
) -> Result<Response, Error> {
next(request).await
}
async fn handle_stream(
&self,
request: Request,
next: NextStreamFn,
) -> Result<StreamEventStream, Error> {
next(request).await
}
}
let mut client = Client::new(HashMap::new(), None, vec![Arc::new(Passthrough)]);
client
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
.await
.unwrap();
client.catalog = Some(priced_mock_catalog());
let response = client.complete(&test_request()).await.unwrap();
assert_eq!(response.cost_source, Some(CostSource::Estimated));
}
#[tokio::test]
async fn stream_stamps_estimated_cost_on_finish() {
use futures::StreamExt;
let mut client = Client::new(HashMap::new(), None, vec![]);
client
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
.await
.unwrap();
client.catalog = Some(priced_mock_catalog());
let mut stream = client.stream(&test_request()).await.unwrap();
let mut finish_response = None;
while let Some(event) = stream.next().await {
if let StreamEvent::Finish { response, .. } = event.unwrap() {
finish_response = Some(response);
}
}
let response = finish_response.expect("stream should yield a Finish event");
// MockProvider's Finish usage is zero tokens — priced, just $0.
assert_eq!(response.cost_source, Some(CostSource::Estimated));
assert_eq!(response.cost_usd, Some(0.0));
}
#[tokio::test]
async fn count_input_tokens_returns_provider_result() {
let (client, calls) =

View file

@ -183,6 +183,8 @@ pub(super) fn decode_response(
raw: Some(raw),
warnings: vec![],
rate_limit,
cost_usd: None,
cost_source: None,
})
}

View file

@ -78,6 +78,8 @@ impl SseAccumulator {
raw: None,
warnings: vec![],
rate_limit: self.rate_limit.take(),
cost_usd: None,
cost_source: None,
}
}
@ -630,6 +632,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
});
let event = StreamEvent::Finish {
finish_reason: FinishReason::ToolCalls,

View file

@ -162,6 +162,8 @@ pub(super) fn decode_response(
raw: Some(raw),
warnings: vec![],
rate_limit,
cost_usd: None,
cost_source: None,
})
}

View file

@ -201,6 +201,8 @@ impl SseAccumulator {
raw: None,
warnings: vec![],
rate_limit: self.rate_limit.clone(),
cost_usd: None,
cost_source: None,
};
StreamEvent::finish(finish_reason, self.usage.clone(), response)

View file

@ -79,5 +79,7 @@ pub(super) fn decode_response(
raw: serde_json::from_str(body).ok(),
warnings: vec![],
rate_limit,
cost_usd: None,
cost_source: None,
})
}

View file

@ -222,6 +222,8 @@ impl StreamState {
raw: None,
warnings: vec![],
rate_limit: self.rate_limit.clone(),
cost_usd: None,
cost_source: None,
};
events.push(StreamEvent::finish(

View file

@ -175,6 +175,8 @@ pub(super) fn decode_response(
raw: Some(raw),
warnings: vec![],
rate_limit,
cost_usd: None,
cost_source: None,
})
}

View file

@ -416,6 +416,8 @@ impl SseAccumulator {
raw: Some(response_data.clone()),
warnings: vec![],
rate_limit: self.rate_limit.clone(),
cost_usd: None,
cost_source: None,
};
events.push(StreamEvent::finish(

View file

@ -0,0 +1,249 @@
//! Catalog-derived cost estimation for completion responses.
//!
//! The estimate is a thin wrapper over the catalog's billing machinery
//! ([`Catalog::price_tokens`]), which is billing-policy- and speed-aware.
//! Costs are stamped onto responses by the [`Client`](crate::Client) as a
//! post-decode step, so codecs stay wire-translation-only and every
//! registered adapter (including custom ones) gets the same treatment.
use fabro_model::billing::{ModelRef, Speed, TokenCounts};
use fabro_model::{Catalog, ProviderId};
use crate::types::{CostSource, Response};
/// Estimate the USD cost of a completion from the catalog's per-token
/// pricing for the model. Returns `None` if the catalog is absent, the
/// model is not in the catalog, or the model has no pricing.
#[must_use]
pub(crate) fn estimate_cost_usd(
catalog: Option<&Catalog>,
provider: &str,
model: &str,
tokens: &TokenCounts,
speed: Option<Speed>,
) -> Option<f64> {
let catalog = catalog?;
// The billing machinery compares ModelRefs against the catalog's
// canonical identity, so resolve model aliases and provider names first.
let model = catalog.get(model)?;
let provider = catalog.provider(&ProviderId::new(provider))?;
let model_ref = ModelRef {
provider: provider.id.clone(),
model_id: model.id.clone(),
speed,
};
let micros = catalog.price_tokens(&model_ref, tokens)?;
#[expect(
clippy::cast_precision_loss,
reason = "micros fit comfortably in f64 for any realistic completion cost"
)]
Some(micros as f64 / 1_000_000.0)
}
/// Stamp a catalog-estimated cost onto `response` unless the provider
/// already supplied one (providers that return authoritative billing data
/// in-band set [`CostSource::Authoritative`] directly and take precedence).
/// `model` is the request's model id or alias (the catalog lookup resolves
/// aliases); the response's provider name selects the billing policy.
pub(crate) fn apply_estimated_cost(
catalog: Option<&Catalog>,
model: &str,
speed: Option<Speed>,
response: &mut Response,
) {
if response.cost_usd.is_some() {
return;
}
let estimate = estimate_cost_usd(catalog, &response.provider, model, &response.usage, speed);
response.cost_usd = estimate;
response.cost_source = estimate.map(|_| CostSource::Estimated);
}
#[cfg(test)]
mod tests {
use fabro_model::catalog::LlmCatalogSettings;
use super::*;
use crate::types::{FinishReason, Message};
/// Single-provider catalog with one `gpt-test` model (alias `gpt-alias`)
/// and the given `[models."gpt-test".costs]` block (empty for unpriced).
fn test_catalog(costs_block: &str) -> Catalog {
let toml = format!(
r#"
[providers.openai]
display_name = "OpenAI"
adapter = "openai"
agent_profile = "openai"
[models."gpt-test"]
provider = "openai"
display_name = "GPT Test"
family = "gpt"
default = true
aliases = ["gpt-alias"]
[models."gpt-test".limits]
context_window = 200000
max_output = 4096
[models."gpt-test".features]
tools = true
vision = false
reasoning = false
{costs_block}
"#
);
let settings: LlmCatalogSettings = toml::from_str(&toml).unwrap();
Catalog::from_settings(&settings).unwrap()
}
fn priced_catalog(input_cost_per_mtok: f64, output_cost_per_mtok: f64) -> Catalog {
test_catalog(&format!(
r#"
[models."gpt-test".costs]
input_cost_per_mtok = {input_cost_per_mtok}
output_cost_per_mtok = {output_cost_per_mtok}
"#
))
}
fn response_with_usage(tokens: TokenCounts) -> Response {
Response {
id: "resp".to_string(),
model: "gpt-test".to_string(),
provider: "openai".to_string(),
message: Message::assistant("hi"),
finish_reason: FinishReason::Stop,
usage: tokens,
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
}
}
#[test]
fn returns_none_when_catalog_is_none() {
let tokens = TokenCounts {
input_tokens: 1000,
output_tokens: 500,
..TokenCounts::default()
};
assert_eq!(
estimate_cost_usd(None, "openai", "gpt-test", &tokens, None),
None
);
}
#[test]
fn returns_estimated_when_model_priced() {
let catalog = priced_catalog(1.0, 2.0);
let tokens = TokenCounts {
input_tokens: 1_000_000, // 1M tokens at $1/Mtok = $1.00
output_tokens: 500_000, // 500k tokens at $2/Mtok = $1.00
..TokenCounts::default()
};
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-test", &tokens, None)
.expect("cost should be Some");
assert!((cost - 2.0).abs() < 1e-9, "expected ~$2.00, got {cost}");
}
#[test]
fn resolves_model_aliases() {
let catalog = priced_catalog(1.0, 2.0);
let tokens = TokenCounts {
input_tokens: 1_000_000,
output_tokens: 0,
..TokenCounts::default()
};
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-alias", &tokens, None);
assert!(cost.is_some());
}
#[test]
fn returns_none_when_model_missing_from_catalog() {
let catalog = priced_catalog(1.0, 2.0);
let tokens = TokenCounts {
input_tokens: 1000,
output_tokens: 500,
..TokenCounts::default()
};
let cost = estimate_cost_usd(Some(&catalog), "openai", "nonexistent-model", &tokens, None);
assert_eq!(cost, None);
}
#[test]
fn returns_none_when_model_has_no_pricing() {
let catalog = test_catalog("");
let tokens = TokenCounts {
input_tokens: 1000,
output_tokens: 500,
..TokenCounts::default()
};
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-test", &tokens, None);
assert_eq!(cost, None);
}
#[test]
fn micros_to_usd_conversion_is_exact_for_integer_amounts() {
// input_cost_per_mtok = 1.5 USD; 1M input tokens with no output
// yields exactly 1_500_000 micros = $1.50 (representable as f64).
let catalog = priced_catalog(1.5, 0.0);
let tokens = TokenCounts {
input_tokens: 1_000_000,
output_tokens: 0,
..TokenCounts::default()
};
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-test", &tokens, None)
.expect("cost should be Some");
assert!(
(cost - 1.5).abs() < f64::EPSILON,
"expected $1.50 exact, got {cost}"
);
}
#[test]
fn apply_estimated_cost_stamps_estimate() {
let catalog = priced_catalog(1.0, 2.0);
let mut response = response_with_usage(TokenCounts {
input_tokens: 1_000_000,
output_tokens: 0,
..TokenCounts::default()
});
apply_estimated_cost(Some(&catalog), "gpt-test", None, &mut response);
assert_eq!(response.cost_source, Some(CostSource::Estimated));
assert!(response.cost_usd.is_some());
}
#[test]
fn apply_estimated_cost_leaves_source_unset_without_estimate() {
let mut response = response_with_usage(TokenCounts::default());
apply_estimated_cost(None, "gpt-test", None, &mut response);
assert_eq!(response.cost_usd, None);
assert_eq!(response.cost_source, None);
}
#[test]
fn apply_estimated_cost_keeps_existing_cost() {
let catalog = priced_catalog(1.0, 2.0);
let mut response = response_with_usage(TokenCounts {
input_tokens: 1_000_000,
output_tokens: 0,
..TokenCounts::default()
});
response.cost_usd = Some(0.42);
response.cost_source = Some(CostSource::Authoritative);
apply_estimated_cost(Some(&catalog), "gpt-test", None, &mut response);
assert_eq!(response.cost_usd, Some(0.42));
assert_eq!(response.cost_source, Some(CostSource::Authoritative));
}
}

View file

@ -1138,6 +1138,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -1166,6 +1168,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
},
)),
];
@ -1274,6 +1278,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
} else {
// Second call: return text
@ -1291,6 +1297,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
}
@ -1360,6 +1368,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
acc.process(&StreamEvent::finish(
@ -1565,6 +1575,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -1596,6 +1608,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
},
)));
@ -1775,6 +1789,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -1941,6 +1957,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -1970,6 +1988,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let events = vec![
Ok(StreamEvent::ToolCallEnd { tool_call }),
@ -1997,6 +2017,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let events = vec![
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
@ -2113,6 +2135,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let tool_calls = vec![ToolCall::new(
@ -2276,6 +2300,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -2307,6 +2333,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let events = vec![
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
@ -2389,6 +2417,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -2405,6 +2435,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let events = vec![
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
@ -2487,6 +2519,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -2512,6 +2546,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let events = vec![
Ok(StreamEvent::ToolCallEnd { tool_call }),
@ -2536,6 +2572,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let events = vec![
Ok(StreamEvent::text_delta(text, Some("t1".into()))),

View file

@ -2,6 +2,7 @@ pub mod adapter_registry;
mod attachments;
pub mod client;
mod codec;
pub(crate) mod cost;
pub mod error;
pub mod generate;
pub mod middleware;

View file

@ -211,6 +211,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
}
}

View file

@ -5,7 +5,9 @@ use tracing::{debug, error};
use crate::error::{Error, error_from_status_code};
use crate::provider::{ProviderAdapter, StreamEventStream};
use crate::transport::{LineReader, parse_sse_block};
use crate::types::{FinishReason, Message, Request, Response, StreamEvent, TokenCounts};
use crate::types::{
CostSource, FinishReason, Message, Request, Response, StreamEvent, TokenCounts,
};
/// Provider adapter that routes LLM requests through an fabro server's
/// `/completions` endpoint, delegating to whatever real provider the server
@ -41,6 +43,8 @@ struct ServerCompletionResponse {
message: Message,
stop_reason: String,
usage: ServerUsage,
cost_usd: Option<f64>,
cost_source: Option<CostSource>,
}
#[derive(serde::Deserialize)]
@ -151,6 +155,10 @@ impl ProviderAdapter for Adapter {
raw: None,
warnings: vec![],
rate_limit: None,
// Carry the server's cost through; the local client's stamping
// never overwrites an already-set cost.
cost_usd: server_resp.cost_usd,
cost_source: server_resp.cost_source,
})
}
@ -307,7 +315,9 @@ data: {\"type\":\"text_delta\",\"delta\":\" world\",\"text_id\":null}\n\
"usage": {
"input_tokens": 10,
"output_tokens": 5
}
},
"cost_usd": 0.000_25,
"cost_source": "estimated"
});
server.mock(|when, then| {
@ -333,6 +343,8 @@ data: {\"type\":\"text_delta\",\"delta\":\" world\",\"text_id\":null}\n\
assert_eq!(response.usage.input_tokens, 10);
assert_eq!(response.usage.output_tokens, 5);
assert_eq!(response.usage.total_tokens(), 15);
assert_eq!(response.cost_usd, Some(0.000_25));
assert_eq!(response.cost_source, Some(CostSource::Estimated));
}
#[tokio::test]

View file

@ -221,6 +221,10 @@ impl ToolChoice {
// --- 3.7 Response ---
// Billing vocabulary shared with the catalog/billing layer and the API
// surface; re-exported here so `fabro_llm::types::*` imports keep working.
pub use fabro_model::CostSource;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response {
pub id: String,
@ -232,6 +236,13 @@ pub struct Response {
pub raw: Option<serde_json::Value>,
pub warnings: Vec<Warning>,
pub rate_limit: Option<RateLimitInfo>,
/// USD cost of this completion, when known or estimable.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub cost_usd: Option<f64>,
/// Whether `cost_usd` came from provider billing data or a catalog
/// estimate.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub cost_source: Option<CostSource>,
}
impl Response {
@ -703,6 +714,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
assert_eq!(response.text(), "Hello world");
}
@ -731,6 +744,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let calls = response.tool_calls();
assert_eq!(calls.len(), 1);
@ -762,6 +777,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
assert_eq!(response.reasoning(), Some("Let me think...".to_string()));
assert_eq!(response.text(), "The answer is 42.");
@ -779,6 +796,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
assert_eq!(response.reasoning(), None);
}
@ -949,6 +968,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
};
let tool_calls = vec![ToolCall::new(
"call_1",

View file

@ -122,6 +122,19 @@ impl Speed {
}
}
/// Source of a USD cost value attached to a completion response.
#[derive(
Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Display, EnumString, IntoStaticStr,
)]
#[serde(rename_all = "snake_case")]
#[strum(serialize_all = "snake_case")]
pub enum CostSource {
/// The provider returned billing data in-band with the response.
Authoritative,
/// Computed from catalog prices and token usage.
Estimated,
}
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct ModelRef {
pub provider: ProviderId,

View file

@ -12,7 +12,7 @@ pub mod types;
pub use adapter::{AdapterKind, AgentProfileKind};
pub use billing::{
AnthropicBillingFacts, AnthropicModelPricing, BilledModelUsage, BilledTokenCounts,
AnthropicBillingFacts, AnthropicModelPricing, BilledModelUsage, BilledTokenCounts, CostSource,
GeminiBillingFacts, GeminiModelPricing, GeminiStoragePricing, GeminiStorageSegment,
ModelBillingFacts, ModelBillingInput, ModelPricing, ModelPricingPolicy, ModelRef, ModelUsage,
OpenAiBillingFacts, OpenAiModelPricing, PricePerMTok, Speed, TokenCounts, UsdMicros,

View file

@ -355,6 +355,8 @@ mod tests {
raw: None,
warnings: Vec::new(),
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}

View file

@ -177,6 +177,8 @@ async fn create_completion(
output_tokens: response.usage.output_tokens,
},
output,
cost_usd: response.cost_usd,
cost_source: response.cost_source,
})
.into_response()
}
@ -197,6 +199,8 @@ async fn create_completion(
output_tokens: response.usage.output_tokens,
},
output: None,
cost_usd: response.cost_usd,
cost_source: response.cost_source,
})
.into_response()
}

View file

@ -1726,6 +1726,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}

View file

@ -732,6 +732,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
})
}
@ -760,6 +762,8 @@ mod tests {
raw: None,
warnings: vec![],
rate_limit: None,
cost_usd: None,
cost_source: None,
},
)),
];

View file

@ -94,6 +94,7 @@ models/completion-tool-choice.ts
models/completion-tool-definition.ts
models/completion-usage.ts
models/conclusion.ts
models/cost-source.ts
models/create-automation-request.ts
models/create-completion-request.ts
models/create-environment-request.ts

View file

@ -19,6 +19,9 @@ import type { CompletionMessage } from './completion-message';
// May contain unused imports in some cases
// @ts-ignore
import type { CompletionUsage } from './completion-usage';
// May contain unused imports in some cases
// @ts-ignore
import type { CostSource } from './cost-source';
export interface CompletionResponse {
'id': string;
@ -30,4 +33,9 @@ export interface CompletionResponse {
'stop_reason': string;
'usage': CompletionUsage;
'output'?: any;
/**
* USD cost of the completion when known: estimated from catalog prices unless the provider returned authoritative billing data.
*/
'cost_usd'?: number;
'cost_source'?: CostSource;
}

View file

@ -0,0 +1,26 @@
/* tslint:disable */
/* eslint-disable */
/**
* Fabro Run API
* HTTP API for managing Fabro workflow run executions.
*
* The version of the OpenAPI document: 0.1.0
*
*
* NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech).
* https://openapi-generator.tech
* Do not edit the class manually.
*/
/**
* Whether `cost_usd` came from provider billing data (authoritative) or catalog price estimation (estimated).
*/
export const CostSource = {
AUTHORITATIVE: 'authoritative',
ESTIMATED: 'estimated'
} as const;
export type CostSource = typeof CostSource[keyof typeof CostSource];

View file

@ -66,6 +66,7 @@ export * from './completion-tool-choice';
export * from './completion-tool-definition';
export * from './completion-usage';
export * from './conclusion';
export * from './cost-source';
export * from './create-automation-request';
export * from './create-completion-request';
export * from './create-environment-request';