mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-10-07 03:00:29 +00:00
feat(llm): catalog-estimated completion cost on Response (#494)
Standalone pre-OpenRouter step, pulled forward from the #438 triage (the gateway-refactor plan's "additive feature PR alongside the redo"): completion responses carry a USD cost with provenance. ## What's here **`Response.cost_usd` + `Response.cost_source`** — new optional fields (`skip_serializing_if` keeps the wire shape byte-identical when unset). `CostSource` (`authoritative` | `estimated`) lives in fabro-model's billing vocabulary next to `UsdMicros`/`TokenCounts`, since the API layer reuses it. **`fabro-llm/src/cost.rs`** — `estimate_cost_usd`, a thin wrapper over the existing `Catalog::price_tokens` billing machinery (billing-policy- and speed-aware), ported from #438's prototype with attribution. One fix over the prototype: model aliases and provider names are canonicalized before building the `ModelRef` — `ModelPricing::bill` rejects non-canonical refs, so the original would silently skip cost on alias requests (caught by a new test). **Client-level stamping** — one generic post-decode site instead of #438's ~8 per-adapter sites (which predate the codec refactor): `Client::complete` stamps blocking responses and `Client::stream` stamps `Finish` events, beneath the middleware chain so middleware observes final responses. Codecs stay wire-translation-only — zero wire-snapshot churn — and every registered adapter (including custom `register_provider` ones) gets the same treatment. Stamping never overwrites an existing cost, so future authoritative in-band costs (OpenRouter) take precedence by construction. **API surface** — `cost_usd`/`cost_source` on `CompletionResponse` (OpenAPI spec + handler + regenerated TS client). The streaming endpoint already carries cost implicitly since `Finish` events serialize the `Response` verbatim; this makes the blocking surface match. `CostSource` reuses the canonical fabro-model type via `with_replacement`, with the standard round-trip test pinning type identity and JSON parity. ## Deliberately not here (stays with the OpenRouter redo per the plan's hard rule) - Authoritative `usage.cost` parsing in the `openai_compatible` codec wire structs - Cached-token usage parsing (changes observable usage values) - Per-model `billing_policy` schema field ## Verification - `cargo nextest run --workspace --no-fail-fast`: 6701 passed; only the known 5 pre-existing environment-dependent fabro-workflow failures (identical on main) - All fabro-llm wire snapshots unmodified; new pins: cost estimation unit tests (incl. alias canonicalization), Client stamping tests (blocking, streaming, beneath middleware, no-catalog), fabro-api `CostSource` round-trip - clippy `-D warnings` + pinned-nightly fmt clean; `bun run typecheck` clean in fabro-web Independent of the route-vocabulary work in #493 — branches directly off main. After both land, the OpenRouter redo shrinks to config + typed codec params + authoritative-cost decode. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
d10fcd5e91
commit
23d3644787
32 changed files with 674 additions and 25 deletions
|
|
@ -7904,6 +7904,21 @@ components:
|
|||
$ref: "#/components/schemas/CompletionUsage"
|
||||
output:
|
||||
description: Parsed structured output when schema was provided.
|
||||
cost_usd:
|
||||
type: number
|
||||
format: double
|
||||
description: >
|
||||
USD cost of the completion when known: estimated from catalog
|
||||
prices unless the provider returned authoritative billing data.
|
||||
cost_source:
|
||||
$ref: "#/components/schemas/CostSource"
|
||||
|
||||
CostSource:
|
||||
type: string
|
||||
description: >
|
||||
Whether `cost_usd` came from provider billing data (authoritative)
|
||||
or catalog price estimation (estimated).
|
||||
enum: [authoritative, estimated]
|
||||
|
||||
# ── Playground ────────────────────────────────────────────────────────
|
||||
|
||||
|
|
|
|||
|
|
@ -1750,6 +1750,8 @@ def farewell(name):
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
},
|
||||
text_response("Done! Updated greet and farewell functions."),
|
||||
];
|
||||
|
|
@ -1826,6 +1828,8 @@ def farewell(name):
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
},
|
||||
text_response("I will correct the patch."),
|
||||
];
|
||||
|
|
|
|||
|
|
@ -187,6 +187,8 @@ pub fn text_response(text: &str) -> Response {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -263,6 +265,8 @@ pub fn tool_call_response(
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -390,5 +394,7 @@ pub fn multi_tool_call_response(calls: Vec<(&str, &str, serde_json::Value)>) ->
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -647,6 +647,7 @@ fn main() {
|
|||
("CompletionMessage", "fabro_types::Message", &[]),
|
||||
("CompletionMessageRole", "fabro_types::Role", &[]),
|
||||
("CompletionContentPart", "fabro_types::ContentPart", &[]),
|
||||
("CostSource", "fabro_model::CostSource", &[]),
|
||||
];
|
||||
for (name, path, impls) in replacements {
|
||||
settings.with_replacement(*name, *path, impls.iter().copied());
|
||||
|
|
|
|||
|
|
@ -20,8 +20,8 @@ pub mod types {
|
|||
};
|
||||
pub use fabro_environment::Environment;
|
||||
pub use fabro_model::{
|
||||
Model, ModelCosts, ModelFeatures, ModelLimits, ModelRef as BillingModelRef, ModelTestMode,
|
||||
Provider, ReasoningEffort, ReasoningEffortFeature, Speed as BillingSpeed,
|
||||
CostSource, Model, ModelCosts, ModelFeatures, ModelLimits, ModelRef as BillingModelRef,
|
||||
ModelTestMode, Provider, ReasoningEffort, ReasoningEffortFeature, Speed as BillingSpeed,
|
||||
};
|
||||
pub use fabro_types::run_event::AgentSessionActivatedProps;
|
||||
pub use fabro_types::settings::ServerNamespace;
|
||||
|
|
|
|||
37
lib/crates/fabro-api/tests/cost_source_round_trip.rs
Normal file
37
lib/crates/fabro-api/tests/cost_source_round_trip.rs
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
use std::any::{TypeId, type_name};
|
||||
|
||||
use fabro_api::types::CostSource as ApiCostSource;
|
||||
use fabro_model::CostSource;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn cost_source_reuses_canonical_type() {
|
||||
assert_same_type::<ApiCostSource, CostSource>();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cost_source_json_matches_openapi_shape() {
|
||||
assert_eq!(
|
||||
serde_json::to_value(CostSource::Authoritative).unwrap(),
|
||||
json!("authoritative")
|
||||
);
|
||||
assert_eq!(
|
||||
serde_json::to_value(CostSource::Estimated).unwrap(),
|
||||
json!("estimated")
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
serde_json::from_value::<ApiCostSource>(json!("estimated")).unwrap(),
|
||||
CostSource::Estimated
|
||||
);
|
||||
}
|
||||
|
||||
fn assert_same_type<T: 'static, U: 'static>() {
|
||||
assert_eq!(
|
||||
TypeId::of::<T>(),
|
||||
TypeId::of::<U>(),
|
||||
"{} should be the same type as {}",
|
||||
type_name::<T>(),
|
||||
type_name::<U>()
|
||||
);
|
||||
}
|
||||
|
|
@ -12,7 +12,7 @@ use fabro_llm::error::{
|
|||
use fabro_llm::provider::{ProviderAdapter, StreamEventStream};
|
||||
use fabro_llm::providers::common::{LineReader, parse_retry_after};
|
||||
use fabro_llm::types::{
|
||||
FinishReason, Message, Request, Response as LlmResponse, StreamEvent, TokenCounts,
|
||||
CostSource, FinishReason, Message, Request, Response as LlmResponse, StreamEvent, TokenCounts,
|
||||
};
|
||||
use fabro_mcp::config::McpServerSettings;
|
||||
use fabro_model::ProviderId;
|
||||
|
|
@ -52,6 +52,8 @@ struct ServerCompletionResponse {
|
|||
message: Message,
|
||||
stop_reason: String,
|
||||
usage: ServerUsage,
|
||||
cost_usd: Option<f64>,
|
||||
cost_source: Option<CostSource>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
|
|
@ -227,6 +229,10 @@ impl ProviderAdapter for AuthenticatedFabroServerAdapter {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
// Carry the server's cost through; the local client's stamping
|
||||
// never overwrites an already-set cost.
|
||||
cost_usd: server_response.cost_usd,
|
||||
cost_source: server_response.cost_source,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -8,13 +8,14 @@ use tracing::debug;
|
|||
use crate::adapter_registry::{
|
||||
AdapterConfig, AdapterKindOptions, OpenAiAdapterOptions, factory_for,
|
||||
};
|
||||
use crate::cost;
|
||||
use crate::error::{Error, ProviderErrorKind};
|
||||
use crate::middleware::{Middleware, NextFn, NextStreamFn};
|
||||
use crate::provider::{ProviderAdapter, StreamEventStream};
|
||||
use crate::token_count::{
|
||||
InputTokenCount, InputTokenCountMethod, InputTokenCountPreference, estimate_input_tokens,
|
||||
};
|
||||
use crate::types::{Request, Response, Speed, Warning};
|
||||
use crate::types::{Request, Response, Speed, StreamEvent, Warning};
|
||||
|
||||
/// The core client that routes requests to provider adapters (Section 2.2, 3).
|
||||
#[derive(Clone)]
|
||||
|
|
@ -336,18 +337,16 @@ impl Client {
|
|||
let provider = self.resolve_provider(request)?;
|
||||
|
||||
if self.middleware.is_empty() {
|
||||
provider.validate_request(request)?;
|
||||
return provider.complete(request).await;
|
||||
return complete_stamped(&provider, self.catalog.as_deref(), request).await;
|
||||
}
|
||||
|
||||
// Build middleware chain
|
||||
let provider_clone = provider.clone();
|
||||
// Build middleware chain. Cost is stamped at the base so middleware
|
||||
// observes the final response.
|
||||
let catalog = self.catalog.clone();
|
||||
let base: NextFn = Arc::new(move |req: Request| {
|
||||
let p = provider_clone.clone();
|
||||
Box::pin(async move {
|
||||
p.validate_request(&req)?;
|
||||
p.complete(&req).await
|
||||
})
|
||||
let provider = provider.clone();
|
||||
let catalog = catalog.clone();
|
||||
Box::pin(async move { complete_stamped(&provider, catalog.as_deref(), &req).await })
|
||||
});
|
||||
|
||||
let chain = self.middleware.iter().rev().fold(base, |next, mw| {
|
||||
|
|
@ -374,18 +373,16 @@ impl Client {
|
|||
let provider = self.resolve_provider(request)?;
|
||||
|
||||
if self.middleware.is_empty() {
|
||||
provider.validate_request(request)?;
|
||||
return provider.stream(request).await;
|
||||
return stream_stamped(&provider, self.catalog.clone(), request).await;
|
||||
}
|
||||
|
||||
// Build streaming middleware chain
|
||||
let provider_clone = provider.clone();
|
||||
// Build streaming middleware chain. Cost is stamped at the base so
|
||||
// middleware observes the final Finish events.
|
||||
let catalog = self.catalog.clone();
|
||||
let base: NextStreamFn = Arc::new(move |req: Request| {
|
||||
let p = provider_clone.clone();
|
||||
Box::pin(async move {
|
||||
p.validate_request(&req)?;
|
||||
p.stream(&req).await
|
||||
})
|
||||
let provider = provider.clone();
|
||||
let catalog = catalog.clone();
|
||||
Box::pin(async move { stream_stamped(&provider, catalog, &req).await })
|
||||
});
|
||||
|
||||
let chain = self.middleware.iter().rev().fold(base, |next, mw| {
|
||||
|
|
@ -492,6 +489,59 @@ impl Client {
|
|||
}
|
||||
}
|
||||
|
||||
/// Validate, run, and cost-stamp a blocking request. Shared by
|
||||
/// [`Client::complete`]'s direct path and its middleware-chain base so cost
|
||||
/// stamping stays single-sited.
|
||||
async fn complete_stamped(
|
||||
provider: &Arc<dyn ProviderAdapter>,
|
||||
catalog: Option<&Catalog>,
|
||||
request: &Request,
|
||||
) -> Result<Response, Error> {
|
||||
provider.validate_request(request)?;
|
||||
let mut response = provider.complete(request).await?;
|
||||
cost::apply_estimated_cost(catalog, &request.model, request.speed, &mut response);
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
/// Validate and run a streaming request, cost-stamping terminal
|
||||
/// [`StreamEvent::Finish`] responses. Shared by [`Client::stream`]'s direct
|
||||
/// path and its middleware-chain base so cost stamping stays single-sited.
|
||||
async fn stream_stamped(
|
||||
provider: &Arc<dyn ProviderAdapter>,
|
||||
catalog: Option<Arc<Catalog>>,
|
||||
request: &Request,
|
||||
) -> Result<StreamEventStream, Error> {
|
||||
provider.validate_request(request)?;
|
||||
let stream = provider.stream(request).await?;
|
||||
Ok(stamp_stream_costs(
|
||||
catalog,
|
||||
request.model.clone(),
|
||||
request.speed,
|
||||
stream,
|
||||
))
|
||||
}
|
||||
|
||||
/// Wrap a provider event stream so terminal [`StreamEvent::Finish`]
|
||||
/// responses carry a catalog-estimated cost, mirroring what
|
||||
/// [`Client::complete`] stamps on blocking responses.
|
||||
fn stamp_stream_costs(
|
||||
catalog: Option<Arc<Catalog>>,
|
||||
model: String,
|
||||
speed: Option<Speed>,
|
||||
stream: StreamEventStream,
|
||||
) -> StreamEventStream {
|
||||
use futures::StreamExt;
|
||||
|
||||
Box::pin(stream.map(move |event| {
|
||||
event.map(|mut event| {
|
||||
if let StreamEvent::Finish { response, .. } = &mut event {
|
||||
cost::apply_estimated_cost(catalog.as_deref(), &model, speed, response);
|
||||
}
|
||||
event
|
||||
})
|
||||
}))
|
||||
}
|
||||
|
||||
fn token_count_fallback_eligible(error: &Error) -> bool {
|
||||
matches!(
|
||||
error,
|
||||
|
|
@ -595,6 +645,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -616,6 +668,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
},
|
||||
)),
|
||||
];
|
||||
|
|
@ -776,6 +830,130 @@ mod tests {
|
|||
assert_eq!(response.provider, "test");
|
||||
}
|
||||
|
||||
/// Hermetic catalog pricing `mock-model` under the `test` provider so
|
||||
/// cost stamping has something to estimate from.
|
||||
fn priced_mock_catalog() -> Arc<Catalog> {
|
||||
let settings: LlmCatalogSettings = toml::from_str(
|
||||
r#"
|
||||
[providers.test]
|
||||
display_name = "Test"
|
||||
adapter = "openai_compatible"
|
||||
base_url = "https://test.invalid/v1"
|
||||
|
||||
[models."mock-model"]
|
||||
provider = "test"
|
||||
display_name = "Mock"
|
||||
family = "mock"
|
||||
default = true
|
||||
|
||||
[models."mock-model".limits]
|
||||
context_window = 100000
|
||||
|
||||
[models."mock-model".features]
|
||||
tools = false
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
[models."mock-model".costs]
|
||||
input_cost_per_mtok = 1.0
|
||||
output_cost_per_mtok = 2.0
|
||||
"#,
|
||||
)
|
||||
.unwrap();
|
||||
Arc::new(Catalog::from_settings(&settings).unwrap())
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn complete_stamps_estimated_cost_from_catalog() {
|
||||
let mut client = Client::new(HashMap::new(), None, vec![]);
|
||||
client
|
||||
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
|
||||
.await
|
||||
.unwrap();
|
||||
client.catalog = Some(priced_mock_catalog());
|
||||
|
||||
let response = client.complete(&test_request()).await.unwrap();
|
||||
|
||||
// 10 input tokens at $1/MTok + 20 output tokens at $2/MTok.
|
||||
assert_eq!(response.cost_source, Some(CostSource::Estimated));
|
||||
let cost = response.cost_usd.expect("cost should be stamped");
|
||||
assert!((cost - 0.000_05).abs() < 1e-12, "got {cost}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn complete_leaves_cost_unset_without_catalog() {
|
||||
let mut client = Client::new(HashMap::new(), None, vec![]);
|
||||
client
|
||||
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let response = client.complete(&test_request()).await.unwrap();
|
||||
|
||||
assert_eq!(response.cost_usd, None);
|
||||
assert_eq!(response.cost_source, None);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn complete_stamps_cost_beneath_middleware() {
|
||||
struct Passthrough;
|
||||
|
||||
#[async_trait]
|
||||
impl Middleware for Passthrough {
|
||||
async fn handle_complete(
|
||||
&self,
|
||||
request: Request,
|
||||
next: NextFn,
|
||||
) -> Result<Response, Error> {
|
||||
next(request).await
|
||||
}
|
||||
|
||||
async fn handle_stream(
|
||||
&self,
|
||||
request: Request,
|
||||
next: NextStreamFn,
|
||||
) -> Result<StreamEventStream, Error> {
|
||||
next(request).await
|
||||
}
|
||||
}
|
||||
|
||||
let mut client = Client::new(HashMap::new(), None, vec![Arc::new(Passthrough)]);
|
||||
client
|
||||
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
|
||||
.await
|
||||
.unwrap();
|
||||
client.catalog = Some(priced_mock_catalog());
|
||||
|
||||
let response = client.complete(&test_request()).await.unwrap();
|
||||
|
||||
assert_eq!(response.cost_source, Some(CostSource::Estimated));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_stamps_estimated_cost_on_finish() {
|
||||
use futures::StreamExt;
|
||||
|
||||
let mut client = Client::new(HashMap::new(), None, vec![]);
|
||||
client
|
||||
.register_provider(Arc::new(MockProvider::new("test", "Hello!")))
|
||||
.await
|
||||
.unwrap();
|
||||
client.catalog = Some(priced_mock_catalog());
|
||||
|
||||
let mut stream = client.stream(&test_request()).await.unwrap();
|
||||
let mut finish_response = None;
|
||||
while let Some(event) = stream.next().await {
|
||||
if let StreamEvent::Finish { response, .. } = event.unwrap() {
|
||||
finish_response = Some(response);
|
||||
}
|
||||
}
|
||||
|
||||
let response = finish_response.expect("stream should yield a Finish event");
|
||||
// MockProvider's Finish usage is zero tokens — priced, just $0.
|
||||
assert_eq!(response.cost_source, Some(CostSource::Estimated));
|
||||
assert_eq!(response.cost_usd, Some(0.0));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn count_input_tokens_returns_provider_result() {
|
||||
let (client, calls) =
|
||||
|
|
|
|||
|
|
@ -183,6 +183,8 @@ pub(super) fn decode_response(
|
|||
raw: Some(raw),
|
||||
warnings: vec![],
|
||||
rate_limit,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -78,6 +78,8 @@ impl SseAccumulator {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: self.rate_limit.take(),
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -630,6 +632,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
});
|
||||
let event = StreamEvent::Finish {
|
||||
finish_reason: FinishReason::ToolCalls,
|
||||
|
|
|
|||
|
|
@ -162,6 +162,8 @@ pub(super) fn decode_response(
|
|||
raw: Some(raw),
|
||||
warnings: vec![],
|
||||
rate_limit,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -201,6 +201,8 @@ impl SseAccumulator {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: self.rate_limit.clone(),
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
|
||||
StreamEvent::finish(finish_reason, self.usage.clone(), response)
|
||||
|
|
|
|||
|
|
@ -79,5 +79,7 @@ pub(super) fn decode_response(
|
|||
raw: serde_json::from_str(body).ok(),
|
||||
warnings: vec![],
|
||||
rate_limit,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -222,6 +222,8 @@ impl StreamState {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: self.rate_limit.clone(),
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
|
||||
events.push(StreamEvent::finish(
|
||||
|
|
|
|||
|
|
@ -175,6 +175,8 @@ pub(super) fn decode_response(
|
|||
raw: Some(raw),
|
||||
warnings: vec![],
|
||||
rate_limit,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -416,6 +416,8 @@ impl SseAccumulator {
|
|||
raw: Some(response_data.clone()),
|
||||
warnings: vec![],
|
||||
rate_limit: self.rate_limit.clone(),
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
|
||||
events.push(StreamEvent::finish(
|
||||
|
|
|
|||
249
lib/crates/fabro-llm/src/cost.rs
Normal file
249
lib/crates/fabro-llm/src/cost.rs
Normal file
|
|
@ -0,0 +1,249 @@
|
|||
//! Catalog-derived cost estimation for completion responses.
|
||||
//!
|
||||
//! The estimate is a thin wrapper over the catalog's billing machinery
|
||||
//! ([`Catalog::price_tokens`]), which is billing-policy- and speed-aware.
|
||||
//! Costs are stamped onto responses by the [`Client`](crate::Client) as a
|
||||
//! post-decode step, so codecs stay wire-translation-only and every
|
||||
//! registered adapter (including custom ones) gets the same treatment.
|
||||
|
||||
use fabro_model::billing::{ModelRef, Speed, TokenCounts};
|
||||
use fabro_model::{Catalog, ProviderId};
|
||||
|
||||
use crate::types::{CostSource, Response};
|
||||
|
||||
/// Estimate the USD cost of a completion from the catalog's per-token
|
||||
/// pricing for the model. Returns `None` if the catalog is absent, the
|
||||
/// model is not in the catalog, or the model has no pricing.
|
||||
#[must_use]
|
||||
pub(crate) fn estimate_cost_usd(
|
||||
catalog: Option<&Catalog>,
|
||||
provider: &str,
|
||||
model: &str,
|
||||
tokens: &TokenCounts,
|
||||
speed: Option<Speed>,
|
||||
) -> Option<f64> {
|
||||
let catalog = catalog?;
|
||||
// The billing machinery compares ModelRefs against the catalog's
|
||||
// canonical identity, so resolve model aliases and provider names first.
|
||||
let model = catalog.get(model)?;
|
||||
let provider = catalog.provider(&ProviderId::new(provider))?;
|
||||
let model_ref = ModelRef {
|
||||
provider: provider.id.clone(),
|
||||
model_id: model.id.clone(),
|
||||
speed,
|
||||
};
|
||||
let micros = catalog.price_tokens(&model_ref, tokens)?;
|
||||
#[expect(
|
||||
clippy::cast_precision_loss,
|
||||
reason = "micros fit comfortably in f64 for any realistic completion cost"
|
||||
)]
|
||||
Some(micros as f64 / 1_000_000.0)
|
||||
}
|
||||
|
||||
/// Stamp a catalog-estimated cost onto `response` unless the provider
|
||||
/// already supplied one (providers that return authoritative billing data
|
||||
/// in-band set [`CostSource::Authoritative`] directly and take precedence).
|
||||
/// `model` is the request's model id or alias (the catalog lookup resolves
|
||||
/// aliases); the response's provider name selects the billing policy.
|
||||
pub(crate) fn apply_estimated_cost(
|
||||
catalog: Option<&Catalog>,
|
||||
model: &str,
|
||||
speed: Option<Speed>,
|
||||
response: &mut Response,
|
||||
) {
|
||||
if response.cost_usd.is_some() {
|
||||
return;
|
||||
}
|
||||
let estimate = estimate_cost_usd(catalog, &response.provider, model, &response.usage, speed);
|
||||
response.cost_usd = estimate;
|
||||
response.cost_source = estimate.map(|_| CostSource::Estimated);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use fabro_model::catalog::LlmCatalogSettings;
|
||||
|
||||
use super::*;
|
||||
use crate::types::{FinishReason, Message};
|
||||
|
||||
/// Single-provider catalog with one `gpt-test` model (alias `gpt-alias`)
|
||||
/// and the given `[models."gpt-test".costs]` block (empty for unpriced).
|
||||
fn test_catalog(costs_block: &str) -> Catalog {
|
||||
let toml = format!(
|
||||
r#"
|
||||
[providers.openai]
|
||||
display_name = "OpenAI"
|
||||
adapter = "openai"
|
||||
agent_profile = "openai"
|
||||
|
||||
[models."gpt-test"]
|
||||
provider = "openai"
|
||||
display_name = "GPT Test"
|
||||
family = "gpt"
|
||||
default = true
|
||||
aliases = ["gpt-alias"]
|
||||
|
||||
[models."gpt-test".limits]
|
||||
context_window = 200000
|
||||
max_output = 4096
|
||||
|
||||
[models."gpt-test".features]
|
||||
tools = true
|
||||
vision = false
|
||||
reasoning = false
|
||||
|
||||
{costs_block}
|
||||
"#
|
||||
);
|
||||
let settings: LlmCatalogSettings = toml::from_str(&toml).unwrap();
|
||||
Catalog::from_settings(&settings).unwrap()
|
||||
}
|
||||
|
||||
fn priced_catalog(input_cost_per_mtok: f64, output_cost_per_mtok: f64) -> Catalog {
|
||||
test_catalog(&format!(
|
||||
r#"
|
||||
[models."gpt-test".costs]
|
||||
input_cost_per_mtok = {input_cost_per_mtok}
|
||||
output_cost_per_mtok = {output_cost_per_mtok}
|
||||
"#
|
||||
))
|
||||
}
|
||||
|
||||
fn response_with_usage(tokens: TokenCounts) -> Response {
|
||||
Response {
|
||||
id: "resp".to_string(),
|
||||
model: "gpt-test".to_string(),
|
||||
provider: "openai".to_string(),
|
||||
message: Message::assistant("hi"),
|
||||
finish_reason: FinishReason::Stop,
|
||||
usage: tokens,
|
||||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn returns_none_when_catalog_is_none() {
|
||||
let tokens = TokenCounts {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 500,
|
||||
..TokenCounts::default()
|
||||
};
|
||||
assert_eq!(
|
||||
estimate_cost_usd(None, "openai", "gpt-test", &tokens, None),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn returns_estimated_when_model_priced() {
|
||||
let catalog = priced_catalog(1.0, 2.0);
|
||||
let tokens = TokenCounts {
|
||||
input_tokens: 1_000_000, // 1M tokens at $1/Mtok = $1.00
|
||||
output_tokens: 500_000, // 500k tokens at $2/Mtok = $1.00
|
||||
..TokenCounts::default()
|
||||
};
|
||||
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-test", &tokens, None)
|
||||
.expect("cost should be Some");
|
||||
assert!((cost - 2.0).abs() < 1e-9, "expected ~$2.00, got {cost}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolves_model_aliases() {
|
||||
let catalog = priced_catalog(1.0, 2.0);
|
||||
let tokens = TokenCounts {
|
||||
input_tokens: 1_000_000,
|
||||
output_tokens: 0,
|
||||
..TokenCounts::default()
|
||||
};
|
||||
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-alias", &tokens, None);
|
||||
assert!(cost.is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn returns_none_when_model_missing_from_catalog() {
|
||||
let catalog = priced_catalog(1.0, 2.0);
|
||||
let tokens = TokenCounts {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 500,
|
||||
..TokenCounts::default()
|
||||
};
|
||||
let cost = estimate_cost_usd(Some(&catalog), "openai", "nonexistent-model", &tokens, None);
|
||||
assert_eq!(cost, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn returns_none_when_model_has_no_pricing() {
|
||||
let catalog = test_catalog("");
|
||||
let tokens = TokenCounts {
|
||||
input_tokens: 1000,
|
||||
output_tokens: 500,
|
||||
..TokenCounts::default()
|
||||
};
|
||||
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-test", &tokens, None);
|
||||
assert_eq!(cost, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn micros_to_usd_conversion_is_exact_for_integer_amounts() {
|
||||
// input_cost_per_mtok = 1.5 USD; 1M input tokens with no output
|
||||
// yields exactly 1_500_000 micros = $1.50 (representable as f64).
|
||||
let catalog = priced_catalog(1.5, 0.0);
|
||||
let tokens = TokenCounts {
|
||||
input_tokens: 1_000_000,
|
||||
output_tokens: 0,
|
||||
..TokenCounts::default()
|
||||
};
|
||||
let cost = estimate_cost_usd(Some(&catalog), "openai", "gpt-test", &tokens, None)
|
||||
.expect("cost should be Some");
|
||||
assert!(
|
||||
(cost - 1.5).abs() < f64::EPSILON,
|
||||
"expected $1.50 exact, got {cost}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn apply_estimated_cost_stamps_estimate() {
|
||||
let catalog = priced_catalog(1.0, 2.0);
|
||||
let mut response = response_with_usage(TokenCounts {
|
||||
input_tokens: 1_000_000,
|
||||
output_tokens: 0,
|
||||
..TokenCounts::default()
|
||||
});
|
||||
|
||||
apply_estimated_cost(Some(&catalog), "gpt-test", None, &mut response);
|
||||
|
||||
assert_eq!(response.cost_source, Some(CostSource::Estimated));
|
||||
assert!(response.cost_usd.is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn apply_estimated_cost_leaves_source_unset_without_estimate() {
|
||||
let mut response = response_with_usage(TokenCounts::default());
|
||||
|
||||
apply_estimated_cost(None, "gpt-test", None, &mut response);
|
||||
|
||||
assert_eq!(response.cost_usd, None);
|
||||
assert_eq!(response.cost_source, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn apply_estimated_cost_keeps_existing_cost() {
|
||||
let catalog = priced_catalog(1.0, 2.0);
|
||||
let mut response = response_with_usage(TokenCounts {
|
||||
input_tokens: 1_000_000,
|
||||
output_tokens: 0,
|
||||
..TokenCounts::default()
|
||||
});
|
||||
response.cost_usd = Some(0.42);
|
||||
response.cost_source = Some(CostSource::Authoritative);
|
||||
|
||||
apply_estimated_cost(Some(&catalog), "gpt-test", None, &mut response);
|
||||
|
||||
assert_eq!(response.cost_usd, Some(0.42));
|
||||
assert_eq!(response.cost_source, Some(CostSource::Authoritative));
|
||||
}
|
||||
}
|
||||
|
|
@ -1138,6 +1138,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -1166,6 +1168,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
},
|
||||
)),
|
||||
];
|
||||
|
|
@ -1274,6 +1278,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
} else {
|
||||
// Second call: return text
|
||||
|
|
@ -1291,6 +1297,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -1360,6 +1368,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
|
||||
acc.process(&StreamEvent::finish(
|
||||
|
|
@ -1565,6 +1575,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -1596,6 +1608,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
},
|
||||
)));
|
||||
|
||||
|
|
@ -1775,6 +1789,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -1941,6 +1957,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -1970,6 +1988,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let events = vec![
|
||||
Ok(StreamEvent::ToolCallEnd { tool_call }),
|
||||
|
|
@ -1997,6 +2017,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let events = vec![
|
||||
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
|
||||
|
|
@ -2113,6 +2135,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
|
||||
let tool_calls = vec![ToolCall::new(
|
||||
|
|
@ -2276,6 +2300,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -2307,6 +2333,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let events = vec![
|
||||
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
|
||||
|
|
@ -2389,6 +2417,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -2405,6 +2435,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let events = vec![
|
||||
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
|
||||
|
|
@ -2487,6 +2519,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -2512,6 +2546,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let events = vec![
|
||||
Ok(StreamEvent::ToolCallEnd { tool_call }),
|
||||
|
|
@ -2536,6 +2572,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let events = vec![
|
||||
Ok(StreamEvent::text_delta(text, Some("t1".into()))),
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ pub mod adapter_registry;
|
|||
mod attachments;
|
||||
pub mod client;
|
||||
mod codec;
|
||||
pub(crate) mod cost;
|
||||
pub mod error;
|
||||
pub mod generate;
|
||||
pub mod middleware;
|
||||
|
|
|
|||
|
|
@ -211,6 +211,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -5,7 +5,9 @@ use tracing::{debug, error};
|
|||
use crate::error::{Error, error_from_status_code};
|
||||
use crate::provider::{ProviderAdapter, StreamEventStream};
|
||||
use crate::transport::{LineReader, parse_sse_block};
|
||||
use crate::types::{FinishReason, Message, Request, Response, StreamEvent, TokenCounts};
|
||||
use crate::types::{
|
||||
CostSource, FinishReason, Message, Request, Response, StreamEvent, TokenCounts,
|
||||
};
|
||||
|
||||
/// Provider adapter that routes LLM requests through an fabro server's
|
||||
/// `/completions` endpoint, delegating to whatever real provider the server
|
||||
|
|
@ -41,6 +43,8 @@ struct ServerCompletionResponse {
|
|||
message: Message,
|
||||
stop_reason: String,
|
||||
usage: ServerUsage,
|
||||
cost_usd: Option<f64>,
|
||||
cost_source: Option<CostSource>,
|
||||
}
|
||||
|
||||
#[derive(serde::Deserialize)]
|
||||
|
|
@ -151,6 +155,10 @@ impl ProviderAdapter for Adapter {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
// Carry the server's cost through; the local client's stamping
|
||||
// never overwrites an already-set cost.
|
||||
cost_usd: server_resp.cost_usd,
|
||||
cost_source: server_resp.cost_source,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -307,7 +315,9 @@ data: {\"type\":\"text_delta\",\"delta\":\" world\",\"text_id\":null}\n\
|
|||
"usage": {
|
||||
"input_tokens": 10,
|
||||
"output_tokens": 5
|
||||
}
|
||||
},
|
||||
"cost_usd": 0.000_25,
|
||||
"cost_source": "estimated"
|
||||
});
|
||||
|
||||
server.mock(|when, then| {
|
||||
|
|
@ -333,6 +343,8 @@ data: {\"type\":\"text_delta\",\"delta\":\" world\",\"text_id\":null}\n\
|
|||
assert_eq!(response.usage.input_tokens, 10);
|
||||
assert_eq!(response.usage.output_tokens, 5);
|
||||
assert_eq!(response.usage.total_tokens(), 15);
|
||||
assert_eq!(response.cost_usd, Some(0.000_25));
|
||||
assert_eq!(response.cost_source, Some(CostSource::Estimated));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
|
|
|||
|
|
@ -221,6 +221,10 @@ impl ToolChoice {
|
|||
|
||||
// --- 3.7 Response ---
|
||||
|
||||
// Billing vocabulary shared with the catalog/billing layer and the API
|
||||
// surface; re-exported here so `fabro_llm::types::*` imports keep working.
|
||||
pub use fabro_model::CostSource;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Response {
|
||||
pub id: String,
|
||||
|
|
@ -232,6 +236,13 @@ pub struct Response {
|
|||
pub raw: Option<serde_json::Value>,
|
||||
pub warnings: Vec<Warning>,
|
||||
pub rate_limit: Option<RateLimitInfo>,
|
||||
/// USD cost of this completion, when known or estimable.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub cost_usd: Option<f64>,
|
||||
/// Whether `cost_usd` came from provider billing data or a catalog
|
||||
/// estimate.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub cost_source: Option<CostSource>,
|
||||
}
|
||||
|
||||
impl Response {
|
||||
|
|
@ -703,6 +714,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
assert_eq!(response.text(), "Hello world");
|
||||
}
|
||||
|
|
@ -731,6 +744,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let calls = response.tool_calls();
|
||||
assert_eq!(calls.len(), 1);
|
||||
|
|
@ -762,6 +777,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
assert_eq!(response.reasoning(), Some("Let me think...".to_string()));
|
||||
assert_eq!(response.text(), "The answer is 42.");
|
||||
|
|
@ -779,6 +796,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
assert_eq!(response.reasoning(), None);
|
||||
}
|
||||
|
|
@ -949,6 +968,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
};
|
||||
let tool_calls = vec![ToolCall::new(
|
||||
"call_1",
|
||||
|
|
|
|||
|
|
@ -122,6 +122,19 @@ impl Speed {
|
|||
}
|
||||
}
|
||||
|
||||
/// Source of a USD cost value attached to a completion response.
|
||||
#[derive(
|
||||
Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Display, EnumString, IntoStaticStr,
|
||||
)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
#[strum(serialize_all = "snake_case")]
|
||||
pub enum CostSource {
|
||||
/// The provider returned billing data in-band with the response.
|
||||
Authoritative,
|
||||
/// Computed from catalog prices and token usage.
|
||||
Estimated,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
pub struct ModelRef {
|
||||
pub provider: ProviderId,
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ pub mod types;
|
|||
|
||||
pub use adapter::{AdapterKind, AgentProfileKind};
|
||||
pub use billing::{
|
||||
AnthropicBillingFacts, AnthropicModelPricing, BilledModelUsage, BilledTokenCounts,
|
||||
AnthropicBillingFacts, AnthropicModelPricing, BilledModelUsage, BilledTokenCounts, CostSource,
|
||||
GeminiBillingFacts, GeminiModelPricing, GeminiStoragePricing, GeminiStorageSegment,
|
||||
ModelBillingFacts, ModelBillingInput, ModelPricing, ModelPricingPolicy, ModelRef, ModelUsage,
|
||||
OpenAiBillingFacts, OpenAiModelPricing, PricePerMTok, Speed, TokenCounts, UsdMicros,
|
||||
|
|
|
|||
|
|
@ -355,6 +355,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: Vec::new(),
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -177,6 +177,8 @@ async fn create_completion(
|
|||
output_tokens: response.usage.output_tokens,
|
||||
},
|
||||
output,
|
||||
cost_usd: response.cost_usd,
|
||||
cost_source: response.cost_source,
|
||||
})
|
||||
.into_response()
|
||||
}
|
||||
|
|
@ -197,6 +199,8 @@ async fn create_completion(
|
|||
output_tokens: response.usage.output_tokens,
|
||||
},
|
||||
output: None,
|
||||
cost_usd: response.cost_usd,
|
||||
cost_source: response.cost_source,
|
||||
})
|
||||
.into_response()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1726,6 +1726,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -732,6 +732,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -760,6 +762,8 @@ mod tests {
|
|||
raw: None,
|
||||
warnings: vec![],
|
||||
rate_limit: None,
|
||||
cost_usd: None,
|
||||
cost_source: None,
|
||||
},
|
||||
)),
|
||||
];
|
||||
|
|
|
|||
|
|
@ -94,6 +94,7 @@ models/completion-tool-choice.ts
|
|||
models/completion-tool-definition.ts
|
||||
models/completion-usage.ts
|
||||
models/conclusion.ts
|
||||
models/cost-source.ts
|
||||
models/create-automation-request.ts
|
||||
models/create-completion-request.ts
|
||||
models/create-environment-request.ts
|
||||
|
|
|
|||
|
|
@ -19,6 +19,9 @@ import type { CompletionMessage } from './completion-message';
|
|||
// May contain unused imports in some cases
|
||||
// @ts-ignore
|
||||
import type { CompletionUsage } from './completion-usage';
|
||||
// May contain unused imports in some cases
|
||||
// @ts-ignore
|
||||
import type { CostSource } from './cost-source';
|
||||
|
||||
export interface CompletionResponse {
|
||||
'id': string;
|
||||
|
|
@ -30,4 +33,9 @@ export interface CompletionResponse {
|
|||
'stop_reason': string;
|
||||
'usage': CompletionUsage;
|
||||
'output'?: any;
|
||||
/**
|
||||
* USD cost of the completion when known: estimated from catalog prices unless the provider returned authoritative billing data.
|
||||
*/
|
||||
'cost_usd'?: number;
|
||||
'cost_source'?: CostSource;
|
||||
}
|
||||
|
|
|
|||
26
lib/packages/fabro-api-client/src/models/cost-source.ts
generated
Normal file
26
lib/packages/fabro-api-client/src/models/cost-source.ts
generated
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
/* tslint:disable */
|
||||
/* eslint-disable */
|
||||
/**
|
||||
* Fabro Run API
|
||||
* HTTP API for managing Fabro workflow run executions.
|
||||
*
|
||||
* The version of the OpenAPI document: 0.1.0
|
||||
*
|
||||
*
|
||||
* NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech).
|
||||
* https://openapi-generator.tech
|
||||
* Do not edit the class manually.
|
||||
*/
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* Whether `cost_usd` came from provider billing data (authoritative) or catalog price estimation (estimated).
|
||||
*/
|
||||
|
||||
export const CostSource = {
|
||||
AUTHORITATIVE: 'authoritative',
|
||||
ESTIMATED: 'estimated'
|
||||
} as const;
|
||||
|
||||
export type CostSource = typeof CostSource[keyof typeof CostSource];
|
||||
|
|
@ -66,6 +66,7 @@ export * from './completion-tool-choice';
|
|||
export * from './completion-tool-definition';
|
||||
export * from './completion-usage';
|
||||
export * from './conclusion';
|
||||
export * from './cost-source';
|
||||
export * from './create-automation-request';
|
||||
export * from './create-completion-request';
|
||||
export * from './create-environment-request';
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue