refactor(cost): calculate over the catalog price model

This commit is contained in:
Devin AI 2026-10-26 02:12:44 +00:00
parent 5052dd3536
commit c8fa5a7a39
11 changed files with 1115 additions and 1014 deletions

View file

@ -3694,11 +3694,13 @@ name = "litellm-cost"
version = "0.1.0"
dependencies = [
"criterion",
"jiff",
"proptest",
"rstest",
"schemars 1.2.2",
"serde",
"serde_json",
"time",
]
[[package]]

View file

@ -10,6 +10,8 @@ serde = ["dep:serde"]
schema = ["serde", "dep:schemars"]
[dependencies]
jiff = { version = "0.2.37", default-features = false, features = ["std", "tzdb-bundle-always"] }
time.workspace = true
schemars = { workspace = true, optional = true }
serde = { workspace = true, optional = true }

View file

@ -1,15 +1,20 @@
# litellm-cost
This crate calculates text token charges from rates and usage supplied by its caller. It is standalone and has no Python bridge or proxy integration
Call `compile(&pricing)` once for an immutable plan, then `plan.calculate(&request)` for each supported request. `calculate(&pricing, &request)` compiles on each call. A successful result exposes pre-multiplier component costs, selected rates, the multiplier, and derived `input()`, `output()`, and `total()` values
The caller states whether `prompt_tokens` includes cache tokens. Threshold selection uses total input tokens for either convention and selects one rate for the whole request. Thresholds are sorted when compiled, and duplicate thresholds or tier overrides fail deterministically. `Fast` selects priority rates; unknown tiers use standard rates
`Rate::Missing`, `Rate::Null`, and `Rate::Value(0.0)` remain distinct. Missing cache rates fall back to the selected input rate, and an absent one-hour write rate falls back to the selected write rate. Missing input or output rates return typed errors, including for zero usage. Python's sparse-entry behavior remains outside this native contract
The supported off-peak shape is one non-wrapping UTC daily window. The caller supplies the applicable regional multiplier after provider-specific selection. Negative or non-finite rates, ambiguous rules, inconsistent cache counts, incomplete write splits, invalid windows and overflow return errors. Callers must decline unsupported inputs before native execution if their public contract accepts those shapes
This crate does not select models, read catalogs, fetch provider prices, normalize multimodal usage, process provider-reported costs, or calculate non-token charges. It does not change proxy behavior. The reference fixture was generated by `tests/generate_python_reference.py` against the Python implementation at the commit recorded in `tests/python_reference.tsv`, using synthetic rates and fixed usage
`money`, `price` and `schedule` hold the typed price model that `litellm-model-catalog` lowers each catalog entry into. `UnitPrice` is USD per unit and `Usd` is an amount, and only the operations that make sense between them exist, so a price is never added to an amount. `Prices::band` and `Prices::rate` resolve a token rate the way Python's `_get_token_base_cost` does. `calculate` does not read this model yet
- `calculate(&prices, &usage, &when)` consumes the same `Prices` that `litellm-model-catalog` produces. There is no separate pricing specification or compilation step, and no runtime caller is migrated here
- `Usage::quantities` contains disjoint billable quantities keyed by `Meter`, including text, audio, image, reasoning, cache reads, writes by TTL, characters, seconds, images and queries. Text counts exclude the quantities assigned to sub-meters. Provider response normalization stays with the caller
- `prompt_tokens` is the total prompt count, including cache and modalities, used only for selecting a prompt band. It is not billed a second time
- `inclusive_threshold` chooses `>=` instead of `>` at prompt-band boundaries
- `search_results` selects a query result band. Query quantity remains the number of queries. Video-second quantity selects a duration interval
- `When` supplies the service tier, channel and billing timestamp. `price::When` remains the catalog key for a tier and band
- `Priced::Known` contains a `Breakdown`. `Priced::Unpriced` lists every used meter without a price and retains the priced lines and their partial total. Zero usage needs no price. An explicit zero price remains a known zero-cost line
- Every line records its billed meter, quantity, unit price and amount. When a sub-meter falls back, `fallback_from` identifies the original usage meter and `meter` identifies the price source
- Audio, image and video tokens fall back to text on the same side, and reasoning falls back to text output
- Cache reads fall back to text input. Long-TTL writes fall back through the default write price to text input
- Missing non-token prices never fall back to token prices
- Online lookup preserves R1's `(tier, band)`, `(standard, band)`, `(tier, base)`, `(standard, base)` order before sub-meter fallback
- Off-peak overrides only declared rates during matching UTC windows. Windows can wrap midnight and restrict weekdays in an IANA timezone
- Batch prices select bands per meter. Explicit batch prices win, missing input sub-meters use the batch input rate, and missing batch token rates can use half the online base rate
- Decisions use their own channel and never silently borrow online prices
- `Breakdown::input()` sums non-output lines and `output()` sums output lines. `total` sums all lines. Prices and amounts remain money newtypes with `f64` internally
- Regional adjustments remain separate from prices. The caller selects and applies `Adjustments` when needed
- Run `cargo run -p litellm-cost --example charge` for a normalized usage example. The existing Python reference TSV is still tested unchanged. Broader fixture generation and full Python pricing parity are deferred

View file

@ -1,64 +1,75 @@
use criterion::{Criterion, criterion_group, criterion_main};
use litellm_cost::{
Pricing, PromptConvention, Rate, Rates, Request, ServiceTier, ThresholdPolicy, ThresholdRates,
Usage, calculate, compile,
};
use std::hint::black_box;
use criterion::{Criterion, criterion_group, criterion_main};
use litellm_cost::money::UnitPrice;
use litellm_cost::price::{
Band, CacheReadMode, Channel, InputUnit, Meter, OutputUnit, Price, Prices, ServiceTier,
TokenKind, When as PriceWhen,
};
use litellm_cost::{Usage, When, calculate};
use time::OffsetDateTime;
fn bench(c: &mut Criterion) {
let pricing = Pricing {
standard: Rates {
input: Rate::Value(0.000002),
output: Rate::Value(0.000008),
cache_read: Rate::Value(0.0000005),
cache_write: Rate::Missing,
cache_write_1h: Rate::Missing,
},
tiers: &[],
thresholds: &[],
off_peak: None,
let input = Meter::Input(InputUnit::Token(TokenKind::Text));
let output = Meter::Output(OutputUnit::Token(TokenKind::Text));
let cache = Meter::CacheRead {
kind: TokenKind::Text,
mode: CacheReadMode::Implicit,
};
let request = Request {
usage: Usage {
prompt_tokens: 1000,
completion_tokens: 200,
cache_read_tokens: 250,
cache_write_tokens: 0,
cache_write_5m_tokens: None,
cache_write_1h_tokens: None,
prompt_convention: PromptConvention::IncludesCache,
},
service_tier: ServiceTier::Standard,
threshold_policy: ThresholdPolicy::Exclusive,
region_multiplier: None,
billed_at_utc_minute: None,
let prices = Prices::new(
[
(input.clone(), Band::Base, 0.000002),
(output.clone(), Band::Base, 0.000008),
(cache.clone(), Band::Base, 0.0000005),
(input.clone(), Band::over_prompt_tokens(1000), 0.000004),
]
.into_iter()
.map(|(meter, band, rate)| {
(
Price::online(
meter,
PriceWhen {
band,
..PriceWhen::default()
},
),
UnitPrice::new(rate).unwrap(),
)
})
.collect(),
None,
);
let usage = Usage {
quantities: [(input, 750), (output, 200), (cache, 250)].into(),
prompt_tokens: 1000,
..Usage::default()
};
let plan = compile(&pricing).unwrap();
c.bench_function("native_compiled_calculation", |b| {
b.iter(|| black_box(plan.calculate(black_box(&request)).unwrap()))
let when = When {
tier: ServiceTier::Standard,
channel: Channel::Online,
at: OffsetDateTime::UNIX_EPOCH,
};
c.bench_function("catalog_calculation", |b| {
b.iter(|| {
black_box(calculate(
black_box(&prices),
black_box(&usage),
black_box(&when),
))
})
});
c.bench_function("native_full_wrapper", |b| {
b.iter(|| black_box(calculate(black_box(&pricing), black_box(&request)).unwrap()))
});
c.bench_function("native_rate_compilation", |b| {
b.iter(|| black_box(compile(black_box(&pricing)).unwrap()))
});
let threshold = ThresholdRates {
above_prompt_tokens: 1000,
standard: Rates {
input: Rate::Value(0.000004),
output: Rate::Value(0.000016),
..Rates::EMPTY
},
tiers: &[],
let above = Usage {
prompt_tokens: 1001,
..usage
};
let threshold_pricing = Pricing {
thresholds: &[threshold],
..pricing
};
let threshold_plan = compile(&threshold_pricing).unwrap();
c.bench_function("native_threshold_boundary", |b| {
b.iter(|| black_box(threshold_plan.calculate(black_box(&request)).unwrap()))
c.bench_function("catalog_threshold_calculation", |b| {
b.iter(|| {
black_box(calculate(
black_box(&prices),
black_box(&above),
black_box(&when),
))
})
});
}

View file

@ -1,40 +1,52 @@
use litellm_cost::{
Pricing, PromptConvention, Rate, Rates, Request, ServiceTier, ThresholdPolicy, Usage, compile,
use litellm_cost::money::UnitPrice;
use litellm_cost::price::{
CacheReadMode, Channel, InputUnit, Meter, OutputUnit, Price, Prices, ServiceTier, TokenKind,
};
use litellm_cost::{Priced, Usage, When, calculate};
use time::OffsetDateTime;
fn main() {
let pricing = Pricing {
standard: Rates {
input: Rate::Value(2.0),
output: Rate::Value(4.0),
cache_read: Rate::Value(0.5),
cache_write: Rate::Value(3.0),
cache_write_1h: Rate::Missing,
},
tiers: &[],
thresholds: &[],
off_peak: None,
let input = Meter::Input(InputUnit::Token(TokenKind::Text));
let output = Meter::Output(OutputUnit::Token(TokenKind::Text));
let cache = Meter::CacheRead {
kind: TokenKind::Text,
mode: CacheReadMode::Implicit,
};
let request = Request {
usage: Usage {
prompt_tokens: 100,
completion_tokens: 20,
cache_read_tokens: 25,
cache_write_tokens: 10,
cache_write_5m_tokens: None,
cache_write_1h_tokens: None,
prompt_convention: PromptConvention::IncludesCache,
},
service_tier: ServiceTier::Standard,
threshold_policy: ThresholdPolicy::Exclusive,
region_multiplier: None,
billed_at_utc_minute: None,
};
let cost = compile(&pricing).unwrap().calculate(&request).unwrap();
println!(
"input={} output={} total={}",
cost.input(),
cost.output(),
cost.total()
let prices = Prices::new(
[
(input.clone(), 2.0),
(output.clone(), 4.0),
(cache.clone(), 0.5),
]
.into_iter()
.map(|(meter, rate)| {
(
Price::online(meter, Default::default()),
UnitPrice::new(rate).unwrap(),
)
})
.collect(),
None,
);
let usage = Usage {
quantities: [(input, 75), (cache, 25), (output, 20)].into(),
prompt_tokens: 100,
..Usage::default()
};
let when = When {
tier: ServiceTier::Standard,
channel: Channel::Online,
at: OffsetDateTime::UNIX_EPOCH,
};
match calculate(&prices, &usage, &when) {
Priced::Known(cost) => println!(
"input={} output={} total={}",
cost.input().get(),
cost.output().get(),
cost.total.get()
),
Priced::Unpriced { missing, partial } => {
println!("unpriced={missing:?} partial={}", partial.total.get())
}
}
}

File diff suppressed because it is too large Load diff

View file

@ -17,6 +17,10 @@ impl UnitPrice {
pub fn get(self) -> f64 {
self.0
}
pub(crate) fn half(self) -> Self {
Self(self.0 / 2.0)
}
}
impl Usd {

View file

@ -119,10 +119,20 @@ impl Prices {
/// The online prompt band a request falls in. Every token meter's `_above_` prices count,
/// and the highest band the prompt passes wins.
pub fn band(&self, tier: ServiceTier, prompt_tokens: u64, inclusive: bool) -> Band {
self.channel_band(Channel::Online, tier, prompt_tokens, inclusive)
}
pub(crate) fn channel_band(
&self,
channel: Channel,
tier: ServiceTier,
prompt_tokens: u64,
inclusive: bool,
) -> Band {
self.rates
.keys()
.filter(|price| {
price.channel == Channel::Online
price.channel == channel
&& price.meter.sets_prompt_band()
&& (price.when.tier == ServiceTier::Standard || price.when.tier == tier)
})
@ -139,6 +149,17 @@ impl Prices {
/// and `(standard, base)` in that order. A missing cache price falls back to the input price
/// resolved the same way.
pub fn rate(&self, meter: &Meter, tier: ServiceTier, band: Band) -> Option<UnitPrice> {
self.direct_rate(meter, Channel::Online, tier, band)
.or_else(|| self.rate(&meter.fallback()?, tier, band))
}
pub(crate) fn direct_rate(
&self,
meter: &Meter,
channel: Channel,
tier: ServiceTier,
band: Band,
) -> Option<UnitPrice> {
[
When { tier, band },
When {
@ -152,8 +173,36 @@ impl Prices {
When::default(),
]
.into_iter()
.find_map(|when| self.get(&Price::online(meter.clone(), when)))
.or_else(|| self.rate(&meter.fallback()?, tier, band))
.find_map(|when| {
self.get(&Price {
meter: meter.clone(),
channel,
when,
})
})
}
}
impl OffPeakRates {
pub(crate) fn get(&self, meter: &Meter) -> Option<UnitPrice> {
match meter {
Meter::Input(InputUnit::Token(TokenKind::Text)) => self.input,
Meter::Output(OutputUnit::Token(TokenKind::Text)) => self.output,
Meter::Output(OutputUnit::ReasoningToken) => self.reasoning,
Meter::CacheRead {
kind: TokenKind::Text,
mode: CacheReadMode::Implicit,
} => self.cache_read,
Meter::CacheRead {
kind: TokenKind::Text,
mode: CacheReadMode::Explicit,
} => self.explicit_cache_read.or(self.cache_read),
Meter::CacheWrite {
kind: TokenKind::Text,
ttl: CacheTtl::Default,
} => self.cache_write,
_ => None,
}
}
}
@ -329,11 +378,11 @@ mod tests {
Band::Base,
Some(2.0)
)]
#[case::no_fallback_for_other_meters(
#[case::audio_falls_back_to_input(
Meter::Input(InputUnit::Token(TokenKind::Audio)),
ServiceTier::Standard,
Band::Base,
None
Some(1.0)
)]
fn rate_walks_tier_then_band(
priority_with_a_standard_band: Prices,

View file

@ -121,11 +121,28 @@ impl Meter {
} => Some(Self::TEXT_INPUT),
Self::CacheWrite {
kind: TokenKind::Text,
ttl: CacheTtl::OneHour,
ttl: CacheTtl::OneHour | CacheTtl::TwelveHours,
} => Some(Self::CacheWrite {
kind: TokenKind::Text,
ttl: CacheTtl::Default,
}),
Self::CacheRead { .. } => Some(Self::CacheRead {
kind: TokenKind::Text,
mode: CacheReadMode::Implicit,
}),
Self::CacheWrite { kind, ttl } if *kind != TokenKind::Text => Some(Self::CacheWrite {
kind: TokenKind::Text,
ttl: *ttl,
}),
Self::Input(InputUnit::Token(
TokenKind::Audio | TokenKind::Image | TokenKind::Video,
)) => Some(Self::TEXT_INPUT),
Self::Output(OutputUnit::Token(
TokenKind::Audio | TokenKind::Image | TokenKind::Video,
))
| Self::Output(OutputUnit::ReasoningToken) => {
Some(Self::Output(OutputUnit::Token(TokenKind::Text)))
}
_ => None,
}
}

View file

@ -1,5 +1,8 @@
use std::str::FromStr;
use jiff::{Timestamp, tz::TimeZone};
use time::{OffsetDateTime, UtcOffset};
use crate::error::{InvalidUtcWindow, InvalidWeekday};
#[derive(Clone, Debug, PartialEq, Eq)]
@ -31,6 +34,35 @@ pub enum Weekday {
Sunday,
}
impl Schedule {
pub fn contains(&self, at: OffsetDateTime) -> bool {
let utc = at.to_offset(UtcOffset::UTC);
let minute = u16::from(utc.hour()) * 60 + u16::from(utc.minute());
let weekday = Timestamp::from_second(at.unix_timestamp())
.ok()
.and_then(|stamp| {
let zone = match self.weekday_timezone.as_deref() {
Some(name) => TimeZone::get(name).ok()?,
None => TimeZone::UTC,
};
Weekday::from_iso(stamp.to_zoned(zone).weekday().to_monday_one_offset() as u8)
});
self.windows.iter().any(|window| {
window
.weekdays
.as_ref()
.is_none_or(|days| weekday.is_some_and(|day| days.contains(&day)))
&& window.hours.iter().any(|hours| {
if hours.start_minute < hours.end_minute {
(hours.start_minute..hours.end_minute).contains(&minute)
} else {
minute >= hours.start_minute || minute < hours.end_minute
}
})
})
}
}
impl FromStr for UtcWindow {
type Err = InvalidUtcWindow;
@ -181,10 +213,67 @@ mod schema {
#[cfg(test)]
mod tests {
use time::format_description::well_known::Rfc3339;
use rstest::rstest;
use super::*;
#[rstest]
#[case::start("2026-01-05T22:00:00Z", "22:00-01:00", None, None, true)]
#[case::wrapped("2026-01-06T00:59:59Z", "22:00-01:00", None, None, true)]
#[case::end("2026-01-06T01:00:00Z", "22:00-01:00", None, None, false)]
#[case::outside("2026-01-05T21:59:59Z", "22:00-01:00", None, None, false)]
#[case::whole_day("2026-01-05T12:00:00Z", "00:00-00:00", None, None, true)]
#[case::utc_monday(
"2026-01-05T01:00:00Z",
"00:00-02:00",
Some(Weekday::Monday),
None,
true
)]
#[case::local_sunday(
"2026-01-05T01:00:00Z",
"00:00-02:00",
Some(Weekday::Sunday),
Some("America/Los_Angeles"),
true
)]
#[case::not_local_monday(
"2026-01-05T01:00:00Z",
"00:00-02:00",
Some(Weekday::Monday),
Some("America/Los_Angeles"),
false
)]
#[case::summer_offset(
"2026-07-06T07:30:00Z",
"07:00-08:00",
Some(Weekday::Monday),
Some("America/Los_Angeles"),
true
)]
#[case::supplied_offset("2026-01-05T23:00:00+02:00", "21:00-22:00", None, None, true)]
fn windows_use_utc_hours_and_local_weekdays(
#[case] at: &str,
#[case] hours: &str,
#[case] weekday: Option<Weekday>,
#[case] zone: Option<&str>,
#[case] expected: bool,
) {
let schedule = Schedule {
windows: vec![Window {
hours: vec![hours.parse().unwrap()],
weekdays: weekday.map(|day| vec![day]),
}],
weekday_timezone: zone.map(str::to_owned),
};
assert_eq!(
schedule.contains(OffsetDateTime::parse(at, &Rfc3339).unwrap()),
expected
);
}
#[rstest]
#[case::same_day("10:00-18:30", Some((600, 1110)))]
#[case::wraps_midnight("22:00-02:00", Some((1320, 120)))]

View file

@ -1,535 +0,0 @@
use litellm_cost::{
OffPeakRates, Pricing, PricingError, PromptConvention, Rate, Rates, Request, ServiceTier,
ThresholdPolicy, ThresholdRates, TierRates, Usage, calculate, compile,
};
use rstest::rstest;
fn rates(input: Rate, output: Rate) -> Rates {
Rates {
input,
output,
..Rates::EMPTY
}
}
fn request() -> Request {
Request {
usage: Usage {
prompt_tokens: 100,
completion_tokens: 20,
cache_read_tokens: 25,
cache_write_tokens: 10,
cache_write_5m_tokens: None,
cache_write_1h_tokens: None,
prompt_convention: PromptConvention::IncludesCache,
},
service_tier: ServiceTier::Standard,
threshold_policy: ThresholdPolicy::Exclusive,
region_multiplier: None,
billed_at_utc_minute: None,
}
}
fn pricing(standard: Rates) -> Pricing<'static> {
Pricing {
standard,
tiers: &[],
thresholds: &[],
off_peak: None,
}
}
#[test]
fn breakdown_and_total_agree() {
let standard = Rates {
cache_read: Rate::Value(0.5),
cache_write: Rate::Value(3.0),
..rates(Rate::Value(2.0), Rate::Value(4.0))
};
let result = calculate(&pricing(standard), &request()).unwrap();
assert_eq!(result.uncached_input, 65.0 * 2.0);
assert_eq!(result.cache_read, 25.0 * 0.5);
assert_eq!(result.cache_write_5m, 10.0 * 3.0);
assert_eq!(result.output(), 20.0 * 4.0);
assert_eq!(result.total(), result.input() + result.output());
assert_eq!(result.rates.cache_read, 0.5);
}
#[rstest]
#[case::missing(Rate::Missing, Rate::Missing, 200.0)]
#[case::null(Rate::Null, Rate::Missing, 200.0)]
#[case::zero(Rate::Value(0.0), Rate::Value(0.0), 130.0)]
fn absent_null_and_zero_cache_rates_are_distinct(
#[case] cache_read: Rate,
#[case] cache_write: Rate,
#[case] expected_input: f64,
) {
let base = rates(Rate::Value(2.0), Rate::Value(4.0));
let standard = Rates {
cache_read,
cache_write,
..base
};
assert_eq!(
calculate(&pricing(standard), &request()).unwrap().input(),
expected_input
);
}
#[test]
fn equivalent_prompt_conventions_select_the_same_threshold() {
let threshold = ThresholdRates {
above_prompt_tokens: 90,
standard: rates(Rate::Value(5.0), Rate::Value(8.0)),
tiers: &[],
};
let specification = Pricing {
standard: rates(Rate::Value(2.0), Rate::Value(4.0)),
tiers: &[],
thresholds: &[threshold],
off_peak: None,
};
let included = request();
let excluded = Request {
usage: Usage {
prompt_tokens: 65,
prompt_convention: PromptConvention::ExcludesCache,
..included.usage
},
..included
};
let plan = compile(&specification).unwrap();
assert_eq!(plan.calculate(&included), plan.calculate(&excluded));
assert_eq!(plan.calculate(&included).unwrap().rates.input, 5.0);
}
#[test]
fn split_writes_and_invalid_accounting() {
let standard = Rates {
cache_read: Rate::Value(0.5),
cache_write: Rate::Value(3.0),
cache_write_1h: Rate::Value(5.0),
..rates(Rate::Value(2.0), Rate::Value(4.0))
};
let base = request();
let split = Request {
usage: Usage {
cache_write_5m_tokens: Some(4),
cache_write_1h_tokens: Some(6),
..base.usage
},
..base
};
let result = calculate(&pricing(standard), &split).unwrap();
assert_eq!(result.cache_write_5m, 12.0);
assert_eq!(result.cache_write_1h, 30.0);
let overlapping = Request {
usage: Usage {
prompt_tokens: 30,
..split.usage
},
..split
};
assert_eq!(
calculate(&pricing(standard), &overlapping),
Err(PricingError::CacheExceedsPrompt)
);
let incomplete = Request {
usage: Usage {
cache_write_1h_tokens: None,
..split.usage
},
..split
};
assert_eq!(
calculate(&pricing(standard), &incomplete),
Err(PricingError::InvalidCacheWriteDetails)
);
}
#[test]
fn threshold_tiers_and_boundaries() {
let priority = TierRates {
tier: ServiceTier::Priority,
rates: rates(Rate::Value(3.0), Rate::Missing),
};
let threshold = ThresholdRates {
above_prompt_tokens: 100,
standard: rates(Rate::Value(5.0), Rate::Value(8.0)),
tiers: &[
TierRates {
tier: ServiceTier::Priority,
rates: rates(Rate::Value(7.0), Rate::Missing),
},
TierRates {
tier: ServiceTier::Flex,
rates: rates(Rate::Value(6.0), Rate::Missing),
},
],
};
let specification = Pricing {
standard: rates(Rate::Value(2.0), Rate::Value(4.0)),
tiers: &[priority],
thresholds: &[threshold],
off_peak: None,
};
let base = request();
let no_cache = Request {
usage: Usage {
cache_read_tokens: 0,
cache_write_tokens: 0,
..base.usage
},
..base
};
let fast = Request {
service_tier: ServiceTier::Fast,
..no_cache
};
let inclusive = Request {
threshold_policy: ThresholdPolicy::Inclusive,
..fast
};
let flex = Request {
service_tier: ServiceTier::Flex,
..inclusive
};
assert_eq!(calculate(&specification, &no_cache).unwrap().input(), 200.0);
assert_eq!(calculate(&specification, &fast).unwrap().input(), 300.0);
assert_eq!(
calculate(&specification, &inclusive).unwrap().input(),
700.0
);
assert_eq!(calculate(&specification, &flex).unwrap().input(), 600.0);
}
#[rstest]
#[case::ultrafast_above_threshold(ServiceTier::Ultrafast, 300_000, 9_301_000.0, 37_000.0)]
#[case::ultrafast_at_threshold(ServiceTier::Ultrafast, 272_000, 544_500.0, 5_000.0)]
#[case::standard_above_threshold(ServiceTier::Standard, 300_000, 3_300_600.0, 13_000.0)]
#[case::priority_above_threshold(ServiceTier::Priority, 300_000, 5_701_000.0, 23_000.0)]
fn tiered_long_context_rates_are_selected_by_service_tier(
#[case] service_tier: ServiceTier,
#[case] prompt_tokens: u64,
#[case] expected_input: f64,
#[case] expected_output: f64,
) {
let standard = Rates {
cache_read: Rate::Value(3.0),
..rates(Rate::Value(1.0), Rate::Value(2.0))
};
let tiers = [
TierRates {
tier: ServiceTier::Priority,
rates: Rates {
cache_read: Rate::Value(5.0),
..rates(Rate::Value(3.0), Rate::Value(4.0))
},
},
TierRates {
tier: ServiceTier::Ultrafast,
rates: Rates {
cache_read: Rate::Value(7.0),
..rates(Rate::Value(2.0), Rate::Value(5.0))
},
},
];
let threshold_tiers = [
TierRates {
tier: ServiceTier::Priority,
rates: Rates {
cache_read: Rate::Value(29.0),
..rates(Rate::Value(19.0), Rate::Value(23.0))
},
},
TierRates {
tier: ServiceTier::Ultrafast,
rates: Rates {
cache_read: Rate::Value(41.0),
..rates(Rate::Value(31.0), Rate::Value(37.0))
},
},
];
let thresholds = [ThresholdRates {
above_prompt_tokens: 272_000,
standard: Rates {
cache_read: Rate::Value(17.0),
..rates(Rate::Value(11.0), Rate::Value(13.0))
},
tiers: &threshold_tiers,
}];
let pricing = Pricing {
standard,
tiers: &tiers,
thresholds: &thresholds,
off_peak: None,
};
let base = request();
let long_context_request = Request {
usage: Usage {
prompt_tokens,
completion_tokens: 1_000,
cache_read_tokens: 100,
cache_write_tokens: 0,
..base.usage
},
service_tier,
..base
};
let cost = calculate(&pricing, &long_context_request).unwrap();
assert_eq!(cost.input(), expected_input);
assert_eq!(cost.output(), expected_output);
}
#[test]
fn compile_rejects_ambiguous_rates() {
let duplicate = ThresholdRates {
above_prompt_tokens: 100,
standard: Rates::EMPTY,
tiers: &[],
};
let specification = Pricing {
standard: rates(Rate::Value(1.0), Rate::Value(1.0)),
tiers: &[],
thresholds: &[duplicate, duplicate],
off_peak: None,
};
assert_eq!(
compile(&specification).err(),
Some(PricingError::DuplicateThreshold)
);
let invalid = pricing(rates(Rate::Value(f64::NAN), Rate::Value(1.0)));
assert_eq!(compile(&invalid).err(), Some(PricingError::InvalidRate));
}
#[test]
fn off_peak_is_one_non_wrapping_utc_window() {
let specification = Pricing {
standard: rates(Rate::Value(2.0), Rate::Value(4.0)),
tiers: &[],
thresholds: &[],
off_peak: Some(OffPeakRates {
start_utc_minute: 60,
end_utc_minute: 120,
rates: rates(Rate::Value(1.0), Rate::Value(2.0)),
}),
};
let base = request();
let start = Request {
billed_at_utc_minute: Some(60),
..base
};
let end = Request {
billed_at_utc_minute: Some(120),
..base
};
assert_eq!(
calculate(&specification, &base),
Err(PricingError::InvalidBillingTime)
);
assert_eq!(calculate(&specification, &start).unwrap().input(), 100.0);
assert_eq!(calculate(&specification, &end).unwrap().input(), 200.0);
}
#[test]
fn missing_rates_and_free_rates_remain_distinct() {
let base = request();
let empty = Request {
usage: Usage {
prompt_tokens: 0,
completion_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
..base.usage
},
..base
};
assert_eq!(
calculate(&pricing(Rates::EMPTY), &empty),
Err(PricingError::MissingInputRate)
);
assert_eq!(
calculate(&pricing(rates(Rate::Value(0.0), Rate::Missing)), &empty),
Err(PricingError::MissingOutputRate)
);
assert_eq!(
calculate(&pricing(rates(Rate::Value(0.0), Rate::Value(0.0))), &empty)
.unwrap()
.total(),
0.0
);
}
#[test]
fn matches_executed_python_reference_cases() {
for row in include_str!("python_reference.tsv")
.lines()
.filter(|line| !line.starts_with('#'))
{
let fields: Vec<_> = row.split('\t').collect();
let count = |index: usize| fields[index].parse::<u64>().unwrap();
let number = |index: usize| fields[index].parse::<f64>().unwrap();
let optional_rate = |index: usize| {
if fields[index].is_empty() {
Rate::Missing
} else {
Rate::Value(number(index))
}
};
let threshold = ThresholdRates {
above_prompt_tokens: if fields[9].is_empty() { 0 } else { count(9) },
standard: rates(optional_rate(10), optional_rate(11)),
tiers: &[],
};
let thresholds = if fields[9].is_empty() {
&[][..]
} else {
std::slice::from_ref(&threshold)
};
let specification = Pricing {
standard: Rates {
cache_read: optional_rate(7),
cache_write: optional_rate(8),
..rates(Rate::Value(number(5)), Rate::Value(number(6)))
},
tiers: &[],
thresholds,
off_peak: None,
};
let base = request();
let input = Request {
usage: Usage {
prompt_tokens: count(1),
completion_tokens: count(2),
cache_read_tokens: count(3),
cache_write_tokens: count(4),
..base.usage
},
..base
};
let actual = calculate(&specification, &input).unwrap();
assert_eq!(actual.input(), number(12), "{}", fields[0]);
assert_eq!(actual.output(), number(13), "{}", fields[0]);
}
}
proptest::proptest! {
#[test]
fn equivalent_usage_conventions_and_breakdown_agree(
regular in 0_u64..1000,
read in 0_u64..1000,
write in 0_u64..1000,
output in 0_u64..1000,
) {
let threshold = ThresholdRates {
above_prompt_tokens: 1000,
standard: rates(Rate::Value(5.0), Rate::Value(8.0)),
tiers: &[],
};
let specification = Pricing {
standard: rates(Rate::Value(2.0), Rate::Value(4.0)),
tiers: &[],
thresholds: &[threshold],
off_peak: None,
};
let base = request();
let included = Request {
usage: Usage {
prompt_tokens: regular + read + write,
completion_tokens: output,
cache_read_tokens: read,
cache_write_tokens: write,
..base.usage
},
..base
};
let excluded = Request {
usage: Usage {
prompt_tokens: regular,
prompt_convention: PromptConvention::ExcludesCache,
..included.usage
},
..included
};
let plan = compile(&specification).unwrap();
let left = plan.calculate(&included).unwrap();
let right = plan.calculate(&excluded).unwrap();
proptest::prop_assert_eq!(left, right);
proptest::prop_assert_eq!(left.total(), left.input() + left.output());
}
}
#[test]
fn regional_multiplier_applies_after_input_components_are_summed() {
let standard = Rates {
cache_read: Rate::Value(0.5),
cache_write: Rate::Value(3.0),
..rates(Rate::Value(2.0), Rate::Value(4.0))
};
let base = request();
let regional = Request {
region_multiplier: Some(1.1),
..base
};
let result = calculate(&pricing(standard), &regional).unwrap();
assert_eq!(result.input(), (65.0 * 2.0 + 25.0 * 0.5 + 10.0 * 3.0) * 1.1);
assert_eq!(result.output(), 20.0 * 4.0 * 1.1);
let invalid = Request {
region_multiplier: Some(f64::NAN),
..base
};
assert_eq!(
calculate(&pricing(standard), &invalid),
Err(PricingError::InvalidRegionMultiplier)
);
}
#[test]
fn compilation_sorts_thresholds_and_rejects_duplicate_tiers() {
let high = ThresholdRates {
above_prompt_tokens: 200,
standard: rates(Rate::Value(7.0), Rate::Missing),
tiers: &[],
};
let low = ThresholdRates {
above_prompt_tokens: 100,
standard: rates(Rate::Value(5.0), Rate::Missing),
tiers: &[],
};
let specification = Pricing {
standard: rates(Rate::Value(2.0), Rate::Value(4.0)),
tiers: &[],
thresholds: &[high, low],
off_peak: None,
};
let base = request();
let above_both = Request {
usage: Usage {
prompt_tokens: 201,
cache_read_tokens: 0,
cache_write_tokens: 0,
..base.usage
},
..base
};
assert_eq!(
compile(&specification)
.unwrap()
.calculate(&above_both)
.unwrap()
.rates
.input,
7.0
);
let duplicate = TierRates {
tier: ServiceTier::Flex,
rates: Rates::EMPTY,
};
let invalid = Pricing {
tiers: &[duplicate, duplicate],
thresholds: &[],
..specification
};
assert_eq!(compile(&invalid).err(), Some(PricingError::DuplicateTier));
}