litellm/tests/integration/cost_calculation
Kerry Lu 455923014a
feat(cost): bill openai 12-hour prompt cache writes at their own rate (#48088)
* feat: add OpenAI 12-hour cache write pricing

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: omit empty cache-write response fields

* style: format cache-write fields

* fix(cost): bill openai 30m and 12h cache writes from the usage split, add gpt-5.6 rates

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost): apply the 12h long-context write rate only above the token threshold

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(cost): key cache write rates by TTL instead of growing the rate tuple

_get_token_base_cost returned a positional 6-tuple and every cache lifetime added a slot to it, to the off-peak helper and to every caller. It now returns a TokenBaseCosts dataclass whose cache write rates live in one mapping keyed by TTL. The TTLs come from the ephemeral_*_input_tokens fields on CacheCreationTokenDetails, so supporting a new lifetime only means adding that usage field and its rate key

A lifetime with no rate of its own now always falls back to the cache creation rate after off-peak pricing is applied. Before this the non-tiered 12h rate and the tiered 1h rate fell back to the rate from before off-peak, so the same request billed differently depending on the pricing path

* refactor(cost): hardcode the cache write TTLs as a Literal instead of parsing usage field names

Parsing ephemeral_*_input_tokens names saved nothing, since a new TTL still needs a usage field and provider parsing code. The TTLs are now a CacheWriteTTL Literal with an explicit rate key per TTL

* refactor(cost): resolve cache write lifetime rates to floats at lookup

Decide the 1h and 12h cost map key in the threshold loop, then resolve each
rate once after off-peak with the base write rate as the default, so no None
flows through the cost code. Add an integration case for a 12h write on a
row without a 12h rate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(cost): name every cache write TTL, marking 5m and 30m as base rate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): list every cache write TTL in one rate key table, None meaning the base write rate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): drop base rate entries from the cache write rate table and rely on fallback

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): key cache write rates by plain string TTL

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): fill missing cache write TTL rates with the base write rate after off-peak

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): default missing cache write TTL rates to the base write rate inline

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): rename cache write rate key table to make clear it only covers long TTLs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cost): list every cache write TTL and bill writes of an unknown TTL at the base write rate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): resolve tiered cache write TTL rates after off-peak instead of passing None

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): carry cache write TTL rates on TokenRates and drop TokenBaseCosts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): define TokenRates before its first use to drop the string annotation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cost): build the TokenRates TTL default with default_factory and type tiers as Mapping

MappingProxyType({}) as a dataclass default is rejected on Python 3.11, where
mappingproxy is unhashable, so import litellm failed there. The tier type
moves from dict[str, object] to Mapping[str, object] (LIT001), with tier_rate
accepting a Mapping to match.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost): type select_tier_for_input tiers as Mapping so tier_rate callers stay typed

The tier returned as a bare dict made every tier_rate call site a partially
unknown argument once tier_rate took Mapping[str, object]. The range bounds are
now validated with a TypeAdapter instead of indexing an unknown list.

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 09:07:02 -07:00
..
bridge_cost_parity test(cost): cover custom deployment aliases, off-peak pricing, bridge usage shapes and duplicate spend rows (#46708) 2026-10-15 10:44:51 -07:00
cases feat(cost): bill openai 12-hour prompt cache writes at their own rate (#48088) 2026-10-24 09:07:02 -07:00
stream_parity test(cost): add BridgeCostParityTestCase with a gpt-6-sol /v1/messages to Responses pair (#46130) 2026-10-13 02:51:55 +00:00
assertions.py test(integration): assert the streamed usage.cost matches the spend row on every cost case (#46762) 2026-10-15 22:34:42 +00:00
catalog.py test(cost): run azure_ai claude bridge pairs, add anthropic usage speed and vertex gemini audio priority price (#47582) 2026-10-20 17:10:16 +00:00
conftest.py test(cost): cover custom deployment aliases, off-peak pricing, bridge usage shapes and duplicate spend rows (#46708) 2026-10-15 10:44:51 -07:00
cost_map.py feat(cost): bill openai 12-hour prompt cache writes at their own rate (#48088) 2026-10-24 09:07:02 -07:00
cost_tracking_case.py feat(cost): bill openai 12-hour prompt cache writes at their own rate (#48088) 2026-10-24 09:07:02 -07:00
test_base_model_cross_provider_spend.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_batch_realtime_cost.py test(integration): move the openai cost tracking cases from cost_tracking_cases.json to CostTrackingTestCase literals (#45879) 2026-10-10 23:29:23 +00:00
test_bridged_previous_response_id_spend.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_cost_tracking.py test(integration): assert the streamed usage.cost matches the spend row on every cost case (#46762) 2026-10-15 22:34:42 +00:00
test_responses_websocket_guardrail_spend.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00