mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-01 07:03:02 +00:00
455923014a
* feat: add OpenAI 12-hour cache write pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: omit empty cache-write response fields
* style: format cache-write fields
* fix(cost): bill openai 30m and 12h cache writes from the usage split, add gpt-5.6 rates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): apply the 12h long-context write rate only above the token threshold
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): key cache write rates by TTL instead of growing the rate tuple
_get_token_base_cost returned a positional 6-tuple and every cache lifetime added a slot to it, to the off-peak helper and to every caller. It now returns a TokenBaseCosts dataclass whose cache write rates live in one mapping keyed by TTL. The TTLs come from the ephemeral_*_input_tokens fields on CacheCreationTokenDetails, so supporting a new lifetime only means adding that usage field and its rate key
A lifetime with no rate of its own now always falls back to the cache creation rate after off-peak pricing is applied. Before this the non-tiered 12h rate and the tiered 1h rate fell back to the rate from before off-peak, so the same request billed differently depending on the pricing path
* refactor(cost): hardcode the cache write TTLs as a Literal instead of parsing usage field names
Parsing ephemeral_*_input_tokens names saved nothing, since a new TTL still needs a usage field and provider parsing code. The TTLs are now a CacheWriteTTL Literal with an explicit rate key per TTL
* refactor(cost): resolve cache write lifetime rates to floats at lookup
Decide the 1h and 12h cost map key in the threshold loop, then resolve each
rate once after off-peak with the base write rate as the default, so no None
flows through the cost code. Add an integration case for a 12h write on a
row without a 12h rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): name every cache write TTL, marking 5m and 30m as base rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): list every cache write TTL in one rate key table, None meaning the base write rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): drop base rate entries from the cache write rate table and rely on fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): key cache write rates by plain string TTL
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): fill missing cache write TTL rates with the base write rate after off-peak
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): default missing cache write TTL rates to the base write rate inline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): rename cache write rate key table to make clear it only covers long TTLs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): list every cache write TTL and bill writes of an unknown TTL at the base write rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): resolve tiered cache write TTL rates after off-peak instead of passing None
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): carry cache write TTL rates on TokenRates and drop TokenBaseCosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): define TokenRates before its first use to drop the string annotation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): build the TokenRates TTL default with default_factory and type tiers as Mapping
MappingProxyType({}) as a dataclass default is rejected on Python 3.11, where
mappingproxy is unhashable, so import litellm failed there. The tier type
moves from dict[str, object] to Mapping[str, object] (LIT001), with tier_rate
accepting a Mapping to match.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): type select_tier_for_input tiers as Mapping so tier_rate callers stay typed
The tier returned as a bare dict made every tier_rate call site a partially
unknown argument once tier_rate took Mapping[str, object]. The range bounds are
now validated with a TypeAdapter instead of indexing an unknown list.
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| anthropic.py | ||
| azure.py | ||
| batch_realtime.py | ||
| bedrock_chat_completions.py | ||
| bedrock_converse.py | ||
| bedrock_mantle.py | ||
| decisions.py | ||
| fireworks_ai.py | ||
| gemini.py | ||
| misc.py | ||
| openai.py | ||
| together_ai.py | ||
| vertex_ai.py | ||
| xai.py | ||