scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-25 01:02:15 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 80

fix(cost): apply regional/service-tier suffix to above-threshold cache keys
Some checks failed
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Details
Unit Tests: Security / security (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Details
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
Details

Browse source 
The above-threshold input/output cost keys were resolved through
_get_service_tier_cost_key (e.g. input_cost_per_token_above_272k_tokens_regional),
but the cache-read/creation above-threshold keys were hardcoded to the
standard form, so regional cache-read pricing above the threshold (e.g.
cache_read_input_token_cost_above_272k_tokens_regional on gpt-5.4/gpt-5.5)
was never consumed.

Resolve cache_creation/cache_creation_1hr/cache_read above-threshold keys
via _get_service_tier_cost_key when service_tier is set, and treat the
standard above-threshold key as the fallback (matching the existing
fallback semantics inside _get_cost_per_unit).

Co-authored-by: Yassin Kortam <yassin@berri.ai>
This commit is contained in:
Cursor Agent 2026-05-17 06:15:36 +00:00
parent cd10c06811
commit ebd580aad8
No known key found for this signature in database
1 changed files with 38 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

45
litellm/litellm_core_utils/llm_cost_calc/utils.py
View file

@ -270,16 +270,41 @@ def _get_token_base_cost(
),
)
# Apply tiered pricing to cache costs
cache_creation_tiered_key = (
# Apply tiered pricing to cache costs. Prefer a
# service_tier-specific above-threshold key when available
# (e.g. cache_read_input_token_cost_above_272k_tokens_regional),
# falling back to the standard above-threshold key.
cache_creation_base_key = (
f"cache_creation_input_token_cost_above_{threshold_str}_tokens"
)
cache_creation_1hr_tiered_key = f"cache_creation_input_token_cost_above_1hr_above_{threshold_str}_tokens"
cache_read_tiered_key = (
cache_creation_1hr_base_key = f"cache_creation_input_token_cost_above_1hr_above_{threshold_str}_tokens"
cache_read_base_key = (
f"cache_read_input_token_cost_above_{threshold_str}_tokens"
)
cache_creation_tiered_key = (
_get_service_tier_cost_key(
cache_creation_base_key, service_tier
)
if service_tier
else cache_creation_base_key
)
cache_creation_1hr_tiered_key = (
_get_service_tier_cost_key(
cache_creation_1hr_base_key, service_tier
)
if service_tier
else cache_creation_1hr_base_key
)
cache_read_tiered_key = (
_get_service_tier_cost_key(cache_read_base_key, service_tier)
if service_tier
else cache_read_base_key
)
if cache_creation_tiered_key in model_info:
if (
cache_creation_tiered_key in model_info
or cache_creation_base_key in model_info
):
cache_creation_cost = cast(
float,
_get_cost_per_unit(
@ -289,7 +314,10 @@ def _get_token_base_cost(
),
)
if cache_creation_1hr_tiered_key in model_info:
if (
cache_creation_1hr_tiered_key in model_info
or cache_creation_1hr_base_key in model_info
):
cache_creation_cost_above_1hr = cast(
float,
_get_cost_per_unit(
@ -299,7 +327,10 @@ def _get_token_base_cost(
),
)
if cache_read_tiered_key in model_info:
if (
cache_read_tiered_key in model_info
or cache_read_base_key in model_info
):
cache_read_cost = cast(
float,
_get_cost_per_unit(