scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-26 01:12:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 98

fix(bedrock): recalculate total_tokens after cache-token inflation in _transform_usage

Browse source 
Bedrock's totalTokens field reflects only inputTokens + outputTokens (non-cached).
_transform_usage inflated input_tokens by adding cacheReadInputTokens/cacheWriteInputTokens
but never updated total_tokens, violating total_tokens == prompt_tokens + completion_tokens.

Fixes by computing total_tokens = input_tokens + output_tokens after the inflation block,
matching the pattern already used in the Anthropic transformation.
This commit is contained in:
Faizzyhon 2026-05-18 04:28:17 -07:00 • committed by GitHub
parent cf9b5e4fa7
commit d2232a1867
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 8 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
litellm/llms/bedrock/chat/converse_transformation.py
View file

@ -1714,7 +1714,6 @@ class AmazonConverseConfig(BaseConfig):
) -> Usage:
input_tokens = usage["inputTokens"]
output_tokens = usage["outputTokens"]
total_tokens = usage["totalTokens"]
cache_creation_input_tokens: int = 0
cache_read_input_tokens: int = 0
@ -1726,6 +1725,14 @@ class AmazonConverseConfig(BaseConfig):
cache_creation_input_tokens = usage["cacheWriteInputTokens"]
input_tokens += cache_creation_input_tokens
# Recalculate total_tokens AFTER inflating input_tokens with cache tokens,
# so that total_tokens == prompt_tokens + completion_tokens.
# Bedrock's native "totalTokens" only covers inputTokens + outputTokens
# (i.e. non-cached tokens), so it would be too low whenever cache tokens
# are present. Mirroring what the Anthropic transformation already does
# (see litellm/llms/anthropic/chat/transformation.py).
total_tokens = input_tokens + output_tokens
prompt_tokens_details = PromptTokensDetailsWrapper(
cached_tokens=cache_read_input_tokens,
cache_creation_tokens=cache_creation_input_tokens,