scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-16 23:41:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102

Fix langfuse input tokens logic for cached tokens (#16203)

Browse source 
* Update langfuse.py

Fixing issue with input_tokens and cache_read_tokens

* Clarify input token calculation in langfuse.py

Add comment to clarify input token calculation based on Langfuse documentation.
This commit is contained in:
Fabrício Ceschin 2025-11-04 22:20:35 -05:00 • committed by GitHub
parent 727fe504bb
commit 387982fcbf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 7 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
litellm/integrations/langfuse/langfuse.py
View file

@ -688,16 +688,19 @@ class LangFuseLogger:
"completion_tokens": _usage_obj.completion_tokens,
"total_cost": cost if self._supports_costs() else None,
}
cache_read_input_tokens = _usage_obj.get(
"cache_read_input_tokens", 0
)
# According to langfuse documentation: "the input value must be reduced by the number of cache_read_input_tokens"
input_tokens = _usage_obj.prompt_tokens - cache_read_input_tokens
usage_details = LangfuseUsageDetails(
input=_usage_obj.prompt_tokens,
input=input_tokens,
output=_usage_obj.completion_tokens,
total=_usage_obj.total_tokens,
cache_creation_input_tokens=_usage_obj.get(
"cache_creation_input_tokens", 0
),
cache_read_input_tokens=_usage_obj.get(
"cache_read_input_tokens", 0
),
cache_read_input_tokens=cache_read_input_tokens,
)
generation_name = clean_metadata.pop("generation_name", None)