mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-10 03:28:53 +00:00
c5eb22381d
Fixes #15263 This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching. **Root Cause:** PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens` for Bedrock, based on the assumption that it would cause double counting (similar to an Anthropic API issue). However, Bedrock's token structure is different: - **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens` are ALL separate values that should be summed for total input tokens - **Anthropic API**: Same structure - all three token types are separate The fix in #9838 was later reverted for Anthropic (correctly re-adding `cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed. **Changes:** 1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation 2. Update test assertions to reflect correct behavior 3. Add regression test for prompt caching cost calculation 4. Fix typo in Anthropic transformation where `cache_creation_tokens` was incorrectly set to `cache_read_input_tokens` **Testing:** - All existing Bedrock transformation tests pass - New test validates correct cost calculation with prompt caching - Verified costs are non-negative and accurate |
||
|---|---|---|
| .. | ||
| chat | ||
| count_tokens | ||
| embed | ||
| files | ||
| image | ||
| invoke_agent | ||
| messages/invoke_transformations | ||
| passthrough | ||
| rerank | ||
| vector_stores | ||
| test_anthropic_beta_support.py | ||
| test_base_aws_llm.py | ||
| test_bedrock_common_utils.py | ||
| test_cross_region_inference_profile_mapping.py | ||