mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-13 08:52:02 +00:00
56388fd13d
* fix(count_tokens): count a compaction block instead of answering 500 A message history that replays Anthropic's signed compaction block made /v1/messages/count_tokens answer HTTP 500: neither the Anthropic nor the Vertex AI partner count-tokens request carried the compact-2026-09-04 beta, the provider rejected the block with a 400, the proxy fell back to the local tokenizer, and that raised on the unknown block type. Both provider count paths now add the compaction beta whenever the history carries a compaction block, and the local tokenizer counts the block's summary text like a text block, skipping the signature. * fix(token_counter): skip the tokenizer for empty thinking and compaction text * fix(count_tokens): forward context_management on Anthropic counts and count chat compaction blocks locally Vertex AI compaction support for inference and counting moves to its own PR, so the Vertex count handler is back to main here * fix(anthropic): map list-form context_management to edits for count_tokens and inference betas Count requests forwarded the OpenAI list form of context_management unchanged, while inference maps it to Anthropic edits, and the inference beta helper read the list as no edits. Both now map it once. A replayed compaction block with an empty summary no longer reaches the tokenizer. * fix(anthropic): forward count context_management only with edits, map Vertex list-form betas, and return provider context_management * fix(anthropic): drop an unmappable context_management list under drop_params --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| batches | ||
| chat | ||
| files | ||
| messages | ||
| pass_through | ||
| __init__.py | ||
| anthropic_docs_refusal.py | ||
| test_anthropic_common_utils.py | ||
| test_anthropic_count_tokens_transformation.py | ||
| test_anthropic_files_and_batches.py | ||
| test_anthropic_output_format_filter.py | ||
| test_anthropic_prompt_cache_prediction.py | ||
| test_anthropic_prompt_cache_pricing.py | ||
| test_anthropic_reasoning_effort.py | ||
| test_anthropic_schema_filter.py | ||
| test_anthropic_structured_output.py | ||
| test_anthropic_wif.py | ||
| test_azure_ai_cache_pricing.py | ||
| test_cost_calculation_dict_safety.py | ||
| test_count_tokens_oauth.py | ||
| test_message_sanitization.py | ||