mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-11 08:38:24 +00:00
a2ab27373f
* fix(proxy): count the context_management edits /v1/messages applies on every bridged provider * fix: apply context_management edits in count_tokens on Responses bridges, compaction blocks and Bedrock Claude The Responses bridge now applies the same edits inference applies for every provider, the chat bridge cuts the history at a client compaction block, and Bedrock Claude forwards the edits AWS accepts with their beta to the mantle and runtime count APIs instead of dropping them. * refactor(count_tokens): return count-view failures as values and raise them once in the caller * fix(bedrock): keep the runtime's 400 when mantle does not serve the model it retries * fix(count_tokens): forward context edits on Vertex and Azure Claude counts and keep Bedrock's mantle answer without edits * fix(proxy): count edited bodies on native routes without a count API, keep counts on editor failures * fix(proxy): keep the Bedrock pass-through count shape and provider 400s on empty bridged bodies The pass-through count picks Converse or InvokeModel from the content again, and the edited InvokeModel count wraps string turns in text blocks. A bridged view with no messages sends the body as before. Azure and Vertex load the beta config off the event loop before a count with edits, and a model with no deployment reads drop_params from the request and router defaults. * fix(bedrock): send an empty-string turn as is when a count forwards context edits Wrapping an empty string into a text block made AWS reject the count with 'text content blocks must be non-empty', while base and /v1/messages accept the empty string. Only non-empty string turns are wrapped now. * test(azure_ai): compare the count's betas with the betas Azure inference sends * test(azure_ai): record the Azure inference request with respx instead of appending to a list --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agents | ||
| anthropic | ||
| chat | ||
| claude | ||
| embed | ||
| image_edit | ||
| image_generation | ||
| passthrough | ||
| rerank | ||
| responses | ||
| vector_stores | ||
| __init__.py | ||
| test_azure_ai_agents_handler.py | ||
| test_azure_ai_cost_calculator.py | ||
| test_azure_ai_entra_auth.py | ||
| test_azure_ai_foundry_catalog_model_metadata.py | ||
| test_azure_ai_fw_models_metadata.py | ||
| test_azure_ai_kimi_k26_metadata.py | ||