mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-10 08:28:25 +00:00
a7b66dd8b9
* fix(openai): stop stripping cache_control from the caller's messages Copy before removing the field, so the caller's own message objects are left alone. * fix(fireworks_ai): strip cache_control on a copy so a fallback keeps the caller's hints * fix: keep the caller's request intact on more providers and trim extra Claude cache hints under drop_params * fix: drop only the oldest extra cache hints inside one message when trimming to the Claude limit * fix: keep replayed session turns yielding their hints as whole turns while a request's own message keeps its newest hints * fix(cache_control): count only hints that reach Claude and cap hints on OpenAI-compatible Claude routes under drop_params * fix(cache_control): keep the capped Claude messages a list in OpenAI-compatible request bodies * fix(caching): cap Claude hints for string drop_params, a Claude base_model, and Converse tool messages The 4-hint cap now reads drop_params through drops_unsupported_params on chat, Converse, and Anthropic routes, names Claude by base_model on OpenAI-compatible routes, and counts a Converse tool message's hints as the one cache point it sends. * fix(caching): trim Claude cache hints in one wire-order pass and reserve a top-level cache_control on OpenAI-compatible routes Dropping the oldest hint one at a time re-walked the whole request per hint, which took seconds on long tool histories. The cap now drops each message's oldest hints in the order the provider counts them, so a string assistant turn loses its text hint before its tool call hints. OpenAI-compatible Claude routes also keep room for a request-level or extra_body cache_control under drop_params. * test: drop narrating comments and stacked generator clauses from cache_control tests * fix(vertex_ai): put every system message in the Gemini context cache so generateContent never sends it beside cachedContent --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| chat | ||
| completion | ||
| decisions | ||
| embeddings | ||
| evals | ||
| image_generation | ||
| image_variations | ||
| live | ||
| realtime | ||
| responses | ||
| speech | ||
| transcriptions | ||
| vector_store_files | ||
| vector_stores | ||
| videos | ||
| __init__.py | ||
| test_cost_calculation.py | ||
| test_data_residency.py | ||
| test_gpt_reasoning_transformation.py | ||
| test_is_model_gpt_5_model.py | ||
| test_o_series_transformation.py | ||
| test_openai.py | ||
| test_openai_authentication_error.py | ||
| test_openai_common_utils.py | ||
| test_openai_empty_response.py | ||
| test_openai_file_content_streaming.py | ||
| test_openai_image_edit_transformation.py | ||
| test_openai_workload_identity.py | ||
| test_organization_costs.py | ||
| test_use_chat_completions_api_no_leak.py | ||