mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-02 07:13:02 +00:00
a7b66dd8b9
* fix(openai): stop stripping cache_control from the caller's messages Copy before removing the field, so the caller's own message objects are left alone. * fix(fireworks_ai): strip cache_control on a copy so a fallback keeps the caller's hints * fix: keep the caller's request intact on more providers and trim extra Claude cache hints under drop_params * fix: drop only the oldest extra cache hints inside one message when trimming to the Claude limit * fix: keep replayed session turns yielding their hints as whole turns while a request's own message keeps its newest hints * fix(cache_control): count only hints that reach Claude and cap hints on OpenAI-compatible Claude routes under drop_params * fix(cache_control): keep the capped Claude messages a list in OpenAI-compatible request bodies * fix(caching): cap Claude hints for string drop_params, a Claude base_model, and Converse tool messages The 4-hint cap now reads drop_params through drops_unsupported_params on chat, Converse, and Anthropic routes, names Claude by base_model on OpenAI-compatible routes, and counts a Converse tool message's hints as the one cache point it sends. * fix(caching): trim Claude cache hints in one wire-order pass and reserve a top-level cache_control on OpenAI-compatible routes Dropping the oldest hint one at a time re-walked the whole request per hint, which took seconds on long tool histories. The cap now drops each message's oldest hints in the order the provider counts them, so a string assistant turn loses its text hint before its tool call hints. OpenAI-compatible Claude routes also keep room for a request-level or extra_body cache_control under drop_params. * test: drop narrating comments and stacked generator clauses from cache_control tests * fix(vertex_ai): put every system message in the Gemini context cache so generateContent never sends it beside cachedContent --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agent | ||
| chat | ||
| decisions | ||
| messages | ||
| responses | ||
| __init__.py | ||
| test_databricks_common_utils.py | ||
| test_databricks_cost_calculator.py | ||
| test_databricks_partner_integration.py | ||
| test_databricks_streaming_utils.py | ||