mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-16 23:41:43 +00:00
142d5aa12b
Bedrock Converse supports cachePoint ttl (1h GA for Claude 4.5+), and _get_cache_point_block maps cache_control.ttl -> cachePoint.ttl, but the model parameter its allow-list gate requires was only threaded through the system-message path. Every message-level path either called _get_cache_point_block without model= (8 call sites in _bedrock_converse_messages_pt / _pt_async) or hardcoded CachePointBlock(type="default") (tool-result blocks and _convert_to_bedrock_tool_call_invoke), so a requested 1h ttl silently degraded to the 5-minute default - exactly on the conversation-tail breakpoint that long-running agents need to survive tool calls longer than 5 minutes. - pass model= at the 8 _get_cache_point_block call sites - tool-result blocks: capture the cache_control dict (was a boolean) and route through _get_cache_point_block so ttl survives - _convert_to_bedrock_tool_call_invoke: accept optional model and route per-tool-call cache_control through _get_cache_point_block Completes the ttl support added for system messages (#19848, #20326): message-level cache_control now behaves identically. Note: message-level cache_control on a content-less assistant message emits no cachePoint at all today; that pre-existing gap is orthogonal to ttl and left out of scope (per-tool-call placement covers it). Co-authored-by: Arash <arashne@glia-ai.com> |
||
|---|---|---|
| .. | ||
| agentcore | ||
| invoke_transformations | ||
| test_converse_transformation.py | ||
| test_converse_transformation_nova_2.py | ||
| test_invoke_handler.py | ||
| test_mistral_config.py | ||
| test_service_tier.py | ||
| test_streaming_choice_index.py | ||
| test_writer_palmyra.py | ||