mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-15 09:12:18 +00:00
3ad164382b
* fix(responses bridge): carry cache_control as a prompt_cache_breakpoint to deployments that read them A chat request with Anthropic-style cache_control on a system or user part got its mark renamed to prompt_cache_breakpoint on the plain chat path, but the chat-to-Responses bridge (taken once a function tool and reasoning are on, and always for bedrock_mantle GPT) dropped it, so explicit-mode caching never wrote. The bridge now renames the marks before converting the messages when the served deployment reads breakpoints, using the same gate the Responses config uses for native /v1/responses input. Deployments that do not read breakpoints keep today's behavior. The rename also leaves unmarked content parts untouched instead of validating them, so a part with non-string keys no longer raises on either path. * fix(responses): keep leading system messages in client order once one goes to input A marked first system string became a list and moved to input while the next unmarked string stayed in instructions, so the model read them in reverse. The leading run that joins instructions now stops at the first system message whose content is not a string * fix(responses): leave Bedrock Mantle and unmarked system order as they were on the bridge Bedrock Mantle already caches the growing prefix on its own, so turning cache_control into breakpoints there added write charges. The client-order rule for leading system messages now applies only when a mark turns a system string into a list, so unmarked chats send the same request as before. * test(responses): cover a marked user turn in the unmarked system split test A cache_control mark on the user message must not switch unmarked leading system messages to the client-order rule. The split test now runs with the user turn plain and marked, so dropping the role check from the system string test fails it. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| litellm_responses_transformation | ||
| __init__.py | ||
| test_litellm_responses_transformation_transformation.py | ||
| test_responses_bridge_provider_propagation.py | ||