mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-13 03:59:12 +00:00
9d3bf29d6d
* fix(responses): keep cache breakpoints on requests that reach the Responses API The chat-completions bridge rebuilt every content block without the prompt_cache_breakpoint marker the cache-control hook had just placed on it, so the request went out with prompt_cache_options set to explicit mode and nothing actually marked. Explicit mode caches only what is marked, so those deployments lost the implicit caching they were getting before the injection point was added The native surface had a second miss. The bridging check looked the provider config up with the still-prefixed model while the config keys on the bare one, so every bedrock_mantle model read as having no native Responses support, and role-targeted injection points were deferred to a chat-completions pass that never runs for them * fix(responses): keep prompt cache breakpoint on stringified content blocks * test: cover the responses/ routing prefix in the native cache point case * fix(responses): keep only the fallback-branch marker carry, main already routes the hook #42281 landed the text, image_url and file branch carry and the implicit default on main, and the Responses routing lookup this branch changed has no observable effect there, so the PR shrinks to the unknown-block fallback branch and its regression test * fix(responses): drop a malformed prompt_cache_breakpoint under drop_params in the chat-to-Responses bridge * fix(responses): accept the 30m prompt_cache_breakpoint ttl under drop_params OpenAI's Responses API takes a ttl of 30m on an explicit breakpoint marker, so the drop_params validation keeps it instead of stripping it as an unknown key. Trims the bridge test docstrings to the dated vendor citation. * test(integration): audit cells for prompt_cache_breakpoint through the chat-to-Responses bridge Adds the /audit cells for the bridge's prompt_cache_breakpoint carry: valid and malformed markers on every carrying block kind and on tool and assistant content, the drop_params on and off contract, the hook-injected system marker, the Anthropic SDK path through the Responses adapter, upstream errors, idempotent spend rows, and the chaos cells (mixed burst, upstream outage, slow streams, worker SIGKILL and a proxy restart mid burst), all against a scripted Responses endpoint with the request it received read back by response id * test(integration): give every scripted responses reply its own id The prompt-cache-breakpoint audit's scripted upstream minted the response id from the request's marker, so three identical marked requests shared one id. The spend-log writer skips rows whose request_id already landed, and the idempotent-logging cell saw one row for three calls on every leg. The upstream now mints a unique id per reply like a real provider, and the cells match a response to its request by the marker inside that id. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| agentic_probe.py | ||
| anthropic_sse.py | ||
| anthropic_thinking.py | ||
| asgi.py | ||
| bedrock_runtime_peer.py | ||
| browser_state.py | ||
| claude_code.py | ||
| client.py | ||
| codex_vendor.py | ||
| daily_activity.py | ||
| daily_spend_rows.py | ||
| database.py | ||
| database_relay.py | ||
| device_login.py | ||
| generation.py | ||
| mail.py | ||
| manifest.py | ||
| mcp.py | ||
| mcp_grants.py | ||
| mcp_stdio_peer.py | ||
| oauth_server.py | ||
| openai_wire.py | ||
| otlp_sink.py | ||
| process.py | ||
| prometheus_series.py | ||
| prompt_cache_breakpoint.py | ||
| provider.py | ||
| proxy.py | ||
| redis_process.py | ||
| responses_stream.py | ||
| responses_vendor.py | ||
| routing.py | ||
| sigv4.py | ||
| tls.py | ||
| tool_rows.py | ||
| upstream.py | ||
| vertex.py | ||
| wire.py | ||