compress() scores text-only copies of the rows, so a content-part cache_control marker was gone by the time get_protected_indices ran and the pinned row could still be stubbed. Read protection from the original rows, which are index-aligned with the normalized copies, and add a regression test that fails without the change.
get_protected_indices() only protected system rows, the last user row,
and the last assistant row. A message carrying its own Anthropic
cache_control breakpoint further back in history (e.g. a large cached
tool result from a few turns ago) was not protected, so the Headroom
guardrail would send it to /v1/compress and rewrite it.
The row came back byte-different but kept its cache_control marker, so
the provider's prompt cache treated the next request as a miss on that
prefix: a cache read silently became a cache write. This reproduces the
production cache-hit-rate collapse reported in #39519 (~65-70% down to
~40-50% within 48h of enabling the guardrail).
get_protected_indices() now also protects any message whose content --
directly on the message, or on any part of a list-of-parts content --
carries a cache_control marker, regardless of its position in history.
Both compress() and the Headroom guardrail already share this function
as their compression-eligibility policy, so both get the fix.
Adds test coverage for cache_control on the message dict itself, on a
content part, mid-history, and de-duplicated against already-protected
indices. Updates the Headroom guardrail's PARTS_MESSAGES fixture, which
previously relied on this exact gap for its all-text merge/flatten test
coverage, to use a separate un-marked row (the cache_control-marked-row
merge behavior is covered directly by compresr's own test, since a
marked row no longer reaches that merge path through Headroom).
Fixes#39519
The Headroom guardrail sent every message to /v1/compress, including the
system prompt and the user's current instruction. On an agentic /v1/messages
request the live turn is the largest compressible blob, so it came back as a
hash marker; the model then called headroom_retrieve and got its own
instruction returned in a tool_result block, which reads as data it fetched
rather than a request to act on, so it described the content instead of doing
the work.
litellm already owns the policy for what a compressor may never rewrite:
get_protected_indices covers the system rows, the last user row and the last
assistant row, and compress() expands it over whole tool exchanges. Headroom
now consults it (promoted from a private name and given tests) and expands it
the same way, so the trailing tool result cannot come back as a marker
standing in for the result of the call the model just made. Protected rows are
withheld from the payload rather than pinned afterwards, so their tokens are
not reported as savings that are never applied; the write-back discards a
compressed system prompt outright, so that saving never existed. The cost is
that a query-aware service no longer sees the newest user message.
A response whose row count differs from what was sent can no longer be
interleaved with the withheld rows, so it goes through the configured fail
policy instead of being adopted. Fail-open now returns the caller's own inputs
object: translation handlers detect a rewrite by identity, so a rebuilt copy
sent an unchanged request through the Anthropic write-back for nothing.
That write-back rebuilt the request with one anthropic_messages_pt call, which
merges every run of consecutive user/tool rows, so a tool_result turn and the
user turn after it arrived fused. Converting a row at a time would separate
them but breaks tool pairing: with modify_params on, an assistant row whose
results are converted separately reads as an orphaned tool call and the
sanitizer answers it with a synthetic "tool execution skipped" result while
dropping the real one. Conversion is now grouped by tool_call_id ownership,
which satisfies both, and the same grouping decides which rows headroom
protects, so the two agree by construction.
The CCR follow-up also dropped any text the model wrote alongside its tool
call, and echoed tool calls it had no results for. Both are fixed by reusing
compresr's extraction helper, now shared instead of duplicated.
Resolves LIT-5018
