The prompt_caching pre-call check keyed a deployment pin on a hash of the
whole cacheable prefix, cache_control markers included. Agent clients
such as Claude Code move the marker to the newest user turn on every
request, so the key changed every turn, the pin never matched, and a
multi-turn session drifted across deployments and lost its provider
cache.
Hash the prefix per content block with the markers stripped, chained so
every block position has a key, and write the pin at the breakpoint
block. Lookup walks back over the last PROMPT_CACHE_LOOKBACK_POSITIONS
positions (a run of tool_use or tool_result blocks counting as one), the
same window the provider probes for a cached prefix, in one batch cache
read. Both sides hash the prefix after base64 truncation so a request
carrying raw image bytes derives the keys the success event stored.
AGENTS.md allows a vendor-owned literal only when its source and date are cited
next to it. A live realtime test cannot avoid naming a model, so record how the
id was checked, and record that a retired id fails as a hang rather than an
error so the next reader does not start by suspecting litellm.
AWS retired amazon.nova-sonic-v1:0. GetFoundationModel now answers
ResourceNotFoundException "This model version has reached the end of its life",
and opening a bidirectional stream against it fails with ValidationException
"The provided model identifier is invalid". amazon.nova-2-sonic-v1:0 is the
active replacement.
The test passed on builds 219 (2026-09-16) and 246 (2026-09-17) and has failed
every run since, three attempts per build, with no litellm change to the
realtime path in between. The symptom was a clean websocket close: Bedrock ends
the stream rather than erroring, the forwarder treats a None receive as a normal
stream end and closes the client socket, so the client sees ConnectionClosedOK
and the test fails waiting for response.done.
litellm already carries both models in the cost map, with
"deprecation_date": "2026-09-14" on the old one, and the promptStart
transformation already sends the audioOutputConfiguration that nova-2-sonic
requires; only the test constant was left behind.
Verified against live Bedrock with the promptStart shape the transformation
builds: amazon.nova-sonic-v1:0 raises "The provided model identifier is
invalid", amazon.nova-2-sonic-v1:0 opens a session and returns a usageEvent.
The mocked handler and provider-cache tests keep the old id: it is only a label
there, no call reaches AWS.
The stored flag is the cluster's desired state and is what the form saves, so it
cannot also stand as proof that this process activated the callback: a pod that
lagged or failed to apply it would still read as on. Report the process's own
registration as a separate read-only field and warn on the page when the two
disagree, so a failed activation is visible instead of only logged.
Deriving enabled from whether this process has the callback registered makes a
pod that has not polled yet report off while the cluster runs it, and the next
save writes that off back for every pod. The stored flag is the cluster's own
answer, so prefer it and fall back to local registration only when none is
stored, which is the config-activated case that has no flag to read.
The member add, delete and role-change audit rows were awaited on the
request path, so a slow audit sink held the response, and the roster was
serialized before checking whether audit logging is enabled at all.
Membership audit work is now scheduled after the transaction commits and
skipped outright when auditing is off.
member_update read the roster outside the team advisory lock and wrote
it back, so a concurrent add or delete could be lost. It now takes the
lock, rereads the roster, and builds the before and after snapshots from
that read.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
