fix: retry transient 'thinking blocks cannot be modified' 400s from Bedrock

Bedrock occasionally returns HTTP 400 claiming the assistant's thinking
blocks have been modified, even when the payload is structurally valid.
The same payload replayed immediately succeeds, indicating a transient
server-side condition rather than a client bug.

Observed on bedrock/us.anthropic.claude-sonnet-4-6 with adaptive thinking
(reasoning_effort=high maps to {type: "adaptive"} on claude-4-6). The
error message is:

  messages.N.content.M: `thinking` or `redacted_thinking` blocks in the
  latest assistant message cannot be modified. These blocks must remain
  as they were in the original response.

Reproduced during a large-scale multi-scan study (800+ Strix runs). Most
failures clear within seconds but we observed one case where the error
persisted across ~2 minutes of backoff, so the retry budget allows up to
~5 minutes total.

Fix:
- Add _is_transient_thinking_error() helper that matches on the status
  code (400) AND the characteristic message ("thinking" + "cannot be
  modified"), to avoid treating unrelated 400s as transient.
- On detection, retry with exponential backoff (5s, 10s, 20s, 40s, 80s,
  160s — total ~5 min) before falling through to the generic retry path.
- Self-contained: the detector checks the exception directly, so it works
  with or without other 400 handlers in place.

Testing:
- Replayed 33 captured payloads that appeared adjacent to a failure: all
  succeeded, confirming the condition is transient and not a client-side
  malformation.
- In a multi-repo scan study, 3-retry budget was insufficient in one case
  (exhausted retries over ~30s). The 6-retry budget implemented here
  succeeded on the same repo on a subsequent attempt.
This commit is contained in:
Sean Turner 2026-04-27 19:01:38 +01:00
parent 9fb101282f
commit 6406a4d6e1
No known key found for this signature in database

View file

@ -159,12 +159,26 @@ class LLM:
messages = self._prepare_messages(conversation_history)
max_retries = int(Config.get("strix_llm_max_retries") or "5")
transient_thinking_retries = 0
for attempt in range(max_retries + 1):
try:
async for response in self._stream(messages):
yield response
return # noqa: TRY300
except Exception as e: # noqa: BLE001
# Bedrock occasionally returns a 400 claiming the assistant's thinking
# blocks have been modified — even when the payload is structurally
# valid and the same payload succeeds on replay. Observed on
# claude-sonnet-4-6 with adaptive thinking. Retry with exponential
# backoff (5s, 10s, 20s, 40s, 80s, 160s — ~5 min total budget) before
# falling through to the generic retry path.
if self._is_transient_thinking_error(e) and transient_thinking_retries < 6:
transient_thinking_retries += 1
if attempt >= max_retries:
self._raise_error(e)
await asyncio.sleep(min(240, 5 * (2 ** (transient_thinking_retries - 1))))
continue
if attempt >= max_retries or not self._should_retry(e):
self._raise_error(e)
wait = min(90, 2 * (2**attempt))
@ -323,6 +337,22 @@ class LLM:
except Exception: # noqa: BLE001
return 0.0
@staticmethod
def _is_transient_thinking_error(e: Exception) -> bool:
"""Detect Bedrock's transient 'thinking blocks cannot be modified' 400.
Observed on claude-sonnet-4-6 with adaptive thinking: Bedrock occasionally
rejects a well-formed payload with this error, and the identical payload
succeeds on replay. Treat it as transient rather than a structural issue.
"""
code = getattr(e, "status_code", None) or getattr(
getattr(e, "response", None), "status_code", None
)
if code != 400:
return False
message = str(e).lower()
return "thinking" in message and "cannot be modified" in message
def _should_retry(self, e: Exception) -> bool:
code = getattr(e, "status_code", None) or getattr(
getattr(e, "response", None), "status_code", None