The prompt_caching pre-call check keyed a deployment pin on a hash of the
whole cacheable prefix, cache_control markers included. Agent clients
such as Claude Code move the marker to the newest user turn on every
request, so the key changed every turn, the pin never matched, and a
multi-turn session drifted across deployments and lost its provider
cache.
Hash the prefix per content block with the markers stripped, chained so
every block position has a key, and write the pin at the breakpoint
block. Lookup walks back over the last PROMPT_CACHE_LOOKBACK_POSITIONS
positions (a run of tool_use or tool_result blocks counting as one), the
same window the provider probes for a cached prefix, in one batch cache
read. Both sides hash the prefix after base64 truncation so a request
carrying raw image bytes derives the keys the success event stored.
A tools/call on a cold worker listed the target server once and then never
again, so a later caller whose credentials expose a wider upstream catalog
got 404 for tools the first caller never had. Gate the pre-call listing on
whether this worker already exposes the requested tool, so callers with
different catalogs no longer mask each other. Removing the per-server guard
also drops the empty-listing case that re-listed on every call.
A tools entry of only {"type": "function"} has nothing for the Responses
bridge to convert, and the bridge raised a 500 for it where the chat
route returns the provider's own 400. The gate now counts a tool as a
function tool only when it carries a function body or a top-level name,
on every provider the gate serves
The startup tool-name fill skips servers whose upstream wants the caller's
own token (true_passthrough, OAuth discovery), and mcp 2 no longer runs the
list handler before an uncached tools/call, so every uvicorn worker that had
not served tools/list answered 404 "Tool not found" for prefixed tools/call
and the REST server_id route on those servers.
On a resolution miss, execute_mcp_tool now lists the prefix-matched (or
server_id-requested) server once, with the caller's credentials, through the
existing tools/list path, then resolves as before. Listing failures fall
through to the existing 404, a worker that already listed the server never
re-lists it, and a server outside the caller's allowed set is never listed.
The /v1/messages handler passed the api_base get_llm_provider resolved to every
provider's native messages config, which shadowed DEEPSEEK_ANTHROPIC_API_BASE and
TENCENT_ANTHROPIC_API_BASE with the chat default and changed the azure_ai
precedence. Messages configs now opt in through uses_get_llm_provider_api_base(),
true only for Bedrock Mantle, whose region-prefixed model must resolve to a
region host before the prefix is stripped. Also registers
BedrockMantleAnthropicMessagesConfig in the lazy import registry.
A conversation that replays a redacted_thinking block (Anthropic redacted reasoning, or the
/v1/messages bridge's stand-in for a reasoning item that carries no summary) made
_count_content_list raise, is_prompt_caching_valid_prompt swallowed that to False, and the
prompt_caching pre-call check neither recorded nor pinned the serving deployment, so the
conversation bounced across the group and paid a cache write on every deployment. The block
now counts like a thinking block with no text: zero tokens for the encrypted payload.
Foundry's OpenAI v1 chat surface rejects function tools with an explicit
reasoning_effort from gpt-5.6 on and with reasoning left on from gpt-6 on,
while gpt-5.4, gpt-5.5 and unset-effort gpt-5.6 serve them. Key the
azure_ai bridge on those measured boundaries instead of the azure
provider's gpt-5.4+ rule so working chat traffic keeps its n, logprobs,
seed and chatcmpl ids.
A string such as "*" or "..." has no base64 characters, so it decoded to no bytes and read as an empty plaintext under any key. The migration would have counted it and overwritten it with a ciphertext of the empty string. Also read from the writer database instead of a read replica, report a database error during the migration instead of crashing the boot, skip columns the connected schema lacks across every schema on the search path, cap the JSON walk depth for the recursion detector, and move the boot wiring into one tested function.
A finished batch reports its whole cost on every poll. The charge-once
marker is now taken with one atomic increment on the shared cache, so two
workers polling the same batch at once cannot both charge it, and the
marker's TTL is refreshed on every poll so a batch polled within every
budget window is never charged again after the marker's first expiry.
Rotating through POST /key/regenerate needs a running proxy, which a refused boot does not have. The refusal now counts the stored values that decrypt under the unsafe key. When there are none it only asks for a new key. When there are some it also asks for LITELLM_MIGRATE_FROM_MASTER_KEY, and the next boot with a safe key re-encrypts them and logs that the variable can be deleted. Leaving the variable set afterwards is a no-op with one notice.
