The changed-test gate booted its stage-mirror stack without files_settings
or finetune_settings, so every raw upload with a custom_llm_provider hit a
500, and it exported the whole provider env into the gateways, so the
AWS_ROLE_NAME the assume-role test needs made the GovCloud deployment run
an AssumeRole with its static keys. The gate also deleted its pytest output,
so a red run left nothing to read. The mirror config now carries the
openai, azure, and vertex_ai file settings, gateways start without
AWS_ROLE_NAME, and the workflow uploads the pass logs and junit files with
every secret value, every field of a JSON-valued secret, and their
XML-escaped forms replaced before the raw files are removed.
Route openai/gpt-image-2.5/{flare,sunburst}/text-to-image through the existing GPT Image config with the xhigh and max quality tiers, add a dedicated fal-ai/flux/dev config, and add a Fal image-edit config so /v1/images/edits works for the gpt-image-2.5 and gpt-image-2 edit endpoints. Add flat and quality-by-size keyed pricing rows so spend is non-zero
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Offload the per-block hashing through offload_token_count on both the pre-call
read and the success-event write, hash raw bytes as base64 instead of raising,
drop the unused serialize_object helper, and bind the chained digest, the
message envelope, and the bytes path in the regression tests
The prompt_caching pre-call check keyed a deployment pin on a hash of the
whole cacheable prefix, cache_control markers included. Agent clients
such as Claude Code move the marker to the newest user turn on every
request, so the key changed every turn, the pin never matched, and a
multi-turn session drifted across deployments and lost its provider
cache.
Hash the prefix per content block with the markers stripped, chained so
every block position has a key, and write the pin at the breakpoint
block. Lookup walks back over the last PROMPT_CACHE_LOOKBACK_POSITIONS
positions (a run of tool_use or tool_result blocks counting as one), the
same window the provider probes for a cached prefix, in one batch cache
read. Both sides hash the prefix after base64 truncation so a request
carrying raw image bytes derives the keys the success event stored.
A tools/call on a cold worker listed the target server once and then never
again, so a later caller whose credentials expose a wider upstream catalog
got 404 for tools the first caller never had. Gate the pre-call listing on
whether this worker already exposes the requested tool, so callers with
different catalogs no longer mask each other. Removing the per-server guard
also drops the empty-listing case that re-listed on every call.
A tools entry of only {"type": "function"} has nothing for the Responses
bridge to convert, and the bridge raised a 500 for it where the chat
route returns the provider's own 400. The gate now counts a tool as a
function tool only when it carries a function body or a top-level name,
on every provider the gate serves
The startup tool-name fill skips servers whose upstream wants the caller's
own token (true_passthrough, OAuth discovery), and mcp 2 no longer runs the
list handler before an uncached tools/call, so every uvicorn worker that had
not served tools/list answered 404 "Tool not found" for prefixed tools/call
and the REST server_id route on those servers.
On a resolution miss, execute_mcp_tool now lists the prefix-matched (or
server_id-requested) server once, with the caller's credentials, through the
existing tools/list path, then resolves as before. Listing failures fall
through to the existing 404, a worker that already listed the server never
re-lists it, and a server outside the caller's allowed set is never listed.
The /v1/messages handler passed the api_base get_llm_provider resolved to every
provider's native messages config, which shadowed DEEPSEEK_ANTHROPIC_API_BASE and
TENCENT_ANTHROPIC_API_BASE with the chat default and changed the azure_ai
precedence. Messages configs now opt in through uses_get_llm_provider_api_base(),
true only for Bedrock Mantle, whose region-prefixed model must resolve to a
region host before the prefix is stripped. Also registers
BedrockMantleAnthropicMessagesConfig in the lazy import registry.
A conversation that replays a redacted_thinking block (Anthropic redacted reasoning, or the
/v1/messages bridge's stand-in for a reasoning item that carries no summary) made
_count_content_list raise, is_prompt_caching_valid_prompt swallowed that to False, and the
prompt_caching pre-call check neither recorded nor pinned the serving deployment, so the
conversation bounced across the group and paid a cache write on every deployment. The block
now counts like a thinking block with no text: zero tokens for the encrypted payload.
