* fix(cohere): map Cohere v2 finish_reason on non-streaming chat responses
* fix(cohere): default a missing Cohere v2 finish_reason to stop instead of raising
A 200 body without finish_reason used to return stop and started raising KeyError after the mapping change, so fall back to COMPLETE
* fix(count_tokens): count grok-4-1, grok-4.3 and DeepSeek V4.1 tool requests like their chat templates
Render grok-4-1, grok-4.3 and DeepSeek V4.1 tool schemas the way each chat
template does, and add a per-deployment token_count_tool_framing next to
token_count_framing for the hidden tool preamble a host bills. The Azure AI
Foundry grok-4-1, xAI grok-4.3 and Fireworks DeepSeek V4.1 cost map rows carry
the measured framing, overridable in model_info
* fix(count_tokens): count Azure grok-4-1 Responses requests with their own framing and use the tool framing in place of the base framing
* fix(count_tokens): keep operator framing on tool and Responses counts, frame bridged Responses like chat, follow base_model for tool rendering, count grok tool_choice
* fix(token_counter): count Azure Responses strict default, skip false strict and built-in-only tools, per-kind operator framing
* fix(token_counter): count an explicit null strict like false on Azure Responses
* fix(count_tokens): follow the chat completions bridge flag and Azure's strict default on azure/ deployments
Keep Azure grok-4-1 chat tool counts at or over the bill by raising its tool framing to 302
* fix(token_counter): keep strict false for non-grok templates, coerce the bridge flag, frame DeepSeek API counts
* fix(proxy): frame Responses counts for the api_base router defaults send inference to
* fix(proxy): keep the api_base helper's name so main's /v1/messages count caller resolves
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): resolve Invoke Claude default max_tokens from the base model id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(bedrock): build Invoke Claude request params without mutating
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): type tool-search normalizer return to clear unknown-type gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): resolve Invoke and Mantle Claude default max_tokens from the deployment id, ARN base_model, and Mantle row
* fix(bedrock): default Invoke Claude max_tokens from the deployment's own cap and honor AnthropicConfig
Read the called deployment's model_info.max_output_tokens before the base model's
cost row, skip the shared regional key the router fills from every deployment, and
leave the default to litellm.AnthropicConfig(max_tokens) when it is set.
* fix(bedrock): send the mapped default max_tokens on bedrock_mantle Claude chat
* fix(bedrock): bound a global AnthropicConfig max_tokens by the deployment's own cap
* test(bedrock): annotate the new Mantle default max_tokens tests with Final
* refactor(bedrock): inject the AnthropicConfig max_tokens reader into the Invoke Claude config
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(databricks): join plain text blocks for non-Claude json_object requests
* fix(databricks): join only the last message for json_object and drop cache_control under drop_params
Databricks checks for "json" only in the last message, so earlier turns now go out as sent and only the
last message, plus any system messages merged into it, is joined into a string. Under drop_params a
text block's cache_control is dropped so that message can be joined too, and without it the request
still goes out as a list.
* fix(databricks): skip dropped assistant turns and blank parts when joining for json_object
The json_object join picked the last message before the request transform drops bare assistant turns,
so a trailing thinking-only or reasoning-only turn (a Responses input ending in a reasoning item included)
left the real last user message as a list and Databricks still returned its 400. The pick now skips the
turns that transform drops.
The join also ran when only one part had text, which sent whitespace-only parts the existing empty block
filter used to drop on requests that already worked. It now joins only two or more non-blank parts, so a
request with one non-blank part goes out exactly as before. Without drop_params, a message with a
message-level cache_control now goes out as sent instead of being rewritten into one cached block.
* fix(databricks): join json_object text parts only on models Databricks hosts
Databricks checks for the word "json" only on the models it hosts
(databricks-* pay per token names and system.ai.* AI Gateway names). An
external model endpoint skips that check and takes a list of text parts
today, so its request now goes out as sent instead of joined.
The drop_params read now goes through drops_unsupported_params, and a
test pins that a text block with a truthy non-string text is kept.
* fix(databricks): reject a text part holding a non-string locally with a clear 400
* refactor(databricks): reject a non-string text part at the transform_request boundary
* fix(databricks): read pydantic message objects in the non-string text check
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(fireworks): keep developer items in place for models that accept a later system message
* fix(fireworks_ai): keep developer items in place for short Fireworks model names and stop flagging gpt-oss-120b
* fix(fireworks): drop the mid-conversation system flag from nemotron-lightning
nemotron-lightning-3p5-30b-a3b followed a later instruction less often with
the developer item kept in place than folded, so both of its cost map keys
go back to folding.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): count the context_management edits /v1/messages applies on every bridged provider
* fix: apply context_management edits in count_tokens on Responses bridges, compaction blocks and Bedrock Claude
The Responses bridge now applies the same edits inference applies for every
provider, the chat bridge cuts the history at a client compaction block, and
Bedrock Claude forwards the edits AWS accepts with their beta to the mantle
and runtime count APIs instead of dropping them.
* refactor(count_tokens): return count-view failures as values and raise them once in the caller
* fix(bedrock): keep the runtime's 400 when mantle does not serve the model it retries
* fix(count_tokens): forward context edits on Vertex and Azure Claude counts and keep Bedrock's mantle answer without edits
* fix(proxy): count edited bodies on native routes without a count API, keep counts on editor failures
* fix(proxy): keep the Bedrock pass-through count shape and provider 400s on empty bridged bodies
The pass-through count picks Converse or InvokeModel from the content again, and the edited InvokeModel count wraps string turns in text blocks. A bridged view with no messages sends the body as before. Azure and Vertex load the beta config off the event loop before a count with edits, and a model with no deployment reads drop_params from the request and router defaults.
* fix(bedrock): send an empty-string turn as is when a count forwards context edits
Wrapping an empty string into a text block made AWS reject the count with 'text content blocks must be non-empty', while base and /v1/messages accept the empty string. Only non-empty string turns are wrapped now.
* test(azure_ai): compare the count's betas with the betas Azure inference sends
* test(azure_ai): record the Azure inference request with respx instead of appending to a list
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(model-catalog): declare the tool and Responses token count framing keys
* fix(model-catalog): read token count framings as strict integers
The cost map guard read the framing keys as lax counts, so a text "214"
or a true passed while litellm's reader drops both and falls back to the
plain framing. Read all four framing keys as plain u64, matching the reader
* fix(model-catalog): accept whole-float token count framings the way the Python reader does
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): price Web Search on the GPT-6 Mantle rows and the GovCloud West Mantle rows
* fix(bedrock): price Web Search on the GPT-5.6 cyber and daybreak-blue Mantle rows
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(openai): trim a malformed prompt_cache_breakpoint under drop_params on chat completions
* fix(openai): leave a non-object content part for OpenAI to reject under drop_params
* fix(openai): trim an unsupported prompt_cache_breakpoint ttl on every path and on azure_ai chat under drop_params
* fix: trim prompt_cache_breakpoint to what Azure and Mantle accept under drop_params
Azure OpenAI chat now gets the same content part trim as OpenAI chat, and
the Azure and Foundry gates also read a deployment's declared base_model.
Each Responses config keeps only the ttl its API takes: Azure keeps 30m,
and Mantle keeps no ttl. A trimmed ttl is logged at debug.
* fix(responses): trim prompt_cache_breakpoint only on APIs known to read it
Other Responses providers get the marker whole or not at all, as on main.
Azure chat now trims for a deployment with no base_model unless the cost
map marks the model as not reading breakpoints, and Azure AI Foundry
Responses keeps Azure's 30m-only shape.
* fix(azure): trim prompt cache breakpoints only for deployments known to read them
Azure and Azure AI Responses keep main's whole-or-nothing rule for unmapped deployment names, an opted-in custom OpenAI host trims the ttl and keeps the mode, and Azure chat strips every marker for a known non-reader under drop_params
* fix(providers): keep main's marker handling where the target won't trim it
A /v1/responses model with unknown breakpoint support on OpenAI's host gets a marker whole or not at all, as on main. The chat to Responses bridge keeps main's whole-marker check when a non-bool global drop_params leaves the target's trim off. A hyphenated name like gpt-5-6-sol now reads as gpt-5.6.
* fix(providers): read a non-version hyphen suffix like -32k as part of the model name
* fix(providers): keep main's reading of a dotted minor version like gpt-5.6o
* fix(providers): keep main's breakpoint shape for unmapped Mantle models and skip the Azure lookup without drop_params
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): keep provider fields on streamed chat completion chunks
The hand-written serializer for plain text chunks dropped every field it did not list, so Azure's per-choice content_filter_results, the chunk's obfuscation, and delta reasoning_details and reasoning_items never reached clients since v1.86.0. Every chunk now serializes with model_dump_json, which keeps them and is faster per chunk.
* fix(proxy): send streamed chunk frames as bytes so plain text frames keep their type
* fix(proxy): send null for chat chunk values JSON cannot encode instead of failing the stream
* fix(proxy): drop chat chunk fields JSON cannot encode instead of failing the stream
A callback or provider value pydantic cannot encode, like invalid UTF-8 bytes, a
lone surrogate, a circular dict, or an unknown type, now leaves only that field out
of the frame with a warning naming it. Responses API events keep raising.
* fix(proxy): drop only the nested chunk value JSON cannot encode and keep lone surrogates
Main removed the StreamingChoices import, so the field drop helpers now take
any BaseModel. A value JSON cannot encode is left out down to the nested key or
list item that holds it, and a lone surrogate goes out as an ASCII JSON escape.
* fix(proxy): keep streaming when an unencodable value sits under non-string dict keys
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): cover per-frame model checks and session pricing on the OpenAI websocket passthrough
* fix(proxy): check the model in every OpenAI websocket passthrough frame and price the session
* fix(proxy): bill each OpenAI websocket turn on its own row and run full auth per frame
Each finished turn (completed, incomplete, or failed) now logs its own spend row under
the model and access groups that allowed it, so group budgets see spend as turns end
instead of at close. Every model a frame names goes through the key's full auth, and a
stored prompt reference with no model is refused for a key held to some models. A frame
still in flight when the client leaves is recorded before the relay gives up on the send
* fix(proxy): keep the stored prompt refusal's first sentence inside the websocket close reason
The close reason carries the refusal's first sentence, and the stored prompt sentence ran past the 123 bytes a close frame allows, so it was cut mid-word. The sentence is shorter now and the rest of the explanation stays in the error event.
* fix(proxy): bill pipelined websocket turns to their own frame and close the review gaps
* fix(proxy): log no extra websocket spend row for frames after the last turn
* fix(proxy): price websocket Responses turns by served tier and hosted tools, and bill turns a client walked out of
A websocket Responses turn is now priced through the same Responses cost path the HTTP passthrough uses, so a priority-tier answer and a hosted web search cost what they cost over HTTP. When the client leaves mid-answer, the gateway keeps reading the provider's socket until that turn ends, up to 10 minutes, so the turn the provider still bills reaches spend
* fix(proxy): bill websocket turns by lane, skip turns OpenAI refuses, and stop frame checks counting the session's own reservation
An error event refuses a queued response.create, so the next turn is no longer charged to the refused frame's model. Turns on separate stream_id lanes are charged to the frame that started each one. The frame model gate settles the connect reservation before it re-runs auth, so a key with less budget left than one turn's max cost is no longer refused on its first frame.
* fix(proxy): bill each websocket lane cut off at session end on its own row
When a client leaves while turns on several stream_id lanes are still
running, each unfinished turn now gets its own spend row under the groups
and model its own frame asked for, instead of one merged row.
Also pins the PR's new single-assigned test variables as Final and swaps
datetime.now() in the logging-handler and success-handler fixtures for
fixed timestamps.
* test(proxy): mark the remaining new websocket test variables Final
Builds the logging mocks with model_call_details in the constructor instead
of setting it afterwards.
* fix(proxy): check every model the realtime URL names before the OpenAI websocket relay connects
* fix(proxy): keep reading running answers after a refused websocket frame so they are billed
* fix(proxy): bill realtime transcription sessions on the OpenAI websocket passthrough
A transcription session names its model only in session.updated under audio.input.transcription, and reports usage on each completed transcription instead of response.done, so its row logged unknown at $0 and a budgeted key never spent.
* test(proxy): mark the turn-taking client's delivered list Final
* fix(proxy): hold a websocket session's parallel slot until the socket closes
Each finished turn logged its spend row under the session's own call id, so the first turn's success callback freed the key's max_parallel_requests slot while the socket was still open, and a key limited to one session could open several at once. The session now claims its slot under its own call id, every turn row logs under a call id of its own, and the slot is released when the session ends, as before
Priced turn rows also take their row's call id as their id, so two turns that start in the same instant no longer share a spend log id and drop one row
* fix(proxy): bill realtime transcription clips that finish outside an answer
A conversation.item.input_audio_transcription.completed frame that arrived after a session's last response.done was never billed, because the session end only owes a row for a turn still running. A clip that completes while no answer is running now logs its own row, and a clip inside a running answer stays in that answer's row
Each row also carries the session's latest session.updated frame, since later rows lost the transcription model it names and priced clips against the realtime model. A newer session.updated replaces the older one, so the stale model is never read first
It also drops the docstrings this PR had added, since the names already say what each helper does
* fix(cost): price a realtime transcription clip at the transcription model the session set
A realtime conversation session opens with session.created naming the realtime model and no
transcription model, and the client sets the transcription model later in session.update. The
cost code took the first session frame and fell back to its session model, so a clip was priced
at the realtime model. That model has no per-second rate, so a gpt-transcribe clip cost $0, and a
token-billed clip used the realtime model's audio rate instead of the transcription model's.
Now the newest session frame that names a transcription model sets the price, and the first
session model is only the fallback when no frame names one, as before
* fix(cost): price each realtime transcription clip at the model set when it ran
A session that changes its transcription model mid-way kept every logged clip
on the newest model. Each clip now takes the transcription model from the last
session event before it, and falls back to the session's own model
* fix(passthrough): keep the session update a buffered clip ran under until its answer is billed
A transcription that completes while an answer streams waits in the
buffer for that answer's response.done. A session.updated in between
used to drop the earlier update, so the clip was priced at the opening
configuration. An earlier update now stays while an unbilled clip
follows it, and drops at the next update once that clip is billed.
* fix(passthrough): keep reading a websocket turn the client left before OpenAI started it
A client that sends response.create and hangs up before OpenAI answers with
response.created made the walkout checks see no turn in flight, so the gateway
closed the provider socket and the turn OpenAI still ran and billed never
reached spend. The relay now counts every forwarded create until the provider
starts or refuses it, and keeps draining while one is owed
* fix(passthrough): settle a websocket turn that ends without its start frame
A turn the provider ended without sending response.created first left its
create counted as not started, so a client leaving later held the provider
socket for the whole walkout drain with nothing left to bill. A turn's end
now settles one waiting create when its lane saw no start frame
* fix(proxy): refuse unparseable and unroutable websocket frames and hold legacy slots per socket
A client frame too deep or too long for the JSON parser closed the provider
socket mid-turn, so the turn in flight was never billed. The gate now refuses
that frame with an error event and keeps draining the open turn.
A stream_id OpenAI rejects comes back as an error that names no lane, which
dropped another turn's authorization. The gate now refuses those frames before
they reach OpenAI.
With the legacy parallel request limiter, each per-turn spend row freed the
key's slot while the socket stayed open. Websocket passthrough rows now free
nothing, and the socket frees its slot once when it closes.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): stream azure_ai natively on an Azure OpenAI host on /v1/messages
The /v1/messages bridges got the requested model (azure_ai/gpt-6-luna) with the resolved provider (azure), so the Responses bridge faked the stream and the chat bridge called a deployment named azure_ai/gpt-6-luna (404). Pass the resolved provider prefix when the requested prefix is a different known provider.
* test(anthropic): cover the bridged model with respx and a regional bedrock_mantle id
* fix(anthropic): carry azure_ai env credentials into bridged /v1/messages calls
When an azure_ai deployment's api_base and key come only from AZURE_AI_API_BASE
and AZURE_AI_API_KEY, the provider the async wrapper derived from the model alone
replaced the model's own resolution, so the bridged call lost both and failed.
The model's own resolution now wins when it names the same provider, an explicit
provider that disagrees still wins, and the bridge carries the resolved key and base.
* fix(anthropic): keep the caller's key, proxy routing and MCP credentials on bridged /v1/messages calls
* fix(anthropic): keep an azure deployment's Entra credential over the env key on bridged /v1/messages calls
* fix(anthropic): skip the sign-in lookup for chatgpt and github_copilot model prefixes on bridged /v1/messages calls
* fix(anthropic): keep the Azure credential of an azure_ai deployment that sets its own api_base on bridged /v1/messages calls
* test(anthropic): annotate the parameters of the bridged /v1/messages tests
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Scope the Prisma migration lock test's advisory lock check to its own database, so a lock another xdist worker holds in its database no longer trips it
Keep in-process MCP peers' idle connections open for 30s instead of 1s, so the gateway's pooled HTTP client doesn't reuse a connection the peer is closing and hang the tool call
Delete database guardrails in the logging-only scope tests through DELETE /guardrails/{id} on the shared gateway instead of raw SQL, so a default-on guardrail stops applying to the next test on the same worker right away
Bound the wedged coordination Redis /customer/update check by the shortest Redis wait the proxy uses, like the burst check, so a slow runner can't fail it while a handler that waits on Redis still does
* fix(proxy): make /update/sso_settings a true PATCH and send only changed SSO fields from the UI
* fix(proxy): mask SSO client secrets in the /update/sso_settings response
* fix(proxy): keep SSO values that only live in the environment_variables row on a partial SSO PATCH
The SSO PATCH strips every SSO env var from the environment_variables config
row. Now that the UI only sends changed fields, an omitted value that lived
only in that row was dropped from both tables and lost on restart. Carry those
values into the SSO row before stripping them
* test(ui): pin CLEARED_SSO_SETTINGS to exactly the backend SSOConfig fields
With PATCH semantics an omitted key keeps its stored value, so a field
missing from the Clear list would silently survive Clear SSO settings
* fix(responses): serialize sequence_number on every bridged streaming event
* test(responses): cover sequence_number on bridged streaming events
* fix(responses): read undeclared fields through dict accessors and number guardrail block continuations
* fix(responses): scope dict reads that see undeclared fields to Responses stream events
Moves the extras-aware __getitem__, get, __contains__ and items from the
shared response base onto a new base for the 37 stream event classes, so
usage, Gemini, Interactions and moderation objects keep reading __dict__
and Gemini responses keep _hidden_params out of .get
* fix(router): renumber Responses stream events only after a fallback takes over
A stream that never falls back now keeps its provider's own sequence numbers, repeats included, so a starting_after resume still matches them. After a fallback the wrapper continues above the highest number the client saw
* fix(guardrails): keep Responses stream numbers unique when LLM Shield flushes held text
The restorer's flush copied the last delta and kept its sequence_number, so a
native stream repeated a number and a bridged stream sent the flush unnumbered.
A flush now takes the done event's slot and later events move up by one, and a
stream the provider never closed numbers its flush after the last event.
Also pins the created_at in the streaming iterator test fixture instead of
reading the clock.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept
* test(e2e): send the azure_ai beta check without the code execution tool
* test(anthropic): require a vertex_ai and azure_ai decision for every anthropic beta
* fix(anthropic): forward the betas Claude Code sends that anthropic, vertex_ai, and azure_ai accept
* test(e2e): send Vertex's header-only betas in the beta acceptance case
* fix(anthropic): keep forwarding Vertex computer-use betas and point beta tests at platforms that still lack them
A live probe found each Claude 4.5 and 4.6 model on Vertex needs one of the two
computer-use betas for the computer tool, so dropping both broke it. The e2e
acceptance test leaves those two out of its all-at-once Vertex call, and the
integration and unit tests that relied on a beta being dropped now use one
the platform still lacks.
* test(anthropic): point unsupported-beta tests at betas the platforms still drop
The Azure AI handler test used claude-code-20250219 as the beta Azure drops, and the
Rust parse test used it as an unknown value. This PR forwards it on Azure AI and maps it,
so both now use values that stay dropped or unknown.
* fix(anthropic): add computer-use-2025-11-24 for the computer_20251124 tool
The computer_20251124 tool fell through to the default computer-use-2024-10-22 beta, which the
provider rejects for that tool version. Map it to computer-use-2025-11-24, which the beta map now
forwards on anthropic and azure_ai
* fix(anthropic): send no computer use beta for an unknown computer tool version
An unknown computer tool version fell back to computer-use-2024-10-22, which the anthropic and azure_ai columns now forward. It now gets no beta, as every platform effectively sent before. Every computer tool version also skips defer_loading and allowed_callers, computer_20251124 included
* test(azure_ai): read the branch's beta map in the auto-injected beta wire test
* fix(anthropic): keep defer_loading and allowed_callers on the computer_20251124 chat tool
* test(anthropic): read the branch's beta map in the azure_ai safeguards beta wire test
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): keep encrypted reasoning on a hop between same-key deployments on the provider's default endpoint
* test(router): type the default-endpoint boundary test helpers
* fix(guardrails): restore the google_text_moderation embeddings fix an earlier main merge dropped
* fix(router): match encryption boundaries on the key, account, and region the request really uses
A deployment's named credential now fills only the values the deployment leaves out, through the
same helper the request path uses, so the boundary names the key and base the request sends.
A deployment with no api_key takes the provider's default key, the boundary carries the OpenAI
organization and project a deployment sends, and a Bedrock Mantle region named in the model
routes through the provider resolver before the Responses URL builder
* fix(router): keep header-signed accounts apart on a shared boundary and skip boundary work no reasoning needs
Headers the request logger masks as secrets (Authorization, api-key, x-api-key) now join the
boundary's account scope, with extra_headers overriding headers the way the request merges them,
so deployments that sign in through a header on one base never share encrypted reasoning.
Target boundaries are built only when a reasoning item from another deployment needs the
comparison, and the Responses input strip asks only about encrypted reasoning items, so a pinned
turn or fallback hop with nothing to compare makes no secret-manager reads.
* fix(router): resolve default boundaries only for openai and drop the unsent organization param
Aggregators and Bedrock Mantle serve several vendors on one key, so a missing key or base there no longer yields a shared boundary. The Responses request never sends the organization param, so it no longer splits the boundary, while a Cookie header now does.
* fix(router): match explicit pairs as on main and key default boundaries by model
An explicit api_base plus api_key pair is the whole boundary again, so tag
headers no longer split deployments that main kept together. The default
OpenAI boundary now carries the model, since one OPENAI_BASE_URL can front
several vendors, and a deployment that declares another provider gets no
boundary before any secret lookup.
* fix(router): match an explicit key and the environment key on one api_base
A deployment with an explicit api_base and api_key now shares a boundary with one on the same api_base that takes the same key from OPENAI_API_KEY, when the model and account headers also match. Two explicit pairs still match on base and key alone, as on main.
* fix(router): skip a deployment whose params cannot be read in boundary checks
A named credential with a value of the wrong type, or an extra header whose name is not a string, made the boundary check raise and fail the whole request. That deployment now has no boundary, so the others still keep the reasoning they can decrypt. Two explicit api_base plus api_key pairs also compare without resolving the provider, so they read no secrets, as on main.
* fix(router): ask a secret manager for default boundaries at most once a minute per deployment
* fix(router): resolve a changed named credential at once in the cached boundary and take an injected clock
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(gemini): answer 400 for an unsupported chat audio format, drop it under drop_params
* fix(gemini): return the audio format error as a value and keep direct mapper callers refused
The format helper returns its UnsupportedParamsError for the caller to raise. The shared voice mapper refuses an unsupported format again for direct SDK callers, the error caps the echoed format at 100 characters, and its advice no longer claims the model only emits pcm16
* fix(gemini): refuse an unsupported audio format nested in speech_config
A format inside audio.speech_config skipped the check and was silently
dropped with a 200. It now gets the same 400 unless drop_params is set.
The 400 test also pins the model name in the chat-path message.
* fix(gemini): show a non-string audio format as sent in the 400
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Deletes the A2A e2e test that registered a third-party agent, which depended on a site the project does not control, along with its helpers and its coverage registry row. Adds an integration test where a scripted A2A agent receives a skill call as a data part and returns a data artifact, asserting the agent and the caller each get theirs unchanged
* test: move Sail and Weave e2e tests to integration tests
The Sail service_tier and completion_window cases now run against the scripted upstream in tests/integration/pricing, asserting the outbound body and the billed rates for chat, responses and messages
The key-scoped Weave delivery cases now run against an OTLP collector in tests/integration/observability. The case where only the otel callback is configured is skipped with a BUG reason
Removes the e2e copies, the Weave read-back client, the Sail registry rows, the Weave docs paragraph and the unused devin MCP server entry from the stage mirror config
* test: move Presidio masking e2e tests to integration tests
Moving them to integration tests. The pre_call, post_call and default scope cases now run against a mock Presidio analyzer and anonymizer that follow the real wire contract, and a scripted model upstream, on chat completions, messages and responses, with and without streaming
The spend log cases check the stored response and the guardrail record with a proxy that stores prompts in spend logs. The e2e copy, its Presidio guardrail params body, its registry rows and the doc paragraphs naming the two PRESIDIO variables are removed
* fix(proxy): support reload with multiple workers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): stop the reload worker pool on a broken edit instead of respawning it
Workers now load the app through a factory that exits with uvicorn's
STARTUP_FAILURE code on any import error, so the pool stops and the
reloader waits for the next save, the same as single-worker reload.
Multi-worker reload needs uvicorn>=0.50.0 for that, so older versions
keep the single-worker reload with a warning.
* test(integration): stop the reload test's proxy through the harness shutdown helper
Main's TQ010 rule rejects a raw .terminate() under tests/integration, so the multi-worker reload test now calls OwnedProxy.begin_shutdown(), which sends the same SIGTERM
* test(proxy): cover the reload worker pool end to end and edit the config the proxy loads
* docs(agents): scope the two-worker reload QA rule to uvicorn proxies that use Redis
* fix(proxy): keep the original traceback when a reloaded worker fails to import the app
* docs(agents): point both router and coordination Redis at the QA Redis and cover single-worker trees
* fix(proxy): stop the reload worker pool when the reloader exits and give reloads a longer worker healthcheck
Closing the terminal or pressing Ctrl-\ killed the reloader and left the pool parent holding the port while it respawned workers that could not start. The pool now stops once its parent reloader is gone. The reload pool also defaults the worker healthcheck to 30 seconds unless --timeout_worker_healthcheck is set, since uvicorn's 5 second default killed workers that were still booting on a reload.
* fix(proxy): watch the reloader through its process sentinel so the pool also stops on Windows
* fix(proxy): carry the graceful shutdown timeout into the reload worker pool
* fix(proxy): stop the reload worker pool on Ctrl-\ instead of waiting out a dying worker's healthcheck
* test(proxy): skip the startup-failure reload tests on uvicorn without a startup failure code
* docs(agents): keep the QA proxy rule to two workers, Redis, and reload
* fix(proxy): keep Windows on single-worker reload and pass unlisted uvicorn args to the reload pool
* fix(proxy): stop reload workers when their pool parent dies
* test(proxy): move the reload pool process tests to the integration suite
The unit test guide keeps tests/unit in-process, so the subprocess pool cases now live in tests/integration/configuration and use the harness port allocator and process helpers
* docs(agents): keep main's proof rules and inline the multi-worker QA command
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): exit reload workers as startup failures when the app factory raises
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): check the reload pool serves the overridden worker app
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Phoenix is unused by the e2e suite and has no account behind it. Drops the callback from the stage mirror gateway config, and updates the OTEL trace test's failure message and the e2e contributing notes that still pointed at the arize_phoenix preset.
* feat(ovalix): file and tool scanning, app discovery from the key alias
Mirror of BerriAI/litellm#34132 at 5ffc268585, rebuilt on main.
* fix(ovalix): keep inspecting requests whose key has no user
The tracker allows a checkpoint call with an empty actor without inspecting
it, so a key with no user (or a /v1/messages or /v1/responses request that
carries its own metadata object) got no Ovalix enforcement. Fall back to
"unknown" as the guardrail did before, so every request is inspected.
* refactor(ovalix): check texts without mutation and test discovery through tracker requests
- _check_texts builds its result from per-text verdicts checked newest first by _check_text
- discovery and routing cache unit tests answer the tracker routes instead of replacing private methods
- integration test adds streaming cells and a tracker outage during a concurrent burst
* fix(ovalix): resolve tool result names in one pass
Each tool result rescanned every earlier message for its call id, so a request with thousands of
tool results spent seconds in the event loop. Declarations are now indexed once per request and
each result takes the nearest earlier one by bisection
* test(ovalix): annotate new test bindings and build the burst plan with product
* fix(ovalix): drop app discovery from the key alias
A config with no application_id no longer boots in discovery mode, where
the [Name] prefix of the key alias picked the Ovalix application and so its
policy. Anyone who could rename a key could move it to a weaker app, or to
an unknown name the tracker created with its default policy. The guardrail
now needs application_id at startup again, as on main, and every checkpoint
call goes by the configured application and checkpoint ids.
Removes enable_routing_cache and fail_if_no_application, the
app_name_regex and resolve_application tracker calls, and the per-alias
routing cache. Restores main's startup check that a requested pre_call or
post_call hook has its checkpoint id.
---------
Co-authored-by: Shalom Jamil <shalom@ovalix.io>
* fix(ui): point the Spend Per entity docs link at the cost tracking page
* fix(docs): replace the remaining docs/proxy/enterprise links with current pages
* revert(terraform): leave the team tags docs link for a code-owned follow-up
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(agents): cover empty search embedding model settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(agents): treat an empty search embedding model as not configured
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): sort search integration imports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): format search embedding tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(agents): annotate search test locals and isolate upstream call checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
