* fix(anthropic): stream azure_ai natively on an Azure OpenAI host on /v1/messages
The /v1/messages bridges got the requested model (azure_ai/gpt-6-luna) with the resolved provider (azure), so the Responses bridge faked the stream and the chat bridge called a deployment named azure_ai/gpt-6-luna (404). Pass the resolved provider prefix when the requested prefix is a different known provider.
* test(anthropic): cover the bridged model with respx and a regional bedrock_mantle id
* fix(anthropic): carry azure_ai env credentials into bridged /v1/messages calls
When an azure_ai deployment's api_base and key come only from AZURE_AI_API_BASE
and AZURE_AI_API_KEY, the provider the async wrapper derived from the model alone
replaced the model's own resolution, so the bridged call lost both and failed.
The model's own resolution now wins when it names the same provider, an explicit
provider that disagrees still wins, and the bridge carries the resolved key and base.
* fix(anthropic): keep the caller's key, proxy routing and MCP credentials on bridged /v1/messages calls
* fix(anthropic): keep an azure deployment's Entra credential over the env key on bridged /v1/messages calls
* fix(anthropic): skip the sign-in lookup for chatgpt and github_copilot model prefixes on bridged /v1/messages calls
* fix(anthropic): keep the Azure credential of an azure_ai deployment that sets its own api_base on bridged /v1/messages calls
* test(anthropic): annotate the parameters of the bridged /v1/messages tests
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Scope the Prisma migration lock test's advisory lock check to its own database, so a lock another xdist worker holds in its database no longer trips it
Keep in-process MCP peers' idle connections open for 30s instead of 1s, so the gateway's pooled HTTP client doesn't reuse a connection the peer is closing and hang the tool call
Delete database guardrails in the logging-only scope tests through DELETE /guardrails/{id} on the shared gateway instead of raw SQL, so a default-on guardrail stops applying to the next test on the same worker right away
Bound the wedged coordination Redis /customer/update check by the shortest Redis wait the proxy uses, like the burst check, so a slow runner can't fail it while a handler that waits on Redis still does
* fix(proxy): make /update/sso_settings a true PATCH and send only changed SSO fields from the UI
* fix(proxy): mask SSO client secrets in the /update/sso_settings response
* fix(proxy): keep SSO values that only live in the environment_variables row on a partial SSO PATCH
The SSO PATCH strips every SSO env var from the environment_variables config
row. Now that the UI only sends changed fields, an omitted value that lived
only in that row was dropped from both tables and lost on restart. Carry those
values into the SSO row before stripping them
* test(ui): pin CLEARED_SSO_SETTINGS to exactly the backend SSOConfig fields
With PATCH semantics an omitted key keeps its stored value, so a field
missing from the Clear list would silently survive Clear SSO settings
* fix(responses): serialize sequence_number on every bridged streaming event
* test(responses): cover sequence_number on bridged streaming events
* fix(responses): read undeclared fields through dict accessors and number guardrail block continuations
* fix(responses): scope dict reads that see undeclared fields to Responses stream events
Moves the extras-aware __getitem__, get, __contains__ and items from the
shared response base onto a new base for the 37 stream event classes, so
usage, Gemini, Interactions and moderation objects keep reading __dict__
and Gemini responses keep _hidden_params out of .get
* fix(router): renumber Responses stream events only after a fallback takes over
A stream that never falls back now keeps its provider's own sequence numbers, repeats included, so a starting_after resume still matches them. After a fallback the wrapper continues above the highest number the client saw
* fix(guardrails): keep Responses stream numbers unique when LLM Shield flushes held text
The restorer's flush copied the last delta and kept its sequence_number, so a
native stream repeated a number and a bridged stream sent the flush unnumbered.
A flush now takes the done event's slot and later events move up by one, and a
stream the provider never closed numbers its flush after the last event.
Also pins the created_at in the streaming iterator test fixture instead of
reading the clock.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept
* test(e2e): send the azure_ai beta check without the code execution tool
* test(anthropic): require a vertex_ai and azure_ai decision for every anthropic beta
* fix(anthropic): forward the betas Claude Code sends that anthropic, vertex_ai, and azure_ai accept
* test(e2e): send Vertex's header-only betas in the beta acceptance case
* fix(anthropic): keep forwarding Vertex computer-use betas and point beta tests at platforms that still lack them
A live probe found each Claude 4.5 and 4.6 model on Vertex needs one of the two
computer-use betas for the computer tool, so dropping both broke it. The e2e
acceptance test leaves those two out of its all-at-once Vertex call, and the
integration and unit tests that relied on a beta being dropped now use one
the platform still lacks.
* test(anthropic): point unsupported-beta tests at betas the platforms still drop
The Azure AI handler test used claude-code-20250219 as the beta Azure drops, and the
Rust parse test used it as an unknown value. This PR forwards it on Azure AI and maps it,
so both now use values that stay dropped or unknown.
* fix(anthropic): add computer-use-2025-11-24 for the computer_20251124 tool
The computer_20251124 tool fell through to the default computer-use-2024-10-22 beta, which the
provider rejects for that tool version. Map it to computer-use-2025-11-24, which the beta map now
forwards on anthropic and azure_ai
* fix(anthropic): send no computer use beta for an unknown computer tool version
An unknown computer tool version fell back to computer-use-2024-10-22, which the anthropic and azure_ai columns now forward. It now gets no beta, as every platform effectively sent before. Every computer tool version also skips defer_loading and allowed_callers, computer_20251124 included
* test(azure_ai): read the branch's beta map in the auto-injected beta wire test
* fix(anthropic): keep defer_loading and allowed_callers on the computer_20251124 chat tool
* test(anthropic): read the branch's beta map in the azure_ai safeguards beta wire test
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): keep encrypted reasoning on a hop between same-key deployments on the provider's default endpoint
* test(router): type the default-endpoint boundary test helpers
* fix(guardrails): restore the google_text_moderation embeddings fix an earlier main merge dropped
* fix(router): match encryption boundaries on the key, account, and region the request really uses
A deployment's named credential now fills only the values the deployment leaves out, through the
same helper the request path uses, so the boundary names the key and base the request sends.
A deployment with no api_key takes the provider's default key, the boundary carries the OpenAI
organization and project a deployment sends, and a Bedrock Mantle region named in the model
routes through the provider resolver before the Responses URL builder
* fix(router): keep header-signed accounts apart on a shared boundary and skip boundary work no reasoning needs
Headers the request logger masks as secrets (Authorization, api-key, x-api-key) now join the
boundary's account scope, with extra_headers overriding headers the way the request merges them,
so deployments that sign in through a header on one base never share encrypted reasoning.
Target boundaries are built only when a reasoning item from another deployment needs the
comparison, and the Responses input strip asks only about encrypted reasoning items, so a pinned
turn or fallback hop with nothing to compare makes no secret-manager reads.
* fix(router): resolve default boundaries only for openai and drop the unsent organization param
Aggregators and Bedrock Mantle serve several vendors on one key, so a missing key or base there no longer yields a shared boundary. The Responses request never sends the organization param, so it no longer splits the boundary, while a Cookie header now does.
* fix(router): match explicit pairs as on main and key default boundaries by model
An explicit api_base plus api_key pair is the whole boundary again, so tag
headers no longer split deployments that main kept together. The default
OpenAI boundary now carries the model, since one OPENAI_BASE_URL can front
several vendors, and a deployment that declares another provider gets no
boundary before any secret lookup.
* fix(router): match an explicit key and the environment key on one api_base
A deployment with an explicit api_base and api_key now shares a boundary with one on the same api_base that takes the same key from OPENAI_API_KEY, when the model and account headers also match. Two explicit pairs still match on base and key alone, as on main.
* fix(router): skip a deployment whose params cannot be read in boundary checks
A named credential with a value of the wrong type, or an extra header whose name is not a string, made the boundary check raise and fail the whole request. That deployment now has no boundary, so the others still keep the reasoning they can decrypt. Two explicit api_base plus api_key pairs also compare without resolving the provider, so they read no secrets, as on main.
* fix(router): ask a secret manager for default boundaries at most once a minute per deployment
* fix(router): resolve a changed named credential at once in the cached boundary and take an injected clock
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(gemini): answer 400 for an unsupported chat audio format, drop it under drop_params
* fix(gemini): return the audio format error as a value and keep direct mapper callers refused
The format helper returns its UnsupportedParamsError for the caller to raise. The shared voice mapper refuses an unsupported format again for direct SDK callers, the error caps the echoed format at 100 characters, and its advice no longer claims the model only emits pcm16
* fix(gemini): refuse an unsupported audio format nested in speech_config
A format inside audio.speech_config skipped the check and was silently
dropped with a 200. It now gets the same 400 unless drop_params is set.
The 400 test also pins the model name in the chat-path message.
* fix(gemini): show a non-string audio format as sent in the 400
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Deletes the A2A e2e test that registered a third-party agent, which depended on a site the project does not control, along with its helpers and its coverage registry row. Adds an integration test where a scripted A2A agent receives a skill call as a data part and returns a data artifact, asserting the agent and the caller each get theirs unchanged
* test: move Sail and Weave e2e tests to integration tests
The Sail service_tier and completion_window cases now run against the scripted upstream in tests/integration/pricing, asserting the outbound body and the billed rates for chat, responses and messages
The key-scoped Weave delivery cases now run against an OTLP collector in tests/integration/observability. The case where only the otel callback is configured is skipped with a BUG reason
Removes the e2e copies, the Weave read-back client, the Sail registry rows, the Weave docs paragraph and the unused devin MCP server entry from the stage mirror config
* test: move Presidio masking e2e tests to integration tests
Moving them to integration tests. The pre_call, post_call and default scope cases now run against a mock Presidio analyzer and anonymizer that follow the real wire contract, and a scripted model upstream, on chat completions, messages and responses, with and without streaming
The spend log cases check the stored response and the guardrail record with a proxy that stores prompts in spend logs. The e2e copy, its Presidio guardrail params body, its registry rows and the doc paragraphs naming the two PRESIDIO variables are removed
* fix(proxy): support reload with multiple workers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): stop the reload worker pool on a broken edit instead of respawning it
Workers now load the app through a factory that exits with uvicorn's
STARTUP_FAILURE code on any import error, so the pool stops and the
reloader waits for the next save, the same as single-worker reload.
Multi-worker reload needs uvicorn>=0.50.0 for that, so older versions
keep the single-worker reload with a warning.
* test(integration): stop the reload test's proxy through the harness shutdown helper
Main's TQ010 rule rejects a raw .terminate() under tests/integration, so the multi-worker reload test now calls OwnedProxy.begin_shutdown(), which sends the same SIGTERM
* test(proxy): cover the reload worker pool end to end and edit the config the proxy loads
* docs(agents): scope the two-worker reload QA rule to uvicorn proxies that use Redis
* fix(proxy): keep the original traceback when a reloaded worker fails to import the app
* docs(agents): point both router and coordination Redis at the QA Redis and cover single-worker trees
* fix(proxy): stop the reload worker pool when the reloader exits and give reloads a longer worker healthcheck
Closing the terminal or pressing Ctrl-\ killed the reloader and left the pool parent holding the port while it respawned workers that could not start. The pool now stops once its parent reloader is gone. The reload pool also defaults the worker healthcheck to 30 seconds unless --timeout_worker_healthcheck is set, since uvicorn's 5 second default killed workers that were still booting on a reload.
* fix(proxy): watch the reloader through its process sentinel so the pool also stops on Windows
* fix(proxy): carry the graceful shutdown timeout into the reload worker pool
* fix(proxy): stop the reload worker pool on Ctrl-\ instead of waiting out a dying worker's healthcheck
* test(proxy): skip the startup-failure reload tests on uvicorn without a startup failure code
* docs(agents): keep the QA proxy rule to two workers, Redis, and reload
* fix(proxy): keep Windows on single-worker reload and pass unlisted uvicorn args to the reload pool
* fix(proxy): stop reload workers when their pool parent dies
* test(proxy): move the reload pool process tests to the integration suite
The unit test guide keeps tests/unit in-process, so the subprocess pool cases now live in tests/integration/configuration and use the harness port allocator and process helpers
* docs(agents): keep main's proof rules and inline the multi-worker QA command
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): exit reload workers as startup failures when the app factory raises
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): check the reload pool serves the overridden worker app
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Phoenix is unused by the e2e suite and has no account behind it. Drops the callback from the stage mirror gateway config, and updates the OTEL trace test's failure message and the e2e contributing notes that still pointed at the arize_phoenix preset.
* feat(ovalix): file and tool scanning, app discovery from the key alias
Mirror of BerriAI/litellm#34132 at 5ffc268585, rebuilt on main.
* fix(ovalix): keep inspecting requests whose key has no user
The tracker allows a checkpoint call with an empty actor without inspecting
it, so a key with no user (or a /v1/messages or /v1/responses request that
carries its own metadata object) got no Ovalix enforcement. Fall back to
"unknown" as the guardrail did before, so every request is inspected.
* refactor(ovalix): check texts without mutation and test discovery through tracker requests
- _check_texts builds its result from per-text verdicts checked newest first by _check_text
- discovery and routing cache unit tests answer the tracker routes instead of replacing private methods
- integration test adds streaming cells and a tracker outage during a concurrent burst
* fix(ovalix): resolve tool result names in one pass
Each tool result rescanned every earlier message for its call id, so a request with thousands of
tool results spent seconds in the event loop. Declarations are now indexed once per request and
each result takes the nearest earlier one by bisection
* test(ovalix): annotate new test bindings and build the burst plan with product
* fix(ovalix): drop app discovery from the key alias
A config with no application_id no longer boots in discovery mode, where
the [Name] prefix of the key alias picked the Ovalix application and so its
policy. Anyone who could rename a key could move it to a weaker app, or to
an unknown name the tracker created with its default policy. The guardrail
now needs application_id at startup again, as on main, and every checkpoint
call goes by the configured application and checkpoint ids.
Removes enable_routing_cache and fail_if_no_application, the
app_name_regex and resolve_application tracker calls, and the per-alias
routing cache. Restores main's startup check that a requested pre_call or
post_call hook has its checkpoint id.
---------
Co-authored-by: Shalom Jamil <shalom@ovalix.io>
* test(agents): cover empty search embedding model settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(agents): treat an empty search embedding model as not configured
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): sort search integration imports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): format search embedding tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(agents): annotate search test locals and isolate upstream call checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cohere): emit streamed tool calls from v2 tool-call-start and tool-call-delta chunks
* fix(cohere): type _parse_tool_call_delta without growing basedpyright counts
The reassigned Final local and the untyped chunk lookups added basedpyright
errors past the base ceiling. Narrow them with casts; per-rule counts for the
file are now at or below the merge-base.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* fix(cohere): parse streamed tool calls with a typed model and map Cohere TOOL_CALL to tool_calls
Read the streamed tool call through a frozen pydantic model instead of casts,
which the strict and type-discipline gates refuse. Map Cohere's TOOL_CALL and
STOP_SEQUENCE finish reasons, so a stream that reads Cohere's message-end still
ends tool_calls. The regression test now drives litellm.acompletion with two
parallel streamed calls and checks the rebuilt calls and the final finish reason
* fix(cohere): close a streamed tool call that sent no arguments with an empty JSON object
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): answer 400 naming a non-string role or tool call id instead of 500
A list role or an object or list tool_call_id crashed the chat conversion with "unhashable type". Depending on the step that hit it, callers got a 500 or a 400 that never named the field. A failed conversion now answers 400 naming the first non-string role or tool call id, and the prompt caching hook reads a non-string answer id as matching no tool call.
* fix(anthropic): name the field the conversion failed on, and answer 400 for a non-string Responses call_id
The chat field check now runs only when the conversion raised a type error, and
scans roles first, then assistant tool call ids, then tool answer ids, so an
orphan tool answer that modify_params drops is no longer blamed. A tool output
call_id that is not a string on /v1/responses answers 400 naming input[N].call_id,
and the clear_tool_uses polyfill leaves a non-string tool_use_id for the provider
* fix(anthropic): name a non-string role whatever error the conversion raised
An int or bool role fails the conversion with an invalid message error, not a type error, so it lost the named 400. Roles are now named for any conversion failure, while tool call ids stay limited to type errors
* fix(responses): name a non-string message item role by its input index
* fix(responses): name a malformed input field only where litellm reads it, by the caller's index
A tool output call_id is named only when it is an object or a list and the
conversion fails, so an orphan numeric id still converts and another field's
failure is never blamed on it. On the bridge to Claude a non-string role or
function_call call_id is named by its input item, counting items before a
compaction item. A numeric assistant tool call id on chat is named too.
* fix(anthropic): name a tool id only for the error it causes and count Codex tool items in input indexes
A numeric tool answer id that modify_params drops was blamed for any later type or attribute error, so the 400 hid the real cause. Tool ids are now named only for the error their value raises, and a hashable answer id never.
The /v1/responses bridge and its input token count now read the input the caller sent, with only additional_tools hoisted into tools, so input[N] counts those items too.
* fix(anthropic): name a tool id only for the error its own value causes and keep Codex tool item text out of the request
* fix(responses): name a tool output call_id only when the items before it convert
* fix(responses): name a non-string input field only on inference so token counting and batch records keep their answer
* fix(responses): let a client field named caller_input or names_non_string_fields reach the bridge
* test(anthropic): build the non-string id histories without editing them in place
* fix(anthropic): never name an empty role for another field's failure
* fix(anthropic): name an empty list or object role when the conversion fails on it
* fix(anthropic): name a list role under the drop_params cache cap and keep the Bedrock batch line prefix
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(router): move litellm/router.py to litellm/router_python_core.py
A pure rename with no content change, so git and review tools track the router body as a move.
The next commit turns it into PythonCore and adds the Router facade back at litellm/router.py
* refactor(router): extract PythonCore behind RouterCore and pass the router its config
The routing logic moves unchanged from litellm/router.py into PythonCore in
litellm/router_python_core.py. Router becomes a facade that builds a RouterConfig, picks a
RouterCore with make_core and delegates every member to it. The core no longer reads or writes
litellm.* directly: process settings come from RouterDynamicConfig, and inference calls, sleeps,
callback lists and the global cache go through PythonHost. A CI check bans litellm.* in the router
modules, and the proxy's construction sites build the config outside the router
Settings are static: the constructor arguments and changeset values are copied when the router is
built or the change applied, and a write to a setting attribute becomes apply(ChangeSet(assign=...)).
A Router subclass's core class also subclasses the facade, so an override's super(Sub, self) works,
and an instance-level patch of a core method is kept on the facade so nested patches unwind.
tests/unit/test_router/core tests the core in isolation: ScriptedHost serves every host operation
(scripted inference answers, recorded sleeps, settings, callbacks, cache), and make_core runs each
test once per core in CORE_BUILDERS
* test(a2a): expect JSON-RPC errors for proxy refusals on /a2a POST routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(a2a): answer proxy refusals on /a2a POST routes as JSON-RPC errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(a2a): build JSON-RPC refusal data from typed fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(a2a): stamp the OTel server span on JSON-RPC refusals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(a2a): cover JSON-RPC refusal route in unit tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A request re-dispatched to a fallback keeps the same logging object but gets a
fresh start_time. completion_start_time was only filled when it was empty, so
an attempt that inherited a value from the earlier attempt reported a negative
TTFT. That value reaches both the spend log and the latency-based routing
cache, where one negative sample empties the buffer window and takes the whole
model group down until the cache entry expires.
Ignore a recorded first-token time that predates the attempt's start and fall
back to end_time, the value already used when none was recorded.
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: AaronHowell <111272993+AaronHowell@users.noreply.github.com>
* fix(caching): stop counting an unrun server tool call mark, and follow #49192 and #35602 in their integration tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): return the server tool call id instead of a TypeGuard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the Gemini cache write carries the system prompt
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(moderations): stop the router image check from 500ing on a list input
* fix(moderations): give the list cast its cast-ok reason
* fix(moderations): type the responses input guard and keep SDK compaction items
* fix(moderations): import the compaction item through litellm.types and mark the TypeIs import
* fix(moderations): sort the compaction item import
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(watsonx): read passthrough credentials from use_in_pass_through deployments
/watsonx/{endpoint} always called the provider config with api_key=None and api_base=None, so it only ever
worked from WATSONX_API_KEY / WATSONX_API_BASE. It now asks PassthroughEndpointRouter for the proxy-wide
watsonx deployment flagged use_in_pass_through, like the other passthrough routes, and keeps the env
fallback when none is flagged.
Co-authored-by: T K Chandra Hasan <t.k.chandra.hasan@ibm.com>
* fix(watsonx): keep env key precedence when no deployment is flagged
Resolve only use_in_pass_through deployments in the route. get_proxy_wide_credentials also falls back to
WATSONX_API_KEY, which put it ahead of WX_API_KEY in generate_iam_token
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: T K Chandra Hasan <t.k.chandra.hasan@ibm.com>
* fix(streaming): fail on clean EOF without finish reason
* fix(streaming): cover shared OpenAI-compatible EOF paths
* test(router): complete mocked streaming responses
* fix(streaming): keep the EOF error message and cover Anthropic clean EOF
The request-less RemoteProtocolError tripped exception mapping into
'The .request property has not been set.' for non-OpenAI providers.
Adds Anthropic and generic-chunk unit tests, an integration test across
chat, the /v1/messages bridge and the /v1/responses bridge, and moves
main's token-estimate test onto a finished stream with a sibling that
checks the cut stream still bills the estimate on its failure log.
* fix(streaming): count a finish reason any chunk sent, keep LangGraph async streams whole
Gemini marks every chunk unfinished and a later usage chunk reset
intermittent_finish_reason, so a finished Gemini stream failed at EOF.
LangGraph's __aiter__ opened a new aiter_lines on every wrapper call,
which hit StreamConsumed after the first chunk and now surfaced as a
false EOF error instead of a silent truncation.
* fix(streaming): keep the synthetic stop for a stream that sent no chunks
Gemini's OpenAI-compatible endpoint answers with only [DONE] when
reasoning uses up max_tokens. Failing every openai/azure stream that
sent nothing turned that 200 into a 500, while every other provider
already kept the stop for an empty stream. Adds integration cells for
Gemini's trailing usage chunk, multi-event LangGraph runs, a mocked
deployment and the [DONE]-only answer.
* test(streaming): keep pytest.raises to one statement, drop the spend read for an empty stream
An empty stream writes no spend row on main either, so the [DONE]-only
cell checks only what the client sees.
* fix(streaming): read the chunk's finish reason for the sticky flag
* fix(agentcore): end a stream on messageStop when the runtime sends no metadata or final message
* test(streaming): pin the logged-chunks guard, inject the LangGraph client, use current model names
* fix(agentcore): say why the end-of-stream flags rebind
* test(streaming): cover an AgentCore runtime that ends on messageStop through the proxy
* test(streaming): cover SDK clients, request fallbacks and a concurrent burst for a cut stream
* test(agentcore): inject the async transport instead of patching the aiohttp flag
* test(guardrails): end the decision model chat fixture with a finish reason, as a real stream does
* test(streaming): shape the AgentCore and LangGraph fixtures like captures from a real runtime and server
* test(streaming): keep each stream's ending and land every spend row once through a spend log outage
* test(streaming): end each stream by its provider's finish after a proxy worker is killed mid burst
* test(streaming): run the outage test as one burst before and one during the outage instead of a capped wave loop
* test(streaming): connect the outage test's proxy to the relay without TLS so its trigger sees the spend-log insert on an ssl=on Postgres
* test(streaming): type the new stream EOF unit tests
* test(streaming): a merged reasoning stream cut mid reasoning now fails
Main's think tag test expected a stream that ends mid reasoning without a finish reason to finish with a closed think block. With this PR that stream fails, so the case moves to its own test that expects MidStreamFallbackError with the open think block as the partial content
---------
Co-authored-by: King Star <mcxin.y@gmail.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(messages): price a native /v1/messages body as served unless its provider reports its own cost
* fix(messages): bill the requested speed when a native body as served reports none
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): split a provider-reported total between input and output by the cost map rate ratio
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): expect the kyma reported-cost headers to follow the cost map rate split
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): keep the kyma test file at its original formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): price the response-cost header split from the served model, not the request alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
