* fix(realtime): surface an upstream handshake refusal as an error event and policy close
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): tidy the handshake refusal e2e
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): keep upstream exception text out of the Azure client error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(realtime): map handshake refusal close codes with a lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 2bab39e374)
Backport of #42048 to stable/1.102.x.
Cherry-picked from 7966f50c34 (main). The safeguards backport maps the dangerous-tool-use-2026-09-03 beta for Bedrock, which Claude Opus 4.5 on Bedrock Invoke rejects as an invalid beta flag, so the all-beta-headers Bedrock cases run on Claude Fable 5.1 as they do on main.
The picked TypedDict fields use read-only Sequence[Mapping[str, object]] annotations and the picked Vertex test carries a test-quality-ok marker, so stable/1.102.x's LIT001, LIT012 and TQ008 budgets hold. Static typing only, no runtime change.
Hand-ported to stable/1.102.x from 47b2479c94 on main (fix(bedrock): gate Invoke tool search on the model map's supports_tool_search flag), the one prerequisite the #42288 handler tests need; the rest of that commit stays on main.
Backport of #42288 to stable/1.102.x.
Cherry-picked from merge commit fc82f6e8fa (litellm_safeguards_bedrock_vertex_messages).
The line has no bedrock_mantle beta-header mapping and no Mantle /v1/messages route, so the Mantle mapping, its test file, and the bedrock_mantle test parameter are left out.
AGENTS.md allows a vendor-owned literal only when its source and date are cited
next to it. A live realtime test cannot avoid naming a model, so record how the
id was checked, and record that a retired id fails as a hang rather than an
error so the next reader does not start by suspecting litellm.
(cherry picked from commit 9f84382a24)
AWS retired amazon.nova-sonic-v1:0. GetFoundationModel now answers
ResourceNotFoundException "This model version has reached the end of its life",
and opening a bidirectional stream against it fails with ValidationException
"The provided model identifier is invalid". amazon.nova-2-sonic-v1:0 is the
active replacement.
The test passed on builds 219 (2026-09-16) and 246 (2026-09-17) and has failed
every run since, three attempts per build, with no litellm change to the
realtime path in between. The symptom was a clean websocket close: Bedrock ends
the stream rather than erroring, the forwarder treats a None receive as a normal
stream end and closes the client socket, so the client sees ConnectionClosedOK
and the test fails waiting for response.done.
litellm already carries both models in the cost map, with
"deprecation_date": "2026-09-14" on the old one, and the promptStart
transformation already sends the audioOutputConfiguration that nova-2-sonic
requires; only the test constant was left behind.
Verified against live Bedrock with the promptStart shape the transformation
builds: amazon.nova-sonic-v1:0 raises "The provided model identifier is
invalid", amazon.nova-2-sonic-v1:0 opens a session and returns a usageEvent.
The mocked handler and provider-cache tests keep the old id: it is only a label
there, no call reaches AWS.
(cherry picked from commit ca18755b64)
Backport of #41347 to rc/1.102.0.
Cherry-picked from merge commit 4bb1ae115b (main), originally by app/devin-ai-integration.
Conflicts: main's #41349 (membership rows written through upsert) is not on this line, so add_new_member keeps its create call and still writes no membership row when no budget resolves. The picked tests are adapted to that and to this line's _check_team_member_budget, which loads the membership itself.
Backport of #41689 to rc/1.102.0.
Cherry-picked from merge commit 07b5051c0d (main), originally by app/devin-ai-integration.
Both conflicts were in test files: the rc line lacks the Codex additional_tools tests that sit next to the new test in test_handler.py on main, and its test_utils.py imports all_litellm_params on a separate line, so only this PR's own additions (the imports, the recording handler, and the two new tests) are taken.
Backport of #41495 to rc/1.102.0.
Cherry-picked from merge commit 0add8c0083 (main), originally by app/devin-ai-integration.
The only conflict was the test import header: the rc line never gained the logging_executor import that main carries next to this change, so only the MockResponseIterator import comes along.
Backport of #41086 to rc/1.102.0.
Cherry-picked from merge commit 4123b4bc2b (main), originally by app/devin-ai-integration.
rc/1.102.0 has no litellm/proxy/auth/resolvers/grants.py, so its one-line import change is dropped; auth_checks.py imports UserNotFoundError from the new types module, so every importer on this line still resolves.
/model/info fills a deployment's missing pricing in from the model cost map so the
Admin UI has a rate to display. Clients echo that whole model_info blob back on save,
and update_db_model merged it into the row, so editing an unrelated setting turned
that day's catalog price into a real per-deployment override. After that the
deployment ignored the cost map and Reload Price Data could no longer move it,
because the reload replays each deployment's stored pricing over the fresh catalog.
Drop the derived pricing from incoming model_info on the two write paths. The
drop-set is read off the same objects the read path uses, CustomPricingLiteLLMParams
plus the tiered *_above_N_tokens pattern that get_model_info passes through and no
model declares, so it cannot drift as new rates are added. output_vector_size is
exempt: it lives on the pricing model but is an embedding dimension, not a rate.
A deployment's own pricing still rides litellm_params, which is untouched, as is the
explicit-null clear, which reads the incoming model rather than the filtered dict.
The filter sits in the endpoint bodies rather than _add_model_to_db, which master-key
rotation reuses to re-serialize every stored deployment.
* fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics
Requests rejected before dispatch (bad key, blocked key, budget, rate limit, malformed body) were spend-logged with an empty call_type because the synthesized logging object never reached the failure lifter. The caching dashboard rolled all of them, plus failed calls on info routes such as /model/info, into one Unknown group.
Resolve call_type from the matched route first, falling back to body shape, and keep the synthesized logging object on request_data so the lifter sees it. Log bare auth exceptions with the 401 ProxyException the client gets so error_code is never empty. Exclude info routes from the cache analytics groups and error breakdown. The dashboard explains the Unknown group when older rows still produce one.
Resolves LIT-5884
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the raw auth exception for failure callbacks
Record the client-facing status in the spend log through a separate client_exception argument so custom failure callbacks still receive the exception auth raised.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the route for multi-operation endpoints and exclude info routes from cache filter options
Routes such as /v1/files map to several operations (create, list) and the
method is not available in the failure hook, so a rejected request there is
filed under its route instead of the first mapped call type. The key alias and
model filter-option queries now apply the same info-route exclusion as the
groups and error breakdown, so every offered filter value returns data. The
info-route exclusion and Unknown grouping are now covered against a real
Postgres in tests/proxy_behavior/spend/test_cache_activity.py
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop client_exception, the spend log row never used it
The DB spend row for a gate rejection is written by _ProxyDBLogger from the
original exception, so the status-bearing copy only reached the in-memory
logging payload. Live runs at the tip still recorded bare auth exceptions as
Unknown/Exception, the same as the base branch. Removing the plumbing keeps
this PR to endpoint attribution and the info-route exclusion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The provider_config path skipped run_realtime_guardrails for transcription
sessions to avoid sending response.create, which also dropped every
realtime_input_transcription guardrail: no violation error reached the
client and on_violation / end_session_after_n_fails never fired. Run the
guardrail for every completed transcript and only suppress response.create
when the session has no assistant turn.
