Partial backport to stable/1.103.x: only the text completion endpoint test moves off the retired gpt-3.5-turbo-instruct to openai/gpt-5.4-nano with reasoning off. This line predates the OpenAI SDK rewrite of the test, so reasoning_effort goes through EndpointsClient.text_completions instead of extra_body. The SAIL, spend-log and coverage registry repairs are not needed for this line's e2e suite
(cherry picked from commit e61733b170)
Partial backport to stable/1.103.x: only the Together AI structured-output e2e case moves to the hybrid reasoning backend with reasoning off, since the reasoning-only model it picked returned no content. The CI infrastructure and other test repairs are not needed for this line's e2e suite
(cherry picked from commit 6ca90b927c)
* test(e2e): tolerate provider-side flakes on five full-suite cells (#42628)
* test(e2e): tolerate provider-side flakes on five full-suite cells
Mistral OCR retries a provider-relayed 429 with backoff, the Vertex vision
probe turns reasoning off so its 32 tokens go to the answer, the Vertex
cache cell spaces eight never-seen prefixes 15s apart around Google's
nondeterministic minimum-token rejection and prices the cached tokens
instead of prompt_tokens, and the Azure content-policy cell resends the
jailbreak prompt while Azure skips its filter
* test(e2e): shorten the new helper docstrings
* test(e2e): accept a relayed provider 429 on the rust OCR cells
The gateway already retries a provider 429 three times per call and the
Mistral key is shared across pipelines, so a throttle can hold across all
four attempts of the OCR cell. After the bounded retries the cell now
accepts the gateway's faithful relay of the provider's 429 (throttling_error,
code 429) as its second expected outcome; the gateway's own 429 and any
other error still fail the cell at once.
* test(e2e): drop the harness unit tests, the live cells cover the helpers
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 41ca4656cd)
* fix(streaming): keep litellm Usage on text-completion usage chunks (#43047)
* fix(streaming): keep litellm Usage on text-completion usage chunks
* fix(streaming): convert provider usage to litellm Usage instead of dropping it
(cherry picked from commit be35b22dfc)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
The batch cleanup leftover warning used a class defined in a test-directory module. The xdist controller cannot import it, so an uncaught leftover warning crashed the whole e2e run. It is now a plain UserWarning.
The integration harness backport missed the workers option and Gateway.request headers that the OTel conversation id tests use.
Backports the batch cleanup half of #43321 so an Azure cancel still in progress after 120s leaves a warning instead of a teardown error, and points the MCP Tools UI test at DeepWiki's renamed ask_wiki_question tool.
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486)
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep the caller's header session under missing_session_id: generate and read replayed payload session ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): drop only the proxy-minted session id so a caller id on the other metadata key survives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a replayed session id hidden when it only echoes the payload trace id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a replayed session id even when the payload trace id fell back to it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop reading the replayed payload's session id, the generated marker does not survive replay
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit gen_ai.conversation.id on otel v2 spans through a real proxy, sink and postgres
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep otel conversation rigs alive for the whole session so shuffled shards do not reboot the proxy per test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): stop the audit rig proxies from probing sibling test peers for model info
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): record accepted OTLP batches in the sink instead of mutating the collector
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): guard the accepted batch deque so snapshots cannot race sink appends
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
(cherry picked from commit a3196907e4)
* fix(jwt): accept a team alias in x-litellm-team-id (#42445)
* fix(jwt): accept a team alias in x-litellm-team-id
The header only matched canonical team ids, so a JWT caller selecting one of their teams by its alias got a 403 even though they belonged to it. The header value is now resolved through the existing alias lookup before the JWT allowed-team check and the DB membership fallback, while a value that is already a team id never costs an alias lookup and denials keep naming the value the caller sent
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): only alias a header team id the database provably lacks
Under fallback_to_db_teams a header value whose team row read fails for any reason other than TeamNotFoundError now keeps the membership denial instead of falling through to the alias lookup, so a degraded read cannot select a different team that carries the value as an alias. Drops the HeaderTeam docstring that only restated its fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 071cb49d32)
* fix(jwt): say x-litellm-team-id matched no team id or alias in the 403 (#42495)
* fix(jwt): say x-litellm-team-id matched no team id or alias in the 403
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): tell the caller when x-litellm-team-id names an alias shared by several teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): deny a shared x-litellm-team-id alias exactly like an unknown value
A distinct 403 for an alias several teams share was raised before the
allowed-teams check, so any JWT could probe which aliases exist. The
alias lookup now treats the duplicate as a miss, and both denials say
the value does not resolve to a team id or a unique team alias, which
is true for unknown, unauthorized and duplicate values alike
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 08639fcf42)
* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim (#43206)
* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(jwt): describe header team selection under fallback_to_db_teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 7b4fd47c6e)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: yassin <yassin@berri.ai>
The test resolves four AWS_GOVCLOUD_* values through os.environ/ references on
the deployment, and none of them is injected into the e2e stack, so the upload
returns 500 "S3 bucket_name is required" and the test fails on configuration
rather than on the GovCloud partition behavior it guards.
The runner podSpec mounts 29 individual secretKeyRef entries from
litellm-provider-keys with no envFrom, and the string "govcloud" appears
nowhere in project-releaser, so nothing supplies these values today. The
commercial-partition Bedrock batch tests resolve their own os.environ/ bucket
and pass, which rules out the reference syntax and leaves the absent secrets as
the only cause.
Skipped rather than weakened, matching the LIT-5027 and LIT-4820 markers in
this file: the assertions stay as the correct contract and the marker names the
condition for removing it.
Reports 1 skipped instead of 1 failed.
AGENTS.md allows a vendor-owned literal only when its source and date are cited
next to it. A live realtime test cannot avoid naming a model, so record how the
id was checked, and record that a retired id fails as a hang rather than an
error so the next reader does not start by suspecting litellm.
AWS retired amazon.nova-sonic-v1:0. GetFoundationModel now answers
ResourceNotFoundException "This model version has reached the end of its life",
and opening a bidirectional stream against it fails with ValidationException
"The provided model identifier is invalid". amazon.nova-2-sonic-v1:0 is the
active replacement.
The test passed on builds 219 (2026-09-16) and 246 (2026-09-17) and has failed
every run since, three attempts per build, with no litellm change to the
realtime path in between. The symptom was a clean websocket close: Bedrock ends
the stream rather than erroring, the forwarder treats a None receive as a normal
stream end and closes the client socket, so the client sees ConnectionClosedOK
and the test fails waiting for response.done.
litellm already carries both models in the cost map, with
"deprecation_date": "2026-09-14" on the old one, and the promptStart
transformation already sends the audioOutputConfiguration that nova-2-sonic
requires; only the test constant was left behind.
Verified against live Bedrock with the promptStart shape the transformation
builds: amazon.nova-sonic-v1:0 raises "The provided model identifier is
invalid", amazon.nova-2-sonic-v1:0 opens a session and returns a usageEvent.
The mocked handler and provider-cache tests keep the old id: it is only a label
there, no call reaches AWS.
Two tests in the config/misc management suite were failing every run against
the Buildkite e2e stack, and one of them took the rest of the build with it.
test_add_allowed_ip_does_not_store_unrelated_config_value posted 127.0.0.1 to
/add/allowed_ip. That route sets the live general_settings["allowed_ips"] that
auth_utils._check_valid_ip reads before it persists anything, and the check is
exact string membership with no CIDR support, so from the moment the POST
returns only 127.0.0.1 can reach the proxy. The runner 403s on its very next
call, and the deferred /delete/allowed_ip sits behind the same auth dependency,
so the cleanup is locked out too and every later test in the build 403s. Build
254's first attempt lost 459 of its 465 failures to that one cascade.
There is no safe way to exercise the route against a shared proxy: nothing
reports the caller's address as the proxy sees it, so a test cannot allowlist
itself first. Move the claim to the route's own TestClient suite, where the
auth dependency is overridden and general_settings is per-test, and record the
route in the module docstring beside /cache/settings and the Vault override so
it is not re-added. save_config's end of the contract was already covered by
test_ProxyConfig_save_config_merges_changed_keys_without_copying_file_settings;
the new test covers the route's end, that what it hands save_config differs
from the loaded config in allowed_ips and nothing else.
The unrelated-key probe also only ever worked on one lane: max_parallel_requests
was added to tests/e2e/gateway/stage_mirror_ci_config.yml and never to the
Buildkite stack's config, where resolve() reports it as "unset" rather than
"config". That key is now unused, so drop it again.
test_config_update_persists_router_setting_to_get wrote router_settings.
num_retries, which both lanes declare in their config file, so the config-
ownership work correctly refuses it with a 400. Switch to retry_after, which is
declared by neither lane, is accepted by /config/update, and is reported back by
GET /router/settings. Verified against a live proxy: max_fallbacks also takes
the write but never reads back, so the read-back poll is what picks the key.
vLLM serves no /v1/files or /v1/batches, so a hosted_vllm deployment can never
host a batch. Batch inputs for such a deployment now land in a LiteLLM-owned
storage backend, the batch is executed line by line through the deployment's
own chat, completion, embedding, or responses route, and the batch plus its
output and error files are served back from the database under the creating key
Format the ported files, annotate mutable wire payloads, give the e2e
OAuth client the SDK 2 httpx2/AuthorizationCodeResult API, tighten the
transport-streams alias to the two-stream SDK 2 shape, and add a
test-quality reason for the MockTransport factory injection.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/config/field/info used to answer from the LiteLLM_Config row, so "the field
400s" proved the row did not carry it. It now answers from the resolved
settings, and the CI stack config declares general_settings.max_parallel_requests,
so the endpoint returns that value and the old assertion could never hold.
The check that /add/allowed_ip writes only what the caller changed moves to
/config/list, which still reports stored_in_db off the row, and the field/info
call now asserts the ownership the endpoint reports: the config file owns the
key, so it reads back as source=config and editable=false.
Verified against a live proxy on an isolated Postgres rather than in CI, where
this check has never run: it waits on protected-environment approval.
Both write paths now go through the same refusal, so /config/field/update and
/config/update answer identically instead of each phrasing its own rule.
A successful write now applies to the SettingsStore, so the next read sees it.
Without this, /config/field/info reported a key the dashboard had just stored
as "not set" until the process reloaded from the database.
resolve() no longer takes a KeyRule it never reads; the store picks the row.
The matrix tests resolve through SettingsStore instead of calling resolve
directly, so the section and key in each case actually route a lookup.
ConfigFieldInfo and ConfigList type `source` as the FieldSource literal, and
the dashboard API types are regenerated for the two new fields.
All three budget specs searched by typing into the search box and moved on
immediately. The search is debounced 300ms, and while the filtered query is in
flight react-query serves the previous page as placeholder data, which the list
hook reports as isLoading, which makes the table swap its whole body for
skeleton rows.
So the row assertion passed against the pre-search rows, and roughly 300ms later
the skeleton swap unmounted the row the spec had just opened the action menu on.
Playwright logged "element is not stable" twice and then "element was detached
from the DOM", and since the menu never reopened the click burned the full 15s
action timeout on all three attempts. Losing that race was pure timing: build
386 and build 387 of the UI suite ran the same commit 4b368bf066, and 386
passed where 387 failed on this spec plus "Delete a budget"
searchForBudget now waits for the GET that carries q=<budget id>, matching what
projectDetachment.spec.ts already does for a key search. That also gives the row
assertion something real to assert, since until now it could pass without the
search having filtered anything
DELETE /v1/files/{id} only lets a proxy admin key delete a raw s3:// or
gs:// file id, because such ids skip the managed-file owner check. The
batch lifecycle cleanup deleted the vertex_ai raw ids with the test's
own virtual key and got a 403 at teardown on every build since #194
Raw cloud-storage ids now go through the master key; managed and
provider-native ids keep using the creating key
The team access-group fixture polled a 403 until its message enumerated
the team's allow-list, because registering a team-scoped deployment
appends that deployment to the list and the fixture has to wait for the
reset to land. #41310 replaced that message with a fixed client-facing
one, so the poll never matched and both tests errored at setup
The allow-list is now read back from /team/info until it holds exactly
the access group
