* fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): streamed alias matching a capability rule bills the deployment price
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): assert every streamed chunk carries the client alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): log the client alias on the priced streamed response, the same as non-streamed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 7b432d78d2)
Hand-ported prerequisite for the #43642 backport to stable/1.103.x.
Taken from dd636373229b1ab3bfb2ed5b9b17b0ebd68b3ac7 (#43288, main), scratch_database only.
The batch cleanup leftover warning used a class defined in a test-directory module. The xdist controller cannot import it, so an uncaught leftover warning crashed the whole e2e run. It is now a plain UserWarning.
The integration harness backport missed the workers option and Gateway.request headers that the OTel conversation id tests use.
Adds the OwnedProxy harness the #42486 backport's OTel test imports, registers the #43029 backport's spend flush test in contracts.json with its covers marker, and aligns the MCP tool failure assertion with main.
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486)
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep the caller's header session under missing_session_id: generate and read replayed payload session ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): drop only the proxy-minted session id so a caller id on the other metadata key survives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a replayed session id hidden when it only echoes the payload trace id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a replayed session id even when the payload trace id fell back to it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop reading the replayed payload's session id, the generated marker does not survive replay
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit gen_ai.conversation.id on otel v2 spans through a real proxy, sink and postgres
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep otel conversation rigs alive for the whole session so shuffled shards do not reboot the proxy per test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): stop the audit rig proxies from probing sibling test peers for model info
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): record accepted OTLP batches in the sink instead of mutating the collector
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): guard the accepted batch deque so snapshots cannot race sink appends
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
(cherry picked from commit a3196907e4)
* fix(jwt): accept a team alias in x-litellm-team-id (#42445)
* fix(jwt): accept a team alias in x-litellm-team-id
The header only matched canonical team ids, so a JWT caller selecting one of their teams by its alias got a 403 even though they belonged to it. The header value is now resolved through the existing alias lookup before the JWT allowed-team check and the DB membership fallback, while a value that is already a team id never costs an alias lookup and denials keep naming the value the caller sent
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): only alias a header team id the database provably lacks
Under fallback_to_db_teams a header value whose team row read fails for any reason other than TeamNotFoundError now keeps the membership denial instead of falling through to the alias lookup, so a degraded read cannot select a different team that carries the value as an alias. Drops the HeaderTeam docstring that only restated its fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 071cb49d32)
* fix(jwt): say x-litellm-team-id matched no team id or alias in the 403 (#42495)
* fix(jwt): say x-litellm-team-id matched no team id or alias in the 403
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): tell the caller when x-litellm-team-id names an alias shared by several teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): deny a shared x-litellm-team-id alias exactly like an unknown value
A distinct 403 for an alias several teams share was raised before the
allowed-teams check, so any JWT could probe which aliases exist. The
alias lookup now treats the duplicate as a miss, and both denials say
the value does not resolve to a team id or a unique team alias, which
is true for unknown, unauthorized and duplicate values alike
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 08639fcf42)
* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim (#43206)
* fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(jwt): describe header team selection under fallback_to_db_teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 7b4fd47c6e)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: yassin <yassin@berri.ai>
Prisma types a raw array parameter from the first batch a connection sees. After a flush in
which every member cost was a whole number (a free model), the connection's cached statement
expected int8[] and every later fractional batch on it failed with "improper binary format in
array element", so member spend silently stopped landing while team spend kept rising.
The rows now travel as one JSON document unpacked by jsonb_to_recordset with the column types
declared in SQL, so Postgres types the numbers and the batch shape no longer matters.
(cherry picked from commit 5e4b1b9df0)
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
pyproject pins mcp>=2.2.0,<3, where FastMCP was renamed to MCPServer and
stateless_http, json_response, transport_security, host and port moved off
the constructor onto streamable_http_app() and run(). The two helpers still
imported mcp.server.fastmcp, so every module reaching them died at collection
and integration-extensions went red on the release candidate.
This ports the helpers verbatim from main, where the same migration already
landed in 479360ae87 after rc/1.103.0 was cut. Only the two helper files are
taken; the rest of that commit restores unrelated test surface.
Collection on the four affected modules: 4 errors and 0 tests before, 0
errors and 6 tests after.
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.
The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.
The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
The check spins up a hypercorn TLS peer and drives the SDK's own httpx
handlers at it, so it needs litellm importable, hypercorn installed and a
loopback socket. It lived under tests/e2e, whose Buildkite runner image
installs neither litellm nor hypercorn by design (the suite drives a
remote proxy over HTTP), so every scheduled e2e build since #230 failed
to import the module and pytest reported it as a collection error. The
unit tree bans sockets, so it does not belong there either
tests/integration is the CircleCI tier built for real TCP against local
protocol peers. This adds an sdk shard to it for cases that exercise the
SDK's clients with no gateway in the path, registers the two HTTP/2
nodes in the contracts manifest, and adds the shard to the CircleCI
matrix. The test now flips the feature through LITELLM_HTTP2 (the user
surface) instead of patching module attributes, and asserts the version
the peer observed on the wire next to the one the client reports
Every one of these fails on main's own scheduled CircleCI run with the same
assertion as on any PR, and each traces to a merged behavior change that
never updated the test that pinned the old behavior
- tests/integration/_support/client.py: #41311 made /key/info serve deleted
keys from the archive with status deleted, so the scenario teardown asserts
the live row is gone and the readback reports deleted instead of a 404.
This alone accounts for nine integration-management and one
integration-providers failure
- tests/integration/authorization/test_warmed_policy.py: #39996 made team
admins unable to edit any team field unless a proxy admin allow-lists it,
and tpm_limit is the only field it accepts today. The demotion test now
enables tpm_limit for the scenario and edits that instead of team_alias
- tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py:
#41337 reads usage off the terminal response and copies the event when it
is missing, which a Mock(spec=ResponsesAPIResponse) cannot survive. The
four mocks now carry a usage object
- tests/test_openai_endpoints.py: #41310 lengthened the access-denied
message, and the test matched against the ExceptionInfo repr, which
saferepr truncates in the middle. It now matches the exception text
- tests/local_testing/test_text_completion.py: Together no longer serves
Qwen2-1.5B serverless, the cheapest cost-map row. The test mocks the
completions call and asserts the request litellm builds, so a vendor
catalog rotation cannot fail it again
test_router_fallbacks_with_cooldowns_and_dynamic_credentials is deliberately
untouched: it passes and fails on main with identical code, and the failing
path is a product question about whether dynamic-credential 429s cool down
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
Since #40994 the proxy relays the upstream error body on a 400, so the
public error type is now whatever the upstream sent instead of the
status-derived name. The fake upstream answered every scripted failure
with type api_error, which made the public-error contract in
test_retry_counts_and_public_errors_match_actual_provider_attempts fail
on main. The fake now sends the type a real OpenAI-compatible upstream
sends for the status, so the assertion holds whether the proxy relays
or maps the type
Adds owned local TCP peers, a Redis process helper and SigV4 helpers to
tests/integration, and integration contracts for Anthropic and Bedrock wire
shapes, Bedrock role configuration, S3 wire, observed routing, Redis
recovery and stream contracts. Consolidates the four commits previously
stacked on litellm_integration_accounting onto its main-merged tip
tests/integration/conftest.py imported the bare `integration` package. Because
tests/__init__.py and tests/integration/__init__.py both exist, pytest's default
prepend import mode puts only the repo root on sys.path, so that name resolved
only under the PYTHONPATH that tests/integration/run.py injects. Every other
invocation died at conftest import with
ModuleNotFoundError: No module named 'integration' and exit 4, including the
command test_oci_integration.py documents in its own docstring.
The imports now use the tests.integration._support path that pytest actually
resolves, matching the 120 other `from tests.` imports in the suite. run.py's
PYTHONPATH still works because it already puts the repo root on the path.
tests/code_coverage_tests/test_integration_suite_imports.py collects every file
under tests/integration with PYTHONPATH scrubbed and asserts a non-zero
collection count, so an unresolvable import fails the code-quality job instead
of only the developers who run these files by hand. CI runs the three
pre-existing files through the allowlist rather than executing them, which is
why nothing caught this.
