The managed file listing cut the page to `limit` first and applied the
purpose filter in Python afterwards, so a page whose rows all failed the
filter came back as `data: []` with `has_more: true`. openai-python stops
paging the moment `data` is empty, so `files.list(purpose="batch", limit=1)`
returned nothing at all instead of every batch file.
Read successive keyset chunks until the page holds `limit + 1` matches or
the caller's rows run out, then return at most `limit` of them. `data` is
now non-empty whenever matching files remain, its last id is always a
usable cursor, and `has_more: false` only ever means the caller has seen
everything. Rows whose stored blob will not parse drop out in the same
loop, so they cannot empty a page either.
That also makes the `next_cursor_id` escape hatch on `build_list_page`
dead, so it goes back to what it was for the batch and vector-store
listings that share it.
Also move `validate_file_list_limit` up into the list_files route, so the
target_model_names and provider branches reject an out-of-range limit the
same way the managed file store already did.
Postgres takes the row source parenthesised, so `INSERT INTO "t" ("a") (SELECT
...)` copies a whole table at boot. Reading only the unparenthesised text let it
through: 777eb8af10 caught it, then e7dea842c3 traded it away to stop a VALUES
list joined to a query by a set operation from bounding nothing.
Read the top level first so set operations still count, then fall back to the
whole statement when no top-level VALUES bounds the insert. `TABLE t` is a row
source as much as a `SELECT` is, and it was passing too
Chat completions, embeddings, the non-streaming /v1/messages tests, and the
OpenAI batch deployment now register through the provider edge, so
E2E_FIXTURE_MODE=record captures their provider calls and replay serves them
back offline. None of them was wired before, so record was a silent no-op over
these suites and replay quietly went live instead of using the bundle
Multipart uploads now key on their parsed parts: every ordinary form field,
plus the field name, filename, content digest, and length of each file part.
The boundary is envelope rather than content, so it stays out of the digest
instead of changing the key on every run. A body that does not parse as its
declared envelope still has the boundary normalized away before hashing, so
the fallback is at least stable, and it records a name that says why
Binary uploads hash byte for byte. Canonicalizing them first meant decoding
with errors="replace", which collapsed every invalid byte to one U+FFFD and
gave two different PDFs of the same length the same key
Bundles stay out of the repo: they hold verbatim provider response bodies and
expire seven days after recording. Publishing them for CI is LIT-5748, and
streaming fidelity is LIT-5742
Four more ruff rules for code the test suite runs but never checks. F601 is the
one that paid: the duplicate key it flagged in a get_form_data fixture was the
mock reproducing the production bug fixed in the previous commit.
B025 removed two unreachable handlers, one of them a pytest.skip shadowed by an
earlier `pass`, so an upstream Vertex flake reported green having asserted
nothing. F632 turned an `is ""` identity check, which passes only on CPython
interning, into the `== ""` it meant. B023 fixed three closures over loop
variables, all latent today but one iteration-order change away from checking the
last case N times.
get_form_data collapsed the FormData multidict with dict(form) before the loop
that rebuilds `foo[]` arrays ever ran, so a request sending
timestamp_granularities[]=word and timestamp_granularities[]=segment reached the
provider as ["segment"] with the first value silently dropped. Read the multidict
with multi_items() instead.
The test could not catch it because its mock was a plain dict carrying the same
key twice, which Python collapses exactly the way the bug did. Every request.form
mock that fed get_form_data now returns real FormData.
* fix(pricing): add undated azure aliases for gpt-audio-mini and gpt-realtime-mini
Azure deployments are commonly created against the undated model name,
and the cost-tracking docs say to set base_model to azure/<model> — but
only the dated -2025-10-06 entries existed for these two models (the
openai provider has undated aliases for both). base_model:
azure/gpt-audio-mini therefore resolved to nothing and, depending on the
fallback path, text tokens billed at $0 while audio tokens billed fine.
Mirror the -2025-10-06 entries as undated aliases, exactly like the
undated openai entries mirror their newest dated variant.
Fixes#33170
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(pricing): assert undated azure audio aliases exactly mirror their dated entries
Review follow-up: COST_FIELDS missed realtime-specific cost keys
(cache_creation_input_audio_token_cost, cache_read_input_token_cost,
input_cost_per_image). Full-entry equality catches drift on every field.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pricing): mirror updated mode=realtime on the undated gpt-realtime-mini alias
Upstream changed the dated entry's mode from chat to realtime after this
branch was cut; the undated alias must stay a byte-for-byte mirror.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pricing): mirror the new deprecation_date onto the undated gpt-audio-mini alias
* test(pricing): use shared local_model_cost_map fixture so get_model_info's lru_cache never crosses maps
---------
Co-authored-by: Mihidum Hettiyahandi <55163074+mihidumh@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A dollar-quoted payload is read as its own region rather than as a handed-off
string, so the marker belongs on the rewrite inside it. Pin that placement, and
pin that a marker on a DO block header never covers the block's body.
* fix(anthropic_messages): gate sampling params on /v1/messages like /chat/completions
/v1/messages forwarded temperature/top_p/top_k raw to models that removed
sampling params (supports_sampling_params: false — Claude 4.7+/Fable 5),
producing provider 400s that router fallbacks mask as silent model
downgrades. The chat path already gates these via
AnthropicModelInfo._apply_sampling_param; reuse it in
get_requested_anthropic_messages_optional_param so both endpoints agree:
drop under drop_params, else raise the clean client-side 400.
Fixes#35053
* test(anthropic): drive new sampling-param tests off the kwarg, not the global
The five tests added here set `litellm.drop_params = True` under a manual
try/finally. That trips TQ005 (module-global mutation, 10 new violations
over the ceiling) and it leaks process-wide if the finally is ever
skipped, which is what the save/restore conftest exists to paper over.
`get_requested_anthropic_messages_optional_param` already takes
`drop_params` as a kwarg, and that is the path /v1/messages actually
uses, so pass it directly. `monkeypatch.setattr` pins the global to
False so each test proves the per-request flag alone is sufficient and
cannot pass on a leaked global.
Verified: TQ gate clean, all 10 tests pass, and the 3 that assert the
new gating still fail with the fix in utils.py reverted.
---------
Co-authored-by: Mihidum Hettiyahandi <55163074+mihidumh@users.noreply.github.com>
The unscoped GET /v1/files limit check accepted 0, which OpenAI's minimum
of 1 does not allow, and the route's except block rebuilt every error with
getattr(e, "status_code", 500). ProxyException has no status_code, so the
400 it raises went out as a 500 and the OpenAI SDK retried it three times.
Errors now go through handle_exception_on_proxy, the helper the sibling
batches route already uses, and the unknown-cursor error is a ProxyException
so it carries type invalid_request_error and param after instead of the
literal "None". The cursor still 400s whether the file belongs to someone
else or does not exist at all
EXPLAIN ANALYZE runs the statement it wraps rather than only planning it, but
ANALYZE sits in the keyword set, so it stood in for the keyword underneath and a
rewrite left under one reached boot unflagged.
* fix(mcp): resolve admin OAuth sessions to the same server set the connect page shows
* fix(mcp): bind admitted admin rows through the entitlement ceiling, not the credential scope clause
DO takes its body as a string literal, and dollar quoting is a convenience
rather than a requirement. A migration spelling the body in single quotes
got its rewrite through untouched, since nothing was reading that literal
as SQL. It is ordinary syntax rather than an attempt to hide anything, so
the miss was reachable by accident.
The module docstring now also records where concatenated dynamic SQL stops
being readable, which is a keyword split across fragments that do not hold
it. Every fragment is scanned, so the shapes people actually write are all
still caught.
add_provider_specific_headers_to_request tagged the client's Authorization header
with the same provider list as anthropic-beta and anthropic-version, so an
sk-ant-oat subscription token was sent to AWS Bedrock and Google Vertex AI as
well. On Bedrock it replaced the SigV4 signature, or the deployment's own API
key, and AWS answered 403 "Invalid API Key format". On Vertex it went out as a
second Authorization header next to the Google one and Google answered 401
ACCESS_TOKEN_TYPE_UNSUPPORTED.
The credential and those API headers need different scopes, so a request can now
carry more than one ProviderSpecificHeader entry. The API headers keep the
provider list they already had and the credential gets its own entry scoped to
anthropic alone. get_provider_specific_headers takes either a single entry or a
sequence and merges only the entries whose provider list matches, so callers that
pass one entry keep working unchanged.
Bedrock SigV4 signing and the deliberate extra_headers Authorization pass-through
in _sign_request are left alone.
The new test set LITELLM_LOCAL_MODEL_COST_MAP and reassigned litellm.model_cost
by hand, leaking both into every test that ran after it and skipping the
get_model_info cache clear. The conftest fixture already does this properly and
restores the original map on the way out.
A marker on an EXECUTE now covers the SQL that EXECUTE runs, so it goes
where the migration reads rather than inside the string. A literal whose
first line sat below its EXECUTE was missing the marker entirely, and the
documented placement failed CI.
A literal assigned with := counts as SQL only when an EXECUTE in the same
body runs that variable by name. An error message naming a DELETE the
application handles is text, and the only way to silence it before was a
marker claiming a bounded data migration that was not there at all.
A page whose rows are all dropped by the purpose filter, or by a row
that does not parse, used to come back with an empty data list, has_more
true and last_id null, so the caller had no cursor to advance with and
stopped one page short of files it owns. last_id now falls back to the
last row the page read.
Also drops the OpenAIFilesPurpose import that the widened purpose
annotation left unused.
The docstring claimed azure pricing mirrors the openai family, which stopped
being true when gpt-5.6-sol took its promotional cut and azure did not. Azure
publishes no sol rate of its own today, so the entries stay where they are.
OpenAI's model page for gpt-5.6 serves the GPT-5.6 Sol page and states
that the gpt-5.6 alias routes requests to GPT-5.6 Sol, so the alias bills
at Sol's rates. The registry entry was left on the pre-cut rates while
gpt-5.6-sol took the cut, overbilling gpt-5.6 callers by 25 percent on
input and 50 percent on output.
All 23 cost fields on gpt-5.6 now match gpt-5.6-sol, and a regression
test pins the two entries together so they cannot drift again.
The owner-scoped listing read every row the caller owns in one query, so an
admin key that owns every file on the proxy pulled the whole table into one
response. Page it with a keyset cursor on unified_file_id instead, and accept
limit and after on GET /v1/files so a client can walk the pages. limit follows
what OpenAI documents for that route: 1 to 10000, default 10000.
An after cursor is resolved inside the caller's own scope, so an id they do not
own gets a 400 rather than a page, and has_more now reflects whether another
row exists instead of always being false.
Refs #37714
The websocket routes under /openai_passthrough and /openai had no e2e
coverage, so nothing catches the regression from issue #36088, where both
prefixes carried HTTP routes only and refused every upgrade with a 403
before a socket ever existed.
Two tests cover it. The realtime one opens /openai_passthrough/v1/realtime
and asserts OpenAI's own session.created frame comes back, which proves the
route is registered and relayed upstream. The responses one asserts
/openai/v1/responses accepts the upgrade, since a responses.connect socket
waits for the client to speak first and has no opening frame to check.
A refused upgrade is an HTTP response rather than a close frame, so both
assert on the handshake. ws_base_url moves into e2e_config now that a
second suite needs it
The three lite-image keys landed on the deploy branch separately while this
branch was open, so merging left every key defined twice in both price maps.
The merge is clean as text and the file still parses, but JSON keeps the last
occurrence of a repeated key, so the first copy's supported_endpoints,
supported_modalities and supports_system_messages were being dropped without
any error.
Each key is now one entry, placed next to its gemini-3.1-flash-image sibling
rather than at the end of the file.
supports_reasoning goes to false on all three, matching every other Gemini
image model. Leaving it off is not neutral: _supports_factory falls through to
the vertex_ai provider config, which answers true, and reasoning_effort then
gets forwarded to an image endpoint that rejects it. That was fixed for the
rest of the family in 75dd70a678 and these entries had drifted back.
Also fills in what the entries were missing against Google's published
pricing: the Vertex implicit cache read rate, batch rates on the Vertex
routes, and the pdf/video input flags.
The two overlapping test files are folded into one, and the price map suite
grows a duplicate-key guard so the next clean-but-lossy merge fails loudly.
Routing rerank through get_request_headers also picked up its
AWS_BEARER_TOKEN_BEDROCK branch. Bedrock API keys are only valid for
Bedrock and Bedrock Runtime actions, not for Agents for Amazon Bedrock
Runtime ones, and rerank is served by bedrock-agent-runtime, so AWS
rejects a bearer-signed rerank call. Opt the rerank handler out of the
bearer path so it keeps signing with SigV4.
scan() recursed into a dollar-quoted body with the sliced text but kept absolute
offsets, so line_of counted newlines in the slice against a position past its end.
Any DO $$ block below the first line reported a wrong line, which also misaligned
the -- data-migration-ok: markers: an unrelated marker earlier in the file could
exempt a rewrite inside a block, and a marker sitting right above one failed to.
Line numbers now always count against the whole migration text.
EXECUTE was treated as harmless while its quoted SQL was masked, so a rewrite
handed over as a string walked through the gate. The literal an EXECUTE runs is
now scanned like a dollar-quoted body.
INSERT was classified by searching the whole statement for SELECT, so a bounded
INSERT ... VALUES holding a scalar subquery, or led by a helper CTE, was flagged
as INSERT ... SELECT. A top-level VALUES now bounds the insert, and a VALUES
buried in a subquery still does not.
* fix(logging): preserve uvicorn color_message args during secret redaction
SecretRedactionFilter clears record.args after substituting record.msg, but
uvicorn's colorized formatter re-renders the separate color_message extra
field against record.args at emit time. With args cleared, uvicorn prints
the raw "%s://%s:%d" template instead of the actual startup URL whenever
output goes to a TTY (colors on).
* fix(logging): narrow color_message fallback to TypeError
Bare except-Exception-pass on the color_message substitution pushed the
BLE001 and S110 strict-rule budgets over their ceiling. Narrow to the one
exception the %-format can actually raise and give it a real fallback
instead of silently swallowing it.
* refactor(logging): move color_message substitution into a helper
The two record.color_message stores put LIT011 over its ceiling. Building the
value in a pure helper leaves one store, marked rebind-ok since scrubbing a
record in place is the logging.Filter contract.
Also pins the ordering that makes the substitution safe: it has to run before
args are cleared, which puts it before the extra-field loop that redacts the
result, so a secret arriving through record.args is still scrubbed out of
color_message.
* fix(proxy): group Codex turns under one session id
Codex puts its conversation uuid in an unprefixed `session-id` header
(`session_id` on builds before the codex-api split), so
`get_chain_id_from_headers` never matched it: the `x-<vendor>-session-id`
regex requires an `x-` prefix. Codex also sends no request metadata the
Anthropic `metadata.user_id` path could parse and no traceparent, so every
turn fell through to a freshly generated per-call trace id and landed as its
own row in the logs.
Read the unprefixed `session-id` / `thread-id` (and the older `session_id` /
`conversation_id`) names, gated on the Codex user agent. Those names are
generic enough that an unrelated client could send one meaning something
else, and colliding values across callers would merge their traces, so the
bare-header path stays Codex-only.
* fix(proxy): match every first-party Codex originator
`is_codex_user_agent` tested `startswith("codex_")`, but the Codex TUI sends
`codex-tui` with a hyphen, and often bare with no version at all. Real values
seen in the wild are `codex-tui` and
`codex-tui/0.149.0 (Mac OS 26.5.1; arm64) ghostty/1.3.1 (codex-tui; 0.149.0)`.
codex-rs's own `is_first_party_originator` lists `codex-tui`, `codex_cli_rs`,
`codex_vscode` and a `Codex ` prefix, which agree only on the `codex` stem.
Match that stem plus a separator so no spelling is missed and an unrelated
`codexfoo` client still is. This also repairs the pre-existing gap where
`should_auto_drop_params_for_agentic_cli` (called on the request path at
litellm_pre_call_utils.py:2049) never fired for the Codex TUI.
* refactor(proxy): take headers as a read-only Mapping in the Codex session lookup
The auto-router savings figure was computed only inside the spend-update
writer, downstream of where logging callbacks consume the standard logging
payload, so Datadog-style callbacks never received it. Compute it once in
the payload builder, stamp it as a top-level payload field beside
cost_breakdown, thread it into the spend log metadata, and have both
spend-writer call sites read the recorded value with recomputation as the
fallback for rows written before the field shipped. Internal sub-calls
(classifier, shadow eval) are never stamped, and a caller-forged metadata
value is discarded by the unconditional overwrite.
Resolves LIT-5973
* fix(mcp): strip root_path before matching the per-server MCP route spelling
The 401 challenge for a gateway-managed oauth2 MCP server advertises the
protected-resource metadata URL in the spelling the client connected on, so a
strict RFC 9728 section 3 client lands on a document whose `resource` equals the
URL it actually called. That spelling test compared `_original_path` against the
root-relative `/{server}/mcp` shape, but `_original_path` and `scope["path"]`
are raw request-line paths that still carry the deployment's `root_path`
On a SERVER_ROOT_PATH deployment the prefix therefore made the legacy test fail
and every request fell through to the standard `/mcp/{server}` branch. A client
connecting on `/litellm/github/mcp` was pointed at the standard-pattern
document, which serves `resource = {base}/litellm/mcp/github`; that is not the
URL the client called, so a strict client aborts discovery before the MCP
request fires
Route the path through `get_route_relative_request_path` first, which removes
`root_path` on a segment boundary the same way
`litellm.proxy.auth.auth_utils.get_request_route` already does for the rest of
the MCP auth path, so `/litellmfoo` is not truncated under `root_path=/litellm`
* fix(mcp): make the gateway-managed 401 challenge root-path aware
The gateway-managed authorization_code challenge in process_mcp_request
built its AS-metadata URL from two root-path-unaware pieces:
- it matched the caller's spelling against `scope["_original_path"]`, a
raw request-line path that still carries the deployment prefix, so on a
SERVER_ROOT_PATH deployment the `/mcp/{server}` branch never matched and
every request fell through to the legacy one-segment form
- it hardcoded `/.well-known/oauth-authorization-server` without the
root-path segment the discovery route decorators bake in, so the URL
404'd under a sub-path deployment regardless of which branch was taken
Route the spelling match through get_route_relative_request_path and the
well-known root through well_known_root_suffix, the same two helpers the
discovery route registrations derive their paths from, so the advertised
URL cannot drift from the route that serves it.
Root-mounted deployments are unaffected: both helpers are no-ops when
SERVER_ROOT_PATH is unset.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Drops the doubled-quote branch in skip_quoted, which masked the same span
either way and so could not be covered, and orders the failure report before
the guidance text.
