Forwarding limit to OpenAI made the ownership filter cut the page down after
the fact, so a key that owned an older container got an empty first page and
its cursor never moved. Non-admin lists now walk upstream pages of 100 until
they have enough owned containers (or five pages), trim to the requested
limit, and report first_id, last_id and has_more off what the caller keeps.
Also assigns tests/test_litellm/proxy/container_endpoints to a CI shard.
The container retrieve, list, delete, create and file routes validated the
provider's error body against the success model, so a deleted or unknown
container and a rejected API key surfaced as 500 pydantic errors instead of
the upstream 404 or 401. The handlers now raise the provider error class with
the upstream status and message before transforming the response.
GET /v1/containers dropped after, limit and order before calling the
provider, and GET /v1/containers/{id}/files dropped the same three, so
paginated list calls ignored their pagination arguments. Both routes now
forward their declared query params.
The dashboard's template picker imported autorouter_presets.json at build time, so every
catalog change needed a dashboard rebuild and artifacts refresh. The catalog now lives in
litellm/proxy/public_endpoints/ and GET /public/autorouter_presets serves it, fetching
litellm.autorouter_presets_url (GitHub raw on main, 1h in-process cache, bundled fallback)
so a merged catalog change propagates to running proxies like the model cost map does.
The dashboard fetches it at runtime via useAutoRouterPresets and keeps no local copy.
Resolves LIT-6764
* fix(ui): paginate request logs by session groups server-side
The logs table server-paginated raw spend logs and then collapsed
multi-call sessions client-side, so a page could render 3 rows while
the footer claimed 25 and sessions straddled pages. Adds an opt-in
group_by_session param to /spend/logs/ui that pages and counts one
representative row per session (DISTINCT ON, newest non-MCP call),
keeps the bounded count contract, enriches whole-session llm/agent
composition counts, and deletes the client-side collapse pipeline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* feat(ui): add a 10 rows-per-page option and default request logs to it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* fix(ui): key session aggregates per api key in the logs enrichment
Grouped pagination splits a reused session id into one row per api key,
but the enrichment still aggregated by session_id alone, so both rows
showed combined spend and counts. The aggregate query now groups by
(session_id, api_key), the count folds into it (the separate group_by
query is deleted), and each row reads its own key's totals.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
* fix(ui): treat an empty api_key as a real session group value
The spend-log schema defaults api_key to an empty string; truthiness
guards in the enrichment treated it as missing, so keyless multi-call
sessions lost their count and spend. Only None means missing now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QxT89fiygmzz2ALcjpu7Ve
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
get_or_compute single-flights concurrent misses under a per-key lock, but
invalidate() deletes outside it. A mint already awaiting the IdP when an
invalidation ran wrote its result into the slot afterwards, so a bearer minted
before the invalidation was served after it for its full TTL; the upstream-401
retry then re-presented the bearer the server had just rejected.
The cache now keeps a per-key generation beside the single-flight lock.
invalidate() bumps it and get_or_compute snapshots it inside the lock right
before minting, skipping the cache write when it no longer matches. The token
is still returned to the caller it was minted for, so only the caching is
fenced. Generation and lock share one object that a running mint binds to a
local, which pins the weak map's entry for exactly as long as a write can still
land; a generation held weakly on its own would be collected across the await
and the re-check would silently pass.
* fix(responses): keep provider response headers in streaming logging callbacks
The responses streaming iterator captures the provider's HTTP response headers into
its own _hidden_params, but never puts them on the completed response, and the
model_validate(model_dump()) copy made for logging drops pydantic private attributes.
Success callbacks and StandardLoggingPayload.hidden_params.additional_headers therefore
saw an empty dict for streaming /v1/responses, so Azure's apim-request-id was unreadable
from the callback payload.
Restore the headers on the nested response of the logging copy, preferring any the
provider transform already set (the fake_stream path) and falling back to the ones the
iterator captured from the stream. Skipped when the copy fell back to the original event,
so a serialization failure never leaves logging-only state on the caller's object.
* fix: satisfy LIT002 mutable-collection gate in header restore
---------
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
The pre-flight token exchange resolved its target from the requested path, so a key not entitled to that server still drove an outbound exchange at the IdP and populated the credential cache before the later access check denied it. Resolve the target through the same allowed-server set the rest of the route uses and skip the exchange when the requested server is not in it.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): forward mode and streaming params to crowdstrike_aidr handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): drop stream_chunk_builder patch from crowdstrike cadence test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): type test params and cover unsupported crowdstrike mode rejection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): stop leaking internal exception details to clients
Public error responses could disclose internal details in two places.
A proxy-layer exception with no recognized provider status code (a bug
in a custom callback, a hook, or litellm's own code) forwarded its raw
str() text verbatim on a 5xx, including any embedded credential,
filesystem path, or internal hostname, or a full stack trace; the
same client-facing message now runs through a redaction layer built
on top of the credential redaction that already runs on log output,
so it also drops an embedded traceback and scrubs path-shaped and
hostname-shaped substrings. It intentionally never runs on server-side
logs, which must keep full detail for debugging.
exception_type(), litellm's core exception mapper, is shared by direct
SDK callers (litellm.completion()) and the proxy, and it deliberately
embeds a traceback into an unmapped exception's message as a debugging
aid for library users; a first pass at this fix stripped that
traceback inside exception_type() itself and broke that convention
(caught by tests asserting on the traceback frame). The traceback stays
in exception_type()'s own output; only the proxy's client-facing
response boundary (and the streaming response generator, which never
needs to embed one at all) strips it.
Full generic-message replacement for the unclassified-exception case
was tried first and reverted too: several routes deliberately raise a
bare exception as an informative, secret-free validation message (e.g.
the OCR endpoint's rejection of provider-native file IDs), and
replacing those wholesale broke that convention; targeted redaction
leaves them untouched.
Also stops the default uvicorn-based proxy from sending a Server
response header.
Resolves LIT-6747
* refactor(proxy): drop the unrelated error-message constant and trim redaction comments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): honor HTTP(S)_PROXY / NO_PROXY when force_ipv4 uses the httpx transport
Passing an explicit transport to httpx.AsyncClient / httpx.Client disables its
automatic environment proxy mounts, so force_ipv4 on the httpx path sent every
LLM request direct and silently bypassed HTTPS_PROXY. Mount the same env-derived
proxy transports next to the IPv4-pinned direct transport in AsyncHTTPHandler,
HTTPHandler and the OpenAI async client factory.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): carry the client's TLS verify and cert settings onto env proxy mounts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The default password policy from #39381 (12+ chars with upper, lower, number, and special) makes globalSetup's POST /user/update with password "test" fail with a 400, so every role login in the UI e2e suite has been failing since that merge. Seed the roles, the noteam@test.local login, and the second-admin spec with policy-compliant passwords and update the scrypt hash in seed.sql to match.
* fix(ollama_chat): stamp finish_reason tool_calls when tool calls streamed before the done chunk
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ollama_chat): trim finish_reason override comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
* fix(proxy): mark session/SSO/SAML cookies Secure behind a TLS-terminating reverse proxy
litellm only sees a plain-HTTP hop when TLS terminates at a reverse proxy,
so cookie Secure attributes previously derived from (or defaulted without
regard to) the literal request scheme could be dropped in production. The
token session cookie set by every login path never carried Secure/HttpOnly/
SameSite at all.
Adds IPAddressUtils.is_request_https, a single trust-aware resolver used by
every cookie-setting call site: PROXY_BASE_URL, then X-Forwarded-Proto only
from a configured trusted proxy (general_settings.use_x_forwarded_for +
mcp_trusted_proxy_ranges), then the literal scheme. An unconfigured or
untrusted caller cannot spoof the header to force Secure on.
Resolves LIT-6748
* fix(proxy): make the shared session-cookie helper public, type new test helpers
set_session_token_cookie is imported across modules (ui_sso.py ->
proxy_server.py), so the leading underscore was misleading and breached
basedpyright's reportPrivateUsage budget with zero headroom. Also adds
missing parameter/return type annotations to the new test helper functions
per repo convention.
* refactor(python-bridge): split non-streaming bridge modules
* refactor(python-bridge): bring shared function tracing into route layer
* feat(dev): list Python route functions and call sites
* feat(dev): list Rust route functions and call sites
* docs(dev): record OCR parity gaps across Python and Rust
* feat(dev): list executed SDK calls with runtime tracing
* feat(dev): report Python vs Rust SDK pipeline steps in one CLI
* feat(dev): side-by-side pipeline step report in compare CLI
* fix(dev): drop invalid Final annotations in compare cell loop
* feat(dev): blue python-only and yellow rust-only steps in compare CLI
* feat(dev): vertical layout with section spacing in compare CLI
* fix(dev): validate SDK trace stages across sync and async routes
* refactor(rust): align SDK route call structure with Python
* refactor(python-bridge): share sync and async route call wrappers
* refactor(dev): split compare CLI into fixtures, runtime, and report modules
* fix(ci): run SDK trace tests and satisfy test lint
* test(proxy): verify NO_DOCS/NO_REDOC/NO_OPENAPI gate every doc surface
Confirms /docs, /redoc and /openapi.json are reachable by default and
each 404 with no schema in the body when NO_DOCS/NO_REDOC/NO_OPENAPI are
set, and that disabling them does not affect other routes.
* test(proxy): pin the production docs_url construction to the real helpers
Addresses Greptile 4/5 feedback: the disabled-path tests built a fresh
FastAPI() app, which could miss drift in the actual
FastAPI(docs_url=..., ...) call in proxy_server.py. This pins that
exact call site against _get_docs_url/_get_redoc_url/_get_openapi_url.
