User-configured pass-through endpoints with ``auth: false`` are
explicitly unauthenticated — the builder short-circuits at
check_api_key_for_custom_headers_or_pass_through_endpoints and returns
a fresh empty UserAPIKeyAuth() without an api_key, user_id, or role.
Pre-refactor, that empty token never reached common_checks. After the
centralization, it does — and common_checks rejects it as admin-only,
breaking every Langfuse / custom unauthenticated pass-through.
This is the same regression class as the public-routes one: a
builder fast-path whose return value cannot survive common_checks.
Honor the same contract here — when the matched endpoint config has
auth != True, skip the centralized gate. auth=True endpoints still
run the full gate (covered by a companion test).
No security regression: ``auth: false`` is the operator's explicit
opt-out from LiteLLM auth on this path. The original commit closed
seven authenticated bypasses; this exemption applies only to a path
the operator has already declared unauthenticated.
The passthrough helper copied the upstream provider's Server: header
(e.g. "cloudflare" from Anthropic) onto the FastAPI response. uvicorn
then added its own Server: header on top, producing two Server: lines
in the wire response. Strict HTTP parsers (aiohttp's, used in CI's
passthrough tests) reject this with "Duplicate 'Server' header found"
and the request fails with a 400.
Same risk for Date, Content-Length, Connection, Keep-Alive: the ASGI
server writes its own copy at serialization time. Forwarding the
upstream's value either duplicates the header or lies about the
re-serialized body length.
Drop these from the forwarded set. Application/business headers
(content-type, x-request-id, anthropic-ratelimit-*, etc.) still pass
through unchanged.
Two regressions introduced by 3737d6a1f3 (centralized common_checks):
1. Public routes (e.g. /health/readiness, /metrics) are exempted by the
builder fast-path but the wrapper then ran common_checks on the
synthetic INTERNAL_USER_VIEW_ONLY token, which has no user_id, no
team, no scopes — so common_checks rejected the request as admin-
only. This broke every k8s readiness probe when master_key is set
(helm chart job confirmed: pod never goes Ready, service has no
endpoints).
2. The admin user_object synthesis only triggered when
user_object is None. After any team-creation flow runs, the row
for litellm_proxy_admin_name (default "default_user_id") exists
in litellm_usertable with the default user_role=internal_user.
get_user_object then returned that row, the synthesis was skipped,
and master_key requests were demoted to internal_user — failing
/team/update, /team/block, etc. The token's user_role is the
source of truth for these paths (set inside the authenticated
master_key / JWT-admin builders); a stale DB row must not override
it.
Fix:
- Short-circuit _run_centralized_common_checks for routes already in
LiteLLMRoutes.public_routes (or general_settings.public_routes).
Same exemption surface the builder already trusts.
- When the token's user_role is PROXY_ADMIN, force the synthesized
admin user_object regardless of what get_user_object returned.
Preserves the spend value from the DB row.
Neither change reopens any of the seven bypasses the original commit
closed: OAuth2, JWT non-admin, DB-fallback, /user/auth, pass-through
headers, etc., still go through the gate. Only paths that were
already admin or already public skip it.
Adds two regression tests:
- test_centralized_common_checks_skips_public_routes
- test_centralized_common_checks_master_key_admin_overrides_db_user_role
* fix(vertex passthrough): log :embedContent and :batchEmbedContents responses
* test(vertex passthrough): add unit tests for :embedContent and :batchEmbedContents logging
* fix(vertex passthrough): extract input text from request body for embedContent token counting
* fix(vertex passthrough): add embedContent and batchEmbedContents to TRACKED_VERTEX_ROUTES
* fix(vertex passthrough): detect Google AI Studio URLs in embedContent handler
* test(vertex passthrough): add unit test for Google AI Studio URL embedContent provider detection
* style: black format vertex_passthrough_logging_handler
Extract the admin team-header attachment into a helper so
auth_builder stays under the 50-statement lint threshold; apply
black formatting to the two files flagged on the prior commit.
No behavior change.
Add _depth/_max_depth guards (default 20) so the nested dict masking
cannot run away, and allowlist the function in the recursive_detector
CI check alongside the other bounded recursive helpers.
`_check_byok_credential` previously returned silently when `prisma_client`
was None, bypassing BYOK ownership validation during database-outage
windows. Any proxy-authenticated user could invoke BYOK-protected MCP
tools without a stored credential during the outage window.
Now raises HTTP 503 with a structured error so the flow fails closed.
Regression test asserts 503 is raised when `prisma_client` is None.
Reported by @brodmart in GHSA-6762-2m23-5mxp.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Scope the header-driven team fetch to LLM API routes so admin
management routes keep the pre-existing bypass behavior (no
phantom teams, no 404s on mgmt calls). Team context is threaded
onto UserAPIKeyAuth so spend logs, rate limits, and team_models
attribution are correctly applied when admins act on behalf of
a team via x-litellm-team-id.
* fix(proxy): honor object_permission for managed vector store access
* perf(proxy): preload team object_permission on UserAPIKeyAuth
Populate team_object_permission during virtual-key and JWT auth when the
team is loaded, so can_user_access_vector_store uses it in memory first
and only falls back to get_object_permission by id when missing.
Made-with: Cursor
* fix(team_endpoints): auto-add SSO team members to org for proxy admins
* test: proxy_admin vs team_admin security boundary for team→org move
* screenshots: before/after for team-org SSO fix
* fix(team_endpoints): restore staging security features dropped in SSO commit
Co-Authored-By: Ishaan Jaff <ishaan@berri.ai>
* style: black formatting for team_endpoints
- _get_masked_values now recurses into nested dict values and covers
additional field name patterns (credentials, password, passwd)
- _row_to_submission_item applies masking before returning litellm_params
- list_guardrails_v2 filters DB and in-memory guardrails to the caller's
team memberships for non-admin users; admins still see all guardrails
- approve_guardrail_submission propagates team_id into the in-memory
guardrail dict so ownership is preserved after approval
MCP server CRUD endpoints (/v1/mcp/server*) were bundled with MCP
tool-call / passthrough endpoints under llm_api_routes, so setting
DISABLE_LLM_API_ENDPOINTS=true on admin-only nodes also blocked the
Admin UI from listing, adding, or attaching MCP servers.
Separate mcp_inference_routes (data-plane, gated by
DISABLE_LLM_API_ENDPOINTS) from mcp_management_routes (control-plane,
gated by DISABLE_ADMIN_ENDPOINTS). Keep mcp_routes as a union for
backward compat with allowed_routes=["mcp_routes"] virtual key configs.
Upgrade is_management_route to pattern-aware matching so
/v1/mcp/server/{path:path} resolves for concrete IDs.
Temporary MCP OAuth sessions were kept in process-local memory, so on
multi-instance/LB proxy deployments a session created on instance A could
not be found when the follow-up /server/oauth/{server_id}/... request
landed on instance B.
Persist temporary session records to Redis (encrypted with the existing
proxy encryption helpers) as a best-effort L2 cache alongside the current
in-memory L1. Convert get_cached_temporary_mcp_server to async and await
it from the authorize/token/register OAuth endpoints.
Made-with: Cursor
Vertex multi-region endpoints (e.g. us, eu) use the rep host pattern, not
{geo}-aiplatform.googleapis.com. Regional IDs still contain a hyphen.
common_utils.get_vertex_base_url centralizes the rule for SDK/API URL building.
Proxy pass-through duplicates the same branching in a local get_vertex_base_url
(with trailing slashes) to avoid importing from common_utils there; live
WebSocket passthrough uses the same multi-region host logic for wss://.
Tests cover us/eu for the common_utils helper.
Made-with: Cursor
Sibling tests were mutating litellm.proxy.proxy_server.master_key and
prisma_client with raw setattr. Values leaked across tests in the same
xdist worker, flipping the auth short-circuit in user_api_key_auth and
causing unrelated tests (e.g. test_ui_view_session_spend_logs_pagination)
to return 401 instead of 200.
Replace raw setattr with monkeypatch in the two offending files and add
an autouse conftest fixture that snapshots/restores the known-leaky
module globals for every proxy test.
Two fixes to proxy-db CI:
1. test_realtime_webrtc_endpoints.py's `proxy_app` fixture mutated the
module-global `proxy_server.master_key` without restoring it, leaking
state into any test that shared the same xdist worker. Under
--dist=loadscope with 2 workers (GHA proxy-endpoints), this caused the
google_endpoints tests to fail with "No api key passed in." because
user_api_key_auth saw a set master_key and a missing API key on the
test request. The fixture now saves and restores the original value.
2. Address the Greptile note that the semantic shard design has no
catch-all, so a new test file added to tests/proxy_unit_tests/ without
a matrix entry would silently skip CI. Adds an assert-shard-coverage
job that enumerates test_*.py files and fails the workflow if any are
not referenced by a matrix entry, with a clear message telling the
author which semantic shard to place it in. All proxy-db shards now
depend on this guard.
The mocked async_increment_cache_pipeline is invoked from Router's
deployment_callback_on_success, registered as an async success callback.
Those callbacks are enqueued to GLOBAL_LOGGING_WORKER and run on a
background task, so the mock may not have been called yet when the test
asserts on it. Flush the worker before asserting.
Two independent deflakes:
1. test_ui_view_spend_logs_unauthorized (unit) was returning 400 instead
of 401/403 when earlier tests in the file left proxy-auth globals
(prisma_client, master_key, user_custom_auth, general_settings,
user_api_key_cache) in a state that let invalid tokens pass auth and
fall through to the endpoint's own start_date/end_date validation.
Add an autouse fixture that pins those globals to their import-time
defaults for every test in the file. Harden the assertion to include
response body so future flakes are diagnosable.
2. test_basic_spend_accuracy (CI job proxy_spend_accuracy_tests) depends
on the Redis transaction buffer flushing spend to Postgres. The buffer
uses a single global pod-lock key (cronjob_lock:db_spend_update_job)
and a single global buffer list key. Pointing the proxy at the shared
remote Redis means concurrent CI pipelines contend for the same lock
and can drain each other's buffer into the wrong database. Add a
start_redis reusable command that boots a per-job redis:7-alpine
container (digest-pinned), and switch proxy_spend_accuracy_tests to
REDIS_HOST=host.docker.internal:6379 so lock and buffer state are
isolated per CI run.
- Add gpt-5.5 to GPT5_MODELS parametrized list so both OpenAIGPT5Config
and AzureOpenAIGPT5Config routing tests cover the new model.
- Add test_generic_cost_per_token_gpt55 verifying the new entry's
cost-map values ($5/$0.50/$30 per 1M) and that generic_cost_per_token
returns the expected prompt/completion costs.
* feat: add gpt-5.5 to model cost map
Add gpt-5.5 entry with pricing from OpenAI flagship page:
input $5/1M, cached input $0.50/1M, output $30/1M, 272K context.
* test: add gpt-5.5 coverage for model cost map and gpt-5 routing
- Add gpt-5.5 to GPT5_MODELS parametrized list so both OpenAIGPT5Config
and AzureOpenAIGPT5Config routing tests cover the new model.
- Add test_generic_cost_per_token_gpt55 verifying the new entry's
cost-map values ($5/$0.50/$30 per 1M) and that generic_cost_per_token
returns the expected prompt/completion costs.
The periodic budget-window reset job filtered keys/teams with
`where={"budget_limits": {"not": None}}`. The prisma-client-python
library does not support null-filtering on `Json?` columns (no
DbNull/JsonNull sentinel — upstream issue #714). The client drops the
`None` value during serialization and the engine rejects the query with
`MissingRequiredValueError: where.budget_limits.not: A value is
required but not set`, so neither the key nor team reset path runs.
Switch those two `find_many` calls to `query_raw` with
`WHERE budget_limits IS NOT NULL`, selecting only the PK and the
`budget_limits` column. Writes still go through the ORM. Add unit tests
covering the expired/unexpired paths for keys and teams, string-encoded
JSON payloads, empty payloads, error isolation between the two paths,
and a regression guard asserting the query still uses `IS NOT NULL`.
