Commit graph

38207 commits

Author SHA1 Message Date
Ryan Crabbe
f9bce31a79
type: narrow _ORG_ADMIN_FEATURE_LABELS annotation to dict[str, str] 2026-04-24 10:18:35 -07:00
Ryan Crabbe
4cdcd87bd6
feat: add UI settings to restrict org admins from creating keys, teams, models
Adds three new UISettings flags gated by check_org_admin_feature_access:
- disable_key_generate_for_org_admin
- disable_team_create_for_org_admin
- disable_model_add_for_org_admin

When enabled by a proxy admin, users with the ORG_ADMIN role receive a 403
on /key/generate, /team/new, and /model/new respectively. All other roles
(proxy admin, internal user, team) are unaffected and continue through
their existing auth checks. Flags are persisted to litellm_uisettings and
synced into general_settings via _RUNTIME_GENERAL_SETTINGS_FLAGS so the
enforcement helper can read them at request time.
2026-04-24 10:18:35 -07:00
yuneng-jiang
09f0a3380f
Merge pull request #26362 from BerriAI/litellm_fix_proxy_test_master_key_leak
[Fix] Tests - Proxy: Isolate master_key/prisma_client module globals between tests
2026-04-24 10:04:09 -07:00
Sameer Kankute
a0c52cda6e
docs(proxy): clarify x-litellm-model-group vs provider model id (#25497)
Made-with: Cursor
2026-04-24 16:59:03 +00:00
yuneng-jiang
8dda834cf9
Merge pull request #25842 from BerriAI/litellm_docs-gemini3-thinking-defaults
docs(gemini): Gemini 3 thinking_level defaults and release note
2026-04-24 09:45:24 -07:00
yuneng-jiang
023dad5bde
Merge pull request #25932 from BerriAI/litellm_docs-code-block-padding-parity
feat(docs): align fenced code block padding on blog and doc pages
2026-04-24 09:45:08 -07:00
yuneng-jiang
61ad127a75
Merge pull request #25935 from BerriAI/litellm_anthropic-stream-strip-gemini-thought-tool-id
fix(anthropic): strip Gemini thought suffix from streaming tool_use id
2026-04-24 09:43:27 -07:00
yuneng-jiang
4e3feda952
Merge pull request #26221 from BerriAI/litellm_responses_strip_custom_tool_call_namespace
feat(responses): strip custom_tool_call namespace for all providers
2026-04-24 09:42:55 -07:00
yuneng-jiang
d73b790cae
Merge pull request #26248 from BerriAI/litellm_anthropic_messages_call_type_fix
fix(proxy): preserve anthropic_messages call type for /v1/messages logging
2026-04-24 09:42:36 -07:00
Ryan Crabbe
6ea95a6379
fix(jwt-auth): apply team TPM/RPM + attribution for admins using x-litellm-team-id
Scope the header-driven team fetch to LLM API routes so admin
management routes keep the pre-existing bypass behavior (no
phantom teams, no 404s on mgmt calls). Team context is threaded
onto UserAPIKeyAuth so spend logs, rate limits, and team_models
attribution are correctly applied when admins act on behalf of
a team via x-litellm-team-id.
2026-04-24 09:40:59 -07:00
Yuneng Jiang
4d5c3476a4
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_docs-gemini3-thinking-defaults 2026-04-24 09:40:04 -07:00
Yuneng Jiang
b2afc70080
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_docs-code-block-padding-parity 2026-04-24 09:39:06 -07:00
yuneng-jiang
78d171b2b9
Merge pull request #26115 from BerriAI/litellm_gpt54_mini_nano_versioned_models
feat(models): add versioned GPT-5.4 mini/nano snapshots
2026-04-24 09:34:55 -07:00
Sameer Kankute
4dbea4e957
fix(responses): enforce spec object on completion bridge (#26327)
Ensure Chat Completions -> Responses bridge always emits object="response" so non-native providers return the same top-level schema as native OpenAI Responses.

Made-with: Cursor
2026-04-24 09:29:06 -07:00
Yuneng Jiang
55ea431c05
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_gpt54_mini_nano_versioned_models 2026-04-24 09:28:54 -07:00
Sameer Kankute
e1466be825
feat(pricing): gemini-embedding-2 GA cost map, blog, and test (#26391)
* feat(pricing): gemini-embedding-2 GA cost map, blog, and test

- Add model_prices entries for gemini-embedding-2 (Gemini + Vertex paths)
- Add docs blog gemini_embedding_2_ga with LiteLLM proxy curl examples
- Add test_gemini_embedding_2_ga_in_cost_map in test_utils

Made-with: Cursor

* Fix greptile reviews
2026-04-24 09:28:18 -07:00
Shivam Rawat
9dcb2bd528
fix(proxy): respect object-level permissions for managed vector store endpoints (#26351)
* fix(proxy): honor object_permission for managed vector store access

* perf(proxy): preload team object_permission on UserAPIKeyAuth

Populate team_object_permission during virtual-key and JWT auth when the
team is loaded, so can_user_access_vector_store uses it in memory first
and only falls back to get_object_permission by id when missing.

Made-with: Cursor
2026-04-24 09:21:13 -07:00
Cesar Garcia
8bd58fb82d
Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-24 13:12:19 -03:00
ishaan-berri
863f922be8
fix(team_endpoints): auto-add SSO team members to org on move (proxy admin only) (#26377)
* fix(team_endpoints): auto-add SSO team members to org for proxy admins

* test: proxy_admin vs team_admin security boundary for team→org move

* screenshots: before/after for team-org SSO fix

* fix(team_endpoints): restore staging security features dropped in SSO commit

Co-Authored-By: Ishaan Jaff <ishaan@berri.ai>

* style: black formatting for team_endpoints
2026-04-24 08:36:25 -07:00
Sameer Kankute
1720903bda
Merge pull request #25346 from BerriAI/litellm_Sameerlite/responses-bridge-optin
feat(responses): add use_chat_completions_api flag for openai/ models with custom api_base
2026-04-24 20:55:22 +05:30
Yuneng Jiang
09d401ed6d
fix: tighten guardrail param handling in list and submission endpoints
- _get_masked_values now recurses into nested dict values and covers
  additional field name patterns (credentials, password, passwd)
- _row_to_submission_item applies masking before returning litellm_params
- list_guardrails_v2 filters DB and in-memory guardrails to the caller's
  team memberships for non-admin users; admins still see all guardrails
- approve_guardrail_submission propagates team_id into the in-memory
  guardrail dict so ownership is preserved after approval
2026-04-23 23:13:06 -07:00
ryan-crabbe-berri
0992bf2271
Merge pull request #26367 from BerriAI/litellm_/split-mcp-routes-management-vs-inference
Split MCP routes into inference vs management (unblock Admin UI on DISABLE_LLM_API_ENDPOINTS nodes)
2026-04-23 22:05:48 -07:00
Ryan Crabbe
2c3c8aa4ea
Move "Store Prompts in Spend Logs" toggle to Admin Settings
Previously, the "Store Prompts in Spend Logs" and "Maximum Spend Logs
Retention Period" settings were surfaced via a gear-icon modal on the
Logs page. The gear was visible to every authenticated user even though
the backend endpoints (/config/update, /config/list) require PROXY_ADMIN
— so non-admins could open the modal but the request would 403 on load
and save, giving a confusing UX.

Move the controls into a new "Logging Settings" tab under Admin Settings,
which is already gated to admins at the sidebar. Remove the gear button
and the onOpenSettings prop chain (ConfigInfoMessage → LogDetailContent →
LogDetailsDrawer). ConfigInfoMessage now points users to
"Admin Settings → Logging Settings" inline.
2026-04-23 21:04:13 -07:00
Sameer Kankute
3c1b27e155
Merge pull request #26381 from BerriAI/litellm_internal_staging
merge main
2026-04-24 09:22:28 +05:30
Sameer Kankute
2378ef7f8c
FIx black formatinig 2026-04-24 09:15:13 +05:30
Yuneng Jiang
e68c60a66e
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_proxy_test_master_key_leak
# Conflicts:
#	tests/test_litellm/proxy/realtime_endpoints/test_realtime_webrtc_endpoints.py
2026-04-23 18:20:01 -07:00
shin-berri
8e652d129d
Merge pull request #26356 from BerriAI/litellm_cci_gha_dedup_and_shard
Some checks are pending
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
[Infra] Remove CCI/GHA test duplication and semantically shard proxy DB tests
2026-04-23 18:17:56 -07:00
yuneng-jiang
654b688c8f
Merge pull request #25746 from BerriAI/litellm_vector-store-team-byok-model-none
fix(router): restore BYOK key injection for vector store endpoints with team-scoped deployments
2026-04-23 18:16:47 -07:00
yuneng-jiang
7b47dffefd
Merge pull request #26375 from BerriAI/litellm_internal_staging
Some checks are pending
CodeQL / Analyze (actions) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Helm unit test / unit-test (push) Waiting to run
Read Version from pyproject.toml / read-version (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Unit Tests: Security / security (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
[Infra] Promote internal staging to main
2026-04-23 17:55:07 -07:00
shivam
e982fe85e9
Merge branch 'litellm_internal_staging' into litellm_vector-store-team-byok-model-none 2026-04-23 17:41:11 -07:00
user
4d74a30412
chore(deps): fix brace-expansion pin and revert risky dev bumps
- Dockerfile: pin the unscoped `brace-expansion@5.0.5` alongside
  `@isaacs/brace-expansion@5.0.1`. The scoped package only has 5.0.0
  and 5.0.1 published; CVE-2026-33750's fix (5.0.5) is on the unscoped
  package which npm also vendors. The override loop now swaps both.
- Revert `black` 26.3.1 -> 24.10.0, `pytest` 9.0.3 -> 8.3.5, and
  `pytest-asyncio` 1.3.0 -> 1.2.0. The major-version bumps cause CI
  lint (black reformats hundreds of files) and code-quality
  (liccheck.ini has no entry for the new versions) failures. Both
  CVEs are dev-only; skipping leaves no runtime exposure.
2026-04-24 00:37:07 +00:00
user
5ba6bc0784
chore(deps): bump uv to 0.11.7 + drop dead npm sed
- UV_IMAGE across all Dockerfiles: 0.10.9 -> 0.11.7.
- Loosen `required-version` in enterprise/ and litellm-proxy-extras/
  from strict `==0.10.9` to `>=0.10.9` so the new Docker image can
  build those workspace members. Matches the main pyproject range.
- Drop the `sed` block that rewrote tar/minimatch version ranges in
  npm's bundled package.json files. The override loop above already
  swaps the vendored directories on disk; npm doesn't re-resolve at
  runtime, so the sed was cosmetic.
2026-04-24 00:36:59 +00:00
user
1cc935e3e4
chore(deps): bump Wolfi base digest to latest
Chainguard rebuilds wolfi-base nightly with picked-up security patches.
The current pin is from 2026-04-01; this moves to the latest digest as
of 2026-04-24 to pick up ~3 weeks of accumulated OS package updates
(openssl, glibc, nodejs apk, etc.).
2026-04-24 00:36:59 +00:00
user
fed1a14646
chore(deps): bump vulnerable dependencies
Closes Nexus IQ policy violations and open Dependabot alerts for
shipped Python deps and runtime-stage npm pins in the Docker image.
2026-04-24 00:36:59 +00:00
Ryan Crabbe
35eef7d92c
chore: apply black formatting to _types.py management_routes block 2026-04-23 17:35:35 -07:00
shivam
812044a805
rerun tests 2026-04-23 17:34:19 -07:00
shivam
b217ad44d3
rerun tests 2026-04-23 17:31:37 -07:00
yuneng-jiang
1f6ce45702
Merge pull request #26370 from BerriAI/litellm_version_bump
[Infra] Bump version 1.83.12 → 1.83.13
2026-04-23 17:30:45 -07:00
yuneng-jiang
08cc1e66cf
Merge pull request #26207 from BerriAI/litellm_team_member_total_spend_frontend
Surface per-member budget cycle in Teams > Members tab
2026-04-23 17:24:03 -07:00
Ryan Crabbe
6b6b8c7418
restore budget_reset_at on TeamMembership type
Members tab column reads this field; dropping it from the type in the
previous revert broke the type check without affecting the reverted
render logic.
2026-04-23 17:07:21 -07:00
Ryan Crabbe
fbaedc36dc
revert TeamInfo budget reset display changes
Out of scope for the members-tab feature and regressed legacy teams
whose budget_reset_at is null (duration was previously shown as a
fallback).
2026-04-23 17:01:32 -07:00
Yuneng Jiang
ffaeff54cd
add uv 2026-04-23 17:00:20 -07:00
Yuneng Jiang
29e30d9ddb
bump: version 1.83.12 → 1.83.13 2026-04-23 16:58:17 -07:00
user
d60734392b
chore(packaging): declare MIT license in litellm-proxy-extras metadata
litellm-proxy-extras ships a LICENSE file with MIT terms but did not
declare a `license` SPDX expression in its pyproject.toml, so tools
that read the metadata (PyPI, Nexus IQ, pip-licenses) reported
License-None for every published version. Add the explicit expression
so downstream scanners resolve the declared license.
2026-04-23 23:57:22 +00:00
Ryan Crabbe
4d2acafa43
Split MCP routes into inference vs management categories
MCP server CRUD endpoints (/v1/mcp/server*) were bundled with MCP
tool-call / passthrough endpoints under llm_api_routes, so setting
DISABLE_LLM_API_ENDPOINTS=true on admin-only nodes also blocked the
Admin UI from listing, adding, or attaching MCP servers.

Separate mcp_inference_routes (data-plane, gated by
DISABLE_LLM_API_ENDPOINTS) from mcp_management_routes (control-plane,
gated by DISABLE_ADMIN_ENDPOINTS). Keep mcp_routes as a union for
backward compat with allowed_routes=["mcp_routes"] virtual key configs.

Upgrade is_management_route to pattern-aware matching so
/v1/mcp/server/{path:path} resolves for concrete IDs.
2026-04-23 16:52:45 -07:00
Ryan Crabbe
ea626d9fb8
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_team_member_total_spend_frontend 2026-04-23 16:47:41 -07:00
yuneng-jiang
87e120d958
Merge pull request #26346 from BerriAI/litellm_reset_budget_is_not_null
[Fix] Reset budget windows failing due to Prisma Json? null filter
2026-04-23 16:37:09 -07:00
Yuneng Jiang
66bf890226
[Infra] Stop attaching push-only postgres workflows to a GHA environment
The `_test-unit-services-base.yml` reusable workflow attached every job
to the `integration-postgres` GHA environment to read three "secrets":
DATABASE_URL, POSTGRES_USER, POSTGRES_PASSWORD. These are not secrets —
the postgres service container is spawned per-job on localhost and
destroyed with the job, so the user/password are bootstrap values for a
throwaway container and the URL is always `postgresql://…@localhost:…`.

Each environment attachment produces a "temporarily deployed to
integration-postgres" deployment record, which the PR timeline renders
as a message per matrix shard per push. With 14 proxy-db shards that's
~14 notifications per push, drowning the PR conversation.

Changes:
* Hardcode POSTGRES_USER/POSTGRES_PASSWORD/POSTGRES_DB and the derived
  DATABASE_URL in `_test-unit-services-base.yml`.
* Delete the `environment: integration-postgres` attachment.
* Delete the `secrets:` declarations on the reusable workflow and on
  the two callers (test-unit-proxy-db.yml, test-unit-security.yml).
* The `services:` container still starts a fresh postgres per job;
  the connection string now matches what the container boots up with.

Security review: no regression. The environment wasn't gating anything
real — no protection rules configured, no approval gates, and the
branch restriction is already enforced by `on: push: branches: [...]`
on both caller workflows. Zizmor pedantic-mode findings are identical
before and after (same 6 pre-existing findings, zero new ones).

The `integration-postgres` environment and its three "secrets" in repo
settings are now unreferenced and can be deleted from repo admin.
2026-04-23 16:32:18 -07:00
Yuneng Jiang
21e08b0bb5
[Infra] Run schema-migration shard serially (workers: 0)
test_db_schema_migration.py has exactly one test, and that test is mostly
waiting on prisma subprocesses (~170s: prisma migrate deploy + prisma
migrate diff). No CPU-bound Python work inside the test body, and only
one test in the file means xdist's parallelism is unused regardless.

Previous run on commit 5df9f397e6: 10.0m wall-clock for the shard, of
which 4:56 was silence between step start and pytest banner — the cost
of 4 xdist workers each cold-starting (pytest plugin load + litellm
import + pytest-cov instrumentation) so that exactly one of them could
pick up the single test.

Switching to workers: 0 takes the serial pytest branch in the base
workflow, which already handles this case correctly (no -n, no --dist).
Single-process startup instead of 4. Expected wall-clock: ~6m.
2026-04-23 16:24:40 -07:00
milan-berri
2001d91b27
fix(mcp): share temporary MCP OAuth sessions across instances via Redis (#26162) (#26318)
Temporary MCP OAuth sessions were kept in process-local memory, so on
multi-instance/LB proxy deployments a session created on instance A could
not be found when the follow-up /server/oauth/{server_id}/... request
landed on instance B.

Persist temporary session records to Redis (encrypted with the existing
proxy encryption helpers) as a best-effort L2 cache alongside the current
in-memory L1. Convert get_cached_temporary_mcp_server to async and await
it from the authorize/token/register OAuth endpoints.

Made-with: Cursor
2026-04-23 16:21:27 -07:00