Commit graph

6178 commits

Author SHA1 Message Date
yuneng-jiang
d701cc6588
Merge pull request #21081 from BerriAI/litellm_yj_release_changes_feb12
[Infra] CI/CD Fixes - Nightly Release Feb 12
2026-02-12 18:07:34 -08:00
Ishaan Jaff
9202e67e33
feat: MCP server discovery UI (#21079)
* feat: add curated MCP server registry for discovery UI

Curated list of 31 well-known MCP servers with names, icons,
categories, transport config, and registry URLs. Includes HTTP
endpoints for GitHub, Atlassian, Sentry, Snowflake, and Cloudflare.

* feat: add GET /v1/mcp/discover endpoint for MCP discovery

Admin-only endpoint that serves the curated MCP registry with
optional query and category filters. Used by the UI discovery modal.

* feat: add DiscoverableMCPServer types for MCP discovery

* feat: add fetchDiscoverableMCPServers network function

* feat: add MCP discovery modal component

Compact list-row layout with category filters, search, and
grouped server list. Follows dev-tool aesthetic.

* feat: wire MCP discovery modal into server management page

Add MCP Server button now opens discovery modal. Card click
pre-fills the create form. Custom Server opens blank form.

* feat: add prefill from discovery and back-to-registry link

Create form accepts prefillData from discovery selection and
shows a Browse MCP Registry link to return to discovery modal.

* test: add unit tests for MCP discovery endpoint and registry

Tests for registry JSON structure validation and endpoint
query/category filtering logic. 15 tests total.

* fix: sync registry with official MCP API and fix stdio prefill

- Updated transport types and URLs from registry.modelcontextprotocol.io API
- GitHub: streamable-http at api.githubcopilot.com/mcp/
- GitLab: streamable-http at gitlab.com/api/v4/mcp (remote only)
- Atlassian: SSE at mcp.atlassian.com/v1/sse (remote only)
- Linear: SSE at mcp.linear.app/sse (remote only)
- Notion: SSE at mcp.notion.com/sse (remote only)
- Stripe: streamable-http at mcp.stripe.com (remote only)
- Exa: streamable-http at mcp.exa.ai/mcp (remote only)
- Cloudflare: SSE at bindings.mcp.cloudflare.com/sse (remote only)
- Sentry: stdio via @sentry/mcp-server (npm, correct package)
- Snowflake: stdio via snowflake-labs-mcp (pypi/uvx, not npm)
- Brave Search: stdio via @brave/brave-search-mcp-server (correct package)
- Fixed stdio prefill to generate stdio_config JSON instead of separate fields
- Discovery modal matches create modal width and header style
- Back arrow positioned on left of create modal header

* Update ui/litellm-dashboard/src/components/mcp_tools/mcp_discovery.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/management_endpoints/mcp_management_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: address Greptile review feedback

- Move `import json` and `import os` to module top level
- Move mcp_registry.json into litellm/proxy/ for pip distribution
- Fix `Text` component: destructure from antd Typography instead of deprecated Tremor
- Update test fixture path to match new registry location

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 17:59:21 -08:00
yuneng-jiang
111593397a fixing core proxy tests 2026-02-12 17:54:32 -08:00
yuneng-jiang
8d10311b4b content filter test fix 2026-02-12 17:54:16 -08:00
yuneng-jiang
e49d094606 fix openai tests 2026-02-12 17:53:47 -08:00
yuneng-jiang
c37e3be933 fixing mcp tests 2026-02-12 17:53:40 -08:00
yuneng-jiang
c41459c8e3 fixing mistral model deprecation, cloud zero transform bug 2026-02-12 17:53:04 -08:00
yuneng-jiang
2864ce73da
Merge pull request #21022 from BerriAI/litellm_unified_ag
[Feature] Access Groups
2026-02-12 15:34:38 -08:00
Ishaan Jaff
5f40f93846
fix: MCP - inject NPM_CONFIG_CACHE into STDIO MCP subprocess env (#21069)
* fix: inject NPM_CONFIG_CACHE into STDIO MCP subprocess env for Docker

npm/npx needs a writable cache directory. In containers the default
(~/.npm) may not exist or be read-only, causing STDIO MCP servers
launched via npx to fail with ENOENT. Inject NPM_CONFIG_CACHE=/tmp/.npm_mcp_cache
into the subprocess env when not already set.

* test: add unit test for NPM_CONFIG_CACHE injection in STDIO MCP

Verifies that NPM_CONFIG_CACHE is auto-injected when not set, and
preserved when explicitly provided. Also moves the import to module
level per code style rules.

* Update litellm/constants.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 15:11:37 -08:00
yuneng-jiang
a45028f623 Merge remote-tracking branch 'origin' into litellm_unified_ag 2026-02-12 14:32:52 -08:00
yuneng-jiang
ea8c89ea3f rename file and add tests 2026-02-12 14:05:51 -08:00
yuneng-jiang
c34cdb29cd remove double auth and add alias 2026-02-12 13:40:58 -08:00
Ishaan Jaff
736daf0a7d
[Feat] Adds Shell tool support for the OpenAI Responses API (#21063)
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool

* add SHELL tool

* test_responses_api_shell_tool

* add SHELL_CALL_IN_PROGRESS

* add SHELL_CALL_IN_PROGRESS events

* TestOpenAIResponsesAPITest

* transform_streaming_response

* test_responses_api_shell_tool_streaming_sees_shell_output

* test_responses_api_shell_tool_streaming_sees_shell_output

* test_responses_api_shell_tool

* docs fix
2026-02-12 13:04:29 -08:00
yuneng-jiang
e6df587bfb adding tests and fixing prisma lookup table 2026-02-12 12:48:05 -08:00
yuneng-jiang
df15456bcc
Merge pull request #20598 from muraliavarma/fix/team-update-empty-premium-fields-403
fix(proxy): skip premium check for empty metadata fields on team/key update
2026-02-12 10:10:27 -08:00
Ishaan Jaff
3d9b145b04
[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058)
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool
2026-02-12 10:00:30 -08:00
Sameer Kankute
e68b970953
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 21:14:08 +05:30
Sameer Kankute
59d6ab8a00
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 20:04:46 +05:30
jquinter
e78d17cd0a Fix/mcp health check cancelled error (#19851)
* Fix MCP health check CancelledError handling for parallel test execution

Add asyncio.CancelledError handler in health_check_server() and missing
@pytest.mark.asyncio decorator on test_mcp_server_manager_config_integration_with_database.

In Python 3.8+, CancelledError inherits from BaseException, not Exception,
so it bypassed the generic exception handler when pytest-xdist cancels
running tasks after a failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Regenerate poetry.lock to resolve merge conflict markers

The lock file had unresolved conflict markers from a previous merge,
causing poetry to fail with "Invalid statement (at line 8534, column 1)".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 19:40:08 +05:30
jquinter
199fbabfb3 Fix MCP streaming test skip logic and metadata None check (#20428)
- Fix skip condition to detect claude models (was only checking for
    "anthropic" in model name, missing "claude-haiku-4-5")
  - Add missing skip for OpenAI tests when OPENAI_API_KEY is not set
  - Fix TypeError in utils.py when metadata is explicitly None instead
    of missing (use `or {}` fallback)
2026-02-12 19:39:05 +05:30
Zero Clover
7044512407 Fix responses bridge metadata isolation (#20484) 2026-02-12 19:39:05 +05:30
Dennis Zheleznyak
19d290a988 fix(sagemaker): Support TEI raw array response format for embeddings (#20487)
HuggingFace Text Embeddings Inference (TEI) returns embeddings as raw
arrays [[0.1, 0.2, ...]] instead of wrapped format {"embedding": [...]}.

This change handles both formats:
- Raw array: [[...]] (TEI, some HF models)
- Wrapped: {"embedding": [[...]]} (standard HF format)

Fixes SagemakerError: "HF response missing 'embedding' field" when using
TEI containers on SageMaker.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 19:39:05 +05:30
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492)
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
2026-02-12 19:39:04 +05:30
Harshit Jain
c867740d5e
Merge pull request #20481 from Harshit28j/litellm_aws_rotation_fix
Fix authorization issues, same alias; verified working
2026-02-12 09:36:10 +05:30
Krish Dholakia
af3acdda18
Guardrails - add toxic/abusive content filter guardrails 2026-02-11 18:08:16 -08:00
Krish Dholakia
5736fd32d9
MCP fixes
* fix(oldteams.tsx): show policies when creating

* fix(proxy/_types.py): ensure mcp rest endpoints can be called by virtual key

ensures UI works with virtual key testing mcp endpoints

* refactor: migrate get object permissions table logic to happen in user api key auth - allows functions to trust user api key object they receive has what they need

* fix(rest_endpoints.py): filter for allowed tools based on what key has access to

* fix(mcp_server_manager.py): ensure only allowed MCP's are returned to the user, via rest endpoints
2026-02-11 18:07:24 -08:00
yuneng-jiang
5fa5657476
Merge pull request #20991 from BerriAI/litellm_spend_logs_sort_03
[Feature] Allow Sorting on /spend/logs/ui
2026-02-11 14:11:11 -08:00
yuneng-jiang
55225a09cf Allow sorting on /spend/logs/ui 2026-02-11 12:48:33 -08:00
Ishaan Jaff
2b00466d3a
fix: support prompt_cache_key for OpenAI and Azure chat completions (#20989)
* fix:fix: prompt_cache_key OAI + Azure OpenAI

* test_prompt_cache_key_supported

* test_azure_openai_with_prompt_cache_key

* fix: remove unnecessary async from test_azure_openai_with_prompt_cache_key

Addresses Greptile feedback: litellm.completion() is synchronous, so
async def is unnecessary and would silently pass without running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused filter_and_transform_beta_headers imports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test_azure_openai_with_prompt_cache_key

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:25:29 -08:00
yuneng-jiang
b88b2520c0
Merge pull request #20987 from BerriAI/litellm_inv_user_org
[Feature] Allow Organization and Team Admins to call /invitation/new
2026-02-11 11:38:15 -08:00
yuneng-jiang
40295595c7 allow team and org admins to call invitation/new 2026-02-11 11:23:27 -08:00
Ishaan Jaff
9975a9e3d4
fix: support Azure AD token auth for non-Claude azure_ai models (#20981)
* fix: _should_use_api_key_header

* test_azure_ai_validate_environment_with_api_key

* fix: remove unused top-level RouteChecks import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add missing env keys to config_settings reference

Add MODEL_COST_MAP_MIN_MODEL_COUNT, MODEL_COST_MAP_MAX_SHRINK_RATIO,
and MAX_POLICY_ESTIMATE_IMPACT_ROWS to the environment variables
reference table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:48:44 -08:00
michelligabriele
81a1cb1318
fix(mcp): merge query params when authorization_url already contains them (#20968) 2026-02-11 08:43:19 -08:00
Sameer Kankute
c27650c4cf
Merge pull request #20935 from BerriAI/litellm_anthropic_filter_bedrock_headers
[Feat]Managing Anthropic Beta Headers
2026-02-11 18:19:01 +05:30
Sameer Kankute
5399dbd1c1 Fix beta header old tests 2026-02-11 18:13:36 +05:30
Cas Steigstra
2ef0d9e80a fix: export PermissionDeniedError from litellm.__init__
PermissionDeniedError (403) is defined in litellm/exceptions.py but was
never added to the import block in litellm/__init__.py. This makes it
the only standard HTTP error exception not accessible as
litellm.PermissionDeniedError, forcing users to import from
litellm.exceptions directly.

Fixes #20959
2026-02-11 13:39:19 +01:00
Sameer Kankute
b92bf3756a Fix beta header old tests 2026-02-11 18:00:25 +05:30
Sameer Kankute
a9255349b6
Merge pull request #20938 from skylarkoo7/fix-20885-deepseek-model-metadata
fix(model-info): sync DeepSeek model metadata and add bare-name fallback
2026-02-11 17:15:03 +05:30
Sameer Kankute
a7b63d3895
Merge pull request #20958 from gotsysdba/main
Fix OCI Cohere system messages by populating preambleOverride
2026-02-11 17:07:34 +05:30
Sameer Kankute
b962b2cc85 Fix beta header old tests 2026-02-11 16:57:52 +05:30
Sameer Kankute
53bc1c8b79 Fix test_bedrock_messages_api_header_forwarding 2026-02-11 16:56:34 +05:30
Sameer Kankute
9083b06ba7 Fix test_provider_specific_header_in_request 2026-02-11 16:56:26 +05:30
Sameer Kankute
64355e6da4 Fix test_vertex_ai_partner_models_anthropic_remove_prompt_caching_scope_beta_header 2026-02-11 16:55:54 +05:30
gotsysdba
13392e0187
Fixes #20957 2026-02-11 11:20:18 +00:00
Sameer Kankute
375ebb333e Fix: phoenix tests issues 2026-02-11 16:13:20 +05:30
mubashir1osmani
6fdfdd27b5 fix(bedrock): address review - cross-account, SSL verify, narrow fallback
1. Cross-account false match: Added _parse_arn_account_and_role_name()
   helper that compares partition + account ID + role name (not just
   role name) to prevent same-name-different-account false matches.

2. SSL verify: _is_already_running_as_role() now passes ssl_verify to
   the STS client via self._get_ssl_verify(), consistent with all other
   boto3 client creation in this module.

3. Overbroad AccessDenied fallback: The catch in _auth_with_aws_role now
   only falls back to ambient credentials when _is_already_running_as_role
   positively confirms the caller is the target role. Genuine trust-policy
   or permission misconfigurations are re-raised.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:47 +05:30
mubashir1osmani
70a1bf92e2 fix(bedrock): skip AssumeRole when ECS/EC2 already running as target IAM role
When aws_role_name is configured but the environment (ECS task role, EC2 instance
profile) is already running as that role, AssumeRole is unnecessary and can fail
with AccessDenied. This adds same-role detection for ECS/EC2 (extending existing
IRSA support) and a fallback to ambient credentials when AssumeRole returns
AccessDenied.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:47 +05:30
mubashir1osmani
2b9b5302ef add test for dynamic project name in metadata 2026-02-11 15:45:47 +05:30
mubashir1osmani
d1c6e25723 added tests 2026-02-11 15:45:47 +05:30
mubashir1osmani
a422e8b9c9 fix(arize): allow OTEL and Arize Phoenix/Arize tracing to coexist in parallel
Arize Phoenix and Arize loggers now create dedicated TracerProviders
instead of fighting over the global singleton, and the otel callback
dedup check no longer incorrectly matches Arize subclasses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:46 +05:30