Commit graph

6181 commits

Author SHA1 Message Date
Sameer Kankute
f2c7156512 Add e2e tests for checking if all beta headers in the mapping works 2026-02-13 15:46:21 +05:30
Sameer Kankute
e0c98d62d4 Fix: LoggingWorker Missing Azure Credentials When Fetching 2026-02-13 13:05:49 +05:30
Sameer Kankute
9a9315043f Fix: Batch Rate Limiter Cannot Access User Files 2026-02-13 12:23:37 +05:30
yuneng-jiang
d701cc6588
Merge pull request #21081 from BerriAI/litellm_yj_release_changes_feb12
[Infra] CI/CD Fixes - Nightly Release Feb 12
2026-02-12 18:07:34 -08:00
Ishaan Jaff
9202e67e33
feat: MCP server discovery UI (#21079)
* feat: add curated MCP server registry for discovery UI

Curated list of 31 well-known MCP servers with names, icons,
categories, transport config, and registry URLs. Includes HTTP
endpoints for GitHub, Atlassian, Sentry, Snowflake, and Cloudflare.

* feat: add GET /v1/mcp/discover endpoint for MCP discovery

Admin-only endpoint that serves the curated MCP registry with
optional query and category filters. Used by the UI discovery modal.

* feat: add DiscoverableMCPServer types for MCP discovery

* feat: add fetchDiscoverableMCPServers network function

* feat: add MCP discovery modal component

Compact list-row layout with category filters, search, and
grouped server list. Follows dev-tool aesthetic.

* feat: wire MCP discovery modal into server management page

Add MCP Server button now opens discovery modal. Card click
pre-fills the create form. Custom Server opens blank form.

* feat: add prefill from discovery and back-to-registry link

Create form accepts prefillData from discovery selection and
shows a Browse MCP Registry link to return to discovery modal.

* test: add unit tests for MCP discovery endpoint and registry

Tests for registry JSON structure validation and endpoint
query/category filtering logic. 15 tests total.

* fix: sync registry with official MCP API and fix stdio prefill

- Updated transport types and URLs from registry.modelcontextprotocol.io API
- GitHub: streamable-http at api.githubcopilot.com/mcp/
- GitLab: streamable-http at gitlab.com/api/v4/mcp (remote only)
- Atlassian: SSE at mcp.atlassian.com/v1/sse (remote only)
- Linear: SSE at mcp.linear.app/sse (remote only)
- Notion: SSE at mcp.notion.com/sse (remote only)
- Stripe: streamable-http at mcp.stripe.com (remote only)
- Exa: streamable-http at mcp.exa.ai/mcp (remote only)
- Cloudflare: SSE at bindings.mcp.cloudflare.com/sse (remote only)
- Sentry: stdio via @sentry/mcp-server (npm, correct package)
- Snowflake: stdio via snowflake-labs-mcp (pypi/uvx, not npm)
- Brave Search: stdio via @brave/brave-search-mcp-server (correct package)
- Fixed stdio prefill to generate stdio_config JSON instead of separate fields
- Discovery modal matches create modal width and header style
- Back arrow positioned on left of create modal header

* Update ui/litellm-dashboard/src/components/mcp_tools/mcp_discovery.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/management_endpoints/mcp_management_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: address Greptile review feedback

- Move `import json` and `import os` to module top level
- Move mcp_registry.json into litellm/proxy/ for pip distribution
- Fix `Text` component: destructure from antd Typography instead of deprecated Tremor
- Update test fixture path to match new registry location

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 17:59:21 -08:00
yuneng-jiang
111593397a fixing core proxy tests 2026-02-12 17:54:32 -08:00
yuneng-jiang
8d10311b4b content filter test fix 2026-02-12 17:54:16 -08:00
yuneng-jiang
e49d094606 fix openai tests 2026-02-12 17:53:47 -08:00
yuneng-jiang
c37e3be933 fixing mcp tests 2026-02-12 17:53:40 -08:00
yuneng-jiang
c41459c8e3 fixing mistral model deprecation, cloud zero transform bug 2026-02-12 17:53:04 -08:00
yuneng-jiang
2864ce73da
Merge pull request #21022 from BerriAI/litellm_unified_ag
[Feature] Access Groups
2026-02-12 15:34:38 -08:00
Ishaan Jaff
5f40f93846
fix: MCP - inject NPM_CONFIG_CACHE into STDIO MCP subprocess env (#21069)
* fix: inject NPM_CONFIG_CACHE into STDIO MCP subprocess env for Docker

npm/npx needs a writable cache directory. In containers the default
(~/.npm) may not exist or be read-only, causing STDIO MCP servers
launched via npx to fail with ENOENT. Inject NPM_CONFIG_CACHE=/tmp/.npm_mcp_cache
into the subprocess env when not already set.

* test: add unit test for NPM_CONFIG_CACHE injection in STDIO MCP

Verifies that NPM_CONFIG_CACHE is auto-injected when not set, and
preserved when explicitly provided. Also moves the import to module
level per code style rules.

* Update litellm/constants.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 15:11:37 -08:00
yuneng-jiang
a45028f623 Merge remote-tracking branch 'origin' into litellm_unified_ag 2026-02-12 14:32:52 -08:00
yuneng-jiang
ea8c89ea3f rename file and add tests 2026-02-12 14:05:51 -08:00
yuneng-jiang
c34cdb29cd remove double auth and add alias 2026-02-12 13:40:58 -08:00
Ishaan Jaff
736daf0a7d
[Feat] Adds Shell tool support for the OpenAI Responses API (#21063)
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool

* add SHELL tool

* test_responses_api_shell_tool

* add SHELL_CALL_IN_PROGRESS

* add SHELL_CALL_IN_PROGRESS events

* TestOpenAIResponsesAPITest

* transform_streaming_response

* test_responses_api_shell_tool_streaming_sees_shell_output

* test_responses_api_shell_tool_streaming_sees_shell_output

* test_responses_api_shell_tool

* docs fix
2026-02-12 13:04:29 -08:00
yuneng-jiang
e6df587bfb adding tests and fixing prisma lookup table 2026-02-12 12:48:05 -08:00
yuneng-jiang
df15456bcc
Merge pull request #20598 from muraliavarma/fix/team-update-empty-premium-fields-403
fix(proxy): skip premium check for empty metadata fields on team/key update
2026-02-12 10:10:27 -08:00
Ishaan Jaff
3d9b145b04
[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058)
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool
2026-02-12 10:00:30 -08:00
Sameer Kankute
e68b970953
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 21:14:08 +05:30
Sameer Kankute
59d6ab8a00
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 20:04:46 +05:30
jquinter
e78d17cd0a Fix/mcp health check cancelled error (#19851)
* Fix MCP health check CancelledError handling for parallel test execution

Add asyncio.CancelledError handler in health_check_server() and missing
@pytest.mark.asyncio decorator on test_mcp_server_manager_config_integration_with_database.

In Python 3.8+, CancelledError inherits from BaseException, not Exception,
so it bypassed the generic exception handler when pytest-xdist cancels
running tasks after a failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Regenerate poetry.lock to resolve merge conflict markers

The lock file had unresolved conflict markers from a previous merge,
causing poetry to fail with "Invalid statement (at line 8534, column 1)".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 19:40:08 +05:30
jquinter
199fbabfb3 Fix MCP streaming test skip logic and metadata None check (#20428)
- Fix skip condition to detect claude models (was only checking for
    "anthropic" in model name, missing "claude-haiku-4-5")
  - Add missing skip for OpenAI tests when OPENAI_API_KEY is not set
  - Fix TypeError in utils.py when metadata is explicitly None instead
    of missing (use `or {}` fallback)
2026-02-12 19:39:05 +05:30
Zero Clover
7044512407 Fix responses bridge metadata isolation (#20484) 2026-02-12 19:39:05 +05:30
Dennis Zheleznyak
19d290a988 fix(sagemaker): Support TEI raw array response format for embeddings (#20487)
HuggingFace Text Embeddings Inference (TEI) returns embeddings as raw
arrays [[0.1, 0.2, ...]] instead of wrapped format {"embedding": [...]}.

This change handles both formats:
- Raw array: [[...]] (TEI, some HF models)
- Wrapped: {"embedding": [[...]]} (standard HF format)

Fixes SagemakerError: "HF response missing 'embedding' field" when using
TEI containers on SageMaker.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 19:39:05 +05:30
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492)
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
2026-02-12 19:39:04 +05:30
Harshit Jain
c867740d5e
Merge pull request #20481 from Harshit28j/litellm_aws_rotation_fix
Fix authorization issues, same alias; verified working
2026-02-12 09:36:10 +05:30
Krish Dholakia
af3acdda18
Guardrails - add toxic/abusive content filter guardrails 2026-02-11 18:08:16 -08:00
Krish Dholakia
5736fd32d9
MCP fixes
* fix(oldteams.tsx): show policies when creating

* fix(proxy/_types.py): ensure mcp rest endpoints can be called by virtual key

ensures UI works with virtual key testing mcp endpoints

* refactor: migrate get object permissions table logic to happen in user api key auth - allows functions to trust user api key object they receive has what they need

* fix(rest_endpoints.py): filter for allowed tools based on what key has access to

* fix(mcp_server_manager.py): ensure only allowed MCP's are returned to the user, via rest endpoints
2026-02-11 18:07:24 -08:00
yuneng-jiang
5fa5657476
Merge pull request #20991 from BerriAI/litellm_spend_logs_sort_03
[Feature] Allow Sorting on /spend/logs/ui
2026-02-11 14:11:11 -08:00
yuneng-jiang
55225a09cf Allow sorting on /spend/logs/ui 2026-02-11 12:48:33 -08:00
Ishaan Jaff
2b00466d3a
fix: support prompt_cache_key for OpenAI and Azure chat completions (#20989)
* fix:fix: prompt_cache_key OAI + Azure OpenAI

* test_prompt_cache_key_supported

* test_azure_openai_with_prompt_cache_key

* fix: remove unnecessary async from test_azure_openai_with_prompt_cache_key

Addresses Greptile feedback: litellm.completion() is synchronous, so
async def is unnecessary and would silently pass without running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused filter_and_transform_beta_headers imports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test_azure_openai_with_prompt_cache_key

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:25:29 -08:00
yuneng-jiang
b88b2520c0
Merge pull request #20987 from BerriAI/litellm_inv_user_org
[Feature] Allow Organization and Team Admins to call /invitation/new
2026-02-11 11:38:15 -08:00
yuneng-jiang
40295595c7 allow team and org admins to call invitation/new 2026-02-11 11:23:27 -08:00
Ishaan Jaff
9975a9e3d4
fix: support Azure AD token auth for non-Claude azure_ai models (#20981)
* fix: _should_use_api_key_header

* test_azure_ai_validate_environment_with_api_key

* fix: remove unused top-level RouteChecks import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add missing env keys to config_settings reference

Add MODEL_COST_MAP_MIN_MODEL_COUNT, MODEL_COST_MAP_MAX_SHRINK_RATIO,
and MAX_POLICY_ESTIMATE_IMPACT_ROWS to the environment variables
reference table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:48:44 -08:00
michelligabriele
81a1cb1318
fix(mcp): merge query params when authorization_url already contains them (#20968) 2026-02-11 08:43:19 -08:00
Sameer Kankute
c27650c4cf
Merge pull request #20935 from BerriAI/litellm_anthropic_filter_bedrock_headers
[Feat]Managing Anthropic Beta Headers
2026-02-11 18:19:01 +05:30
Sameer Kankute
5399dbd1c1 Fix beta header old tests 2026-02-11 18:13:36 +05:30
Cas Steigstra
2ef0d9e80a fix: export PermissionDeniedError from litellm.__init__
PermissionDeniedError (403) is defined in litellm/exceptions.py but was
never added to the import block in litellm/__init__.py. This makes it
the only standard HTTP error exception not accessible as
litellm.PermissionDeniedError, forcing users to import from
litellm.exceptions directly.

Fixes #20959
2026-02-11 13:39:19 +01:00
Sameer Kankute
b92bf3756a Fix beta header old tests 2026-02-11 18:00:25 +05:30
Sameer Kankute
a9255349b6
Merge pull request #20938 from skylarkoo7/fix-20885-deepseek-model-metadata
fix(model-info): sync DeepSeek model metadata and add bare-name fallback
2026-02-11 17:15:03 +05:30
Sameer Kankute
a7b63d3895
Merge pull request #20958 from gotsysdba/main
Fix OCI Cohere system messages by populating preambleOverride
2026-02-11 17:07:34 +05:30
Sameer Kankute
b962b2cc85 Fix beta header old tests 2026-02-11 16:57:52 +05:30
Sameer Kankute
53bc1c8b79 Fix test_bedrock_messages_api_header_forwarding 2026-02-11 16:56:34 +05:30
Sameer Kankute
9083b06ba7 Fix test_provider_specific_header_in_request 2026-02-11 16:56:26 +05:30
Sameer Kankute
64355e6da4 Fix test_vertex_ai_partner_models_anthropic_remove_prompt_caching_scope_beta_header 2026-02-11 16:55:54 +05:30
gotsysdba
13392e0187
Fixes #20957 2026-02-11 11:20:18 +00:00
Sameer Kankute
375ebb333e Fix: phoenix tests issues 2026-02-11 16:13:20 +05:30
mubashir1osmani
6fdfdd27b5 fix(bedrock): address review - cross-account, SSL verify, narrow fallback
1. Cross-account false match: Added _parse_arn_account_and_role_name()
   helper that compares partition + account ID + role name (not just
   role name) to prevent same-name-different-account false matches.

2. SSL verify: _is_already_running_as_role() now passes ssl_verify to
   the STS client via self._get_ssl_verify(), consistent with all other
   boto3 client creation in this module.

3. Overbroad AccessDenied fallback: The catch in _auth_with_aws_role now
   only falls back to ambient credentials when _is_already_running_as_role
   positively confirms the caller is the target role. Genuine trust-policy
   or permission misconfigurations are re-raised.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:47 +05:30
mubashir1osmani
70a1bf92e2 fix(bedrock): skip AssumeRole when ECS/EC2 already running as target IAM role
When aws_role_name is configured but the environment (ECS task role, EC2 instance
profile) is already running as that role, AssumeRole is unnecessary and can fail
with AccessDenied. This adds same-role detection for ECS/EC2 (extending existing
IRSA support) and a fallback to ambient credentials when AssumeRole returns
AccessDenied.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:47 +05:30