Commit graph

6178 commits

Author SHA1 Message Date
yuneng-jiang
231aedeeff DO NOT MERGE TO MAIN - skipping main schema check 2026-02-12 17:01:31 -08:00
yuneng-jiang
e4c9f9693b DO NOT MERGE TO MAIN - skipping tests without model access 2026-02-12 16:40:42 -08:00
yuneng-jiang
495814c886 fixing core proxy tests 2026-02-12 16:27:44 -08:00
yuneng-jiang
0ad5748b36 Revert "Merge pull request #20598 from muraliavarma/fix/team-update-empty-premium-fields-403"
This reverts commit df15456bcc, reversing
changes made to 89565c97cc.
2026-02-12 16:22:29 -08:00
yuneng-jiang
7021da85d2 content filter test fix 2026-02-12 15:31:19 -08:00
yuneng-jiang
1af3db4460 DO NOT MERGE TO MAIN - skipping model tests we do not have access to 2026-02-12 14:27:51 -08:00
yuneng-jiang
ac63e619fb Revert "[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058)"
This reverts commit 3d9b145b04.
2026-02-12 13:47:05 -08:00
yuneng-jiang
4b1cd28531 revert changes to azure files 2026-02-12 13:46:43 -08:00
yuneng-jiang
1872906e6c test fixing azure 2026-02-12 12:16:04 -08:00
yuneng-jiang
d64ea0ee30 fixing azure bad request error strategies 2026-02-12 11:55:36 -08:00
yuneng-jiang
11f9b47710 fix openai tests 2026-02-12 11:49:37 -08:00
yuneng-jiang
fd5bbe070b fixing mcp tests 2026-02-12 11:42:41 -08:00
yuneng-jiang
206b6bad41 fixing azure bad request error 2026-02-12 11:33:45 -08:00
yuneng-jiang
295d1d443b fixing mistral model deprecation, cloud zero transform bug 2026-02-12 11:33:28 -08:00
yuneng-jiang
df15456bcc
Merge pull request #20598 from muraliavarma/fix/team-update-empty-premium-fields-403
fix(proxy): skip premium check for empty metadata fields on team/key update
2026-02-12 10:10:27 -08:00
Ishaan Jaff
3d9b145b04
[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058)
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool
2026-02-12 10:00:30 -08:00
Sameer Kankute
e68b970953
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 21:14:08 +05:30
Sameer Kankute
59d6ab8a00
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 20:04:46 +05:30
jquinter
e78d17cd0a Fix/mcp health check cancelled error (#19851)
* Fix MCP health check CancelledError handling for parallel test execution

Add asyncio.CancelledError handler in health_check_server() and missing
@pytest.mark.asyncio decorator on test_mcp_server_manager_config_integration_with_database.

In Python 3.8+, CancelledError inherits from BaseException, not Exception,
so it bypassed the generic exception handler when pytest-xdist cancels
running tasks after a failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Regenerate poetry.lock to resolve merge conflict markers

The lock file had unresolved conflict markers from a previous merge,
causing poetry to fail with "Invalid statement (at line 8534, column 1)".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 19:40:08 +05:30
jquinter
199fbabfb3 Fix MCP streaming test skip logic and metadata None check (#20428)
- Fix skip condition to detect claude models (was only checking for
    "anthropic" in model name, missing "claude-haiku-4-5")
  - Add missing skip for OpenAI tests when OPENAI_API_KEY is not set
  - Fix TypeError in utils.py when metadata is explicitly None instead
    of missing (use `or {}` fallback)
2026-02-12 19:39:05 +05:30
Zero Clover
7044512407 Fix responses bridge metadata isolation (#20484) 2026-02-12 19:39:05 +05:30
Dennis Zheleznyak
19d290a988 fix(sagemaker): Support TEI raw array response format for embeddings (#20487)
HuggingFace Text Embeddings Inference (TEI) returns embeddings as raw
arrays [[0.1, 0.2, ...]] instead of wrapped format {"embedding": [...]}.

This change handles both formats:
- Raw array: [[...]] (TEI, some HF models)
- Wrapped: {"embedding": [[...]]} (standard HF format)

Fixes SagemakerError: "HF response missing 'embedding' field" when using
TEI containers on SageMaker.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 19:39:05 +05:30
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492)
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
2026-02-12 19:39:04 +05:30
Harshit Jain
c867740d5e
Merge pull request #20481 from Harshit28j/litellm_aws_rotation_fix
Fix authorization issues, same alias; verified working
2026-02-12 09:36:10 +05:30
Krish Dholakia
af3acdda18
Guardrails - add toxic/abusive content filter guardrails 2026-02-11 18:08:16 -08:00
Krish Dholakia
5736fd32d9
MCP fixes
* fix(oldteams.tsx): show policies when creating

* fix(proxy/_types.py): ensure mcp rest endpoints can be called by virtual key

ensures UI works with virtual key testing mcp endpoints

* refactor: migrate get object permissions table logic to happen in user api key auth - allows functions to trust user api key object they receive has what they need

* fix(rest_endpoints.py): filter for allowed tools based on what key has access to

* fix(mcp_server_manager.py): ensure only allowed MCP's are returned to the user, via rest endpoints
2026-02-11 18:07:24 -08:00
yuneng-jiang
5fa5657476
Merge pull request #20991 from BerriAI/litellm_spend_logs_sort_03
[Feature] Allow Sorting on /spend/logs/ui
2026-02-11 14:11:11 -08:00
yuneng-jiang
55225a09cf Allow sorting on /spend/logs/ui 2026-02-11 12:48:33 -08:00
Ishaan Jaff
2b00466d3a
fix: support prompt_cache_key for OpenAI and Azure chat completions (#20989)
* fix:fix: prompt_cache_key OAI + Azure OpenAI

* test_prompt_cache_key_supported

* test_azure_openai_with_prompt_cache_key

* fix: remove unnecessary async from test_azure_openai_with_prompt_cache_key

Addresses Greptile feedback: litellm.completion() is synchronous, so
async def is unnecessary and would silently pass without running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused filter_and_transform_beta_headers imports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test_azure_openai_with_prompt_cache_key

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:25:29 -08:00
yuneng-jiang
b88b2520c0
Merge pull request #20987 from BerriAI/litellm_inv_user_org
[Feature] Allow Organization and Team Admins to call /invitation/new
2026-02-11 11:38:15 -08:00
yuneng-jiang
40295595c7 allow team and org admins to call invitation/new 2026-02-11 11:23:27 -08:00
Ishaan Jaff
9975a9e3d4
fix: support Azure AD token auth for non-Claude azure_ai models (#20981)
* fix: _should_use_api_key_header

* test_azure_ai_validate_environment_with_api_key

* fix: remove unused top-level RouteChecks import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add missing env keys to config_settings reference

Add MODEL_COST_MAP_MIN_MODEL_COUNT, MODEL_COST_MAP_MAX_SHRINK_RATIO,
and MAX_POLICY_ESTIMATE_IMPACT_ROWS to the environment variables
reference table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:48:44 -08:00
michelligabriele
81a1cb1318
fix(mcp): merge query params when authorization_url already contains them (#20968) 2026-02-11 08:43:19 -08:00
Sameer Kankute
c27650c4cf
Merge pull request #20935 from BerriAI/litellm_anthropic_filter_bedrock_headers
[Feat]Managing Anthropic Beta Headers
2026-02-11 18:19:01 +05:30
Sameer Kankute
5399dbd1c1 Fix beta header old tests 2026-02-11 18:13:36 +05:30
Cas Steigstra
2ef0d9e80a fix: export PermissionDeniedError from litellm.__init__
PermissionDeniedError (403) is defined in litellm/exceptions.py but was
never added to the import block in litellm/__init__.py. This makes it
the only standard HTTP error exception not accessible as
litellm.PermissionDeniedError, forcing users to import from
litellm.exceptions directly.

Fixes #20959
2026-02-11 13:39:19 +01:00
Sameer Kankute
b92bf3756a Fix beta header old tests 2026-02-11 18:00:25 +05:30
Sameer Kankute
a9255349b6
Merge pull request #20938 from skylarkoo7/fix-20885-deepseek-model-metadata
fix(model-info): sync DeepSeek model metadata and add bare-name fallback
2026-02-11 17:15:03 +05:30
Sameer Kankute
a7b63d3895
Merge pull request #20958 from gotsysdba/main
Fix OCI Cohere system messages by populating preambleOverride
2026-02-11 17:07:34 +05:30
Sameer Kankute
b962b2cc85 Fix beta header old tests 2026-02-11 16:57:52 +05:30
Sameer Kankute
53bc1c8b79 Fix test_bedrock_messages_api_header_forwarding 2026-02-11 16:56:34 +05:30
Sameer Kankute
9083b06ba7 Fix test_provider_specific_header_in_request 2026-02-11 16:56:26 +05:30
Sameer Kankute
64355e6da4 Fix test_vertex_ai_partner_models_anthropic_remove_prompt_caching_scope_beta_header 2026-02-11 16:55:54 +05:30
gotsysdba
13392e0187
Fixes #20957 2026-02-11 11:20:18 +00:00
Sameer Kankute
375ebb333e Fix: phoenix tests issues 2026-02-11 16:13:20 +05:30
mubashir1osmani
6fdfdd27b5 fix(bedrock): address review - cross-account, SSL verify, narrow fallback
1. Cross-account false match: Added _parse_arn_account_and_role_name()
   helper that compares partition + account ID + role name (not just
   role name) to prevent same-name-different-account false matches.

2. SSL verify: _is_already_running_as_role() now passes ssl_verify to
   the STS client via self._get_ssl_verify(), consistent with all other
   boto3 client creation in this module.

3. Overbroad AccessDenied fallback: The catch in _auth_with_aws_role now
   only falls back to ambient credentials when _is_already_running_as_role
   positively confirms the caller is the target role. Genuine trust-policy
   or permission misconfigurations are re-raised.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:47 +05:30
mubashir1osmani
70a1bf92e2 fix(bedrock): skip AssumeRole when ECS/EC2 already running as target IAM role
When aws_role_name is configured but the environment (ECS task role, EC2 instance
profile) is already running as that role, AssumeRole is unnecessary and can fail
with AccessDenied. This adds same-role detection for ECS/EC2 (extending existing
IRSA support) and a fallback to ambient credentials when AssumeRole returns
AccessDenied.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:47 +05:30
mubashir1osmani
2b9b5302ef add test for dynamic project name in metadata 2026-02-11 15:45:47 +05:30
mubashir1osmani
d1c6e25723 added tests 2026-02-11 15:45:47 +05:30
mubashir1osmani
a422e8b9c9 fix(arize): allow OTEL and Arize Phoenix/Arize tracing to coexist in parallel
Arize Phoenix and Arize loggers now create dedicated TracerProviders
instead of fighting over the global singleton, and the otel callback
dedup check no longer incorrectly matches Arize subclasses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-11 15:45:46 +05:30