scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88
32127 commits 2519 branches 1411 tags 3.1 GiB
Commit graph

32127 commits

This branch
This branch
All branches
Author SHA1 Message Date
yuneng-jiang
231aedeeff DO NOT MERGE TO MAIN - skipping main schema check 2026-02-12 17:01:31 -08:00
yuneng-jiang
e4c9f9693b DO NOT MERGE TO MAIN - skipping tests without model access 2026-02-12 16:40:42 -08:00
yuneng-jiang
495814c886 fixing core proxy tests 2026-02-12 16:27:44 -08:00
yuneng-jiang
0ad5748b36 Revert "Merge pull request #20598 from muraliavarma/fix/team-update-empty-premium-fields-403" 
This reverts commit df15456bcc, reversing
changes made to 89565c97cc.
 2026-02-12 16:22:29 -08:00
yuneng-jiang
7021da85d2 content filter test fix 2026-02-12 15:31:19 -08:00
yuneng-jiang
1f61794c12 DO NOT MERGE TO MAIN - remove publish to pypi due to remote source updating 2026-02-12 15:26:48 -08:00
yuneng-jiang
5d93af47ab Revert "Add semgrep & Fix OOMs (#20912)" 
This reverts commit b7993b14cf.
 2026-02-12 15:25:34 -08:00
yuneng-jiang
46a5626db6 DO NOT MERGE TO MAIN - removing mapped tests core due to flakiness 2026-02-12 14:30:01 -08:00
yuneng-jiang
1af3db4460 DO NOT MERGE TO MAIN - skipping model tests we do not have access to 2026-02-12 14:27:51 -08:00
yuneng-jiang
ac63e619fb Revert "[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058)" 
This reverts commit 3d9b145b04.
 2026-02-12 13:47:05 -08:00
yuneng-jiang
4b1cd28531 revert changes to azure files 2026-02-12 13:46:43 -08:00
yuneng-jiang
1872906e6c test fixing azure 2026-02-12 12:16:04 -08:00
yuneng-jiang
a3f6cbf6bb bump: version 1.81.10 → 1.81.11 2026-02-12 11:57:48 -08:00
yuneng-jiang
d64ea0ee30 fixing azure bad request error strategies 2026-02-12 11:55:36 -08:00
yuneng-jiang
11f9b47710 fix openai tests 2026-02-12 11:49:37 -08:00
yuneng-jiang
fd5bbe070b fixing mcp tests 2026-02-12 11:42:41 -08:00
yuneng-jiang
206b6bad41 fixing azure bad request error 2026-02-12 11:33:45 -08:00
yuneng-jiang
295d1d443b fixing mistral model deprecation, cloud zero transform bug 2026-02-12 11:33:28 -08:00
yuneng-jiang
15352c4bc3 adding key envs to docs 2026-02-12 11:15:19 -08:00
yuneng-jiang
71a397230b bumping pillow and cryptography for security fixes 2026-02-12 11:05:58 -08:00
yuneng-jiang
ed0c5fdbee fix mypy linting 2026-02-12 10:51:21 -08:00
yuneng-jiang
aef37075cf Merge pull request #21061 from BerriAI/migration_yj_feb12 
[Infra] Add Mmigration for Tags Adjustment on Policy Table
 2026-02-12 10:36:57 -08:00
yuneng-jiang
8a9b41a5cf fix ruff check 2026-02-12 10:21:13 -08:00
yuneng-jiang
df15456bcc
Merge pull request #20598 from muraliavarma/fix/team-update-empty-premium-fields-403 
fix(proxy): skip premium check for empty metadata fields on team/key update
 2026-02-12 10:10:27 -08:00
Ishaan Jaff
89565c97cc
[Feat] AI Gateway - Add Tracing for MCP Calls running through AI Gateway (#21018) 
* commit new expansion

* fix MCP

* fix: LiteLLMProxyRequestSetup

* _process_mcp_tools_without_openai_transform

* UI fixes

* UI refactor view logs/sessions

* index

* _add_mcp_tool_metadata_to_final_chunk

* add badges

* add getEventDisplayName

* ui fixes

* backend fix

* fix

* UI fix

* UI fix

* fix row

* fix: address Greptile review feedback on PR #21018 (#21057)

- Fix session time range calculation: use Math.min/Math.max across all
  entries instead of relying on array order (sessionLogs is sorted by
  type, not time).

Other Greptile comments were already addressed in the branch:
- LogDetailContent.tsx exists
- Clipboard call already wrapped in try/catch
- Dedup already uses O(1) Map lookup
- model_dump() serialization is documented
- GROUP BY performance comment already present

---------

Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
 2026-02-12 10:02:03 -08:00
Ishaan Jaff
3d9b145b04
[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool
 2026-02-12 10:00:30 -08:00
Krrish Dholakia
f5382ebac9 docs: fix docs 2026-02-12 08:45:57 -08:00
Sameer Kankute
556bcd7203
Merge pull request #21055 from BerriAI/litellm_day_0_MiniMax-M2.1 
fix docs
 2026-02-12 22:04:23 +05:30
Sameer Kankute
9f15eca6b6 fix docs 2026-02-12 22:03:21 +05:30
Sameer Kankute
4e62386c65
Merge pull request #21054 from BerriAI/litellm_day_0_MiniMax-M2.1 
Add support for MiniMax-M2.1 and MiniMax-M2.1-lightining
 2026-02-12 21:51:46 +05:30
Sameer Kankute
7a3b227aeb
Apply suggestion from @greptile-apps[bot] 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-12 21:50:55 +05:30
Sameer Kankute
9b32c516ad Add support for MiniMax-M2.1 and MiniMax-M2.1-lightining 2026-02-12 21:45:49 +05:30
Sameer Kankute
a7179797f7
Merge pull request #20930 from BerriAI/litellm_oss_staging_02_11_2026 
oss staging 02 / 11/ 2026
 2026-02-12 21:28:59 +05:30
Sameer Kankute
5471b14c56
Merge pull request #21051 from BerriAI/revert-20569-fix/20557-gemini-multiturn-tool-calling 
Revert "Fix #20557: Fix Gemini multi-turn tool calling message formatting"
 2026-02-12 21:17:04 +05:30
Sameer Kankute
4931bac3da
Revert "Fix #20557: Fix Gemini multi-turn tool calling message formatting (#2…" 
This reverts commit 49078b3c6b.
 2026-02-12 21:16:33 +05:30
Sameer Kankute
e68b970953
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 21:14:08 +05:30
Sameer Kankute
3d7126b11a
Merge pull request #20587 from BerriAI/litellm_oss_staging_02_06_2026 
Litellm oss staging 02 06 2026
 2026-02-12 20:07:35 +05:30
Sameer Kankute
59d6ab8a00
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 20:04:46 +05:30
jquinter
e78d17cd0a Fix/mcp health check cancelled error (#19851) 
* Fix MCP health check CancelledError handling for parallel test execution

Add asyncio.CancelledError handler in health_check_server() and missing
@pytest.mark.asyncio decorator on test_mcp_server_manager_config_integration_with_database.

In Python 3.8+, CancelledError inherits from BaseException, not Exception,
so it bypassed the generic exception handler when pytest-xdist cancels
running tasks after a failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Regenerate poetry.lock to resolve merge conflict markers

The lock file had unresolved conflict markers from a previous merge,
causing poetry to fail with "Invalid statement (at line 8534, column 1)".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-12 19:40:08 +05:30
jquinter
2875fe8e49 ci: add matrix-based parallel test workflow (#19942) 
Split tests/test_litellm into 10 parallel CI jobs using GitHub Actions
matrix strategy to reduce PR feedback time from ~25 min to ~8-10 min.

Changes:
- Add new test-litellm-matrix.yml workflow with 10 matrix jobs:
  - llms (~225 files, 4 workers)
  - proxy-guardrails (~51 files, 4 workers)
  - proxy-core (~52 files, 4 workers)
  - proxy-misc (~77 files, 4 workers)
  - integrations (~60 files, 4 workers)
  - core-utils (~32 files, 2 workers)
  - other (~69 files, 4 workers) - includes all previously uncovered dirs
  - root (~34 files, 4 workers)
  - proxy-unit-a (~20 files, 2 workers)
  - proxy-unit-b (~28 files, 2 workers)

- Deprecate test-litellm.yml (moved to workflow_dispatch for manual use)

- Add matching Makefile targets for local testing:
  - make test-unit-llms
  - make test-unit-proxy-guardrails
  - make test-unit-proxy-core
  - make test-unit-proxy-misc
  - make test-unit-integrations
  - make test-unit-core-utils
  - make test-unit-other
  - make test-unit-root
  - make test-proxy-unit-a
  - make test-proxy-unit-b

Benefits:
- ~3x faster wall-clock time through parallelization
- Dependency caching for faster subsequent runs
- Concurrency control to cancel stale runs
- Better failure isolation per test group

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-12 19:39:05 +05:30
jquinter
199fbabfb3 Fix MCP streaming test skip logic and metadata None check (#20428) 
- Fix skip condition to detect claude models (was only checking for
    "anthropic" in model name, missing "claude-haiku-4-5")
  - Add missing skip for OpenAI tests when OPENAI_API_KEY is not set
  - Fix TypeError in utils.py when metadata is explicitly None instead
    of missing (use `or {}` fallback)
 2026-02-12 19:39:05 +05:30
Zero Clover
7044512407 Fix responses bridge metadata isolation (#20484) 2026-02-12 19:39:05 +05:30
Dennis Zheleznyak
19d290a988 fix(sagemaker): Support TEI raw array response format for embeddings (#20487) 
HuggingFace Text Embeddings Inference (TEI) returns embeddings as raw
arrays [[0.1, 0.2, ...]] instead of wrapped format {"embedding": [...]}.

This change handles both formats:
- Raw array: [[...]] (TEI, some HF models)
- Wrapped: {"embedding": [[...]]} (standard HF format)

Fixes SagemakerError: "HF response missing 'embedding' field" when using
TEI containers on SageMaker.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-12 19:39:05 +05:30
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492) 
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
 2026-02-12 19:39:04 +05:30
Sameer Kankute
3d70a84eed
Merge pull request #21017 from BerriAI/litellm__supports_tool_search_on_bedrock 
Fix: add claude opus 4.6 in _supports_tool_search_on_bedrock
 2026-02-12 19:30:35 +05:30
yuneng-jiang
9cee51abb9
Merge pull request #21004 from BerriAI/litellm_ui_auto_router 
[Fix] UI - Add Auto Router: Description Text Input Focus
 2026-02-11 20:48:44 -08:00
yuneng-jiang
af6ff6957d
Update ui/litellm-dashboard/src/components/add_model/RouterConfigBuilder.tsx 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-11 20:47:53 -08:00
Harshit Jain
c867740d5e
Merge pull request #20481 from Harshit28j/litellm_aws_rotation_fix 
Fix authorization issues, same alias; verified working
 2026-02-12 09:36:10 +05:30
Sameer Kankute
11ea8b6660 hot fix: add claude opus 4.6 in _supports_tool_search_on_bedrock 2026-02-12 08:35:58 +05:30
Krish Dholakia
af3acdda18
Guardrails - add toxic/abusive content filter guardrails 2026-02-11 18:08:16 -08:00