scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-09 22:31:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 87
33245 commits 4181 branches 1403 tags 2.3 GiB
Commit graph

33245 commits

This branch
This branch
All branches
Author SHA1 Message Date
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492) 
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
 2026-02-12 19:39:04 +05:30
Sameer Kankute
3d70a84eed
Merge pull request #21017 from BerriAI/litellm__supports_tool_search_on_bedrock 
Fix: add claude opus 4.6 in _supports_tool_search_on_bedrock
 2026-02-12 19:30:35 +05:30
Sameer Kankute
e8f97bbfca Return 200 response for stale dlete requests from client 2026-02-12 19:21:14 +05:30
yuneng-jiang
9cee51abb9
Merge pull request #21004 from BerriAI/litellm_ui_auto_router 
[Fix] UI - Add Auto Router: Description Text Input Focus
 2026-02-11 20:48:44 -08:00
yuneng-jiang
af6ff6957d
Update ui/litellm-dashboard/src/components/add_model/RouterConfigBuilder.tsx 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-11 20:47:53 -08:00
yuneng-jiang
658a283be3 unified access groups v0 2026-02-11 20:44:30 -08:00
Harshit Jain
c867740d5e
Merge pull request #20481 from Harshit28j/litellm_aws_rotation_fix 
Fix authorization issues, same alias; verified working
 2026-02-12 09:36:10 +05:30
Harshit Jain
7e958bbf67
Apply suggestion from @greptile-apps[bot] 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-12 09:03:43 +05:30
Harshit Jain
931e9ff88d
Update tests/test_litellm/proxy/test_docker_no_network_on_deploy.py 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-12 09:01:13 +05:30
Sameer Kankute
11ea8b6660 hot fix: add claude opus 4.6 in _supports_tool_search_on_bedrock 2026-02-12 08:35:58 +05:30
Harshit Jain
095fa517e0
fix lint and format errors 2026-02-12 08:09:20 +05:30
Krish Dholakia
af3acdda18
Guardrails - add toxic/abusive content filter guardrails 2026-02-11 18:08:16 -08:00
Krish Dholakia
5736fd32d9
MCP fixes 
* fix(oldteams.tsx): show policies when creating

* fix(proxy/_types.py): ensure mcp rest endpoints can be called by virtual key

ensures UI works with virtual key testing mcp endpoints

* refactor: migrate get object permissions table logic to happen in user api key auth - allows functions to trust user api key object they receive has what they need

* fix(rest_endpoints.py): filter for allowed tools based on what key has access to

* fix(mcp_server_manager.py): ensure only allowed MCP's are returned to the user, via rest endpoints
 2026-02-11 18:07:24 -08:00
Krish Dholakia
b019638716
docs: add reference to example_openai_endpoint repo for self-hosting fake OpenAI proxy (#21006) 
- Updated benchmarks.md with a section on setting up fake OpenAI endpoints
- Updated load_test.md to mention the self-hosted option
- Updated load_test_advanced.md with a tip box about the example repo

Reference: https://github.com/BerriAI/example_openai_endpoint

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-02-11 18:00:21 -08:00
Alejandro Tapia
ca94095e24 playground-test-fallbacks: Added fallback testing as a toggle to ChatUI. Purposely fails first call, then attempts to use fallbacks. 2026-02-11 17:31:05 -08:00
Harshit Jain
8249cce157 add tests for hotpath & docker container 2026-02-12 01:26:22 +00:00
yuneng-jiang
be23583d30 Merge remote-tracking branch 'origin' into litellm_ui_auto_router 2026-02-11 16:21:47 -08:00
yuneng-jiang
afebf559aa fix auto router description textinput 2026-02-11 16:20:56 -08:00
yuneng-jiang
d47b7b763f
Merge pull request #21002 from BerriAI/litellm_ui_content_filter 
[Fix] UI - Guardrail Edit: LiteLLM Content Filter Categories
 2026-02-11 16:20:34 -08:00
yuneng-jiang
83f92d6ba5 Fixing content filter guardrail update 2026-02-11 15:37:57 -08:00
Ishaan Jaff
d5e8d957b2
Rename HTTP transport type to 'Streamable HTTP (Recommended)' in MCP pages (#21000) 
- Updated create_mcp_server.tsx to show 'Streamable HTTP (Recommended)' label
- Updated mcp_server_edit.tsx to show 'Streamable HTTP (Recommended)' label
- Both Add New MCP Server and Edit MCP Server pages now display the updated label

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
 2026-02-11 15:30:38 -08:00
yuneng-jiang
5fa5657476
Merge pull request #20991 from BerriAI/litellm_spend_logs_sort_03 
[Feature] Allow Sorting on /spend/logs/ui
 2026-02-11 14:11:11 -08:00
yuneng-jiang
6c3a6ba4dc change default values 2026-02-11 13:45:11 -08:00
yuneng-jiang
55225a09cf Allow sorting on /spend/logs/ui 2026-02-11 12:48:33 -08:00
Ryan Crabbe
7dda6b0cbb perf: skip Usage Pydantic round-trip in logging payload (6.8x faster) 2026-02-11 12:39:45 -08:00
Ishaan Jaff
2b00466d3a
fix: support prompt_cache_key for OpenAI and Azure chat completions (#20989) 
* fix:fix: prompt_cache_key OAI + Azure OpenAI

* test_prompt_cache_key_supported

* test_azure_openai_with_prompt_cache_key

* fix: remove unnecessary async from test_azure_openai_with_prompt_cache_key

Addresses Greptile feedback: litellm.completion() is synchronous, so
async def is unnecessary and would silently pass without running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused filter_and_transform_beta_headers imports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test_azure_openai_with_prompt_cache_key

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-11 12:25:29 -08:00
yuneng-jiang
b88b2520c0
Merge pull request #20987 from BerriAI/litellm_inv_user_org 
[Feature] Allow Organization and Team Admins to call /invitation/new
 2026-02-11 11:38:15 -08:00
yuneng-jiang
40295595c7 allow team and org admins to call invitation/new 2026-02-11 11:23:27 -08:00
Ishaan Jaff
9975a9e3d4
fix: support Azure AD token auth for non-Claude azure_ai models (#20981) 
* fix: _should_use_api_key_header

* test_azure_ai_validate_environment_with_api_key

* fix: remove unused top-level RouteChecks import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add missing env keys to config_settings reference

Add MODEL_COST_MAP_MIN_MODEL_COUNT, MODEL_COST_MAP_MAX_SHRINK_RATIO,
and MAX_POLICY_ESTIMATE_IMPACT_ROWS to the environment variables
reference table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-11 10:48:44 -08:00
Rahul Dhanawade
49078b3c6b
Fix #20557: Fix Gemini multi-turn tool calling message formatting (#20569) 
- Add missing 'role' field to tool result ContentType objects
- Tool result messages must have role='user' to match Gemini API requirements
- Fixes 'contents[2].parts[0].data: required oneof field data' error on second turn
- Prevents uninitialized data field in tool result messages
 2026-02-11 10:27:09 -08:00
Alexsander Hamir
f32322cd41
chore: improve Semgrep rules documentation and organization (#20978) 
- Add organizing rules (language/domain structure, naming, metadata)
- Require all rules to fail CI (severity: ERROR, no warn-only)
- Move unbounded-memory.yml to python/reliability/ per structure
- Enhance unbounded-memory rule metadata (tags, confidence, source)
 2026-02-11 09:39:44 -08:00
michelligabriele
81a1cb1318
fix(mcp): merge query params when authorization_url already contains them (#20968) 2026-02-11 08:43:19 -08:00
milan-berri
78a3abe6fc
fix: enable verbose_logger when LITELLM_LOG=DEBUG for callback debug logs (#20496) 
- Add verbose_logger to imports when LITELLM_LOG=DEBUG is set
- Set verbose_logger to DEBUG level alongside verbose_proxy_logger and verbose_router_logger
- Fixes issue where callback integrations (Langsmith, Langfuse, etc.) don't show debug logs with LITELLM_LOG=DEBUG
- Makes LITELLM_LOG=DEBUG behavior consistent with --detailed_debug flag
 2026-02-11 08:42:48 -08:00
Sameer Kankute
c27650c4cf
Merge pull request #20935 from BerriAI/litellm_anthropic_filter_bedrock_headers 
[Feat]Managing Anthropic Beta Headers
 2026-02-11 18:19:01 +05:30
Sameer Kankute
44ddcfa7ac
Merge pull request #20951 from BerriAI/litellm_x-anthropic-billing 
Fix: remove x-anthropic-billing block
 2026-02-11 18:14:34 +05:30
Sameer Kankute
5399dbd1c1 Fix beta header old tests 2026-02-11 18:13:36 +05:30
Sameer Kankute
ac695373e6
Merge pull request #20960 from CSteigstra/fix/export-permission-denied-error 
fix: export PermissionDeniedError from litellm.__init__
 2026-02-11 18:10:23 +05:30
Cas Steigstra
2ef0d9e80a fix: export PermissionDeniedError from litellm.__init__ 
PermissionDeniedError (403) is defined in litellm/exceptions.py but was
never added to the import block in litellm/__init__.py. This makes it
the only standard HTTP error exception not accessible as
litellm.PermissionDeniedError, forcing users to import from
litellm.exceptions directly.

Fixes #20959
 2026-02-11 13:39:19 +01:00
Sameer Kankute
b92bf3756a Fix beta header old tests 2026-02-11 18:00:25 +05:30
Sameer Kankute
a9255349b6
Merge pull request #20938 from skylarkoo7/fix-20885-deepseek-model-metadata 
fix(model-info): sync DeepSeek model metadata and add bare-name fallback
 2026-02-11 17:15:03 +05:30
Sameer Kankute
a7b63d3895
Merge pull request #20958 from gotsysdba/main 
Fix OCI Cohere system messages by populating preambleOverride
 2026-02-11 17:07:34 +05:30
Sameer Kankute
b962b2cc85 Fix beta header old tests 2026-02-11 16:57:52 +05:30
Sameer Kankute
53bc1c8b79 Fix test_bedrock_messages_api_header_forwarding 2026-02-11 16:56:34 +05:30
Sameer Kankute
9083b06ba7 Fix test_provider_specific_header_in_request 2026-02-11 16:56:26 +05:30
Sameer Kankute
64355e6da4 Fix test_vertex_ai_partner_models_anthropic_remove_prompt_caching_scope_beta_header 2026-02-11 16:55:54 +05:30
Sameer Kankute
936cebb7d0 Fix mcp and structured output header tests 2026-02-11 16:51:44 +05:30
gotsysdba
13392e0187
Fixes #20957 2026-02-11 11:20:18 +00:00
Sameer Kankute
31a2bbfe85
Merge pull request #20931 from BerriAI/litellm_oss_staging_02_10_2026 
Litellm oss staging 02 10 2026
 2026-02-11 16:28:21 +05:30
Sameer Kankute
375ebb333e Fix: phoenix tests issues 2026-02-11 16:13:20 +05:30
Sameer Kankute
d3426d55f9 Fix: litellm import error 2026-02-11 16:09:49 +05:30