scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36
32494 commits 4135 branches 1402 tags 2.1 GiB
Commit graph

5481 commits

Author SHA1 Message Date
Yuneng Jiang
0bc2c48dce
chore: fixes 2026-04-04 22:37:10 -07:00
Harshit Jain
dd2111e664
Merge pull request #20321 from Harshit28j/litellm_fix-virtual-key-grace-period 
fix: virutal key grace period from env/UI
 2026-02-16 19:36:18 +05:30
Sameer Kankute
86a254a215
Merge pull request #21307 from BerriAI/litellm_oss_staging_02_14_20262 
Litellm oss staging 02 14 20262
 2026-02-16 19:10:59 +05:30
Sameer Kankute
be9df253dd
Update docs/my-website/sidebars.js 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-16 19:03:15 +05:30
Sameer Kankute
5b6e232da6 Add doc for OpenAI Agents SDK with LiteLLM 2026-02-16 19:00:53 +05:30
Sameer Kankute
a38b4c8924 Add doc for OpenAI Agents SDK with LiteLLM 2026-02-16 18:59:39 +05:30
Fly
9c71d8b61b feat(s3): add support for virtual-hosted-style URLs (#21094) 
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).

This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
 2026-02-16 18:26:44 +05:30
Harshit Jain
56b53cb502
Merge branch 'main' into litellm_fix-virtual-key-grace-period 2026-02-15 08:09:13 +05:30
yuneng-jiang
227b3551a6 access groups docs 2026-02-14 18:00:22 -08:00
Ishaan Jaffer
45235ce5d5 docs 2026-02-14 17:09:13 -08:00
Ishaan Jaffer
8a341ebd5d docs fix 2026-02-14 17:09:13 -08:00
Ishaan Jaffer
74a9900fe6 docs fix 2026-02-14 16:58:00 -08:00
Ishaan Jaffer
c93e0dcf9f docs 2026-02-14 16:56:38 -08:00
Ishaan Jaffer
ff502e8b91 DEFAULT_ACCESS_GROUP_CACHE_TTL 2026-02-14 16:48:57 -08:00
Ishaan Jaffer
d24869c7ce docs fix 2026-02-14 16:47:01 -08:00
Ishaan Jaffer
9e7dde508f docs fix 2026-02-14 16:45:57 -08:00
Ishaan Jaff
eb432bf911
Litellm notes 181 12 (#21231) 
* docs

* docs

* docs

* docs

* fix SCHEMA

* LiteLLM_PolicyTable

* litellm-proxy-extras = {version = "0.4.39",
 2026-02-14 16:39:22 -08:00
Ishaan Jaffer
80b080e00c code QA fx 2026-02-14 13:29:03 -08:00
Alexsander Hamir
30b28da2b7
Add pyroscope for observability (#21167) 
* Pyroscope: require PYROSCOPE_APP_NAME and PYROSCOPE_SERVER_ADDRESS, add UTF-8 locale hint

- No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled
- Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections
- Startup message suggests PYTHONUTF8=1 if server rejects profiles
- Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default
- Add pyroscope_profiling to sidebar (Alerting & Monitoring)
- pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra

* proxy: add PYROSCOPE_SAMPLE_RATE env, use verbose logging, fix int type

- Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default)
- Pass sample_rate to pyroscope.configure() as int for pyroscope-io
- Replace print with verbose_proxy_logger (info/warning)
- Document PYROSCOPE_SAMPLE_RATE in config_settings.md

* Address Greptile PR feedback: Pyroscope optional, docs, tests, docstring

- pyproject.toml: mark pyroscope-io as optional=true (proxy extra only)
- Add docs/my-website/docs/proxy/pyroscope_profiling.md (fix broken sidebar link)
- Add tests/test_litellm/proxy/test_pyroscope.py for _init_pyroscope()
- proxy_server: fix _init_pyroscope docstring (required server/app name, sample rate as int)

* Update litellm/proxy/proxy_server.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-13 17:32:29 -08:00
Harshit Jain
f87b12a2f7
fix grace period with better error handling on frontend and as per best practices 2026-02-14 04:18:08 +05:30
Ishaan Jaff
a06113ec82
feat: MCP OAuth2 client-side debug headers (#21151) 
* fix: SCOPES on Atlassian issue

* feat: add MCPDebug class for client-side MCP OAuth2 debugging

* feat: inject MCP debug headers into streamable HTTP response path

* test: add unit tests for MCPDebug class

* fix: refactor MCPDebug - move all logic into class static methods

* fix: collapse server.py debug code to two-liner using MCPDebug methods

* test: add tests for resolve_auth_resolution and wrap_send_with_debug_headers

* docs: add MCP debug headers section to troubleshooting guide

* docs: add Debugging OAuth section to mcp_oauth.md

* docs: replace inline debug section with cross-link to mcp_oauth

* docs: extract UI troubleshooting into its own page

* docs: simplify troubleshoot.md to issue reporting only

* docs: add quick-start debug command to MCP troubleshoot page

* docs: restructure sidebar - UI, MCP, Performance, Issue Reporting
 2026-02-13 12:55:47 -08:00
Ishaan Jaff
568edde87a
docs: fix Claude Code MCP tutorial with correct config and URL patterns (#21145) 
- Fix Atlassian config: use http transport and correct URL (/v1/mcp not /v1/sse)
- Fix URL pattern: use standard /mcp/<server_name> not legacy /<server_name>/mcp
- Add parameter breakdown table explaining each claude mcp add argument
- Add warning that server name in proxy config must match URL path
- Add ngrok step for OAuth callback accessibility
- Add ~/.claude.json config option alongside claude mcp add
- Fix auth header guidance: use x-litellm-api-key for OAuth servers
 2026-02-13 12:12:38 -08:00
Krish Dholakia
24b56a14eb
Guardrails - new Policy Templates (pre-configured guardrail combinations for specific use-cases) (#21025) 
* feat(patterns.json): add australia specific pii patterns - tax file number, abn, medicare number

Improve PII detection for australian contexts

* feat(patterns.json): add iban + street address pattern detection

* feat: support policy templates on ui

allows admin to enable pre-configured guardrails

helps cover specific use-cases well

* feat: create missing guardrails, working policy templates

* feat: policy_templates.json

support hosted policy templates

allows others to contribute to the policy templates

* docs: document new policy templates

* fix: address greptile feedback

* fix: fix linting error
 2026-02-13 11:53:02 -08:00
Sameer Kankute
066e694f5e
Merge pull request #21110 from BerriAI/litellm_litellm_anthropic_remote_url3 
Add support for remote URL fetching for anthropic beta header mapping
 2026-02-14 00:30:51 +05:30
fpagny
37157ee35f
feat(scaleway): add scaleway provider (#21121) 
* feat(scaleway): add scaleway provider

* Fix link format

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* remove unused tabs and tabitem

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: add scaleway to sidebar menu

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-13 08:37:58 -08:00
Alexsander Hamir
68122ede2c
docs: fix DEFAULT_NUM_WORKERS_LITELLM_PROXY default (1, not 4) (#21127) 
- config_settings.md: state default is 1, clarify NUM_WORKERS recommendation
- proxy_cli.py: correct --num_workers help text to match actual default
 2026-02-13 07:36:20 -08:00
Sameer Kankute
4a9d851dc4
Merge branch 'main' into litellm_litellm_anthropic_remote_url3 2026-02-13 18:40:00 +05:30
Harshit Jain
673b7d1fea
Merge branch 'main' into litellm_fix-virtual-key-grace-period 2026-02-13 18:14:47 +05:30
Sameer Kankute
d8f114e363
Merge branch 'main' into litellm_oss_staging_02_07_20262 2026-02-13 17:53:03 +05:30
Sameer Kankute
77dc742fde Add support sync with remote URL for beta headers 2026-02-13 15:49:42 +05:30
Cesar Garcia
64d8f1a601
docs: add native thinking param examples for Claude Opus 4.6 (#20799) 
* docs: add native thinking param examples for Claude Opus 4.6

Add documentation for using the native `thinking` parameter directly
with adaptive thinking and explicit budgets for Claude Opus 4.6.

* docs: add note about reasoning_effort mapping to adaptive for Opus 4.6
 2026-02-12 20:29:28 -08:00
Shivam Rawat
60390df4e2
Merge pull request #21083 from BerriAI/litellm_docs_clarity_on_dashscope 
docs: add API base URLs for Dashscope (International and China/Beijing)
 2026-02-12 18:47:38 -08:00
shivam
3c12f6f896 minor change 2026-02-12 18:44:16 -08:00
shivam
d4aa41daf6 docs: add API base URLs for Dashscope (International and China/Beijing) 2026-02-12 18:40:45 -08:00
yuneng-jiang
514645777b adding key envs to docs 2026-02-12 17:52:35 -08:00
Ishaan Jaff
736daf0a7d
[Feat] Adds Shell tool support for the OpenAI Responses API (#21063) 
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool

* add SHELL tool

* test_responses_api_shell_tool

* add SHELL_CALL_IN_PROGRESS

* add SHELL_CALL_IN_PROGRESS events

* TestOpenAIResponsesAPITest

* transform_streaming_response

* test_responses_api_shell_tool_streaming_sees_shell_output

* test_responses_api_shell_tool_streaming_sees_shell_output

* test_responses_api_shell_tool

* docs fix
 2026-02-12 13:04:29 -08:00
Ishaan Jaff
3d9b145b04
[Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 
* test_responses_api_context_management_server_side_compaction

* Server-side compaction

* docs fix

* test_responses_api_shell_tool
 2026-02-12 10:00:30 -08:00
Krrish Dholakia
f5382ebac9 docs: fix docs 2026-02-12 08:45:57 -08:00
Sameer Kankute
556bcd7203
Merge pull request #21055 from BerriAI/litellm_day_0_MiniMax-M2.1 
fix docs
 2026-02-12 22:04:23 +05:30
Sameer Kankute
9f15eca6b6 fix docs 2026-02-12 22:03:21 +05:30
Sameer Kankute
4e62386c65
Merge pull request #21054 from BerriAI/litellm_day_0_MiniMax-M2.1 
Add support for MiniMax-M2.1 and MiniMax-M2.1-lightining
 2026-02-12 21:51:46 +05:30
Sameer Kankute
7a3b227aeb
Apply suggestion from @greptile-apps[bot] 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-12 21:50:55 +05:30
Sameer Kankute
9b32c516ad Add support for MiniMax-M2.1 and MiniMax-M2.1-lightining 2026-02-12 21:45:49 +05:30
Sameer Kankute
e68b970953
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 21:14:08 +05:30
Sameer Kankute
59d6ab8a00
Merge branch 'main' into litellm_oss_staging_02_11_2026 2026-02-12 20:04:46 +05:30
Cesar Garcia
df38de5683 docs(web_search): add gpt-5-search-api usage examples for SDK and AI Gateway (#20616) 
- Document two OpenAI web search approaches: search models (/chat/completions) vs web_search_preview tool (/responses)
- Add gpt-5-search-api examples across all sections in web_search.md
- Update /responses examples to use gpt-5 with web_search_preview tool
- Add OpenAI Web Search Models section to providers/openai.md
- Add web search example to providers/openai/responses_api.md
 2026-02-12 19:58:12 +05:30
Achilleas Athanasiou Fragkoulis
cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492) 
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
 2026-02-12 19:39:04 +05:30
Krish Dholakia
b019638716
docs: add reference to example_openai_endpoint repo for self-hosting fake OpenAI proxy (#21006) 
- Updated benchmarks.md with a section on setting up fake OpenAI endpoints
- Updated load_test.md to mention the self-hosted option
- Updated load_test_advanced.md with a tip box about the example repo

Reference: https://github.com/BerriAI/example_openai_endpoint

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-02-11 18:00:21 -08:00
Ishaan Jaff
9975a9e3d4
fix: support Azure AD token auth for non-Claude azure_ai models (#20981) 
* fix: _should_use_api_key_header

* test_azure_ai_validate_environment_with_api_key

* fix: remove unused top-level RouteChecks import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add missing env keys to config_settings reference

Add MODEL_COST_MAP_MIN_MODEL_COUNT, MODEL_COST_MAP_MAX_SHRINK_RATIO,
and MAX_POLICY_ESTIMATE_IMPACT_ROWS to the environment variables
reference table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-11 10:48:44 -08:00
Sameer Kankute
c27650c4cf
Merge pull request #20935 from BerriAI/litellm_anthropic_filter_bedrock_headers 
[Feat]Managing Anthropic Beta Headers
 2026-02-11 18:19:01 +05:30