Commit graph

32251 commits

Author SHA1 Message Date
Yuneng Jiang
32899f6b53
chore: fixes
Some checks failed
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
2026-04-05 00:37:17 -07:00
OpenClaw
29a3345d05 fix(mypy): Fix type errors across multiple files
- vertex_ai/gemini/transformation.py: Fix TypedDict assignment via dict alias
- mcp_server/server.py: Convert ASGI scope to dict for type compatibility
- pass_through_endpoints.py: Add explicit Optional[dict] type annotation
- vector_store_endpoints/endpoints.py: Add Any type for dynamic proxy hook
- responses transformation.py: Use dict(Reasoning()) and setattr for compatibility
- zscaler_ai_guard.py: Add assert for api_base nullability
2026-02-14 02:17:43 +00:00
shin-bot-litellm
df54e1b452
fix(model_info): Add missing tpm/rpm for Gemini models (#21175)
Several Gemini models (TTS, native-audio, robotics, gemma) were missing
tpm/rpm values, causing test_get_model_info_gemini to fail.

Added conservative default values (tpm=250000, rpm=10) for preview models.
gemini-2.5-flash-preview-tts gets tpm=4000000, rpm=10.

Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
2026-02-13 18:07:01 -08:00
Alexsander Hamir
30b28da2b7
Add pyroscope for observability (#21167)
* Pyroscope: require PYROSCOPE_APP_NAME and PYROSCOPE_SERVER_ADDRESS, add UTF-8 locale hint

- No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled
- Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections
- Startup message suggests PYTHONUTF8=1 if server rejects profiles
- Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default
- Add pyroscope_profiling to sidebar (Alerting & Monitoring)
- pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra

* proxy: add PYROSCOPE_SAMPLE_RATE env, use verbose logging, fix int type

- Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default)
- Pass sample_rate to pyroscope.configure() as int for pyroscope-io
- Replace print with verbose_proxy_logger (info/warning)
- Document PYROSCOPE_SAMPLE_RATE in config_settings.md

* Address Greptile PR feedback: Pyroscope optional, docs, tests, docstring

- pyproject.toml: mark pyroscope-io as optional=true (proxy extra only)
- Add docs/my-website/docs/proxy/pyroscope_profiling.md (fix broken sidebar link)
- Add tests/test_litellm/proxy/test_pyroscope.py for _init_pyroscope()
- proxy_server: fix _init_pyroscope docstring (required server/app name, sample rate as int)

* Update litellm/proxy/proxy_server.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 17:32:29 -08:00
yuneng-jiang
1828e68515
Merge pull request #21166 from BerriAI/litellm_access_group_rename
[Refactor] Access Group model_ids to model_names for backwards Compatability
2026-02-13 17:22:17 -08:00
yuneng-jiang
3ebafc4a10
Merge pull request #21165 from BerriAI/litellm_access_group_ui
[Feature] UI - Access Groups: Table and Details Page
2026-02-13 17:17:18 -08:00
yuneng-jiang
41445ff8ad migration + build files 2026-02-13 17:15:39 -08:00
yuneng-jiang
38e2ecb52c bump: version 0.4.36 → 0.4.37 2026-02-13 17:15:05 -08:00
yuneng-jiang
60954d22e3 Merge remote-tracking branch 'origin' into litellm_access_group_rename 2026-02-13 17:04:59 -08:00
Ishaan Jaff
b48bec0e4b
fix: populate identity fields in proxy admin JWT early-return path (#21169)
* fix: populate identity fields in proxy admin JWT early-return path

When is_proxy_admin is True, the UserAPIKeyAuth early-return now includes
user_id, team_id, team_alias, team_metadata, org_id, and end_user_id
resolved from the JWT. Previously only user_role and parent_otel_span
were set, causing blank Team Name and Internal User in Request Logs UI.

* test: add unit tests for proxy admin JWT identity fields
2026-02-13 17:04:38 -08:00
yuneng-jiang
ea2e5ff0b9 allow editing of access group names 2026-02-13 17:03:41 -08:00
yuneng-jiang
7561c1a3c3 addressing comments 2026-02-13 16:54:34 -08:00
yuneng-jiang
72848f4c08 change to model name for backwards compat 2026-02-13 16:44:49 -08:00
The Mavik
ab4b6197ef
fix: add custom_body parameter to endpoint_func in create_pass_through_route (#20849)
* fix: add custom_body parameter to endpoint_func in create_pass_through_route

The bedrock_proxy_route calls `endpoint_func(custom_body=data)` to
pass a pre-parsed, SigV4-signed request body. However, the
`endpoint_func` closure created by `create_pass_through_route` does
not accept a `custom_body` keyword argument, causing:

    TypeError: endpoint_func() got an unexpected keyword argument 'custom_body'

Add `custom_body: Optional[dict] = None` to both `endpoint_func`
definitions (adapter-based and URL-based). In the URL-based path,
when `custom_body` is provided by the caller, use it instead of
re-parsing the body from the raw request.

Fixes #16999

* Add tests for custom_body handling in create_pass_through_route

Address reviewer feedback on PR #20849:

- Document why the adapter-based endpoint_func accepts custom_body
  for signature compatibility but does not forward it (the underlying
  chat_completion_pass_through_endpoint does not support it).
- Add test_create_pass_through_route_custom_body_url_target: verifies
  that when a caller (e.g. bedrock_proxy_route) supplies custom_body,
  it takes precedence over the body parsed from the raw request.
- Add test_create_pass_through_route_no_custom_body_falls_back:
  verifies that the default path (no custom_body) correctly uses the
  request-parsed body, preserving existing behavior.

Both tests are fully mocked following the project's CONTRIBUTING.md
guidelines and the patterns established in the existing test file.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: themavik <themavik@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-13 16:44:40 -08:00
yuneng-jiang
ca2ce4a946 adding tests 2026-02-13 16:30:24 -08:00
yuneng-jiang
2f3dc4860f new badge changes 2026-02-13 16:22:07 -08:00
yuneng-jiang
5dbcca8d43 Access groups UI 2026-02-13 16:20:31 -08:00
Ishaan Jaff
a06113ec82
feat: MCP OAuth2 client-side debug headers (#21151)
* fix: SCOPES on Atlassian issue

* feat: add MCPDebug class for client-side MCP OAuth2 debugging

* feat: inject MCP debug headers into streamable HTTP response path

* test: add unit tests for MCPDebug class

* fix: refactor MCPDebug - move all logic into class static methods

* fix: collapse server.py debug code to two-liner using MCPDebug methods

* test: add tests for resolve_auth_resolution and wrap_send_with_debug_headers

* docs: add MCP debug headers section to troubleshooting guide

* docs: add Debugging OAuth section to mcp_oauth.md

* docs: replace inline debug section with cross-link to mcp_oauth

* docs: extract UI troubleshooting into its own page

* docs: simplify troubleshoot.md to issue reporting only

* docs: add quick-start debug command to MCP troubleshoot page

* docs: restructure sidebar - UI, MCP, Performance, Issue Reporting
2026-02-13 12:55:47 -08:00
Ishaan Jaffer
817de99177 UI build 2026-02-13 12:51:17 -08:00
Ishaan Jaff
40b290aa84
fix: MCP Gateway SCOPES on Atlassian issue (#21150)
* fix: SCOPES on Atlassian issue

* test: add regression tests for scopes=None in OAuth discovery endpoints
2026-02-13 12:43:56 -08:00
yuneng-jiang
71107b3fd0
Merge pull request #21149 from BerriAI/litellm_spendlogs_reset_custom_time
[Fix] UI - Spend Logs: Reset Filters Resets Custom Date Range
2026-02-13 12:30:45 -08:00
yuneng-jiang
eb144a44d0
Merge pull request #21144 from BerriAI/litellm_ui_fallbacks_10
[Refactor] UI - Fallbacks: Default Configurable to 10 Models
2026-02-13 12:30:35 -08:00
yuneng-jiang
82fcc39e6b reset custom time range when reset filters is clicked 2026-02-13 12:12:50 -08:00
Ishaan Jaff
568edde87a
docs: fix Claude Code MCP tutorial with correct config and URL patterns (#21145)
- Fix Atlassian config: use http transport and correct URL (/v1/mcp not /v1/sse)
- Fix URL pattern: use standard /mcp/<server_name> not legacy /<server_name>/mcp
- Add parameter breakdown table explaining each claude mcp add argument
- Add warning that server name in proxy config must match URL path
- Add ngrok step for OAuth callback accessibility
- Add ~/.claude.json config option alongside claude mcp add
- Fix auth header guidance: use x-litellm-api-key for OAuth servers
2026-02-13 12:12:38 -08:00
yuneng-jiang
6eeb9b109b
Merge pull request #21143 from BerriAI/litellm_model_id_search
[Feature] UI - Spend Logs: Sorting Columns
2026-02-13 11:53:16 -08:00
yuneng-jiang
724ab1c154
Update ui/litellm-dashboard/src/components/networking.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 11:53:04 -08:00
Krish Dholakia
24b56a14eb
Guardrails - new Policy Templates (pre-configured guardrail combinations for specific use-cases) (#21025)
* feat(patterns.json): add australia specific pii patterns - tax file number, abn, medicare number

Improve PII detection for australian contexts

* feat(patterns.json): add iban + street address pattern detection

* feat: support policy templates on ui

allows admin to enable pre-configured guardrails

helps cover specific use-cases well

* feat: create missing guardrails, working policy templates

* feat: policy_templates.json

support hosted policy templates

allows others to contribute to the policy templates

* docs: document new policy templates

* fix: address greptile feedback

* fix: fix linting error
2026-02-13 11:53:02 -08:00
yuneng-jiang
25fccad127 Change default fallbacks to 10 models 2026-02-13 11:50:13 -08:00
yuneng-jiang
adc2859f0a sorting spend logs in ui 2026-02-13 11:33:41 -08:00
datzscaler
aab8edde67
fix(guardrails): Zscaler AI Guard bug fixes and support during post-call (#20801)
* fix(guardrails): fixed post-call issue with Zscaler guardrail and invalid headers. Added unittests

* fix(guardrails): Addressed greptil comments. Make policyid hanlding more clear

* fix(guardrails): Address greptile comment

* Update litellm/proxy/guardrails/guardrail_hooks/zscaler_ai_guard/zscaler_ai_guard.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 11:28:14 -08:00
Sameer Kankute
066e694f5e
Merge pull request #21110 from BerriAI/litellm_litellm_anthropic_remote_url3
Add support for remote URL fetching for anthropic beta header mapping
2026-02-14 00:30:51 +05:30
Ishaan Jaff
a1b7261502
Remove exposed API key from example configuration in mcp_server_manager.py (#21129)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-02-13 09:52:38 -08:00
Aidan Kovacic
7a164ba2cc
fix: shared health check serialization (#21119)
* Add BaseModel serialization to safe_json_dumps

* Use safe_dumps in shared health check caching

* remove redundant seen.add
2026-02-13 09:15:43 -08:00
yuneng-jiang
5ddba48df9
Merge pull request #21088 from BerriAI/litellm_spend_rebase
[Fix] Spend Management Tests
2026-02-13 09:11:42 -08:00
Sameer Kankute
d4dbb460cc Make tests run with local beta header mapping json 2026-02-13 22:31:42 +05:30
Sameer Kankute
aaf5938864
Merge pull request #21089 from BerriAI/litellm_vector_store_endpoints
Add target_model_names for vector store endpoints
2026-02-13 22:18:29 +05:30
jquinter
93e8127562
chore: remove redundant test-complete job from test workflow (#21106)
The test-complete aggregate job adds no value as GitHub Actions
already provides visibility into matrix job results.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 08:39:00 -08:00
jquinter
1e0b1c8334
fix: remove duplicate PerplexityResponsesConfig from LLM_CONFIG_NAMES (#21105)
Removes duplicate entry of PerplexityResponsesConfig that was causing
Ruff linting error F601 (dictionary key literal repeated).

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 08:38:50 -08:00
jquinter
d32b654d6b
fix: regenerate poetry.lock to sync with pyproject.toml (#21108)
poetry.lock was out of sync with pyproject.toml, causing CI failures
across all PRs with "Run `poetry lock` to fix the lock file" error.

Regenerated the lock file to resolve the sync issue.

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 08:38:29 -08:00
fpagny
37157ee35f
feat(scaleway): add scaleway provider (#21121)
* feat(scaleway): add scaleway provider

* Fix link format

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* remove unused tabs and tabitem

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: add scaleway to sidebar menu

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 08:37:58 -08:00
Sameer Kankute
a653a8ddde
Merge pull request #21092 from BerriAI/litellm_azure_batches_issues
Fix azure batches issues
2026-02-13 22:06:21 +05:30
Sameer Kankute
fedfec51ab
Merge pull request #21120 from BerriAI/litellm_add_rag_ingest_vertex_ai
Add rag ingest vertex ai
2026-02-13 22:04:57 +05:30
Sameer Kankute
4ad0ecd9eb Fix mypy issues 2026-02-13 22:02:27 +05:30
Sameer Kankute
dc874fc98f Fix mypy issues 2026-02-13 21:54:46 +05:30
Sameer Kankute
ef456cafc8
Merge pull request #21040 from BerriAI/litellm_fix_stale_mcp_issue
[Bug] Fix Session not found errors
2026-02-13 21:52:27 +05:30
Alexsander Hamir
68122ede2c
docs: fix DEFAULT_NUM_WORKERS_LITELLM_PROXY default (1, not 4) (#21127)
- config_settings.md: state default is 1, clarify NUM_WORKERS recommendation
- proxy_cli.py: correct --num_workers help text to match actual default
2026-02-13 07:36:20 -08:00
Sameer Kankute
1c0464fac0 Remove print statement 2026-02-13 20:13:14 +05:30
Sameer Kankute
c5d7418e9e Add VertexAIRAGIngestion class 2026-02-13 20:11:25 +05:30
Sameer Kankute
9e02fd065a Add support for vertex ai for rag/ingest 2026-02-13 20:11:10 +05:30
Sameer Kankute
4a9d851dc4
Merge branch 'main' into litellm_litellm_anthropic_remote_url3 2026-02-13 18:40:00 +05:30