Commit graph

6640 commits

Author SHA1 Message Date
Sameer Kankute
e3c00027b9 Handle image in assitant message for gemini 2026-02-10 14:06:18 +05:30
Sameer Kankute
cb8ce09b0d Add support for langchain_aws via litellm passthrough 2026-02-10 13:37:17 +05:30
Sameer Kankute
45c8edbe25 Fix: Bug: Batch Rate Limiter Cannot Access User Files 2026-02-10 12:25:26 +05:30
Sameer Kankute
9bdb163269 Add error file ids as managed files 2026-02-10 12:06:40 +05:30
Harshit Jain
ac4bd34461
Merge branch 'main' into litellm_aws_rotation_fix 2026-02-10 11:47:45 +05:30
michelligabriele
969710477f
fix(mcp): resolve OAuth2 root endpoints returning "MCP server not found" (#20784)
When MCP SDK hits root-level /register, /authorize, /token without
server name prefix, auto-resolve to the single configured OAuth2
server. Also fix WWW-Authenticate header to use correct public URL
behind reverse proxy.
2026-02-09 19:58:37 -08:00
Shivam Rawat
1ee43b11de
[Fix] prevent shared backend model key from being polluted by per-deployment custom pricing (#20679)
* bug: custom price override for models

* added associated test
2026-02-09 19:38:44 -08:00
Ishaan Jaff
36e0361187
[UI] M2M OAuth2 UI Flow (#20794)
* add has_client_credentials

* MCPOAuth2TokenCache

* init MCP Oauth2 constants

* MCPOAuth2TokenCache

* resolve_mcp_auth

* test fixes

* docs fix

* address greptile review: min TTL, env-configurable constants, tests, docs

- Fix zero-TTL edge case: floor at MCP_OAUTH2_TOKEN_CACHE_MIN_TTL (10s)
- Make all MCP OAuth2 constants env-configurable via os.getenv()
- Move test file to follow 1:1 mapping convention (test_oauth2_token_cache.py)
- Add MCP OAuth doc page (mcp_oauth.md) with M2M and PKCE sections
- Update FAQ in mcp.md to reflect M2M support
- Add E2E test script and config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix mypy lint

* fix oauth2

* ui feat fixes

* test M2M

* test fix

* ui feats

* ui fixes

* ui fix client ID

* fix: backend endpoints

* docs fix

* fixes greptile

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 19:28:02 -08:00
Ishaan Jaff
19024e0602
[Feat] MCP Oauth2 Fixes - Add support for MCP M2M Oauth2 support (#20788)
* add has_client_credentials

* MCPOAuth2TokenCache

* init MCP Oauth2 constants

* MCPOAuth2TokenCache

* resolve_mcp_auth

* test fixes

* docs fix

* address greptile review: min TTL, env-configurable constants, tests, docs

- Fix zero-TTL edge case: floor at MCP_OAUTH2_TOKEN_CACHE_MIN_TTL (10s)
- Make all MCP OAuth2 constants env-configurable via os.getenv()
- Move test file to follow 1:1 mapping convention (test_oauth2_token_cache.py)
- Add MCP OAuth doc page (mcp_oauth.md) with M2M and PKCE sections
- Update FAQ in mcp.md to reflect M2M support
- Add E2E test script and config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix mypy lint

* fix oauth2

* remove old files

* docs fix

* address greptile comments

* fix: atomic lock creation + validate JSON response shape

- Use dict.setdefault() for atomic per-server lock creation
- Add isinstance(body, dict) check before accessing token response fields

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace asserts with proper guards, wrap HTTP errors with context

- Replace `assert` statements with `if/raise ValueError` (asserts can be
  disabled with python -O in production)
- Wrap `httpx.HTTPStatusError` to provide a clear error message with
  server_id and status code
- Add tests for HTTP error and non-dict JSON response error paths
- Remove unused imports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 17:35:11 -08:00
yuneng-jiang
6b938f81d1 guardrails list sensitive values fix 2026-02-09 17:11:54 -08:00
michelligabriele
35eb303098
fix(prometheus): sanitize label values to prevent metric scrape failures (#20600)
* fix(prometheus): sanitize label values to prevent metric scrape failures

Unicode characters like U+2028 (Line Separator) in Prometheus label values
break the text exposition format, causing scrapers (e.g. Datadog) to fail
parsing the entire /metrics endpoint. One bad label value causes ALL metrics
to be lost, not just the affected metric.

Add _sanitize_prometheus_label_value() and apply it in prometheus_label_factory()
and all direct .labels() call sites.

* fix(prometheus): handle non-string label values in sanitization

Coerce non-string values (int, bool, float) to str before applying
sanitization, preventing AttributeError on .replace() calls.

* fix(prometheus): run sanitization on coerced non-string values

Non-string values should be coerced to str and then sanitized (not
returned early), so their string representations also get cleaned.

* fix(prometheus): widen type hint to Optional[Any] for label value sanitization
2026-02-09 15:48:59 -08:00
Sameer Kankute
136fc698ef
Merge pull request #20601 from Harshit28j/litellm_fix_budget_model_v2
fix conflicts with main- (this PR is from upstream/main)
2026-02-09 20:08:27 +05:30
Sameer Kankute
6b2bcdb870
Merge pull request #20483 from BerriAI/litellm_completion_websearch
[Feat] Chat completion - Add Websearch support using LiteLLM /search (using web search interception hook)
2026-02-09 17:52:52 +05:30
Sameer Kankute
6158e46f00
Merge pull request #20747 from BerriAI/litellm_image_gen_bas_model_fix
Fix: base_model name for body and deplyment name in URL
2026-02-09 17:51:41 +05:30
Sameer Kankute
ef55d37bf0
Merge branch 'main' into litellm_v1_messages_claude_4_6 2026-02-09 17:14:36 +05:30
Sameer Kankute
30d17c29e4 handle when litellm_parrams might be none 2026-02-09 17:13:39 +05:30
Sameer Kankute
a322db02ab
Merge pull request #20696 from BerriAI/litellm_oss_staging_02_08_2026
Litellm oss staging 02 08 2026
2026-02-09 16:59:54 +05:30
Sameer Kankute
285c00ac20
Merge pull request #20710 from tsachis/fix-video-list-pagination-id-encoding
Fix video list pagination cursors not encoded with provider metadata
2026-02-09 16:41:52 +05:30
Sameer Kankute
f929461fc6
Merge pull request #20702 from emerzon/fix/issue-20698-stream-chunk-thinking-blocks
fix(streaming): preserve interleaved thinking/redacted_thinking blocks
2026-02-09 16:32:43 +05:30
Sameer Kankute
d35691aa0c Fix: base_model name for body and deplyment name in URL 2026-02-09 16:25:46 +05:30
Sameer Kankute
2f33445054
Merge pull request #20738 from BerriAI/main
merge main
2026-02-09 15:07:39 +05:30
Sameer Kankute
4e94ecb08d Add tests for WebSearch interception with chat completions API 2026-02-09 13:41:29 +05:30
Sameer Kankute
b822e2e0ff Add support for fast param 2026-02-09 11:28:00 +05:30
Sameer Kankute
8cd8a01d5a Add compaction for vertex ai 2026-02-09 10:18:19 +05:30
Emerson Gomes
cf17a440cd fix(responses): guard ambiguous tool-call index reuse 2026-02-08 08:53:17 -06:00
Emerson Gomes
68d788c84d fix(responses): preserve streamed tool deltas when id is omitted 2026-02-08 08:48:39 -06:00
tshushan
381c3756f4 Fix video list pagination cursors not encoded with provider metadata
first_id and last_id in the video list response were returned as raw
provider IDs while data[].id was properly wrapped with
encode_video_id_with_provider(). This caused pagination to break when
clients passed unencoded cursors back as the `after` parameter.

- Encode first_id/last_id in transform_video_list_response
- Decode the `after` param in transform_video_list_request via
  extract_original_video_id()
- Add 6 unit tests covering encoding, decoding, passthrough, and
  full round-trip pagination

Fixes #20708

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 16:24:14 +02:00
Emerson Gomes
c63d5fa0b5 test(streaming): build thinking chunks with typed Delta/StreamingChoices 2026-02-08 05:08:50 -06:00
Emerson Gomes
f7d03f8a43 fix(streaming): preserve interleaved thinking/redacted blocks 2026-02-08 05:04:47 -06:00
Elias Högbom Aronsson
0458e734b2
fix(vertex_ai): propagate extra_headers anthropic-beta to request body (#20666)
Vertex AI requires Anthropic beta flags in the request body
(anthropic_beta array), not as HTTP headers. The Bedrock handler
already extracts user-specified beta headers from the headers dict,
but the Vertex handler was missing this, causing extra_headers like
interleaved-thinking-2025-05-14 to be silently dropped.

This extracts anthropic-beta values from optional_params extra_headers
and merges them into the anthropic_beta request body field, and also
removes extra_headers from the request body since the parent's
transform_request spreads optional_params into data.
2026-02-07 23:05:17 -08:00
John Lathouwers
8dcd183013
Fixes #20582 (#20663) 2026-02-07 22:58:59 -08:00
nuernber
55a89f279f
feat: add support for anthropic_messages call type in prompt caching (#19233)
* feat: add support for anthropic_messages call type in prompt caching

* test: move anthropic_messages prompt caching test to main router test file

* add tutorial on using claude code with prompt cache routing
2026-02-07 22:51:06 -08:00
jwang-gif
c9df996b77
Add team policy mapping for zguard (#20608)
* support policy mapping on team key level

* update document

* update document

* address comments

* update document

* add unit test for new feature

* add more test case
2026-02-07 22:44:17 -08:00
Harshit Jain
3b043ee8bf
fix critical CVE vulnerabliltes (#20683) 2026-02-07 22:23:01 -08:00
Varun Chawla
e24ea2897a
fix: empty guardrails/policies arrays should not trigger enterprise license check (#20567)
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20304)

The UI sends empty arrays for enterprise-only fields (guardrails, policies,
logging) even when the user has not configured these features. The backend
`is not None` check treated `[]` as a truthy intent to use the feature,
falsely requiring an enterprise license for basic team operations.

Backend: Add `and updated_kv[field] != [] and updated_kv[field] != {}`
guards in `_update_metadata_fields` so empty collections are skipped.

UI: Conditionally omit guardrails, logging, and policies from the
payload when empty instead of defaulting to `[]`.

Fixes #20304

* fix: allow clearing fields with empty collections while skipping enterprise check

Address PR review feedback:

1. Move the empty-collection guard into _update_metadata_field (singular)
   so that empty lists/dicts skip only the premium license check but still
   get written into metadata. This lets users intentionally clear a
   previously-set field (e.g. guardrails: []) without being blocked, while
   the UI's default empty arrays still don't trigger a false enterprise
   error.

2. Remove sys.path hack from test file; use standard imports that work
   with pytest discovery.

3. Add tests verifying that empty collections are moved into metadata
   (field clearing works) even though they bypass the premium check.

Fixes #20304
2026-02-07 22:22:12 -08:00
shivam
fef5c5fd3e fix: preserve key alias and team_id after key regeneration, deletion 2026-02-07 17:08:01 -08:00
yuneng-jiang
70ebdeec8b modernize /team/available endpoint, migrate link to UI 2026-02-07 16:44:29 -08:00
Shivam Rawat
dd5c14baf8
posthog serilization fix (#20668) 2026-02-07 14:24:32 -08:00
Ishaan Jaff
14e2714162
Revert "perf: skip enterprise callback check when no dynamic disabling config…" (#20659)
This reverts commit ffc6af3f2f.
2026-02-07 11:52:07 -08:00
shin-bot-litellm
1477b4b46b
fix(tests): Add missing mocks for MCP IP filtering and updated APIs (#20652)
Fixes 15 failing tests in the MCP test suite:

1. **OAuth discoverable endpoints** (test_discoverable_endpoints.py):
   - Added autouse fixture to mock IPAddressUtils.get_mcp_client_ip
   - This bypasses IP-based access control which was blocking server lookup
   - Fixes: test_authorize_*, test_token_*, test_oauth_*, test_register_*

2. **A2A endpoints** (test_a2a_endpoints.py):
   - Fixed mock path for add_litellm_data_to_request
   - Was patching litellm_pre_call_utils but function is called from common_request_processing

3. **MCP guardrail handler** (test_mcp_guardrail_handler.py):
   - Updated tests to match new handler behavior
   - Handler now passes tools (not texts) to guardrail
   - Handler checks for mcp_tool_name (not messages array)

4. **MCP path-based segregation** (test_user_api_key_auth_mcp.py):
   - Added client_ip to get_auth_context unpacking (7 values now)
   - get_auth_context was updated to include client_ip

5. **MCP registry** (test_mcp_management_endpoints.py):
   - Added mock for get_filtered_registry (not just get_registry)
   - Registry endpoint uses get_filtered_registry for IP filtering

Co-authored-by: Shin <shin@openclaw.ai>
2026-02-07 11:30:49 -08:00
ryan-crabbe
ffc6af3f2f
perf: skip enterprise callback check when no dynamic disabling configured (#20449)
Add cheap .get() guards in should_run_callback() to short-circuit
the expensive EnterpriseCallbackControls.is_callback_disabled_dynamically()
call. When neither litellm_disabled_callbacks nor x-litellm-disable-callbacks
header is set (the common case), the enterprise function is never entered,
reducing should_run_callback from ~485ms to ~93-165ms across 54k calls.
2026-02-07 11:12:47 -08:00
Harshit Jain
37aee120a0
fixes content blocked by bedrock (#20606) 2026-02-07 11:02:12 -08:00
ryan-crabbe
14c2b5da91
perf: replace enum construction with frozenset lookup in _is_streaming_request (#20302)
CallTypes(call_type) was constructing an enum from string on every call,
taking ~4.6µs/call (69.6% of function time). Replace with a frozenset
membership test for ~0.8µs/call (8.3x faster).
2026-02-07 10:50:57 -08:00
ryan-crabbe
616350f058
perf: Optimize get_litellm_params with sparse kwargs extraction (#19884)
* perf: Optimize get_litellm_params with sparse kwargs extraction

- Add _OPTIONAL_KWARGS_KEYS frozenset for O(1) lookups
- Replace 28 unconditional kwargs.get() calls with sparse extraction
- Only add kwargs keys that are actually present in the dict
- Simplify _get_base_model_from_litellm_call_metadata by removing redundant None checks

This reduces get_litellm_params() time by ~31% (743ms → 509ms across 6000 calls)
and Logging.__init__ total time by ~24% (1.61s → 1.23s).

* test: add unit tests for get_litellm_params sparse kwargs extraction
2026-02-07 10:38:43 -08:00
ryan-crabbe
874bafbbb4
perf: add early-exit guards in completion_cost for unused features (#20020)
* perf: add early-exit guards in completion_cost for unused features

Skip function calls to get_cost_for_built_in_tools, _apply_cost_discount,
_apply_cost_margin, and _store_cost_breakdown_in_logging_obj when their
respective features are not configured. Reduces completion_cost() time
by ~20% (4.39s → 3.53s over 6K requests) for the common case where
built-in tools, discounts, margins, and logging object are not active.

* fix: always call get_cost_for_built_in_tools regardless of standard_built_in_tools_params

The function can detect web search usage from the usage object (e.g.
server_tool_use.web_search_requests, prompt_tokens_details.web_search_requests)
even when standard_built_in_tools_params is None, so guarding on it can
under-count cost for providers like Vertex AI and Anthropic.

Adds regression test for completion_cost with web search in usage but
no standard_built_in_tools_params.
2026-02-07 10:10:00 -08:00
Ishaan Jaffer
1780b1716f filter_server_ids_by_ip 2026-02-07 10:08:20 -08:00
shin-bot-litellm
df299d3193
fix(tests): Fix flaky container and scientific notation tests (#20650)
* fix(tests): Mock async_container_create_handler for async router test

The test was mocking container_create_handler (sync), but
router.acreate_container uses _is_async=True which calls
async_container_create_handler. This caused the test to hit
the real OpenAI API.

Fixed by using AsyncMock on async_container_create_handler.

* fix(tests): Use uuid for unique model name in scientific notation test

The test was using a static "unique" model name which could cause
conflicts when running tests in parallel (-n 16 in CI). Using uuid
ensures truly unique names to prevent test pollution.

---------

Co-authored-by: Shin <shin@openclaw.ai>
2026-02-07 09:57:08 -08:00
ryan-crabbe
8c7051686b
perf: optimize get_standard_logging_metadata with set intersection (#19685)
* perf: Optimize get_standard_logging_metadata with set intersection

- Cache StandardLoggingMetadata.__annotations__.keys() as module-level frozenset
- Use set intersection to iterate only keys present in both metadata and supported keys
- Single lookup for user_api_key instead of 3 separate .get() calls

Results:
- get_standard_logging_metadata: 1.55s → 1.41s (9.2% faster)

* test: add unit tests for get_standard_logging_metadata non-string user_api_key handling
2026-02-07 09:35:03 -08:00
shin-bot-litellm
9eb71574d8
fix(tests): Fix sendgrid email tests to properly mock httpx client (#20628)
* fix(tests): Fix sendgrid email tests to properly mock httpx client

The tests were potentially hitting the real SendGrid API because the mock
was patching get_async_httpx_client() but the actual client could be cached
or the mock timing could be off.

Fix by directly replacing logger.async_httpx_client after instantiation,
which guarantees the mock is used regardless of caching or initialization
timing issues.

Changes:
- Replace mock_httpx_client fixture with simpler mock_async_client fixture
- Directly inject mock client into logger instance after creation
- Remove respx decorator (no longer needed with direct injection)
- Simplify test structure while maintaining same assertions

* fix(lint): remove unused imports from SendGrid test
2026-02-07 09:19:02 -08:00
shin-bot-litellm
537f7af583
fix(test): update deprecated gemini embedding model (#20621)
Replace text-embedding-004 with gemini-embedding-001.

The old model was deprecated and returns 404:
'models/text-embedding-004 is not found for API version v1beta'

Co-authored-by: Shin <shin@openclaw.ai>
2026-02-06 18:35:40 -08:00