Commit graph

34138 commits

Author SHA1 Message Date
Harshit28j
6d535e5639 fix(proxy): allow custom auth routes to bypass route authorization checks
Custom user-added routes (e.g. /ldap/ngs/ready) used with Depends(user_api_key_auth) were being rejected as admin-only after _run_post_custom_auth_checks was introduced in commit 14badde13c.

The route authorization check in common_checks is designed for LiteLLM's own management routes. Custom auth flows that add their own routes should be trusted since the custom auth function already validated the request. Budget and expiry checks still run.

Add skip_route_check parameter to common_checks() and pass skip_route_check=True from _run_post_custom_auth_checks() to skip route authorization while preserving budget/team/model checks.

Regression test added: test_common_checks_skip_route_check_for_custom_auth

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 20:20:47 +05:30
Sameer Kankute
c1b39a6425
Merge pull request #22651 from BerriAI/litellm_encrypted_content_affinity_2
Add support for encrypted content affinity
2026-03-03 19:43:47 +05:30
Sameer Kankute
a9ee8ff5d2
Merge pull request #22641 from BerriAI/litellm_remove_default_litellm_thinking
[Chore]Remove defualt hardcoded thinking levels for gemini 3 family
2026-03-03 19:42:40 +05:30
Sameer Kankute
bdfc602dbf
Merge pull request #22625 from BerriAI/litellm_azure_ai_finetune
Fix: Azure ai finetuning api
2026-03-03 19:42:17 +05:30
Harshit Jain
bb1da7091a
Merge pull request #22631 from Harshit28j/litellm_bump_litellm_08_03
bump: version 1.82.0 → 1.82.1
2026-03-03 19:35:56 +05:30
Sameer Kankute
88dd1b5801
Merge pull request #22657 from BerriAI/revert-22654-litellm_add_Qwen3.5-397B-A17B
Revert "feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model"
2026-03-03 19:35:39 +05:30
Sameer Kankute
24ec7f882f
Revert "feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model" 2026-03-03 19:35:26 +05:30
Sameer Kankute
974c02fae4
Merge pull request #22654 from Varad2001/litellm_add_Qwen3.5-397B-A17B
feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model
2026-03-03 19:19:41 +05:30
Varad Khonde
d66f8bc15d feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model 2026-03-03 19:17:28 +05:30
Sameer Kankute
9ffbd9e30e
Merge pull request #22464 from Point72/ephrimstanley/batch-fixes-feb27
Managed batches fixes for vertex
2026-03-03 18:53:53 +05:30
Sameer Kankute
2f6279d189 Fix import issue 2026-03-03 18:41:15 +05:30
Sameer Kankute
ca597e18c8 Fix routing of encrypted content 2026-03-03 18:37:28 +05:30
Sameer Kankute
521f804350 Fix encrypted content streaming affinity issue 2026-03-03 18:37:22 +05:30
Sameer Kankute
2bc4da76ce Update the tests 2026-03-03 18:36:07 +05:30
Sameer Kankute
37834f1d2a Update the docs 2026-03-03 18:36:07 +05:30
Sameer Kankute
7928d41e9a Update the routing 2026-03-03 18:36:07 +05:30
Sameer Kankute
122f534d87 Add encoding method for Encrypted-content-aware deployment 2026-03-03 18:36:07 +05:30
Sameer Kankute
adec115db8 Fix logging for error 2026-03-03 18:36:07 +05:30
Sameer Kankute
18bf3f2df6 Fix mock github test 2026-03-03 18:36:07 +05:30
Sameer Kankute
a88a17796b Fix logging and encrypted content extraction 2026-03-03 18:36:07 +05:30
Sameer Kankute
37612bdf56 ADd incident report 2026-03-03 18:36:07 +05:30
Sameer Kankute
fbec5c5ccf Add docs for encrypted_content_affinity 2026-03-03 18:36:07 +05:30
Sameer Kankute
9f627c67d8 Add tests for encrypted_content_affinity 2026-03-03 18:36:07 +05:30
Sameer Kankute
394c49d303 Add tests for encrypted_content_affinity 2026-03-03 18:36:07 +05:30
Sameer Kankute
92b2556282 Add encrypted_content_affinity in router 2026-03-03 18:34:46 +05:30
Sameer Kankute
058fac848e Add Encrypted-content-aware deployment affinity for the Router 2026-03-03 18:34:46 +05:30
Sameer Kankute
213423cb45 Fix test case 2026-03-03 15:05:20 +05:30
Sameer Kankute
f8034f15ad Remove defualt hardcoded thinking levels for gemini 3 family 2026-03-03 14:51:12 +05:30
Harshit28j
d07689d2d7 bump: version 1.82.0 → 1.82.1 2026-03-03 11:59:58 +05:30
Krish Dholakia
67f90254ed
feat(guardrails): team-based guardrail registration and approval workflow (#22459)
* feat(guardrails): team-based guardrail registration and approval workflow

Add team-based guardrail submission system where teams can register
Generic Guardrail API guardrails for admin review. Includes:

- POST /guardrails/register endpoint for team-scoped submissions
- Admin review endpoints (list/get/approve/reject submissions)
- Team Guardrails tab in the UI dashboard
- extra_headers support for forwarding client headers to guardrail APIs
- Prisma schema migration for status, submitted_at, reviewed_at fields
- Documentation for team-based guardrails and static/dynamic headers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(guardrails): address review feedback - SSRF, silent failure, redundant query

- Validate api_base URL scheme (http/https only) and hostname in
  register_guardrail to prevent SSRF via team submissions
- Return warning field in approve response when in-memory initialization
  fails so admins know the guardrail won't work until next sync cycle
- Eliminate redundant DB query in list_guardrail_submissions by fetching
  all team guardrails once and deriving both filtered list and summary
  counts from the single result set

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(guardrails): add pending_review status guard to reject endpoint

Prevent rejecting already-active or already-rejected guardrails, which
would create a DB/memory inconsistency (active in memory but rejected
in DB). Now mirrors the approve endpoint's status check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 22:06:49 -08:00
Shivaang
213799282b
fix(openrouter): register OpenRouter as native Responses API provider (#22355)
OpenRouter supports the Responses API at /api/v1/responses with
encrypted_content for multi-turn stateless reasoning workflows.
Without native registration, requests fall through to the chat
completion bridge, which uses a different format (reasoning_details)
and drops encrypted_content entirely.

This adds OpenRouterResponsesAPIConfig to route requests directly to
OpenRouter's Responses API endpoint, preserving encrypted_content.

Fixes https://github.com/BerriAI/litellm/issues/22189

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-02 22:02:59 -08:00
Jaeyeon Kim(김재연)
6bcba46dda
fix: set mock status_code in JWT OIDC discovery tests (#22361)
The _resolve_jwks_url method checks response.status_code != 200, but
MagicMock returns a MagicMock object for status_code which is always
truthy (!= 200). Explicitly set mock_response.status_code = 200 so the
tests exercise the intended code path.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:57:54 -08:00
ryan-crabbe
5b0238736c
Add incident report: cache eviction closes in-use httpx clients (#22309) 2026-03-02 21:49:48 -08:00
Sameer Kankute
18216ac07c Fix: Azure ai finetuning api 2026-03-03 10:48:00 +05:30
Ishaan Jaff
86d5b4c632
feat: add Nebius AI Studio models to model_prices_and_context_window.json (#22614)
Add 30 Nebius AI Studio models covering:
- Text-to-text: DeepSeek (R1, R1-0528, R1-Distill, V3, V3-0324), Meta Llama
  (3.1-8B/70B/405B, 3.3-70B), Qwen (3-235B/32B/30B/14B/4B, 2.5-72B/32B,
  2.5-Coder-7B, QwQ-32B), Mistral Nemo, NousResearch Hermes-3, NVIDIA
  Nemotron Ultra/Super, Google Gemma-3-27B, Llama-Guard-3
- Vision: Qwen2.5-VL-72B, Qwen2-VL-72B, Qwen2-VL-7B
- Embedding: BAAI/bge-en-icl, BAAI/bge-multilingual-gemma2, intfloat/e5-mistral-7b

Pricing sourced from https://nebius.com/prices-ai-studio (base flavor).
Context windows sourced from https://docs.nebius.com/studio/inference/models/

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-02 18:43:07 -08:00
Krish Dholakia
dfa2798169
Fix PR template: correct test directory path from tests/litellm/ to tests/test_litellm/ (#22612)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-03-02 17:49:53 -08:00
Ishaan Jaff
bfceb7fc3f
feat(perplexity): add embedding support for pplx-embed-v1 models (#22610)
* feat: add Perplexity embedding support (pplx-embed-v1)

Add support for Perplexity AI's embedding models via the LLM HTTP handler:

Models:
- pplx-embed-v1-0.6b (1024 dims, 32K context, $0.004/1M tokens)
- pplx-embed-v1-4b (2560 dims, 32K context, $0.03/1M tokens)

Implementation:
- PerplexityEmbeddingConfig in litellm/llms/perplexity/embedding/
- Registered in ProviderConfigManager, __init__.py lazy imports, main.py dispatch
- Model pricing added to model_prices_and_context_window.json
- Supports dimensions and encoding_format parameters
- Uses base_llm_http_handler.embedding() pattern

Tests:
- 19 unit tests covering transformation, params, URLs, provider config, model info

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* docs: add Perplexity AI embeddings documentation

- Create providers/perplexity_embedding.md with SDK and proxy usage examples
- Convert Perplexity from flat doc to category in sidebars.js
- Category includes existing chat/responses doc + new embeddings doc
- Covers pplx-embed-v1-0.6b and pplx-embed-v1-4b models
- Documents supported parameters (dimensions, encoding_format)
- Includes proxy config and curl examples

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: decode Perplexity base64_int8 embeddings to OpenAI-format float arrays

Perplexity returns embeddings as base64-encoded signed int8 values by default,
not float arrays like OpenAI. This commit adds decoding in
transform_embedding_response so the proxy returns standard OpenAI-compatible
float arrays (normalized to [-1, 1]).

- Added _decode_base64_embedding() static method
- Handles both base64 strings (decoded) and float lists (passthrough)
- Added 3 new tests for base64 decoding + passthrough

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-02 17:37:50 -08:00
Kenan Yildirim
b8befb3403
Add CrowdStrike AIDR guardrail hook (#17876)
* Add CrowdStrike AIDR guardrail hook

* fixup! use apply_guardrail event hook

* fixup! update imports

* fix(guardrails): include AI response in CrowdStrike AIDR output events

Issue:
_build_guard_input_for_response() was:
- Sending only the original user input (messages).
- Not sending the AI provider response.

This fix will:
  - Extract response.choices from the ModelResponse object and include them in guard_input payload.
  - Thus, ensure AIDR output rules receive the AI-generated content for analysis.
  - Fix and update tests.

* fix(guardrails): prevent duplicate input events in CrowdStrike AIDR guardrail

Issue:
The CrowdStrike AIDR guardrail was running on during_call hooks wihtout event_hook configured.

This fix will:
- Set event_hook to ["pre_call", "post_call"] (AIDR admins will control what policy is applied)

This change will:
- Require default_on parameter
- Prevent duplicate API calls to AIDR for the same input
- Avoid unchecked AI provider API calls on during_call hook

* docs: add CrowdStrike AIDR to the list of Guardrails under Integrations

* docs: update CrowdStrike AIDR documentation page

---------

Co-authored-by: Konstantin Lapine <konstantin.lapine@crowdstrike.com>
2026-03-02 17:26:54 -08:00
Cesar Garcia
2525d66dbe
Merge pull request #22584 from BerriAI/litellm_oss_staging_02_27_2026
Litellm oss staging 02 27 2026
2026-03-02 19:05:02 -03:00
Cesar Garcia
e559d4dd11
Merge pull request #22582 from BerriAI/litellm_oss_staging_02_26_2026
Litellm oss staging 02 26 2026
2026-03-02 18:51:37 -03:00
Chesars
6292c3dbdf merge: resolve conflicts with upstream/main
- anthropic.md: keep claude-opus-4-6 alias and claude-sonnet-4-6 entry
- transformation.py: take upstream's formatted effort_map with fallback
2026-03-02 18:49:24 -03:00
Cesar Garcia
835a2c3dc6
Merge pull request #22583 from Chesars/fix/add-bedrock-cache-token-pricing
fix(pricing): add missing cache token pricing for 24 Bedrock Claude models
2026-03-02 18:46:28 -03:00
Cesar Garcia
680b9ee9f2
Merge pull request #22586 from Chesars/fix/update-gemini-deprecation-dates
fix: update Gemini model deprecation dates
2026-03-02 18:45:30 -03:00
Cesar Garcia
a54a1d27d7
Merge pull request #22596 from Chesars/fix/add-dashscope-models-pricing
fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus
2026-03-02 18:45:08 -03:00
Cesar Garcia
229eb5234d
Merge pull request #22601 from Chesars/fix/update-mistral-models-pricing
feat: add missing Mistral models and update pricing
2026-03-02 18:44:38 -03:00
Chesars
884f7c5e4e fix: update mistral-small-latest to match Small 3.2 specs
mistral-small-latest now points to Small 3.2 (since June 2025).
Updated pricing from $0.10/$0.30 to $0.06/$0.18 per 1M tokens,
context from 32k to 131k, and added vision support to match
mistral-small-3-2-2506.
2026-03-02 18:36:05 -03:00
Chesars
87fe521f46 fix: remove unused OpenAIImageGenerationOptionalParams import
Fixes ruff F401 in check_code_and_doc_quality CI check.
2026-03-02 18:24:29 -03:00
Chesars
abb7eb250a fix: remove retired Saba model from new entries
Saba was retired on 9/30/2025 per Mistral docs, replaced by Small 3.2.
2026-03-02 18:19:46 -03:00
Chesars
bd822a7a68 fix: add supports_response_schema to Ministral 3 models
Ministral 3 (3B, 8B, 14B) support structured outputs per Mistral docs.
2026-03-02 18:19:02 -03:00
Shivam Rawat
d5355602d5
added configurable env for mcp timeouts (#22287) 2026-03-02 13:13:41 -08:00