Commit graph

8758 commits

Author SHA1 Message Date
Sameer Kankute
4710e772be Add support for bedrock qwen 2 imported model 2025-12-04 11:08:57 +05:30
Cesar Garcia
6d831ffff4
fix(ocr): add missing ocr and aocr to CallTypes enum (#17435)
Add `ocr` and `aocr` entries to the CallTypes enum to fix the
ValueError that occurs when using the /v1/ocr endpoint with
guardrails enabled.

The OCR endpoint uses route_type="aocr", but the CallTypes enum
was missing these values, causing guardrail hooks to fail when
trying to instantiate CallTypes("aocr").

Fixes #17381
2025-12-03 21:28:13 -08:00
Andrew Truong
1879623502
feat: Basic Weave OTEL integration (#17439)
* test

* testfix
2025-12-03 21:26:22 -08:00
Haiyi
06d42fbd30
Fix: Skip empty text blocks in Anthropic system messages (#17442)
When translating system messages for the Anthropic API, empty text
content blocks cause the error "messages: text content blocks must be
non-empty". This fix skips empty string content and empty text blocks
in list content to prevent this error.

Fixes issue with Vertex AI Anthropic API calls.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-03 21:25:06 -08:00
Devaj Mody
a6ce118989
fix(openai): add clear error message for empty LLM endpoint responses (#17445)
Fixes #17209

  - Add validation after response.parse() to detect empty/invalid responses
  - Raise clear OpenAIError with actionable message
  - Add 3 unit tests
2025-12-03 21:23:09 -08:00
Krish Dholakia
32013f63a0
Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages (#17459)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* fix: support multiple kinds of input to the guardrail api

* feat(guardrail_translation/handler.py): support extracting tool calls from openai chat completions for guardrail api's

* feat(generic_guardrail_api.py): support extracting + returning modified tool calls on generic_guardrails_api

allows guardrail api to analyze tool call being sent to provider - to run any analysis on it

* fix(guardrails.py): support anthropic /v1/messages tool calls

* feat(responses_api/): extract tool calls for guardrail processing

* docs(generic_guardrail_api.md): document tools param support

* docs: generic_guardrail_api.md

improve documentation
2025-12-03 21:20:39 -08:00
Krish Dholakia
be0530a6b3
fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail (#17424)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* test: update tests

* fix(bedrock_guardrails.py): fix post call streaming iterator logic

* fix: fix return

* fix(bedrock_guardrails.py): fix
2025-12-03 20:54:56 -08:00
Ishaan Jaffer
8a824b7c17 fix mypy linting 2025-12-03 19:43:29 -08:00
Sameer Kankute
50fc30644e
Merge pull request #17311 from Chesars/fix/azure-gpt5.1-reasoning-effort-none
Fix: Allow reasoning_effort='none' for Azure gpt-5.1 models
2025-12-04 09:08:32 +05:30
Ishaan Jaff
585aee2ae4
[Feat] Agent Gateway - Allow tracking request / response in "Logs" Page (#17449)
* init litellm A2a client

* simpler a2a client interface

* test a2a

* move a2a invoking tests

* test fix

* ensure a2a send message is tracked n logs

* rename tags

* add streaming handlng

* add a2a invocation

* add a2a invocation i cost calc

* test_a2a_logging_payload

* update invoke_agent_a2a

* test_invoke_agent_a2a_adds_litellm_data

* add A2a agent
2025-12-03 18:57:18 -08:00
yuneng-jiang
e29acb2f64
Merge pull request #16850 from BerriAI/litellm_user_info_404
[Fix] Return 404 when a user is not found on /user/info
2025-12-03 17:15:55 -08:00
yuneng-jiang
73824c278a
Merge pull request #17443 from BerriAI/litellm_v2_login
[Feature] New Login Page
2025-12-03 16:23:47 -08:00
yuneng-jiang
37c598441f Change is_sso_configured to auto_redirect_to_sso 2025-12-03 15:48:50 -08:00
yuneng-jiang
8a1cf104e0 Merge remote-tracking branch 'origin' into litellm_ui_config_add_sso 2025-12-03 15:36:33 -08:00
Ishaan Jaff
100cfc11ac
[Bug Fix] Parallel Request Limiter with /messages (#17426)
* fix: use standard_logging_object for parallel request limiter

* fix test parallel request limtier
2025-12-03 14:13:28 -08:00
yuneng-jiang
9bb292f478 V2 login route 2025-12-03 12:41:45 -08:00
Haji Akhundov
69b32dd870 Fix ruff linting errors: remove unused imports and variables 2025-12-03 21:06:32 +01:00
Haji Akhundov
e5baff780a Enforce SCIM 2.0 compliance: reject non-existent users in groups
- Reject group operations when users don't exist (security fix)
- Prevents unauthorized user provisioning via group membership
- Aligns with SCIM 2.0 protocol: users must exist before group membership
- Add validation for empty user IDs
- Update tests to verify rejection behavior

This is a breaking change but necessary for security and SCIM compliance.
Users must be created via POST /Users before being added to groups.
2025-12-03 20:44:54 +01:00
yuneng-jiang
cc92fdf90f Merge remote-tracking branch 'origin' into litellm_ui_callback_fix 2025-12-03 11:02:59 -08:00
yuneng-jiang
457840afed Merge remote-tracking branch 'origin' into litellm_user_info_404 2025-12-03 11:02:23 -08:00
yuneng-jiang
e6620fcdad Ruff checks 2025-12-03 11:01:10 -08:00
yuneng-jiang
b3c0ea5414 Merge remote-tracking branch 'origin' into litellm_login_route_refactor 2025-12-03 10:40:11 -08:00
Sameer Kankute
ece1e49fda
Merge pull request #17414 from BerriAI/litellm_ragflow_vector_store
Add vector store support for ragflow
2025-12-03 20:48:52 +05:30
Sameer Kankute
7e9c1ffb33
Merge pull request #17407 from BerriAI/litellm_enforce_enforce_user_param
Enforce support of enforce_user_param to openai post endpoints
2025-12-03 20:45:21 +05:30
Sameer Kankute
1fbe310444
Merge pull request #17405 from BerriAI/litellm_gemini_thought_sig_tool_beta
Make thought sign in tool call id as a beta feat
2025-12-03 20:43:53 +05:30
Haji Akhundov
8871a5670b Simplify SCIM tests to focus on core issue 2025-12-03 14:50:03 +01:00
Haji Akhundov
c01a4c6e57 Add tests for SCIM user_email UUID validation fix
- Test user with UUID in user_email (defensive fix scenario)
- Test user with None email (root cause fix scenario)
- Verifies transformation doesn't fail and emails array is empty for invalid emails
2025-12-03 14:41:10 +01:00
Sameer Kankute
8eaabb4ad7 Add vector store support for ragflow 2025-12-03 15:29:47 +05:30
Sameer Kankute
52090c3f3e
Merge pull request #17350 from BerriAI/litellm_rag_chat_completion_api
Add ragflow support for chat completions API
2025-12-03 13:29:32 +05:30
Sameer Kankute
ba59cebb57
Merge pull request #17408 from BerriAI/main
merge main
2025-12-03 12:33:21 +05:30
Krish Dholakia
8edcc4ecc3
Guardrails API - add streaming support (#17400)
* fix(initial-commit): adding a way to get the right response type based on the api route

* feat(unified_guardrail.py): support streaming guardrails

* test: update tests

* fix: fix linting errors

* test: update tests
2025-12-02 22:52:09 -08:00
Sameer Kankute
54e29e7828 Enforce support of enforce_user_param to openai post endpoints 2025-12-03 12:19:21 +05:30
rioiart
1ac2655b17
Fix/organization max budget not enforced (#17334)
* test: add failing tests for organization budget enforcement bug

Add comprehensive tests exposing that organization-level budgets are
retrieved but never enforced during request authentication. Tests verify:

1. Basic org budget exceeded scenario (team under budget, org over)
2. Multiple teams collectively exceeding org budget
3. Organization budget fields exist but are never checked
4. Inconsistency between team budget enforcement (works) and org (doesn't)

Tests intentionally fail to document the bug. Will be fixed in next commit.

Related to organization_max_budget not being enforced in auth_checks.py

* fix: enforce organization budget in auth checks

Add organization budget enforcement to common_checks() in auth_checks.py.
Previously, organization_max_budget was retrieved from DB but never checked,
allowing teams to collectively exceed their organization's budget limit.

Changes:
- Add _organization_max_budget_check() function following team budget pattern
- Call org budget check after team budget check in common_checks()
- Add "organization_budget" to budget_alerts type literals
- Update tests to verify org budget is enforced

Budget hierarchy is now properly enforced:
  Organization Budget (hard ceiling)
    └─ Team Budget (sub-allocation)
        └─ Team Member Budget (per-user within team)
            └─ Key Budget (per-key)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: add organization_id to budget alerts, fix enum comparison and linting of newly added code

- Add organization_id field to CallInfo class for better alert context
- Include organization_id in budget alerts (token, soft, team, org)
- Fix event_group enum comparison (was comparing enum to string)
- Add OrganizationBudgetAlert class for organization budget alerting
- Add organization_budget to test parameterizations
- Apply Black formatting to slack_alerting.py

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 22:46:03 -08:00
Matt Greathouse
f22bc0aab2
Support Deepseek 3.2 with Reasoning (#17384)
* Add openrouter/deepseek/deepseek-v3.2

* Added deepseek-provided v3.2

* Allow reasoning effort param for openrouter models that support it

* Added tests
2025-12-02 22:00:19 -08:00
Richard Song
099ccf56a7
Refactor add_schema_to_components to move definitions to components/schemas and add corresponding unit test (#17389) 2025-12-02 21:57:07 -08:00
Sameer Kankute
40c203e32b Make thought sign in tool call id as a beta feat 2025-12-03 11:04:53 +05:30
Sameer Kankute
a0819d6df0
Merge branch 'main' into litellm_vertex-bge-cherrypick 2025-12-03 08:37:04 +05:30
yuneng-jiang
7fea97a0c0 Add is_sso_configured to UI Config 2025-12-02 17:53:16 -08:00
flozonn
31cad8e6e6
feat: Add Nova lite 2 reasoning support with reasoningConfig (#17371) 2025-12-02 16:33:07 -08:00
Leslie Cheng
de4ff120eb
🐛 Fix proxy caching between requests in aiohttp transport (#17122)
* write a regression test

* impl fix

* add test for host case

* use the host as cache key
2025-12-02 14:37:45 -08:00
yuneng-jiang
6ee9d9c344 /login route refactor 2025-12-02 11:19:27 -08:00
Sameer Kankute
2201e12acc Fix import error 2025-12-02 22:08:23 +05:30
Ishaan Jaffer
fe03833d3b test_vertex_ai_bge_psc_endpoint_url_construction 2025-12-02 21:59:53 +05:30
Ishaan Jaffer
075a80b747 test_vertex_ai_bge_with_endpoint_id_pattern 2025-12-02 21:59:02 +05:30
Ishaan Jaffer
58d9531869 test_is_bge_model_detection 2025-12-02 21:59:02 +05:30
Ishaan Jaffer
0abf450b7e tes BGE 2025-12-02 21:59:02 +05:30
Ishaan Jaffer
8957770e68 test_vertex_ai_bge_embedding_with_custom_api_base 2025-12-02 21:59:02 +05:30
Ishaan Jaffer
f2befcf657 test_vertex_ai_bge_embedding_with_custom_api_base 2025-12-02 21:59:02 +05:30
Sameer Kankute
bf1308e86b Support for Custom Vertex AI Models via PSC Endpoint with api_base (#15953)
* Support for Custom Vertex AI Models via PSC Endpoint with api_base

* Add docs related psc

* remove not needed files

* remove print statemnt

* fix mypy errors
2025-12-02 21:58:58 +05:30
Sameer Kankute
831ad45c4d Add ragflow support 2025-12-02 18:18:08 +05:30