Commit graph

6832 commits

Author SHA1 Message Date
Alexsander Hamir
c0d149e0a9
Fix: Lack of None value checks & update publicai_chat_transformation tests (#17539)
* fix: handle none content

* fix: defensive check on none value

* Fix test failures: Azure OCR skip, None content handling, PublicAI JSON config

- Skip aocr/ocr call types in Azure test (they don't use Azure SDK client)
- Handle None content in Responses API transformation (skip message creation)
- Update PublicAI tests to use JSON-based configuration system
- Add None check in PublicAI test fixture to fix type error
2025-12-05 09:43:52 -08:00
Sameer Kankute
a21f1ce21f
Merge pull request #17528 from BerriAI/litellm_save_background_checks
Add background health checks to db
2025-12-05 22:25:03 +05:30
Sameer Kankute
558c8f92d1
Merge pull request #17519 from BerriAI/litellm_cursor_integration
Add support for cursor BYOK with its own configuration
2025-12-05 22:23:45 +05:30
Sameer Kankute
49a344ebd9
Merge pull request #17525 from BerriAI/litellm_fix_in_memory_vector_store
Fix vector store configuration synchronization failure
2025-12-05 22:23:17 +05:30
Sameer Kankute
5f23d94b7e Fixed media resoltion for gemini 3 2025-12-05 22:16:36 +05:30
Sameer Kankute
3d6b7f0d3d Add background health checks to db 2025-12-05 14:27:37 +05:30
Sameer Kankute
acc0b5fe27
Merge pull request #17362 from BerriAI/litellm_vertex-bge-cherrypick
[Feat] VertexAI - Add BGE Embeddings support
2025-12-05 11:53:42 +05:30
Sameer Kankute
99fd96687f Fix vector store configuration synchronization failure 2025-12-05 11:46:14 +05:30
Krish Dholakia
51cc102c30
fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514)
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
2025-12-04 22:06:13 -08:00
Cesar Garcia
316f7671a9
fix(gemini): handle partial JSON chunks after first valid chunk (#17496)
* fix(gemini): allow JSON accumulation on any chunk, not just first

* test(gemini): add tests for partial JSON chunk handling
2025-12-04 22:01:59 -08:00
Kristian Brünn
63fae79493
fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage (#17504)
* fix: optimize SpendLogs queries to use timestamp filtering (#17487)

* use timestamptz & enhance test
2025-12-04 21:52:57 -08:00
Devaj Mody
48b5100c18
fix(guardrails): mask all matching keywords in content filter (#17521)
Fixes #17517

  - Fixed bug where only the first matching blocked keyword was masked
  - Now iterates through ALL blocked keywords and masks each one
  - Added 3 regression tests for multiple keyword masking
2025-12-04 21:50:15 -08:00
yuneng-jiang
a96677c299 Fix /get/config/callbacks callback variables 2025-12-04 21:48:49 -08:00
Sameer Kankute
a6006e698c Add support for cursor BYOK with its own configuration 2025-12-05 09:34:49 +05:30
Dominic Feliton
6af693de36
(fix): empty response + vllm streaming (#17516)
* Fix empty response + vllm streaming

* Add unit test
2025-12-04 19:24:28 -08:00
Ishaan Jaff
b2e8d3fd42
[Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider (#17448)
* feat: Add JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Add simple JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Implement JSON-based provider config and migrate PublicAI

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* docs fix

* undo change

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 17:59:25 -08:00
yuneng-jiang
de8f0a3409 Ensure fresh data to prevent race condition in scim v2 2025-12-04 17:04:59 -08:00
Ishaan Jaff
8f592088ed
[Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
2025-12-04 16:31:00 -08:00
Ishaan Jaff
e3116da653
feat: Add /global/spend/tags to admin viewer routes (#17501)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 14:43:33 -08:00
Raghav Jhavar
72eb4c3a1c
🆕 feat: support routing to only websearch supported deployments (#17500)
* support routing to only websearch supported deployments

* add docs
2025-12-04 14:18:20 -08:00
yuneng-jiang
2e65c464ad Adding tests 2025-12-04 12:36:15 -08:00
Sameer Kankute
f2c0029939
Merge pull request #17470 from BerriAI/litellm_batches_bedrock_content
Add support for file content download for bedrock batches
2025-12-04 21:57:04 +05:30
Sameer Kankute
5b4542304d
Merge pull request #17461 from BerriAI/litellm_qwen2_imported_model_support
Add support for bedrock qwen 2 imported model
2025-12-04 21:56:22 +05:30
Sameer Kankute
edd392b50d Add support for file content download for bedrock batches 2025-12-04 13:27:53 +05:30
Sameer Kankute
4710e772be Add support for bedrock qwen 2 imported model 2025-12-04 11:08:57 +05:30
Cesar Garcia
6d831ffff4
fix(ocr): add missing ocr and aocr to CallTypes enum (#17435)
Add `ocr` and `aocr` entries to the CallTypes enum to fix the
ValueError that occurs when using the /v1/ocr endpoint with
guardrails enabled.

The OCR endpoint uses route_type="aocr", but the CallTypes enum
was missing these values, causing guardrail hooks to fail when
trying to instantiate CallTypes("aocr").

Fixes #17381
2025-12-03 21:28:13 -08:00
Andrew Truong
1879623502
feat: Basic Weave OTEL integration (#17439)
* test

* testfix
2025-12-03 21:26:22 -08:00
Haiyi
06d42fbd30
Fix: Skip empty text blocks in Anthropic system messages (#17442)
When translating system messages for the Anthropic API, empty text
content blocks cause the error "messages: text content blocks must be
non-empty". This fix skips empty string content and empty text blocks
in list content to prevent this error.

Fixes issue with Vertex AI Anthropic API calls.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-03 21:25:06 -08:00
Devaj Mody
a6ce118989
fix(openai): add clear error message for empty LLM endpoint responses (#17445)
Fixes #17209

  - Add validation after response.parse() to detect empty/invalid responses
  - Raise clear OpenAIError with actionable message
  - Add 3 unit tests
2025-12-03 21:23:09 -08:00
Krish Dholakia
32013f63a0
Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages (#17459)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* fix: support multiple kinds of input to the guardrail api

* feat(guardrail_translation/handler.py): support extracting tool calls from openai chat completions for guardrail api's

* feat(generic_guardrail_api.py): support extracting + returning modified tool calls on generic_guardrails_api

allows guardrail api to analyze tool call being sent to provider - to run any analysis on it

* fix(guardrails.py): support anthropic /v1/messages tool calls

* feat(responses_api/): extract tool calls for guardrail processing

* docs(generic_guardrail_api.md): document tools param support

* docs: generic_guardrail_api.md

improve documentation
2025-12-03 21:20:39 -08:00
Krish Dholakia
be0530a6b3
fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail (#17424)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* test: update tests

* fix(bedrock_guardrails.py): fix post call streaming iterator logic

* fix: fix return

* fix(bedrock_guardrails.py): fix
2025-12-03 20:54:56 -08:00
Ishaan Jaffer
8a824b7c17 fix mypy linting 2025-12-03 19:43:29 -08:00
Sameer Kankute
50fc30644e
Merge pull request #17311 from Chesars/fix/azure-gpt5.1-reasoning-effort-none
Fix: Allow reasoning_effort='none' for Azure gpt-5.1 models
2025-12-04 09:08:32 +05:30
Ishaan Jaff
585aee2ae4
[Feat] Agent Gateway - Allow tracking request / response in "Logs" Page (#17449)
* init litellm A2a client

* simpler a2a client interface

* test a2a

* move a2a invoking tests

* test fix

* ensure a2a send message is tracked n logs

* rename tags

* add streaming handlng

* add a2a invocation

* add a2a invocation i cost calc

* test_a2a_logging_payload

* update invoke_agent_a2a

* test_invoke_agent_a2a_adds_litellm_data

* add A2a agent
2025-12-03 18:57:18 -08:00
yuneng-jiang
e29acb2f64
Merge pull request #16850 from BerriAI/litellm_user_info_404
[Fix] Return 404 when a user is not found on /user/info
2025-12-03 17:15:55 -08:00
yuneng-jiang
73824c278a
Merge pull request #17443 from BerriAI/litellm_v2_login
[Feature] New Login Page
2025-12-03 16:23:47 -08:00
yuneng-jiang
37c598441f Change is_sso_configured to auto_redirect_to_sso 2025-12-03 15:48:50 -08:00
yuneng-jiang
8a1cf104e0 Merge remote-tracking branch 'origin' into litellm_ui_config_add_sso 2025-12-03 15:36:33 -08:00
Ishaan Jaff
100cfc11ac
[Bug Fix] Parallel Request Limiter with /messages (#17426)
* fix: use standard_logging_object for parallel request limiter

* fix test parallel request limtier
2025-12-03 14:13:28 -08:00
yuneng-jiang
9bb292f478 V2 login route 2025-12-03 12:41:45 -08:00
Haji Akhundov
69b32dd870 Fix ruff linting errors: remove unused imports and variables 2025-12-03 21:06:32 +01:00
Haji Akhundov
e5baff780a Enforce SCIM 2.0 compliance: reject non-existent users in groups
- Reject group operations when users don't exist (security fix)
- Prevents unauthorized user provisioning via group membership
- Aligns with SCIM 2.0 protocol: users must exist before group membership
- Add validation for empty user IDs
- Update tests to verify rejection behavior

This is a breaking change but necessary for security and SCIM compliance.
Users must be created via POST /Users before being added to groups.
2025-12-03 20:44:54 +01:00
yuneng-jiang
cc92fdf90f Merge remote-tracking branch 'origin' into litellm_ui_callback_fix 2025-12-03 11:02:59 -08:00
yuneng-jiang
457840afed Merge remote-tracking branch 'origin' into litellm_user_info_404 2025-12-03 11:02:23 -08:00
yuneng-jiang
e6620fcdad Ruff checks 2025-12-03 11:01:10 -08:00
yuneng-jiang
b3c0ea5414 Merge remote-tracking branch 'origin' into litellm_login_route_refactor 2025-12-03 10:40:11 -08:00
Sameer Kankute
ece1e49fda
Merge pull request #17414 from BerriAI/litellm_ragflow_vector_store
Add vector store support for ragflow
2025-12-03 20:48:52 +05:30
Sameer Kankute
7e9c1ffb33
Merge pull request #17407 from BerriAI/litellm_enforce_enforce_user_param
Enforce support of enforce_user_param to openai post endpoints
2025-12-03 20:45:21 +05:30
Sameer Kankute
1fbe310444
Merge pull request #17405 from BerriAI/litellm_gemini_thought_sig_tool_beta
Make thought sign in tool call id as a beta feat
2025-12-03 20:43:53 +05:30
Haji Akhundov
8871a5670b Simplify SCIM tests to focus on core issue 2025-12-03 14:50:03 +01:00