scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
28214 commits 3222 branches 1436 tags 4.7 GiB
Commit graph

28214 commits

This branch
This branch
All branches
Author SHA1 Message Date
Sameer Kankute
acc0b5fe27
Merge pull request #17362 from BerriAI/litellm_vertex-bge-cherrypick 
[Feat] VertexAI - Add BGE Embeddings support
 2025-12-05 11:53:42 +05:30
Krish Dholakia
b3a3081e8e
Guardrails API - new structured_messages param (#17518) 
* fix(generic_guardrail_api.py): add 'structured_messages' support

allows guardrail provider to know if text is from system or user

* fix(generic_guardrail_api.md): document 'structured_messages' parameter

give api provider a way to distinguish between user and system messages

* feat(anthropic/): return openai chat completion format structured messages when calls made via `/v1/messages` on Anthropic

* feat(responses/guardrail_translation): support 'structured_messages' param for guardrails

structured openai chat completion spec messages, for guardrail checks when using /v1/responses api

allows guardrail checks to work consistently across APIs
 2025-12-04 22:08:00 -08:00
Krish Dholakia
51cc102c30
fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514) 
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
 2025-12-04 22:06:13 -08:00
Cesar Garcia
316f7671a9
fix(gemini): handle partial JSON chunks after first valid chunk (#17496) 
* fix(gemini): allow JSON accumulation on any chunk, not just first

* test(gemini): add tests for partial JSON chunk handling
 2025-12-04 22:01:59 -08:00
Kristian Brünn
63fae79493
fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage (#17504) 
* fix: optimize SpendLogs queries to use timestamp filtering (#17487)

* use timestamptz & enhance test
 2025-12-04 21:52:57 -08:00
Krish Dholakia
8776336c3c
Enable detailed debugging for reference (#17508) 
* Deprecate set_verbose in favor of LITELLM_LOG

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Update debugging documentation links

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-12-04 21:51:56 -08:00
Devaj Mody
48b5100c18
fix(guardrails): mask all matching keywords in content filter (#17521) 
Fixes #17517

  - Fixed bug where only the first matching blocked keyword was masked
  - Now iterates through ALL blocked keywords and masks each one
  - Added 3 regression tests for multiple keyword masking
 2025-12-04 21:50:15 -08:00
Dominic Feliton
6af693de36
(fix): empty response + vllm streaming (#17516) 
* Fix empty response + vllm streaming

* Add unit test
 2025-12-04 19:24:28 -08:00
YutaSaito
7106509ef0
chore: add stack trace to mcp error message (#17269) 2025-12-04 19:19:00 -08:00
Ishaan Jaffer
4f3b843efe docs openai 2025-12-04 18:32:23 -08:00
Ishaan Jaff
b2e8d3fd42
[Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider (#17448) 
* feat: Add JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Add simple JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Implement JSON-based provider config and migrate PublicAI

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* docs fix

* undo change

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-12-04 17:59:25 -08:00
Ishaan Jaff
fadfbb13d3
[Docs] A2a - Permission management (#17515) 
* docs add a2a gateway + mcp gateway

* docs a2a permissions

* docs a2a permission

* docs

* docs a2a

* docs a2a

* add new img

* docs agent permissions
 2025-12-04 17:29:47 -08:00
Ishaan Jaffer
dfe3124335 UI new build 2025-12-04 16:34:59 -08:00
Ishaan Jaffer
5cc1a5031b fix invoke_agent_a2a 2025-12-04 16:32:15 -08:00
Ishaan Jaff
575e769bff
[Feat] UI - Agent Gateway - set allowed agents by key, team (#17511) 
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route

* add AgentPermissionsProps

* add agents on team/key create

* add agent selector on team/key

* add agent selector on key edit /info

* add AgentPermissions

* docs list + invoke agents
 2025-12-04 16:31:17 -08:00
Ishaan Jaff
8f592088ed
[Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502) 
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
 2025-12-04 16:31:00 -08:00
Ishaan Jaff
e3116da653
feat: Add /global/spend/tags to admin viewer routes (#17501) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-12-04 14:43:33 -08:00
Ishaan Jaffer
c6e26a20b1 refactor invoke 2025-12-04 14:21:01 -08:00
Raghav Jhavar
72eb4c3a1c
🆕 feat: support routing to only websearch supported deployments (#17500) 
* support routing to only websearch supported deployments

* add docs
 2025-12-04 14:18:20 -08:00
Anas AbdelR
2abcc77944
fix: resolve ruff lint errors (#17490) 
Fixed 20 of 22 lint errors:

- batches/batch_utils.py: Added missing model_name parameter to functions
- integrations/custom_guardrail.py: Removed unused Tuple import
- llms/custom_httpx/http_handler.py: Removed unused AIOHTTP_NEEDS_CLEANUP_CLOSED import
- llms/anthropic/chat/guardrail_translation/handler.py: Prefixed unused variable with underscore
- llms/openai/responses/guardrail_translation/handler.py: Removed duplicate BaseModel import, prefixed unused variable
- llms/pass_through/guardrail_translation/handler.py: Prefixed unused variables with underscore
- proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py: Removed unused List and Tuple imports
- proxy/hooks/parallel_request_limiter_v3.py: Removed unused import

Remaining 2 errors are PLR0915 (too many statements) which require refactoring.
 2025-12-04 14:12:57 -08:00
Yannay Hammer
2627f0d519
Fix aim security guardrail tests (#17499) 2025-12-04 13:29:20 -08:00
Krrish Dholakia
5aeba81538 docs(multi_tenant_architecture.md): add new architecture doc 2025-12-04 11:13:50 -08:00
Ishaan Jaff
8ee8cad4f1
feat - new model amazon.titan-image-generator-v2:0 (#17488) 2025-12-04 09:26:55 -08:00
Sameer Kankute
31f1bf0539
Merge pull request #17485 from BerriAI/litellm_ragflow_test_fix 
fix: test_default_api_base for ragfow
 2025-12-04 21:57:18 +05:30
Sameer Kankute
f2c0029939
Merge pull request #17470 from BerriAI/litellm_batches_bedrock_content 
Add support for file content download for bedrock batches
 2025-12-04 21:57:04 +05:30
Sameer Kankute
5b4542304d
Merge pull request #17461 from BerriAI/litellm_qwen2_imported_model_support 
Add support for bedrock qwen 2 imported model
 2025-12-04 21:56:22 +05:30
Sameer Kankute
1bd4050921
Merge pull request #17458 from BerriAI/litellm_map_output_tokens_details 
Map output_tokens_details of responses API to completion_tokens_details
 2025-12-04 21:55:21 +05:30
Sameer Kankute
6751badf3a fix: test_default_api_base for ragfow 2025-12-04 21:49:33 +05:30
Sameer Kankute
46ebf425d5 Fix : test_vertexai_model_garden_model_completion 2025-12-04 21:41:02 +05:30
Sameer Kankute
edd392b50d Add support for file content download for bedrock batches 2025-12-04 13:27:53 +05:30
yuneng-jiang
94b944e8ec
Merge pull request #17465 from BerriAI/litellm_ui_model_page_perf_2 
[Infra] UI - Change Credentials to use React Query
 2025-12-03 23:53:40 -08:00
yuneng-jiang
ad86dfbf7d
Merge pull request #17451 from BerriAI/litellm_new_login_page_sso_changes 
[Feature] Add Auto Redirect to SSO to New Login Page
 2025-12-03 23:53:21 -08:00
yuneng-jiang
5496e622c1 Unit tests 2025-12-03 23:46:09 -08:00
yuneng-jiang
59b32c28a8 Change selector in test 2025-12-03 23:37:15 -08:00
yuneng-jiang
0483336c53 change test to follow other logins 2025-12-03 23:21:09 -08:00
yuneng-jiang
19dc5d750c Adding waits 2025-12-03 23:10:32 -08:00
yuneng-jiang
45a9723467 view_user_info debugging 2025-12-03 22:20:04 -08:00
yuneng-jiang
b8190c6b6d resolving build issues 2025-12-03 22:15:19 -08:00
yuneng-jiang
7519a3e30b Change credentials to use react-query 2025-12-03 22:08:41 -08:00
Krish Dholakia
dc7c2b9b05
Update docs to link agent hub (#17462) 
* Docs: Add AI Hub agent registry documentation

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Fix: Update AI Hub link in A2A documentation

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-12-03 21:59:45 -08:00
yuneng-jiang
c634c08818 Find inputs by placeholder 2025-12-03 21:51:45 -08:00
Sameer Kankute
4710e772be Add support for bedrock qwen 2 imported model 2025-12-04 11:08:57 +05:30
Daniel Klein
3134fc825b
fix: make bedrock converse messages respect modify_params as expected (#17427) 2025-12-03 21:34:50 -08:00
Cesar Garcia
664b384dce
fix(pricing): add missing Moonshot turbo models and fix incorrect pricing (#17432) 
Fixes #17417

- Add moonshot/kimi-k2-turbo-preview ($1.15/$8.00 input/output)
- Add moonshot/kimi-k2-thinking-turbo ($1.15/$8.00 input/output)
- Add moonshot/kimi-k2-0905-preview ($0.60/$2.50 input/output, 262K context)
- Fix moonshot/kimi-thinking-preview pricing ($30/$30 -> $0.60/$2.50)
- Fix wandb/moonshotai/Kimi-K2-Instruct absurd pricing ($135K/$400K -> $0.60/$2.50)
 2025-12-03 21:31:49 -08:00
Cesar Garcia
b05f6d3cfd
fix(groq): check supports_response_schema before using json_tool_call workaround (#17438) 
Fixes issue where LiteLLM used the json_tool_call workaround for all Groq
models, even those that support native json_schema (e.g., gpt-oss-120b,
llama-4, kimi-k2). This caused errors when users passed their own tools
alongside structured outputs.

Changes:
- Check `supports_response_schema()` before using the workaround
- Only use json_tool_call workaround for models without native support
- Add clear error message when using workaround with user-provided tools
- Update model config to set `supports_response_schema: false` for models
  that don't support native json_schema
- Add unit tests for structured output handling
 2025-12-03 21:31:13 -08:00
Cesar Garcia
6d831ffff4
fix(ocr): add missing ocr and aocr to CallTypes enum (#17435) 
Add `ocr` and `aocr` entries to the CallTypes enum to fix the
ValueError that occurs when using the /v1/ocr endpoint with
guardrails enabled.

The OCR endpoint uses route_type="aocr", but the CallTypes enum
was missing these values, causing guardrail hooks to fail when
trying to instantiate CallTypes("aocr").

Fixes #17381
 2025-12-03 21:28:13 -08:00
Andrew Truong
1879623502
feat: Basic Weave OTEL integration (#17439) 
* test

* testfix
 2025-12-03 21:26:22 -08:00
Haiyi
06d42fbd30
Fix: Skip empty text blocks in Anthropic system messages (#17442) 
When translating system messages for the Anthropic API, empty text
content blocks cause the error "messages: text content blocks must be
non-empty". This fix skips empty string content and empty text blocks
in list content to prevent this error.

Fixes issue with Vertex AI Anthropic API calls.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
 2025-12-03 21:25:06 -08:00
Devaj Mody
a6ce118989
fix(openai): add clear error message for empty LLM endpoint responses (#17445) 
Fixes #17209

  - Add validation after response.parse() to detect empty/invalid responses
  - Raise clear OpenAIError with actionable message
  - Add 3 unit tests
 2025-12-03 21:23:09 -08:00
codgician
adfbb1c308
docs: document responses and embedding api for github copilot (#17456) 2025-12-03 21:22:08 -08:00