Commit graph

1087 commits

Author SHA1 Message Date
yuneng-jiang
5afd03fef3 Mock server_root_path for test 2025-12-05 16:13:56 -08:00
Dominic Fallows
2ffe8ee204
fix(presidio): handle empty content and error dict responses (#17489)
- Skip empty/whitespace text before calling Presidio API
- Handle error dict responses gracefully (e.g., {'error': 'No text provided'})
- Add defensive error handling for invalid result items
- Add comprehensive test coverage for empty content scenarios

Fixes crash in tool/function calling where assistant messages have empty content.
2025-12-05 15:45:19 -08:00
yuneng-jiang
cb18af542e
Merge pull request #17498 from BerriAI/litellm_customer_usage_backend
[Feature] Customer (end user) Usage
2025-12-05 15:31:08 -08:00
Devaj Mody
6ff7ed14f6
fix(team): use organization.members instead of deprecated organization.users (#17557)
Fixes #17552

  - Change Prisma include from 'users' to 'members'
  - Use LiteLLM_OrganizationTableWithMembers type for membership validation
  - Access organization.members instead of organization.users
  - Add tests for membership validation
2025-12-05 15:30:59 -08:00
Ishaan Jaff
769f3cc310
[Bug fix] Secret Managers Integration - Make email and secret manager operations independent in key management hooks (#17551)
* TestKeyManagementEventHooksIndependentOperations

* KeyManagementEventHooks - make ops independant
2025-12-05 15:26:00 -08:00
Ishaan Jaff
a78f40f75a
[Fixes] Dynamic Rate Limiter - Dynamic rate limiting token count increases/decreases by 1 instead of actual count + Redis TTL (#17558)
* fix async_log_success_event for _PROXY_DynamicRateLimitHandlerV3

* test_async_log_success_event_increments_by_actual_tokens

* fix redis TTL

* Potential fix for code scanning alert no. 3873: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-05 15:25:45 -08:00
yuneng-jiang
f6804333b8 New user route user_id conflict handling 2025-12-05 15:08:13 -08:00
yuneng-jiang
0dd4db34bd Working setting generic callbacks on UI 2025-12-05 14:37:48 -08:00
yuneng-jiang
98f9124444 Merge remote-tracking branch 'origin' into litellm_custom_webhook_fix 2025-12-05 12:48:37 -08:00
yuneng-jiang
62045477ba
Merge pull request #16335 from BerriAI/litellm_ui_callback_fix
[Feature] Show all callbacks on UI
2025-12-05 12:35:58 -08:00
Ishaan Jaff
6021f31ebc
Fix: Allow null max_budget in budget update endpoint (#17545)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-05 11:45:23 -08:00
yuneng-jiang
4a0893ca22 Merge remote-tracking branch 'origin' into litellm_ui_callback_fix 2025-12-05 11:43:35 -08:00
Sameer Kankute
a21f1ce21f
Merge pull request #17528 from BerriAI/litellm_save_background_checks
Add background health checks to db
2025-12-05 22:25:03 +05:30
Sameer Kankute
558c8f92d1
Merge pull request #17519 from BerriAI/litellm_cursor_integration
Add support for cursor BYOK with its own configuration
2025-12-05 22:23:45 +05:30
Sameer Kankute
49a344ebd9
Merge pull request #17525 from BerriAI/litellm_fix_in_memory_vector_store
Fix vector store configuration synchronization failure
2025-12-05 22:23:17 +05:30
Sameer Kankute
3d6b7f0d3d Add background health checks to db 2025-12-05 14:27:37 +05:30
Sameer Kankute
99fd96687f Fix vector store configuration synchronization failure 2025-12-05 11:46:14 +05:30
Krish Dholakia
51cc102c30
fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514)
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
2025-12-04 22:06:13 -08:00
Kristian Brünn
63fae79493
fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage (#17504)
* fix: optimize SpendLogs queries to use timestamp filtering (#17487)

* use timestamptz & enhance test
2025-12-04 21:52:57 -08:00
Devaj Mody
48b5100c18
fix(guardrails): mask all matching keywords in content filter (#17521)
Fixes #17517

  - Fixed bug where only the first matching blocked keyword was masked
  - Now iterates through ALL blocked keywords and masks each one
  - Added 3 regression tests for multiple keyword masking
2025-12-04 21:50:15 -08:00
yuneng-jiang
a96677c299 Fix /get/config/callbacks callback variables 2025-12-04 21:48:49 -08:00
Sameer Kankute
a6006e698c Add support for cursor BYOK with its own configuration 2025-12-05 09:34:49 +05:30
yuneng-jiang
de8f0a3409 Ensure fresh data to prevent race condition in scim v2 2025-12-04 17:04:59 -08:00
Ishaan Jaff
8f592088ed
[Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
2025-12-04 16:31:00 -08:00
Ishaan Jaff
e3116da653
feat: Add /global/spend/tags to admin viewer routes (#17501)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 14:43:33 -08:00
yuneng-jiang
2e65c464ad Adding tests 2025-12-04 12:36:15 -08:00
Krish Dholakia
32013f63a0
Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages (#17459)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* fix: support multiple kinds of input to the guardrail api

* feat(guardrail_translation/handler.py): support extracting tool calls from openai chat completions for guardrail api's

* feat(generic_guardrail_api.py): support extracting + returning modified tool calls on generic_guardrails_api

allows guardrail api to analyze tool call being sent to provider - to run any analysis on it

* fix(guardrails.py): support anthropic /v1/messages tool calls

* feat(responses_api/): extract tool calls for guardrail processing

* docs(generic_guardrail_api.md): document tools param support

* docs: generic_guardrail_api.md

improve documentation
2025-12-03 21:20:39 -08:00
Krish Dholakia
be0530a6b3
fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail (#17424)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* test: update tests

* fix(bedrock_guardrails.py): fix post call streaming iterator logic

* fix: fix return

* fix(bedrock_guardrails.py): fix
2025-12-03 20:54:56 -08:00
Ishaan Jaffer
8a824b7c17 fix mypy linting 2025-12-03 19:43:29 -08:00
Ishaan Jaff
585aee2ae4
[Feat] Agent Gateway - Allow tracking request / response in "Logs" Page (#17449)
* init litellm A2a client

* simpler a2a client interface

* test a2a

* move a2a invoking tests

* test fix

* ensure a2a send message is tracked n logs

* rename tags

* add streaming handlng

* add a2a invocation

* add a2a invocation i cost calc

* test_a2a_logging_payload

* update invoke_agent_a2a

* test_invoke_agent_a2a_adds_litellm_data

* add A2a agent
2025-12-03 18:57:18 -08:00
yuneng-jiang
e29acb2f64
Merge pull request #16850 from BerriAI/litellm_user_info_404
[Fix] Return 404 when a user is not found on /user/info
2025-12-03 17:15:55 -08:00
yuneng-jiang
73824c278a
Merge pull request #17443 from BerriAI/litellm_v2_login
[Feature] New Login Page
2025-12-03 16:23:47 -08:00
yuneng-jiang
37c598441f Change is_sso_configured to auto_redirect_to_sso 2025-12-03 15:48:50 -08:00
yuneng-jiang
8a1cf104e0 Merge remote-tracking branch 'origin' into litellm_ui_config_add_sso 2025-12-03 15:36:33 -08:00
Ishaan Jaff
100cfc11ac
[Bug Fix] Parallel Request Limiter with /messages (#17426)
* fix: use standard_logging_object for parallel request limiter

* fix test parallel request limtier
2025-12-03 14:13:28 -08:00
yuneng-jiang
9bb292f478 V2 login route 2025-12-03 12:41:45 -08:00
Haji Akhundov
69b32dd870 Fix ruff linting errors: remove unused imports and variables 2025-12-03 21:06:32 +01:00
Haji Akhundov
e5baff780a Enforce SCIM 2.0 compliance: reject non-existent users in groups
- Reject group operations when users don't exist (security fix)
- Prevents unauthorized user provisioning via group membership
- Aligns with SCIM 2.0 protocol: users must exist before group membership
- Add validation for empty user IDs
- Update tests to verify rejection behavior

This is a breaking change but necessary for security and SCIM compliance.
Users must be created via POST /Users before being added to groups.
2025-12-03 20:44:54 +01:00
yuneng-jiang
cc92fdf90f Merge remote-tracking branch 'origin' into litellm_ui_callback_fix 2025-12-03 11:02:59 -08:00
yuneng-jiang
457840afed Merge remote-tracking branch 'origin' into litellm_user_info_404 2025-12-03 11:02:23 -08:00
yuneng-jiang
e6620fcdad Ruff checks 2025-12-03 11:01:10 -08:00
yuneng-jiang
b3c0ea5414 Merge remote-tracking branch 'origin' into litellm_login_route_refactor 2025-12-03 10:40:11 -08:00
Sameer Kankute
7e9c1ffb33
Merge pull request #17407 from BerriAI/litellm_enforce_enforce_user_param
Enforce support of enforce_user_param to openai post endpoints
2025-12-03 20:45:21 +05:30
Haji Akhundov
8871a5670b Simplify SCIM tests to focus on core issue 2025-12-03 14:50:03 +01:00
Haji Akhundov
c01a4c6e57 Add tests for SCIM user_email UUID validation fix
- Test user with UUID in user_email (defensive fix scenario)
- Test user with None email (root cause fix scenario)
- Verifies transformation doesn't fail and emails array is empty for invalid emails
2025-12-03 14:41:10 +01:00
Krish Dholakia
8edcc4ecc3
Guardrails API - add streaming support (#17400)
* fix(initial-commit): adding a way to get the right response type based on the api route

* feat(unified_guardrail.py): support streaming guardrails

* test: update tests

* fix: fix linting errors

* test: update tests
2025-12-02 22:52:09 -08:00
Sameer Kankute
54e29e7828 Enforce support of enforce_user_param to openai post endpoints 2025-12-03 12:19:21 +05:30
rioiart
1ac2655b17
Fix/organization max budget not enforced (#17334)
* test: add failing tests for organization budget enforcement bug

Add comprehensive tests exposing that organization-level budgets are
retrieved but never enforced during request authentication. Tests verify:

1. Basic org budget exceeded scenario (team under budget, org over)
2. Multiple teams collectively exceeding org budget
3. Organization budget fields exist but are never checked
4. Inconsistency between team budget enforcement (works) and org (doesn't)

Tests intentionally fail to document the bug. Will be fixed in next commit.

Related to organization_max_budget not being enforced in auth_checks.py

* fix: enforce organization budget in auth checks

Add organization budget enforcement to common_checks() in auth_checks.py.
Previously, organization_max_budget was retrieved from DB but never checked,
allowing teams to collectively exceed their organization's budget limit.

Changes:
- Add _organization_max_budget_check() function following team budget pattern
- Call org budget check after team budget check in common_checks()
- Add "organization_budget" to budget_alerts type literals
- Update tests to verify org budget is enforced

Budget hierarchy is now properly enforced:
  Organization Budget (hard ceiling)
    └─ Team Budget (sub-allocation)
        └─ Team Member Budget (per-user within team)
            └─ Key Budget (per-key)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: add organization_id to budget alerts, fix enum comparison and linting of newly added code

- Add organization_id field to CallInfo class for better alert context
- Include organization_id in budget alerts (token, soft, team, org)
- Fix event_group enum comparison (was comparing enum to string)
- Add OrganizationBudgetAlert class for organization budget alerting
- Add organization_budget to test parameterizations
- Apply Black formatting to slack_alerting.py

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 22:46:03 -08:00
Richard Song
099ccf56a7
Refactor add_schema_to_components to move definitions to components/schemas and add corresponding unit test (#17389) 2025-12-02 21:57:07 -08:00
yuneng-jiang
7fea97a0c0 Add is_sso_configured to UI Config 2025-12-02 17:53:16 -08:00