Commit graph

45 commits

Author SHA1 Message Date
yuneng-jiang
b60d94d655 addressing comments 2026-02-05 21:27:42 -08:00
yuneng-jiang
ec5dc0be51 email soft budgets 2026-02-05 20:42:43 -08:00
yuneng-jiang
dad2a29b2b Max budget alerts 2025-12-17 15:04:20 -08:00
yuneng-jiang
14feda694b Email budget alerts working 2025-12-15 16:51:25 -08:00
yuneng-jiang
7a18c81cf4 bump: version 0.1.25 → 0.1.26 2025-12-15 16:00:37 -08:00
yuneng-jiang
d8663efeab Revert "Base commit"
This reverts commit 1186cc6966.
2025-12-15 10:25:22 -08:00
yuneng-jiang
1186cc6966 Base commit 2025-12-15 10:24:47 -08:00
yuneng-jiang
439d42ba67 bump: version 0.1.24 → 0.1.25 2025-12-10 18:26:07 -08:00
yuneng-jiang
0769a290da Sendgrid integration + bump enterprise package 2025-12-09 19:34:09 -08:00
Krish Dholakia
1cb5fcddba
make generic api OSS + support multiple generic API's (#17152)
* feat(generic_api_callback.py): make generic api OSS + support multiple generic API's

Enables https://github.com/BerriAI/litellm/pull/17094#discussion_r2562832967

* feat(callback_utils.py): support custom generic api callbacks

* feat(generic_api_callback.py): support specifying which event types to run the generic api for

* fix(litellm_logging.py): log system prompt for anthropic messages

* feat(generic_api_callback.py): support generic api compatible api's - e.g. rubrik agent cloud

* docs(sidebars.js): document new OSS generic api

* docs(generic_api.md): document new OSS Generic API

* docs(custom_webhook_api.md): document custom webhook api integration tutorial

* docs(custom_webhook_api.md): cleanup

* docs(custom_webhook_api.md): document what get's logged to custom webhook api

* Refactor: Pass callback config to GenericAPILogger

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Fix: Handle empty messages list in logging payload

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Checkpoint before follow-up message

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: Cache GenericAPILogger instances to improve performance

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-11-26 18:38:38 -08:00
Ishaan Jaff
83ea037bc9
[Feat] AI Gateway - Allow admins to disable, dynamic callback controls (#16750)
* add flag to allow_dynamic_callback_disabling

* fix EnterpriseCallbackControls

* test controls

* add docs on dynamic logging
2025-11-17 18:29:07 -08:00
Krish Dholakia
06906534b3
feat(audio_transcriptions/): calculate duration of audio file for cost calculation + feat (image_generations): cost tracking accuracy improved with output_format, quality, size values fixed per openai model
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks

* fix: fix aembedding

* fix: fix ruff errors

* fix: modify to catch errors

* fix: test

* fix: loosen test to handle openai lib out of sync

* fix: fix base models

* fix: fix usage object
2025-11-08 16:24:31 -08:00
Krish Dholakia
202eaeb1a2
Revert "(feat) Audio transcription - cost tracking + (feat) image generation …" (#16409)
This reverts commit c96da44265.
2025-11-08 15:38:16 -08:00
Krish Dholakia
c96da44265
(feat) Audio transcription - cost tracking + (feat) image generation - accurate cost tracking based on output_format/quality/size
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks

* fix: fix aembedding

* fix: fix ruff errors

* fix: modify to catch errors

* fix: test

* fix: loosen test to handle openai lib out of sync
2025-11-08 15:30:46 -08:00
Ishaan Jaffer
0a1fc0eeb2 Revert "fix: fix ruff errors"
This reverts commit eef864360e.
2025-11-08 14:33:51 -08:00
Krrish Dholakia
eef864360e fix: fix ruff errors 2025-11-08 14:15:12 -08:00
Ishaan Jaff
2541a54023
[Feat] Email Notifications - Ensure Users get Key Rotated Email (#16292)
* add virtual_key_rotated

* add KEY_ROTATED_EMAIL_TEMPLATE

* fix base email

* add _send_key_rotated_email

* fix _send_key_rotated_email

* key rotated email fix

* fix: async_key_generated_hook ensure behavior matches regenerate

* test_send_key_rotated_email

* expose EMAIL_INCLUDE_API_KEY

* docs mask api keys

* add email regen

* docs key regen

* docs email

* docs email

* docs fix

* fix code qa check
2025-11-05 15:56:49 -08:00
Ishaan Jaff
cea318330e
[Feat] Add Guardrails for /v1/messages and /v1/responses API (#15686)
* add get_guardrails_messages_for_call_type

* fix call type for /messages

* add anthropic endpoints

* fix bedrock guardrails

* fix config.yaml

* fix types

* fix async_pre_call_hook

* ruff fix

* fix guard

* fix test bedrock guardrail

* fix linting

* fix linting

* docs guardrails

* fix mypy linting
2025-10-17 18:09:00 -07:00
Krrish Dholakia
b4c4776f56 fix: fix linting errors 2025-10-04 14:47:13 -07:00
Alexsander Hamir
eaa04cd8ce
fix: use fastuuid helper (#14903)
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
2025-09-25 15:47:01 -07:00
Mubashir Osmani
8b804303ed
fix: ci/cd tests + lint errors (#14646)
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-09-17 17:06:43 -07:00
Ishaan Jaff
555efd60b5
[Security] Fix: Ensure .info() logs are not used for request/responses + Add code QA check for possible violations (#14386)
* test: info log

* fixes for debug

* add info_log_check

* fix: debug level

* fix: debug level

* fix: use .debug

* fix info statements

* fix info log check

* fix: fix emit_standard_logging_payload

* fix RequestBody

* Potential fix for code scanning alert no. 3339: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix check

* fix code qa check

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-09-09 13:55:56 -07:00
Ishaan Jaff
4d941c914e
[Feat] Responses API Session Handling - Multi media support (#13347)
* rename ResponsesSessionHandler

* use ResponsesSessionHandler

* test session handler

* refactor ResponsesSessionHandler

* fix get_proxy_server_request_from_spend_log

* use constant for LITELLM_TRUNCATED_PAYLOAD_FIELD

* add _should_check_cold_storage_for_full_payload

* add get_class_type_for_custom_logger_name

* get_active_custom_logger_for_callback_name

* add get_proxy_server_request_from_cold_storage to CustomLogger

* add ColdStorageHandler

* start using cold storage integration

* add get_proxy_server_request_from_cold_storage

* fixes from manual testing

* s3 v2 fix getting region name

* ChatCompletionImageUrlObject

* use _get_configured_cold_storage_custom_logger

* fixes for _should_check_cold_storage_for_full_payload

* fix _download_object_from_s3

* test_s3_v2_with_cold_storage

* add cold_storage_object_key to StandardLoggingMetadata

* use get_proxy_server_request_from_cold_storage_with_object_key

* add cold_storage_object_key to SpendLogsMetadata

* add cold_storage_object_key

* get_proxy_server_request_from_cold_storage_with_object_key

* use get_proxy_server_request_from_cold_storage_with_object_key

* test responses API

* add get_proxy_server_request_from_cold_storage_with_object_key

* session handler fixes

* test session handler

* fix ruff checks

* _download_object_from_s3

* cleanup

* test

* lint fix

* test_e2e_cold_storage_successful_retrieval

* test_e2e_generate_cold_storage_object_key_successful

* test_async_gcs_pub_sub_v1

* test fix

* test fix

* test fix

* test_standard_logging_metadata_has_cold_storage_object_key_field

* test_sanitize_request_body_for_spend_logs_payload_basic

* test_transform_input_image_item_to_image_item_with_image_data
2025-08-07 10:59:53 -07:00
Jugal D. Bhatt
900c7f45c0
[MCP Gateway] Litellm mcp pre and during guardrails (#13188)
* add guardrail support

* add guardrail support

* guardrails for MCP

* added changes

* add mcp guardrails

* added test

* add ui

* fix guardrail form

* working with cursor

* remvoe print

* fix mcp servertests

* fix mypy and remove console logs

* fix mypy and remove console logs

* fix mypy tests
2025-08-01 20:02:25 -07:00
Ishaan Jaff
06574a72b5
[Feat] Backend - Add support for disabling callbacks in request body (#12762)
* allow using standard_callback_dynamic_params to disable callbacks

* fix is_callback_disabled_dynamically

* test_callback_disabled_via_request_body_multiple
2025-07-19 10:10:30 -07:00
Ishaan Jaff
57b0b4edf3
[Bug fix] [Bug]: Verbose log is enabled by default (#12596)
* test find_set_verbose_assignments

* fix set verbose

* test set verbose

* fix unused import
2025-07-14 20:06:16 -07:00
Jugal D. Bhatt
d55712b783
Add MCP url masking on frontend (#12247)
* add url masking on frontend

* fix test

* add signature
2025-07-02 16:32:26 -07:00
Jugal Bhatt
e53eb6eb37 change logger 2025-07-01 16:28:53 -07:00
Jugal Bhatt
7e6d27e6c3 change logger 2025-07-01 16:28:34 -07:00
Jugal Bhatt
fefc5fc33a add warning message for non premium users 2025-07-01 16:27:31 -07:00
Jugal Bhatt
69670001de email_templates 2025-07-01 14:34:15 -07:00
Ishaan Jaff
cfc07f361b
[Bug Fix] Invite links email should contain the correct invite id (#12130)
* use common helper create_invitation_for_user

* use common util in proxy

* fix create_invitation_for_user

* refactor base email

* test_get_invitation_link_creates_new_when_none_exist

* fix code QA checks
2025-06-28 09:19:32 -07:00
Ishaan Jaff
8c5fb6f539
[Feat] Enterprise - Allow dynamically disabling callbacks in request headers (#11985)
* Add support for disabling callbacks via x-litellm-disable-callbacks header

* add _is_callback_disabled_via_headers

* add get_proxy_server_request_headers

* _is_callback_disabled_via_headers

* X_LITELLM_DISABLE_CALLBACKS

* add EnterpriseCallbackControls

* use EnterpriseCallbackControls

* use CustomLoggerRegistry

* use CustomLoggerRegistry

* CustomLoggerRegistry

* EnterpriseCallbackControls

* TestEnterpriseCallbackControls

* docs clean up

* docs dynamic callbacks

* doc fixes

* fix code qa checks

* fix CustomLoggerRegistry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-23 14:32:05 -07:00
Krish Dholakia
c92b6c175c
Prometheus - fix request increment + add route tracking for streaming requests (#11731)
* fix(prometheus.py): remove request increment from inside the log success event

it's only done on post-call success/failure

* fix(litellm_logging.py): add additional validation step for checking if 'stream' is true

prevent double counting on non-stream requests

* test: add unit testing to ensure stream is not incorrectly set to true

* feat(litellm_logging.py): emit request route in standard logging payload

used by prometheus streaming metrics for route

* fix: fix otel test

* fix: fix linting errors

* test: update test

* fix: fix linting error
2025-06-14 16:26:48 -07:00
Ishaan Jaff
7fcbb38d91
[Fix] Responses API - Session management (#11254)
* fix: import session handling

* fix: imports for session handler

* tests: tests for session handler

* Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-29 20:36:02 -07:00
Ishaan Jaff
0d4c85c419 Revert "fix: import session handling"
This reverts commit deb257dc10.
2025-05-29 17:54:24 -07:00
Ishaan Jaff
deb257dc10 fix: import session handling 2025-05-29 17:48:34 -07:00
Krish Dholakia
a44bc9f613
Remove + Check for unsafe enterprise/ folder imports (#11107)
* refactor(litellm/): wrap enterprise package usage in safe try-excepts

prevents missing package error for OSS users

Fixes https://github.com/BerriAI/litellm/issues/10795

* ci(config.yml): add script to check for unsafe imports to ci/cd

Closes https://github.com/BerriAI/litellm/issues/10795

* fix: add missing docs

* test: test_openai_image_edit_litellm_sdk

* use n 4 for mapped tests (#11109)

* Fix/background health check (#10887)

* fix: improve health check logic by deep copying model list on each iteration

* test: add async test for background health check reflecting model list changes

* fix: validate health check interval before executing background health check

* fix: specify type for health check results dictionary

* fix(user_api_key_auth.py): handle user custom auth set with no custom settings

* bump: version 0.1.21 → 0.2.0

* ci(config.yml): run enterprise and litellm tests separately

* fix: fix linting error

* docs: add missing docs

* [Feat] Add content policy violation error mapping for image editd (#11113)

* feat: add image edit mapping for content policy violations

* test fix

* Expose `/list` and `/info` endpoints for Audit Log events (#11102)

* feat(audit_logging_endpoints.py): expose list endpoint to show all audit logs

make it easier for user to retrieve individual endpoints

* feat(enterprise/): add audit logging endpoint

* feat(audit_logging_endpoints.py): expose new GET `/audit/{id}` endpoint

make it easier to retrieve view individual audit logs

* feat(key_management_event_hooks.py): correctly show the key of the user who initiated the change

* fix(key_management_event_hooks.py): add key rotations as an audit log event

'

* test(test_audit_logging_endpoints.py): add simple unit testing for audit log endpoint

* fix: testing fixes

* fix: fix ruff check

* [Feat] Use aiohttp transport by default - 97% lower median latency  (#11097)

* fix: add flag for disabling use_aiohttp_transport

* feat: add _create_async_transport

* feat: fixes for transport

* add httpx-aiohttp

* feat: fixes for transport

* refactor: fixes for transport

* build: fix deps

* fixes: test fixes

* fix: ensure aiohttp does not auto set content type

* test: test fixes

* feat: add LiteLLMAiohttpTransport

* fix: fixes for responses API handling

* test: fixes for responses API handling

* test: fixes for responses API handling

* feat: fixes for transport

* fix: base embedding handler

* test: test_async_http_handler_force_ipv4

* test: fix failing deepeval test

* fix: add YARL for bedrock urls

* fix: issues with transport

* fix: comment out linting issues

* test fix

* test: XAI is unstable

* test: fixes for using respx

* test: XAI fixes

* test: XAI fixes

* test: infinity testing fixes

* docs(config_settings.md): document param

* test: test_openai_image_edit_litellm_sdk

* test: remove deprecated test

* bump respx==0.22.0

* test: test_xai_message_name_filtering

* test: fix anthropic test after bumping httpx

* use n 4 for mapped tests (#11109)

* fix: use 1 session per event loop

* test: test_client_session_helper

* fix: linting error

* fix: resolving GET requests on httpx 0.28.1

* test fixes proxy unit tests

* fix: add ssl verify settings

* fix: proxy unit tests

* fix: refactor

* tests: basic unit tests for aiohttp transports

* tests: fixes xai

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>

* test: cleanup redundant test

* fix: fix import

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: JuHyun Bae <jhyun0408@nate.com>
2025-05-24 08:59:02 -07:00
jmorenoc-o
1cd2595006
Fixes the InvitationLink Prisma find_many query (#11031)
Related: 3b6c6d05dd (r157675103)

We should use "order", according to the prisma python docs https://prisma-client-py.readthedocs.io/en/stable/reference/limitations/#order-argument 

Also we are using "order" in other files of the project: 
https://github.com/search?q=repo%3ABerriAI%2Flitellm%20order%3D%7B&type=code
2025-05-21 21:42:57 -07:00
Ishaan Jaff
3b6c6d05dd
[Fix] Invitation Email does not include the invitation link (#10958)
* fix: email invites should link to the invitation

* fix: email invites should link to the invitation

* fix: email invites should link to the invitation
2025-05-19 15:38:38 -07:00
Ishaan Jaff
42e6e664b2
[Refactor] Make Pagerduty a free feature (#10857)
* refactor: make pagerduty free

* refactor: make pagerduty free

* fix: pagerduty loc

* fix: linting error
2025-05-15 10:12:06 -07:00
Ishaan Jaff
3130c4f8f9
[Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea (#10782)
* refactor: move guardrails to pip

* refactor: move guardrails to pip

* testing fix: move guardrails to pip

* git commit setup_litellm_enterprise_pip
2025-05-13 09:42:22 -07:00
Ishaan Jaff
a4fb1da2d9
fix: pass application/json for GenericAPILogger (#10772)
* fix: pass application/json for GenericAPILogger

* fix: test_generic_api_callback
2025-05-12 14:15:33 -07:00
Ishaan Jaff
07cb100fdc
Litellm emails smtp fixes (#10730)
* feat: smtp email logger

* feat: smtp email logger

* fix: linting errors
2025-05-10 17:23:59 -07:00
Ishaan Jaff
3731ee436a
[Refactor] Use pip package for enterprise/ folder (#10709)
* init enterprise pip

* init enterprise pip

* init enterprise pip

* test: enterprise pip

* add litellm-enterprise to pip

* litellm ent check

* litellm ent check

* fix import email router

* fix setup_litellm_enterprise_pip

* fix local testing with enterprise pip
2025-05-09 17:18:48 -07:00