scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 79
26500 commits 4106 branches 1402 tags 4.2 GiB
Commit graph

251 commits

Author SHA1 Message Date
Copilot
f5359ba007
Fix apply_guardrail endpoint returning raw string instead of ApplyGuardrailResponse (#15436) 
* Initial plan

* Fix apply_guardrail endpoint to return ApplyGuardrailResponse

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-11 17:50:37 -07:00
Krrish Dholakia
b4c4776f56 fix: fix linting errors 2025-10-04 14:47:13 -07:00
Krrish Dholakia
a1a0e99638 fix(prometheus.py): working e2e calls w/ userapikeymetadata 2025-09-30 21:23:25 -07:00
Krrish Dholakia
d6800ee706 feat(prometheus.py): initial working commit of passing team/key metadata as prometheus metrics 
Closes LIT-1006
 2025-09-30 17:02:40 -07:00
Krrish Dholakia
6ca7752381 fix(prometheus.py): don't require metadata labels to be set for all requests 
add a default value if metadata label not set
 2025-09-30 15:46:30 -07:00
Krrish Dholakia
c2e2b699e6 fix: fix linting error 2025-09-27 12:43:06 -07:00
Alexsander Hamir
eaa04cd8ce
fix: use fastuuid helper (#14903) 
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
 2025-09-25 15:47:01 -07:00
Ishaan Jaff
b9ffa98c55
[Feat] Proxy CLI: Create a python method to login using litellm proxy (#14782) 
* fix: cli auth with SSO okta

* fix: add LITTELM_CLI_SERVICE_ACCOUNT_NAME

* fix: get_litellm_cli_user_api_key_auth

* use existing_key CLI

* fix: use existing key

* test auth commands

* test_cli_sso_callback_regenerate_vs_create_flow

* feat: add CLI Token Utilities

* fix: get_stored_api_key

* move file

* fix: get_valid_models

* fix config.yaml

* TestCLITokenUtils

* TestGetValidModelsWithCLI

* fix: tie user id to keys created through CLI

* fix: add teams interface to CLI

* add /keys/update to the list client commands

* fix /sso/cli/poll to return the user_id

* fix: working TeamsManagementClient

* fix CLI Login command

* fixes for auth

* Potential fix for code scanning alert no. 3400: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* ruff fix

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-09-22 21:28:38 -07:00
Ishaan Jaffer
c656982f18 test fix 2025-09-20 16:55:23 -07:00
Krish Dholakia
270d612029
Merge branch 'main' into litellm_dev_09_10_2025_p1 2025-09-19 22:01:57 -07:00
Krrish Dholakia
565eeca92a build(pyproject.toml): bump versions 2025-09-18 19:24:34 -07:00
Krrish Dholakia
4d87199266 fix(prometheus.py): fix spend metrics 2025-09-18 19:12:07 -07:00
Mubashir Osmani
8b804303ed
fix: ci/cd tests + lint errors (#14646) 
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-09-17 17:06:43 -07:00
Krrish Dholakia
fc2d1f2646 fix: fix import errors 2025-09-14 09:32:21 -07:00
Krrish Dholakia
459e66a9cd fix: fix test 2025-09-13 13:38:35 -07:00
Krrish Dholakia
a9fddbf4ad fix(prometheus.py): make prometheus work for multiple workers 2025-09-10 16:20:17 -07:00
Ishaan Jaff
555efd60b5
[Security] Fix: Ensure .info() logs are not used for request/responses + Add code QA check for possible violations (#14386) 
* test: info log

* fixes for debug

* add info_log_check

* fix: debug level

* fix: debug level

* fix: use .debug

* fix info statements

* fix info log check

* fix: fix emit_standard_logging_payload

* fix RequestBody

* Potential fix for code scanning alert no. 3339: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix check

* fix code qa check

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-09-09 13:55:56 -07:00
mubashir1osmani
2626123aca added litellm_spend_metric 2025-09-01 11:33:34 -04:00
mubashir1osmani
5649c0f0b9 Revert "feat: add persistence and management for guardrails on virtual keys" 
This reverts commit 6cd5afa8b1.
 2025-09-01 10:45:46 -04:00
mubashir1osmani
a6790ac158 Revert "feat: add UI support for displaying and managing guardrails on virtual keys" 
This reverts commit 83654c5608.
 2025-09-01 10:45:44 -04:00
mubashir1osmani
31cfe1fe0a Revert "fix: import verbose_proxy_logger to resolve undefined name errors" 
This reverts commit a8f2ebed9c.
 2025-09-01 10:45:38 -04:00
mubashir1osmani
8de65e8448 fix: added prometheus missing metrics 2025-09-01 10:36:51 -04:00
mubashir1osmani
af2f447698 added prometheus metrics 2025-09-01 10:28:48 -04:00
mubashir1osmani (aider)
a8f2ebed9c fix: import verbose_proxy_logger to resolve undefined name errors 2025-09-01 01:17:46 -04:00
mubashir1osmani (aider)
83654c5608 feat: add UI support for displaying and managing guardrails on virtual keys 2025-09-01 01:17:08 -04:00
mubashir1osmani (aider)
6cd5afa8b1 feat: add persistence and management for guardrails on virtual keys 2025-09-01 00:56:22 -04:00
Krrish Dholakia
dba78046b9 fix(internal_user_endpoints.py): show remaining users when free SSO users is enabled 2025-08-20 22:26:01 -07:00
Ishaan Jaff
4d941c914e
[Feat] Responses API Session Handling - Multi media support (#13347) 
* rename ResponsesSessionHandler

* use ResponsesSessionHandler

* test session handler

* refactor ResponsesSessionHandler

* fix get_proxy_server_request_from_spend_log

* use constant for LITELLM_TRUNCATED_PAYLOAD_FIELD

* add _should_check_cold_storage_for_full_payload

* add get_class_type_for_custom_logger_name

* get_active_custom_logger_for_callback_name

* add get_proxy_server_request_from_cold_storage to CustomLogger

* add ColdStorageHandler

* start using cold storage integration

* add get_proxy_server_request_from_cold_storage

* fixes from manual testing

* s3 v2 fix getting region name

* ChatCompletionImageUrlObject

* use _get_configured_cold_storage_custom_logger

* fixes for _should_check_cold_storage_for_full_payload

* fix _download_object_from_s3

* test_s3_v2_with_cold_storage

* add cold_storage_object_key to StandardLoggingMetadata

* use get_proxy_server_request_from_cold_storage_with_object_key

* add cold_storage_object_key to SpendLogsMetadata

* add cold_storage_object_key

* get_proxy_server_request_from_cold_storage_with_object_key

* use get_proxy_server_request_from_cold_storage_with_object_key

* test responses API

* add get_proxy_server_request_from_cold_storage_with_object_key

* session handler fixes

* test session handler

* fix ruff checks

* _download_object_from_s3

* cleanup

* test

* lint fix

* test_e2e_cold_storage_successful_retrieval

* test_e2e_generate_cold_storage_object_key_successful

* test_async_gcs_pub_sub_v1

* test fix

* test fix

* test fix

* test_standard_logging_metadata_has_cold_storage_object_key_field

* test_sanitize_request_body_for_spend_logs_payload_basic

* test_transform_input_image_item_to_image_item_with_image_data
 2025-08-07 10:59:53 -07:00
Ishaan Jaff
da40b5ec47 bump litellm enterprise 2025-08-05 17:50:40 -07:00
Ishaan Jaff
0308a835b8 bump: version 0.1.17 → 0.1.18 2025-08-05 17:47:16 -07:00
Ishaan Jaff
dab8ba03e3
[Feat] - When using custom tags on prometheus allow using wildcard patterns (#13316) 
* _tag_matches_wildcard_configured_pattern

* test_get_custom_labels_from_tags_wildcard_patterns

* docs Custom Tags

* docs how custom tags work

* fix
 2025-08-05 17:46:13 -07:00
Krish Dholakia
eb49f987de
Ensure disable_llm_api_endpoints works + Add wildcard model support for 'team-byok' model (#13278) 
* fix(route_checks.py): ensure disable llm api endpoints is correctly set

* fix(route_checks.py): raise httpexception

raise expected exceptions

* fix(router.py): handle team only wildcard models

fixes issue where team only wildcard models were not considered during auth checks

* fix(router.py): handle team only wildcard models

fixes issue where team only wildcard models were not considered during auth checks
 2025-08-04 23:19:51 -07:00
Ishaan Jaff
ad26cba8b6 add litellm-enterprise==0.1.17 2025-08-04 17:26:50 -07:00
Ishaan Jaff
ba1882fdd5
[Bug Fix] Prometheus - fix for litellm_input_tokens_metric, litellm_output_tokens_metric - Note this updates the metric name (#13271) 
* fixes for litellm_tokens_metric

* test_prometheus_token_metrics_with_prometheus_config
 2025-08-04 17:22:21 -07:00
Jugal D. Bhatt
900c7f45c0
[MCP Gateway] Litellm mcp pre and during guardrails (#13188) 
* add guardrail support

* add guardrail support

* guardrails for MCP

* added changes

* add mcp guardrails

* added test

* add ui

* fix guardrail form

* working with cursor

* remvoe print

* fix mcp servertests

* fix mypy and remove console logs

* fix mypy and remove console logs

* fix mypy tests
 2025-08-01 20:02:25 -07:00
Krish Dholakia
b7c5073d28
Custom Auth - bubble up custom exceptions (#13093) 
* fix(enterprise/litellm_enterprise/proxy/auth/user_api_key_auth.py): bubble up exception if type is ProxyException

* docs(custom_auth.md): doc on bubbling up custom exceptions
 2025-07-29 16:28:25 -07:00
Krish Dholakia
d6423d913d
build: update pip package (#12998) 2025-07-25 16:23:53 -07:00
Krish Dholakia
550211ba47
Prometheus - tags, fix '[tag]="false"' when tag is set (#12916) 
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )

* fix(prometheus.py): working tag based metrics

* fix(prometheus.py): emit request tags on post call success hook

* fix(prometheus.py): add user agent tags on request failure

* fix(prometheus.py): add request tags to deployment failure metric

s
 2025-07-24 16:39:13 -07:00
Ishaan Jaff
bf300f8ca7 Revert "Litellm dev 07 21 2025 p1 (#12848)" 
This reverts commit e4e10aa4ed.
 2025-07-22 18:28:36 -07:00
Krish Dholakia
e4e10aa4ed
Litellm dev 07 21 2025 p1 (#12848) 
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang

* test(tests/): update tests
 2025-07-21 22:09:39 -07:00
Krish Dholakia
00677afefe
Litellm batch cost tracking debug (#12782) 
* feat(proxy_server.py): support batch polling interval

allows admin to control batch polling interval (default is 3600s)

easier debugging

* fix(proxy_settings_endpoint.py): ensure value is actually set before updating env var
 2025-07-21 20:17:56 -07:00
Ishaan Jaff
96f7eb6f78 bump litellm enterprise version 2025-07-19 10:12:33 -07:00
Ishaan Jaff
06574a72b5
[Feat] Backend - Add support for disabling callbacks in request body (#12762) 
* allow using standard_callback_dynamic_params to disable callbacks

* fix is_callback_disabled_dynamically

* test_callback_disabled_via_request_body_multiple
 2025-07-19 10:10:30 -07:00
Ishaan Jaff
99e2ea081d
[Feat] UI - Allow clicking into Vector Stores (#12741) 
* Add View Vector Store

* add /info for vector store

* fix updated_at
 2025-07-18 14:28:57 -07:00
Jugal D. Bhatt
a46b9d376f
[Prometheus] Move Prometheus to enterprise folder (#12659) 
* fix tools fetch for keys

* add promethues to enterprise

* remove old prom

* remove old prom

* fix tests

* safe imports

* add if

* fix enterprise test

* rename imports

* added label import

* added label import

* move tests to enterprise

* fix tests

* add log

* build: update versions

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
 2025-07-18 11:54:47 -07:00
Ishaan Jaff
3051a9c68a
[Bug Fix] QA - Use PG Vector Vector Store with LiteLLM (#12716) 
* store in generic litellm params

* handle storing litellm_params_json

* add litellm_params

* update file loc

* fix startup issue

* ui fix litellm_params

* fix typing

* fix adding PG Vector

* use litellm params

* fix transform_search_vector_store_request

* fix transform

* fix URL

* test fix

* test_pg_vector_search_request_construction
 2025-07-18 08:41:18 -07:00
Ishaan Jaff
95f7290e44
[Feat] Proxy - New LLM API Routes /v1/vector_stores and /v1/vector_stores/vs_abc123/search (#12699) 
* add Search + Create vector stores

* add new vector_stores routes

* add avector_store_search routes

* docs vector stores

* docs vector stores

* Create Vector Store

* fix - refactor

* add vector_store_router

* fix imports

* fix ruff
 2025-07-17 11:19:25 -07:00
yeahyung
782969bb05
(#11794) use upsert for managed object table rather than create to avoid UniqueViolationError (#11795) 
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError

* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
 2025-07-15 20:20:01 -07:00
Krish Dholakia
7c392475e6
Control Plane + Data Plane support (#12601) 
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance

useful for preventing multiple instances from doing admin actions

* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances

provide guidance on scaling proxy

* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm

* fix(route_checks.py): allow disabling llm api endpoints on an instance

allows pure admin instance to exist

* refactor(enterprise/route_checks.py): refactor env var checks

* refactor: finish refactoring

* docs(control_plane_and_data_plane.md): refactor docs

* test: update tests
 2025-07-14 21:31:56 -07:00
Ishaan Jaff
57b0b4edf3
[Bug fix] [Bug]: Verbose log is enabled by default (#12596) 
* test find_set_verbose_assignments

* fix set verbose

* test set verbose

* fix unused import
 2025-07-14 20:06:16 -07:00