Ishaan Jaff
93d6e9bf49
pip install "google-genai==1.22.0"
2025-09-12 19:30:47 -07:00
Ishaan Jaff
555efd60b5
[Security] Fix: Ensure .info() logs are not used for request/responses + Add code QA check for possible violations ( #14386 )
...
* test: info log
* fixes for debug
* add info_log_check
* fix: debug level
* fix: debug level
* fix: use .debug
* fix info statements
* fix info log check
* fix: fix emit_standard_logging_payload
* fix RequestBody
* Potential fix for code scanning alert no. 3339: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix check
* fix code qa check
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-09-09 13:55:56 -07:00
Ishaan Jaff
a0b380a296
fix install
2025-09-08 18:30:01 -07:00
Ishaan Jaff
e71e06510c
fix test
2025-09-08 17:38:44 -07:00
Ishaan Jaff
ac2d349161
[Security] Ensure LiteLLM Images have 0 Critical, High, Medium vulnerabilities with CVSS ≥ 4.0 ( #14357 )
...
* updated scans
* fix scans
* fix litellm_security_tests
* fix
* fix count CVEE
* add readme
* fix Security Scans
* fix: starlette==0.47.2 # starlette fastapi dep
* bump fastapi==0.115.14
* bump 0.116.1
2025-09-08 16:49:52 -07:00
Ishaan Jaff
3915a4455a
fix pip install
2025-09-06 14:29:08 -07:00
Ishaan Jaff
5310bba35b
[Feat] Litellm x CloudZero Integration - Cost Tracking ( #14296 )
...
* fix: just pull LiteLLM_DailyUserSpend
* get the team_id from user daily spend table
* cloudzero_dry_run_export
* fix CZ endpoints
* trace entity_id
* fix: get_usage_data
* fix get_usage_data
* fix _create_cbf_record
* fix get_usage_data
* ensure start and end time is used for exporting data
* fix init_cloudzero_background_job
* fix CloudZeroExportRequest
* fix initialize_cloudzero_export_job
* fix initialize_cloudzero_export_job
* allow init with env + config.yaml for cloudzero
* fix: init CZ through config.yaml
* fix DRY run on CZ
* TestCloudZeroDryRunEndpoint
* fix: CLOUDZERO_EXPORT_INTERVAL_MINUTES
* fix init_cloudzero_background_job
* fix exporting data
* fix transform
* stash cloudzero docs
* docs: CloudZero
* ruff fix
* fix rendering key alias
* fix polars
2025-09-05 21:29:41 -07:00
Ishaan Jaff
e127820bf1
fix proxy_logging_guardrails_model_info_tests
2025-09-03 16:10:30 -07:00
Ishaan Jaff
47c04a8a8c
proxy_logging_guardrails_model_info_tests
2025-09-02 14:13:17 -07:00
Ishaan Jaff
5ab3d74371
DEFAULT_NUM_WORKERS_LITELLM_PROXY
2025-09-01 20:34:23 -07:00
Ishaan Jaff
c839b2c751
installing_litellm_on_python
2025-08-26 19:51:02 -07:00
Ishaan Jaff
80d4cc1283
[Perf] Use fastuuid for fast UUID generations - 2.1x Faster ( #13992 )
...
* use fastuuid
* add fastuuid
* add fastuuid==0.12.0
2025-08-26 19:47:25 -07:00
Ishaan Jaff
b9132968b2
[Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS ( #13905 )
...
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895 )
* fix: litellm.configured_cold_storage_logger
* fix Session Management - Non-OpenAI Models docs
* ruff fix
* test fix
* create LoggingWorker
* add GLOBAL_LOGGING_WORKER for async task handling
* fix logging tests
* add conftest
* fix conftest
* test fix location of encode bedrock runtime modelid arn
* fix conftest.py
* tuning LoggingWorker
* conftest.py
* fix conftest batches/
* test_async_chat_azure
* event_loop
* test_bedrock_streaming_passthrough_test2
* fix GLOBAL_LOGGING_WORKER
* logging worker
* add flush for global logging worker
* Revert "fix GLOBAL_LOGGING_WORKER"
This reverts commit d254f508f4
2025-08-23 13:13:23 -07:00
Ishaan Jaff
76f1064229
[Bug Fix] litellm incompatible with newest release of openAI v1.100.0 ( #13728 )
...
* fix imports OpenAI SDK
* ResponseText fixes
* fixes ResponseText
* fix imports
* catch AttributeError
* fix import
* use openai==1.100.1
* fix build from PIP
* fix lint test
* Print OpenAI version
* fix Install dependencies
2025-08-18 18:26:17 -07:00
0x-fang
9f17bedb48
bump up boto3 and aioboto3 versions ( #13665 )
2025-08-15 17:06:08 -07:00
Ishaan Jaff
f5a1c14929
Revert "update boto3 version to 1.37.38 ( #13656 )"
...
This reverts commit 6f2264a13d
2025-08-15 11:24:59 -07:00
Ishaan Jaff
c331473b75
Revert "fix aioboto3"
...
This reverts commit 82d1cb48b7
2025-08-15 11:24:50 -07:00
Ishaan Jaff
82d1cb48b7
fix aioboto3
2025-08-15 11:19:14 -07:00
0x-fang
6f2264a13d
update boto3 version to 1.37.38 ( #13656 )
2025-08-15 11:06:54 -07:00
Ishaan Jaff
eb4bd26f24
[Bug Fix] - Get Routes ( #13466 )
...
* fixes get_routes_for_mounted_app
* fix - use _safe_get_endpoint_name
* fix code QA check
* test_get_routes_for_mounted_app_with_static_files
* test fixes
2025-08-09 12:52:23 -07:00
Ishaan Jaff
3905cee579
test fixes
2025-08-08 18:50:09 -07:00
Jugal D. Bhatt
29a8c583c2
added redis iam auth ( #13275 )
2025-08-05 10:56:34 -07:00
Krrish Dholakia
fedd805b6a
test: loosen check
2025-08-01 09:21:13 -07:00
Krrish Dholakia
4a66617d71
build(config.yml): use ci/cd postgres in test
2025-08-01 08:48:07 -07:00
Krish Dholakia
c7e4435bdc
Fix - using managed files w/ OTEL + UI - add model group alias on UI ( #13171 )
...
* fix(router.py): safe deep copy kwargs
OTEL adds a parent_otel_span which cannot be deepcopied
* fix: use safe deep copy in other places as well
* test: add script to check and ban copy.deepcopy of kwargs
enforce safe_deep_copy usage
* build(ui/): new component for adding model group alias on UI
* fix(proxy_server.py): support updating model_group_alias via /config/update
allows ui component to work
* fix(router.py): update model_group_alias in router settings based on db value
* fix: fix code qa error
2025-07-31 21:22:04 -07:00
Ishaan Jaff
ee70d593c1
[Feat] Allow redacting message / response content for specific logging integrations - DD LLM Observability ( #13158 )
...
* fix redact_standard_logging_payload
* add StandardCustomLoggerInitParams
* allow defining DatadogLLMObsInitParams
* fix init DataDogLLMObsLogger
* fix import
* update redact_standard_logging_payload_from_model_call_details
* test_dd_llms_obs_redaction
* docs DD logging
* docs DD
* docs DD
* Redacting Messages, Response docs DD LLM Obs
* fix redaction logic
* fix create_llm_obs_payload
* fix logging response
* fixes
* ruff fix
* fix test
* test_dd_llms_obs_redaction
* test_create_llm_obs_payload
* redact_standard_logging_payload_from_model_call_details
* img - dd_llm_obs
* docs DD
* fix linting
* fix linting
* fix mypy
* test_create_llm_obs_payload
* test_create_llm_obs_payload
* fix mock_env_vars
* fix _handle_anthropic_messages_response_logging
2025-07-31 16:44:16 -07:00
Ishaan Jaff
cbb922b1bb
[Bug Fix] Gemini-CLI Integration - ensure tool calling works as expected on generateContent ( #13189 )
...
* transform_generate_content_request
* add tools in GenerateContentRequestDict
* add generate_content_handler tool calling
* google_generate_content_endpoint_testing
* test_mock_stream_generate_content_with_tools
* test_validate_post_request_parameters
* fixes for generate_content_handler
* fix VertexAIGoogleGenAIConfig
* fixes veretx ai
* google_generate_content_endpoint_testing
* test_async_streaming_with_logging
* load_vertex_ai_credentials
* test_vertex_anthropic.py
2025-07-31 16:42:57 -07:00
Krish Dholakia
c6a8733234
build(config.yml): migrate build_and_test to ci/cd pg db ( #13166 )
2025-07-30 18:19:36 -07:00
Krrish Dholakia
e3ba350345
ci: migrate to postgres in ci/cd
2025-07-30 17:00:06 -07:00
Krrish Dholakia
2266938184
ci(config.yml): remove check
2025-07-30 16:33:42 -07:00
Krrish Dholakia
88aed06bbb
ci: migrate to db in pipeline
2025-07-30 15:48:15 -07:00
Ishaan Jaff
5aa4a70bd3
fix grype scan
2025-07-30 15:35:16 -07:00
Krrish Dholakia
3bf7411491
ci(config.yml): run prisma generate before running enterprise tests
2025-07-30 15:28:45 -07:00
Krrish Dholakia
6da36049fe
ci: remove bad script
2025-07-30 15:25:32 -07:00
Krish Dholakia
3ca99f1def
Litellm explore postgres db ci cd ( #13156 )
...
* ci(config.yml): testing with ci/cd db
* build: spin up pg db in ci/cd test
2025-07-30 15:17:12 -07:00
Ishaan Jaff
8f86355361
fix ci/cd testing
2025-07-25 09:05:27 -07:00
Ishaan Jaff
29cff4f4e6
pip install semantic_router --no-deps
2025-07-24 20:09:37 -07:00
Ishaan Jaff
b8e404dd95
[Feat] Backend Router - Add Auto-Router powered by semantic-router ( #12955 )
...
* add router.json
* test_router_auto_router
* async_pre_routing_hook
* fixes for auto router
* add async_pre_routing_hook
* add LiteLLMRouterEncoder
* update test auto_router_embedding_model
* add auto_router_embedding_model
* add AutoRouter
* fix async_pre_routing_hook
* update async_pre_routing_hook
* fix auto router
* fix router.json
* working router init
* working embedding encoder
* working auto router
* test_router_auto_router
* test auto router
* add semantic-router as optional for litellm
* add extras
* semantic_router==0.1.10
* ruff fix
* use aiohttp==3.10.11
* python-dotenv==1.0.1
* test auto router
* test_router_auto_router
* semantic_router
* test_is_auto_router_deployment
* fix check
* fix docker build step
* add semantic_router
* Revert "add semantic_router"
This reverts commit 537b672887
2025-07-24 18:32:56 -07:00
Ishaan Jaff
d93e85282e
test_bad_database_url
2025-07-22 18:37:51 -07:00
Krrish Dholakia
aeb270b759
build: move build_and_test to use prisma migrate
2025-07-17 12:16:06 -07:00
Ishaan Jaff
57b0b4edf3
[Bug fix] [Bug]: Verbose log is enabled by default ( #12596 )
...
* test find_set_verbose_assignments
* fix set verbose
* test set verbose
* fix unused import
2025-07-14 20:06:16 -07:00
Ishaan Jaff
57d9ca5f3a
ci/cd remove flaky load testing scripts. We will do load testing on post releases
2025-07-12 11:17:54 -07:00
Ishaan Jaff
a0a57687ca
fix security - mcp ( #12362 )
2025-07-07 15:53:54 -07:00
Ishaan Jaff
08f3b06d82
[Feat] Bump langfuse python SDK version and LANGFUSE_TRACING_ENVIRONMENT ( #12376 )
...
* bump langfuse to 2.59.7
* _set_langfuse_specific_attributes
* fix files
* test_set_langfuse_environment_attribute
2025-07-07 15:53:43 -07:00
Ishaan Jaff
f285765862
add code_qa_check_tests
2025-07-04 10:06:40 -07:00
Ishaan Jaff
39955129f5
fix mapped tests ( #12320 )
...
* fix - use flush llm client cache
* faster mapped tests
* test_async_multiple_response_ids_routing
* fix tests
* test_ateam_member_update_admin_requires_premium
* regular mapped tests
* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206 )"
This reverts commit 2c60c316ec
2025-07-04 10:04:43 -07:00
Ishaan Jaff
0c19414b36
[ ⚡ ️ Python SDK import] - reduce python sdk import time by .3s ( #12140 )
...
* use 1 file for KeyManagementSystem
* move key management settings
* fix import locs
* test_proxy_types_not_imported
* test the import loc
* fix import item
* fix imports
* fix import loc
* fix imports
2025-06-28 14:57:10 -07:00
Ishaan Jaff
7f11b5b76c
[Security] - Add Trivy Security Scan for UI + Docs folder - remove all vulnerabilities ( #11778 )
...
* docs - sec scan
* ui - fix 0 security issues
* fix - 0 medium vuln on docs
* add MEDIUM scan for Docs
* Run Trivy scan on LiteLLM UI
2025-06-16 17:13:19 -07:00
Ishaan Jaff
327868fff6
speed up llm_translation_testing
2025-06-14 16:56:02 -07:00
Ishaan Jaff
d477c1915d
fix - speed up litellm_proxy_unit_testing
2025-06-14 16:30:28 -07:00
