Commit graph

22391 commits

Author SHA1 Message Date
Cole McIntosh
08239357cf Add ExceptionCheckers class for improved error string detection
Introduce the ExceptionCheckers class to encapsulate methods for checking error conditions in exception strings, specifically for identifying rate limit errors. Update the Fireworks AI exception mapping tests to cover various scenarios, including standard 429 errors and text-based detection, ensuring accurate mapping to RateLimitError. Enhance test coverage for both positive and negative cases of rate limit detection.
2025-06-05 17:15:53 -06:00
Cole McIntosh
fda99ecb41 Enhance exception mapping for Fireworks AI: add better handling for 429 status codes and text-based rate limit detection. Update tests to verify correct mapping to RateLimitError for both 429 and related error messages. 2025-06-05 15:47:25 -06:00
Cole McIntosh
7c513856dc
Fix None values in usage field for gpt-image-1 model responses (#11448)
* fix(convert_dict_to_response.py): handle None values in usage field for gpt-image-1

* test: add tests for handling None and partial values in usage fields for gpt-image-1 responses
2025-06-05 13:19:18 -07:00
Krish Dholakia
69c9d75f20
fix(prometheus.py): pass custom metadata labels in litellm_total_toke… (#11414)
* fix(prometheus.py): pass custom metadata labels in litellm_total_tokens metric

* fix(handler.py): handle /v1 for openai realtime translation

Closes https://github.com/BerriAI/litellm/pull/11398

* fix(prometheus.py): fix incrementing total tokens metric
2025-06-05 00:15:23 -07:00
Low Jian Sheng
a3e5bc4856
Support no reasoning option for gemini models (#11393)
* support no reasoning for gemini models

* change none to disable

* remove print statements

* update docs
2025-06-05 00:11:45 -07:00
Krrish Dholakia
505d2fe0c7 build: bump 2025-06-05 00:08:53 -07:00
Krish Dholakia
db23016536
fix(redis_cache.py): support pipeline redis lpop for older redis vers… (#11425)
* fix(redis_cache.py): support pipeline redis lpop for older redis versions

Fixes https://github.com/BerriAI/litellm/issues/10379

* test: add mock host
2025-06-05 00:05:54 -07:00
Sha
a301ef873e
added gemini url context support (#11351)
* added gemini url context support

* lint issue fix
2025-06-04 23:56:21 -07:00
Tom Bocklisch
d7982bb0af
Use proper attribute for sagemaker request (#11362) 2025-06-04 22:47:43 -07:00
Jimmy Tsai
4019f79808
feat: add deepseek-r1 family model configuration to pricing JSON (#11394) 2025-06-04 22:39:06 -07:00
Ishaan Jaff
02a34d319a
bump to ddtrace==3.8.0 (#11426) 2025-06-04 22:18:07 -07:00
Ishaan Jaff
f0e0007eaf fix: gemini-2.0-flash-preview-image-generation test 2025-06-04 21:21:28 -07:00
Cole McIntosh
049e65a84e
Merge pull request #11417 from colesmcintosh/sso-config-ui 2025-06-04 20:51:59 -06:00
Ishaan Jaff
de306cfcb3
[Performance] Performance improvements for /v1/messages route (#11421)
* fix: perf anthropic /v1/messages

* fix: perf anthropic /v1/messages

* fix: linting checks

* fix: linting checks
2025-06-04 18:47:53 -07:00
raz-alon
fada9c79be
Add User ID validation to ensure it is not an email or phone number (#10102) 2025-06-04 18:38:02 -07:00
Cole McIntosh
c1a324c2fb Merge remote-tracking branch 'origin/main' into sso-config-ui 2025-06-04 18:42:43 -06:00
Cole McIntosh
65b28826a6 Add uiAuditLogsCall function 2025-06-04 18:19:53 -06:00
Krish Dholakia
9da32d9e14
Litellm audit log staging (#11418)
* Audit logs added (#11226)

* audit logs added

* audit logs populated

* adding json response

* collapsible json columns

* add created at column

* added changed field

* added premiumUser description

* added paginated filtered logs

* convert table names

* remove test file

* added new ui for audit logs

* only show the difference in before value and updated value

* fix: add lucide-react to package json

---------

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
2025-06-04 14:34:17 -07:00
Cole McIntosh
3a946933ee Refactor settings response models in proxy_setting_endpoints.py
- Renamed SSOSettingsResponse to inherit from a new base class SettingsResponse for better structure.
- Introduced InternalUserSettingsResponse and DefaultTeamSettingsResponse models for internal user and default team settings.
- Updated endpoint responses to use field_schema instead of schema for consistency.
- Enhanced test cases to validate the new response structure and ensure proper functionality of SSO settings.
2025-06-04 15:08:05 -06:00
Cole McIntosh
3cc9460922 Add SSO settings response model in proxy_setting_endpoints.py
- Introduced SSOSettingsResponse model to encapsulate SSO configuration values and schema information.
- Updated the get_sso_settings endpoint to utilize the new response model, enhancing API clarity and usability.
2025-06-04 14:55:29 -06:00
Cole McIntosh
c41b14e27b Add clear SSO settings functionality in SSOModals component
- Introduced a confirmation modal for clearing SSO settings.
- Implemented handleClearSSO function to reset SSO settings and provide user feedback.
- Updated UI to include a 'Clear' button for SSO settings, enhancing user experience.
- Added state management for the confirmation modal visibility.
2025-06-04 14:39:26 -06:00
Cole McIntosh
72c7fd63bf Implement SSO configuration check in AdminPanel and update SSOModals to reflect SSO status
- Added logic to check SSO configuration and set state in AdminPanel.
- Introduced a new function to handle SSO configuration checks.
- Updated UI to conditionally render SSO button text based on configuration status.
- Passed SSO configuration status as a prop to SSOModals for better integration.
2025-06-04 14:34:18 -06:00
Cole McIntosh
58b1f78ff0 Remove console log from handleShowInstructions in AdminPanel component 2025-06-04 14:25:57 -06:00
Cole McIntosh
9c481e3ba8 Add SSO configuration endpoints and UI integration
- Introduced new SSOConfig class to manage SSO settings.
- Added endpoints for fetching and updating SSO settings in proxy_setting_endpoints.py.
- Created a new __init__.py file to expose the SSO router.
- Updated AdminPanel and SSOModals components to handle SSO settings retrieval and updates.
- Removed deprecated SSO update logic from AdminPanel.
- Enhanced error handling and logging for SSO operations.
2025-06-04 14:09:19 -06:00
Cole McIntosh
7d4a70bfe3 Enhance Admin Panel UI: Adjust button styles and layout for better accessibility and user experience 2025-06-04 13:32:43 -06:00
Lik Wei
a24077fb4e
fix: add enterprise feature gating to RegenerateKeyModal in KeyInfoView (#11400)
Pass premiumUser prop to RegenerateKeyModal to control access
Disable Regenerate Key button and show tooltip for non-premium users
Ensure consistent enterprise feature messaging across components
2025-06-04 12:05:56 -07:00
Krrish Dholakia
26891c23c5 docs: update docs 2025-06-04 11:42:53 -07:00
Walt Wells
f782cc7e4d
update GCSBucketBase to handle GSM project ID if passed (#11409) 2025-06-04 10:02:26 -07:00
Krish Dholakia
e74ff23b45
Litellm dev 06 03 2025 p3 (#11388)
* fix(vertex_ai/common_utils.py): Close https://github.com/BerriAI/litellm/issues/11383

* feat(anthropic/batches): transformation.py

new transformation config for anthropic batches

* feat(anthropic/batches): working token tracking for anthropic batch calls via `/anthropic` passthrough route

* fix(anthropic_passthrough_logging_handler.py): ruff check fixes
2025-06-03 22:35:51 -07:00
Krrish Dholakia
b21efd435a test(test_sso_sign_in.py): update test 2025-06-03 21:46:34 -07:00
Krish Dholakia
3bd128630d
Anthropic - pass file url's as Document content type + Gemini - cache token tracking on streaming calls (#11387)
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter

add test and ensure anthropic file url's are correctly sent as 'document' blocks

* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming

Closes https://github.com/BerriAI/litellm/issues/10667

* test(test_vertex_and_google_ai_studio_gemini.py): update test
2025-06-03 21:36:49 -07:00
Krrish Dholakia
0273b72b68 build: bump poetry 2025-06-03 21:25:53 -07:00
Krish Dholakia
e0fa33f099
UI / SSO - Update proxy admin id role in DB + Handle SSO redirects with custom root path (#11384)
* fix(ui_sso.py): update user as proxy admin in db table, when checking for proxy_admin_id

Fixes issue where existing internal user, unable to make calls when set as proxy admin id

* fix(utils.py): fix custom base path
2025-06-03 21:16:55 -07:00
Ishaan Jaff
a1f3a1c5dc
[Feat] Performance - Don't create 1 task for every hanging request alert (#11385)
* feat: add async_get_oldest_n_keys in memory cache

* fix: add add_request_to_hanging_request_check

* test: alerting

* feat: v2 hanging request check

* fix: HangingRequestData

* fix: AlertingHangingRequestCheck

* fix: check_for_hanging_requests

* fix: use correct metadata location for hanging requests

* fix: formatting alert

* test hanging request check

* fix: add guard flags for background tasks alerting
2025-06-03 21:12:54 -07:00
Krrish Dholakia
affc4524e1 build: cascade budget changes 2025-06-03 21:02:19 -07:00
Krrish Dholakia
2fe0a2750b test: ensure aws region correctly set 2025-06-03 20:58:21 -07:00
Cole McIntosh
7bbd8262ed
Add Claude 4 Sonnet & Opus, DeepSeek R1, and fix Llama Vision model pricing configurations (#11339)
* fix: update model path for llama-v3p2-90b-vision-instruct in pricing configuration (missing fireworks_ai/ prefix)

* feat: add deepseek-r1-0528 model configuration to pricing JSON

* feat: add configurations for new Claude 4 model alias to pricing JSON

* undo prefix change

* fix: update supports_response_schema to false in pricing JSON for litellm_provider

* update supports_tool_choice and supports_response_schema

* Update model configuration to disable function calling and tool choice for multiple models in fireworks_ai. Adjusted supported parameters in FireworksAIConfig to conditionally include tools and tool_choice based on model compatibility.

* Refactor FireworksAIConfig to use supports_function_calling from utils

* Enhance FireworksAIConfig to conditionally support tool_choice based on model capabilities
2025-06-03 20:39:47 -07:00
Ishaan Jaff
c0c04b0e63
[Performance]: Add debugging endpoint to track active /asyncio-tasks (#11382)
* feat: add debug/asyncio-tasks

* Update litellm/proxy/common_utils/debug_utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-03 19:04:30 -07:00
Marty Sullivan
d247a390bd
add gemini-embeddings-001 model prices and context window (#11332)
* add gemini-embeddings-001 model prices and context window

* use scientific notation
2025-06-03 15:59:30 -07:00
hagan
0f449bf038
[Bug Fix] Create/Update team member api 500 errror (#10479)
* Fixes issue with team_endpoints on member budget update

* refactored location of budget membership fix

* added test for _upsert_budget_membership func
2025-06-03 15:57:33 -07:00
Ishaan Jaff
7c1e1790e3
fix: health liveliness check auth (#11378) 2025-06-03 15:16:12 -07:00
Ishaan Jaff
209362664f add Dockerfile.dev 2025-06-03 12:03:52 -07:00
Ishaan Jaff
99c91fe41f
[Feat]: Performance add DD profiler to monitor python profile of LiteLLM CPU% (#11375)
* feat: add DD profile

* fix: test_should_use_dd_profiler

* docs dd profiler

* docs DD profiler
2025-06-03 12:03:08 -07:00
Ishaan Jaff
41a2a62511 fix: bedrock kb test 2025-06-03 11:55:41 -07:00
Cole McIntosh
50d5ee9138
Fix linting error (#11374) 2025-06-03 11:32:39 -07:00
Cole McIntosh
621d609879
feat: add cerebras/qwen-3-32b model pricing and capabilities to model_prices_and_context_window.json (#11373) 2025-06-03 11:32:13 -07:00
Krrish Dholakia
d5842edf09 docs(vllm.md): add vllm - model list loadbalancing tutorial to docs 2025-06-03 09:38:33 -07:00
Krrish Dholakia
cff8c3971c fix: fix import 2025-06-03 09:22:37 -07:00
AnilAren
2486743904
Doc : Nvidia embedding models (#11352)
* fix: bedrock ai21 jamba models will work now

* Update supported_embedding.md

* Update supported_embedding.md
2025-06-03 09:17:07 -07:00
Krrish Dholakia
5ff6db5062 fix: fix merge conflicts 2025-06-03 08:26:33 -07:00