Commit graph

23993 commits

Author SHA1 Message Date
Tasmay Pankaj Tibrewal
a85ab9d204
added qwen3, deepseek r1 0528 throughput, glm 4.5 and gpt oss models 2025-08-15 04:34:55 +05:30
Tasmay Pankaj Tibrewal
4b51e5787c
added qwen3, deepseek r1 0528 throughput, glm 4.5 and gpt oss models 2025-08-15 04:33:49 +05:30
Krrish Dholakia
aaf9c38a10 test: skip test - ran out of credits 2025-08-14 15:01:26 -07:00
Jugal D. Bhatt
3a38912bce
[Proxy] Litellm fix mapped tests (#13634)
* Fix - add safe divide by 0 for most places to prevent crash

* mock prisma client

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e390.
2025-08-14 14:19:36 -07:00
Krish Dholakia
cedba81f4c
Merge pull request #13588 from BerriAI/litellm_admin_viewer_updates
[Management/UI]Allow routes for admin viewer
2025-08-14 12:03:00 -07:00
Krish Dholakia
f360e0ead2
Merge pull request #13590 from BerriAI/litellm_bedrock_api_header
[LLM translation] Refactor Anthropic Configurations and Add Support for `anthropic_beta` Headers
2025-08-14 11:32:18 -07:00
Jugal D. Bhatt
1beba93cc8
Fix - add safe divide by 0 for most places to prevent crash (#13624) 2025-08-14 11:17:49 -07:00
Krish Dholakia
8d76935457
Merge pull request #13607 from huangyafei/update_price
Add deepseek-chat-v3-0324 to OpenRouter cost map
2025-08-14 11:05:53 -07:00
Krish Dholakia
fe2833817e
Merge pull request #13625 from BerriAI/litellm_dev_08_13_2025_p1
perf(main.py): new 'EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER' flag
2025-08-14 11:04:35 -07:00
Ishaan Jaff
5bb96af818
[Feat] Add reasoning_effort param for hosted_vllm provider (#13620)
* add reasoning_effort to hosted_vllm

* test_hosted_vllm_supports_reasoning_effort

* Reasoning Effort
2025-08-14 10:10:30 -07:00
Cole McIntosh
dea98a315b
fix(volcengine): handle thinking disabled parameter properly (#13598)
* fix(volcengine): handle thinking disabled parameter properly

When thinking is set to {"type": "disabled"} in volcengine provider config,
skip adding the parameter to extra_body entirely instead of passing it through.
This prevents "thinking: undefined" from appearing in request logs.

Fixes #13039

* test(volcengine): fix and enhance thinking parameter tests

- Fixed existing test that expected broken behavior
- Added comprehensive test coverage for all thinking parameter scenarios:
  * thinking disabled → omitted from extra_body
  * thinking enabled → included in extra_body
  * thinking None → included in extra_body as None
  * custom thinking values → included in extra_body
  * no thinking parameter → empty result

All tests passing, verifying the fix for issue #13039
2025-08-14 09:10:06 -07:00
tanjiro
b5d0a7eb95
adding missing imports + removing unused imports (#13610) 2025-08-14 08:26:15 -07:00
Krrish Dholakia
0288ed35da test: update tests 2025-08-13 23:33:32 -07:00
Krrish Dholakia
b53962dee2 test: update test 2025-08-13 23:09:18 -07:00
huangyafei
2b89f95e37 Add deepseek-chat-v3-0324 to OpenRouter model list 2025-08-14 10:44:15 +08:00
Krrish Dholakia
5a7a889d93 perf(main.py): new 'EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER' flag
improves RPS for openai calls by 100 (100 users, 10 start-up)

 Moves to using litellm's asynchttphandler vs. openais's sdk for llm calling
2025-08-13 19:12:39 -07:00
Ishaan Jaff
cd893134b7 test team endpoints 2025-08-13 18:43:50 -07:00
Ishaan Jaff
ce4210a17a bump litellm proxy extras 2025-08-13 17:50:10 -07:00
Ishaan Jaff
086621e3d3 test_handle_jwt.py 2025-08-13 17:27:50 -07:00
Ishaan Jaff
8e76f8e7d0
[Feat] Team Member Rate Limits + Support for using with JWT Auth (#13601)
* fix - assign tpm/rpm limit onJWT

* add team member rpm/tpm limits

* update - rate limiter v3 with team member rate limits

* update utils

* fixes for LiteLLM_BudgetTable

* undo change

* add TeamMemberBudgetHandler

* add _process_team_member_budget_data

* add get_team_membership

* add safe_get_team_member_rpm_limit and safe_get_team_member_tpm_limit

* LiteLLM_TeamMembership

* add LiteLLM_TeamMembership rate limit for JWTs

* fix

* tests
2025-08-13 17:21:36 -07:00
Ishaan Jaff
76d25926d4
[Feat] New provider - Azure AI Flux Image Generation (#13592)
* init files

* add AzureFoundryModelInfo

* fix api_version property

* add azure_ai img gen

* use AzureFoundryModelInfo

* get_base_image_generation_call_args

* add azure_ai/FLUX-1.1-pro

* add util for route_image_generation_cost_calculator

* docs azure ai flux

* fixes for flux

* fixes for AzureFoundryFluxImageGenerationConfig

* ruff fix
2025-08-13 17:20:30 -07:00
Ishaan Jaff
fb325cbb5e
fix Build from litellm pip package (#13603) 2025-08-13 17:19:43 -07:00
Krrish Dholakia
2e21f1d07a fix(router.py): fix test 2025-08-13 16:27:13 -07:00
Jugal Bhatt
3b47355449 Enhance route access checks for PROXY_ADMIN_VIEW_ONLY role in RouteChecks class
- Added additional routes for key management operations to the access control logic.
- Improved handling of routes that start with "/key/" and end with "/regenerate" to ensure proper access restrictions.
2025-08-13 16:04:43 -07:00
Ishaan Jaff
75bcfbb76a
[Feat] New model vertex_ai/deepseek-ai/deepseek-r1-0528-maas (#13594)
* add ertex_ai/deepseek-ai/deepseek-r1-0528-maas

* fix init

* test_model_info_for_vertex_ai_deepseek_model
2025-08-13 13:44:45 -07:00
Jugal Bhatt
4201f0aa79 Enhance Bedrock Provider Configuration and Header Management
- Added `forward_client_headers_to_llm_api` setting in the Bedrock documentation to facilitate client-side header forwarding.
- Updated `completion` function to use merged headers instead of original `extra_headers`.
- Improved request handling in `BedrockConverseLLM` and `AmazonInvokeConfig` to ensure proper header management for `anthropic-beta` parameters.
- Refactored request transformation logic to return the transformed request for better clarity and functionality.
2025-08-13 13:41:34 -07:00
Jugal Bhatt
26e62c9bd8 Update Test for Anthropic Messages Configuration
- Renamed `AmazonAnthropicClaude3MessagesConfig` to `AmazonAnthropicClaudeMessagesConfig` in the test file to align with recent refactoring.
- Adjusted the instantiation of the configuration class in the test to reflect the new naming convention.
2025-08-13 11:53:11 -07:00
Jugal Bhatt
c2310bcccc Refactor Anthropic Configurations in Tests
- Updated test cases to use the renamed `AmazonAnthropicClaudeConfig` instead of `AmazonAnthropicClaude3Config` for consistency with recent changes.
- Adjusted imports and assertions in test files to reflect the new configuration class name.
2025-08-13 11:52:41 -07:00
Jugal Bhatt
3990f61bed Refactor Anthropic Configurations and Add Support for anthropic_beta Headers
- Renamed `AmazonAnthropicClaude3Config` and `AmazonAnthropicClaude3MessagesConfig` to `AmazonAnthropicClaudeConfig` and `AmazonAnthropicClaudeMessagesConfig` respectively for consistency.
- Implemented `get_anthropic_beta_from_headers` function to extract and handle `anthropic-beta` headers across various transformations.
- Updated request transformations in `AmazonConverseConfig` and `AmazonInvokeConfig` to include `anthropic_beta` parameters based on user headers.
- Added tests to ensure proper handling of `anthropic_beta` headers in different scenarios.
2025-08-13 11:47:59 -07:00
Jugal Bhatt
a74056e707 Refactor access checks for PROXY_ADMIN_VIEW_ONLY role in RouteChecks class
- Consolidated access control logic into a new static method `_check_proxy_admin_viewer_access`.
- Improved readability and maintainability by reducing code duplication in route access checks.
- Ensured proper handling of write operations and parameter validation for management routes.
2025-08-13 11:00:32 -07:00
Krish Dholakia
ffd165c9b7
Merge pull request #13549 from BerriAI/litellm_mcp_config_fix
[fix] Enhance MCPServerManager with access groups and description support
2025-08-13 10:51:19 -07:00
Krish Dholakia
a1f6ade9f6
Merge pull request #13582 from BerriAI/remove-network-response-error
Remove ambiguous network response error
2025-08-13 10:50:05 -07:00
Jugal Bhatt
89a1150033 Allow routes for admin viewer 2025-08-13 10:49:02 -07:00
Michael Verunica
37e57a0e5f
fix(azure): remove trailing semicolon in Content-Type header for image generation (#13584) 2025-08-13 09:43:02 -07:00
Krish Dholakia
5e45fcdff0
Merge pull request #13375 from colesmcintosh/fix/ollama-gpt-oss-thinking-field 2025-08-13 07:58:57 -07:00
Krzysztof Gąsiorowski
0b28930ad4
[Fix] Hide sensitive data in /model/info - azure entra client_secret (#13577)
* Remove litellm_params.client_secret from /model/info

Added pop of client_secret (Azure provider secret) from litellm_params in remove_sensitive_info_from_deployment used by /model/info endpoints

* Added test for litellm.proxy.common_utils.openai_endpoint_utils.remove_sensitive_info_from_deployment
2025-08-13 07:35:53 -07:00
tanjiro
be109c2180 put the error toast on the ui 2025-08-13 23:05:11 +09:00
tanjiro
06ee35f74c replace text error with json error 2025-08-13 22:32:57 +09:00
Krrish Dholakia
5ae44e3275 fix(router.py): fix cooldown increment logic 2025-08-12 23:33:56 -07:00
Krish Dholakia
6afaf5721a
[Fix] Streaming - consistent 'finish_reason' chunk index (#13560)
* feat(model_response_utils.py): new function to check if modelresponsestream is empty

used for checking https://github.com/BerriAI/litellm/issues/13348

* fix(streaming_handler.py): skip chunk if empty

Fixes https://github.com/BerriAI/litellm/issues/13348

* fix(streaming_handler.py): add is_empty logic to async flow
2025-08-12 23:21:57 -07:00
Krish Dholakia
f00e891004
LiteLLM SDK <-> Proxy: support user param + Prisma - remove use_prisma_migrate flag - redundant as this is now default (#13555)
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params

Fixes issue where 'user' was not being sent in request to litellm proxy via sdk

* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default

* docs: cleanup docs

* fix(proxy_cli.py): remove --use_prisma_migrate flag

* refactor: remove references to use_prisma_migrate env var

This is now the default flow for db migrations
2025-08-12 22:03:39 -07:00
Ishaan Jaff
38d0736599 bump fastapi-offline 2025-08-12 21:19:29 -07:00
Sameer Kankute
74496eefea
Implement GitHub Action to auto-label issues with provider keywords (#13537)
* Add auto issue labelling for llm_translation

* Add logging

* make the formatting better

* make the formatting better

* make the formatting better

* make the formatting better
2025-08-12 21:18:20 -07:00
Jugal D. Bhatt
e51c15705c
[Feat] Add CredentialDeleteModal component and integrate with CredentialsPanel (#13550)
* [Feat] Add CredentialDeleteModal component and integrate with CredentialsPanel

* Implement CredentialDeleteModal for confirming credential deletions.
* Update CredentialsPanel to manage delete modal state and handle credential deletion.
* Ensure user confirmation before deleting credentials to prevent accidental removals.

* Update CredentialDeleteModal to use smart quotes for credential name in confirmation message

* Enhance CredentialDeleteModal with confirmation input and validation

* Added input field for user to confirm credential deletion by typing the credential name.
* Updated modal behavior to handle confirmation and cancellation actions.
* Removed unnecessary state management in CredentialsPanel for delete modal visibility.

* Fix HTML entity for apostrophe in CredentialDeleteModal confirmation message
2025-08-12 21:17:39 -07:00
Ishaan Jaff
1249385a99
[Feat] GEMINI CLI - Add Token Counter for VertexAI Models (#13558)
* add VertexAIModelInfo

* working API call to vertex ai

* add count_tokens MODE

* _construct_url

* test_vertex_ai_gemini_token_counting_with_contents
2025-08-12 20:53:47 -07:00
Ishaan Jaff
74ddd21067
[Bug fix] SCIM Team Memberships - handle metadata (#13553)
* test_update_group_metadata_serialization_issue

* fixes for update_group

* test_update_group_metadata_serialization_issue
2025-08-12 20:53:12 -07:00
tanjiro
ebae03cf93
Enhance chart readability with short-form notation for large numbers (#12370)
* format y-axis value for total tokens

* format y-axis for the rest of the charts on model-activity

* revert changes for requests per day

* labels modified to plain text

* added plain text label for api_requests and spend

* minor

* move components to utils
2025-08-12 17:24:03 -07:00
tanjiro
88e4d302a2
Fix internal users table overflow (#12736)
* modify column name

* fix overflow

* remove height
2025-08-12 17:23:23 -07:00
tanjiro
e4364ad1c4
Team Member Permissions Page - Access Column Changes (#13145)
* revert prettier

* move allow access to right
2025-08-12 17:22:24 -07:00
tanjiro
911b0cfd73
Display Error from Backend on the UI - Keys Page (#13435)
* fix sso logout

- add a new login page with sso button

* lint fix

* lint fix

* lint fix

* fix tests

* fix test

* Revert "fix test"

This reverts commit 74eb734571.

* Reapply "fix test"

This reverts commit 72d0b2d4c6.

* add host to add modal

* close modal after save is clicked. and auto-refresh

* show old values in edit modal

* send the whole payload on edit

* Update settings.tsx

* resolve conflict

* fix conflict

* merge main

* first draft of notifications added to settings

* add error compatibility by taking errors from the backend

- db errors
- auth errors

* add support for different types of errors

* minor

* name change

* email alerts page notifications modified

* remove unused code

* move create_key to organisms/ folder

* move view_key_table to templates

* keys page notifications

* move regenerate_key to organisms

* notifications for regenerate key modal

* move key info view and key edit view to templates/ folder

* replace "Network response was not ok" with error data

* resolve merge conflict
2025-08-12 17:21:04 -07:00