Commit graph

24475 commits

Author SHA1 Message Date
Jugal Bhatt
3b47355449 Enhance route access checks for PROXY_ADMIN_VIEW_ONLY role in RouteChecks class
- Added additional routes for key management operations to the access control logic.
- Improved handling of routes that start with "/key/" and end with "/regenerate" to ensure proper access restrictions.
2025-08-13 16:04:43 -07:00
Ishaan Jaff
75bcfbb76a
[Feat] New model vertex_ai/deepseek-ai/deepseek-r1-0528-maas (#13594)
* add ertex_ai/deepseek-ai/deepseek-r1-0528-maas

* fix init

* test_model_info_for_vertex_ai_deepseek_model
2025-08-13 13:44:45 -07:00
Jugal Bhatt
4201f0aa79 Enhance Bedrock Provider Configuration and Header Management
- Added `forward_client_headers_to_llm_api` setting in the Bedrock documentation to facilitate client-side header forwarding.
- Updated `completion` function to use merged headers instead of original `extra_headers`.
- Improved request handling in `BedrockConverseLLM` and `AmazonInvokeConfig` to ensure proper header management for `anthropic-beta` parameters.
- Refactored request transformation logic to return the transformed request for better clarity and functionality.
2025-08-13 13:41:34 -07:00
Jugal Bhatt
26e62c9bd8 Update Test for Anthropic Messages Configuration
- Renamed `AmazonAnthropicClaude3MessagesConfig` to `AmazonAnthropicClaudeMessagesConfig` in the test file to align with recent refactoring.
- Adjusted the instantiation of the configuration class in the test to reflect the new naming convention.
2025-08-13 11:53:11 -07:00
Jugal Bhatt
c2310bcccc Refactor Anthropic Configurations in Tests
- Updated test cases to use the renamed `AmazonAnthropicClaudeConfig` instead of `AmazonAnthropicClaude3Config` for consistency with recent changes.
- Adjusted imports and assertions in test files to reflect the new configuration class name.
2025-08-13 11:52:41 -07:00
Jugal Bhatt
3990f61bed Refactor Anthropic Configurations and Add Support for anthropic_beta Headers
- Renamed `AmazonAnthropicClaude3Config` and `AmazonAnthropicClaude3MessagesConfig` to `AmazonAnthropicClaudeConfig` and `AmazonAnthropicClaudeMessagesConfig` respectively for consistency.
- Implemented `get_anthropic_beta_from_headers` function to extract and handle `anthropic-beta` headers across various transformations.
- Updated request transformations in `AmazonConverseConfig` and `AmazonInvokeConfig` to include `anthropic_beta` parameters based on user headers.
- Added tests to ensure proper handling of `anthropic_beta` headers in different scenarios.
2025-08-13 11:47:59 -07:00
Jugal Bhatt
a74056e707 Refactor access checks for PROXY_ADMIN_VIEW_ONLY role in RouteChecks class
- Consolidated access control logic into a new static method `_check_proxy_admin_viewer_access`.
- Improved readability and maintainability by reducing code duplication in route access checks.
- Ensured proper handling of write operations and parameter validation for management routes.
2025-08-13 11:00:32 -07:00
Krish Dholakia
ffd165c9b7
Merge pull request #13549 from BerriAI/litellm_mcp_config_fix
[fix] Enhance MCPServerManager with access groups and description support
2025-08-13 10:51:19 -07:00
Krish Dholakia
a1f6ade9f6
Merge pull request #13582 from BerriAI/remove-network-response-error
Remove ambiguous network response error
2025-08-13 10:50:05 -07:00
Jugal Bhatt
89a1150033 Allow routes for admin viewer 2025-08-13 10:49:02 -07:00
Michael Verunica
37e57a0e5f
fix(azure): remove trailing semicolon in Content-Type header for image generation (#13584) 2025-08-13 09:43:02 -07:00
Krish Dholakia
5e45fcdff0
Merge pull request #13375 from colesmcintosh/fix/ollama-gpt-oss-thinking-field 2025-08-13 07:58:57 -07:00
Krzysztof Gąsiorowski
0b28930ad4
[Fix] Hide sensitive data in /model/info - azure entra client_secret (#13577)
* Remove litellm_params.client_secret from /model/info

Added pop of client_secret (Azure provider secret) from litellm_params in remove_sensitive_info_from_deployment used by /model/info endpoints

* Added test for litellm.proxy.common_utils.openai_endpoint_utils.remove_sensitive_info_from_deployment
2025-08-13 07:35:53 -07:00
tanjiro
be109c2180 put the error toast on the ui 2025-08-13 23:05:11 +09:00
tanjiro
06ee35f74c replace text error with json error 2025-08-13 22:32:57 +09:00
Yuki Imajuku
b75961fb20 update openrouter claude sonnet 2025-08-13 21:05:14 +09:00
Dor Zion
6a1f5bdc2f
Add Noma Security guardrail support 2025-08-13 13:39:21 +03:00
nielsbosma
fe54da79a1 test(braintrust-logging): add span_name tests for events
Add tests to verify custom and default span_name in BraintrustLogger,
including async, metadata merging, and span name behavior.
2025-08-13 12:26:14 +02:00
nielsbosma
f2b0b6124c feat(logging): add support for custom span names in Braintrust logging 2025-08-13 12:18:17 +02:00
TensorNull
8de6f0b3ce
Merge branch 'BerriAI:main' into main 2025-08-13 15:23:47 +08:00
Krrish Dholakia
5ae44e3275 fix(router.py): fix cooldown increment logic 2025-08-12 23:33:56 -07:00
Krish Dholakia
6afaf5721a
[Fix] Streaming - consistent 'finish_reason' chunk index (#13560)
* feat(model_response_utils.py): new function to check if modelresponsestream is empty

used for checking https://github.com/BerriAI/litellm/issues/13348

* fix(streaming_handler.py): skip chunk if empty

Fixes https://github.com/BerriAI/litellm/issues/13348

* fix(streaming_handler.py): add is_empty logic to async flow
2025-08-12 23:21:57 -07:00
iamkankute
38635b9e07 fix: remove incorrect web search support for azure/gpt-4.1 family 2025-08-13 11:41:50 +05:30
Yuki Imajuku
86d8fcf5fd update model prices and context window 2025-08-13 14:56:32 +09:00
iamkankute
3d0f417829 fix: remove incorrect web search support for azure/gpt-4.1 family 2025-08-13 11:13:56 +05:30
sudu
595319cfbd
Merge branch 'BerriAI:main' into main 2025-08-13 13:07:34 +08:00
Krish Dholakia
f00e891004
LiteLLM SDK <-> Proxy: support user param + Prisma - remove use_prisma_migrate flag - redundant as this is now default (#13555)
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params

Fixes issue where 'user' was not being sent in request to litellm proxy via sdk

* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default

* docs: cleanup docs

* fix(proxy_cli.py): remove --use_prisma_migrate flag

* refactor: remove references to use_prisma_migrate env var

This is now the default flow for db migrations
2025-08-12 22:03:39 -07:00
sudu
902e5e71e1
Update simple_shuffle.py, choose weights by 'weight', 'rpm', 'tpm' in one loop 2025-08-13 12:40:34 +08:00
Ishaan Jaff
38d0736599 bump fastapi-offline 2025-08-12 21:19:29 -07:00
Sameer Kankute
74496eefea
Implement GitHub Action to auto-label issues with provider keywords (#13537)
* Add auto issue labelling for llm_translation

* Add logging

* make the formatting better

* make the formatting better

* make the formatting better

* make the formatting better
2025-08-12 21:18:20 -07:00
Jugal D. Bhatt
e51c15705c
[Feat] Add CredentialDeleteModal component and integrate with CredentialsPanel (#13550)
* [Feat] Add CredentialDeleteModal component and integrate with CredentialsPanel

* Implement CredentialDeleteModal for confirming credential deletions.
* Update CredentialsPanel to manage delete modal state and handle credential deletion.
* Ensure user confirmation before deleting credentials to prevent accidental removals.

* Update CredentialDeleteModal to use smart quotes for credential name in confirmation message

* Enhance CredentialDeleteModal with confirmation input and validation

* Added input field for user to confirm credential deletion by typing the credential name.
* Updated modal behavior to handle confirmation and cancellation actions.
* Removed unnecessary state management in CredentialsPanel for delete modal visibility.

* Fix HTML entity for apostrophe in CredentialDeleteModal confirmation message
2025-08-12 21:17:39 -07:00
Ishaan Jaff
1249385a99
[Feat] GEMINI CLI - Add Token Counter for VertexAI Models (#13558)
* add VertexAIModelInfo

* working API call to vertex ai

* add count_tokens MODE

* _construct_url

* test_vertex_ai_gemini_token_counting_with_contents
2025-08-12 20:53:47 -07:00
Ishaan Jaff
74ddd21067
[Bug fix] SCIM Team Memberships - handle metadata (#13553)
* test_update_group_metadata_serialization_issue

* fixes for update_group

* test_update_group_metadata_serialization_issue
2025-08-12 20:53:12 -07:00
TensorNull
7cce327d09
Merge branch 'BerriAI:main' into main 2025-08-13 09:05:33 +08:00
tanjiro
ebae03cf93
Enhance chart readability with short-form notation for large numbers (#12370)
* format y-axis value for total tokens

* format y-axis for the rest of the charts on model-activity

* revert changes for requests per day

* labels modified to plain text

* added plain text label for api_requests and spend

* minor

* move components to utils
2025-08-12 17:24:03 -07:00
tanjiro
88e4d302a2
Fix internal users table overflow (#12736)
* modify column name

* fix overflow

* remove height
2025-08-12 17:23:23 -07:00
tanjiro
e4364ad1c4
Team Member Permissions Page - Access Column Changes (#13145)
* revert prettier

* move allow access to right
2025-08-12 17:22:24 -07:00
tanjiro
911b0cfd73
Display Error from Backend on the UI - Keys Page (#13435)
* fix sso logout

- add a new login page with sso button

* lint fix

* lint fix

* lint fix

* fix tests

* fix test

* Revert "fix test"

This reverts commit 74eb734571.

* Reapply "fix test"

This reverts commit 72d0b2d4c6.

* add host to add modal

* close modal after save is clicked. and auto-refresh

* show old values in edit modal

* send the whole payload on edit

* Update settings.tsx

* resolve conflict

* fix conflict

* merge main

* first draft of notifications added to settings

* add error compatibility by taking errors from the backend

- db errors
- auth errors

* add support for different types of errors

* minor

* name change

* email alerts page notifications modified

* remove unused code

* move create_key to organisms/ folder

* move view_key_table to templates

* keys page notifications

* move regenerate_key to organisms

* notifications for regenerate key modal

* move key info view and key edit view to templates/ folder

* replace "Network response was not ok" with error data

* resolve merge conflict
2025-08-12 17:21:04 -07:00
Krrish Dholakia
280ad6f049 fix(main.py): add sambanova api base support to embeddings 2025-08-12 17:17:19 -07:00
Jorge Piedrahita Ortiz
6a77780c0e
Feat/sambanova embeddings (#13308)
* add sambanova embeddings

* fmt

* minor fix

* add sambanova embeddings call

* fmt

* include embeddings in sambanova docs

* add sambanova embeddigns unit test

* remove unused import

* minor fix

* fmt

* update sambanova embeding to inherit from BaseEmbeddingConfig

* fmt

* fmt

* hot fix

* fmt
2025-08-12 17:15:26 -07:00
Jugal Bhatt
bb32ec991f Merge branch 'main' into litellm_mcp_config_fix 2025-08-12 17:10:18 -07:00
Ishaan Jaff
afe159bb8b
[Feat] GEMINI CLI Integration - Add /countTokens endpoint support (#13545)
* stash changes for token counter

* working TokenCountRequest

* working acount_tokens

* add GoogleAIStudioTokenCounter

* re-use validate_environment

* fixes count_tokens

* fixes google_count_tokens

* fixes token counter base class

* fix TokenCountResponse

* fix - use BaseTokenCounter

* add should_use_token_counting_api

* fixes for GoogleAIStudioTokenCounter

* fixes for should_use_token_counting_api

* fixes for google_count_tokens

* fixes for /messages count_tokens

* fixes for should_use_token_counting_api

* working e2e gemini token counter

* ruff check fixes

* fixes for token counter

* fixes for TokenCountResponse

* cleanup TokenCountRequest

* add TokenCountDetailsResponse

* fix use well typed Responses

* fix typing for TokenCountDetailsResponse

* test_vertex_ai_gemini_token_counting_with_contents

* fixes for TokenCountDetailsResponse

* test fixes

* test_factory_registration

* test_proxy_token_counter.py

* TestGoogleAIStudioTokenCounter

* fix token_counter
2025-08-12 16:19:58 -07:00
Jugal Bhatt
f487816b9f [fix] Enhance MCPServerManager with access groups and description support
* Added access_groups and description fields to MCPServerManager for better server configuration.
* Implemented tests to verify integration of config-based servers with database servers, ensuring correct handling of access_groups and description.
* Updated add_update_server method to accommodate new fields and validate server addition in the registry.
2025-08-12 14:22:40 -07:00
Ishaan Jaff
828e3e3deb
[Feat] Add Streaming support + Docs for bedrock gpt-oss model family (#13346)
* add openai.gpt-oss-20b-1:0

* update BEDROCK_CONVERSE_MODELS

* openai.gpt-oss-20b-1:0 fixes

* fix PDF input

* fix for should_fake_stream

* TestBedrockGPTOSS

* should_fake_stream

* update supports vision field for openai.gpt-oss models

* fixes for bedrock gpt oss

* fixes for should_fake_stream

* docs bedrock gpt oss models
2025-08-12 08:39:36 -07:00
Edward Samuel Pasaribu
36f160b582
Update openrouter/openai/gpt-5-mini pricing 2025-08-12 18:54:06 +08:00
Edward Samuel Pasaribu
d898f9e0dd
Add openrouter gpt-5 family models pricing 2025-08-12 18:51:56 +08:00
TensorNull
a3bf332697
Merge pull request #3 from cometapi-dev/dev
docs: Add CometAPI documentation with authentication, usage examples, and error handling
2025-08-12 17:35:48 +08:00
TensorNull
fa81c20df6 fix: Remove outdated models from the model list in the CometAPI document 2025-08-12 17:28:10 +08:00
TensorNull
8b602f9507 [Feat] - Add CometAPI documentation with authentication, usage examples, and error handling 2025-08-12 17:18:33 +08:00
NULL
4d2b5076a0
Merge branch 'BerriAI:main' into dev 2025-08-12 16:51:25 +08:00