Commit graph

24054 commits

Author SHA1 Message Date
Michal Otmianowski
405e74ec16 use namespace as prefix in s3 2025-08-18 13:43:51 +02:00
Krish Dholakia
c99277c517
Merge pull request #13694 from BerriAI/litellm_dev_08_16_2025_p3
Litellm dev 08 16 2025 p3
2025-08-17 13:02:17 -07:00
Ishaan Jaff
6c4ced275d ui new build 2025-08-16 19:08:18 -07:00
Ishaan Jaff
5c4ba732a6
fix client side exception (#13684) 2025-08-16 19:04:06 -07:00
Krrish Dholakia
21549a3463 docs(index.md): write up on session management 2025-08-16 17:55:18 -07:00
Krrish Dholakia
cb95f2d5d7 docs(index.md): clarify risk of upgrade 2025-08-16 17:38:50 -07:00
Krrish Dholakia
788a1d03a1 docs(index.md): cleanup 2025-08-16 17:33:07 -07:00
Krrish Dholakia
235b6b040f docs(index.md): add highlight note on redis latency improvements 2025-08-16 17:32:45 -07:00
Krrish Dholakia
8a75ab8388 docs(index.md): cleanup doc 2025-08-16 17:16:04 -07:00
Krish Dholakia
50dec7cdee
Merge pull request #13682 from BerriAI/litellm_stable_release_fixes
Litellm stable release fixes
2025-08-16 15:10:55 -07:00
Krrish Dholakia
e035efeec7 fix(public_model_hub.tsx): fix ui 2025-08-16 14:30:26 -07:00
Krrish Dholakia
6322aef0e3 fix(streaming_handler.py): fix streaming chunk calculation 2025-08-16 14:25:29 -07:00
Krrish Dholakia
eb66daeef7 test: update test
we now return correct token usage on clientside
2025-08-16 14:14:35 -07:00
Ishaan Jaff
d96df5e9be Revert "test_stream_token_counting_anthropic_with_include_usage"
This reverts commit c3aee1194b.
2025-08-16 13:07:00 -07:00
Ishaan Jaff
c3aee1194b test_stream_token_counting_anthropic_with_include_usage 2025-08-16 13:06:31 -07:00
Krrish Dholakia
8bb5ee2ba5 test: update unit test 2025-08-16 12:39:03 -07:00
Krrish Dholakia
1ec0808713 test: update test - anthropic returns input tokens in final usage chunk now 2025-08-16 12:31:52 -07:00
Krish Dholakia
73f89d51a1
Merge pull request #13629 from BerriAI/budget-duration
Edit Budget Duration + Other Improvements in User Settings
2025-08-16 11:19:07 -07:00
Ishaan Jaff
5889311b10
[Docs] v1.75.8-stable (#13676)
* v1-75-8

* fix responses API docs

* docs fix

* docs fix new models

* docs links

* docs fix links

* docs sambanova

* docs fix
2025-08-16 09:20:04 -07:00
Ishaan Jaff
f522f40228 test_passing_tool_result_as_list 2025-08-16 08:08:25 -07:00
Ishaan Jaff
76939c5e73 ui new build 2025-08-16 07:59:18 -07:00
Ishaan Jaff
cac27be56a bump: version 1.75.7 → 1.75.8 2025-08-16 07:58:17 -07:00
Krish Dholakia
1b2ec16eee
Merge pull request #13640 from yytdfc/fix_bedrock_epc
[Bug Fix] Add cachePoint support for assistant and tool messages in Bedrock
2025-08-16 01:20:28 -07:00
Ishaan Jaff
40395598da
[Feat] UI - Allow editing team member rpm/tpm limits (#13669)
* show team member tpm/rpm limits

* ui - allow setting team settings

* fix better debugging

* fix types: TeamMemberUpdateRequest

* add _upsert_budget_and_membership

* allow updating team member RPM/TPM in teamMemberUpdateCall

* editing team member rpm/tpm

* UI - fixes for team member component

* fix info

* test_upsert_rpm_only_creates_new_budget
2025-08-15 17:29:44 -07:00
0x-fang
9f17bedb48
bump up boto3 and aioboto3 versions (#13665) 2025-08-15 17:06:08 -07:00
Ishaan Jaff
38e5af3580
[Feat] UI QA for Team Member Rate Limits (#13664)
* show team member tpm/rpm limits

* ui - allow setting team settings

* fix better debugging
2025-08-15 16:21:11 -07:00
Ishaan Jaff
4a10eeb75e
[Feat] Team Member Rate Limits - show team member tpm/rpm limits (#13662)
* show team member tpm/rpm limits

* ui - allow setting team settings
2025-08-15 12:18:13 -07:00
Ishaan Jaff
f5a1c14929 Revert "update boto3 version to 1.37.38 (#13656)"
This reverts commit 6f2264a13d.
2025-08-15 11:24:59 -07:00
Ishaan Jaff
c331473b75 Revert "fix aioboto3"
This reverts commit 82d1cb48b7.
2025-08-15 11:24:50 -07:00
Ishaan Jaff
49bef09a3a Revert "fix aioboto3"
This reverts commit 92c638ec59.
2025-08-15 11:24:41 -07:00
Ishaan Jaff
92c638ec59 fix aioboto3 2025-08-15 11:19:21 -07:00
Ishaan Jaff
82d1cb48b7 fix aioboto3 2025-08-15 11:19:14 -07:00
James Page
46e8e50b47
trivy/secrets false positives (#13631)
* alert_to_webhook_url: avoid secret false positive

Trivy detects the example as a potential secret; tweak the hostname
a bit to make it obviously not hooks.slack.com to avoid this.

* Remove log file

Drop log file checked into the git tree; this contains strings that
trivy detects as potential secrets.
2025-08-15 11:10:54 -07:00
Daniel Barker
47edecd5bc
Fixed incorrect key info endpoint (#13633) 2025-08-15 11:10:06 -07:00
William Finnigan
99f30fe5b7
Fix LangfuseOtelSpanAttributes constants to match expected values at https://python.reference.langfuse.com/langfuse#LangfuseOtelSpanAttributes (#13659)
- Update TRACE_USER_ID from 'langfuse.trace.user_id' to 'user.id'
- Update SESSION_ID from 'langfuse.trace.session_id' to 'session.id'
2025-08-15 11:08:32 -07:00
0x-fang
6f2264a13d
update boto3 version to 1.37.38 (#13656) 2025-08-15 11:06:54 -07:00
Ishaan Jaff
c3608adb0f
[Bug Fix] Using stream=True + background=True with Responses API (#13654)
* test_update_responses_api_response_id_with_model_id_handles_dict

* fix linting
2025-08-15 08:58:28 -07:00
Ishaan Jaff
d6fa6b60d7
[Feat] UI - Add Confirmation Modal Before Deleting Keys (#13655)
* Enhance key deletion with confirmation input and improved modal UI

Co-authored-by: ishaan <ishaan@berri.ai>

* remove file

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-08-15 08:51:32 -07:00
FuChen
bb96d4e23c Add cachePoint support for assistant and tool messages in Bedrock 2025-08-15 12:17:59 +08:00
Ishaan Jaff
d29bc4255b bump: version 1.75.6 → 1.75.7 2025-08-14 21:13:21 -07:00
FuChen
511363d3a7 test: Add comprehensive test cases for cachePoint support
- test_assistant_message_cache_control: Tests assistant messages with string content and cache_control
- test_assistant_message_list_content_cache_control: Tests assistant messages with list content and cache_control
- test_tool_message_cache_control: Tests tool messages with list content and cache_control
- test_tool_message_string_content_cache_control: Tests tool messages with string content and cache_control
- test_assistant_tool_calls_cache_control: Tests assistant tool_calls with cache_control
- test_multiple_tool_calls_with_mixed_cache_control: Tests multiple tool calls with mixed cache_control
- test_no_cache_control_no_cache_point: Tests that messages without cache_control don't generate cachePoint blocks

These tests ensure that cachePoint blocks are correctly generated for all message types when cache_control is present.
2025-08-15 11:56:34 +08:00
FuChen
210fff585d feat: Add cachePoint support for assistant and tool messages in Bedrock
- Add cachePoint support for assistant messages (both string and list content)
- Add cachePoint support for tool messages (both message-level and content-level cache_control)
- Add cachePoint support for assistant tool_calls
- Move CachePointBlock import to file header for better code organization
- Ensure cachePoint blocks are created as separate content blocks alongside main content

This enables comprehensive cache control across all message types in Bedrock conversations.
2025-08-15 11:40:40 +08:00
Ishaan Jaff
48c89812c4
[Bug Fix] /messages endpoint - ensure tool use arguments are returned for non-anthropic models (#13638)
* bug fix _translate_streaming_openai_chunk_to_anthropic

* test  test_translate_streaming_openai_chunk_to_anthropic_with_partial_json
2025-08-14 17:29:42 -07:00
Ishaan Jaff
17db9edd85
UI - Fix image overflow in LiteLLM model (#13639)
* Improve LiteLLM model name display with better styling and overflow handling

Co-authored-by: ishaan <ishaan@berri.ai>

* Add Tooltip to LiteLLM model name for improved text display

Co-authored-by: ishaan <ishaan@berri.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-08-14 17:29:23 -07:00
Krrish Dholakia
5631d97964 docs(readme.md): cleanup 2025-08-14 17:16:05 -07:00
Krrish Dholakia
41f7901cfa docs(readme.md): add note, saying poetry is required 2025-08-14 17:15:29 -07:00
Krrish Dholakia
817b8408fa docs(readme.md): fix readme 2025-08-14 17:13:04 -07:00
Ishaan Jaff
b78495d398
[Fix] Ensure /messages works when using `bedrock/converse/<model> with LiteLLM (#13627)
* get_bedrock_provider_config_for_messages_api

* fixes for get_bedrock_provider_config_for_messages_api

* test_anthropic_messages_litellm_router_bedrock

* fix merge conflicts

* fix - refactor based on jugal's comment
2025-08-14 16:50:05 -07:00
Jugal D. Bhatt
b8fe5f7b17
[MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e390.
2025-08-14 16:32:18 -07:00
Jugal D. Bhatt
aea0605eed
[LLM Translation] Fix Realtime API endpoint for no intent (#13476)
* fix intent params

* Add responses

* fix unrelated test

* test fix - fireworks API endpoint is down

* test fix fireworks ai is having an active outage

* test_completion_cost_databricks

* dbrx fix test API currently not responding

* Update OpenAI Realtime handler to use the correct endpoint and include all query parameters. Adjusted error messages for missing API base and key. Updated health check URL construction to pass model as a query parameter.

* Enhance OpenAI Realtime handler tests to ensure model parameter inclusion in WebSocket URL. Added new tests to verify correct URL construction with model and additional parameters, preventing 'missing_model' errors. Updated existing tests for consistency.

* Remove debug print statements for API base and key in OpenAIRealtime handler to clean up the code.

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-08-14 16:24:14 -07:00