Commit graph

14665 commits

Author SHA1 Message Date
Krrish Dholakia
f7f1a0d0b7 test: add unit test 2025-08-18 22:32:36 -07:00
Krrish Dholakia
2e16f2cb13 test: add unit tests 2025-08-18 21:19:43 -07:00
Krish Dholakia
422447b7f1
Responses API - add default api version for openai responses api calls + Openrouter - fix claude-sonnet-4 on openrouter + Azure - Handle openai/v1/responses
Responses API - add default api version for openai responses api calls + Openrouter - fix claude-sonnet-4 on openrouter + Azure - Handle `openai/v1/responses`
2025-08-18 18:59:28 -07:00
Krish Dholakia
3b52545db3
Merge pull request #13529 from BerriAI/litellm_dev_08_11_2025_p1
[Fix] Cooldowns - don't return raw Azure Exceptions to client
2025-08-18 18:54:19 -07:00
Ishaan Jaff
ba0881d728
[Bug Fix] image_edit() function returns APIConnectionError with litellm_proxy - Support for both image edits and image generations (#13735)
* add image edits litellm proxy on SDK

* add image gen provider

* add IMG Gen support for litellm_proxy provider
2025-08-18 18:26:32 -07:00
Ishaan Jaff
76f1064229
[Bug Fix] litellm incompatible with newest release of openAI v1.100.0 (#13728)
* fix imports OpenAI SDK

* ResponseText fixes

* fixes ResponseText

* fix imports

* catch AttributeError

* fix import

* use openai==1.100.1

* fix build from PIP

* fix lint test

* Print OpenAI version

* fix Install dependencies
2025-08-18 18:26:17 -07:00
Ishaan Jaff
ba1d2e8749
[Feat] DD LLM Observability - Add time to first token, litellm overhead, guardrail overhead latency metrics (#13734)
* fixes for DDLLMObsLatencyMetrics

* use _get_latency_metrics

* DD LLM Obs - track latency metrics

* fixes for bedrock guardrails

* DD unit tests

* test DD
2025-08-18 17:38:04 -07:00
Ishaan Jaff
ef08e18c66
[Feat] Datadog LLM Observability - Add support for Failure Logging (#13726)
* add async_log_failure_event for DD LLM Obs

* update types

* DataDogLLMObsLogger  add failure logging support

* test_async_log_failure_event

* dd test failure
2025-08-18 15:19:48 -07:00
Ryan Means
a05330fcd8 Fix unit tests 2025-08-18 12:00:03 -07:00
Thiago Salvatore
169a17400f
fix(vertexai-batch): fix vertexai batch file format (#13576)
* fix(access group): allow access group on mcp tool retrieval

* fix(test): fix broken tests and add test case for access group

* fix(mypy): fix typing issues

* fix(memory file): add content type to in memory file
2025-08-18 10:19:23 -07:00
Michal Otmianowski
405e74ec16 use namespace as prefix in s3 2025-08-18 13:43:51 +02:00
Tim Elfrink
0febdf8c1c feat: add thinking and reasoning parameter support for GitHub Copilot provider
- Add dynamic parameter support for anthropic models through GitHub Copilot
- Include thinking parameter for anthropic model compatibility
- Support reasoning_effort parameter for both anthropic and reasoning models
- Update test coverage for parameter validation logic
- Ensure proper parameter filtering based on model type
2025-08-17 17:51:58 +02:00
Tim Elfrink
1e81a1bd7c feat: Add thinking and reasoning_effort parameter support for GitHub Copilot provider
- Add github_copilot case to get_supported_openai_params function
- Implement get_supported_openai_params method in GithubCopilotConfig
- Dynamically add thinking and reasoning_effort params for Anthropic models
- Add comprehensive tests for parameter support validation
- Ensure case-insensitive model detection for parameter inclusion

Fixes UnsupportedParamsError when using advanced reasoning parameters
with Anthropic models through GitHub Copilot proxy.
2025-08-17 17:13:29 +02:00
Krrish Dholakia
6322aef0e3 fix(streaming_handler.py): fix streaming chunk calculation 2025-08-16 14:25:29 -07:00
Krrish Dholakia
eb66daeef7 test: update test
we now return correct token usage on clientside
2025-08-16 14:14:35 -07:00
Ishaan Jaff
d96df5e9be Revert "test_stream_token_counting_anthropic_with_include_usage"
This reverts commit c3aee1194b.
2025-08-16 13:07:00 -07:00
Ishaan Jaff
c3aee1194b test_stream_token_counting_anthropic_with_include_usage 2025-08-16 13:06:31 -07:00
Krrish Dholakia
8bb5ee2ba5 test: update unit test 2025-08-16 12:39:03 -07:00
Krrish Dholakia
1ec0808713 test: update test - anthropic returns input tokens in final usage chunk now 2025-08-16 12:31:52 -07:00
Ishaan Jaff
f522f40228 test_passing_tool_result_as_list 2025-08-16 08:08:25 -07:00
Cole McIntosh
000ecad4e2 Fix Groq streaming ASCII encoding issue
Replace iter_lines()/aiter_lines() with iter_text()/aiter_text() using explicit
UTF-8 encoding to handle non-ASCII characters like µ in streaming responses.

- Added utf8_iter_lines() and utf8_aiter_lines() helper functions
- Ensures proper UTF-8 decoding of streaming response content
- Added comprehensive tests for Unicode character handling

Fixes #12660
2025-08-16 08:32:22 -05:00
Krrish Dholakia
5b641380ab fix(openai/responses/transformation.py): update supported openai params 2025-08-16 01:49:54 -07:00
Krrish Dholakia
b5c4ee60bb fix(openai/image_edits): Support 'mask' parameter for openai image edits
Closes https://github.com/BerriAI/litellm/issues/13528
2025-08-16 01:40:24 -07:00
Krish Dholakia
1b2ec16eee
Merge pull request #13640 from yytdfc/fix_bedrock_epc
[Bug Fix] Add cachePoint support for assistant and tool messages in Bedrock
2025-08-16 01:20:28 -07:00
Krrish Dholakia
675d73fb9b fix(mistral/chat/transformation.py): Support new mistral thinking block
Closes https://github.com/BerriAI/litellm/issues/13416
2025-08-16 01:16:01 -07:00
Krrish Dholakia
ff7bdb6290 fix(mistral/chat/transformation.py): handle empty message content for mistral calls
Fixes https://github.com/BerriAI/litellm/issues/13355
2025-08-16 00:41:11 -07:00
Krrish Dholakia
b83b1686c2 feat(support-allowed_openai_params-for-responses-api): Fixes https://github.com/BerriAI/litellm/issues/13559 2025-08-16 00:23:37 -07:00
Ishaan Jaff
40395598da
[Feat] UI - Allow editing team member rpm/tpm limits (#13669)
* show team member tpm/rpm limits

* ui - allow setting team settings

* fix better debugging

* fix types: TeamMemberUpdateRequest

* add _upsert_budget_and_membership

* allow updating team member RPM/TPM in teamMemberUpdateCall

* editing team member rpm/tpm

* UI - fixes for team member component

* fix info

* test_upsert_rpm_only_creates_new_budget
2025-08-15 17:29:44 -07:00
0x-fang
9f17bedb48
bump up boto3 and aioboto3 versions (#13665) 2025-08-15 17:06:08 -07:00
Ishaan Jaff
f5a1c14929 Revert "update boto3 version to 1.37.38 (#13656)"
This reverts commit 6f2264a13d.
2025-08-15 11:24:59 -07:00
Daniel Barker
47edecd5bc
Fixed incorrect key info endpoint (#13633) 2025-08-15 11:10:06 -07:00
0x-fang
6f2264a13d
update boto3 version to 1.37.38 (#13656) 2025-08-15 11:06:54 -07:00
Ishaan Jaff
c3608adb0f
[Bug Fix] Using stream=True + background=True with Responses API (#13654)
* test_update_responses_api_response_id_with_model_id_handles_dict

* fix linting
2025-08-15 08:58:28 -07:00
FuChen
511363d3a7 test: Add comprehensive test cases for cachePoint support
- test_assistant_message_cache_control: Tests assistant messages with string content and cache_control
- test_assistant_message_list_content_cache_control: Tests assistant messages with list content and cache_control
- test_tool_message_cache_control: Tests tool messages with list content and cache_control
- test_tool_message_string_content_cache_control: Tests tool messages with string content and cache_control
- test_assistant_tool_calls_cache_control: Tests assistant tool_calls with cache_control
- test_multiple_tool_calls_with_mixed_cache_control: Tests multiple tool calls with mixed cache_control
- test_no_cache_control_no_cache_point: Tests that messages without cache_control don't generate cachePoint blocks

These tests ensure that cachePoint blocks are correctly generated for all message types when cache_control is present.
2025-08-15 11:56:34 +08:00
Ishaan Jaff
48c89812c4
[Bug Fix] /messages endpoint - ensure tool use arguments are returned for non-anthropic models (#13638)
* bug fix _translate_streaming_openai_chunk_to_anthropic

* test  test_translate_streaming_openai_chunk_to_anthropic_with_partial_json
2025-08-14 17:29:42 -07:00
Ishaan Jaff
b78495d398
[Fix] Ensure /messages works when using `bedrock/converse/<model> with LiteLLM (#13627)
* get_bedrock_provider_config_for_messages_api

* fixes for get_bedrock_provider_config_for_messages_api

* test_anthropic_messages_litellm_router_bedrock

* fix merge conflicts

* fix - refactor based on jugal's comment
2025-08-14 16:50:05 -07:00
Jugal D. Bhatt
b8fe5f7b17
[MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e390.
2025-08-14 16:32:18 -07:00
Jugal D. Bhatt
aea0605eed
[LLM Translation] Fix Realtime API endpoint for no intent (#13476)
* fix intent params

* Add responses

* fix unrelated test

* test fix - fireworks API endpoint is down

* test fix fireworks ai is having an active outage

* test_completion_cost_databricks

* dbrx fix test API currently not responding

* Update OpenAI Realtime handler to use the correct endpoint and include all query parameters. Adjusted error messages for missing API base and key. Updated health check URL construction to pass model as a query parameter.

* Enhance OpenAI Realtime handler tests to ensure model parameter inclusion in WebSocket URL. Added new tests to verify correct URL construction with model and additional parameters, preventing 'missing_model' errors. Updated existing tests for consistency.

* Remove debug print statements for API base and key in OpenAIRealtime handler to clean up the code.

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-08-14 16:24:14 -07:00
Krish Dholakia
ab8bd70c5e
Merge pull request #13635 from BerriAI/litellm_mock_test_fix
[Proxy] LiteLLM mock test fix
2025-08-14 15:47:41 -07:00
TomuHirata
f6e53deacd
Update mlflow logger usage span attributes (#13561)
* test: sync mlflow request tags

* fix test
2025-08-14 15:20:50 -07:00
Jugal Bhatt
f53599825d Merge branch 'main' into litellm_mock_test_fix 2025-08-14 15:17:31 -07:00
Jugal Bhatt
45f188b041 Add mock user API key authentication in tag management tests
This update integrates mock user API key authentication into the tag management endpoint tests, ensuring accurate simulation of user roles for creating, updating, and deleting tags. The changes enhance the reliability of the tests by properly setting up user authentication before executing test cases.
2025-08-14 15:03:21 -07:00
Krrish Dholakia
aaf9c38a10 test: skip test - ran out of credits 2025-08-14 15:01:26 -07:00
Jugal Bhatt
d21f467264 Revert "Add mock user API key authentication in tag management tests"
This reverts commit 5fc0803b94.
2025-08-14 14:58:49 -07:00
Jugal Bhatt
5fc0803b94 Add mock user API key authentication in tag management tests
This update introduces a helper function to create a mock user API key authentication object, which is utilized in the tag management endpoint tests. The mock authentication is integrated into the test cases for creating, updating, and deleting tags, enhancing the reliability of the tests by simulating user roles accurately.
2025-08-14 14:40:49 -07:00
Jugal Bhatt
a6e55c0447 Revert "Revert "Fix - add safe divide by 0 for most places to prevent crash""
This reverts commit 5ad698f2cc.
2025-08-14 14:23:06 -07:00
Jugal Bhatt
bfb0a3854e Enhance logging in cost calculation tests to ensure DEBUG level captures are accurate. Updated tests to set logger level before assertions and restored original logger level after execution. This improves reliability of log level checks in both cost and batch cost calculation tests. 2025-08-14 14:21:22 -07:00
Jugal Bhatt
5ad698f2cc Revert "Fix - add safe divide by 0 for most places to prevent crash"
This reverts commit 265d40e390.
2025-08-14 14:21:07 -07:00
Jugal D. Bhatt
3a38912bce
[Proxy] Litellm fix mapped tests (#13634)
* Fix - add safe divide by 0 for most places to prevent crash

* mock prisma client

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e390.
2025-08-14 14:19:36 -07:00
Krish Dholakia
f360e0ead2
Merge pull request #13590 from BerriAI/litellm_bedrock_api_header
[LLM translation] Refactor Anthropic Configurations and Add Support for `anthropic_beta` Headers
2025-08-14 11:32:18 -07:00