scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
24940 commits 3222 branches 1436 tags 4.7 GiB
Commit graph

3090 commits

Author SHA1 Message Date
Tim Elfrink
e1329b03c6 Add comprehensive tests for CompactifAI provider 
- Test basic and streaming completions with proper mocking
- Cover authentication, parameter handling, and error scenarios
- Test provider detection and async functionality
- Verify request headers and response transformation
- Follow LiteLLM testing patterns with respx/httpx mocking
- Ensure full compatibility with OpenAI-style responses
 2025-09-13 08:41:30 +02:00
Krish Dholakia
663dbc6080
Merge pull request #14477 from BerriAI/litellm_dev_09_11_2025_p2 
`/v1/messages` - don't send content block after message w/ finish reason + usage block + `/key/unblock` - support hashed tokens
 2025-09-12 19:51:44 -07:00
Ishaan Jaff
93af8fd6ba
[QA] E2E - Testing for bedrock batches api (#14525) 
* add bedrock/batch-anthropic.claude-3-5-sonnet-20240620-v1:0

* test_bedrock_batches_api

* fix

* fix import

* test_bedrock_batches_api
 2025-09-12 19:31:19 -07:00
Ishaan Jaff
075a089d82
[Feat] Bedrock Batches - Ensure correct transformation applied to incoming requests (#14522) 
* use is_batch_jsonl_file

* fix valid_content_type

* fix transform_create_file_request

* fix _transform_openai_jsonl_content_to_bedrock_jsonl_content

* test_transform_openai_jsonl_content_to_bedrock_jsonl_content

* fix mypy linting errors

* fix BEDROCK_BATCH_MODEL

* fix working sample

* fix comment

* fix model list

* fix: use with managed batches

* refactor
 2025-09-12 18:32:57 -07:00
Arseny Boykov
f4318bccd3
[Performance] Use _PROXY_MaxParallelRequestsHandler_v3 by default again (#14450) 
* Use _PROXY_MaxParallelRequestsHandler_v3 by default (#14352)

(cherry picked from commit f3fa45cf8fbd5f5cce2f45a7312776d5005fb08e)
(cherry picked from commit 5b680bb4a3)

* Use random api_key for parallel requests test

* Fix off-by-one error in parallel request rate limit

The rate limiter was incorrectly rejecting requests when the limit was met, but not exceeded. The check in `is_cache_list_over_limit` was `int(counter_value) + 1 > current_limit`, which caused the first request to be rejected if the limit was 1.

This commit removes the `+ 1`, changing the logic to `int(counter_value) > current_limit`. The check now correctly allows requests up to the specified parallel limit.

* Test actual parallel requests

* Ensure rate limiting works correctly for multiple users

* Add sequential rate-limit test

* Revert random key usage
 2025-09-12 17:33:55 -07:00
Ishaan Jaff
e87e50328e
[Feat] Bedrock Batches - Working e2e flow to upload file + create batch (#14518) 
* fix: bedrock batches transform

* fix: upload_url

* fixes for model name

* fix upload_url

* fix bedrock batch test

* test_mock_bedrock_file_url_mapping
 2025-09-12 15:37:09 -07:00
Ishaan Jaff
18372f9ebe
Revert "fix vertex ai file upload" (#14501) 2025-09-12 12:02:24 -07:00
Sameerlite
fa175e8d90
Fix gemini cli error (#14417) 
* Fix gemini cli error

* Added better handling

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-09-12 11:56:51 -07:00
Sameer Kankute
1a123b2cd5
Litellm gemini cli bug fix (#14451) 
* Fix gemini cli error

* Add reasoning request support

* Added better handling

* remove other PR code

* refactored code for better structure following

---------

Co-authored-by: sameer@berri.ai <sameer@berri.ai>
 2025-09-12 11:55:26 -07:00
Boopesh Shanmugam
8b338a4d8c
User Headers X LiteLLM Users Mapping feature (#14485) 
* Draft commit.

* user header mapping feature with backward compatibility with user_header_name field.

* user header mapping feature with backward compatibility with user_header_name field optimizations.

* Added unit tests.
 2025-09-12 11:49:37 -07:00
Krish Dholakia
f8036a25a2
Merge pull request #14455 from lmnr-ai/fix/async-logging-tasks-context 
propagate execution context into logging tasks
 2025-09-12 00:36:08 -07:00
Krish Dholakia
113d2a8c5a
Merge pull request #14459 from holzman/fix-provider-budget 
Fix provider budgets
 2025-09-12 00:34:52 -07:00
Krish Dholakia
0413a701a2
Merge pull request #14460 from Sameerlite/litellm_vertex_gcs_bucket_issue 
fix vertex ai file upload
 2025-09-12 00:04:21 -07:00
Ishaan Jaff
32d87c242b
[Fixes] Using Qwen API Tiered Pricing (#14479) 
* fix: use dashscope cost calc

* add qwen logo
 2025-09-11 20:07:41 -07:00
Ishaan Jaff
69ef062f55 fix tiered_pricing test 2025-09-11 19:56:44 -07:00
Ishaan Jaff
51d5255452
[Bug]: Azure OpenAI & AI Foundry Reject Image Generation Payload Due to extra_body Injection in LiteLLM v1.76.3 (#14475) 
* add request body azure img gen

* fix test_get_optional_params_image_gen_filters_empty_values

* test_azure_image_generation_request_body

* test_azure_image_generation_request_body
 2025-09-11 19:39:06 -07:00
Krrish Dholakia
0c8b311155 test: add unit testing for both flows on key unblock 2025-09-11 19:15:15 -07:00
Krrish Dholakia
805069c287 fix(adapters/streaming_iterator.py): Don't send content block after message delta block is sent 
Fixes https://github.com/BerriAI/litellm/issues/14315
 2025-09-11 18:52:02 -07:00
Ishaan Jaff
dda115cc6d
[Feat] Cost Tracking - Add support for Tiered Cost Tracking for Qwen API (Dashscope) (#14471) 
* add dashscope logo

* docs fix

* docs fix

* fix supports_batch_calling

* fix naming

* fix input_cost_per_audio_token

* use output_cost_per_reasoning_token

* add tiered_pricing in get_model_info

* test fixes

* fix cost calc

* ruff fix
 2025-09-11 18:14:39 -07:00
Burt Holzman
e9e548d797 Fix provider budgets 2025-09-11 12:08:43 -05:00
Sameer Kankute
090e0fddf4 fix vertex ai file upload 2025-09-11 22:20:54 +05:30
Din
ee5a9d0aa0 propagate execution context into logging tasks 2025-09-11 15:54:40 +01:00
Ishaan Jaff
258b674dbb fix deepinfra test 2025-09-10 19:39:23 -07:00
Ishaan Jaff
a13aa4740a
[Fixes] Bug fixes to using LiteLLM MCP Gateway (#14392) 
* fix: use _get_mcp_servers_in_path

* fix checks for using litellm_proxy as MCP tool provider

* fix: fix mcp_tools_with_litellm_proxy

* fix: fix aresponses_api_with_mcp

* aresponses_api_with_mcp

* test_mcp_allowed_tools_filtering

* fix: _filter_mcp_tools_by_allowed_tools

* fix: _filter_mcp_tools_by_allowed_tools

* test_streaming_responses_api_with_mcp_tools

* fixes: test tools transfrom MCP->OpenaI spec

* test_streaming_responses_api_with_mcp_tools

* fix: chat ui allow multi select with allowed tools

* fix: use correct MCP events with litellm proxy response API

* fix get_event_model_class

* fix litellm proxy MCP handler

* fix MCPEnhancedStreamingIterator

* chat ui show list tools result

* UI: show MCP events

* fix stream iterator

* fixes: litellm proxy mcp handler

* test responses + mcp

* fix: update responses api with mcp handling

* ruff check fix

* central: _process_mcp_tools_to_openai_format

* fix: refactor code

* test_mcp_allowed_tools_filtering

* test mcp with litellm proxy

* fix mcp call

* demo: video using MCP ui

* fixes for using stream iterator

* test_no_duplicate_mcp_tools_in_streaming_e2e

* docs fix

* fix code snippet
 2025-09-10 19:12:11 -07:00
Ishaan Jaff
1f42e41c8d
[Bug]: Fix Authorization header not being sent to configured MCP servers (#14422) 
* test: test_mcp_server_config_auth_value_header_used

* fix: authentication_token

* docs: fix instructions on using responses api with MCPs

* mcp fixes
 2025-09-10 16:41:08 -07:00
Ishaan Jaff
dc5650eeda
Revert "fix: remove anthropic-beta header for Vertex AI requests with prompt caching" (#14421) 2025-09-10 15:47:18 -07:00
Krish Dholakia
a80cfe5fb5
Merge branch 'main' into feature/databricks-function-call-missing-pass-description 2025-09-09 22:42:32 -07:00
Krish Dholakia
53bf023876
Merge pull request #14092 from gotsysdba/main 
OCI Provder: Update OCIPromptTokensDetails
 2025-09-09 22:39:16 -07:00
Krish Dholakia
d72081113e
Merge pull request #14111 from dharamendrak/feature/aiohttp-dependency-injection 
feat: Add dependency injection support to BaseLLMAIOHTTPHandler
 2025-09-09 22:35:30 -07:00
Krish Dholakia
03a457842d
Merge pull request #14310 from swarna1101/fix-anthropic-prompt-caching-vertex-ai 
fix: remove anthropic-beta header for Vertex AI requests with prompt caching
 2025-09-09 22:32:40 -07:00
Krrish Dholakia
a504c7dae3 test: update tests 2025-09-09 21:43:37 -07:00
Krrish Dholakia
bdd7255bab test: remove redundant tests - moved to parallel_request_limiter_v3.py 2025-09-09 21:35:50 -07:00
Krrish Dholakia
c45ede7187 test: update test 2025-09-09 21:31:34 -07:00
Krrish Dholakia
f0de7d1dfd fix: remove EOL model name 2025-09-09 21:15:12 -07:00
Krrish Dholakia
d05f58721e test: remove end of life model from tests 2025-09-09 21:01:45 -07:00
Krrish Dholakia
e443d01925 test: remove redundant test 2025-09-09 20:37:09 -07:00
Krrish Dholakia
bff76715a5 test: skip test with invalid arn 2025-09-09 20:35:44 -07:00
Krrish Dholakia
07a83056a6 test: update test 2025-09-09 20:30:50 -07:00
Krrish Dholakia
44566977f1 test: update test 2025-09-09 20:18:03 -07:00
Krrish Dholakia
0854c35d3e test: remove eol bedrock model from tests 2025-09-09 19:48:35 -07:00
Ishaan Jaff
555efd60b5
[Security] Fix: Ensure .info() logs are not used for request/responses + Add code QA check for possible violations (#14386) 
* test: info log

* fixes for debug

* add info_log_check

* fix: debug level

* fix: debug level

* fix: use .debug

* fix info statements

* fix info log check

* fix: fix emit_standard_logging_payload

* fix RequestBody

* Potential fix for code scanning alert no. 3339: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix check

* fix code qa check

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-09-09 13:55:56 -07:00
pinkgu
285b14bd69
Fix/gcp model armor (#14381) 
* chore: fix model armor always success

Signed-off-by: bjorn <bjornjee95@gmail.com>

* chore: fix sanitisation field extraction

Signed-off-by: bjorn <bjornjee95@gmail.com>

---------

Signed-off-by: bjorn <bjornjee95@gmail.com>
 2025-09-09 09:18:37 -07:00
swarnabhasinha
f8ddd123d1
fix: add missing patch import in vertex_ai test 2025-09-09 16:16:59 +05:30
swarnabhasinha
7c5ec380c1
fix: remove anthropic-beta header for Vertex AI requests with prompt caching 2025-09-09 15:57:53 +05:30
Krish Dholakia
351896cd1d
Merge pull request #12414 from dotmobo/feature/fix-timestamp-granularities 
The parameter timestamp_granularities is broken for openai-like transcription
 2025-09-08 23:13:13 -07:00
Krish Dholakia
b9ce3a1587
Merge pull request #12416 from dotmobo/feature/fix-alloy 
feat: add a health_check_voice parameter in model_info
 2025-09-08 23:12:48 -07:00
Krish Dholakia
8c5dd753cf
Merge pull request #12651 from ocraviotto/fix-gemini-400-when-no-user-msg 
fix: ensure Gemini calls include a user message (fixes #9733)
 2025-09-08 23:08:32 -07:00
Krish Dholakia
a92b521ad4
Merge branch 'main' into propagate_metadata_to_vertexai_labels 2025-09-08 19:08:22 -07:00
Krish Dholakia
54c3ae15c8
Merge pull request #14312 from mubashir1osmani/main 
fix: spend tags endpoint returns 500
 2025-09-08 19:05:13 -07:00
Ishaan Jaff
87f116bded test fix 2025-09-08 18:46:20 -07:00