scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-18 04:44:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 125
40949 commits 3608 branches 1460 tags 6.7 GiB
Commit graph

10801 commits

Author SHA1 Message Date
Krish Dholakia
64083111d3
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Henry Wang
acc23b9757 fix issue from pr review 2025-10-01 11:57:10 +08:00
Henry H Wang
7c4439ba0a
Merge branch 'BerriAI:main' into gemini-adapter-fixes 2025-10-01 09:53:51 +08:00
Ishaan Jaffer
8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer
04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair Ali
fcfe856e10
Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Ishaan Jaff
0ca11eefde
[Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Henry Wang
4eee54b157 fix the test issue from the pr review 2025-10-01 09:08:25 +08:00
Ishaan Jaffer
f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
malags
68189d1c04
[Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Krish Dholakia
c475723c67
Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish Dholakia
5619d350b6
Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan Jaff
0476a33d9f
[Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan Jaff
69a464fc97
[Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
Copilot
71b9b58fa9
[Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Krrish Dholakia
0cd61a6a6a fix: simplify testing 2025-09-30 12:37:25 -07:00
Eddie Richter
929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
Jack Temple
7212116d8c test: add UI theme settings retrieval and update tests 2025-09-30 10:01:44 -05:00
Henry Wang
fcd539af33 fix the issue from the tests for pr review 2025-09-30 18:15:25 +08:00
Henry Wang
cce05ac2b4 fix test issues from pr review 2025-09-30 16:44:15 +08:00
Henry Wang
d838c96ffb fix test issues from pr review 2025-09-30 16:05:17 +08:00
Krish Dholakia
fd1a986596
Merge pull request #15013 from serializer/patch-1 
[bug]: Update request handling for original exceptions
 2025-09-29 21:59:26 -07:00
Krish Dholakia
79d071c25c
Merge pull request #15022 from BerriAI/litellm_gemini_tool_search 
Ignore type param for gemini tools
 2025-09-29 21:57:50 -07:00
Krish Dholakia
9a44fcdc32
Merge pull request #15050 from uc4w6c/fix/regression-mcp-protocol-version 
fix: resolve regression with duplicate Mcp-Protocol-Version header
 2025-09-29 21:53:00 -07:00
Ishaan Jaff
708c0bd78d
[Feat] Return Cost for Responses API Streaming requests (#15053) 
* test_basic_openai_responses_api_streaming

* _transform_chat_completion_usage_to_responses_usage

* ResponseAPIUsage.cost

* test fixes for anthropic cost with /responses

* fix mypy typng
 2025-09-29 19:47:04 -07:00
Ishaan Jaffer
f1578b49e2 vertex_httpx_mock_post 2025-09-29 18:25:54 -07:00
Copilot
f22fd4cddd
Fix: Add /v1/messages/count_tokens to Anthropic routes for non-admin user access (#15034) 
* Initial plan

* Fix: Add /v1/messages/count_tokens to Anthropic routes for user access

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-29 18:16:52 -07:00
Ishaan Jaff
ebf72f5eb9
[Fix] Parallel Request Limiter v3 - use well known redis cluster hashing algorithm (#15052) 
* test_keyslot_for_redis_cluster

* fix _is_redis_cluster

* Update litellm/proxy/hooks/parallel_request_limiter_v3.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-09-29 18:12:44 -07:00
Ishaan Jaff
f6d7683261
[Feat] LiteLLM Overhead metric tracking - Add support for tracking litellm overhead on cache hits (#15045) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* add CachingDetails

* fix update_response_metadata import

* add CachingDetails metrics

* add CachingDetails

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit
 2025-09-29 17:33:27 -07:00
Ishaan Jaffer
f303860881 Revert "test_keyslot_for_redis_cluster" 
This reverts commit 52de33787b.
 2025-09-29 16:42:15 -07:00
Ishaan Jaffer
52de33787b test_keyslot_for_redis_cluster 2025-09-29 16:41:32 -07:00
Alexsander Hamir
d4830e34e5
fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 
* fix: remove redundant deep copy

set_model_list already does the deep copy at the beginning of the call.

* fix: remove unused model_list arguments

The `model_list` parameter was being passed to classes that did not use it.

* fix: reduce per-request memory and time from O(N×M) to O(N)

No need to create a whole array for a simple look up.

* add: missing test

* fix: remove unused parameter
 2025-09-29 15:49:46 -07:00
Ishaan Jaffer
e0172b86e2 test_litellm_overhead_non_streaming 2025-09-29 15:48:32 -07:00
Yuta Saito
5359a0d6a6 fix: test 2025-09-30 07:25:32 +09:00
Ishaan Jaff
619577d4e8
[Feat] Add litellm overhead metric for VertexAI (#15040) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* Revert "add update_response_metadata for caching handler"

This reverts commit f2a891f2b4.
 2025-09-29 15:15:25 -07:00
Krrish Dholakia
9ed83d44e3 test: remove unnecessary test 2025-09-29 13:35:34 -07:00
Sameer Kankute
3ab1c31e4e
(Feat) Add cost tracking for Vertex AI Passthrough /predict endpoint (#15019) 
* Add cost tracking for passthrough for predict endpoint

* restore file
 2025-09-29 13:19:04 -07:00
Krrish Dholakia
bc6e6e7a28 fix(auth_checks.py): add auth checks to mcp server on call tools 2025-09-29 13:13:10 -07:00
Ishaan Jaff
4dc23e8079
Revert "feat: improve vertex AI/gemini api_base handling for proxy services (…" (#15042) 
This reverts commit ff2d19e4ca.
 2025-09-29 13:03:46 -07:00
Zero Clover
ff2d19e4ca
feat: improve vertex AI/gemini api_base handling for proxy services (#15039) 2025-09-29 12:12:32 -07:00
Shubham Pathak
3f312d0527 Added test 2025-09-29 21:43:31 +05:30
Sameerlite
58955a0348 Ignore type param for gemini tools 2025-09-29 16:32:21 +05:30
Henry Wang
99a884019b test(gemini): Add unit tests for Google GenAI adapter 
This commit adds a comprehensive suite of unit tests for the Google GenAI adapter to ensure compliance with the project's contribution guidelines.

The new tests cover four main areas:
- Request parameter translation
- Streaming response handling
- Router methods for Google GenAI
- Proxy endpoints for Google GenAI

Additionally, this commit includes minor formatting and linting fixes identified during development.
 2025-09-29 18:51:35 +08:00
anthony-liner
9ca73d5504 fix: set usage_details.total in langfuse integration 2025-09-29 16:30:44 +09:00
Krish Dholakia
0d738b2899
Merge pull request #14986 from uc4w6c/fix/remove-servername-prefix-mcp_tools-tests 
Fix/remove servername prefix mcp tools tests
 2025-09-28 18:00:23 -07:00
Krish Dholakia
e7939b0521
Merge pull request #14983 from abhijitjavelin/main 
Feat: Add Javelin standalone guardrails integration for LiteLLM Proxy
 2025-09-28 17:59:07 -07:00
Krrish Dholakia
f32b0364c4 fix: fix tests 2025-09-28 17:55:46 -07:00
Yuta Saito
be5010a675 test: fix tests 2025-09-29 07:21:32 +09:00
Yuta Saito
dae7d08ff2 Revert "Revert "Merge pull request #14720 from uc4w6c/feat/remove-servername-prefix-mcp_tools"" 
This reverts commit a88d774f94.
 2025-09-29 07:12:24 +09:00
TobiMayr
3c99d2236a feature/add max requests env var 2025-09-28 19:18:56 +01:00