scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 79
25896 commits 4106 branches 1402 tags 4.2 GiB
Commit graph

3549 commits

Author SHA1 Message Date
Ishaan Jaff
e73d053de3
[Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115) 
* test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints

* fix: is_registered_pass_through_route

* docs fix
 2025-10-01 14:09:01 -07:00
Sameer Kankute
7ec7e5332c
Add generateContent cost tracking (#15014) 2025-10-01 09:03:45 -07:00
Krish Dholakia
1503435d91
Merge pull request #15029 from henryhwang/gemini-adapter-fixes 
feat(gemini): Add full support for native Gemini API translation
 2025-09-30 21:19:18 -07:00
Ishaan Jaff
73bfef1a1f
Revert "[Feature]: Replace HTTPException with ParallelRequestLimitError in pa…" (#15095) 
This reverts commit 71b9b58fa9.
 2025-09-30 21:17:04 -07:00
Ishaan Jaffer
395c32c38d test_azure_openai_assistants_e2e_operations_stream 2025-09-30 21:16:29 -07:00
Krish Dholakia
64083111d3
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Henry Wang
acc23b9757 fix issue from pr review 2025-10-01 11:57:10 +08:00
Henry H Wang
7c4439ba0a
Merge branch 'BerriAI:main' into gemini-adapter-fixes 2025-10-01 09:53:51 +08:00
Ishaan Jaffer
8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer
04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair Ali
fcfe856e10
Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Ishaan Jaff
0ca11eefde
[Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Henry Wang
4eee54b157 fix the test issue from the pr review 2025-10-01 09:08:25 +08:00
Ishaan Jaffer
f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
malags
68189d1c04
[Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Krish Dholakia
c475723c67
Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish Dholakia
5619d350b6
Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan Jaff
0476a33d9f
[Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan Jaff
69a464fc97
[Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
Copilot
71b9b58fa9
[Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Krrish Dholakia
0cd61a6a6a fix: simplify testing 2025-09-30 12:37:25 -07:00
Eddie Richter
929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
Jack Temple
7212116d8c test: add UI theme settings retrieval and update tests 2025-09-30 10:01:44 -05:00
Henry Wang
fcd539af33 fix the issue from the tests for pr review 2025-09-30 18:15:25 +08:00
Henry Wang
cce05ac2b4 fix test issues from pr review 2025-09-30 16:44:15 +08:00
Henry Wang
d838c96ffb fix test issues from pr review 2025-09-30 16:05:17 +08:00
Krish Dholakia
fd1a986596
Merge pull request #15013 from serializer/patch-1 
[bug]: Update request handling for original exceptions
 2025-09-29 21:59:26 -07:00
Krish Dholakia
79d071c25c
Merge pull request #15022 from BerriAI/litellm_gemini_tool_search 
Ignore type param for gemini tools
 2025-09-29 21:57:50 -07:00
Krish Dholakia
9a44fcdc32
Merge pull request #15050 from uc4w6c/fix/regression-mcp-protocol-version 
fix: resolve regression with duplicate Mcp-Protocol-Version header
 2025-09-29 21:53:00 -07:00
Ishaan Jaff
708c0bd78d
[Feat] Return Cost for Responses API Streaming requests (#15053) 
* test_basic_openai_responses_api_streaming

* _transform_chat_completion_usage_to_responses_usage

* ResponseAPIUsage.cost

* test fixes for anthropic cost with /responses

* fix mypy typng
 2025-09-29 19:47:04 -07:00
Ishaan Jaffer
f1578b49e2 vertex_httpx_mock_post 2025-09-29 18:25:54 -07:00
Copilot
f22fd4cddd
Fix: Add /v1/messages/count_tokens to Anthropic routes for non-admin user access (#15034) 
* Initial plan

* Fix: Add /v1/messages/count_tokens to Anthropic routes for user access

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-29 18:16:52 -07:00
Ishaan Jaff
ebf72f5eb9
[Fix] Parallel Request Limiter v3 - use well known redis cluster hashing algorithm (#15052) 
* test_keyslot_for_redis_cluster

* fix _is_redis_cluster

* Update litellm/proxy/hooks/parallel_request_limiter_v3.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-09-29 18:12:44 -07:00
Ishaan Jaff
f6d7683261
[Feat] LiteLLM Overhead metric tracking - Add support for tracking litellm overhead on cache hits (#15045) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* add CachingDetails

* fix update_response_metadata import

* add CachingDetails metrics

* add CachingDetails

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit
 2025-09-29 17:33:27 -07:00
Ishaan Jaffer
f303860881 Revert "test_keyslot_for_redis_cluster" 
This reverts commit 52de33787b.
 2025-09-29 16:42:15 -07:00
Ishaan Jaffer
52de33787b test_keyslot_for_redis_cluster 2025-09-29 16:41:32 -07:00
Alexsander Hamir
d4830e34e5
fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 
* fix: remove redundant deep copy

set_model_list already does the deep copy at the beginning of the call.

* fix: remove unused model_list arguments

The `model_list` parameter was being passed to classes that did not use it.

* fix: reduce per-request memory and time from O(N×M) to O(N)

No need to create a whole array for a simple look up.

* add: missing test

* fix: remove unused parameter
 2025-09-29 15:49:46 -07:00
Ishaan Jaffer
e0172b86e2 test_litellm_overhead_non_streaming 2025-09-29 15:48:32 -07:00
Yuta Saito
5359a0d6a6 fix: test 2025-09-30 07:25:32 +09:00
Ishaan Jaff
619577d4e8
[Feat] Add litellm overhead metric for VertexAI (#15040) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* Revert "add update_response_metadata for caching handler"

This reverts commit f2a891f2b4.
 2025-09-29 15:15:25 -07:00
Krrish Dholakia
9ed83d44e3 test: remove unnecessary test 2025-09-29 13:35:34 -07:00
Sameer Kankute
3ab1c31e4e
(Feat) Add cost tracking for Vertex AI Passthrough /predict endpoint (#15019) 
* Add cost tracking for passthrough for predict endpoint

* restore file
 2025-09-29 13:19:04 -07:00
Krrish Dholakia
bc6e6e7a28 fix(auth_checks.py): add auth checks to mcp server on call tools 2025-09-29 13:13:10 -07:00
Ishaan Jaff
4dc23e8079
Revert "feat: improve vertex AI/gemini api_base handling for proxy services (…" (#15042) 
This reverts commit ff2d19e4ca.
 2025-09-29 13:03:46 -07:00
Zero Clover
ff2d19e4ca
feat: improve vertex AI/gemini api_base handling for proxy services (#15039) 2025-09-29 12:12:32 -07:00
Shubham Pathak
3f312d0527 Added test 2025-09-29 21:43:31 +05:30
Sameerlite
58955a0348 Ignore type param for gemini tools 2025-09-29 16:32:21 +05:30
Henry Wang
99a884019b test(gemini): Add unit tests for Google GenAI adapter 
This commit adds a comprehensive suite of unit tests for the Google GenAI adapter to ensure compliance with the project's contribution guidelines.

The new tests cover four main areas:
- Request parameter translation
- Streaming response handling
- Router methods for Google GenAI
- Proxy endpoints for Google GenAI

Additionally, this commit includes minor formatting and linting fixes identified during development.
 2025-09-29 18:51:35 +08:00
Krish Dholakia
0d738b2899
Merge pull request #14986 from uc4w6c/fix/remove-servername-prefix-mcp_tools-tests 
Fix/remove servername prefix mcp tools tests
 2025-09-28 18:00:23 -07:00
Krish Dholakia
e7939b0521
Merge pull request #14983 from abhijitjavelin/main 
Feat: Add Javelin standalone guardrails integration for LiteLLM Proxy
 2025-09-28 17:59:07 -07:00