scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78
25907 commits 4106 branches 1402 tags 4.2 GiB
Commit graph

25907 commits

This branch
This branch
All branches
Author SHA1 Message Date
Krrish Dholakia
83facb761f fix: fix placement 2025-10-01 18:57:27 -07:00
Krrish Dholakia
6589c924c8 fix: add linting 2025-10-01 18:54:50 -07:00
Krrish Dholakia
34366d8fe2 docs(key_management_endpoints.py): document new fields 2025-10-01 18:50:22 -07:00
Krrish Dholakia
32036e2093 feat(ui/): add common rate limit type form, for reuse for tpm/rpm across create + edit key 2025-10-01 18:46:43 -07:00
Krrish Dholakia
20b6f011f7 fix(create_key_button.tsx): working ui controls to set guaranteed throughput/best effort throughput on key 2025-10-01 18:34:29 -07:00
Krrish Dholakia
ff866aae7b feat(create_key_button.tsx): add tpm/rpm rate limit type options to UI 
allows user to set the type of tpm/rpm limit they're trying to set
 2025-10-01 18:30:34 -07:00
Krrish Dholakia
d8e3a62fbe feat(key_management_endpoints.py): add support for guaranteed throughput on key update and service account key creation 2025-10-01 17:58:31 -07:00
Krrish Dholakia
757436f302 test: add unit test 2025-10-01 17:51:10 -07:00
Krrish Dholakia
3ce074564e feat(key_management_endpoints.py): add guaranteed throughput support for model specific tpm / rpm limits 
prevents admin from granting keys more tpm/rpm than created for a key
 2025-10-01 17:48:23 -07:00
Krrish Dholakia
a83238a2db test: add unit tests 2025-10-01 17:02:05 -07:00
Krrish Dholakia
005aec69c7 feat(key_management_endpoints.py): allow specifying rate limit type when creating tpm/rpm limits on keys 
prevents overallocating tpm/rpm limits
 2025-10-01 16:57:07 -07:00
Ishaan Jaff
d9664a3ee4
fix gpt-5-chat-latest on model cost map (#15116) 2025-10-01 14:35:57 -07:00
Ishaan Jaff
e73d053de3
[Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115) 
* test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints

* fix: is_registered_pass_through_route

* docs fix
 2025-10-01 14:09:01 -07:00
Luiz Rennó Costa
7e56600896
fix: model_group not always present in litellm_params, and metadata reference location (#15108) 
Co-authored-by: Luiz Rennó Costa <luiz.renno@ifood.com.br>
 2025-10-01 11:39:49 -07:00
Patrick Lafleur
e7fd1fb96b
Fix missing HTTPException import (#15111) 2025-10-01 11:39:30 -07:00
Sameer Kankute
7ec7e5332c
Add generateContent cost tracking (#15014) 2025-10-01 09:03:45 -07:00
Ishaan Jaffer
ab00ca2de9 bump: version 1.77.6 → 1.77.7 2025-10-01 08:54:03 -07:00
Krrish Dholakia
7fd24a2632 bump: version 1.77.6 → 1.77.7 2025-09-30 21:23:42 -07:00
Krish Dholakia
1503435d91
Merge pull request #15029 from henryhwang/gemini-adapter-fixes 
feat(gemini): Add full support for native Gemini API translation
 2025-09-30 21:19:18 -07:00
Ishaan Jaff
73bfef1a1f
Revert "[Feature]: Replace HTTPException with ParallelRequestLimitError in pa…" (#15095) 
This reverts commit 71b9b58fa9.
 2025-09-30 21:17:04 -07:00
Ishaan Jaffer
395c32c38d test_azure_openai_assistants_e2e_operations_stream 2025-09-30 21:16:29 -07:00
Krish Dholakia
64083111d3
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Henry Wang
acc23b9757 fix issue from pr review 2025-10-01 11:57:10 +08:00
Henry H Wang
7c4439ba0a
Merge branch 'BerriAI:main' into gemini-adapter-fixes 2025-10-01 09:53:51 +08:00
Ishaan Jaffer
8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer
04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair Ali
fcfe856e10
Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Alexsander Hamir
26145da3e7
perf(router): optimize _filter_cooldown_deployments to O(n) (#15091) 
Refactored to use set-based lookup and list comprehension instead
of two-pass approach with list.remove().

Old complexity: O(n×m + k×n)
- First loop: n deployments × m list lookups = O(n×m)
- Second loop: k removals × n list.remove() scans = O(k×n)

New complexity: O(m + n)
- Convert to set: O(m)
- Filter with O(1) set lookups: O(n)

Example with 100 deployments, 5 in cooldown:
- Old: ~1000 operations
- New: ~105 operations

Called on every request - high impact for production.
 2025-09-30 18:39:12 -07:00
Ishaan Jaff
0ca11eefde
[Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Henry Wang
4eee54b157 fix the test issue from the pr review 2025-10-01 09:08:25 +08:00
Ishaan Jaffer
f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
Ishaan Jaffer
aac1129761 fix is_sensitive_key 2025-09-30 17:05:57 -07:00
malags
68189d1c04
[Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Ishaan Jaffer
f46f9d3fd9 docs azure passthrough api fixes 2025-09-30 15:54:09 -07:00
Krish Dholakia
c475723c67
Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish Dholakia
5619d350b6
Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan Jaff
0476a33d9f
[Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan Jaff
69a464fc97
[Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
Ishaan Jaffer
75d22d3d79 fix code qa check 2025-09-30 14:03:05 -07:00
Copilot
71b9b58fa9
[Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Alexsander Hamir
05dd104ce6
perf(router): Cache nested dict lookups in hot path (#15084) 
Cache deployment["litellm_params"] and deployment["model_info"]
at loop start to avoid repeated dict hash lookups.

- _pre_call_checks: 3 fewer lookups per deployment per request
- deployment_callback_on_failure: 1 fewer lookup per failure
- _set_model_group_info: 4 fewer lookups per model

Saves CPU cycles on every routing decision and failure callback.
 2025-09-30 13:36:42 -07:00
Alexsander Hamir
927e15996e
perf(router): Remove unnecessary hasattr checks in get_model_list() (#15082) 
Remove redundant hasattr() checks for model_list and model_group_alias
in get_model_list() method. Both attributes are always initialized in
__init__, making these runtime checks unnecessary overhead.

Changes:
- Remove hasattr(self, "model_list") check
- Remove hasattr(self, "model_group_alias") check
- Move model_group_alias initialization earlier in __init__ to ensure
  it's available when set_model_list() calls get_model_names()
- Simplify control flow by removing nested conditional blocks

Performance Impact:
- hasattr should not appear in the profile of a proxy server handling thousands of requests. This change ensures it no longer does.
 2025-09-30 13:35:27 -07:00
Ishaan Jaff
60230e5666
[Feat] UI - add snowflake on UI (#15083) 
* UI - add snowflake on UI

* fixes snowflake creds
 2025-09-30 13:16:04 -07:00
Ishaan Jaff
862736e74b
feat: add groq/moonshotai/kimi-k2-instruct-0905 (#15079) 2025-09-30 12:51:21 -07:00
Krrish Dholakia
0cd61a6a6a fix: simplify testing 2025-09-30 12:37:25 -07:00
Krish Dholakia
39d1d2a379
Merge pull request #14840 from eddierichter-amd/lemonade-integration 
Add AMD Lemonade provider support
 2025-09-30 12:34:30 -07:00
Ishaan Jaffer
45bfd2599c docs fix 2025-09-30 11:35:15 -07:00
Eddie Richter
7da05df534 Removing unecessary import 2025-09-30 12:12:24 -06:00
Eddie Richter
19e7070b73 Removing get_model_info from Lemonade provider. Implemented get_models which gets hooked into get_valid_models litellm utility. Also, added a simple cost calculator implementation for Lemonade so calling cost_calculator.completion_cost() doesn't return an error when a model is not found in the model_cost json. 2025-09-30 12:12:24 -06:00
Eddie Richter
1e1e4c36ac Fixing key name 2025-09-30 12:12:24 -06:00