scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 115
25295 commits 3285 branches 1443 tags 4.8 GiB
Commit graph

5590 commits

Author SHA1 Message Date
Ishaan Jaffer
c656982f18 test fix 2025-09-20 16:55:23 -07:00
Ishaan Jaffer
4bafbc5c08 ui new build 2025-09-20 16:27:21 -07:00
Krrish Dholakia
204c208d24 fix(model_connection_test.tsx): pass modelinfo to /health/test_connection 
ensures team admins can test new models
 2025-09-20 12:33:28 -07:00
Ishaan Jaffer
3c2c14ffe4 test fix 2025-09-20 11:33:08 -07:00
Krrish Dholakia
5164c484d9 build(ui/): new ui build 2025-09-20 10:17:37 -07:00
Ishaan Jaffer
7d91df4dbe test fix _get_mcp_servers_in_path 2025-09-20 10:09:59 -07:00
Krish Dholakia
1f9afcb349
Merge pull request #14438 from hakasecurity/change-aim-headers 
rename aim headers + tests
 2025-09-19 23:32:50 -07:00
Krish Dholakia
270d612029
Merge branch 'main' into litellm_dev_09_10_2025_p1 2025-09-19 22:01:57 -07:00
Krish Dholakia
6142c3ac3d
Merge pull request #14738 from BerriAI/litellm_dev_09_19_2025_p1 
UI SSO - consider token info endpoint on generic SSO route for access control groups
 2025-09-19 17:58:37 -07:00
Krrish Dholakia
7694e9f6a9 test: refactoring + testing 2025-09-19 17:20:15 -07:00
Krrish Dholakia
75b98a7909 fix(ui_sso.py): initial commit, adding checking token endpoint response for allowed team ids 
Allows ui access control to work for the given team ids
 2025-09-19 17:02:49 -07:00
Ishaan Jaff
c918e18852
Enrich rate limit error message with specific limit type and reset time (#14736) 
- Add specific rate limit type (requests/tokens/max_parallel_requests) to error message
- Include current limit value for better context
- Display reset time in human-readable format
- Handle negative remaining values gracefully by showing 0 instead
- Add reset_at header with timestamp for programmatic use

Fixes issue where rate limit errors were ambiguous about which type of limit
was exceeded and when the limit would reset.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-09-19 16:32:13 -07:00
Ishaan Jaff
90ee9e4587
[Feat] Dynamic Rate Limiter v3 - fixes to ensure priority routing works as expected (#14734) 
* fix: dynamic limiter v3

* fix: dynamic limiter v3

* feat: add dynamic limiter v3

* feat: add dynamic limiter v3

* feat: add dynamic limiter v3 in init litellm_logging

* feat: add dynamic limiter v3 in init litellm_logging

* fix: priority rate limiting

* Potential fix for code scanning alert no. 3397: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: priority rate limiting

* fix: ruff

* fix: mypy lint

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-09-19 16:04:45 -07:00
Krish Dholakia
ad6ba8f5c5
Merge pull request #14695 from uc4w6c/fix/mcp-gateway-tools-list 
Fix/mcp gateway tools list
 2025-09-18 23:40:14 -07:00
Krish Dholakia
d5a839d971
Merge pull request #14700 from BerriAI/litellm_contributor_prs_09_18_2025_p2 
Update Bedrock documentation for Titan V2 encoding_format support + Anthropic - account for 1h vs. 5m cache creation token cost difference + UI - add langsmith_sampling_rate as a dynamic param
 2025-09-18 23:38:29 -07:00
Krish Dholakia
664c83cfb5
Merge branch 'litellm_contributor_prs_09_18_2025_p2' into litellm_dev_09_17_2025_p2_v2 2025-09-18 19:50:55 -07:00
=
5b080e20c4 removes HTTP exception for faulty guardrail 2025-09-18 18:57:15 -07:00
Krish Dholakia
f387803655
Merge pull request #14658 from ARajan1084/bedrock-custom-guardrail-fix 
fix: check for AWS exceptions despite a 200 response
 2025-09-18 18:34:32 -07:00
Krish Dholakia
3671c6a97f
Merge pull request #14690 from ARajan1084/bedrock_guardrail_logging_fix 
fix: Amazon Bedrock incorrect guardrailResponse bug
 2025-09-18 18:33:23 -07:00
Krish Dholakia
661a42b626
Merge pull request #14698 from BerriAI/litellm_contributor_prs_09_18_2025_p1 
Litellm contributor prs 09 18 2025 p1
 2025-09-18 18:32:51 -07:00
=
4f468e9563 Update bedrock_guardrails.py 2025-09-18 17:27:40 -07:00
=
12fed102ba Update bedrock_guardrails.py 2025-09-18 17:25:31 -07:00
Alexsander Hamir
59409429d4
fix: reduced __inits__ overhead in 7% (#14689) 
* fix: avoid redundant __init__ calls on hot path

Previously, imports on the request hot path caused __init__ to run
excessively for every request. This change ensures initialization
happens once, reducing cpu overhead.

* fix: remove redundant __init__ import

The current implementation no longer requires an import at the top of the function.

* fix: placed on core utils for future reuse

* test: add coverage & remove inline import

A general import-checking tool across all endpoints would be a large PR.
This commit focuses on a smaller, targeted fix for the discussed case.

* added import check to CI
 2025-09-18 17:18:05 -07:00
=
77e4008e7b lint fix 2025-09-18 17:13:55 -07:00
=
2bbcf5a851 Update bedrock_guardrails.py 2025-09-18 16:05:30 -07:00
=
911918474a removed duplicate code 2025-09-18 15:49:16 -07:00
=
a86b9a1808 check for AWS exceptions despite a 200 response 2025-09-18 15:42:36 -07:00
=
78cb8c71fd prelim changes for new Bedrock guardrail info view 2025-09-18 15:09:10 -07:00
Yuta Saito
654f1d3290 fix: stop including spec_version in MCP server registration inserts 2025-09-19 07:06:15 +09:00
Yuta Saito
6c291093e9 fix: remove adding Mcp-Protocol-Version header (#14069) 
The Mcp-Protocol-Version header is already handled in the MCP Python SDK, so the explicit addition on LiteLLM Proxy was redundant.
 2025-09-19 07:05:20 +09:00
=
c1963f7f02 removed wrapper which overwrote the guardrailResponse value incorrectly 2025-09-18 12:05:38 -07:00
Krish Dholakia
bfaab8ad7e
Merge pull request #14557 from timelfrink/fix/issue-14478-bedrock-count-tokens-endpoint 
Implement AWS Bedrock CountTokens API support
 2025-09-17 23:51:06 -07:00
Krish Dholakia
ff36dfdc76
Merge pull request #14637 from akraines/feature/middle-truncate-spend-logs 
feat: implement middle-truncation for spend log payloads
 2025-09-17 23:47:04 -07:00
Tim Elfrink
c234b13275 Apply code formatting and linting fixes 
- Apply Black formatting to all Bedrock CountTokens files
- Clean up imports and remove unused variables in tests
- Fix indentation and simplify test structure
- Fix pyright type error with type ignore annotation
- All tests continue to pass after cleanup
 2025-09-18 08:28:17 +02:00
Krrish Dholakia
c620d76fe4 fix(team_callback_endpoints.py): fix adding callbacks to teams 
Resolves error caused by the migration to a standard 'logging' field in metadata
 2025-09-17 19:01:34 -07:00
Krrish Dholakia
83522016f2 feat(langsmith.py): add per request sampling_rate support 
allows setting langsmith sampling rate per team/per key

Closes LIT-879
 2025-09-17 18:39:34 -07:00
Krrish Dholakia
e5c1d09937 feat(langsmith.py): add langsmith sampling rate 
Closes LIT-879
 2025-09-17 18:02:33 -07:00
=
36e01b7881 updates in memory custom guardrail liteLLM params 2025-09-17 18:00:59 -07:00
Mubashir Osmani
dc267e9032 fix: ci/cd tests + lint errors (#14646) 
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-09-17 18:00:59 -07:00
Krrish Dholakia
32c6019ecc fix(_health_endpoints.py): protect /health/test_connection - only allow users who are allowed to create models, to call this endpoint 
Closes LIT-989
 2025-09-17 18:00:59 -07:00
Krrish Dholakia
1598d3e955 fix(bedrock_guardrails.py): respect bedrock runtime endpoint when using guardrails 
Closes LIT-983
 2025-09-17 18:00:58 -07:00
Krish Dholakia
44e0c730b9
Merge pull request #14653 from ARajan1084/in-memory-guardrail-fix 
fix: In Memory Guardrail fails to update
 2025-09-17 17:42:23 -07:00
Mubashir Osmani
8b804303ed
fix: ci/cd tests + lint errors (#14646) 
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-09-17 17:06:43 -07:00
Krish Dholakia
f51003538a
Merge pull request #14650 from BerriAI/litellm_dev_09_17_2025_p1 
Bedrock Guardrails - support setting bedrock runtime endpoint + Protect `/health/test_connect` to prevent users without model creation permissions from calling it
 2025-09-17 16:55:53 -07:00
=
66202d0ab7 updates in memory custom guardrail liteLLM params 2025-09-17 16:28:05 -07:00
Krish Dholakia
895c41efa3
Merge pull request #14619 from BerriAI/litellm_dev_09_16_2025_p1 
UI - allow team member to view service account keys they create + Anthropic - include cache creation tokens in prompt token total (separate out during cost tracking)
 2025-09-17 15:43:04 -07:00
Krrish Dholakia
d96e397de5 fix(_health_endpoints.py): protect /health/test_connection - only allow users who are allowed to create models, to call this endpoint 
Closes LIT-989
 2025-09-17 15:28:35 -07:00
Krrish Dholakia
1ada663959 fix(bedrock_guardrails.py): respect bedrock runtime endpoint when using guardrails 
Closes LIT-983
 2025-09-17 14:41:40 -07:00
Akiva Kraines
115a3e9ded feat: implement middle-truncation for spend log payloads 
- Change truncation strategy from head-only to middle-truncation (35% start, 65% end)
- Preserve both beginning and end of long strings for better debugging context
- Apply same sanitization to response payloads when store_prompts_in_spend_logs is enabled
- Increase default MAX_STRING_LENGTH_PROMPT_IN_DB from 1000 to 2048 characters
- Update tests to verify new truncation behavior with 35%-65% split

This provides better diagnostic value by keeping the more important end context
while still maintaining storage limits.
 2025-09-17 16:30:25 +03:00
Krish Dholakia
fcf84027e8
Merge pull request #14555 from mubashir1osmani/dd_spend_metric 
DataDog shows spend metrics
 2025-09-16 22:44:47 -07:00