scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-16 23:41:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
49744 commits 2457 branches 1410 tags 2.9 GiB
Commit graph

49744 commits

This branch
This branch
All branches
Author SHA1 Message Date
Yujong Lee
b70ddc2fd8 ci(rust): split rust jobs, use nextest and Swatinem/rust-cache 
Split the Rust workflow into fmt, clippy, nextest and wheel jobs so they run in parallel, replace manual actions/cache with Swatinem/rust-cache, and install a pinned checksum-verified cargo-nextest. Make two python-bridge tests self-contained so they pass when nextest runs each test in its own process.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:14:32 +00:00
yucheng-berri
b04d530ecf
Merge pull request #41386 from BerriAI/litellm_otel_trace_correlation 
fix(proxy): default litellm_trace_id to the OTel server span trace id
 2026-09-16 11:58:45 -07:00
yujonglee
560200da21
Merge pull request #41464 from BerriAI/litellm_rust_extract_auth_cache_crates 
refactor(rust): extract auth and cache crates
 2026-09-16 11:45:04 -07:00
Mateo Wang
78848d01a3
Merge pull request #34427 from BerriAI/litellm_bedrock_rag_retrieval_filter 
fix(rag): forward retrieval_filter from retrieval_config to vector store search
 2026-09-16 11:34:44 -07:00
Yujong Lee
44a518ea25 fix(rust): evict in-memory cache entries at the expiry boundary 
Matches Python evict_cache, which uses expiration <= now

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:30:45 +00:00
Yujong Lee
fc93c9ccba ci(rust): drop removed bedrock-auth feature steps 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:20:33 +00:00
Yujong Lee
23633445d7 refactor(rust): extract auth and cache crates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:12:11 +00:00
yucheng
1b5dacc717 fix(proxy): tolerate malformed auth spans and read the OTel span from request state for custom auth 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:05:38 +00:00
kerry-berri
4e996400e2
Merge pull request #41154 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated
 2026-09-16 10:37:21 -07:00
Yassin Kortam
8aebd4ff63
Merge pull request #40571 from BerriAI/litellm_presidio_mcp_mode_no_post_call_scan 
fix(guardrails): don't add post_call output scan for MCP-only Presidio modes
 2026-09-16 10:09:12 -07:00
yujonglee
a7cfdd4cd5
Merge pull request #41432 from BerriAI/litellm_remove_rust_gateway_router_realtime 
refactor(rust): remove gateway, config, router, realtime, and Rust trace-parity instrumentation
 2026-09-16 10:06:31 -07:00
Yassin Kortam
8491d01668
Merge pull request #41268 from BerriAI/litellm_outbound_http2_opt_in 
feat(http): opt-in outbound HTTP/2 for httpx clients
 2026-09-16 10:05:37 -07:00
kerry
b9ab36279b fix(prices): add tpm and rpm to gemini 3.8 live rows 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:17 +00:00
kerry
eed760a37d test: drop azure gpt-5.6 rate tests pinned to a dated price page 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:15 +00:00
yassin
8a43fed20c fix(guardrails): use explicit returns in _configured_event_hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:48:15 +00:00
Yassin Kortam
f0474bb70e
Merge pull request #41310 from BerriAI/litellm_model_access_denied_message 
fix(proxy): hide model allowlist from client-facing model access denied errors
 2026-09-16 09:38:29 -07:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
7b3582aa66 fix(guardrails): treat tag-based Mode as MCP-only when all hooks are MCP hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:31:44 +00:00
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
 2026-09-16 16:27:18 +00:00
berriai-litellm-provider-info-sync[bot]
809685603a
chore(prices): sync Azure prices: 247 models 
azure_ai/Codestral-2501: 
azure_ai/cohere-command-a: 
azure_ai/deepseek-r1: 
azure_ai/deepseek-v3: 
azure_ai/deepseek-v3-0324: 
azure_ai/deepseek-v3.1: 
azure_ai/deepseek-v3.2: 
azure_ai/deepseek-v3.2-speciale: 
azure_ai/deepseek-v4-flash: 
azure_ai/DeepSeek-V4-Flash-0731: 
azure_ai/deepseek-v4-pro: 
azure_ai/embed-v-4-0: 
azure_ai/FW-DeepSeek-V3.2: 
azure_ai/FW-DeepSeek-V4-Pro: 
azure_ai/FW-GLM-5: 
azure_ai/FW-GLM-5.1: 
azure_ai/FW-GLM-5.2: 
azure_ai/FW-GLM-5.2-Fast: 
azure_ai/FW-Inkling: 
azure_ai/FW-Kimi-K2.5: 
azure_ai/FW-Kimi-K2.6: 
azure_ai/FW-Kimi-K2.7-Code: 
azure_ai/FW-Kimi-K3: 
azure_ai/FW-MiniMax-M2.5: 
azure_ai/FW-MiniMax-M3: 
azure_ai/FW-Nemotron-3-Ultra-NVFP4: 
azure_ai/FW-Nemotron-Lightning-3.5-30B-A3B: 
azure_ai/gpt-oss-120b: 
azure_ai/grok-3: 
azure_ai/global/grok-3: 
azure_ai/grok-3-mini: 
azure_ai/global/grok-3-mini: 
azure_ai/grok-4: 
azure_ai/grok-4-1-fast-non-reasoning: 
azure_ai/grok-4-1-fast-reasoning: 
azure_ai/grok-4-20-non-reasoning: 
azure_ai/grok-4-20-reasoning: 
azure_ai/grok-4-fast-non-reasoning: 
azure_ai/grok-4-fast-reasoning: 
azure_ai/grok-4.3: 
azure_ai/grok-4.6: 
azure_ai/grok-code-fast-1: 
azure_ai/kimi-k2.5: 
azure_ai/kimi-k2.6: 
azure_ai/kimi-k2.7-code: 
azure_ai/Llama-3.3-70B-Instruct: 
azure_ai/Llama-4-Maverick-17B-128E-Instruct-FP8: 
azure_ai/MAI-DS-R1: 
azure_ai/MAI-Image-2.5: 
azure_ai/MAI-Image-2.5-Flash: 
azure_ai/MAI-Image-2e: 
azure_ai/MAI-Thinking-1: 
azure_ai/mistral-large-3: 
azure_ai/Phi-3-medium-128k-instruct: 
azure_ai/Phi-3-medium-4k-instruct: 
azure_ai/Phi-3-mini-128k-instruct: 
azure_ai/Phi-3-mini-4k-instruct: 
azure_ai/Phi-3-small-128k-instruct: 
azure_ai/Phi-3-small-8k-instruct: 
azure_ai/Phi-3.5-mini-instruct:
 2026-09-16 16:26:23 +00:00
Yassin Kortam
e941edd08a
Merge pull request #41407 from BerriAI/litellm_content_filter_stream_bounded_scan 
perf(content_filter): scan a bounded window per streamed chunk
 2026-09-16 09:25:23 -07:00
kerry
d398c9199c Merge remote-tracking branch 'origin/main' into HEAD 2026-09-16 16:18:47 +00:00
jesus
47117d880c fix(guardrails): don't add post_call output scan for MCP-only Presidio modes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:49 +00:00
Yujong Lee
6330d80efa refactor(rust): glob workspace members 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:46 +00:00
Yassin Kortam
29a712b186
Merge pull request #41368 from BerriAI/litellm_shadow_streaming_multi_target 
feat(router): stream shadow traffic and fan out silent_model to multiple targets
 2026-09-16 09:08:24 -07:00
Yujong Lee
96baeb8b04 refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:00:07 +00:00
joshua-berri
9cd787386e
Merge pull request #41314 from BerriAI/litellm_fix_mcp_jwt_oauth_persistence 
fix(mcp): authorize JWT OAuth credential persistence
 2026-09-16 06:47:40 -07:00
berriai-litellm-provider-info-sync[bot]
0443605c40
chore(prices): sync Together AI prices: 4 models, 4 deprecated 
together_ai/google/gemma-4-31B-it: deprecation_date
together_ai/intfloat/multilingual-e5-large-instruct: deprecation_date
together_ai/openai/gpt-oss-20b: deprecation_date
together_ai/thinkingmachines/Inkling-Small: deprecation_date
 2026-09-16 13:16:12 +00:00
yassin
a8fff5b091 fix(content_filter): refuse a trim that splits a conditional word across the cut 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:45:58 +00:00
yassin
4fbe631146 test(content_filter): annotate streaming test locals as Final and type the logging metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:17:47 +00:00
yassin
60642e875b perf(content_filter): back off refused streamed buffer cuts by one context length 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:51:06 +00:00
yassin
7e429dee87 fix(content_filter): keep exception phrases and open conditional sentences in the streamed buffer 
Trimming the streamed buffer to the retained tail could drop a category
exception phrase that suppresses a later keyword, or the identifier word
of an unfinished sentence that a conditional category pairs with a later
block word. Refuse the cut while either would leave the buffer so the
bounded scan masks and blocks exactly like a scan of the full text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:18:10 +00:00
yassin
52f06906fe fix(content_filter): widen the streamed scan tail to the longest configured keyword 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 10:58:30 +00:00
yassin
62ecb11ab9 perf(content_filter): scan a bounded window per streamed chunk 
The streaming post-call hook rescanned the whole accumulated choice buffer on every chunk, so scan cost grew quadratically with output length. Keep a bounded per-choice buffer instead: once it exceeds twice the scan context, drop the head when masking the head and tail separately yields the same output as masking the whole buffer, so no pattern, phrase or exception straddles the cut. Detections from the dropped head are kept and merged, deduplicated, into the final log row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 09:59:53 +00:00
yucheng
3628025aae fix(proxy): keep caller metadata.trace_id ahead of the OTel fallback on litellm_metadata routes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 08:30:54 +00:00
yucheng
898fbd37a7 test(proxy): mark locals Final in the OTel trace id fallback tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 08:02:04 +00:00
yucheng
4cb4493fa7 fix(proxy): let the OTel trace id fallback fill a null litellm_trace_id 
A body that serializes litellm_trace_id as null or an empty string carries no identity, so it must not
block the server span fallback. Also mark the nested metadata write as an out-param store

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:48:58 +00:00
yucheng
49417d4fa2 fix(proxy): ignore non-span parent_otel_span when deriving litellm_trace_id 
UserAPIKeyAuth.parent_otel_span is Any at runtime (opentelemetry is an optional extra), so the OTel
trace-id fallback must only format an int trace id, otherwise an object that merely quacks like a span
turns the whole request into a 500

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:41:48 +00:00
yucheng
f1fd1c8996 fix(proxy): default litellm_trace_id to the OTel server span trace id 
When the otel callback is enabled and the client sends no trace or session identity, the request now inherits the W3C trace id of the proxy's server span as litellm_trace_id and metadata.trace_id. The missing_session_id policy and SpendLogs then persist that value as session_id, so a trace in the OTel backend and its row in the Logs UI carry the same id. Explicit x-litellm-trace-id, traceparent, body metadata.trace_id and litellm_trace_id keep priority.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:22:03 +00:00
Joshua Valluru
ee676d59f2 fix(mcp): preserve browser OAuth for unrelated bearer tokens 2026-09-15 23:08:14 -07:00
Joshua Valluru
e035682ed1 refactor(auth): separate JWT identity and OAuth authorization 2026-09-15 22:15:57 -07:00
yassin
252c69b532 fix(router): snapshot shadow kwargs per target so concurrent shadows never share metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 04:31:43 +00:00
tin-berri
a8979fe054
Merge pull request #41371 from BerriAI/litellm_forecast_simplify_advanced_ui 
fix(ui): simplify Capability and Fuse advanced routing options
 2026-09-15 21:19:50 -07:00
kerry-berri
5960881640
Merge pull request #41337 from BerriAI/litellm_fix_responses_stream_absent_usage_recount 
fix(responses): recount tokens when a streamed response completes without usage
 2026-09-15 21:19:42 -07:00
yuneng-jiang
bbffddd517
Merge pull request #41366 from BerriAI/litellm_/back-002-litellm-e2e-replay-8de5b1 
fix(e2e): record cookie-setting provider responses and keep prompt-caching tests live
 2026-09-15 21:18:14 -07:00
yuneng-jiang
e3c1f78e28
Merge branch 'main' into litellm_/back-002-litellm-e2e-replay-8de5b1 2026-09-15 21:08:29 -07:00
yassin
0f7ed4433b fix(router): snapshot shadow kwargs before fan-out so shadows never see primary mutations 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 04:04:01 +00:00
Tin Chi Lo
9d640b86ed fix(ui): simplify Capability and Fuse routing options 2026-09-15 20:58:38 -07:00
kerry
ff878e7df0 refactor(responses): copy the terminal event instead of mutating stubbed chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:54:04 +00:00