scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-05 02:41:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 49
49766 commits 3118 branches 1436 tags 4.4 GiB
Commit graph

49766 commits

This branch
This branch
All branches
Author SHA1 Message Date
Mateo Wang
737929f338
Merge pull request #41335 from BerriAI/litellm_fireworks_dict_reasoning_effort 
fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string
 2026-09-16 13:12:37 -07:00
yujonglee
560065df16
Merge pull request #41480 from BerriAI/litellm_rust_ci_nextest_rust_cache 
ci(rust): split rust jobs, use nextest and Swatinem/rust-cache
 2026-09-16 13:08:20 -07:00
Mateo Wang
2b33201a09
Merge pull request #41475 from BerriAI/litellm_bedrock_kb_user_context 
fix(bedrock): forward userContext in Knowledge Base Retrieve requests
 2026-09-16 13:07:03 -07:00
mateo-berri
884087f01c test(bedrock): type the vector store search test helper 2026-09-16 12:55:29 -07:00
Yujong Lee
48df3d5a48 ci(rust): install nextest via pinned taiki-e/install-action 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:54:46 +00:00
Yujong Lee
3f15dcd96b ci(rust): fold fmt into the clippy job 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:47:48 +00:00
Yujong Lee
e850232f02 test(rust): assert merge cost scales linearly instead of a wall-clock bound 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:43:42 +00:00
kerry-berri
b6143b3711
Merge pull request #41457 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Google Gemini prices: 22 models
 2026-09-16 12:37:16 -07:00
Mateo Wang
a1b1f1ef9d
Merge pull request #34455 from BerriAI/litellm_lit4767_empty_choices_streaming_guard 
fix(responses): guard empty-choices chunks in the Responses API streaming bridge
 2026-09-16 12:28:46 -07:00
Yujong Lee
e8b5632c20 ci(rust): run the token counter timing test alone under nextest 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:22:02 +00:00
kerry
e46106e20b test: drop gemini-3.1-flash-lite-image capability pins 
The per-route capability test hardcoded vendor facts, including function calling support on the gemini route, which the live model card says is not supported. Keep the backup-matches-main invariant and the routing tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:21:50 +00:00
Mateo Wang
734038a83c
Merge pull request #41333 from BerriAI/litellm_cost_json_codeowners 
chore(codeowners): add ryan and kerry as owners of the cost map
 2026-09-16 12:17:13 -07:00
Yujong Lee
b70ddc2fd8 ci(rust): split rust jobs, use nextest and Swatinem/rust-cache 
Split the Rust workflow into fmt, clippy, nextest and wheel jobs so they run in parallel, replace manual actions/cache with Swatinem/rust-cache, and install a pinned checksum-verified cargo-nextest. Make two python-bridge tests self-contained so they pass when nextest runs each test in its own process.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:14:32 +00:00
berriai-litellm-provider-info-sync[bot]
b893e6b926
chore(prices): sync Google Gemini prices: 22 models 
gemini/gemini-2.5-flash: max_tokens, max_output_tokens, supports_audio_input
gemini/gemini-2.5-flash-image: max_input_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-lite: max_tokens, max_output_tokens, supports_audio_input
gemini-2.5-flash-native-audio-preview-12-2025: supports_vision, max_input_tokens, supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-native-audio-preview-12-2025: supports_vision, max_input_tokens, supports_web_search, supports_response_schema, supports_function_calling
gemini-2.5-flash-preview-tts: max_tokens, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-preview-tts: max_tokens, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-pro: max_tokens, max_output_tokens
gemini/gemini-2.5-pro-preview-tts: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-3-flash-preview: max_tokens, max_output_tokens, supports_audio_input
gemini/gemini-3-pro-image: supports_response_schema
gemini/gemini-3.1-flash-image: max_input_tokens, supports_response_schema
gemini/gemini-3.1-flash-lite-image: supports_web_search, supports_function_calling
gemini-3.1-flash-live-preview: supports_response_schema
gemini/gemini-3.1-flash-live-preview: supports_response_schema
gemini/gemini-3.1-flash-tts-preview: supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-3.5-flash: max_tokens, max_output_tokens
gemini/gemini-3.5-live-translate-preview: supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-3.5-transcribe: supports_function_calling
gemini/gemini-3.5-transcribe-live: supports_function_calling
gemini/gemini-embedding-2: supports_vision, supports_audio_input
gemini/gemini-omni-1.1-flash: max_input_tokens
 2026-09-16 19:06:19 +00:00
yucheng-berri
b04d530ecf
Merge pull request #41386 from BerriAI/litellm_otel_trace_correlation 
fix(proxy): default litellm_trace_id to the OTel server span trace id
 2026-09-16 11:58:45 -07:00
mateo-berri
033aa8ba6d fix(bedrock): forward userContext in Knowledge Base Retrieve requests 
The Bedrock vector store search only lifted retrievalConfiguration out of extra_body, so the caller's userContext (the Retrieve API's ACL identity) never reached Bedrock and ACL-enabled data sources answered with zero results. The transform now forwards userContext, taken from extra_body first and then from the top-level params where the OpenAI SDK's extra_body merge lands, as the caller sent it.
 2026-09-16 11:50:36 -07:00
yujonglee
560200da21
Merge pull request #41464 from BerriAI/litellm_rust_extract_auth_cache_crates 
refactor(rust): extract auth and cache crates
 2026-09-16 11:45:04 -07:00
Mateo Wang
78848d01a3
Merge pull request #34427 from BerriAI/litellm_bedrock_rag_retrieval_filter 
fix(rag): forward retrieval_filter from retrieval_config to vector store search
 2026-09-16 11:34:44 -07:00
Yujong Lee
44a518ea25 fix(rust): evict in-memory cache entries at the expiry boundary 
Matches Python evict_cache, which uses expiration <= now

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:30:45 +00:00
Yujong Lee
fc93c9ccba ci(rust): drop removed bedrock-auth feature steps 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:20:33 +00:00
Yujong Lee
23633445d7 refactor(rust): extract auth and cache crates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:12:11 +00:00
yucheng
1b5dacc717 fix(proxy): tolerate malformed auth spans and read the OTel span from request state for custom auth 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:05:38 +00:00
berriai-litellm-provider-info-sync[bot]
f8fb31db3a
chore(prices): sync Azure prices: 2 models 
azure_ai/grok-4.3: input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens
azure_ai/grok-4.6: input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens
 2026-09-16 17:51:09 +00:00
kerry-berri
4e996400e2
Merge pull request #41154 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated
 2026-09-16 10:37:21 -07:00
Yassin Kortam
8aebd4ff63
Merge pull request #40571 from BerriAI/litellm_presidio_mcp_mode_no_post_call_scan 
fix(guardrails): don't add post_call output scan for MCP-only Presidio modes
 2026-09-16 10:09:12 -07:00
yujonglee
a7cfdd4cd5
Merge pull request #41432 from BerriAI/litellm_remove_rust_gateway_router_realtime 
refactor(rust): remove gateway, config, router, realtime, and Rust trace-parity instrumentation
 2026-09-16 10:06:31 -07:00
Yassin Kortam
8491d01668
Merge pull request #41268 from BerriAI/litellm_outbound_http2_opt_in 
feat(http): opt-in outbound HTTP/2 for httpx clients
 2026-09-16 10:05:37 -07:00
kerry
b9ab36279b fix(prices): add tpm and rpm to gemini 3.8 live rows 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:17 +00:00
kerry
eed760a37d test: drop azure gpt-5.6 rate tests pinned to a dated price page 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:15 +00:00
yassin
8a43fed20c fix(guardrails): use explicit returns in _configured_event_hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:48:15 +00:00
Yassin Kortam
f0474bb70e
Merge pull request #41310 from BerriAI/litellm_model_access_denied_message 
fix(proxy): hide model allowlist from client-facing model access denied errors
 2026-09-16 09:38:29 -07:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
7b3582aa66 fix(guardrails): treat tag-based Mode as MCP-only when all hooks are MCP hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:31:44 +00:00
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
 2026-09-16 16:27:18 +00:00
berriai-litellm-provider-info-sync[bot]
809685603a
chore(prices): sync Azure prices: 247 models 
azure_ai/Codestral-2501: 
azure_ai/cohere-command-a: 
azure_ai/deepseek-r1: 
azure_ai/deepseek-v3: 
azure_ai/deepseek-v3-0324: 
azure_ai/deepseek-v3.1: 
azure_ai/deepseek-v3.2: 
azure_ai/deepseek-v3.2-speciale: 
azure_ai/deepseek-v4-flash: 
azure_ai/DeepSeek-V4-Flash-0731: 
azure_ai/deepseek-v4-pro: 
azure_ai/embed-v-4-0: 
azure_ai/FW-DeepSeek-V3.2: 
azure_ai/FW-DeepSeek-V4-Pro: 
azure_ai/FW-GLM-5: 
azure_ai/FW-GLM-5.1: 
azure_ai/FW-GLM-5.2: 
azure_ai/FW-GLM-5.2-Fast: 
azure_ai/FW-Inkling: 
azure_ai/FW-Kimi-K2.5: 
azure_ai/FW-Kimi-K2.6: 
azure_ai/FW-Kimi-K2.7-Code: 
azure_ai/FW-Kimi-K3: 
azure_ai/FW-MiniMax-M2.5: 
azure_ai/FW-MiniMax-M3: 
azure_ai/FW-Nemotron-3-Ultra-NVFP4: 
azure_ai/FW-Nemotron-Lightning-3.5-30B-A3B: 
azure_ai/gpt-oss-120b: 
azure_ai/grok-3: 
azure_ai/global/grok-3: 
azure_ai/grok-3-mini: 
azure_ai/global/grok-3-mini: 
azure_ai/grok-4: 
azure_ai/grok-4-1-fast-non-reasoning: 
azure_ai/grok-4-1-fast-reasoning: 
azure_ai/grok-4-20-non-reasoning: 
azure_ai/grok-4-20-reasoning: 
azure_ai/grok-4-fast-non-reasoning: 
azure_ai/grok-4-fast-reasoning: 
azure_ai/grok-4.3: 
azure_ai/grok-4.6: 
azure_ai/grok-code-fast-1: 
azure_ai/kimi-k2.5: 
azure_ai/kimi-k2.6: 
azure_ai/kimi-k2.7-code: 
azure_ai/Llama-3.3-70B-Instruct: 
azure_ai/Llama-4-Maverick-17B-128E-Instruct-FP8: 
azure_ai/MAI-DS-R1: 
azure_ai/MAI-Image-2.5: 
azure_ai/MAI-Image-2.5-Flash: 
azure_ai/MAI-Image-2e: 
azure_ai/MAI-Thinking-1: 
azure_ai/mistral-large-3: 
azure_ai/Phi-3-medium-128k-instruct: 
azure_ai/Phi-3-medium-4k-instruct: 
azure_ai/Phi-3-mini-128k-instruct: 
azure_ai/Phi-3-mini-4k-instruct: 
azure_ai/Phi-3-small-128k-instruct: 
azure_ai/Phi-3-small-8k-instruct: 
azure_ai/Phi-3.5-mini-instruct:
 2026-09-16 16:26:23 +00:00
Yassin Kortam
e941edd08a
Merge pull request #41407 from BerriAI/litellm_content_filter_stream_bounded_scan 
perf(content_filter): scan a bounded window per streamed chunk
 2026-09-16 09:25:23 -07:00
kerry
d398c9199c Merge remote-tracking branch 'origin/main' into HEAD 2026-09-16 16:18:47 +00:00
jesus
47117d880c fix(guardrails): don't add post_call output scan for MCP-only Presidio modes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:49 +00:00
Yujong Lee
6330d80efa refactor(rust): glob workspace members 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:46 +00:00
Yassin Kortam
29a712b186
Merge pull request #41368 from BerriAI/litellm_shadow_streaming_multi_target 
feat(router): stream shadow traffic and fan out silent_model to multiple targets
 2026-09-16 09:08:24 -07:00
Yujong Lee
96baeb8b04 refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:00:07 +00:00
joshua-berri
9cd787386e
Merge pull request #41314 from BerriAI/litellm_fix_mcp_jwt_oauth_persistence 
fix(mcp): authorize JWT OAuth credential persistence
 2026-09-16 06:47:40 -07:00
berriai-litellm-provider-info-sync[bot]
0443605c40
chore(prices): sync Together AI prices: 4 models, 4 deprecated 
together_ai/google/gemma-4-31B-it: deprecation_date
together_ai/intfloat/multilingual-e5-large-instruct: deprecation_date
together_ai/openai/gpt-oss-20b: deprecation_date
together_ai/thinkingmachines/Inkling-Small: deprecation_date
 2026-09-16 13:16:12 +00:00
yassin
a8fff5b091 fix(content_filter): refuse a trim that splits a conditional word across the cut 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:45:58 +00:00
yassin
4fbe631146 test(content_filter): annotate streaming test locals as Final and type the logging metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:17:47 +00:00
yassin
60642e875b perf(content_filter): back off refused streamed buffer cuts by one context length 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:51:06 +00:00
yassin
7e429dee87 fix(content_filter): keep exception phrases and open conditional sentences in the streamed buffer 
Trimming the streamed buffer to the retained tail could drop a category
exception phrase that suppresses a later keyword, or the identifier word
of an unfinished sentence that a conditional category pairs with a later
block word. Refuse the cut while either would leave the buffer so the
bounded scan masks and blocks exactly like a scan of the full text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:18:10 +00:00
yassin
52f06906fe fix(content_filter): widen the streamed scan tail to the longest configured keyword 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 10:58:30 +00:00
yassin
62ecb11ab9 perf(content_filter): scan a bounded window per streamed chunk 
The streaming post-call hook rescanned the whole accumulated choice buffer on every chunk, so scan cost grew quadratically with output length. Keep a bounded per-choice buffer instead: once it exceeds twice the scan context, drop the head when masking the head and tail separately yields the same output as masking the whole buffer, so no pattern, phrase or exception straddles the cut. Detections from the dropped head are kept and merged, deduplicated, into the final log row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 09:59:53 +00:00