scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 139
49820 commits 2515 branches 1410 tags 3 GiB
Commit graph

3237 commits

Author SHA1 Message Date
Mateo Wang
8e524370e1
Merge pull request #41343 from BerriAI/litellm_lit5030_bedrock_invoke_nova_prompt_caching 
fix(bedrock): make prompt caching work on the Nova InvokeModel route
 2026-09-16 13:42:21 -07:00
Yassin Kortam
c4a9341ef6
Merge pull request #34829 from max-sixty/bugfix/http-handler-del-closes-streaming-client 
fix(http_handler): keep a handler alive while a response it issued is still reading
 2026-09-16 13:39:04 -07:00
mateo-berri
6015437d67 test(bedrock): assert the Nova cache-read rate as a discount instead of pinning the vendor ratio 2026-09-16 13:19:58 -07:00
Mateo Wang
737929f338
Merge pull request #41335 from BerriAI/litellm_fireworks_dict_reasoning_effort 
fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string
 2026-09-16 13:12:37 -07:00
mateo-berri
884087f01c test(bedrock): type the vector store search test helper 2026-09-16 12:55:29 -07:00
mateo-berri
0dff64ce1a fix(bedrock): move a Nova invoke cache point behind an image or tool result to the last text block 2026-09-16 12:42:15 -07:00
mateo-berri
033aa8ba6d fix(bedrock): forward userContext in Knowledge Base Retrieve requests 
The Bedrock vector store search only lifted retrievalConfiguration out of extra_body, so the caller's userContext (the Retrieve API's ACL identity) never reached Bedrock and ACL-enabled data sources answered with zero results. The transform now forwards userContext, taken from extra_body first and then from the top-level params where the OpenAI SDK's extra_body merge lands, as the caller sent it.
 2026-09-16 11:50:36 -07:00
kerry-berri
4e996400e2
Merge pull request #41154 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated
 2026-09-16 10:37:21 -07:00
Yassin Kortam
8491d01668
Merge pull request #41268 from BerriAI/litellm_outbound_http2_opt_in 
feat(http): opt-in outbound HTTP/2 for httpx clients
 2026-09-16 10:05:37 -07:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
04aad317c7 Merge remote-tracking branch 'origin/main' into pr34829 2026-09-16 16:02:08 +00:00
kerry-berri
4b84fa9230
Merge pull request #41336 from BerriAI/litellm_fix_anthropic_stream_absent_usage 
fix(anthropic): tolerate message_delta events without usage when streaming
 2026-09-15 20:23:01 -07:00
Yassin Kortam
d108cdc431
Merge pull request #38254 from BerriAI/litellm_fix_xai_responses_instructions 2026-09-15 19:55:52 -07:00
kerry-berri
399420bb25
Merge pull request #41339 from BerriAI/litellm_fix_fireworks_cost_components 
fix(fireworks-ai): bill cache-write, reasoning and audio tokens via the shared cost calculator
 2026-09-15 19:16:24 -07:00
kerry
133f1e8ef5 test(fireworks-ai): drop the explanatory comment on the cache-read constant 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 02:06:42 +00:00
kerry-berri
54e1b9e3de
Merge pull request #41338 from BerriAI/litellm_fix_gemini_model_version 
fix(gemini): propagate the provider's modelVersion to the response model
 2026-09-15 19:04:50 -07:00
kerry
091a38cce6 test(anthropic): import Final for the annotated locals 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:43:08 +00:00
kerry
c4c96180e3 fix(gemini): strip version suffix from modelVersion and keep it on blocked streams 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:42:36 +00:00
kerry
8352045f32 style(fireworks-ai): apply repository conventions to the cost component change 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:39:24 +00:00
Devin AI
8d625d0400 fix(xai): keep 'instructions' on the xAI Responses API so system messages survive web_search bridging 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:35:13 +00:00
kerry
10aef224e7 style(anthropic): apply repository conventions to the missing-usage change 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:34:28 +00:00
kerry
d1aa37b0a8 merge main into litellm_fix_fireworks_cost_components 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:28:29 +00:00
kerry
44a6d19889 test(gemini): drop review narration from the wrapper test docstring 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:14:37 +00:00
kerry
801f6a92b4 test(gemini): assert the served modelVersion reaches the assembled stream through CustomStreamWrapper 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
mateo-berri
3c15f64fd4 fix(bedrock): make prompt caching work on the Nova InvokeModel route 
Nova InvokeModel rejects the standalone cachePoint blocks the shared Converse transform emits, so each one is folded into the block it caches and tool_config injection points are dropped before the transform runs, since this route has no tool caching to credit. Usage reads Bedrock's Count-suffixed cache keys and adds cached tokens into prompt_tokens, streaming routes every wrapped InvokeModel event through the Converse chunk parser and tolerates the missing totalTokens, and the Nova 1 cost-map entries gain cache_read_input_token_cost at a quarter of the input rate
 2026-09-15 18:01:21 -07:00
kerry
5eab1feb20 fix(fireworks-ai): bill cache-write, reasoning, and audio tokens via the shared cost calculator 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:42:13 +00:00
Devin AI
a7a61db78d fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:37:55 +00:00
kerry
d5a36eb2ca fix(gemini): propagate provider modelVersion onto model responses 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:36:07 +00:00
kerry
ba6bcd747f chore: merge origin/main into test cleanup 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:33:25 +00:00
Yassin Kortam
7eeba69016
Merge pull request #41316 from BerriAI/litellm_nvidia_nim_infer_passthrough 
feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer
 2026-09-15 17:28:25 -07:00
kerry
4657d43fb9 fix(anthropic): tolerate message_delta chunks without usage in streams 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:27:09 +00:00
Yassin Kortam
636313e9fc
Merge pull request #41322 from BerriAI/litellm_gcp_video_usage 
fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough
 2026-09-15 17:21:15 -07:00
yassin
771b2509d1 fix(proxy): reject mixed NIM model groups and strip the deployment model before the group in /nvidia_nim URLs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:34:22 +00:00
kerry
e64371062c test: remove test files left empty by the cleanup 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:04:40 +00:00
Yassin Kortam
9d75cdd502
Merge pull request #41304 from BerriAI/litellm_lit1698_openai_system_messages_first 2026-09-15 15:56:13 -07:00
kerry
c94a4f8696 test: restore xai reported-cost passthrough tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:49:57 +00:00
yassin
b13cf21fd1 fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough 
Route Vertex and Gemini /interactions create responses through the shared
Interactions usage transformation so SpendLogs carry per-modality tokens and
cost instead of zeros, and carry parameters.sampleCount into Veo usage as
video_count so duration-based video cost is multiplied by the number of
generated videos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:42:27 +00:00
kerry
357e0fca8e chore: merge origin/main into test cleanup 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:36:58 +00:00
kerry
2586b21893 test: keep tests that survive correct cost-map updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:16:08 +00:00
yassin
69be041da7 feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:11:08 +00:00
Yassin Kortam
1b8daf20e0
Merge pull request #38268 from BerriAI/litellm_fix_xai_web_search_nested_filters 
fix(xai): honor nested web_search filters on the xAI Responses API
 2026-09-15 14:56:36 -07:00
yassin
77d913958d feat(openai): add openai_system_messages_first to put system messages first for prompt caching 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:53:29 +00:00
kerry
a428cc8d84 test: restore runtime-derived tests dropped by mistake 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:44:39 +00:00
kerry
62aa21e810 test: drop remaining tests that pin cost-map vendor facts 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:34:00 +00:00
yassin
afb6f8be65 fix(xai): treat an explicit empty web_search filters object as unrestricted 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:24:32 +00:00
Yassin Kortam
a6127d2363
Merge pull request #35180 from BerriAI/litellm_lit4995_vertex_rerank_search_units 
fix(rerank): bill Vertex search_units from input records and give every rerank response a unique id
 2026-09-15 14:23:23 -07:00
Devin AI
3cb5ceb98c fix(xai): honor nested web_search filters on the xAI Responses API 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:02:10 +00:00
yassin
113e43b87a Merge remote-tracking branch 'origin/main' into litellm_fix_xai_chat_web_search_live_search 2026-09-15 20:46:51 +00:00
Yassin Kortam
cdb4eaac59
Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts 
fix(proxy): run the remaining inline token counts off the event loop
 2026-09-15 13:38:00 -07:00
yassin
95cc3bfabb Merge remote-tracking branch 'origin/main' into litellm_fix_xai_chat_web_search_live_search 2026-09-15 20:27:48 +00:00