scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-16 23:41:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
49726 commits 2457 branches 1410 tags 2.9 GiB
Commit graph

12111 commits

Author SHA1 Message Date
yassin
4513f78df5 fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:14:36 +00:00
yassin
bf39aebcf1 ci(migrations): flag defaulted ADD COLUMN on request-log tables 
Postgres 10 has no fast default path, so ADD COLUMN ... DEFAULT on
LiteLLM_SpendLogs rewrites the heap and every index under an ACCESS
EXCLUSIVE lock inside the boot-time migrate deploy. The checker now
reports it on LiteLLM_SpendLogs and LiteLLM_ErrorLogs; the two shipped
migrations that already do it are grandfathered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:02:00 +00:00
kerry-berri
4e996400e2
Merge pull request #41154 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated
 2026-09-16 10:37:21 -07:00
Yassin Kortam
8aebd4ff63
Merge pull request #40571 from BerriAI/litellm_presidio_mcp_mode_no_post_call_scan 
fix(guardrails): don't add post_call output scan for MCP-only Presidio modes
 2026-09-16 10:09:12 -07:00
yujonglee
a7cfdd4cd5
Merge pull request #41432 from BerriAI/litellm_remove_rust_gateway_router_realtime 
refactor(rust): remove gateway, config, router, realtime, and Rust trace-parity instrumentation
 2026-09-16 10:06:31 -07:00
Yassin Kortam
8491d01668
Merge pull request #41268 from BerriAI/litellm_outbound_http2_opt_in 
feat(http): opt-in outbound HTTP/2 for httpx clients
 2026-09-16 10:05:37 -07:00
kerry
eed760a37d test: drop azure gpt-5.6 rate tests pinned to a dated price page 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:15 +00:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
7b3582aa66 fix(guardrails): treat tag-based Mode as MCP-only when all hooks are MCP hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:31:44 +00:00
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
 2026-09-16 16:27:18 +00:00
Yassin Kortam
e941edd08a
Merge pull request #41407 from BerriAI/litellm_content_filter_stream_bounded_scan 
perf(content_filter): scan a bounded window per streamed chunk
 2026-09-16 09:25:23 -07:00
jesus
47117d880c fix(guardrails): don't add post_call output scan for MCP-only Presidio modes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:49 +00:00
Yassin Kortam
29a712b186
Merge pull request #41368 from BerriAI/litellm_shadow_streaming_multi_target 
feat(router): stream shadow traffic and fan out silent_model to multiple targets
 2026-09-16 09:08:24 -07:00
Yujong Lee
96baeb8b04 refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:00:07 +00:00
joshua-berri
9cd787386e
Merge pull request #41314 from BerriAI/litellm_fix_mcp_jwt_oauth_persistence 
fix(mcp): authorize JWT OAuth credential persistence
 2026-09-16 06:47:40 -07:00
yassin
a8fff5b091 fix(content_filter): refuse a trim that splits a conditional word across the cut 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:45:58 +00:00
yassin
4fbe631146 test(content_filter): annotate streaming test locals as Final and type the logging metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:17:47 +00:00
yassin
60642e875b perf(content_filter): back off refused streamed buffer cuts by one context length 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:51:06 +00:00
yassin
7e429dee87 fix(content_filter): keep exception phrases and open conditional sentences in the streamed buffer 
Trimming the streamed buffer to the retained tail could drop a category
exception phrase that suppresses a later keyword, or the identifier word
of an unfinished sentence that a conditional category pairs with a later
block word. Refuse the cut while either would leave the buffer so the
bounded scan masks and blocks exactly like a scan of the full text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:18:10 +00:00
yassin
52f06906fe fix(content_filter): widen the streamed scan tail to the longest configured keyword 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 10:58:30 +00:00
yassin
62ecb11ab9 perf(content_filter): scan a bounded window per streamed chunk 
The streaming post-call hook rescanned the whole accumulated choice buffer on every chunk, so scan cost grew quadratically with output length. Keep a bounded per-choice buffer instead: once it exceeds twice the scan context, drop the head when masking the head and tail separately yields the same output as masking the whole buffer, so no pattern, phrase or exception straddles the cut. Detections from the dropped head are kept and merged, deduplicated, into the final log row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 09:59:53 +00:00
Joshua Valluru
ee676d59f2 fix(mcp): preserve browser OAuth for unrelated bearer tokens 2026-09-15 23:08:14 -07:00
Joshua Valluru
e035682ed1 refactor(auth): separate JWT identity and OAuth authorization 2026-09-15 22:15:57 -07:00
yassin
252c69b532 fix(router): snapshot shadow kwargs per target so concurrent shadows never share metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 04:31:43 +00:00
kerry-berri
5960881640
Merge pull request #41337 from BerriAI/litellm_fix_responses_stream_absent_usage_recount 
fix(responses): recount tokens when a streamed response completes without usage
 2026-09-15 21:19:42 -07:00
yassin
0f7ed4433b fix(router): snapshot shadow kwargs before fan-out so shadows never see primary mutations 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 04:04:01 +00:00
kerry
ff878e7df0 refactor(responses): copy the terminal event instead of mutating stubbed chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:54:04 +00:00
yassin
9cabde90dd test(router): cover _run_silent_experiment directly for router coverage gate 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:48:32 +00:00
kerry
7cc07d437a fix(responses): build the billed terminal response immutably and guard the cache dump 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:47:03 +00:00
yassin
06fcc1f733 feat(router): stream shadow traffic and fan out silent_model to multiple targets 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:28:08 +00:00
kerry-berri
4b84fa9230
Merge pull request #41336 from BerriAI/litellm_fix_anthropic_stream_absent_usage 
fix(anthropic): tolerate message_delta events without usage when streaming
 2026-09-15 20:23:01 -07:00
kerry
7121e64db4 fix(responses): type the dict terminal response so the estimated usage is billed 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:13:53 +00:00
Yassin Kortam
d108cdc431
Merge pull request #38254 from BerriAI/litellm_fix_xai_responses_instructions 2026-09-15 19:55:52 -07:00
yuneng-jiang
65d607d161
Merge pull request #41346 from BerriAI/litellm_e2e_provider_cache 
feat(e2e): reuse exact provider responses for 24 hours
 2026-09-15 19:45:56 -07:00
Yuneng Jiang
3dda798c8a
ci(e2e): consolidate cache contracts in filtered CircleCI job 2026-09-15 19:28:50 -07:00
yucheng-berri
c14ac1dbe9
Merge pull request #41329 from BerriAI/litellm_singulr_v2_contract 2026-09-15 19:27:30 -07:00
kerry-berri
399420bb25
Merge pull request #41339 from BerriAI/litellm_fix_fireworks_cost_components 
fix(fireworks-ai): bill cache-write, reasoning and audio tokens via the shared cost calculator
 2026-09-15 19:16:24 -07:00
Joshua Valluru
c7e4160ee6 fix(mcp): enforce OAuth write policy across signed callbacks 2026-09-15 19:15:34 -07:00
Yucheng He
146085669c fix(guardrails): run Singulr logging_only through the base hook, key MCP scans off the proxy call type and type the payloads 
Removes the Singulr async_logging_hook and logging_hook overrides so logging_only runs through CustomGuardrail.async_logging_hook: the response scope reaches Singulr as an assistant message instead of a raw ModelResponse dump, a vendor timeout is recorded as guardrail_failed_to_respond, a request-scope block ends the scan, and the sync success callback thread makes no Singulr call.

Decides MCP versus LLM by the proxy logging object's call_type (then the call_type or server-only markers in request_data), never by name, arguments or mcp_tool_name keys a client can put in a chat body. REST /mcp-rest/tools/call pre-scans reach Singulr as mcp_request and a non-mapping arguments value is forwarded as tool_arguments instead of raising.

should_block is a strict bool defaulting to false so a null verdict is an invalid response that block_on_error decides; payload fields drop Any for Sequence, Mapping and AssistantMessage types; metadata carries only the keys present; docstrings and section comments removed per the repo comment policy.
 2026-09-15 19:14:49 -07:00
aniket-kardile
8cce2b196a feat(guardrails): singulr v2 API contract with logging_only, pre_mcp_call and post_mcp_call 
Squash of BerriAI/litellm#37464 (head da298ca7) by @aniket-kardile, adopted onto main: v2 gateway payload contract with request, response, mcp_request and mcp_response scopes, typed payload models, proxy user, org and team metadata forwarded to Singulr, and the logging_only, pre_mcp_call and post_mcp_call modes.
 2026-09-15 19:14:49 -07:00
kerry
133f1e8ef5 test(fireworks-ai): drop the explanatory comment on the cache-read constant 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 02:06:42 +00:00
kerry
7680d3de86 fix(responses): tolerate dict terminal responses when estimating usage 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 02:05:44 +00:00
kerry-berri
54e1b9e3de
Merge pull request #41338 from BerriAI/litellm_fix_gemini_model_version 
fix(gemini): propagate the provider's modelVersion to the response model
 2026-09-15 19:04:50 -07:00
Joshua Valluru
97211bc356 fix(mcp): authorize per-user OAuth credential writes 2026-09-15 19:00:47 -07:00
yucheng-berri
41eb2dbfeb
Merge pull request #41128 from BerriAI/litellm_llm_judge_pre_call 
feat(guardrails): support pre_call and during_call modes for llm_as_a_judge
 2026-09-15 18:58:21 -07:00
tin-berri
106365fbac
Merge pull request #41341 from BerriAI/litellm_autorouter_pricing_fields 
fix(proxy): preserve Anthropic pricing modifiers in router savings
 2026-09-15 18:56:15 -07:00
yassin
b085a3c151 fix(mcp): return fixed client message on sampling model access denial 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:51:29 +00:00
kerry
7ed20406d7 test(responses): narrow the ValueError assertion to satisfy PT011 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:47:45 +00:00
yassin
15f2e25e8a refactor(proxy): replace configurable model access denied message with a fixed clean client message 
Drop the model_access_denied_message setting, its {model} template, the DB
override entry and the Admin UI field. Model access denials now always return
the fixed client message while the allowlist diagnostic is logged at the final
HTTP, realtime and MCP boundaries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:44:36 +00:00