scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-04 02:31:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 69
49802 commits 3112 branches 1436 tags 4.4 GiB
Commit graph

49802 commits

This branch
This branch
All branches
Author SHA1 Message Date
kerry
eed760a37d test: drop azure gpt-5.6 rate tests pinned to a dated price page 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:15 +00:00
yassin
8a43fed20c fix(guardrails): use explicit returns in _configured_event_hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:48:15 +00:00
Yassin Kortam
f0474bb70e
Merge pull request #41310 from BerriAI/litellm_model_access_denied_message 
fix(proxy): hide model allowlist from client-facing model access denied errors
 2026-09-16 09:38:29 -07:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
7b3582aa66 fix(guardrails): treat tag-based Mode as MCP-only when all hooks are MCP hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:31:44 +00:00
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
 2026-09-16 16:27:18 +00:00
berriai-litellm-provider-info-sync[bot]
809685603a
chore(prices): sync Azure prices: 247 models 
azure_ai/Codestral-2501: 
azure_ai/cohere-command-a: 
azure_ai/deepseek-r1: 
azure_ai/deepseek-v3: 
azure_ai/deepseek-v3-0324: 
azure_ai/deepseek-v3.1: 
azure_ai/deepseek-v3.2: 
azure_ai/deepseek-v3.2-speciale: 
azure_ai/deepseek-v4-flash: 
azure_ai/DeepSeek-V4-Flash-0731: 
azure_ai/deepseek-v4-pro: 
azure_ai/embed-v-4-0: 
azure_ai/FW-DeepSeek-V3.2: 
azure_ai/FW-DeepSeek-V4-Pro: 
azure_ai/FW-GLM-5: 
azure_ai/FW-GLM-5.1: 
azure_ai/FW-GLM-5.2: 
azure_ai/FW-GLM-5.2-Fast: 
azure_ai/FW-Inkling: 
azure_ai/FW-Kimi-K2.5: 
azure_ai/FW-Kimi-K2.6: 
azure_ai/FW-Kimi-K2.7-Code: 
azure_ai/FW-Kimi-K3: 
azure_ai/FW-MiniMax-M2.5: 
azure_ai/FW-MiniMax-M3: 
azure_ai/FW-Nemotron-3-Ultra-NVFP4: 
azure_ai/FW-Nemotron-Lightning-3.5-30B-A3B: 
azure_ai/gpt-oss-120b: 
azure_ai/grok-3: 
azure_ai/global/grok-3: 
azure_ai/grok-3-mini: 
azure_ai/global/grok-3-mini: 
azure_ai/grok-4: 
azure_ai/grok-4-1-fast-non-reasoning: 
azure_ai/grok-4-1-fast-reasoning: 
azure_ai/grok-4-20-non-reasoning: 
azure_ai/grok-4-20-reasoning: 
azure_ai/grok-4-fast-non-reasoning: 
azure_ai/grok-4-fast-reasoning: 
azure_ai/grok-4.3: 
azure_ai/grok-4.6: 
azure_ai/grok-code-fast-1: 
azure_ai/kimi-k2.5: 
azure_ai/kimi-k2.6: 
azure_ai/kimi-k2.7-code: 
azure_ai/Llama-3.3-70B-Instruct: 
azure_ai/Llama-4-Maverick-17B-128E-Instruct-FP8: 
azure_ai/MAI-DS-R1: 
azure_ai/MAI-Image-2.5: 
azure_ai/MAI-Image-2.5-Flash: 
azure_ai/MAI-Image-2e: 
azure_ai/MAI-Thinking-1: 
azure_ai/mistral-large-3: 
azure_ai/Phi-3-medium-128k-instruct: 
azure_ai/Phi-3-medium-4k-instruct: 
azure_ai/Phi-3-mini-128k-instruct: 
azure_ai/Phi-3-mini-4k-instruct: 
azure_ai/Phi-3-small-128k-instruct: 
azure_ai/Phi-3-small-8k-instruct: 
azure_ai/Phi-3.5-mini-instruct:
 2026-09-16 16:26:23 +00:00
Yassin Kortam
e941edd08a
Merge pull request #41407 from BerriAI/litellm_content_filter_stream_bounded_scan 
perf(content_filter): scan a bounded window per streamed chunk
 2026-09-16 09:25:23 -07:00
yassin
23e775796c Merge remote-tracking branch 'origin/main' into pr34829 2026-09-16 16:23:53 +00:00
kerry
d398c9199c Merge remote-tracking branch 'origin/main' into HEAD 2026-09-16 16:18:47 +00:00
jesus
47117d880c fix(guardrails): don't add post_call output scan for MCP-only Presidio modes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:49 +00:00
Yujong Lee
6330d80efa refactor(rust): glob workspace members 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:46 +00:00
Yassin Kortam
29a712b186
Merge pull request #41368 from BerriAI/litellm_shadow_streaming_multi_target 
feat(router): stream shadow traffic and fan out silent_model to multiple targets
 2026-09-16 09:08:24 -07:00
yassin
04aad317c7 Merge remote-tracking branch 'origin/main' into pr34829 2026-09-16 16:02:08 +00:00
Yujong Lee
96baeb8b04 refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:00:07 +00:00
joshua-berri
9cd787386e
Merge pull request #41314 from BerriAI/litellm_fix_mcp_jwt_oauth_persistence 
fix(mcp): authorize JWT OAuth credential persistence
 2026-09-16 06:47:40 -07:00
berriai-litellm-provider-info-sync[bot]
0443605c40
chore(prices): sync Together AI prices: 4 models, 4 deprecated 
together_ai/google/gemma-4-31B-it: deprecation_date
together_ai/intfloat/multilingual-e5-large-instruct: deprecation_date
together_ai/openai/gpt-oss-20b: deprecation_date
together_ai/thinkingmachines/Inkling-Small: deprecation_date
 2026-09-16 13:16:12 +00:00
yassin
a8fff5b091 fix(content_filter): refuse a trim that splits a conditional word across the cut 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:45:58 +00:00
yassin
4fbe631146 test(content_filter): annotate streaming test locals as Final and type the logging metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 12:17:47 +00:00
yassin
60642e875b perf(content_filter): back off refused streamed buffer cuts by one context length 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:51:06 +00:00
yassin
7e429dee87 fix(content_filter): keep exception phrases and open conditional sentences in the streamed buffer 
Trimming the streamed buffer to the retained tail could drop a category
exception phrase that suppresses a later keyword, or the identifier word
of an unfinished sentence that a conditional category pairs with a later
block word. Refuse the cut while either would leave the buffer so the
bounded scan masks and blocks exactly like a scan of the full text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 11:18:10 +00:00
yassin
52f06906fe fix(content_filter): widen the streamed scan tail to the longest configured keyword 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 10:58:30 +00:00
yassin
62ecb11ab9 perf(content_filter): scan a bounded window per streamed chunk 
The streaming post-call hook rescanned the whole accumulated choice buffer on every chunk, so scan cost grew quadratically with output length. Keep a bounded per-choice buffer instead: once it exceeds twice the scan context, drop the head when masking the head and tail separately yields the same output as masking the whole buffer, so no pattern, phrase or exception straddles the cut. Detections from the dropped head are kept and merged, deduplicated, into the final log row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 09:59:53 +00:00
yucheng
3628025aae fix(proxy): keep caller metadata.trace_id ahead of the OTel fallback on litellm_metadata routes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 08:30:54 +00:00
yucheng
898fbd37a7 test(proxy): mark locals Final in the OTel trace id fallback tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 08:02:04 +00:00
yucheng
4cb4493fa7 fix(proxy): let the OTel trace id fallback fill a null litellm_trace_id 
A body that serializes litellm_trace_id as null or an empty string carries no identity, so it must not
block the server span fallback. Also mark the nested metadata write as an out-param store

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:48:58 +00:00
yucheng
49417d4fa2 fix(proxy): ignore non-span parent_otel_span when deriving litellm_trace_id 
UserAPIKeyAuth.parent_otel_span is Any at runtime (opentelemetry is an optional extra), so the OTel
trace-id fallback must only format an int trace id, otherwise an object that merely quacks like a span
turns the whole request into a 500

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:41:48 +00:00
yucheng
f1fd1c8996 fix(proxy): default litellm_trace_id to the OTel server span trace id 
When the otel callback is enabled and the client sends no trace or session identity, the request now inherits the W3C trace id of the proxy's server span as litellm_trace_id and metadata.trace_id. The missing_session_id policy and SpendLogs then persist that value as session_id, so a trace in the OTel backend and its row in the Logs UI carry the same id. Explicit x-litellm-trace-id, traceparent, body metadata.trace_id and litellm_trace_id keep priority.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:22:03 +00:00
Joshua Valluru
ee676d59f2 fix(mcp): preserve browser OAuth for unrelated bearer tokens 2026-09-15 23:08:14 -07:00
Joshua Valluru
e035682ed1 refactor(auth): separate JWT identity and OAuth authorization 2026-09-15 22:15:57 -07:00
yassin
252c69b532 fix(router): snapshot shadow kwargs per target so concurrent shadows never share metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 04:31:43 +00:00
tin-berri
a8979fe054
Merge pull request #41371 from BerriAI/litellm_forecast_simplify_advanced_ui 
fix(ui): simplify Capability and Fuse advanced routing options
 2026-09-15 21:19:50 -07:00
kerry-berri
5960881640
Merge pull request #41337 from BerriAI/litellm_fix_responses_stream_absent_usage_recount 
fix(responses): recount tokens when a streamed response completes without usage
 2026-09-15 21:19:42 -07:00
yuneng-jiang
bbffddd517
Merge pull request #41366 from BerriAI/litellm_/back-002-litellm-e2e-replay-8de5b1 
fix(e2e): record cookie-setting provider responses and keep prompt-caching tests live
 2026-09-15 21:18:14 -07:00
yuneng-jiang
e3c1f78e28
Merge branch 'main' into litellm_/back-002-litellm-e2e-replay-8de5b1 2026-09-15 21:08:29 -07:00
yassin
0f7ed4433b fix(router): snapshot shadow kwargs before fan-out so shadows never see primary mutations 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 04:04:01 +00:00
Tin Chi Lo
9d640b86ed fix(ui): simplify Capability and Fuse routing options 2026-09-15 20:58:38 -07:00
kerry
ff878e7df0 refactor(responses): copy the terminal event instead of mutating stubbed chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:54:04 +00:00
yassin
9cabde90dd test(router): cover _run_silent_experiment directly for router coverage gate 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:48:32 +00:00
kerry
7cc07d437a fix(responses): build the billed terminal response immutably and guard the cache dump 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:47:03 +00:00
berriai-litellm-provider-info-sync[bot]
641dcd5f8f
chore(prices): sync Google Gemini prices: 1 model 
gemini/gemini-robotics-er-2-preview: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost, input_cost_per_token_batches, output_cost_per_token_batches
 2026-09-16 03:46:34 +00:00
yuneng-jiang
174c1ac4ed
Merge pull request #41348 from BerriAI/litellm_fix_models_reload_test 
fix(e2e): expect models filters to persist after reload
 2026-09-15 20:31:47 -07:00
yassin
06fcc1f733 feat(router): stream shadow traffic and fan out silent_model to multiple targets 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:28:08 +00:00
yuneng-jiang
9d54c2e159
Merge pull request #41359 from BerriAI/litellm_fix_budget_reset_decrement_tests 
test(proxy): assert budget resets decrement the cleared spend
 2026-09-15 20:27:35 -07:00
yuneng-jiang
ef06790e39
Merge pull request #41358 from BerriAI/litellm_fix_stale_router_tests 
test(router): ignore deployment-selection logs in the fallback log assertion
 2026-09-15 20:27:21 -07:00
Yuneng Jiang
a6fb21c3f8
fix(e2e): record cookie-setting provider responses and keep prompt-caching tests live 
The first cache-enabled litellm-e2e build (211) showed three gaps in the shared provider cache:

Every OpenAI response carries Cloudflare bot-management Set-Cookie headers, and the capture rejected any response with Set-Cookie, so no OpenAI response was ever recorded (179 of 372 misses rejected). The edge already withholds Set-Cookie from the proxy, so drop it before validating and storing instead of rejecting.

The provider prompt-caching tests need fresh provider state: a replayed priming response reports cache creation rather than a cache read, and the TPM test then trips the key limit. Mark both modules provider_live.

TestApiBaseSeam::test_live_mode_returns_none ran inside the cache-enabled runner and saw the shared edge; isolate it from E2E_PROVIDER_CACHE.
 2026-09-15 20:26:20 -07:00
kerry-berri
4b84fa9230
Merge pull request #41336 from BerriAI/litellm_fix_anthropic_stream_absent_usage 
fix(anthropic): tolerate message_delta events without usage when streaming
 2026-09-15 20:23:01 -07:00
Yuneng Jiang
8c046e13bd
test(together_ai): move request-shape checks to the mapped file, drop the live ones 
Together moved openai/gpt-oss-20b off serverless and three tests in
test_completion.py died on a live 400. None of them needed Together to be up:
streaming is already covered live by tests/e2e/llm_translation/test_together_ai_e2e.py,
which picks its model from the cost map instead of pinning one, and the other
two are request-shape questions. Delete all three and assert the two shapes in
the mapped transformation file: the provider prefix is stripped without eating
the rest of a slashed model name, and custom role wrappers never reach the
request.
 2026-09-15 20:17:42 -07:00
tin-berri
3b14631f06
Merge pull request #41315 from BerriAI/litellm_capability_fuse_classifier_ui 
feat(ui): configure capability and Fuse v2 classifiers
 2026-09-15 20:13:57 -07:00