scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-23 00:41:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 99
50106 commits 2669 branches 1413 tags 3.3 GiB
Commit graph

12335 commits

Author SHA1 Message Date
yucheng
fc77914df3 test(proxy): type the moderation override stub in hook detection tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-17 01:59:04 +00:00
yucheng
b1255a6f2c fix(proxy): run prompt injection heuristics off the event loop and dispatch llm_api_check moderation 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-17 01:33:54 +00:00
yucheng-berri
0add8c0083
Merge pull request #41495 from BerriAI/litellm_converted_stream_post_call_hook 
fix(utils): run post-call deployment hook on converted chat streams
 2026-09-16 17:49:14 -07:00
Yassin Kortam
617a40bb1c
Merge pull request #40842 from BerriAI/litellm_guardrail_tag_budget_enforcement 
fix(proxy): enforce tag budgets for tags added by guardrails
 2026-09-16 17:44:29 -07:00
kerry-berri
db408f68ae
Merge pull request #41494 from BerriAI/litellm_auto_merge_price_sync 
ci: auto-merge provider-info-sync PRs when CI, Greptile and Bugbot are clean
 2026-09-16 17:36:02 -07:00
yucheng
e1cce943de Merge remote-tracking branch 'origin/main' into litellm_converted_stream_post_call_hook 2026-09-17 00:31:24 +00:00
Mateo Wang
9e1eb546e4
Merge pull request #41514 from BerriAI/litellm_mcp_api_key_static_header_slot 
fix(mcp): count admin static headers as api_key credential slots
 2026-09-16 17:22:08 -07:00
kerry-berri
2e4840ee18
Merge pull request #41509 from BerriAI/litellm_mantle_gpt5_verbosity 
fix(bedrock_mantle): accept and forward verbosity on gpt-5.x chat completions
 2026-09-16 17:20:33 -07:00
Mateo Wang
d4a22acb66
Merge pull request #41513 from BerriAI/litellm_internal_copy_31400 
fix(bedrock): neutralize orphaned tool blocks instead of raising or injecting a dummy tool (internal copy of #31400)
 2026-09-16 17:12:35 -07:00
ryan-crabbe-berri
43713f7508
Merge pull request #39996 from BerriAI/litellm_team_admin_editable_fields 
feat(proxy): let proxy admins choose which team fields team admins may edit
 2026-09-16 17:07:13 -07:00
kerry
0a47fe160d Merge remote-tracking branch 'origin/main' into litellm_mantle_gpt5_verbosity 2026-09-17 00:06:50 +00:00
Mateo Wang
09a188b583
Merge pull request #41094 from BerriAI/litellm_model_group_info_proxy_admin_all_models 
fix(proxy): show all model groups to proxy admins in /model_group/info
 2026-09-16 17:06:44 -07:00
Mateo Wang
913ef6ed49
Merge pull request #33856 from BerriAI/litellm_azure_ai_responses_native 
fix(azure_ai): route Responses API to native /openai/v1/responses for Foundry Models
 2026-09-16 17:06:29 -07:00
kerry-berri
f93b31679b
Merge pull request #41503 from BerriAI/litellm_fix_interrupted_anthropic_reasoning_usage 
fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content
 2026-09-16 17:04:59 -07:00
Mateo Wang
6cdf398bea
Merge pull request #41504 from BerriAI/litellm_bedrock_agent_runtime_strip_virtual_key 
fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough
 2026-09-16 17:02:17 -07:00
kerry
a3d3fe4ada fix(ci): pin the auto-merge request to the evaluated head sha 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:57:44 +00:00
kerry
42bf9f0ece Merge remote-tracking branch 'origin/main' into litellm_auto_merge_price_sync 2026-09-16 23:56:48 +00:00
yassin
d6b13f938d test(proxy): cover guardrail tag budget edge cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:52:39 +00:00
yassin
488666ccae fix(proxy): enforce tag budgets for tags added by guardrails 
Auth runs the tag budget check before pre_call_hook, so a tag that a custom guardrail adds is attributed spend but never budget checked. After the pre-call hook, budget check only the newly added tags with the same exemptions auth applied (budget-free routes, zero-cost models), keep the pre-guardrail tag baseline across fallback retries, and surface an over-budget tag as the same budget_exceeded 429 auth returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:52:39 +00:00
Mateo Wang
319f427c40
Merge pull request #41201 from BerriAI/litellm_gemini_37_38_flash_no_minimal_thinking 
fix(gemini): map minimal thinking to low for Gemini 3.7 and 3.8 Flash
 2026-09-16 16:52:08 -07:00
mateo-berri
ada0a1ad3a fix(azure_ai): strip the azure_ai/ prefix when a Responses call is remapped to azure 
A catalog OpenAI name on an .openai.azure.com host (or with AZURE_AI_API_BASE set to one) is remapped from azure_ai to azure before the Responses request is built, and the azure_ai/ prefix stayed in the wire model, so Azure answered DeploymentNotFound. The Azure Responses config now strips azure_ai/ next to responses/ and o_series/.
 2026-09-16 16:49:29 -07:00
yucheng-berri
c5325b1492
Merge pull request #40596 from BerriAI/litellm_lit_7470_rate_limit_fallback_pristine_data 
fix(proxy): retry rate-limit fallbacks from a pristine request snapshot
 2026-09-16 16:47:35 -07:00
yucheng-berri
672f43fd54
Merge pull request #41356 from BerriAI/litellm_lit7836_call_id_endpoint_logs 
fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses
 2026-09-16 16:43:12 -07:00
mateo-berri
574ea15b8f fix(mcp): count admin static headers as api_key credential slots 
An api_key server whose key lives in static_headers, the documented
shape for upstreams that expect a custom header name, dispatched fine
before the fail-closed check and was rejected as misconfigured after
it. The check now treats every static header the admin configured as a
credential slot for api_key mode, on both the MCP client path and the
OpenAPI tool path, with regression tests at all three layers.
 2026-09-16 16:42:20 -07:00
mateo-berri
41737aeda8 test(proxy): drop the docstring from the agent-runtime passthrough regression class 2026-09-16 16:37:32 -07:00
kerry
e8f246bb6b fix(responses_bridge): forward verbosity as text.verbosity 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:32:32 +00:00
Mateo Wang
390ab45448
Merge pull request #37506 from BerriAI/litellm_dashscope_reasoning_effort
Some checks are pending
LiteLLM Rust / rust-wheel (push) Waiting to run
Details
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Details
Unit Tests / proxy-server (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Details
Unit Tests / caching-local (push) Waiting to run
Details
Unit Tests / core-utils (push) Waiting to run
Details
Unit Tests / enterprise-package (push) Waiting to run
Details
Unit Tests / enterprise-routing (push) Waiting to run
Details
Unit Tests / integrations (push) Waiting to run
Details
Unit Tests / All Other Providers (push) Waiting to run
Details
Unit Tests / Vertex AI (push) Waiting to run
Details
Unit Tests / misc (push) Waiting to run
Details
Unit Tests / proxy-auth (push) Waiting to run
Details
Unit Tests / proxy-endpoints (push) Waiting to run
Details
Unit Tests / proxy-extras (push) Waiting to run
Details
Unit Tests / proxy-infra (push) Waiting to run
Details
Unit Tests / responses-caching-types (push) Waiting to run
Details
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Details 
fix(dashscope): forward reasoning_effort to the provider
 2026-09-16 16:29:25 -07:00
kerry
884e96958c fix(streaming): cap estimated reasoning tokens to the provider total and cover dict chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:27:48 +00:00
kerry
15c18ad6cd fix(bedrock_mantle): accept verbosity on gpt-5.x chat completions 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:27:07 +00:00
joshua-berri
41410e9556
Merge pull request #41364 from BerriAI/litellm_fix_mcp_auth_fail_closed_4501 
fix(mcp): fail closed on missing upstream credentials
 2026-09-16 23:26:38 +00:00
mateo-berri
3712d8de92 Merge remote-tracking branch 'origin/main' into litellm_model_group_info_proxy_admin_all_models
Some checks failed
LiteLLM Rust / rust-wheel (push) Has been cancelled
Details
LiteLLM Rust / rust-lint (push) Has been cancelled
Details
LiteLLM Rust / rust-test (push) Has been cancelled
Details
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Details
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Details
2026-09-16 16:18:45 -07:00
mateo-berri
2a9fa48730 fix: expand wildcard deployments for proxy admins on /model_group/info 2026-09-16 16:18:44 -07:00
mateo-berri
a1ad95dbbd fix(gemini): read the minimal thinking floor from the cost map and cover the /v1/messages bridge
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
Details
LiteLLM Rust / rust-lint (push) Has been cancelled
Details
LiteLLM Rust / rust-test (push) Has been cancelled
Details
2026-09-16 16:13:41 -07:00
kerry
43514a7ffe Merge remote-tracking branch 'origin/main' into litellm_fix_interrupted_anthropic_reasoning_usage 2026-09-16 23:13:06 +00:00
kerry
02bccfd89f fix(streaming): fill text_tokens when reasoning is counted from stream content 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:13:05 +00:00
ryan-crabbe-berri
81ae5caa7e fix(proxy): pass only a team admin's changed fields on to the team update 2026-09-16 16:11:52 -07:00
mateo-berri
610249c663 fix(bedrock): carry the tool_call_id into the rewritten tool call text 2026-09-16 16:08:46 -07:00
yuneng-jiang
030901211e
Merge pull request #41360 from BerriAI/litellm_unpin_together_ai_serverless_model 
test(together_ai): move request-shape checks to the mapped file, drop the live ones
 2026-09-16 16:07:33 -07:00
ryan-crabbe-berri
be6e7ef173
Merge pull request #41345 from BerriAI/litellm_remove_duplicate_user_budget_hook 
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
 2026-09-16 16:06:41 -07:00
yucheng-berri
659c4ce5a1
Merge pull request #40669 from BerriAI/litellm_otel_passthrough_trace_propagation 
fix(otel): propagate W3C trace context on HTTP and WebSocket passthrough
 2026-09-16 16:02:46 -07:00
mateo-berri
02d9aae8c8 fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough 
The agent-runtime branch of /bedrock/{endpoint} (agents, knowledgebases, flows,
retrieveAndGenerate, rerank, generateQuery, optimize-prompt) forwarded every
caller header to AWS next to the SigV4 signature, so a LiteLLM key presented in
x-api-key or x-litellm-api-key reached bedrock-agent-runtime verbatim. Build the
upstream header set explicitly: drop LiteLLM credential headers by name and any
authenticated secret by value, keep the remaining caller headers, and let the
signed headers win on collisions.
 2026-09-16 16:01:52 -07:00
mateo-berri
8045794c37 Merge remote-tracking branch 'origin/main' into litellm_dashscope_reasoning_effort 2026-09-16 15:59:08 -07:00
yuneng-jiang
ceaa0adef8
Merge branch 'main' into litellm_unpin_together_ai_serverless_model 2026-09-16 15:57:17 -07:00
ryan-crabbe-berri
36eb9cdb35 test(proxy): drop the route-list membership test that the PATCH gate tests already cover 2026-09-16 15:56:27 -07:00
ryan-crabbe-berri
c37d0a2e66 fix(proxy): read general_settings without a cast and test the /team/update gate by behavior only 2026-09-16 15:51:04 -07:00
kerry
4b70696afa fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content 
Interrupted Anthropic streams that die before message_delta were billed at
the message_start placeholder (any value above 1 was trusted) or at 0 when
the partial response was reasoning-only, because the token_counter fallback
only looked at visible text. Reset the placeholder whenever no finish_reason
or second usage event arrived, fold the already-counted reasoning tokens into
the fallback estimate, and drop the stale completion_tokens_details so cost
is computed from the recovered count

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 22:49:25 +00:00
ryan
dae16264c1 test(auth): cover over-budget user on zero-cost vs paid model in common_checks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 15:42:15 -07:00
ryan
287bbaa6c1 fix(proxy): remove duplicate user budget hook that 429'd zero-cost models 
_PROXY_MaxBudgetLimiter re-checked spend:user:{id} against user_max_budget in
async_pre_call_hook without the zero-cost model exemption that
_user_max_budget_check applies in auth, so free models were rejected with
"Max budget limit reached." once a user was over budget. Auth already owns
this check, so the hook is deleted rather than taught the exemption again

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 15:42:15 -07:00
ryan-crabbe-berri
6e2ae19670 fix(proxy): enforce org budget ceilings on /team/update 
update_team loaded the org without its budget row, so the org max_budget,
tpm_limit and rpm_limit checks silently passed. It now loads the budget the
same way /team/new does
 2026-09-16 15:38:30 -07:00
ryan-crabbe-berri
a44a58e91a feat(proxy): let team admins edit tpm_limit when a proxy admin enables it 
tpm_limit is the first field in the team-admin allow-list registry. The team
settings tab gives a team admin a form with only the enabled fields and sends
only those on save, and UI Settings labels the checkbox the same way
 2026-09-16 15:38:30 -07:00