Commit graph

49820 commits

Author SHA1 Message Date
yuneng-jiang
1be89d28b8
Merge pull request #39990 from BerriAI/litellm_e2e_jwt_harness
test(e2e): reusable JWT fixtures and management lifecycle coverage
2026-09-11 21:18:59 -07:00
yucheng
13596700ec fix(mcp): challenge Agent 365 connects on the mounted /mcp/{server} route
Starlette's /mcp Mount moves the prefix into root_path and leaves the app root in
app_root_path, which is an empty string on a default deployment. Reading it with
`or` fell through to root_path=/mcp and stripped the prefix, so the route-relative
path became /{server} and the per-server connect challenge was skipped there.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 04:12:12 +00:00
yucheng
3c536bfc9a fix(mcp): keep server-wide listed tool metadata across per-user OAuth token invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 04:00:47 +00:00
yucheng
485f4522e4 fix(mcp): scope the Agent 365 connect challenge to single-server routes and drop stale listed tools on invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 03:52:06 +00:00
Mateo Wang
7ad6c628de
Merge pull request #40773 from BerriAI/litellm_e2e_memory_regression_failing_requests
test(e2e): memory regression test for failing requests on the release gate
2026-09-11 20:46:33 -07:00
mateo-berri
5ab6e0858c fix(health): resolve a model name the way a request routes before matching the provider model string 2026-09-11 20:40:00 -07:00
yucheng
b1c3573390 Merge branch 'litellm_internal_staging' into litellm_agent365_mcp_guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 03:31:49 +00:00
Mateo Wang
99e14fc2e5
Merge pull request #40798 from BerriAI/litellm_lit7523_mantle_reasoning_summary
fix(bedrock_mantle): gate reasoning.summary on the OpenAI Responses path
2026-09-11 20:30:49 -07:00
joshua-berri
70cf348aa5
Merge pull request #40791 from BerriAI/litellm_fix_mcp_root_discovery_6634
fix(mcp): use gateway authentication for root discovery
2026-09-11 20:23:10 -07:00
joshua-berri
e86adf98ac
Merge pull request #40808 from BerriAI/litellm_fix_mcp_oauth_issuer_7078
fix(mcp): match per-server OAuth metadata issuers
2026-09-11 20:23:00 -07:00
joshua-berri
108cacdd2f
Merge pull request #40790 from BerriAI/litellm_fix_mcp_discovery_cache_6585
fix(mcp): cache upstream discovery lists
2026-09-11 20:16:36 -07:00
mateo-berri
eaf3d8ad3e fix(health): target a model name the way a request for it routes
A team's copies published under the name win, then deployments named that way, then a public name only another team's deployment carries (an admin reaches it, routing does too). The endpoint resolver and the live narrowing share one rule.
2026-09-11 20:12:39 -07:00
ryan-crabbe-berri
17fde7a261 refactor(realtime): move Meta Muse Voice onto BaseRealtimeConfig
Replace the hand-rolled Meta realtime handler with a MetaRealtimeConfig
that plugs into the shared realtime handler and RealTimeStreaming relay.
Clients keep speaking the OpenAI realtime wire: session.update,
input_audio_buffer.append/commit and the OpenAI transcription events.
Unsupported transcription settings are logged and dropped, matching the
Gemini realtime precedent, and the Meta-specific session.mode, keywords,
language_bias, DIARIZATION and speaker extensions are removed.

Drop the MODEL_API_KEY env var in favor of the standard META_API_KEY,
remove the private-logging flag so spend logs record the transcript the
same way other realtime models do, and add per-second pricing for
muse-voice-transcribe-1.0.

The relay now sends raw bytes from transform_realtime_request straight to
the backend after pace_backend_send, and transcription sessions never
trigger response.create.
2026-09-11 20:12:15 -07:00
Young Han
1acb994998 fix(realtime): bound Muse audio before decoding 2026-09-11 20:10:53 -07:00
Young Han
b82b31a44f feat(realtime): add Meta Muse Voice transcription 2026-09-11 20:10:53 -07:00
Mateo Wang
19c8553052
Merge pull request #40820 from BerriAI/litellm_sanitize_unknown_model_error_message
fix(proxy): keep the raw model string out of the unknown-model spend-log error message
2026-09-11 19:54:34 -07:00
mateo-berri
6209f0694b fix(health): keep team public names to the owning team and let model_id win over model
A public name a team publishes its own deployment copy under now targets that copy only for a caller from that team, so an admin or another team probing the shared name gets the global deployment alone

model_id wins when paired with model: a foreign id still gets the 403, and an id no deployment carries gets the 404 of the lone-id path, before any probe runs or a result is stored under it

cache_health_check_results accepts the Mapping sequences perform_health_check returns
2026-09-11 19:49:37 -07:00
mateo-berri
9ba7ec2964 test(e2e): keep the memory regression case in a class and drop the helper docstrings 2026-09-11 19:46:06 -07:00
ryan-crabbe-berri
e65b7ff0b8
Merge pull request #40659 from BerriAI/litellm_team_member_table_search_sort_filter
feat(ui): search, sort and role filter for the team member table
2026-09-11 19:37:29 -07:00
devin-ai-integration[bot]
98f6c14a76
test(pgbouncer): stop the never-listens replacement test flaking under CI load (#40830) 2026-09-11 19:32:56 -07:00
mateo-berri
a8ffc852f2 test(e2e): trim the reliability helper docstrings to the cooldown rationale 2026-09-11 19:32:40 -07:00
mateo-berri
9375719feb Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_e2e_memory_regression_failing_requests
# Conflicts:
#	tests/e2e/CLAUDE.md
#	tests/e2e/models.py
2026-09-11 19:31:11 -07:00
mateo-berri
6a96a47e9f style(proxy): drop the docstring bullet and wrap the sanitizer test call 2026-09-11 19:29:53 -07:00
ryan-crabbe-berri
c1aaaae4e3
Merge pull request #40831 from BerriAI/litellm_dismissible_env_credential_banner 2026-09-11 19:23:23 -07:00
Mateo Wang
55c34951e3
Merge pull request #40609 from csoni-cweave/fix/guardrail-len-mismatch
fix(guardrails): fail closed with a named error when a Responses input rewrite cannot be applied
2026-09-11 19:22:53 -07:00
ryan
2db51046d4 fix(ui): make the env-credential login warning banner dismissible
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 02:20:32 +00:00
mateo-berri
3dafa7aac1 test(proxy): expect the sanitized unknown-model message in the spend-log error test 2026-09-11 19:18:10 -07:00
Yuneng Jiang
dea10d4fdb
fix(ui): label explicit empty tool selections 2026-09-11 19:16:09 -07:00
devin-ai-integration[bot]
b5bf09d22d
fix(db): carry DATABASE_SSLMODE/DATABASE_SSLROOTCERT into the assembled writer and reader URLs (#40815)
* fix(db): carry DATABASE_SSLMODE/DATABASE_SSLROOTCERT into the assembled writer and reader URLs

The componentized gateway supervisor starts the in-container PgBouncer from the
DATABASE_URL assembled out of the discrete DATABASE_* vars before config.yaml is
read, so an IAM URL had no way to request verified TLS: PgBouncer dialed the
server with server_tls_sslmode = prefer (no SNI, no verification) and public
RDS endpoints rejected the handshake. Two new env vars, exposed by the chart as
database.writer.sslMode / sslRootCert, are appended as libpq sslmode/sslrootcert
to every writer and reader URL the settings assemble (never to a pinned URL),
then translated for Prisma as before. Token refresh now also carries Prisma's
sslmode/sslcert/sslaccept over into the re-minted URL

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(db): keep TLS params on the CLI password URL and the initial IAM reader mint

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(db): treat DATABASE_SSLROOTCERT on its own as verify-full and cover collector and migrations TLS env

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(db): type the reader mint TLS test double and drop its mutable capture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 02:14:39 +00:00
devin-ai-integration[bot]
1fde15c1ec
fix(shadow-eval): skip hosted web search samples (#40827)
(cherry picked from commit a78cd2fe02)

Co-authored-by: Tin Chi Lo <tin@berri.ai>
2026-09-12 02:11:48 +00:00
Yuneng Jiang
0587e3b8f8
test(ui): isolate search provider query state 2026-09-11 19:07:04 -07:00
Yuneng Jiang
707b779c5d
fix(ui): preserve clear and default semantics in local forms 2026-09-11 18:58:58 -07:00
yassin
28f2d1f016 test(redis): cover the write and list timeout paths going through the shared log throttle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 01:58:36 +00:00
devin-ai-integration[bot]
7057b2f6c4
fix(fireworks_ai): keep reasoning_content on replayed assistant messages (#40682)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 18:47:37 -07:00
mateo-berri
e4030597d8 test(guardrails): prove the structured write-back lands CrowdStrike redactions on Responses instructions and tool items 2026-09-11 18:47:18 -07:00
Mateo Wang
eed11ba59b
Merge pull request #40812 from BerriAI/litellm_fireworks_deepseek_v4p1_flash
feat(model_prices): add DeepSeek V4.1 Flash on Fireworks
2026-09-11 18:43:00 -07:00
mateo-berri
832da2950f fix(proxy): keep the raw model string out of the unknown-model spend-log error message 2026-09-11 18:42:08 -07:00
yassin
f681a978f0 fix(redis): use a monotonic clock for the timeout log throttle and trim test docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 01:37:36 +00:00
yuneng-jiang
cf97b757a4
fix(ui): preserve cleared shared select values (#40795)
Preserve explicit null when shared selectors clear and adapt affected forms, validation, and request payloads. Clear stale dependent relationships and retain required-selection checks. Document project detachment, user model-budget clearing, and routing-compression clearing as deferred follow-ups.
2026-09-11 18:37:08 -07:00
Mateo Wang
44ce8bb1ef
Merge pull request #40769 from BerriAI/litellm_azure_realtime_ga_default
fix(realtime): dial Azure's GA realtime upstream for GA clients
2026-09-11 18:34:20 -07:00
yassin
9c84e98fb2 fix(proxy): treat a Redis timeout in spend counter increments as an already-logged cache failure
The cost tracking callback logged its own ERROR with a traceback for every request whose spend counter increment timed out, on top of the cache layer's throttled line. Timeouts now take the same path as breaker-open refusals: invalidate the counters and return. Also exposes is_redis_timeout_failure publicly for that caller and drops the comment on the new constant

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 01:25:32 +00:00
devin-ai-integration[bot]
f84f986b4e
fix(guardrails): keep post_call guardrail info on streamed chat completions (#40806)
* fix(guardrails): sync logging_obj guardrail info on every record so post_call entries survive streamed chat completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(guardrails): hoist regression test imports to module scope

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 18:16:33 -07:00
mateo-berri
105dc77109 fix(realtime): probe Azure's GA realtime upstream in health checks when no protocol is pinned 2026-09-11 18:14:21 -07:00
ryan-crabbe-berri
e72d1cfa8d
Merge pull request #40572 from dclarksymmetry/fix/spend-counter-reseed-double-count
fix(proxy): prevent spend counter double counting
2026-09-11 18:14:04 -07:00
yassin
107b4ec4db fix(redis): log a timeout streak once per interval instead of one line per cache call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 01:12:39 +00:00
Maximilian Roos
8dd20af4c9
Merge branch 'litellm_internal_staging' into bugfix/http-handler-del-closes-streaming-client 2026-09-11 18:11:10 -07:00
Joshua Valluru
9d31de2f20 fix(mcp): bound discovery cache result bytes 2026-09-11 18:08:27 -07:00
ryan-crabbe-berri
50cd26cd9c
Merge pull request #40396 from jon-walton/litellm_user_budget_webhook_alerts
fix(proxy): emit internal user budget webhook alerts
2026-09-11 18:03:35 -07:00
ryan-crabbe-berri
0c9fda8c1e fix(proxy): gate the webhook test alert on proxy admins
/health/services?service=webhook fired a budget_crossed alert for the
caller's own user_id with any authenticated key. That alert writes the
same dedup cache entry the auth-time user budget alert uses, so a
non-admin could pre-populate it and suppress their real budget alert
for the cache TTL. Match the newrelic and pointfive branches and reject
non-admin callers with a 403 before the alert fires.
2026-09-11 18:00:13 -07:00
mateo
6c07876dcf test(model_prices): use local cost map for Fireworks cost coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 01:00:11 +00:00