Commit graph

16473 commits

Author SHA1 Message Date
mateo
a7836ede15 fix(models): absorb open registry PRs: govcloud bedrock and mantle, azure gov, openai tiered long-context, scaleway, together qwen3.8, azure ai cache and kimi k2.7 code, azure mai deprecations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 14:48:51 +00:00
michelligabriele
f0f6ff8de4
test(router): cover _widest_configured_limit directly for the coverage gate 2026-09-02 14:12:23 +02:00
mateo
25c5f0d993 test: deflake JWT tamper assertions and fuzzy picker widget driver
Tamper tests rewrote the last two base64url characters of the signature,
which on roughly 1 in 250 RS256 tokens (1 in 1000 HS256) only touched
padding bits, so the decoded signature was unchanged and still verified.
Corrupt the decoded signature bytes instead.

The fuzzy picker driver sent keys after fixed sleeps, so a slow worker
could receive the filter text before the widget had highlighted the match.
Wait on the widget's highlighted choice instead.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 10:05:50 +00:00
michelligabriele
d5270890c4
fix(proxy): report the widest window across a model group, not the first deployment's 2026-09-02 12:03:36 +02:00
michelligabriele
c60c60e6fe
fix(proxy): resolve /v1/models limits from the deployment, not the alias 2026-09-02 10:59:18 +02:00
mateo
385957e830 fix: reject constructor-managed router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 07:49:25 +00:00
mateo
77aa22e065 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_techdebt_20260901 2026-09-02 07:44:53 +00:00
Devin AI
2d643a8683 chore: merge litellm_internal_staging into litellm_realtime_router_session_model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:37:16 +00:00
mateo
974b331a4d fix: accept persistable router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:15:52 +00:00
mateo-berri
eb49135f2a test(e2e): remove the throwaway pathspec probe 2026-09-01 22:39:47 -07:00
mateo-berri
7cde2cd77f test(guardrails): type the presidio sibling test helpers precisely 2026-09-01 22:39:17 -07:00
mateo-berri
9d6b8bb14a test(e2e): throwaway top-level probe with a type error to prove the gate now runs 2026-09-01 22:33:16 -07:00
mateo-berri
b0aafabdee ci(lint): gate top-level tests/e2e and litellm files in the diff-scoped lint steps
Without :(glob), git matches 'tests/e2e/**/*.py' with * crossing slashes, so the
pattern needs at least one directory below tests/e2e and a top-level file never
matches. PR #39209 added tests/e2e/test_junit_properties.py with three
basedpyright errors and the e2e step printed "No changed tests/e2e Python files;
skipping." The ruff format step's 'litellm/**/*.py' skipped litellm/main.py and
the other top-level modules the same way.

:(glob) makes /**/ match zero or more directories, so both gates now select
top-level and nested files alike. A regression test runs the workflow's own
pathspecs against a throwaway repo and locks that in for every diff-scoped gate.
2026-09-01 22:30:46 -07:00
mateo-berri
c18511be7d fix(guardrails): track and tear down presidio sibling callbacks
initialize_presidio registers up to three callbacks per guardrail but the
registry only kept the first, so deleting or re-syncing the guardrail left
the post_call siblings serving the old config. The initializer now returns
every callback it registered, the registry tracks primary and siblings per
guardrail id, delete purges all of them from every callback list, and
update pushes the new params into each while siblings keep their stage.
2026-09-01 22:28:38 -07:00
mateo-berri
4a5d0b8163 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_headroom_ccr_streaming_responses
# Conflicts:
#	litellm/llms/openai/responses/guardrail_translation/handler.py
#	tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_handler.py
2026-09-01 22:25:59 -07:00
Mateo Wang
31ca4ddf32
Merge pull request #39166 from BerriAI/litellm_bedrock_bearer_token_converse_crash
fix(bedrock): stop Converse crashing on bearer-token auth without SigV4 credentials
2026-09-01 22:24:30 -07:00
mateo-berri
b45670b4b4 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_mcp_test_connection_oauth_bearer 2026-09-01 22:23:03 -07:00
mateo-berri
222cb293ca fix(mcp): name the access group that resolved when an agent vetoes a mixed scope 2026-09-01 22:14:18 -07:00
mateo-berri
c09db7c7a3 Fail closed on rewrites for buffers that never reached their terminal event
An Anthropic buffer without a stop_reason only ran the flat text scan, so a
rewrite there was dropped while the executor trusted the translation to have
delivered it. A Responses buffer ending at response.output_item.done returned
after the tool-call scan without ever checking the text. Both now reach the
flat scan and raise UndeliverableStreamRewrite when a caller expects the
rewrite delivered, matching the existing Responses no-envelope fallback.
2026-09-01 22:01:04 -07:00
mateo-berri
681eb0eced Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants 2026-09-01 21:58:15 -07:00
James Liounis
e2c3f51c46
fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping (#37883)
* fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping

SearchAPIRouter dropped every parameter configured on a search tool, forwarding
only per-request kwargs. Any tool-level setting (mode, max_results, ...) was
silently lost on the way to the adapter, for every search provider.

Also completes the Parallel AI v1 search surface: after_date, fetch_policy,
location and include_domains now nest under advanced_settings instead of being
sent as unknown top-level fields, responses preserve search_id / session_id /
warnings / raw excerpts, and search cost is derived from the request mode and
the provider's reported usage rather than a single flat rate.

* fix(parallel_ai): stop a caller from pricing its own search request

`_parallel_ai_usage` carries the provider's reported usage into cost
calculation. It was only written when the response contained a usage block, so
a caller could pass `_parallel_ai_usage=[{"name": "sku_search", "count": 0}]`
and, whenever the provider omitted usage, bill $0.00 instead of $0.005 — the
value also reached the upstream request body as an unknown field.

The key is now stripped from inbound params and written unconditionally from
the parsed response, so only the provider can populate it.

* fix(parallel_ai): price fast search mode correctly

* test(parallel_ai): fake search at HTTP boundary

* fix(parallel_ai): tolerate null search result fields

---------

Co-authored-by: khushishelat <shelatkhushi@gmail.com>
2026-09-01 21:46:46 -07:00
moe-berri
6adc14b4b1 fix(router): preserve Claude subagent fallbacks 2026-09-01 21:13:45 -07:00
moe-berri
8a0967443d fix(router): isolate Claude session binding cache 2026-09-01 20:38:13 -07:00
mateo
1d3e26fd98 fix: preserve shared optional callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 03:28:04 +00:00
mateo
70a4f74a0d test: allow callback state fixture mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:56:23 +00:00
mateo
3dea586d3b test: cover runtime callback reconciliation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:48:09 +00:00
mateo
7b86b7f4cd test: isolate router callback state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:45:45 +00:00
mateo
e67f98feb1 fix: reconcile runtime pre-call checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:44:17 +00:00
mateo
29f0110fe0 test: pass request to config update test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:29:41 +00:00
mateo
cb511f70cc fix: preserve config update authorization order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:20:51 +00:00
mateo
0608f0a00f fix: reject unknown runtime router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:17:02 +00:00
mateo-berri
3c2138f037 fix(mcp): count agent toolsets in the agent MCP ceiling and tool grants
An agent's object_permission.mcp_toolsets could be persisted through the
new edit form and PATCH /v1/agents but never reached the request-time
checks: _get_allowed_mcp_servers_for_agent read only mcp_servers and
mcp_access_groups, so an agent granted a toolset alone resolved to [] and
placed no ceiling on its keys, and _get_agent_tool_permissions_for_server
ignored the tools those toolsets grant. Both helpers now resolve toolsets
through the shared _toolset_tool_permissions / _toolset_tools_for_server
helpers the key, team, and org levels already use, and a declared toolset
that resolves to nothing raises UnloadableEntitlementError so the
resolver denies instead of reading the agent as unrestricted
2026-09-01 19:12:24 -07:00
mateo-berri
f49a3e15a8 test(e2e): read JUnit properties off the real collected pytest Item
tests/e2e/test_junit_properties.py fed a hand-rolled FakeItem to
result_properties and attach_result_properties, both typed pytest.Item,
so uv run basedpyright tests/e2e reported 3 reportArgumentType errors on
litellm_internal_staging and every make check that scopes a litellm/ or
tests/e2e/ Python file failed.

Each test now looks up its own collected Item in request.session.items
and applies the covers marker at run time through request.applymarker,
so the coverage registry's collect-only pass never sees the test ids and
the production functions keep their pytest.Item signatures. No casts, no
ignores.

Resolves LIT-6669
2026-09-01 19:12:07 -07:00
mateo-berri
863199c09b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_bearer_token_converse_crash 2026-09-01 19:07:43 -07:00
mateo-berri
2d7697569a Merge branch 'litellm_fix_post_call_policy_pipeline' into litellm_post_call_pipeline_stream_rewrite
Brings the stack base up to origin/litellm_internal_staging. Staging's
get_streaming_string_so_far now joins delta-only Responses parts itself, so
the handler's separate delta joiner and the flag-gated delta scan go away
and the fallback keeps failing closed on a rewrite it cannot deliver. The
two tests that pinned the old flag-gated behaviour are dropped in favour of
staging's delta-assembly tests, and _process_ended_stream takes the same
UserAPIKeyAuth | None the typed process_output_response now expects
2026-09-01 19:04:22 -07:00
mateo-berri
de0a4c1b0a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_stream_usage_cost_default
# Conflicts:
#	litellm/responses/streaming_iterator.py
2026-09-01 18:53:50 -07:00
mateo-berri
6eee2f080c Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_post_call_policy_pipeline 2026-09-01 18:52:23 -07:00
mateo-berri
31a3c55737 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_guardrail_event_hook_resync 2026-09-01 18:50:05 -07:00
mateo-berri
0e7a05d878 test(bedrock): read the sent bearer header off the mock instead of a hand-rolled recorder 2026-09-01 18:46:41 -07:00
Mateo Wang
92d453373a
Merge pull request #39036 from BerriAI/litellm_fix_stream_modify_response_chunks
fix(guardrails): deliver modify_response block as valid SSE on streaming chat and Responses
2026-09-01 18:44:10 -07:00
mateo-berri
2286091a9a fix(guardrails): skip None fields in in-memory guardrail updates so constructor defaults survive 2026-09-01 18:38:38 -07:00
mateo-berri
e4c6badca2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants 2026-09-01 18:37:31 -07:00
mateo-berri
659eb0462e fix(mcp): deny every zero-resolution scoped request uniformly and persist agent MCP toolsets from the UI
Any scoped MCP request (/mcp/<name> path or x-mcp-servers header) that resolves to zero allowed servers now returns one generic 403, so unknown, unauthorized, and access-group names are indistinguishable and the error cannot be used to enumerate servers. The agent-attributed variant reruns the same scope resolver with the agent binding stripped and fires only when that rerun resolves, naming the vetoed server or access group and both fix paths

AgentObjectPermission now declares mcp_toolsets so PATCH /v1/agents keeps it, and the agent edit form round-trips toolsets alongside servers and access groups instead of dropping them on save. The two scope-resolver params only iterate their names, so they take Sequence[str] and the LIT001 budget ratchets down by the two annotations this widens
2026-09-01 18:37:23 -07:00
mateo-berri
8c72342ad5 fix(guardrails): resync event_hook and accept raw dicts in in-memory guardrail updates 2026-09-01 18:25:34 -07:00
Moe Khalil
46502f5804 Revert "fix(router): keep serving when Claude Code session router cleanup fails"
This reverts commit 2d4301589c.
2026-09-02 01:11:23 +00:00
Moe Khalil
2d4301589c fix(router): keep serving when Claude Code session router cleanup fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 01:10:31 +00:00
yucheng-berri
62f032cca5
fix(proxy): keep passthrough logging metadata and model_info dicts when team callbacks are wired (#39216)
* fix(proxy): keep passthrough logging metadata and model_info dicts when team callbacks are wired

Passing team callback vars into Logging(kwargs=...) makes get_litellm_params materialize a full litellm_params, where metadata and model_info default to None instead of being absent. Readers that resolve them as .get(key, {}).get(...) then raise, so any passthrough request from a team with logging callbacks 500s once a pre-call guardrail is on, and the router strategy loggers log a traceback per request.

* test(proxy): annotate the closure dicts the passthrough logging tests record into
2026-09-01 18:07:04 -07:00
Mateo Wang
364c9be2e0
Merge pull request #39211 from BerriAI/litellm_fix_default_api_key_limit_leak
fix: stop deployment default API key limits leaking into provider requests
2026-09-01 18:06:43 -07:00
yucheng-berri
4b87fd5718
fix: normalize provider-specific cache token fields in OTel v2 usage (#39202)
* fix: normalize provider-specific cache token fields in OTel v2 usage

* fix: use an immutable empty mapping for the cache token details fallback

* fix: ignore malformed cache token values instead of emitting or raising
2026-09-01 18:06:35 -07:00
devin-ai-integration[bot]
2b616fc479
feat(scim): add placeholder listing and merge so a shadowed account can be healed (#39231)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 01:04:53 +00:00