Commit graph

48995 commits

Author SHA1 Message Date
michelligabriele
d5270890c4
fix(proxy): report the widest window across a model group, not the first deployment's 2026-09-02 12:03:36 +02:00
mateo-berri
f94bd6d903 refactor(typing): replace Any with proven types in 65 backend files
Typing-only pass over backend modules that carried the most reportAny and
reportExplicitAny errors. Every new annotation is backed by a construction
site, a call site, or an isinstance narrowing that already existed; untyped
JSON boundaries were left alone rather than declared without validation.

Tree-wide basedpyright errors drop 138,481 to 138,007. reportAny drops 8,854
to 8,645 and reportExplicitAny drops 3,119 to 2,814.
2026-09-02 09:11:36 +00:00
michelligabriele
c60c60e6fe
fix(proxy): resolve /v1/models limits from the deployment, not the alias 2026-09-02 10:59:18 +02:00
mateo
1400070d71 chore(techdebt): clear fresh debt from the 2026-09-01 window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 07:58:09 +00:00
mateo
385957e830 fix: reject constructor-managed router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 07:49:25 +00:00
mateo
77aa22e065 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_techdebt_20260901 2026-09-02 07:44:53 +00:00
mateo
aba9644297 fix: avoid router settings update name collision
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 07:10:04 +00:00
mateo
95e09db661 style: apply ruff format to router settings merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:56:27 +00:00
Devin AI
2d643a8683 chore: merge litellm_internal_staging into litellm_realtime_router_session_model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:37:16 +00:00
mateo
974b331a4d fix: accept persistable router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:15:52 +00:00
mateo-berri
eb49135f2a test(e2e): remove the throwaway pathspec probe 2026-09-01 22:39:47 -07:00
mateo-berri
7cde2cd77f test(guardrails): type the presidio sibling test helpers precisely 2026-09-01 22:39:17 -07:00
Mateo Wang
bca3bade40
Merge pull request #39233 from BerriAI/litellm_post_call_pipeline_stream_rewrite
fix(policy_engine): apply post_call pipeline text rewrites on streams
2026-09-01 22:35:55 -07:00
mateo-berri
9d6b8bb14a test(e2e): throwaway top-level probe with a type error to prove the gate now runs 2026-09-01 22:33:16 -07:00
mateo-berri
b0aafabdee ci(lint): gate top-level tests/e2e and litellm files in the diff-scoped lint steps
Without :(glob), git matches 'tests/e2e/**/*.py' with * crossing slashes, so the
pattern needs at least one directory below tests/e2e and a top-level file never
matches. PR #39209 added tests/e2e/test_junit_properties.py with three
basedpyright errors and the e2e step printed "No changed tests/e2e Python files;
skipping." The ruff format step's 'litellm/**/*.py' skipped litellm/main.py and
the other top-level modules the same way.

:(glob) makes /**/ match zero or more directories, so both gates now select
top-level and nested files alike. A regression test runs the workflow's own
pathspecs against a throwaway repo and locks that in for every diff-scoped gate.
2026-09-01 22:30:46 -07:00
mateo-berri
c18511be7d fix(guardrails): track and tear down presidio sibling callbacks
initialize_presidio registers up to three callbacks per guardrail but the
registry only kept the first, so deleting or re-syncing the guardrail left
the post_call siblings serving the old config. The initializer now returns
every callback it registered, the registry tracks primary and siblings per
guardrail id, delete purges all of them from every callback list, and
update pushes the new params into each while siblings keep their stage.
2026-09-01 22:28:38 -07:00
mateo-berri
4a5d0b8163 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_headroom_ccr_streaming_responses
# Conflicts:
#	litellm/llms/openai/responses/guardrail_translation/handler.py
#	tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_handler.py
2026-09-01 22:25:59 -07:00
Mateo Wang
31ca4ddf32
Merge pull request #39166 from BerriAI/litellm_bedrock_bearer_token_converse_crash
fix(bedrock): stop Converse crashing on bearer-token auth without SigV4 credentials
2026-09-01 22:24:30 -07:00
mateo-berri
b45670b4b4 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_mcp_test_connection_oauth_bearer 2026-09-01 22:23:03 -07:00
mateo-berri
222cb293ca fix(mcp): name the access group that resolved when an agent vetoes a mixed scope 2026-09-01 22:14:18 -07:00
mateo-berri
c09db7c7a3 Fail closed on rewrites for buffers that never reached their terminal event
An Anthropic buffer without a stop_reason only ran the flat text scan, so a
rewrite there was dropped while the executor trusted the translation to have
delivered it. A Responses buffer ending at response.output_item.done returned
after the tool-call scan without ever checking the text. Both now reach the
flat scan and raise UndeliverableStreamRewrite when a caller expects the
rewrite delivered, matching the existing Responses no-envelope fallback.
2026-09-01 22:01:04 -07:00
mateo-berri
681eb0eced Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants 2026-09-01 21:58:15 -07:00
James Liounis
e2c3f51c46
fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping (#37883)
* fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping

SearchAPIRouter dropped every parameter configured on a search tool, forwarding
only per-request kwargs. Any tool-level setting (mode, max_results, ...) was
silently lost on the way to the adapter, for every search provider.

Also completes the Parallel AI v1 search surface: after_date, fetch_policy,
location and include_domains now nest under advanced_settings instead of being
sent as unknown top-level fields, responses preserve search_id / session_id /
warnings / raw excerpts, and search cost is derived from the request mode and
the provider's reported usage rather than a single flat rate.

* fix(parallel_ai): stop a caller from pricing its own search request

`_parallel_ai_usage` carries the provider's reported usage into cost
calculation. It was only written when the response contained a usage block, so
a caller could pass `_parallel_ai_usage=[{"name": "sku_search", "count": 0}]`
and, whenever the provider omitted usage, bill $0.00 instead of $0.005 — the
value also reached the upstream request body as an unknown field.

The key is now stripped from inbound params and written unconditionally from
the parsed response, so only the provider can populate it.

* fix(parallel_ai): price fast search mode correctly

* test(parallel_ai): fake search at HTTP boundary

* fix(parallel_ai): tolerate null search result fields

---------

Co-authored-by: khushishelat <shelatkhushi@gmail.com>
2026-09-01 21:46:46 -07:00
mateo-berri
bf6aa9c390 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_hide_admin_tabs_view_only
# Conflicts:
#	ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/page.test.tsx
2026-09-01 21:31:39 -07:00
moe-berri
6adc14b4b1 fix(router): preserve Claude subagent fallbacks 2026-09-01 21:13:45 -07:00
moe-berri
8a0967443d fix(router): isolate Claude session binding cache 2026-09-01 20:38:13 -07:00
mateo
1d3e26fd98 fix: preserve shared optional callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 03:28:04 +00:00
mateo
70a4f74a0d test: allow callback state fixture mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:56:23 +00:00
mateo
3dea586d3b test: cover runtime callback reconciliation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:48:09 +00:00
mateo
7b86b7f4cd test: isolate router callback state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:45:45 +00:00
mateo
e67f98feb1 fix: reconcile runtime pre-call checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:44:17 +00:00
mateo
29f0110fe0 test: pass request to config update test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:29:41 +00:00
mateo
cb511f70cc fix: preserve config update authorization order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:20:51 +00:00
mateo
0608f0a00f fix: reject unknown runtime router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:17:02 +00:00
mateo-berri
3c2138f037 fix(mcp): count agent toolsets in the agent MCP ceiling and tool grants
An agent's object_permission.mcp_toolsets could be persisted through the
new edit form and PATCH /v1/agents but never reached the request-time
checks: _get_allowed_mcp_servers_for_agent read only mcp_servers and
mcp_access_groups, so an agent granted a toolset alone resolved to [] and
placed no ceiling on its keys, and _get_agent_tool_permissions_for_server
ignored the tools those toolsets grant. Both helpers now resolve toolsets
through the shared _toolset_tool_permissions / _toolset_tools_for_server
helpers the key, team, and org levels already use, and a declared toolset
that resolves to nothing raises UnloadableEntitlementError so the
resolver denies instead of reading the agent as unrestricted
2026-09-01 19:12:24 -07:00
mateo-berri
f49a3e15a8 test(e2e): read JUnit properties off the real collected pytest Item
tests/e2e/test_junit_properties.py fed a hand-rolled FakeItem to
result_properties and attach_result_properties, both typed pytest.Item,
so uv run basedpyright tests/e2e reported 3 reportArgumentType errors on
litellm_internal_staging and every make check that scopes a litellm/ or
tests/e2e/ Python file failed.

Each test now looks up its own collected Item in request.session.items
and applies the covers marker at run time through request.applymarker,
so the coverage registry's collect-only pass never sees the test ids and
the production functions keep their pytest.Item signatures. No casts, no
ignores.

Resolves LIT-6669
2026-09-01 19:12:07 -07:00
mateo-berri
863199c09b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_bearer_token_converse_crash 2026-09-01 19:07:43 -07:00
mateo-berri
2d7697569a Merge branch 'litellm_fix_post_call_policy_pipeline' into litellm_post_call_pipeline_stream_rewrite
Brings the stack base up to origin/litellm_internal_staging. Staging's
get_streaming_string_so_far now joins delta-only Responses parts itself, so
the handler's separate delta joiner and the flag-gated delta scan go away
and the fallback keeps failing closed on a rewrite it cannot deliver. The
two tests that pinned the old flag-gated behaviour are dropped in favour of
staging's delta-assembly tests, and _process_ended_stream takes the same
UserAPIKeyAuth | None the typed process_output_response now expects
2026-09-01 19:04:22 -07:00
mateo-berri
de0a4c1b0a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_stream_usage_cost_default
# Conflicts:
#	litellm/responses/streaming_iterator.py
2026-09-01 18:53:50 -07:00
mateo-berri
6eee2f080c Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_post_call_policy_pipeline 2026-09-01 18:52:23 -07:00
mateo-berri
31a3c55737 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_guardrail_event_hook_resync 2026-09-01 18:50:05 -07:00
mateo-berri
0e7a05d878 test(bedrock): read the sent bearer header off the mock instead of a hand-rolled recorder 2026-09-01 18:46:41 -07:00
Mateo Wang
92d453373a
Merge pull request #39036 from BerriAI/litellm_fix_stream_modify_response_chunks
fix(guardrails): deliver modify_response block as valid SSE on streaming chat and Responses
2026-09-01 18:44:10 -07:00
mateo-berri
2286091a9a fix(guardrails): skip None fields in in-memory guardrail updates so constructor defaults survive 2026-09-01 18:38:38 -07:00
mateo-berri
e4c6badca2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_agent_mcp_grants 2026-09-01 18:37:31 -07:00
mateo-berri
659eb0462e fix(mcp): deny every zero-resolution scoped request uniformly and persist agent MCP toolsets from the UI
Any scoped MCP request (/mcp/<name> path or x-mcp-servers header) that resolves to zero allowed servers now returns one generic 403, so unknown, unauthorized, and access-group names are indistinguishable and the error cannot be used to enumerate servers. The agent-attributed variant reruns the same scope resolver with the agent binding stripped and fires only when that rerun resolves, naming the vetoed server or access group and both fix paths

AgentObjectPermission now declares mcp_toolsets so PATCH /v1/agents keeps it, and the agent edit form round-trips toolsets alongside servers and access groups instead of dropping them on save. The two scope-resolver params only iterate their names, so they take Sequence[str] and the LIT001 budget ratchets down by the two annotations this widens
2026-09-01 18:37:23 -07:00
mateo-berri
8c72342ad5 fix(guardrails): resync event_hook and accept raw dicts in in-memory guardrail updates 2026-09-01 18:25:34 -07:00
Moe Khalil
46502f5804 Revert "fix(router): keep serving when Claude Code session router cleanup fails"
This reverts commit 2d4301589c.
2026-09-02 01:11:23 +00:00
Moe Khalil
2d4301589c fix(router): keep serving when Claude Code session router cleanup fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 01:10:31 +00:00
yucheng-berri
62f032cca5
fix(proxy): keep passthrough logging metadata and model_info dicts when team callbacks are wired (#39216)
* fix(proxy): keep passthrough logging metadata and model_info dicts when team callbacks are wired

Passing team callback vars into Logging(kwargs=...) makes get_litellm_params materialize a full litellm_params, where metadata and model_info default to None instead of being absent. Readers that resolve them as .get(key, {}).get(...) then raise, so any passthrough request from a team with logging callbacks 500s once a pre-call guardrail is on, and the router strategy loggers log a traceback per request.

* test(proxy): annotate the closure dicts the passthrough logging tests record into
2026-09-01 18:07:04 -07:00