Commit graph

7 commits

Author SHA1 Message Date
yujonglee
323d3956d6
feat(rust): one typed decline from core up to each host (#47772)
* feat(rust): one typed decline from core up to each host

Rust core says it cannot serve a request with one Decline that travels
unchanged through llms and inference up to the host. The Python bridge
raises RustBridgeDeclined for it in route_failure before the public
exception mapping, so dispatch falls back to legacy Python instead of
failing the call, and the gateway answers 501. Every route collapses
errors after the send with Error::after_send, which keeps a provider
answer and a response fault as they are and nests anything else as the
source of an invalid response, so a billed call can never read as a
decline. Response-side cases that used to decline are now
ErrorDetail::Unhandled response faults.

The rust-error skill documents declined() as the first question both
boundaries ask, and the Python suites pin that a native decline reaches
the dispatcher before any provider IO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(rust): keep post-provider hook failures terminal

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-28 09:22:11 -07:00
yujonglee
730db4d228
refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines (#46714)
* refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines

* refactor(rust-bridge): split and document route-specific detectors

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 15:06:25 -07:00
yujonglee
0ccae7ab35
refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates (#46750)
* refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates

* refactor(rust-bridge): simplify catalog compatibility checks

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-20 15:06:24 -07:00
yujonglee
a1c9fa951a
test(rust-bridge): relocate dispatch tests before routing changes (#46926)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-16 12:29:33 -07:00
yujonglee
7d79ccb7cc
test(rust-bridge): select native routes per test without LITELLM_RUST (#46620)
* feat(rust-bridge): scope catalog rules with a contextvar

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): select native integration routes with scoped rules

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): run the Rust extension suite without LITELLM_RUST

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): use scoped catalog rules in dispatch tests

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* refactor(rust-bridge): keep catalog rules a static global and override them once per test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:12:08 -07:00
yujonglee
8302f006dd
refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python (#46299)
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python

Rust's run_call built Python streams itself and imported
litellm.rust_bridge.streams by name, so sync calls were driven in Rust
while async calls came back as a Rust-made coroutine. Every native
inference entrypoint now returns one unstarted Execution and
public_execution.py drives it with lifecycle.run or lifecycle.drive,
wrapping streams and _hidden_params on the Python side. Rust only
imports the lifecycle step classes through StepTypes, and
python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone.

The adapter trusts the _native.pyi contract with a single cast instead
of a per-request runtime_checkable isinstance. Binding tests assert the
adapter starts the execution and returns its completed value rather
than checking callable identity.

* refactor(rust-bridge): fold lifecycle and public_execution into one execution module

The public stream wrappers only forwarded to the lifecycle streams and added the
header dict, so the streams now carry _hidden_params themselves. lifecycle.py no
longer held any call lifecycle, just the Execution driver, so it is renamed to
execution.py and absorbs the synchronous and asynchronous adapters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(rust-bridge): keep execution.py free of third-party imports

The host-python tests load execution.py into a bare interpreter, so the
pydantic header validation broke them. The stream head comes from the
binding's encode_stream_head, which always builds a str-keyed dict

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(host-python): preserve pre-async hook coverage after execution handoff

* style(rust-bridge): sort litellm_host_python imports

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* fix(rust-bridge): mark execution TypeVar as Final

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): cover stream close, read failure, and sync suspension in execution

Co-Authored-By: Yujong Lee <yujong@berri.ai>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 17:50:44 -07:00
yujonglee
0fc11653e2
test(rust-bridge): pin Messages and Responses callback contracts on the native path (#46443)
* test(rust-bridge): pin Messages and Responses callback contracts on the native path

Each test runs the native route and asserts what the Python path does today, so a
known gap is a strict xfail that names its reason and the exception it fails with
Messages and Responses share one Surface and hook recorder in support/callbacks.py,
and each surface folder lists its own expected counts and gaps.

Body-level hooks from the function body (pre-request hook, websearch short-circuit,
Messages interceptors, planned and legacy agentic loops, streaming deployment hook)
are xfail until native runs them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(rust-bridge): give every route folder the same one-concern-per-file layout

Route data, the route Surface and the server fixture move into each route folder, and support/ keeps only
route-neutral helpers. test_inference.py splits into chat_completions/ and responses/, ocr/test_requests.py
splits into requests, failures and settings, and Messages and Responses get streaming and body hook files.
Messages request tests that duplicate inference-messages request.rs are removed, and drop_params read
from litellm.drop_params is now covered

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(rust-bridge): expect the required-route decline error for unsupported Responses requests

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): import route data from the route folders in the secrets unit tests

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): import NoReturn in the OCR dispatch tests

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): move route-neutral helpers out of support/callbacks.py

Co-Authored-By: Yujong Lee <yujong@berri.ai>

* test(rust-bridge): cover deployment rejection and concurrent callbacks

* test(rust-bridge): assert native serving via the x-litellm-rust marker in the route folder tests

Co-Authored-By: Yujong Lee <yujong@berri.ai>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 17:07:20 -07:00