The test spawned a real Python child that installs a SIGTERM handler, slept
0.3s, then cancelled the supervisor and expected the child to exit 0. CI runs
with --cov, and pytest-cov starts coverage in every spawned Python process,
so the child is often still starting when SIGTERM lands and dies with -15.
open_children now takes the process spawner the same way it takes the
platform, and only needs the platform to signal and stop children. The test
uses a fake child whose wait() blocks on an event and a fake platform that
records stop_group, so it checks the shielded stop-then-wait with no real
process and no sleep. It also checks the child is still being waited on,
because anyio's wait_all_tasks_blocked on asyncio can return one loop tick
before a task group finishes.
* feat(proxy): supervise the opt-in Rust forwarding server
With LITELLM_RUST on, `litellm` and `python -m gateway.launch` start a
supervisor that runs the Rust gateway on the public port and Python on a
private loopback listener. Python still serves every route and sees each
request as if the client had connected to it directly.
Rust sends the client's TCP peer in `x-litellm-rust-peer`. The supervised
server turns server-level proxy headers off, and a middleware restores
the peer and then applies the operator's own trust, so client IPs,
trusted proxies and every IP-based setting behave the same with Rust on
or off.
The supervisor is transparent to Python. Signals pass through, Python's
exit status is the result, and both children shut down when the
supervisor's end of their stdin closes, so nothing outlives it. Rust
starts only once Python listens. When forwarding cannot start (no
gateway binary, TLS, a Unix socket, Hypercorn or Granian), the entry
point warns and serves from Python alone.
Every forwarding decision lives in litellm/proxy/rust_forward, paired
with the gateway-forward crate. OS differences sit behind a Platform
protocol, with Windows supported for local CLI runs. AGENTS.md files in
the touched folders state the boundary. The `--rust_gateway` flag is
gone in favor of the environment variable.
* refactor(proxy): switch Rust forwarding on with LITELLM_RUST_FIRST
Forwarding gets its own switch, independent of LITELLM_RUST, which keeps
selecting only the Rust paths inside the SDK. Neither turns the other on.
---------
Co-authored-by: narmstrong-berri <narmstrong@berri.ai>
