mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-30 06:43:27 +00:00
8893c75eb3
The gateway acts on the core decline. A replay layer records each native request before its handler runs. When the error response carries a Decline it forwards the recording to Python as the client sent it, or keeps the 501 when the gateway runs standalone. Paths and methods with no native route go to Python through the merged router's fallbacks, before authentication. The 501 stubs for embeddings and completions and the gateway's own Unsupported variant are gone, since a matched stub would never reach Python. LITELLM_PYTHON_UPSTREAM picks the Fallback at startup. Forward uses its own hyper client to the loopback proxy, strips hop-by-hop headers, adds x-litellm-gateway-declined with the reason and appends the client address to X-Forwarded-For. An upstream that is not an http origin is rejected with InvalidUpstream, which keeps the URI parse failure as its source. Error responses attach the decline in one place, so the messages handler builds its Anthropic envelope through Error::anthropic_response instead of a tuple that dropped it. Forwarding is a gateway concern, so the dispatch module lives in the gateway crate and replays declines for every mounted router. The replay layer is installed only when a Python upstream is set, so a standalone gateway does not buffer request bodies. Tests run the gateway and a stand-in Python on real listeners through litellm_gateway::serve, which main.rs uses too. The rust-error skill says where an unserved path and a forwarded decline go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| dispatch.rs | ||
| mcp.rs | ||
| server.rs | ||