mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-01 07:03:02 +00:00
4 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
yujonglee
|
7b60fd061a
|
refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042)
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http The workspace moves to reqwest 0.13, so litellm-http no longer carries a second reqwest just for rmcp. The mcp feature, its pool and its proxy conversion go away, and the gateway hands rmcp a pooled NoRedirect client that already carries the TLS, proxy and timeout settings The Client test helpers collapse into for_test, a no-redirect client without TLS or proxy settings, and wrap_for_test for the few tests that need a hand-built reqwest client Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(http): honor transport settings for Rust WebSockets The Responses WebSocket connected through tokio-tungstenite with its own native-roots TLS, so it ignored the configured CA bundle, verify flag, client certificate, proxies and timeouts. The handshake now goes through a pooled reqwest client (ClientVariant::WebSocket, HTTP/1.1, no redirects) and upgrades the response, so it shares the settings every other outbound request uses The upgrade response is checked for the Sec-WebSocket-Accept key, the Upgrade and Connection headers and a subprotocol that was offered Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(http): build outbound requests as reqwest::Request OutboundRequest was a hand-rolled copy of reqwest::Request. Routes now build a reqwest::Request directly and RequestSigner reads the body bytes back out of it, so the signer still covers exactly what is sent. SigV4 signing moves onto HeaderMap and takes the method, which drops the BTreeMap plumbing and the header casing table Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(http): make litellm_http::Error the one opaque error transport::Error and media::Error fold into litellm_http::Error, an Arc<Repr> newtype whose private variants keep the typed reqwest, io, rustls and tungstenite causes and own every message. Callers can no longer build an error with a message of their own. They ask through dispatched, is_request, status, upstream_response and tls_source One From<reqwest::Error> now classifies connect, network and timeout and strips the URL. The WebSocket handshake reports each failed check as its own variant and owns its connect timeout. OCR drops the media variants it mirrored, and inference::Error maps a header the signer computes to Unsupported for every route, not only chat The error rules move out of AGENTS.md into the rust-error skill, which also records the decisions behind this design Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(rust): split rust-error skill into thin SKILL.md with references Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): render causes once and ask predicates at the boundaries The four WebSocket handshake variants collapse into Handshake(HandshakeFault), and media download failures no longer answer dispatched() since they happen before the provider call ErrorDetail in llms and auth-types, the inference error and the OCR error now store their cause in a field named cause with no #[source], so to_string() carries it exactly once and source() is None. Tests that walked the chain now destructure the variant and downcast the cause The gateway asks is_unsupported() and is_missing_credential() on the inference and OCR errors instead of matching through nested enums, and litellm_auth::Error answers is_missing_credential() for both The rust-error skill gains axum and pyo3 references, boundaries.md describes the real predicate order and the known divergences, the testing example asserts dispatched() correctly, and shape, classification, conversions, messages-and-causes and recipes agree with the code again * docs(rust): focus the rust-error gateway page on the error flow and add a rust-gateway skill The former axum page toured gateway-inference: its status match, the envelope table, an extractor, the unsupported route and the Sse closure. Those details belong to the gateway crates and go stale with them. The page is now gateway.md and follows an error from the provider failure through each From into IntoResponse, where it is classified once and rendered by surface and framing. The gateway-structure material moves to a new rust-gateway skill: envelopes per API surface, extractors and path registration, SSE relay, and the router test harness. * test(inference): pin provider URL redaction on every route's transport error Chat, Messages, Responses and transcription each get a route-level test that fails if a reqwest error reaches the caller with its URL. The Messages stream decoder also takes body errors through litellm_http::Error like every other reqwest error Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(rust-error): adopt std source chains and add litellm-error Report Messages describe their own layer and the cause is the source, boundaries render the chain once with litellm_error::Report. Corrects skill claims the audit found stale against the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): keep causes as sources across http, auth, secrets, cache and utility crates Messages describe their own layer and every cause is a typed source, predicates match exhaustively, and boundaries render with litellm_error::Report. Media downloads no longer count as dispatched, Error::upstream bounds the provider body itself, and a forwarded header SigV4 cannot sign is the caller's fault. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): keep causes as sources in llms and the gateway crates ErrorDetail loses its string variant, OCR provider variants become data, host faults stop counting as the caller's, a tunnelled transport error classifies the same on every stream path, and each gateway crate has one Error with one status() and extractors that reject in its envelope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): nest llms errors in inference and ask predicates in route tests litellm_inference::Error nests litellm_llms::Error instead of mirroring its variants, host driver faults get their own variant, every predicate matches exhaustively, and route tests ask predicates instead of matching transport variants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): map bridge and gateway-inference errors once and render them with Report RustUpstreamError has one constructor that always sets headers, argument checks raise typed errors mapped once, OCR asks upstream_response(), the Responses stream error event comes from the error type, gateway-inference loses its string variants, and auth faults classify through one litellm_auth predicate on every route. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(rust-error): carry OCR provider headers as a HeaderMap and keep outages off the caller The OCR provider error and get_error_class take reqwest's HeaderMap, the bridge turns it into Python's (name, value) pairs once. Credential acquisition and provider SDK failures are no longer the caller's fault, the UI hides auth and backend causes, and chat keeps a provider answer as is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(rust-error): correct skill claims against the aligned code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): fix the remaining sweep findings in gateway, llms, auth and cache crates Auth rejections hide their backend cause on every inference renderer, path parameters reject in the surface envelope, Textract's unsupported document and Azure DI's unparseable poll URL get typed variants, the credential chain exposes its last failure as the source, gateway-mcp asks predicates over rmcp errors, and the S3 and Azure blob transports classify through litellm_http::Error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): fix the remaining sweep findings on the Python side host-python keeps a native error's chain and returns blocking-pool cancellation instead of panicking, Vault transport failures convert through litellm_http::Error so no URL reaches Python, the bridge asks predicates instead of matching three levels into Vault errors, and the last string-built PyErrs become typed errors mapped once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): refresh compatible dependencies flagged by OSV * refactor(rust): remove redundant client construction comment * test(rust): cover configured MCP proxy and CA bundle * fix(http): preserve native WebSocket trust and optional protocols * fix(http): allow plain WebSockets without native roots * fix(http): normalize provider URLs before building outbound requests * fix(rust): preserve OCR download status and reduce error size * fix(rust): preserve upstream bodies and bound diagnostics only * test(rust): preserve bridge serialization boundary in body regression --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yujonglee
|
8302f006dd
|
refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python (#46299)
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python Rust's run_call built Python streams itself and imported litellm.rust_bridge.streams by name, so sync calls were driven in Rust while async calls came back as a Rust-made coroutine. Every native inference entrypoint now returns one unstarted Execution and public_execution.py drives it with lifecycle.run or lifecycle.drive, wrapping streams and _hidden_params on the Python side. Rust only imports the lifecycle step classes through StepTypes, and python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone. The adapter trusts the _native.pyi contract with a single cast instead of a per-request runtime_checkable isinstance. Binding tests assert the adapter starts the execution and returns its completed value rather than checking callable identity. * refactor(rust-bridge): fold lifecycle and public_execution into one execution module The public stream wrappers only forwarded to the lifecycle streams and added the header dict, so the streams now carry _hidden_params themselves. lifecycle.py no longer held any call lifecycle, just the Execution driver, so it is renamed to execution.py and absorbs the synchronous and asynchronous adapters Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(rust-bridge): keep execution.py free of third-party imports The host-python tests load execution.py into a bare interpreter, so the pydantic header validation broke them. The stream head comes from the binding's encode_stream_head, which always builds a str-keyed dict Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(host-python): preserve pre-async hook coverage after execution handoff * style(rust-bridge): sort litellm_host_python imports Co-Authored-By: Yujong Lee <yujong@berri.ai> * test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests Co-Authored-By: Yujong Lee <yujong@berri.ai> * fix(rust-bridge): mark execution TypeVar as Final Co-Authored-By: Yujong Lee <yujong@berri.ai> * test(rust-bridge): cover stream close, read failure, and sync suspension in execution Co-Authored-By: Yujong Lee <yujong@berri.ai> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yujonglee
|
f50f14117f
|
refactor(rust-bridge): generator-style execution protocol with stream step names (#46282)
* refactor(rust-bridge): drive executions with generator-style send, throw and close Co-Authored-By: Yujong Lee <yujong@berri.ai> * refactor(rust-bridge): name stream steps StreamOpen and StreamChunk and reject out-of-phase steps Co-Authored-By: Yujong Lee <yujong@berri.ai> * refactor(rust-bridge): drive with ExitStack and require the stream factory Co-Authored-By: Yujong Lee <yujong@berri.ai> * test(host-python): give the hook_chain lifecycle binding the stream factory Co-Authored-By: Yujong Lee <yujong@berri.ai> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yujonglee
|
327bdcaa7f
|
test(host-python): move driver tests out of the inline module (#46325)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |