diff --git a/litellm-rust/.agents/skills/rust-error/references/messages-and-causes.md b/litellm-rust/.agents/skills/rust-error/references/messages-and-causes.md index abf2c0cde4a..651d7cb8856 100644 --- a/litellm-rust/.agents/skills/rust-error/references/messages-and-causes.md +++ b/litellm-rust/.agents/skills/rust-error/references/messages-and-causes.md @@ -66,7 +66,7 @@ Store the lower-level error itself, never its string, and never drop it with `ma | downcasting | the type is gone | | redaction | `reqwest::Error`'s `Display` includes the request URL, which can carry credentials in the path or query | -If the cause is not `Clone` and the error must be, hold it as `#[source] litellm_error::Source`, a cloneable `Arc` that is transparent to `Display` and `source()`, or make the crate's error opaque. `litellm_error::find::(&error)` walks the chain and looks through `Source`, so it is how anyone reaches a typed cause. A third-party `Box`, or an `anyhow::Error` a dependency returns, converts with `Source::from`, and a dependency's own boxed alias such as `tokenizers::Error` is held as it is. Never put `#[error(transparent)]` on a `Source`, the erased error would then be no node of the chain and `find` could not reach it. Do not stringify or drop a cause to get `Clone` or `PartialEq` +If the cause is not `Clone` and the error must be, hold it as `#[source] litellm_error::Source`, a cloneable `Arc` that is transparent to `Display` and `source()`, or make the crate's error opaque. `litellm_error::find::(&error)` walks the chain and looks through `Source`, so it is how anyone reaches a typed cause. A third-party `Box`, or an `anyhow::Error` a dependency returns, converts with `Source::from`, and a dependency's own boxed alias such as `tokenizers::Error` is held as it is. A source boxed to keep the variant small is a `Box` node of the chain, so it is reached as `find::>`. Never put `#[error(transparent)]` on a `Source`, the erased error would then be no node of the chain and `find` could not reach it. Do not stringify or drop a cause to get `Clone` or `PartialEq` ```rust // loses the type and leaks the URL diff --git a/litellm-rust/.agents/skills/rust-error/references/testing.md b/litellm-rust/.agents/skills/rust-error/references/testing.md index fdf1699e190..4aa22b4a293 100644 --- a/litellm-rust/.agents/skills/rust-error/references/testing.md +++ b/litellm-rust/.agents/skills/rust-error/references/testing.md @@ -20,7 +20,7 @@ In order of preference 2. A real cause, such as `tungstenite::Error::ConnectionClosed` or a `serde_json::from_str("{")` error, passed through our `From` 3. A public constructor, such as `Error::upstream(500, "boom".into())` -Inside the owning crate a test may match the private representation through the `#[cfg(test)] fn repr(&self) -> &Repr` accessor in `crates/http/src/error.rs`, as the `media.rs` and `tls.rs` tests do. Anywhere else, reach the cause the test expects with `litellm_error::find::(&error)` +Inside the owning crate a test may match the private representation through the `#[cfg(test)] fn repr(&self) -> &Repr` accessor in `crates/http/src/error.rs`, as the `media.rs` and `tls.rs` tests do. Anywhere else, reach the cause the test expects with `litellm_error::find::(&error)`. A transparent layer is no node of the chain, so `find` reaches foreign causes underneath it but never the nested error of ours itself. Ask that error's predicates through the outer type instead, which is what callers do too ## What to cover diff --git a/litellm-rust/.agents/skills/rust-gateway/references/surfaces.md b/litellm-rust/.agents/skills/rust-gateway/references/surfaces.md index a0b408abde6..c3f74325c26 100644 --- a/litellm-rust/.agents/skills/rust-gateway/references/surfaces.md +++ b/litellm-rust/.agents/skills/rust-gateway/references/surfaces.md @@ -8,10 +8,12 @@ Each surface mirrors what the Python proxy returns there, so the shapes differ o | OpenAI-shaped routes | `{"error": {"message", "type", "param", "code"}}` | `Error::openai_response()` and `openai_sse_frame()` in the same file | | MCP | `{"detail": message}` | `crates/gateway-mcp/src/error.rs` | | UI | `{"error": {"message"}}`, plus `Retry-After` on 429 | `crates/gateway-ui/src/error.rs` | -| auth rejections | plain text | `crates/gateway-auth/src/error.rs` | +| auth rejections | plain text, the top-level message only | `crates/gateway-auth/src/error.rs` | Every surface takes its status from one `status()` on its crate's `Error` and renders the message of anything that is not a provider answer with `litellm_error::Report`. In `gateway-inference` the error `type` string is derived from the status by `error_type()`, so a new failure never names its own type. The Anthropic envelope passes a provider body already in its shape through untouched and extracts the message from anything else, and the OpenAI envelope always wraps, as the Python proxy does. A provider body is bounded by `litellm_http::Error::upstream` when the error is built +Two surfaces show less than the chain on purpose, and a test in each pins it. An auth rejection goes to a caller who has not authenticated, so it never shows what a secret manager or key store said. An MCP upstream failure shows only its own layer, because rmcp renders the transport error, URL included, into its own `Display` + `/v1/messages` adds the request id to the envelope when the provider did not set one, which is why its handler takes the body as `Result` and renders the rejection itself, see [requests](requests.md) A new surface gets its own renderer next to the others, named after the shape it produces, and its status still comes from `Error::status()`. Picking the status is the [`rust-error`](../../rust-error/references/gateway.md) skill's concern, not the surface's diff --git a/litellm-rust/Cargo.lock b/litellm-rust/Cargo.lock index ebcd2b4b31f..7131cbb2bdd 100644 --- a/litellm-rust/Cargo.lock +++ b/litellm-rust/Cargo.lock @@ -4040,6 +4040,7 @@ dependencies = [ "litellm-cache-memory", "litellm-cache-response", "litellm-core-utils", + "litellm-error", "litellm-host", "litellm-host-native", "litellm-http", @@ -4132,6 +4133,7 @@ dependencies = [ "litellm-cache-memory", "litellm-cache-response", "litellm-core-utils", + "litellm-error", "litellm-host", "litellm-host-native", "litellm-http", @@ -4169,6 +4171,7 @@ dependencies = [ "futures-util", "litellm-auth", "litellm-core-utils", + "litellm-error", "litellm-http", "litellm-inference", "litellm-inference-testing", diff --git a/litellm-rust/crates/gateway-inference/src/error.rs b/litellm-rust/crates/gateway-inference/src/error.rs index af0f65449fc..590e4e03196 100644 --- a/litellm-rust/crates/gateway-inference/src/error.rs +++ b/litellm-rust/crates/gateway-inference/src/error.rs @@ -236,7 +236,7 @@ mod tests { #[rstest] #[case::upstream_status(upstream(429, ""), StatusCode::TOO_MANY_REQUESTS)] #[case::rejected_request( - Error::Route(InferenceError::InvalidRequest(litellm_llms::ErrorDetail::Expected { + Error::Route(InferenceError::invalid_request(litellm_llms::ErrorDetail::Expected { subject: "top_k", expected: "an integer", })), diff --git a/litellm-rust/crates/http/src/error.rs b/litellm-rust/crates/http/src/error.rs index fb6d6f7a216..6f025876b21 100644 --- a/litellm-rust/crates/http/src/error.rs +++ b/litellm-rust/crates/http/src/error.rs @@ -71,6 +71,8 @@ pub(crate) enum Repr { HandshakeTimeout(#[source] Elapsed), #[error("upstream WebSocket error")] WebSocket(#[source] tungstenite::Error), + #[error("the WebSocket connection is closed")] + WebSocketClosed, #[error("media URL rejected by network policy")] BlockedUrl, @@ -151,6 +153,10 @@ impl Error { Repr::UnsignableHeader(litellm_error::Source::new(cause)).into() } + pub fn websocket_closed() -> Self { + Repr::WebSocketClosed.into() + } + pub fn signature(cause: impl std::error::Error + Send + Sync + 'static) -> Self { Repr::Signature(litellm_error::Source::new(cause)).into() } @@ -165,7 +171,8 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) => true, - Repr::Read { .. } + Repr::WebSocketClosed + | Repr::Read { .. } | Repr::InvalidPem { .. } | Repr::Client(_) | Repr::TlsConfig(_) @@ -220,6 +227,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::Resolve(_) | Repr::MissingRedirectLocation | Repr::InvalidRedirect(_) @@ -253,6 +261,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::BlockedUrl | Repr::Resolve(_) | Repr::DownloadDisabled @@ -287,6 +296,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::BlockedUrl | Repr::Resolve(_) | Repr::DownloadDisabled @@ -323,6 +333,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::BlockedUrl | Repr::Resolve(_) | Repr::DownloadDisabled @@ -359,6 +370,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::BlockedUrl | Repr::Resolve(_) | Repr::DownloadDisabled @@ -395,6 +407,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::BlockedUrl | Repr::Resolve(_) | Repr::DownloadDisabled @@ -430,6 +443,7 @@ impl Error { | Repr::Handshake(_) | Repr::HandshakeTimeout(_) | Repr::WebSocket(_) + | Repr::WebSocketClosed | Repr::BlockedUrl | Repr::Resolve(_) | Repr::DownloadDisabled diff --git a/litellm-rust/crates/inference-chat/Cargo.toml b/litellm-rust/crates/inference-chat/Cargo.toml index 9815f3b5c64..8a779d44976 100644 --- a/litellm-rust/crates/inference-chat/Cargo.toml +++ b/litellm-rust/crates/inference-chat/Cargo.toml @@ -22,6 +22,7 @@ tracing.workspace = true [dev-dependencies] litellm-cache-memory.workspace = true +litellm-error.workspace = true litellm-host-native.workspace = true litellm-http = { workspace = true, features = ["test-support"] } litellm-inference-testing.workspace = true diff --git a/litellm-rust/crates/inference-chat/src/common_utils.rs b/litellm-rust/crates/inference-chat/src/common_utils.rs index 97b9ac71464..1178a8fa893 100644 --- a/litellm-rust/crates/inference-chat/src/common_utils.rs +++ b/litellm-rust/crates/inference-chat/src/common_utils.rs @@ -50,16 +50,18 @@ pub(super) fn resolve_request( config: &'static dyn BaseConfig, ) -> Result { let messages = serde_json::from_value::>(call.messages).map_err(|err| { - Error::InvalidRequest(litellm_llms::ErrorDetail::invalid( + Error::invalid_request(litellm_llms::ErrorDetail::invalid( "chat completions messages", err, )) })?; if messages.is_empty() { - return Err(Error::InvalidRequest(litellm_llms::ErrorDetail::Expected { - subject: "chat completions messages", - expected: "non-empty", - })); + return Err(Error::invalid_request( + litellm_llms::ErrorDetail::Expected { + subject: "chat completions messages", + expected: "non-empty", + }, + )); } if let Some(reason) = config.unsupported_reason(&messages, &call.optional_params) { return Err(Error::Unsupported(reason.0)); diff --git a/litellm-rust/crates/inference-chat/src/handler.rs b/litellm-rust/crates/inference-chat/src/handler.rs index bfce8ce5c23..73df38df6b7 100644 --- a/litellm-rust/crates/inference-chat/src/handler.rs +++ b/litellm-rust/crates/inference-chat/src/handler.rs @@ -120,7 +120,7 @@ pub(super) async fn execute( .map_err(Error::post_call)?; let body: Value = serde_json::from_str(&text).map_err(|err| { - Error::InvalidResponse(litellm_llms::ErrorDetail::invalid( + Error::invalid_response(litellm_llms::ErrorDetail::invalid( "chat completions response JSON", err, )) @@ -144,14 +144,13 @@ pub(super) async fn execute( /// implementation must not retry those, so collapse them to one variant that /// can only mean the provider was already called. pub(super) fn as_response_error(err: Error) -> Error { - match err { - already @ Error::InvalidResponse(_) => already, - upstream if upstream.upstream_response().is_some() => upstream, - other => Error::InvalidResponse(litellm_llms::ErrorDetail::invalid( - "chat completions response", - other, - )), + if err.is_response() { + return err; } + Error::invalid_response(litellm_llms::ErrorDetail::invalid( + "chat completions response", + err, + )) } #[cfg(test)] @@ -307,30 +306,38 @@ mod tests { .expect_err("the upstream failure fails the call"); assert!( - matches!(&error, Error::Http(http) if http.upstream_response().is_some_and(|(status, _)| status == 500)), + error + .upstream_response() + .is_some_and(|(status, _)| status == 500), "{error:?}" ); assert!(interceptors.raw.into_inner().unwrap().is_empty()); } #[rstest::rstest] - fn response_errors_collapse_to_one_variant_that_can_only_mean_already_sent() { - for original in [ - Error::MissingField("usage"), - Error::Unsupported("non-text response content block"), - Error::InvalidRequest(litellm_llms::ErrorDetail::Expected { - subject: "whatever", - expected: "anything else", - }), - Error::Auth(litellm_auth::Error::InvalidHeader), - ] { - let label = format!("{original:?}"); - assert!( - matches!(as_response_error(original), Error::InvalidResponse(_)), - "{label} must not stay retryable once the provider has answered" - ); - } - let upstream = Error::from(litellm_http::Error::upstream(500, "boom".into())); - assert_eq!(as_response_error(upstream.clone()), upstream); + #[case::missing_field(litellm_llms::Error::MissingField("usage").into())] + #[case::unsupported(Error::Unsupported("non-text response content block"))] + #[case::invalid_request(Error::invalid_request(litellm_llms::ErrorDetail::Expected { + subject: "whatever", + expected: "anything else", + }))] + #[case::auth(Error::Auth(litellm_auth::Error::InvalidHeader))] + fn response_errors_collapse_to_a_fault_that_can_only_mean_already_sent( + #[case] original: Error, + ) { + let collapsed = as_response_error(original); + assert!(collapsed.is_response(), "{collapsed:?}"); + assert!(!collapsed.is_request(), "{collapsed:?}"); + assert!(!collapsed.is_unsupported(), "{collapsed:?}"); + } + + #[rstest::rstest] + fn a_response_fault_is_not_wrapped_twice() { + let original = Error::invalid_response(litellm_llms::ErrorDetail::MissingField { + subject: "response", + field: "usage", + }); + let report = litellm_error::Report::new(&as_response_error(original)).to_string(); + assert!(!report.contains("chat completions response"), "{report}"); } } diff --git a/litellm-rust/crates/inference-chat/tests/chat_completions.rs b/litellm-rust/crates/inference-chat/tests/chat_completions.rs index df07ba82f74..6ed513ba32a 100644 --- a/litellm-rust/crates/inference-chat/tests/chat_completions.rs +++ b/litellm-rust/crates/inference-chat/tests/chat_completions.rs @@ -179,7 +179,7 @@ async fn a_response_it_cannot_normalize_is_reported_as_already_sent( .await .expect_err("response cannot be normalized"); - assert!(matches!(error, Error::InvalidResponse(_)), "{error:?}"); + assert!(error.is_response(), "{error:?}"); } #[rstest] @@ -215,10 +215,7 @@ async fn a_connection_that_is_never_established_returns_a_connect_error( .await .expect_err("nothing is listening"); - assert!( - matches!(&error, Error::Http(http) if !http.dispatched()), - "{error:?}" - ); + assert!(!error.dispatched(), "{error:?}"); } #[rstest] @@ -235,7 +232,8 @@ async fn a_transport_error_does_not_echo_the_provider_url( .await .expect_err("nothing is listening"); - assert!(!error.to_string().contains("tenant-token-1234"), "{error}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(!report.contains("tenant-token-1234"), "{report}"); } #[rstest] @@ -253,10 +251,8 @@ async fn a_timeout_after_sending_returns_a_network_error(request: ChatCompletion .await .expect_err("the call times out"); - assert!( - matches!(&error, Error::Http(http) if http.dispatched()), - "{error:?}" - ); + assert!(error.dispatched(), "{error:?}"); + assert!(!error.is_request(), "{error:?}"); } #[rstest] @@ -352,7 +348,7 @@ async fn a_post_call_hook_failure_never_looks_safe_to_retry( Ok(wire) } async fn after_provider_response(&self, _: RawResponse) -> Result<(), Error> { - Err(Error::MissingField("callback rejected")) + Err(litellm_llms::Error::MissingField("callback rejected").into()) } } let upstream = upstream([anthropic_response(ANTHROPIC_MESSAGE)]).await; @@ -368,10 +364,10 @@ async fn a_post_call_hook_failure_never_looks_safe_to_retry( ) .await .unwrap_err(); - let Error::PostCallHook(source) = error else { - panic!("expected retained callback error") - }; - assert_eq!(*source, Error::MissingField("callback rejected")); + assert!(error.dispatched(), "{error:?}"); + assert!(!error.is_request(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains("callback rejected"), "{report}"); assert_eq!(received(&upstream).await.len(), 1); } diff --git a/litellm-rust/crates/inference-chat/tests/request.rs b/litellm-rust/crates/inference-chat/tests/request.rs index d35d67e0a48..a648a637652 100644 --- a/litellm-rust/crates/inference-chat/tests/request.rs +++ b/litellm-rust/crates/inference-chat/tests/request.rs @@ -211,7 +211,7 @@ async fn rejects_empty_messages_before_resolving_credentials() { }) .await .expect_err("empty messages are rejected"); - assert!(matches!(error, Error::InvalidRequest(_))); + assert!(error.is_request(), "{error:?}"); } #[rstest] @@ -233,32 +233,27 @@ async fn rejects_an_unknown_provider() { #[rstest] #[tokio::test] async fn rejects_a_model_with_no_resolvable_provider() { - assert!(matches!( - complete(request( - "claude-sonnet-4-5", - None, - json!([{"role": "user", "content": "hi"}]), - json!({}), - )) - .await, - Err(Error::InvalidProvider(_)) - )); + let error = complete(request( + "claude-sonnet-4-5", + None, + json!([{"role": "user", "content": "hi"}]), + json!({}), + )) + .await + .unwrap_err(); + assert!(error.is_request(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!( + report.contains("unable to resolve custom_llm_provider"), + "{report}" + ); } #[rstest] -#[case::empty( - json!([]), - Some(litellm_llms::ErrorDetail::Expected { - subject: "chat completions messages", - expected: "non-empty", - }) -)] -#[case::malformed(json!("not a list"), None)] +#[case::empty(json!([]), "chat completions messages must be non-empty")] +#[case::malformed(json!("not a list"), "invalid chat completions messages")] #[tokio::test] -async fn rejects_an_empty_or_malformed_message_list( - #[case] messages: Value, - #[case] expected: Option, -) { +async fn rejects_an_empty_or_malformed_message_list(#[case] messages: Value, #[case] reason: &str) { let error = complete(request( "anthropic/claude-sonnet-4-5", None, @@ -267,11 +262,9 @@ async fn rejects_an_empty_or_malformed_message_list( )) .await .unwrap_err(); - if let Some(expected) = expected { - assert_eq!(error, Error::InvalidRequest(expected)); - } else { - assert!(matches!(error, Error::InvalidRequest(_))); - } + assert!(error.is_request(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains(reason), "{report}"); } #[rstest] diff --git a/litellm-rust/crates/inference-messages/src/handler.rs b/litellm-rust/crates/inference-messages/src/handler.rs index 5b7fd2d45b9..67c3b8f98b5 100644 --- a/litellm-rust/crates/inference-messages/src/handler.rs +++ b/litellm-rust/crates/inference-messages/src/handler.rs @@ -188,7 +188,7 @@ async fn provider_call( None | Some(Value::Null) => false, Some(Value::Bool(stream)) => *stream, Some(value) => { - return Err(Error::InvalidRequest( + return Err(Error::invalid_request( litellm_llms::ErrorDetail::InvalidValue { field: "stream", expected: "a boolean", @@ -261,7 +261,7 @@ async fn call_provider( provider_name, )); } - let text = response.text().await.map_err(network)?; + let text = response.text().await.map_err(litellm_http::Error::from)?; log_response_body(&text); context.response_received(&text).await?; decode_response(config, &identity.model, &text) @@ -269,16 +269,12 @@ async fn call_provider( } fn serialize_failure(err: serde_json::Error) -> Error { - Error::InvalidRequest(litellm_llms::ErrorDetail::failed( + Error::invalid_request(litellm_llms::ErrorDetail::failed( "Anthropic messages request serialization", err, )) } -fn network(error: reqwest::Error) -> Error { - litellm_http::Error::from(error).into() -} - async fn send( http: &litellm_http::Client, authenticated: Authenticated, @@ -302,7 +298,7 @@ async fn provider_error(response: reqwest::Response) -> Error { log_error_body(status, &text); litellm_http::Error::upstream(status, text).into() } - Err(error) => network(error), + Err(error) => litellm_http::Error::from(error).into(), } } @@ -312,7 +308,7 @@ fn decode_response( text: &str, ) -> Result { let response = serde_json::from_str(text).map_err(|err| { - Error::InvalidResponse(litellm_llms::ErrorDetail::invalid( + Error::invalid_response(litellm_llms::ErrorDetail::invalid( "messages response JSON", err, )) @@ -334,7 +330,7 @@ fn streaming_response( .collect(); let chunks = match decoder { None => futures_util::stream::try_unfold(response, move |mut response| async move { - let chunk = response.chunk().await.map_err(network)?; + let chunk = response.chunk().await.map_err(litellm_http::Error::from)?; Ok(chunk.map(|chunk| { log_chunk(provider, "provider_response", &chunk); (chunk, response) @@ -536,14 +532,10 @@ mod tests { shaping, }) .await; - assert!( - matches!( - result, - Err(Error::Auth(litellm_auth::Error::MissingApiKey { .. })) - ), - "{:?}", - result.err() - ); + let Err(error) = result else { + panic!("a call without credentials fails"); + }; + assert!(error.is_missing_credential(), "{error:?}"); } #[rstest] @@ -766,19 +758,18 @@ mod tests { }, ..shaping }; - assert!(matches!( - wire_body( - json!({ - "model": "claude-test", - "messages": [{"role": "user", "content": "hi"}], - "max_tokens": 16, - "metadata": {"user_id": 123} - }), - shaping, - ) - .await, - Err(Error::InvalidRequest(_)) - )); + let error = wire_body( + json!({ + "model": "claude-test", + "messages": [{"role": "user", "content": "hi"}], + "max_tokens": 16, + "metadata": {"user_id": 123} + }), + shaping, + ) + .await + .unwrap_err(); + assert!(error.is_request(), "{error:?}"); } #[rstest] @@ -795,7 +786,7 @@ mod tests { shaping, ) .await, - Err(Error::InvalidRequest( + Err(Error::invalid_request( litellm_llms::ErrorDetail::InvalidValue { field: "metadata.user_id", expected: "a string", @@ -897,7 +888,7 @@ mod tests { let chunk = chunks.next().await.unwrap(); match expected { Some(expected) => assert_eq!(chunk.unwrap().as_ref(), expected.as_bytes()), - None => assert!(matches!(chunk, Err(Error::InvalidResponse(_))), "{chunk:?}"), + None => assert!(chunk.unwrap_err().is_response()), } assert!(chunks.next().await.is_none()); } diff --git a/litellm-rust/crates/inference-messages/src/types.rs b/litellm-rust/crates/inference-messages/src/types.rs index 21eac0574ed..ae346259559 100644 --- a/litellm-rust/crates/inference-messages/src/types.rs +++ b/litellm-rust/crates/inference-messages/src/types.rs @@ -34,11 +34,11 @@ pub fn messages_body(body: Map) -> Result /// Python's `GenericLiteLLMParams(**kwargs)`. pub fn litellm_params(fields: Map) -> Result { serde_json::from_value(Value::Object(fields)) - .map_err(|err| Error::InvalidRequest(ErrorDetail::invalid("litellm params", err))) + .map_err(|err| Error::invalid_request(ErrorDetail::invalid("litellm params", err))) } pub(super) fn invalid_request(err: serde_json::Error) -> Error { - Error::InvalidRequest(ErrorDetail::invalid("Anthropic messages request", err)) + Error::invalid_request(ErrorDetail::invalid("Anthropic messages request", err)) } pub type MessagesCallResponse = diff --git a/litellm-rust/crates/inference-messages/tests/bedrock_native/main.rs b/litellm-rust/crates/inference-messages/tests/bedrock_native/main.rs index 5420477c34a..f91d5c79f7c 100644 --- a/litellm-rust/crates/inference-messages/tests/bedrock_native/main.rs +++ b/litellm-rust/crates/inference-messages/tests/bedrock_native/main.rs @@ -309,7 +309,14 @@ async fn deployment_workspace_is_required_and_sent_as_a_header( }) .await; if !configured { - assert!(matches!(result, Err(Error::Auth(_)))); + let Err(error) = result else { + panic!("the call fails"); + }; + assert!(!error.dispatched(), "{error:?}"); + assert!( + error.is_request() || error.is_missing_credential(), + "{error:?}" + ); assert!(received(&upstream).await.is_empty()); return; } diff --git a/litellm-rust/crates/inference-messages/tests/messages/host.rs b/litellm-rust/crates/inference-messages/tests/messages/host.rs index ef158ed29e7..48afae99cb5 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/host.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/host.rs @@ -301,7 +301,7 @@ async fn response_mode_follows_the_intercepted_request( let summaries = traces.summaries("litellm.route"); assert_eq!(summaries.len(), 1); let Some(expected_stream) = expected_stream else { - assert!(matches!(result, Err(Error::InvalidRequest(_)))); + assert!(result.unwrap_err().is_request()); assert!(received(&upstream).await.is_empty()); assert_eq!(summaries[0]["outcome"], "failure"); return; @@ -322,14 +322,17 @@ async fn a_before_send_failure_never_sends(call: MessagesCall) { let upstream = upstream([message_response()]).await; let host = RecordingHost::new( authenticated(call, upstream.uri()), - Box::new(|_| Err(Error::MissingField("vetoed by the host"))), + Box::new(|_| Err(litellm_llms::Error::MissingField("vetoed by the host").into())), ); let error = run_through(&host) .await .expect_err("the host failure fails the call"); - assert_eq!(error, Error::MissingField("vetoed by the host")); + assert_eq!( + error, + Error::from(litellm_llms::Error::MissingField("vetoed by the host")) + ); assert!(received(&upstream).await.is_empty()); assert!(host.raw_responses().is_empty()); } diff --git a/litellm-rust/crates/inference-messages/tests/messages/main.rs b/litellm-rust/crates/inference-messages/tests/messages/main.rs index 48431148b08..a01e571ec75 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/main.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/main.rs @@ -144,7 +144,7 @@ impl LocalMessagesHost { .lock() .unwrap_or_else(|error| error.into_inner()) .take() - .ok_or(Error::MissingField("messages request")) + .ok_or(litellm_llms::Error::MissingField("messages request").into()) } pub fn runtime(&self) -> litellm_host_native::in_process::Host<'_, (), Self, ()> { litellm_host_native::in_process::Host { diff --git a/litellm-rust/crates/inference-messages/tests/messages/request.rs b/litellm-rust/crates/inference-messages/tests/messages/request.rs index 0c4a1eba549..e53bdb8d252 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/request.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/request.rs @@ -108,13 +108,7 @@ async fn a_call_without_credentials_fails_before_sending( .await .expect_err("a call without credentials fails"); - assert!( - matches!( - error, - Error::Auth(litellm_auth::Error::MissingApiKey { .. }) - ), - "{error:?}" - ); + assert!(error.is_missing_credential(), "{error:?}"); assert!(received(&upstream).await.is_empty()); } @@ -185,7 +179,9 @@ async fn unsupported_providers_are_rejected_before_sending( .await .expect_err("unsupported provider errors"); - assert_eq!(error, Error::InvalidProvider(reported.into())); + assert!(error.is_request(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains(reported), "{report}"); } #[rstest] @@ -513,10 +509,9 @@ async fn unsupported_params_are_dropped_under_drop_params_and_rejected_without_i let error = run(shaped(false)) .await .expect_err("an unsupported param is rejected without drop_params"); - assert!( - matches!(&error, Error::InvalidRequest(message) if message.to_string().contains(rejected_as)), - "{error:?}" - ); + assert!(error.is_request(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains(rejected_as), "{report}"); assert!(received(&upstream).await.is_empty()); run_message(shaped(true)).await; diff --git a/litellm-rust/crates/inference-messages/tests/messages/response.rs b/litellm-rust/crates/inference-messages/tests/messages/response.rs index 8bf4038b88e..a0969860822 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/response.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/response.rs @@ -235,7 +235,7 @@ async fn an_unreadable_success_body_is_an_invalid_response( .await .expect_err("an unreadable body fails"); - assert!(matches!(error, Error::InvalidResponse(_)), "{error:?}"); + assert!(error.is_response(), "{error:?}"); } #[rstest] @@ -252,7 +252,9 @@ async fn a_provider_slower_than_the_timeout_fails_the_call(call: MessagesCall) { .await .expect_err("the call times out"); - assert!(matches!(error, Error::Http(_)), "{error:?}"); + assert!(error.dispatched(), "{error:?}"); + assert!(!error.is_request(), "{error:?}"); + assert_eq!(error.upstream_response(), None); } #[rstest] @@ -312,7 +314,7 @@ async fn the_facade_sends_through_the_injected_http_pool_configuration(call: Mes fn a_body_that_does_not_parse_is_an_invalid_request(#[case] raw: Value) { let error = messages_body(object(raw)).expect_err("the body is rejected"); - assert!(matches!(error, Error::InvalidRequest(_)), "{error:?}"); + assert!(error.is_request(), "{error:?}"); assert!( litellm_error::Report::new(&error) .to_string() diff --git a/litellm-rust/crates/inference-messages/tests/messages/secrets.rs b/litellm-rust/crates/inference-messages/tests/messages/secrets.rs index d4f7d2bf3a7..b4a3018ab93 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/secrets.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/secrets.rs @@ -109,7 +109,7 @@ async fn a_secret_manager_failure_fails_the_call_before_sending(call: MessagesCa .expect_err("a secret manager failure fails the call"); assert!( - matches!(&error, Error::Secret(source) if matches!(source.source_error(), litellm_secrets::Error::ManagedSecretMissing)), + matches!(&error, Error::Secret(_)) && !error.is_request(), "{error:?}" ); assert!(received(&upstream).await.is_empty()); diff --git a/litellm-rust/crates/inference-messages/tests/messages/stream.rs b/litellm-rust/crates/inference-messages/tests/messages/stream.rs index 777f7e7c3ba..4a8a061f764 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/stream.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/stream.rs @@ -326,7 +326,9 @@ async fn the_timeout_covers_a_stalled_stream_body(call: MessagesCall) { .expect("the stalled stream gives up within the timeout") .expect_err("a stalled body fails the call"); - assert!(matches!(error, Error::Http(_)), "{error:?}"); + assert!(error.dispatched(), "{error:?}"); + assert!(!error.is_request(), "{error:?}"); + assert_eq!(error.upstream_response(), None); let seen = host.seen.into_inner().unwrap(); assert!( matches!(seen.as_slice(), [Seen::Open(_), Seen::Deliver(chunk)] if chunk.as_ref() == b"event: message_start\ndata: {}\n\n"), @@ -458,7 +460,9 @@ async fn the_sdk_yields_a_body_error_once_after_delivered_chunks(call: MessagesC .expect("the stalled body times out") .unwrap() .unwrap_err(); - assert!(matches!(error, Error::Http(_)), "{error:?}"); + assert!(error.dispatched(), "{error:?}"); + assert!(!error.is_request(), "{error:?}"); + assert_eq!(error.upstream_response(), None); assert!(chunks.next().await.is_none()); tokio::time::timeout(Duration::from_secs(5), connection) .await diff --git a/litellm-rust/crates/inference-messages/tests/messages/tencent.rs b/litellm-rust/crates/inference-messages/tests/messages/tencent.rs index dce187f4ea2..4bab7de65e1 100644 --- a/litellm-rust/crates/inference-messages/tests/messages/tencent.rs +++ b/litellm-rust/crates/inference-messages/tests/messages/tencent.rs @@ -79,13 +79,9 @@ async fn missing_credentials_fail_before_sending(tencent_call: MessagesCall) { }) .await .unwrap_err(); - assert!(matches!( - error, - Error::Auth(litellm_auth::Error::MissingApiKey { - environment_variable: "TENCENT_API_KEY", - .. - }) - )); + assert!(error.is_missing_credential(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains("TENCENT_API_KEY"), "{report}"); assert!(received(&upstream).await.is_empty()); } diff --git a/litellm-rust/crates/inference-messages/tests/openrouter/main.rs b/litellm-rust/crates/inference-messages/tests/openrouter/main.rs index 74444142d37..80243718768 100644 --- a/litellm-rust/crates/inference-messages/tests/openrouter/main.rs +++ b/litellm-rust/crates/inference-messages/tests/openrouter/main.rs @@ -1,7 +1,7 @@ use std::{sync::Arc, time::Duration}; use futures_util::TryStreamExt; -use litellm_inference_messages::{Error, MessagesCall, MessagesCallResponse, MessagesShaping}; +use litellm_inference_messages::{MessagesCall, MessagesCallResponse, MessagesShaping}; use litellm_inference_testing::{RecordingSecrets, no_secrets}; use rstest::{fixture, rstest}; use serde_json::json; @@ -92,13 +92,12 @@ async fn an_anthropic_key_alone_is_not_an_openrouter_credential(call: MessagesCa None, ) .await; - assert!(matches!( - result, - Err(Error::Auth(litellm_auth::Error::MissingApiKey { - provider: "OpenRouter", - .. - })) - )); + let Err(error) = result else { + panic!("the call fails"); + }; + assert!(error.is_missing_credential(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains("OpenRouter"), "{report}"); assert!(received(&upstream).await.is_empty()); } diff --git a/litellm-rust/crates/inference-ocr/src/provider_config.rs b/litellm-rust/crates/inference-ocr/src/provider_config.rs index 09041e9b72b..19e50af0dd1 100644 --- a/litellm-rust/crates/inference-ocr/src/provider_config.rs +++ b/litellm-rust/crates/inference-ocr/src/provider_config.rs @@ -1,6 +1,8 @@ use litellm_auth::{InputSource, SecretValue, Sourced}; -use litellm_core_utils::get_llm_provider_logic::{LlmProviders, get_custom_llm_provider}; -use litellm_inference::provider::{ResolvedProvider, resolve_llm_provider}; +use litellm_core_utils::get_llm_provider_logic::{ + CustomLlmProvider, LlmProviders, get_custom_llm_provider, +}; +use litellm_inference::provider::ResolvedProvider; use litellm_llms::{ aws_textract::ocr::{ analyze_transformation::TextractAnalyzeDocumentConfig, common_utils::TextractOperation, @@ -241,18 +243,20 @@ pub(crate) fn resolve_provider_config<'a>( model: &'a str, custom_llm_provider: Option<&'a str>, ) -> Result<(ResolvedProvider<'a>, OcrConfigKind), Error> { - let provider = match resolve_llm_provider(model, custom_llm_provider, "ocr") { - Ok(provider) => provider, - Err(_) if get_custom_llm_provider(model, custom_llm_provider).is_none() => { - ResolvedProvider { - model, - provider: LlmProviders::Mistral, - } - } - Err(litellm_inference::Error::InvalidProvider(provider)) => { - return Err(Error::InvalidProvider(provider)); - } - Err(error) => return Err(Error::InvalidProvider(error.to_string())), + let provider = match get_custom_llm_provider(model, custom_llm_provider) { + None => ResolvedProvider { + model, + provider: LlmProviders::Mistral, + }, + Some(CustomLlmProvider { + model, + custom_llm_provider, + }) => ResolvedProvider { + model, + provider: custom_llm_provider + .parse() + .map_err(|_| Error::InvalidProvider(custom_llm_provider.to_owned()))?, + }, }; let config = match provider.provider { LlmProviders::AwsTextract => match TextractOperation::from_model(provider.model)? { diff --git a/litellm-rust/crates/inference-ocr/tests/ocr/azure_ai.rs b/litellm-rust/crates/inference-ocr/tests/ocr/azure_ai.rs index 4c6add753ce..6f591af73e0 100644 --- a/litellm-rust/crates/inference-ocr/tests/ocr/azure_ai.rs +++ b/litellm-rust/crates/inference-ocr/tests/ocr/azure_ai.rs @@ -222,10 +222,8 @@ async fn credential_precedence( false, json!({}), numbered_token, - |error: &Error| matches!(error, Error::Auth(litellm_auth::Error::MissingApiBase { - provider: "Azure AI", - guidance: "Set AZURE_AI_API_BASE environment variable or pass api_base parameter", - })), + |error: &Error| litellm_error::Report::new(error).to_string() + == "Missing Azure AI API Base - Set AZURE_AI_API_BASE environment variable or pass api_base parameter", 0 )] #[case::unsupported_oidc_reference( @@ -239,7 +237,8 @@ async fn credential_precedence( true, json!({"azure_ad_token": "static-token"}), |_| String::new(), - |error: &Error| error.is_missing_credential() && error.to_string().contains("Azure AI credentials"), + |error: &Error| error.is_missing_credential() + && litellm_error::Report::new(error).to_string().contains("Azure AI credentials"), 1 )] #[tokio::test] diff --git a/litellm-rust/crates/inference-ocr/tests/ocr/documents.rs b/litellm-rust/crates/inference-ocr/tests/ocr/documents.rs index 96cb164cdb6..35de1326386 100644 --- a/litellm-rust/crates/inference-ocr/tests/ocr/documents.rs +++ b/litellm-rust/crates/inference-ocr/tests/ocr/documents.rs @@ -216,10 +216,8 @@ async fn configured_client_preserves_document_url_policy(#[case] allowed: bool) .await; if !allowed { - assert!( - matches!(&result, Err(Error::Http(error)) if error.is_request()), - "{result:?}" - ); + let error = result.unwrap_err(); + assert!(error.is_request(), "{error:?}"); assert!(received(&documents).await.is_empty()); assert!(received(&upstream).await.is_empty()); return; diff --git a/litellm-rust/crates/inference-responses/Cargo.toml b/litellm-rust/crates/inference-responses/Cargo.toml index 3dcd9976e0a..083179c50c8 100644 --- a/litellm-rust/crates/inference-responses/Cargo.toml +++ b/litellm-rust/crates/inference-responses/Cargo.toml @@ -28,6 +28,7 @@ tracing.workspace = true litellm-http = { workspace = true, features = ["test-support"] } litellm-cache.workspace = true litellm-cache-memory.workspace = true +litellm-error.workspace = true litellm-host-native.workspace = true litellm-inference-testing.workspace = true litellm-tracing.workspace = true diff --git a/litellm-rust/crates/inference-responses/src/handler.rs b/litellm-rust/crates/inference-responses/src/handler.rs index 5160533fae1..8d3d1075e8e 100644 --- a/litellm-rust/crates/inference-responses/src/handler.rs +++ b/litellm-rust/crates/inference-responses/src/handler.rs @@ -73,10 +73,12 @@ pub(super) async fn execute( None => false, Some(serde_json::Value::Bool(value)) => *value, Some(_) => { - return Err(Error::InvalidRequest(litellm_llms::ErrorDetail::Expected { - subject: "stream", - expected: "a boolean", - })); + return Err(Error::invalid_request( + litellm_llms::ErrorDetail::Expected { + subject: "stream", + expected: "a boolean", + }, + )); } }; let outbound = litellm_inference::outbound::outbound_request( @@ -91,7 +93,7 @@ pub(super) async fn execute( let response = litellm_inference::outbound::send(outbound, &route.http).await?; let status = response.status().as_u16(); if !response.status().is_success() { - let body = response.text().await.map_err(network)?; + let body = response.text().await.map_err(litellm_http::Error::from)?; return Err(litellm_http::Error::upstream(status, body).into()); } if stream { @@ -104,14 +106,14 @@ pub(super) async fn execute( .collect(); let chunks = response .bytes_stream() - .map(|chunk| chunk.map_err(network)) + .map(|chunk| chunk.map_err(|error| litellm_http::Error::from(error).into())) .boxed(); return Ok(ResponsesOutput::Stream { head: ResponsesStreamHead { headers }, chunks, }); } - let body = response.text().await.map_err(network)?; + let body = response.text().await.map_err(litellm_http::Error::from)?; let raw = RawResponse { body: body.clone() }; if let Some(observers) = observers { observers.emit(litellm_host::lifecycle::CallEvent::Execution( @@ -123,7 +125,7 @@ pub(super) async fn execute( .await .map_err(Error::post_call)?; let value = serde_json::from_str(&body).map_err(|error| { - Error::InvalidResponse(litellm_llms::ErrorDetail::invalid( + Error::invalid_response(litellm_llms::ErrorDetail::invalid( "Responses API response JSON", error, )) @@ -136,7 +138,3 @@ pub(super) async fn execute( ) .await } - -fn network(error: reqwest::Error) -> Error { - litellm_http::Error::from(error).into() -} diff --git a/litellm-rust/crates/inference-responses/src/websocket.rs b/litellm-rust/crates/inference-responses/src/websocket.rs index c515493d0ef..2afcde0a7a4 100644 --- a/litellm-rust/crates/inference-responses/src/websocket.rs +++ b/litellm-rust/crates/inference-responses/src/websocket.rs @@ -7,7 +7,7 @@ use litellm_http::{ }; use tokio::sync::Mutex; use tokio_tungstenite::tungstenite::{ - self, Message, + Message, http::{HeaderMap, HeaderName, HeaderValue}, }; @@ -39,13 +39,13 @@ impl ResponsesWebSocketConnection { .map(|(name, value)| { Ok(( name.parse::().map_err(|error| { - Error::InvalidRequest(ErrorDetail::invalid( + Error::invalid_request(ErrorDetail::invalid( "WebSocket header name", error, )) })?, HeaderValue::from_str(value).map_err(|error| { - Error::InvalidRequest(ErrorDetail::invalid( + Error::invalid_request(ErrorDetail::invalid( "WebSocket header value", error, )) @@ -71,7 +71,7 @@ impl ResponsesWebSocketConnection { litellm_inference::diagnostic::operation("litellm.websocket.send_text", async { let mut socket = self.socket.lock().await; let Some(socket) = socket.as_mut() else { - return Err(litellm_http::Error::from(tungstenite::Error::AlreadyClosed).into()); + return Err(litellm_http::Error::websocket_closed().into()); }; socket .send(Message::Text(text)) @@ -98,7 +98,7 @@ impl ResponsesWebSocketConnection { Some(Ok(Message::Binary(bytes))) => String::from_utf8(bytes.to_vec()) .map(Some) .map_err(|error| { - Error::InvalidResponse(ErrorDetail::invalid("WebSocket text frame", error)) + Error::invalid_response(ErrorDetail::invalid("WebSocket text frame", error)) }), Some(Ok(Message::Close(_))) | None => Ok(None), Some(Ok(_)) => Ok(None), diff --git a/litellm-rust/crates/inference-responses/tests/openrouter.rs b/litellm-rust/crates/inference-responses/tests/openrouter.rs index e52cdba8ce8..8cdb68c376f 100644 --- a/litellm-rust/crates/inference-responses/tests/openrouter.rs +++ b/litellm-rust/crates/inference-responses/tests/openrouter.rs @@ -1,10 +1,7 @@ use std::sync::Arc; use futures_util::TryStreamExt; -use litellm_inference_responses::{ - Error, - types::{ResponsesCall, ResponsesOutput}, -}; +use litellm_inference_responses::types::{ResponsesCall, ResponsesOutput}; use litellm_inference_testing::{RecordingSecrets, no_secrets}; use rstest::{fixture, rstest}; use serde_json::json; @@ -108,13 +105,12 @@ async fn a_missing_key_fails_before_transport(call: ResponsesCall) { None, ) .await; - assert!(matches!( - result, - Err(Error::Auth(litellm_auth::Error::MissingApiKey { - provider: "OpenRouter", - .. - })) - )); + let Err(error) = result else { + panic!("the call fails"); + }; + assert!(error.is_missing_credential(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains("OpenRouter"), "{report}"); assert!(received(&upstream).await.is_empty()); } diff --git a/litellm-rust/crates/inference-responses/tests/responses.rs b/litellm-rust/crates/inference-responses/tests/responses.rs index 0d7767fd004..66402e595eb 100644 --- a/litellm-rust/crates/inference-responses/tests/responses.rs +++ b/litellm-rust/crates/inference-responses/tests/responses.rs @@ -576,12 +576,11 @@ async fn websocket_operations_trace_outcomes_without_capturing_frames_or_credent Some("private-frame-sentinel") ); connection.close().await.unwrap(); - assert!( - connection - .send_text("private-frame-sentinel".into()) - .await - .is_err() - ); + let error = connection + .send_text("private-frame-sentinel".into()) + .await + .expect_err("a closed connection sends nothing"); + assert!(!error.dispatched(), "{error:?}"); }) .await; server.await.unwrap(); diff --git a/litellm-rust/crates/inference-transcription/Cargo.toml b/litellm-rust/crates/inference-transcription/Cargo.toml index c501d329852..cf50b3f84fd 100644 --- a/litellm-rust/crates/inference-transcription/Cargo.toml +++ b/litellm-rust/crates/inference-transcription/Cargo.toml @@ -17,6 +17,7 @@ serde_json.workspace = true tracing.workspace = true [dev-dependencies] +litellm-error.workspace = true litellm-inference-testing.workspace = true litellm-tracing.workspace = true rstest.workspace = true diff --git a/litellm-rust/crates/inference-transcription/src/handler.rs b/litellm-rust/crates/inference-transcription/src/handler.rs index 93be7387121..ba7eb740e2f 100644 --- a/litellm-rust/crates/inference-transcription/src/handler.rs +++ b/litellm-rust/crates/inference-transcription/src/handler.rs @@ -62,7 +62,7 @@ pub(super) async fn execute( return Err(litellm_http::Error::upstream(status.as_u16(), text).into()); } let response_json = serde_json::from_str(&text).map_err(|error| { - Error::InvalidResponse(litellm_llms::ErrorDetail::invalid( + Error::invalid_response(litellm_llms::ErrorDetail::invalid( "audio response JSON", error, )) diff --git a/litellm-rust/crates/inference-transcription/tests/audio_transcription.rs b/litellm-rust/crates/inference-transcription/tests/audio_transcription.rs index d99b995fc21..9e48edce0c0 100644 --- a/litellm-rust/crates/inference-transcription/tests/audio_transcription.rs +++ b/litellm-rust/crates/inference-transcription/tests/audio_transcription.rs @@ -152,13 +152,7 @@ async fn invalid_audio_is_rejected_before_sending( .await .expect_err("invalid audio is rejected"); - assert!( - matches!( - error, - Error::InvalidRequest(_) | Error::MissingField(_) | Error::InvalidType { .. } - ), - "{error:?}" - ); + assert!(error.is_request(), "{error:?}"); assert!(received(&upstream).await.is_empty()); } @@ -185,7 +179,9 @@ async fn unsupported_providers_are_rejected_before_sending( .await .expect_err("unsupported provider errors"); - assert_eq!(error, Error::InvalidProvider(reported.into())); + assert!(error.is_request(), "{error:?}"); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains(reported), "{report}"); } #[rstest] @@ -262,7 +258,7 @@ async fn an_unreadable_success_body_is_an_invalid_response( .await .expect_err("an unreadable body fails"); - assert!(matches!(error, Error::InvalidResponse(_)), "{error:?}"); + assert!(error.is_response(), "{error:?}"); } #[rstest] diff --git a/litellm-rust/crates/inference/AGENTS.md b/litellm-rust/crates/inference/AGENTS.md index 7f2bd0387be..fe9830581ec 100644 --- a/litellm-rust/crates/inference/AGENTS.md +++ b/litellm-rust/crates/inference/AGENTS.md @@ -53,10 +53,10 @@ ## Error placement -- The workspace `Error definitions` rules shape each crate's error. This section decides which crate and module a failure belongs to +- The [rust-error skill](../../.agents/skills/rust-error/SKILL.md) shapes each crate's error. This section decides which crate and module a failure belongs to - A failure is declared once, by the lowest crate that raises it - - every crate above nests it unchanged (`#[error(transparent)] Auth(#[from] litellm_auth::Error)`) or maps it once at its boundary, as `src/error.rs` does for `litellm_llms::Error` - - `Error` collects route failures for every format crate and never re-declares a variant a lower crate raises + - every crate above nests it in one variant and delegates its predicates, as `Error::Llm`, `Error::Auth` and `Error::Http` do, or translates it once in the shared `From` when its meaning changes, as `From` does for `Unsupported` and a header the signer computes + - `Error` collects route failures for every format crate and never re-declares a variant a lower crate raises. Routes build transformation failures with `Error::invalid_request` and `Error::invalid_response`, and callers ask `is_request()`, `is_response()`, `is_missing_credential()`, `dispatched()` and `upstream_response()` instead of matching nested variants - Scope follows the concept, not the first caller - an error type under `litellm-llms`'s `/` is private to that provider: no other provider and nothing in `base_llm` may import it - a failure two providers or two routes can hit (wire framing, stream event decoding, a malformed provider response) belongs to the crate that owns the concept: `litellm-framer` for framing, `litellm_llms::Error` for the transformation layer diff --git a/litellm-rust/crates/inference/src/error.rs b/litellm-rust/crates/inference/src/error.rs index 12a3aa2f6b2..684f788eb24 100644 --- a/litellm-rust/crates/inference/src/error.rs +++ b/litellm-rust/crates/inference/src/error.rs @@ -1,32 +1,19 @@ -//! One error for every route in this crate. OCR still carries its own, richer enum. -//! -//! A variant is declared by the layer that produces it and nested here as is: -//! credentials by `litellm_auth` (AWS folds into it at that crate's boundary), the wire by -//! `litellm_http`, secrets by `litellm_secrets`. The transformation layer's [`LlmError`] -//! maps onto the same-named variants once, here, so no route re-declares them. - use std::sync::Arc; -use litellm_llms::Error as LlmError; +use litellm_host::machine::MachineFault; +use litellm_llms::{Error as LlmError, ErrorDetail}; #[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)] pub enum Error { - #[error("expected {expected}, got {actual}")] - InvalidType { - expected: &'static str, - actual: &'static str, - }, - #[error("missing required field: {0}")] - MissingField(&'static str), #[error("invalid provider: {0}")] InvalidProvider(String), - #[error("invalid request")] - InvalidRequest(#[source] litellm_llms::ErrorDetail), - #[error("invalid response")] - InvalidResponse(#[source] litellm_llms::ErrorDetail), + #[error("invalid provider: unable to resolve custom_llm_provider for {route} request")] + UnresolvedProvider { route: &'static str }, #[error("unsupported by the rust path: {0}")] Unsupported(&'static str), #[error(transparent)] + Llm(LlmError), + #[error(transparent)] Auth(#[from] litellm_auth::Error), #[error(transparent)] Headers(#[from] litellm_http::request::HeaderError), @@ -34,17 +21,27 @@ pub enum Error { Http(litellm_http::Error), #[error(transparent)] Secret(#[from] SecretError), - #[error("post-call hook failed: {0}")] - PostCallHook(Arc), + #[error("the host driver failed")] + Host(#[source] MachineFault), + #[error("post-call hook failed")] + PostCallHook(#[source] Arc), } -impl From for Error { - fn from(fault: litellm_host::machine::MachineFault) -> Self { - Self::InvalidRequest(litellm_llms::ErrorDetail::failed("host driver", fault)) +impl From for Error { + fn from(fault: MachineFault) -> Self { + Self::Host(fault) } } impl Error { + pub fn invalid_request(detail: ErrorDetail) -> Self { + Self::Llm(LlmError::InvalidRequest(detail)) + } + + pub fn invalid_response(detail: ErrorDetail) -> Self { + Self::Llm(LlmError::InvalidResponse(detail)) + } + pub fn post_call(error: Self) -> Self { Self::PostCallHook(Arc::new(error)) } @@ -53,32 +50,88 @@ impl Error { /// the provider's answer. pub fn is_request(&self) -> bool { match self { - Self::InvalidType { .. } - | Self::MissingField(_) - | Self::InvalidProvider(_) - | Self::InvalidRequest(_) + Self::InvalidProvider(_) + | Self::UnresolvedProvider { .. } | Self::Unsupported(_) | Self::Headers(_) => true, + Self::Llm(error) => error.is_request(), Self::Auth(error) => !error.is_missing_credential(), Self::Http(error) => error.is_request(), - Self::InvalidResponse(_) | Self::Secret(_) | Self::PostCallHook(_) => false, + Self::Secret(_) | Self::Host(_) | Self::PostCallHook(_) => false, + } + } + + pub fn is_response(&self) -> bool { + match self { + Self::Llm(error) => error.is_response(), + Self::Http(error) => error.is_response(), + Self::InvalidProvider(_) + | Self::UnresolvedProvider { .. } + | Self::Unsupported(_) + | Self::Auth(_) + | Self::Headers(_) + | Self::Secret(_) + | Self::Host(_) + | Self::PostCallHook(_) => false, } } pub fn is_missing_credential(&self) -> bool { - matches!(self, Self::Auth(error) if error.is_missing_credential()) + match self { + Self::Llm(error) => error.is_missing_credential(), + Self::Auth(error) => error.is_missing_credential(), + Self::InvalidProvider(_) + | Self::UnresolvedProvider { .. } + | Self::Unsupported(_) + | Self::Headers(_) + | Self::Http(_) + | Self::Secret(_) + | Self::Host(_) + | Self::PostCallHook(_) => false, + } } pub fn is_unsupported(&self) -> bool { - matches!(self, Self::Unsupported(_)) + match self { + Self::Unsupported(_) => true, + Self::InvalidProvider(_) + | Self::UnresolvedProvider { .. } + | Self::Llm(_) + | Self::Auth(_) + | Self::Headers(_) + | Self::Http(_) + | Self::Secret(_) + | Self::Host(_) + | Self::PostCallHook(_) => false, + } + } + + pub fn dispatched(&self) -> bool { + match self { + Self::Llm(error) => error.dispatched(), + Self::Http(error) => error.dispatched(), + Self::Host(_) | Self::PostCallHook(_) => true, + Self::InvalidProvider(_) + | Self::UnresolvedProvider { .. } + | Self::Unsupported(_) + | Self::Auth(_) + | Self::Headers(_) + | Self::Secret(_) => false, + } } -} -impl Error { pub fn upstream_response(&self) -> Option<(u16, &str)> { match self { + Self::Llm(error) => error.upstream_response(), Self::Http(error) => error.upstream_response(), - _ => None, + Self::InvalidProvider(_) + | Self::UnresolvedProvider { .. } + | Self::Unsupported(_) + | Self::Auth(_) + | Self::Headers(_) + | Self::Secret(_) + | Self::Host(_) + | Self::PostCallHook(_) => None, } } } @@ -97,13 +150,13 @@ impl From for Error { impl From for Error { fn from(error: LlmError) -> Self { match error { - LlmError::InvalidType { expected, actual } => Self::InvalidType { expected, actual }, - LlmError::MissingField(field) => Self::MissingField(field), - LlmError::InvalidRequest(message) => Self::InvalidRequest(message), - LlmError::InvalidResponse(message) => Self::InvalidResponse(message), LlmError::Unsupported(reason) => Self::Unsupported(reason), - LlmError::Auth(error) => Self::Auth(error), LlmError::Http(error) => Self::from(error), + LlmError::Auth(error) => Self::Auth(error), + error @ (LlmError::InvalidType { .. } + | LlmError::MissingField(_) + | LlmError::InvalidRequest(_) + | LlmError::InvalidResponse(_)) => Self::Llm(error), } } } @@ -135,45 +188,120 @@ impl Eq for SecretError {} #[cfg(test)] mod tests { use super::Error; + use litellm_host::machine::MachineFault; use litellm_llms::{Error as LlmError, ErrorDetail}; use rstest::rstest; - #[test] - fn a_missing_api_key_is_the_environment_not_the_request() { - assert!( - !Error::Auth(litellm_auth::Error::MissingApiKey { - provider: "Anthropic", - environment_variable: "ANTHROPIC_API_KEY", - }) - .is_request() - ); - assert!(Error::Auth(litellm_auth::Error::InvalidHeader).is_request()); - assert!( - Error::InvalidRequest(ErrorDetail::Expected { - subject: "top_k", - expected: "an integer", - }) - .is_request() - ); - assert!( - !Error::InvalidResponse(ErrorDetail::MissingField { - subject: "response", - field: "usage", - }) - .is_request() - ); + #[rstest] + #[case::missing_api_key( + litellm_auth::Error::MissingApiKey { + provider: "Anthropic", + environment_variable: "ANTHROPIC_API_KEY", + } + .into(), + false + )] + #[case::invalid_auth_header(litellm_auth::Error::InvalidHeader.into(), true)] + #[case::transformation_missing_api_key( + LlmError::Auth(litellm_auth::Error::MissingApiKey { + provider: "Anthropic", + environment_variable: "ANTHROPIC_API_KEY", + }) + .into(), + false + )] + #[case::invalid_request( + LlmError::InvalidRequest(ErrorDetail::Expected { + subject: "top_k", + expected: "an integer", + }) + .into(), + true + )] + #[case::invalid_response( + LlmError::InvalidResponse(ErrorDetail::MissingField { + subject: "response", + field: "usage", + }) + .into(), + false + )] + #[case::host_driver(MachineFault::Abandoned.into(), false)] + #[case::post_call_hook_over_a_request_fault( + Error::post_call(LlmError::MissingField("usage").into()), + false + )] + fn only_the_callers_input_is_a_request_fault(#[case] error: Error, #[case] is_request: bool) { + assert_eq!(error.is_request(), is_request); } - #[test] - fn a_header_the_signer_computes_is_left_to_the_host_on_every_route() { - let error = Error::from(litellm_http::Error::computed_header("authorization")); - assert!(matches!(error, Error::Unsupported(_)), "{error:?}"); + + #[rstest] + #[case::own_auth(litellm_auth::Error::MissingApiKey { + provider: "Anthropic", + environment_variable: "ANTHROPIC_API_KEY", + } + .into())] + #[case::transformation_auth( + LlmError::Auth(litellm_auth::Error::MissingApiKey { + provider: "Anthropic", + environment_variable: "ANTHROPIC_API_KEY", + }) + .into() + )] + fn a_missing_api_key_is_a_missing_credential_wherever_it_is_raised(#[case] error: Error) { + assert!(error.is_missing_credential()); + } + + #[rstest] + #[case::direct(litellm_http::Error::computed_header("authorization").into())] + #[case::through_a_transformation( + LlmError::Http(litellm_http::Error::computed_header("authorization")).into() + )] + fn a_header_the_signer_computes_is_left_to_the_host_on_every_route(#[case] error: Error) { + assert!(error.is_unsupported(), "{error:?}"); assert!(error.is_request()); } + #[test] + fn a_transformation_unsupported_stays_a_decline() { + assert!(Error::from(LlmError::Unsupported("test capability")).is_unsupported()); + } + + #[test] + fn an_upstream_failure_through_a_transformation_keeps_the_provider_answer() { + let error = Error::from(LlmError::Http(litellm_http::Error::upstream( + 429, + "slow down".into(), + ))); + assert_eq!(error.upstream_response(), Some((429, "slow down"))); + assert!(error.dispatched()); + } + + #[test] + fn a_post_call_hook_failure_is_not_the_providers_answer() { + let error = + Error::post_call(litellm_http::Error::upstream(500, "hook backend down".into()).into()); + assert_eq!(error.upstream_response(), None); + assert!(!error.is_request()); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.starts_with("post-call hook failed: "), "{report}"); + assert!(report.contains("hook backend down"), "{report}"); + } + + #[test] + fn a_host_driver_fault_keeps_the_fault_as_its_source() { + let error = Error::from(MachineFault::Abandoned); + assert_eq!( + litellm_error::find::(&error), + Some(&MachineFault::Abandoned) + ); + assert!(error.dispatched()); + } + #[rstest] #[case::request(true)] #[case::response(false)] - fn contextual_errors_render_their_cause_and_route_classification(#[case] request: bool) { + fn contextual_errors_keep_their_cause_and_route_classification(#[case] request: bool) { let source = serde_json::from_str::("{").unwrap_err(); let source_message = source.to_string(); let detail = ErrorDetail::invalid("test payload", source); @@ -183,11 +311,10 @@ mod tests { LlmError::InvalidResponse(detail) }); assert_eq!(error.is_request(), request); - let category = if request { "request" } else { "response" }; - assert_eq!( - litellm_error::Report::new(&error).to_string(), - format!("invalid {category}: invalid test payload: {source_message}") - ); + assert_eq!(error.is_response(), !request); + let report = litellm_error::Report::new(&error).to_string(); + assert!(report.contains("test payload"), "{report}"); + assert_eq!(report.matches(&source_message).count(), 1, "{report}"); assert!(litellm_error::find::(&error).is_some()); } } diff --git a/litellm-rust/crates/inference/src/provider.rs b/litellm-rust/crates/inference/src/provider.rs index 305aca85f97..fa226a4ef8d 100644 --- a/litellm-rust/crates/inference/src/provider.rs +++ b/litellm-rust/crates/inference/src/provider.rs @@ -25,11 +25,7 @@ pub fn resolve_llm_provider<'a>( custom_llm_provider: provider, }) }) - .ok_or_else(|| { - Error::InvalidProvider(format!( - "unable to resolve custom_llm_provider for {route} request" - )) - })?; + .ok_or(Error::UnresolvedProvider { route })?; let provider = custom_llm_provider .parse() .map_err(|_| Error::InvalidProvider(custom_llm_provider.to_string()))?; diff --git a/litellm-rust/crates/llms/src/error.rs b/litellm-rust/crates/llms/src/error.rs index 6728f68e3b8..ce4d086cf73 100644 --- a/litellm-rust/crates/llms/src/error.rs +++ b/litellm-rust/crates/llms/src/error.rs @@ -21,6 +21,70 @@ pub enum Error { Http(#[from] litellm_http::Error), } +impl Error { + pub fn is_request(&self) -> bool { + match self { + Self::InvalidType { .. } + | Self::MissingField(_) + | Self::InvalidRequest(_) + | Self::Unsupported(_) => true, + Self::Auth(error) => !error.is_missing_credential(), + Self::Http(error) => error.is_request(), + Self::InvalidResponse(_) => false, + } + } + + pub fn is_response(&self) -> bool { + match self { + Self::InvalidResponse(_) => true, + Self::Http(error) => error.is_response(), + Self::InvalidType { .. } + | Self::MissingField(_) + | Self::InvalidRequest(_) + | Self::Unsupported(_) + | Self::Auth(_) => false, + } + } + + pub fn is_missing_credential(&self) -> bool { + match self { + Self::Auth(error) => error.is_missing_credential(), + Self::InvalidType { .. } + | Self::MissingField(_) + | Self::InvalidRequest(_) + | Self::InvalidResponse(_) + | Self::Unsupported(_) + | Self::Http(_) => false, + } + } + + /// A transformation runs on both sides of the send, so only the transport and credential + /// resolution, which always runs before it, can tell whether the request reached the provider. + pub fn dispatched(&self) -> bool { + match self { + Self::Http(error) => error.dispatched(), + Self::Auth(_) => false, + Self::InvalidType { .. } + | Self::MissingField(_) + | Self::InvalidRequest(_) + | Self::InvalidResponse(_) + | Self::Unsupported(_) => true, + } + } + + pub fn upstream_response(&self) -> Option<(u16, &str)> { + match self { + Self::Http(error) => error.upstream_response(), + Self::InvalidType { .. } + | Self::MissingField(_) + | Self::InvalidRequest(_) + | Self::InvalidResponse(_) + | Self::Unsupported(_) + | Self::Auth(_) => None, + } + } +} + #[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)] pub enum ErrorDetail { #[error("invalid {subject}")] diff --git a/litellm-rust/crates/python-bridge/src/errors.rs b/litellm-rust/crates/python-bridge/src/errors.rs index c51b17b499a..b835e2ee63d 100644 --- a/litellm-rust/crates/python-bridge/src/errors.rs +++ b/litellm-rust/crates/python-bridge/src/errors.rs @@ -45,7 +45,7 @@ mod tests { #[case::unsupported(Error::Unsupported("test capability"), true)] #[case::invalid_provider(Error::InvalidProvider("unknown".into()), true)] #[case::invalid_request( - Error::InvalidRequest(litellm_llms::ErrorDetail::Expected { + Error::invalid_request(litellm_llms::ErrorDetail::Expected { subject: "messages", expected: "non-empty", }), @@ -56,7 +56,7 @@ mod tests { false )] #[case::invalid_response( - Error::InvalidResponse(litellm_llms::ErrorDetail::MissingField { + Error::invalid_response(litellm_llms::ErrorDetail::MissingField { subject: "response", field: "usage", }), diff --git a/litellm-rust/crates/python-bridge/src/routes/chat_completions/host.rs b/litellm-rust/crates/python-bridge/src/routes/chat_completions/host.rs index a221e3369ff..752a78dcda7 100644 --- a/litellm-rust/crates/python-bridge/src/routes/chat_completions/host.rs +++ b/litellm-rust/crates/python-bridge/src/routes/chat_completions/host.rs @@ -79,7 +79,7 @@ impl PythonBinding for ChatCompletionsPythonHost { self.0.error(py, error) } fn host_error(_: &PyErr) -> Error { - Error::InvalidRequest(litellm_llms::ErrorDetail::HostCall) + Error::invalid_request(litellm_llms::ErrorDetail::HostCall) } } diff --git a/litellm-rust/crates/python-bridge/src/routes/messages/host.rs b/litellm-rust/crates/python-bridge/src/routes/messages/host.rs index a46061dcaec..0d5b5d5b122 100644 --- a/litellm-rust/crates/python-bridge/src/routes/messages/host.rs +++ b/litellm-rust/crates/python-bridge/src/routes/messages/host.rs @@ -76,12 +76,12 @@ fn native_error(py: Python<'_>, error: Error) -> PyResult { return Ok(error); } match error { - Error::InvalidRequest(message) => { + Error::Llm(litellm_llms::Error::InvalidRequest(message)) => { let error = PyValueError::new_err(litellm_error::Report::new(&message).to_string()); error.value(py).setattr(REQUEST_ERROR_MARKER, true)?; Ok(error) } - Error::MissingField(field) => { + Error::Llm(litellm_llms::Error::MissingField(field)) => { let error = PyValueError::new_err(format!("missing required field: {field}")); error.value(py).setattr(REQUEST_ERROR_MARKER, true)?; Ok(error) @@ -284,7 +284,7 @@ impl PythonBinding for MessagesPythonHost { } fn host_error(_: &PyErr) -> Error { - Error::InvalidRequest(litellm_llms::ErrorDetail::HostCall) + Error::invalid_request(litellm_llms::ErrorDetail::HostCall) } } @@ -458,7 +458,7 @@ mod tests { projected, litellm_params(serde_json::from_value(fields).unwrap()).unwrap() ), - (Err(error), Err(())) => assert!(matches!(error, Error::InvalidRequest(_))), + (Err(error), Err(())) => assert!(error.is_request(), "{error:?}"), (projected, expected) => panic!("got {projected:?}, expected {expected:?}"), } }); @@ -466,7 +466,7 @@ mod tests { #[rstest] #[case::rejected_request( - Error::InvalidRequest(litellm_llms::ErrorDetail::UnsupportedParameter { + Error::invalid_request(litellm_llms::ErrorDetail::UnsupportedParameter { model: "claude".into(), param: "top_k".into(), value: "5".into(), @@ -474,7 +474,7 @@ mod tests { }), true )] - #[case::missing_field(Error::MissingField("max_tokens"), true)] + #[case::missing_field(litellm_llms::Error::MissingField("max_tokens").into(), true)] #[case::unresolvable_provider(Error::InvalidProvider("openai".into()), false)] #[case::upstream_failure( litellm_http::Error::upstream(400, "bad".into()).into(), diff --git a/litellm-rust/crates/python-bridge/src/routes/responses/host.rs b/litellm-rust/crates/python-bridge/src/routes/responses/host.rs index 4d9455435d7..fdbc716d84e 100644 --- a/litellm-rust/crates/python-bridge/src/routes/responses/host.rs +++ b/litellm-rust/crates/python-bridge/src/routes/responses/host.rs @@ -161,7 +161,7 @@ impl PythonBinding for ResponsesPythonHost { self.host.error(py, error) } fn host_error(_: &PyErr) -> Error { - Error::InvalidRequest(litellm_llms::ErrorDetail::HostCall) + Error::invalid_request(litellm_llms::ErrorDetail::HostCall) } }