scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-30 06:43:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 382

refactor(rust-error): nest llms errors in inference and ask predicates in route tests

Browse source 
litellm_inference::Error nests litellm_llms::Error instead of mirroring
its variants, host driver faults get their own variant, every predicate
matches exhaustively, and route tests ask predicates instead of matching
transport variants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Yujong Lee 2026-10-23 13:20:38 -07:00
parent 204f146dc9
commit b5b542a68a
41 changed files with 505 additions and 311 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
litellm-rust/.agents/skills/rust-error/references/messages-and-causes.md
View file

@ -66,7 +66,7 @@ Store the lower-level error itself, never its string, and never drop it with `ma
| downcasting | the type is gone |
| redaction | `reqwest::Error`'s `Display` includes the request URL, which can carry credentials in the path or query |
If the cause is not `Clone` and the error must be, hold it as `#[source] litellm_error::Source`, a cloneable `Arc<dyn Error + Send + Sync>` that is transparent to `Display` and `source()`, or make the crate's error opaque. `litellm_error::find::<T>(&error)` walks the chain and looks through `Source`, so it is how anyone reaches a typed cause. A third-party `Box<dyn Error + Send + Sync>`, or an `anyhow::Error` a dependency returns, converts with `Source::from`, and a dependency's own boxed alias such as `tokenizers::Error` is held as it is. Never put `#[error(transparent)]` on a `Source`, the erased error would then be no node of the chain and `find` could not reach it. Do not stringify or drop a cause to get `Clone` or `PartialEq`
If the cause is not `Clone` and the error must be, hold it as `#[source] litellm_error::Source`, a cloneable `Arc<dyn Error + Send + Sync>` that is transparent to `Display` and `source()`, or make the crate's error opaque. `litellm_error::find::<T>(&error)` walks the chain and looks through `Source`, so it is how anyone reaches a typed cause. A third-party `Box<dyn Error + Send + Sync>`, or an `anyhow::Error` a dependency returns, converts with `Source::from`, and a dependency's own boxed alias such as `tokenizers::Error` is held as it is. A source boxed to keep the variant small is a `Box<E>` node of the chain, so it is reached as `find::<Box<E>>`. Never put `#[error(transparent)]` on a `Source`, the erased error would then be no node of the chain and `find` could not reach it. Do not stringify or drop a cause to get `Clone` or `PartialEq`
```rust
// loses the type and leaks the URL

2
litellm-rust/.agents/skills/rust-error/references/testing.md
View file

@ -20,7 +20,7 @@ In order of preference
2. A real cause, such as `tungstenite::Error::ConnectionClosed` or a `serde_json::from_str("{")` error, passed through our `From`
3. A public constructor, such as `Error::upstream(500, "boom".into())`
Inside the owning crate a test may match the private representation through the `#[cfg(test)] fn repr(&self) -> &Repr` accessor in `crates/http/src/error.rs`, as the `media.rs` and `tls.rs` tests do. Anywhere else, reach the cause the test expects with `litellm_error::find::<T>(&error)`
Inside the owning crate a test may match the private representation through the `#[cfg(test)] fn repr(&self) -> &Repr` accessor in `crates/http/src/error.rs`, as the `media.rs` and `tls.rs` tests do. Anywhere else, reach the cause the test expects with `litellm_error::find::<T>(&error)`. A transparent layer is no node of the chain, so `find` reaches foreign causes underneath it but never the nested error of ours itself. Ask that error's predicates through the outer type instead, which is what callers do too
## What to cover

4
litellm-rust/.agents/skills/rust-gateway/references/surfaces.md
View file

@ -8,10 +8,12 @@ Each surface mirrors what the Python proxy returns there, so the shapes differ o
| OpenAI-shaped routes | `{"error": {"message", "type", "param", "code"}}` | `Error::openai_response()` and `openai_sse_frame()` in the same file |
| MCP | `{"detail": message}` | `crates/gateway-mcp/src/error.rs` |
| UI | `{"error": {"message"}}`, plus `Retry-After` on 429 | `crates/gateway-ui/src/error.rs` |
| auth rejections | plain text | `crates/gateway-auth/src/error.rs` |
| auth rejections | plain text, the top-level message only | `crates/gateway-auth/src/error.rs` |
Every surface takes its status from one `status()` on its crate's `Error` and renders the message of anything that is not a provider answer with `litellm_error::Report`. In `gateway-inference` the error `type` string is derived from the status by `error_type()`, so a new failure never names its own type. The Anthropic envelope passes a provider body already in its shape through untouched and extracts the message from anything else, and the OpenAI envelope always wraps, as the Python proxy does. A provider body is bounded by `litellm_http::Error::upstream` when the error is built
Two surfaces show less than the chain on purpose, and a test in each pins it. An auth rejection goes to a caller who has not authenticated, so it never shows what a secret manager or key store said. An MCP upstream failure shows only its own layer, because rmcp renders the transport error, URL included, into its own `Display`
`/v1/messages` adds the request id to the envelope when the provider did not set one, which is why its handler takes the body as `Result<JsonObject, Error>` and renders the rejection itself, see [requests](requests.md)
A new surface gets its own renderer next to the others, named after the shape it produces, and its status still comes from `Error::status()`. Picking the status is the [`rust-error`](../../rust-error/references/gateway.md) skill's concern, not the surface's

3
litellm-rust/Cargo.lock generated
View file

@ -4040,6 +4040,7 @@ dependencies = [
"litellm-cache-memory",
"litellm-cache-response",
"litellm-core-utils",
"litellm-error",
"litellm-host",
"litellm-host-native",
"litellm-http",
@ -4132,6 +4133,7 @@ dependencies = [
"litellm-cache-memory",
"litellm-cache-response",
"litellm-core-utils",
"litellm-error",
"litellm-host",
"litellm-host-native",
"litellm-http",
@ -4169,6 +4171,7 @@ dependencies = [
"futures-util",
"litellm-auth",
"litellm-core-utils",
"litellm-error",
"litellm-http",
"litellm-inference",
"litellm-inference-testing",

2
litellm-rust/crates/gateway-inference/src/error.rs
View file

@ -236,7 +236,7 @@ mod tests {
#[rstest]
#[case::upstream_status(upstream(429, ""), StatusCode::TOO_MANY_REQUESTS)]
#[case::rejected_request(
Error::Route(InferenceError::InvalidRequest(litellm_llms::ErrorDetail::Expected {
Error::Route(InferenceError::invalid_request(litellm_llms::ErrorDetail::Expected {
subject: "top_k",
expected: "an integer",
})),

16
litellm-rust/crates/http/src/error.rs
View file

@ -71,6 +71,8 @@ pub(crate) enum Repr {
HandshakeTimeout(#[source] Elapsed),
#[error("upstream WebSocket error")]
WebSocket(#[source] tungstenite::Error),
#[error("the WebSocket connection is closed")]
WebSocketClosed,
#[error("media URL rejected by network policy")]
BlockedUrl,
@ -151,6 +153,10 @@ impl Error {
Repr::UnsignableHeader(litellm_error::Source::new(cause)).into()
}
pub fn websocket_closed() -> Self {
Repr::WebSocketClosed.into()
}
pub fn signature(cause: impl std::error::Error + Send + Sync + 'static) -> Self {
Repr::Signature(litellm_error::Source::new(cause)).into()
}
@ -165,7 +171,8 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_) => true,
Repr::Read { .. }
Repr::WebSocketClosed
| Repr::Read { .. }
| Repr::InvalidPem { .. }
| Repr::Client(_)
| Repr::TlsConfig(_)
@ -220,6 +227,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::Resolve(_)
| Repr::MissingRedirectLocation
| Repr::InvalidRedirect(_)
@ -253,6 +261,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::BlockedUrl
| Repr::Resolve(_)
| Repr::DownloadDisabled
@ -287,6 +296,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::BlockedUrl
| Repr::Resolve(_)
| Repr::DownloadDisabled
@ -323,6 +333,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::BlockedUrl
| Repr::Resolve(_)
| Repr::DownloadDisabled
@ -359,6 +370,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::BlockedUrl
| Repr::Resolve(_)
| Repr::DownloadDisabled
@ -395,6 +407,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::BlockedUrl
| Repr::Resolve(_)
| Repr::DownloadDisabled
@ -430,6 +443,7 @@ impl Error {
| Repr::Handshake(_)
| Repr::HandshakeTimeout(_)
| Repr::WebSocket(_)
| Repr::WebSocketClosed
| Repr::BlockedUrl
| Repr::Resolve(_)
| Repr::DownloadDisabled

1
litellm-rust/crates/inference-chat/Cargo.toml
View file

@ -22,6 +22,7 @@ tracing.workspace = true
[dev-dependencies]
litellm-cache-memory.workspace = true
litellm-error.workspace = true
litellm-host-native.workspace = true
litellm-http = { workspace = true, features = ["test-support"] }
litellm-inference-testing.workspace = true

12
litellm-rust/crates/inference-chat/src/common_utils.rs
View file

@ -50,16 +50,18 @@ pub(super) fn resolve_request(
config: &'static dyn BaseConfig,
) -> Result<ResolvedChatCompletionsRequest, Error> {
let messages = serde_json::from_value::<Vec<ChatMessage>>(call.messages).map_err(|err| {
Error::InvalidRequest(litellm_llms::ErrorDetail::invalid(
Error::invalid_request(litellm_llms::ErrorDetail::invalid(
"chat completions messages",
err,
))
})?;
if messages.is_empty() {
return Err(Error::InvalidRequest(litellm_llms::ErrorDetail::Expected {
subject: "chat completions messages",
expected: "non-empty",
}));
return Err(Error::invalid_request(
litellm_llms::ErrorDetail::Expected {
subject: "chat completions messages",
expected: "non-empty",
},
));
}
if let Some(reason) = config.unsupported_reason(&messages, &call.optional_params) {
return Err(Error::Unsupported(reason.0));

61
litellm-rust/crates/inference-chat/src/handler.rs
View file

@ -120,7 +120,7 @@ pub(super) async fn execute(
.map_err(Error::post_call)?;
let body: Value = serde_json::from_str(&text).map_err(|err| {
Error::InvalidResponse(litellm_llms::ErrorDetail::invalid(
Error::invalid_response(litellm_llms::ErrorDetail::invalid(
"chat completions response JSON",
err,
))
@ -144,14 +144,13 @@ pub(super) async fn execute(
/// implementation must not retry those, so collapse them to one variant that
/// can only mean the provider was already called.
pub(super) fn as_response_error(err: Error) -> Error {
match err {
already @ Error::InvalidResponse(_) => already,
upstream if upstream.upstream_response().is_some() => upstream,
other => Error::InvalidResponse(litellm_llms::ErrorDetail::invalid(
"chat completions response",
other,
)),
if err.is_response() {
return err;
}
Error::invalid_response(litellm_llms::ErrorDetail::invalid(
"chat completions response",
err,
))
}
#[cfg(test)]
@ -307,30 +306,38 @@ mod tests {
.expect_err("the upstream failure fails the call");
assert!(
matches!(&error, Error::Http(http) if http.upstream_response().is_some_and(|(status, _)| status == 500)),
error
.upstream_response()
.is_some_and(|(status, _)| status == 500),
"{error:?}"
);
assert!(interceptors.raw.into_inner().unwrap().is_empty());
}
#[rstest::rstest]
fn response_errors_collapse_to_one_variant_that_can_only_mean_already_sent() {
for original in [
Error::MissingField("usage"),
Error::Unsupported("non-text response content block"),
Error::InvalidRequest(litellm_llms::ErrorDetail::Expected {
subject: "whatever",
expected: "anything else",
}),
Error::Auth(litellm_auth::Error::InvalidHeader),
] {
let label = format!("{original:?}");
assert!(
matches!(as_response_error(original), Error::InvalidResponse(_)),
"{label} must not stay retryable once the provider has answered"
);
}
let upstream = Error::from(litellm_http::Error::upstream(500, "boom".into()));
assert_eq!(as_response_error(upstream.clone()), upstream);
#[case::missing_field(litellm_llms::Error::MissingField("usage").into())]
#[case::unsupported(Error::Unsupported("non-text response content block"))]
#[case::invalid_request(Error::invalid_request(litellm_llms::ErrorDetail::Expected {
subject: "whatever",
expected: "anything else",
}))]
#[case::auth(Error::Auth(litellm_auth::Error::InvalidHeader))]
fn response_errors_collapse_to_a_fault_that_can_only_mean_already_sent(
#[case] original: Error,
) {
let collapsed = as_response_error(original);
assert!(collapsed.is_response(), "{collapsed:?}");
assert!(!collapsed.is_request(), "{collapsed:?}");
assert!(!collapsed.is_unsupported(), "{collapsed:?}");
}
#[rstest::rstest]
fn a_response_fault_is_not_wrapped_twice() {
let original = Error::invalid_response(litellm_llms::ErrorDetail::MissingField {
subject: "response",
field: "usage",
});
let report = litellm_error::Report::new(&as_response_error(original)).to_string();
assert!(!report.contains("chat completions response"), "{report}");
}
}

26
litellm-rust/crates/inference-chat/tests/chat_completions.rs
View file

@ -179,7 +179,7 @@ async fn a_response_it_cannot_normalize_is_reported_as_already_sent(
.await
.expect_err("response cannot be normalized");
assert!(matches!(error, Error::InvalidResponse(_)), "{error:?}");
assert!(error.is_response(), "{error:?}");
}
#[rstest]
@ -215,10 +215,7 @@ async fn a_connection_that_is_never_established_returns_a_connect_error(
.await
.expect_err("nothing is listening");
assert!(
matches!(&error, Error::Http(http) if !http.dispatched()),
"{error:?}"
);
assert!(!error.dispatched(), "{error:?}");
}
#[rstest]
@ -235,7 +232,8 @@ async fn a_transport_error_does_not_echo_the_provider_url(
.await
.expect_err("nothing is listening");
assert!(!error.to_string().contains("tenant-token-1234"), "{error}");
let report = litellm_error::Report::new(&error).to_string();
assert!(!report.contains("tenant-token-1234"), "{report}");
}
#[rstest]
@ -253,10 +251,8 @@ async fn a_timeout_after_sending_returns_a_network_error(request: ChatCompletion
.await
.expect_err("the call times out");
assert!(
matches!(&error, Error::Http(http) if http.dispatched()),
"{error:?}"
);
assert!(error.dispatched(), "{error:?}");
assert!(!error.is_request(), "{error:?}");
}
#[rstest]
@ -352,7 +348,7 @@ async fn a_post_call_hook_failure_never_looks_safe_to_retry(
Ok(wire)
}
async fn after_provider_response(&self, _: RawResponse) -> Result<(), Error> {
Err(Error::MissingField("callback rejected"))
Err(litellm_llms::Error::MissingField("callback rejected").into())
}
}
let upstream = upstream([anthropic_response(ANTHROPIC_MESSAGE)]).await;
@ -368,10 +364,10 @@ async fn a_post_call_hook_failure_never_looks_safe_to_retry(
)
.await
.unwrap_err();
let Error::PostCallHook(source) = error else {
panic!("expected retained callback error")
};
assert_eq!(*source, Error::MissingField("callback rejected"));
assert!(error.dispatched(), "{error:?}");
assert!(!error.is_request(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains("callback rejected"), "{report}");
assert_eq!(received(&upstream).await.len(), 1);
}

49
litellm-rust/crates/inference-chat/tests/request.rs
View file

@ -211,7 +211,7 @@ async fn rejects_empty_messages_before_resolving_credentials() {
})
.await
.expect_err("empty messages are rejected");
assert!(matches!(error, Error::InvalidRequest(_)));
assert!(error.is_request(), "{error:?}");
}
#[rstest]
@ -233,32 +233,27 @@ async fn rejects_an_unknown_provider() {
#[rstest]
#[tokio::test]
async fn rejects_a_model_with_no_resolvable_provider() {
assert!(matches!(
complete(request(
"claude-sonnet-4-5",
None,
json!([{"role": "user", "content": "hi"}]),
json!({}),
))
.await,
Err(Error::InvalidProvider(_))
));
let error = complete(request(
"claude-sonnet-4-5",
None,
json!([{"role": "user", "content": "hi"}]),
json!({}),
))
.await
.unwrap_err();
assert!(error.is_request(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(
report.contains("unable to resolve custom_llm_provider"),
"{report}"
);
}
#[rstest]
#[case::empty(
json!([]),
Some(litellm_llms::ErrorDetail::Expected {
subject: "chat completions messages",
expected: "non-empty",
})
)]
#[case::malformed(json!("not a list"), None)]
#[case::empty(json!([]), "chat completions messages must be non-empty")]
#[case::malformed(json!("not a list"), "invalid chat completions messages")]
#[tokio::test]
async fn rejects_an_empty_or_malformed_message_list(
#[case] messages: Value,
#[case] expected: Option<litellm_llms::ErrorDetail>,
) {
async fn rejects_an_empty_or_malformed_message_list(#[case] messages: Value, #[case] reason: &str) {
let error = complete(request(
"anthropic/claude-sonnet-4-5",
None,
@ -267,11 +262,9 @@ async fn rejects_an_empty_or_malformed_message_list(
))
.await
.unwrap_err();
if let Some(expected) = expected {
assert_eq!(error, Error::InvalidRequest(expected));
} else {
assert!(matches!(error, Error::InvalidRequest(_)));
}
assert!(error.is_request(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains(reason), "{report}");
}
#[rstest]

57
litellm-rust/crates/inference-messages/src/handler.rs
View file

@ -188,7 +188,7 @@ async fn provider_call(
None | Some(Value::Null) => false,
Some(Value::Bool(stream)) => *stream,
Some(value) => {
return Err(Error::InvalidRequest(
return Err(Error::invalid_request(
litellm_llms::ErrorDetail::InvalidValue {
field: "stream",
expected: "a boolean",
@ -261,7 +261,7 @@ async fn call_provider(
provider_name,
));
}
let text = response.text().await.map_err(network)?;
let text = response.text().await.map_err(litellm_http::Error::from)?;
log_response_body(&text);
context.response_received(&text).await?;
decode_response(config, &identity.model, &text)
@ -269,16 +269,12 @@ async fn call_provider(
}
fn serialize_failure(err: serde_json::Error) -> Error {
Error::InvalidRequest(litellm_llms::ErrorDetail::failed(
Error::invalid_request(litellm_llms::ErrorDetail::failed(
"Anthropic messages request serialization",
err,
))
}
fn network(error: reqwest::Error) -> Error {
litellm_http::Error::from(error).into()
}
async fn send(
http: &litellm_http::Client,
authenticated: Authenticated,
@ -302,7 +298,7 @@ async fn provider_error(response: reqwest::Response) -> Error {
log_error_body(status, &text);
litellm_http::Error::upstream(status, text).into()
}
Err(error) => network(error),
Err(error) => litellm_http::Error::from(error).into(),
}
}
@ -312,7 +308,7 @@ fn decode_response(
text: &str,
) -> Result<MessagesResponse, Error> {
let response = serde_json::from_str(text).map_err(|err| {
Error::InvalidResponse(litellm_llms::ErrorDetail::invalid(
Error::invalid_response(litellm_llms::ErrorDetail::invalid(
"messages response JSON",
err,
))
@ -334,7 +330,7 @@ fn streaming_response(
.collect();
let chunks = match decoder {
None => futures_util::stream::try_unfold(response, move |mut response| async move {
let chunk = response.chunk().await.map_err(network)?;
let chunk = response.chunk().await.map_err(litellm_http::Error::from)?;
Ok(chunk.map(|chunk| {
log_chunk(provider, "provider_response", &chunk);
(chunk, response)
@ -536,14 +532,10 @@ mod tests {
shaping,
})
.await;
assert!(
matches!(
result,
Err(Error::Auth(litellm_auth::Error::MissingApiKey { .. }))
),
"{:?}",
result.err()
);
let Err(error) = result else {
panic!("a call without credentials fails");
};
assert!(error.is_missing_credential(), "{error:?}");
}
#[rstest]
@ -766,19 +758,18 @@ mod tests {
},
..shaping
};
assert!(matches!(
wire_body(
json!({
"model": "claude-test",
"messages": [{"role": "user", "content": "hi"}],
"max_tokens": 16,
"metadata": {"user_id": 123}
}),
shaping,
)
.await,
Err(Error::InvalidRequest(_))
));
let error = wire_body(
json!({
"model": "claude-test",
"messages": [{"role": "user", "content": "hi"}],
"max_tokens": 16,
"metadata": {"user_id": 123}
}),
shaping,
)
.await
.unwrap_err();
assert!(error.is_request(), "{error:?}");
}
#[rstest]
@ -795,7 +786,7 @@ mod tests {
shaping,
)
.await,
Err(Error::InvalidRequest(
Err(Error::invalid_request(
litellm_llms::ErrorDetail::InvalidValue {
field: "metadata.user_id",
expected: "a string",
@ -897,7 +888,7 @@ mod tests {
let chunk = chunks.next().await.unwrap();
match expected {
Some(expected) => assert_eq!(chunk.unwrap().as_ref(), expected.as_bytes()),
None => assert!(matches!(chunk, Err(Error::InvalidResponse(_))), "{chunk:?}"),
None => assert!(chunk.unwrap_err().is_response()),
}
assert!(chunks.next().await.is_none());
}

4
litellm-rust/crates/inference-messages/src/types.rs
View file

@ -34,11 +34,11 @@ pub fn messages_body(body: Map<String, Value>) -> Result<MessagesRequest, Error>
/// Python's `GenericLiteLLMParams(**kwargs)`.
pub fn litellm_params(fields: Map<String, Value>) -> Result<LitellmParams, Error> {
serde_json::from_value(Value::Object(fields))
.map_err(|err| Error::InvalidRequest(ErrorDetail::invalid("litellm params", err)))
.map_err(|err| Error::invalid_request(ErrorDetail::invalid("litellm params", err)))
}
pub(super) fn invalid_request(err: serde_json::Error) -> Error {
Error::InvalidRequest(ErrorDetail::invalid("Anthropic messages request", err))
Error::invalid_request(ErrorDetail::invalid("Anthropic messages request", err))
}
pub type MessagesCallResponse =

9
litellm-rust/crates/inference-messages/tests/bedrock_native/main.rs
View file

@ -309,7 +309,14 @@ async fn deployment_workspace_is_required_and_sent_as_a_header(
})
.await;
if !configured {
assert!(matches!(result, Err(Error::Auth(_))));
let Err(error) = result else {
panic!("the call fails");
};
assert!(!error.dispatched(), "{error:?}");
assert!(
error.is_request() || error.is_missing_credential(),
"{error:?}"
);
assert!(received(&upstream).await.is_empty());
return;
}

9
litellm-rust/crates/inference-messages/tests/messages/host.rs
View file

@ -301,7 +301,7 @@ async fn response_mode_follows_the_intercepted_request(
let summaries = traces.summaries("litellm.route");
assert_eq!(summaries.len(), 1);
let Some(expected_stream) = expected_stream else {
assert!(matches!(result, Err(Error::InvalidRequest(_))));
assert!(result.unwrap_err().is_request());
assert!(received(&upstream).await.is_empty());
assert_eq!(summaries[0]["outcome"], "failure");
return;
@ -322,14 +322,17 @@ async fn a_before_send_failure_never_sends(call: MessagesCall) {
let upstream = upstream([message_response()]).await;
let host = RecordingHost::new(
authenticated(call, upstream.uri()),
Box::new(|_| Err(Error::MissingField("vetoed by the host"))),
Box::new(|_| Err(litellm_llms::Error::MissingField("vetoed by the host").into())),
);
let error = run_through(&host)
.await
.expect_err("the host failure fails the call");
assert_eq!(error, Error::MissingField("vetoed by the host"));
assert_eq!(
error,
Error::from(litellm_llms::Error::MissingField("vetoed by the host"))
);
assert!(received(&upstream).await.is_empty());
assert!(host.raw_responses().is_empty());
}

2
litellm-rust/crates/inference-messages/tests/messages/main.rs
View file

@ -144,7 +144,7 @@ impl LocalMessagesHost {
.lock()
.unwrap_or_else(|error| error.into_inner())
.take()
.ok_or(Error::MissingField("messages request"))
.ok_or(litellm_llms::Error::MissingField("messages request").into())
}
pub fn runtime(&self) -> litellm_host_native::in_process::Host<'_, (), Self, ()> {
litellm_host_native::in_process::Host {

19
litellm-rust/crates/inference-messages/tests/messages/request.rs
View file

@ -108,13 +108,7 @@ async fn a_call_without_credentials_fails_before_sending(
.await
.expect_err("a call without credentials fails");
assert!(
matches!(
error,
Error::Auth(litellm_auth::Error::MissingApiKey { .. })
),
"{error:?}"
);
assert!(error.is_missing_credential(), "{error:?}");
assert!(received(&upstream).await.is_empty());
}
@ -185,7 +179,9 @@ async fn unsupported_providers_are_rejected_before_sending(
.await
.expect_err("unsupported provider errors");
assert_eq!(error, Error::InvalidProvider(reported.into()));
assert!(error.is_request(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains(reported), "{report}");
}
#[rstest]
@ -513,10 +509,9 @@ async fn unsupported_params_are_dropped_under_drop_params_and_rejected_without_i
let error = run(shaped(false))
.await
.expect_err("an unsupported param is rejected without drop_params");
assert!(
matches!(&error, Error::InvalidRequest(message) if message.to_string().contains(rejected_as)),
"{error:?}"
);
assert!(error.is_request(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains(rejected_as), "{report}");
assert!(received(&upstream).await.is_empty());
run_message(shaped(true)).await;

8
litellm-rust/crates/inference-messages/tests/messages/response.rs
View file

@ -235,7 +235,7 @@ async fn an_unreadable_success_body_is_an_invalid_response(
.await
.expect_err("an unreadable body fails");
assert!(matches!(error, Error::InvalidResponse(_)), "{error:?}");
assert!(error.is_response(), "{error:?}");
}
#[rstest]
@ -252,7 +252,9 @@ async fn a_provider_slower_than_the_timeout_fails_the_call(call: MessagesCall) {
.await
.expect_err("the call times out");
assert!(matches!(error, Error::Http(_)), "{error:?}");
assert!(error.dispatched(), "{error:?}");
assert!(!error.is_request(), "{error:?}");
assert_eq!(error.upstream_response(), None);
}
#[rstest]
@ -312,7 +314,7 @@ async fn the_facade_sends_through_the_injected_http_pool_configuration(call: Mes
fn a_body_that_does_not_parse_is_an_invalid_request(#[case] raw: Value) {
let error = messages_body(object(raw)).expect_err("the body is rejected");
assert!(matches!(error, Error::InvalidRequest(_)), "{error:?}");
assert!(error.is_request(), "{error:?}");
assert!(
litellm_error::Report::new(&error)
.to_string()

2
litellm-rust/crates/inference-messages/tests/messages/secrets.rs
View file

@ -109,7 +109,7 @@ async fn a_secret_manager_failure_fails_the_call_before_sending(call: MessagesCa
.expect_err("a secret manager failure fails the call");
assert!(
matches!(&error, Error::Secret(source) if matches!(source.source_error(), litellm_secrets::Error::ManagedSecretMissing)),
matches!(&error, Error::Secret(_)) && !error.is_request(),
"{error:?}"
);
assert!(received(&upstream).await.is_empty());

8
litellm-rust/crates/inference-messages/tests/messages/stream.rs
View file

@ -326,7 +326,9 @@ async fn the_timeout_covers_a_stalled_stream_body(call: MessagesCall) {
.expect("the stalled stream gives up within the timeout")
.expect_err("a stalled body fails the call");
assert!(matches!(error, Error::Http(_)), "{error:?}");
assert!(error.dispatched(), "{error:?}");
assert!(!error.is_request(), "{error:?}");
assert_eq!(error.upstream_response(), None);
let seen = host.seen.into_inner().unwrap();
assert!(
matches!(seen.as_slice(), [Seen::Open(_), Seen::Deliver(chunk)] if chunk.as_ref() == b"event: message_start\ndata: {}\n\n"),
@ -458,7 +460,9 @@ async fn the_sdk_yields_a_body_error_once_after_delivered_chunks(call: MessagesC
.expect("the stalled body times out")
.unwrap()
.unwrap_err();
assert!(matches!(error, Error::Http(_)), "{error:?}");
assert!(error.dispatched(), "{error:?}");
assert!(!error.is_request(), "{error:?}");
assert_eq!(error.upstream_response(), None);
assert!(chunks.next().await.is_none());
tokio::time::timeout(Duration::from_secs(5), connection)
.await

10
litellm-rust/crates/inference-messages/tests/messages/tencent.rs
View file

@ -79,13 +79,9 @@ async fn missing_credentials_fail_before_sending(tencent_call: MessagesCall) {
})
.await
.unwrap_err();
assert!(matches!(
error,
Error::Auth(litellm_auth::Error::MissingApiKey {
environment_variable: "TENCENT_API_KEY",
..
})
));
assert!(error.is_missing_credential(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains("TENCENT_API_KEY"), "{report}");
assert!(received(&upstream).await.is_empty());
}

15
litellm-rust/crates/inference-messages/tests/openrouter/main.rs
View file

@ -1,7 +1,7 @@
use std::{sync::Arc, time::Duration};
use futures_util::TryStreamExt;
use litellm_inference_messages::{Error, MessagesCall, MessagesCallResponse, MessagesShaping};
use litellm_inference_messages::{MessagesCall, MessagesCallResponse, MessagesShaping};
use litellm_inference_testing::{RecordingSecrets, no_secrets};
use rstest::{fixture, rstest};
use serde_json::json;
@ -92,13 +92,12 @@ async fn an_anthropic_key_alone_is_not_an_openrouter_credential(call: MessagesCa
None,
)
.await;
assert!(matches!(
result,
Err(Error::Auth(litellm_auth::Error::MissingApiKey {
provider: "OpenRouter",
..
}))
));
let Err(error) = result else {
panic!("the call fails");
};
assert!(error.is_missing_credential(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains("OpenRouter"), "{report}");
assert!(received(&upstream).await.is_empty());
}

32
litellm-rust/crates/inference-ocr/src/provider_config.rs
View file

@ -1,6 +1,8 @@
use litellm_auth::{InputSource, SecretValue, Sourced};
use litellm_core_utils::get_llm_provider_logic::{LlmProviders, get_custom_llm_provider};
use litellm_inference::provider::{ResolvedProvider, resolve_llm_provider};
use litellm_core_utils::get_llm_provider_logic::{
CustomLlmProvider, LlmProviders, get_custom_llm_provider,
};
use litellm_inference::provider::ResolvedProvider;
use litellm_llms::{
aws_textract::ocr::{
analyze_transformation::TextractAnalyzeDocumentConfig, common_utils::TextractOperation,
@ -241,18 +243,20 @@ pub(crate) fn resolve_provider_config<'a>(
model: &'a str,
custom_llm_provider: Option<&'a str>,
) -> Result<(ResolvedProvider<'a>, OcrConfigKind), Error> {
let provider = match resolve_llm_provider(model, custom_llm_provider, "ocr") {
Ok(provider) => provider,
Err(_) if get_custom_llm_provider(model, custom_llm_provider).is_none() => {
ResolvedProvider {
model,
provider: LlmProviders::Mistral,
}
}
Err(litellm_inference::Error::InvalidProvider(provider)) => {
return Err(Error::InvalidProvider(provider));
}
Err(error) => return Err(Error::InvalidProvider(error.to_string())),
let provider = match get_custom_llm_provider(model, custom_llm_provider) {
None => ResolvedProvider {
model,
provider: LlmProviders::Mistral,
},
Some(CustomLlmProvider {
model,
custom_llm_provider,
}) => ResolvedProvider {
model,
provider: custom_llm_provider
.parse()
.map_err(|_| Error::InvalidProvider(custom_llm_provider.to_owned()))?,
},
};
let config = match provider.provider {
LlmProviders::AwsTextract => match TextractOperation::from_model(provider.model)? {

9
litellm-rust/crates/inference-ocr/tests/ocr/azure_ai.rs
View file

@ -222,10 +222,8 @@ async fn credential_precedence(
false,
json!({}),
numbered_token,
|error: &Error| matches!(error, Error::Auth(litellm_auth::Error::MissingApiBase {
provider: "Azure AI",
guidance: "Set AZURE_AI_API_BASE environment variable or pass api_base parameter",
})),
|error: &Error| litellm_error::Report::new(error).to_string()
== "Missing Azure AI API Base - Set AZURE_AI_API_BASE environment variable or pass api_base parameter",
0
)]
#[case::unsupported_oidc_reference(
@ -239,7 +237,8 @@ async fn credential_precedence(
true,
json!({"azure_ad_token": "static-token"}),
|_| String::new(),
|error: &Error| error.is_missing_credential() && error.to_string().contains("Azure AI credentials"),
|error: &Error| error.is_missing_credential()
&& litellm_error::Report::new(error).to_string().contains("Azure AI credentials"),
1
)]
#[tokio::test]

6
litellm-rust/crates/inference-ocr/tests/ocr/documents.rs
View file

@ -216,10 +216,8 @@ async fn configured_client_preserves_document_url_policy(#[case] allowed: bool)
.await;
if !allowed {
assert!(
matches!(&result, Err(Error::Http(error)) if error.is_request()),
"{result:?}"
);
let error = result.unwrap_err();
assert!(error.is_request(), "{error:?}");
assert!(received(&documents).await.is_empty());
assert!(received(&upstream).await.is_empty());
return;

1
litellm-rust/crates/inference-responses/Cargo.toml
View file

@ -28,6 +28,7 @@ tracing.workspace = true
litellm-http = { workspace = true, features = ["test-support"] }
litellm-cache.workspace = true
litellm-cache-memory.workspace = true
litellm-error.workspace = true
litellm-host-native.workspace = true
litellm-inference-testing.workspace = true
litellm-tracing.workspace = true

22
litellm-rust/crates/inference-responses/src/handler.rs
View file

@ -73,10 +73,12 @@ pub(super) async fn execute(
None => false,
Some(serde_json::Value::Bool(value)) => *value,
Some(_) => {
return Err(Error::InvalidRequest(litellm_llms::ErrorDetail::Expected {
subject: "stream",
expected: "a boolean",
}));
return Err(Error::invalid_request(
litellm_llms::ErrorDetail::Expected {
subject: "stream",
expected: "a boolean",
},
));
}
};
let outbound = litellm_inference::outbound::outbound_request(
@ -91,7 +93,7 @@ pub(super) async fn execute(
let response = litellm_inference::outbound::send(outbound, &route.http).await?;
let status = response.status().as_u16();
if !response.status().is_success() {
let body = response.text().await.map_err(network)?;
let body = response.text().await.map_err(litellm_http::Error::from)?;
return Err(litellm_http::Error::upstream(status, body).into());
}
if stream {
@ -104,14 +106,14 @@ pub(super) async fn execute(
.collect();
let chunks = response
.bytes_stream()
.map(|chunk| chunk.map_err(network))
.map(|chunk| chunk.map_err(|error| litellm_http::Error::from(error).into()))
.boxed();
return Ok(ResponsesOutput::Stream {
head: ResponsesStreamHead { headers },
chunks,
});
}
let body = response.text().await.map_err(network)?;
let body = response.text().await.map_err(litellm_http::Error::from)?;
let raw = RawResponse { body: body.clone() };
if let Some(observers) = observers {
observers.emit(litellm_host::lifecycle::CallEvent::Execution(
@ -123,7 +125,7 @@ pub(super) async fn execute(
.await
.map_err(Error::post_call)?;
let value = serde_json::from_str(&body).map_err(|error| {
Error::InvalidResponse(litellm_llms::ErrorDetail::invalid(
Error::invalid_response(litellm_llms::ErrorDetail::invalid(
"Responses API response JSON",
error,
))
@ -136,7 +138,3 @@ pub(super) async fn execute(
)
.await
}
fn network(error: reqwest::Error) -> Error {
litellm_http::Error::from(error).into()
}

10
litellm-rust/crates/inference-responses/src/websocket.rs
View file

@ -7,7 +7,7 @@ use litellm_http::{
};
use tokio::sync::Mutex;
use tokio_tungstenite::tungstenite::{
self, Message,
Message,
http::{HeaderMap, HeaderName, HeaderValue},
};
@ -39,13 +39,13 @@ impl ResponsesWebSocketConnection {
.map(|(name, value)| {
Ok((
name.parse::<HeaderName>().map_err(|error| {
Error::InvalidRequest(ErrorDetail::invalid(
Error::invalid_request(ErrorDetail::invalid(
"WebSocket header name",
error,
))
})?,
HeaderValue::from_str(value).map_err(|error| {
Error::InvalidRequest(ErrorDetail::invalid(
Error::invalid_request(ErrorDetail::invalid(
"WebSocket header value",
error,
))
@ -71,7 +71,7 @@ impl ResponsesWebSocketConnection {
litellm_inference::diagnostic::operation("litellm.websocket.send_text", async {
let mut socket = self.socket.lock().await;
let Some(socket) = socket.as_mut() else {
return Err(litellm_http::Error::from(tungstenite::Error::AlreadyClosed).into());
return Err(litellm_http::Error::websocket_closed().into());
};
socket
.send(Message::Text(text))
@ -98,7 +98,7 @@ impl ResponsesWebSocketConnection {
Some(Ok(Message::Binary(bytes))) => String::from_utf8(bytes.to_vec())
.map(Some)
.map_err(|error| {
Error::InvalidResponse(ErrorDetail::invalid("WebSocket text frame", error))
Error::invalid_response(ErrorDetail::invalid("WebSocket text frame", error))
}),
Some(Ok(Message::Close(_))) | None => Ok(None),
Some(Ok(_)) => Ok(None),

18
litellm-rust/crates/inference-responses/tests/openrouter.rs
View file

@ -1,10 +1,7 @@
use std::sync::Arc;
use futures_util::TryStreamExt;
use litellm_inference_responses::{
Error,
types::{ResponsesCall, ResponsesOutput},
};
use litellm_inference_responses::types::{ResponsesCall, ResponsesOutput};
use litellm_inference_testing::{RecordingSecrets, no_secrets};
use rstest::{fixture, rstest};
use serde_json::json;
@ -108,13 +105,12 @@ async fn a_missing_key_fails_before_transport(call: ResponsesCall) {
None,
)
.await;
assert!(matches!(
result,
Err(Error::Auth(litellm_auth::Error::MissingApiKey {
provider: "OpenRouter",
..
}))
));
let Err(error) = result else {
panic!("the call fails");
};
assert!(error.is_missing_credential(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains("OpenRouter"), "{report}");
assert!(received(&upstream).await.is_empty());
}

11
litellm-rust/crates/inference-responses/tests/responses.rs
View file

@ -576,12 +576,11 @@ async fn websocket_operations_trace_outcomes_without_capturing_frames_or_credent
Some("private-frame-sentinel")
);
connection.close().await.unwrap();
assert!(
connection
.send_text("private-frame-sentinel".into())
.await
.is_err()
);
let error = connection
.send_text("private-frame-sentinel".into())
.await
.expect_err("a closed connection sends nothing");
assert!(!error.dispatched(), "{error:?}");
})
.await;
server.await.unwrap();

1
litellm-rust/crates/inference-transcription/Cargo.toml
View file

@ -17,6 +17,7 @@ serde_json.workspace = true
tracing.workspace = true
[dev-dependencies]
litellm-error.workspace = true
litellm-inference-testing.workspace = true
litellm-tracing.workspace = true
rstest.workspace = true

2
litellm-rust/crates/inference-transcription/src/handler.rs
View file

@ -62,7 +62,7 @@ pub(super) async fn execute(
return Err(litellm_http::Error::upstream(status.as_u16(), text).into());
}
let response_json = serde_json::from_str(&text).map_err(|error| {
Error::InvalidResponse(litellm_llms::ErrorDetail::invalid(
Error::invalid_response(litellm_llms::ErrorDetail::invalid(
"audio response JSON",
error,
))

14
litellm-rust/crates/inference-transcription/tests/audio_transcription.rs
View file

@ -152,13 +152,7 @@ async fn invalid_audio_is_rejected_before_sending(
.await
.expect_err("invalid audio is rejected");
assert!(
matches!(
error,
Error::InvalidRequest(_) | Error::MissingField(_) | Error::InvalidType { .. }
),
"{error:?}"
);
assert!(error.is_request(), "{error:?}");
assert!(received(&upstream).await.is_empty());
}
@ -185,7 +179,9 @@ async fn unsupported_providers_are_rejected_before_sending(
.await
.expect_err("unsupported provider errors");
assert_eq!(error, Error::InvalidProvider(reported.into()));
assert!(error.is_request(), "{error:?}");
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains(reported), "{report}");
}
#[rstest]
@ -262,7 +258,7 @@ async fn an_unreadable_success_body_is_an_invalid_response(
.await
.expect_err("an unreadable body fails");
assert!(matches!(error, Error::InvalidResponse(_)), "{error:?}");
assert!(error.is_response(), "{error:?}");
}
#[rstest]

6
litellm-rust/crates/inference/AGENTS.md
View file

@ -53,10 +53,10 @@
## Error placement
- The workspace `Error definitions` rules shape each crate's error. This section decides which crate and module a failure belongs to
- The [rust-error skill](../../.agents/skills/rust-error/SKILL.md) shapes each crate's error. This section decides which crate and module a failure belongs to
- A failure is declared once, by the lowest crate that raises it
- every crate above nests it unchanged (`#[error(transparent)] Auth(#[from] litellm_auth::Error)`) or maps it once at its boundary, as `src/error.rs` does for `litellm_llms::Error`
- `Error` collects route failures for every format crate and never re-declares a variant a lower crate raises
- every crate above nests it in one variant and delegates its predicates, as `Error::Llm`, `Error::Auth` and `Error::Http` do, or translates it once in the shared `From` when its meaning changes, as `From<litellm_llms::Error>` does for `Unsupported` and a header the signer computes
- `Error` collects route failures for every format crate and never re-declares a variant a lower crate raises. Routes build transformation failures with `Error::invalid_request` and `Error::invalid_response`, and callers ask `is_request()`, `is_response()`, `is_missing_credential()`, `dispatched()` and `upstream_response()` instead of matching nested variants
- Scope follows the concept, not the first caller
- an error type under `litellm-llms`'s `<provider>/` is private to that provider: no other provider and nothing in `base_llm` may import it
- a failure two providers or two routes can hit (wire framing, stream event decoding, a malformed provider response) belongs to the crate that owns the concept: `litellm-framer` for framing, `litellm_llms::Error` for the transformation layer

273
litellm-rust/crates/inference/src/error.rs
View file

@ -1,32 +1,19 @@
//! One error for every route in this crate. OCR still carries its own, richer enum.
//!
//! A variant is declared by the layer that produces it and nested here as is:
//! credentials by `litellm_auth` (AWS folds into it at that crate's boundary), the wire by
//! `litellm_http`, secrets by `litellm_secrets`. The transformation layer's [`LlmError`]
//! maps onto the same-named variants once, here, so no route re-declares them.
use std::sync::Arc;
use litellm_llms::Error as LlmError;
use litellm_host::machine::MachineFault;
use litellm_llms::{Error as LlmError, ErrorDetail};
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("expected {expected}, got {actual}")]
InvalidType {
expected: &'static str,
actual: &'static str,
},
#[error("missing required field: {0}")]
MissingField(&'static str),
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request")]
InvalidRequest(#[source] litellm_llms::ErrorDetail),
#[error("invalid response")]
InvalidResponse(#[source] litellm_llms::ErrorDetail),
#[error("invalid provider: unable to resolve custom_llm_provider for {route} request")]
UnresolvedProvider { route: &'static str },
#[error("unsupported by the rust path: {0}")]
Unsupported(&'static str),
#[error(transparent)]
Llm(LlmError),
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Headers(#[from] litellm_http::request::HeaderError),
@ -34,17 +21,27 @@ pub enum Error {
Http(litellm_http::Error),
#[error(transparent)]
Secret(#[from] SecretError),
#[error("post-call hook failed: {0}")]
PostCallHook(Arc<Error>),
#[error("the host driver failed")]
Host(#[source] MachineFault),
#[error("post-call hook failed")]
PostCallHook(#[source] Arc<Error>),
}
impl From<litellm_host::machine::MachineFault> for Error {
fn from(fault: litellm_host::machine::MachineFault) -> Self {
Self::InvalidRequest(litellm_llms::ErrorDetail::failed("host driver", fault))
impl From<MachineFault> for Error {
fn from(fault: MachineFault) -> Self {
Self::Host(fault)
}
}
impl Error {
pub fn invalid_request(detail: ErrorDetail) -> Self {
Self::Llm(LlmError::InvalidRequest(detail))
}
pub fn invalid_response(detail: ErrorDetail) -> Self {
Self::Llm(LlmError::InvalidResponse(detail))
}
pub fn post_call(error: Self) -> Self {
Self::PostCallHook(Arc::new(error))
}
@ -53,32 +50,88 @@ impl Error {
/// the provider's answer.
pub fn is_request(&self) -> bool {
match self {
Self::InvalidType { .. }
| Self::MissingField(_)
| Self::InvalidProvider(_)
| Self::InvalidRequest(_)
Self::InvalidProvider(_)
| Self::UnresolvedProvider { .. }
| Self::Unsupported(_)
| Self::Headers(_) => true,
Self::Llm(error) => error.is_request(),
Self::Auth(error) => !error.is_missing_credential(),
Self::Http(error) => error.is_request(),
Self::InvalidResponse(_) | Self::Secret(_) | Self::PostCallHook(_) => false,
Self::Secret(_) | Self::Host(_) | Self::PostCallHook(_) => false,
}
}
pub fn is_response(&self) -> bool {
match self {
Self::Llm(error) => error.is_response(),
Self::Http(error) => error.is_response(),
Self::InvalidProvider(_)
| Self::UnresolvedProvider { .. }
| Self::Unsupported(_)
| Self::Auth(_)
| Self::Headers(_)
| Self::Secret(_)
| Self::Host(_)
| Self::PostCallHook(_) => false,
}
}
pub fn is_missing_credential(&self) -> bool {
matches!(self, Self::Auth(error) if error.is_missing_credential())
match self {
Self::Llm(error) => error.is_missing_credential(),
Self::Auth(error) => error.is_missing_credential(),
Self::InvalidProvider(_)
| Self::UnresolvedProvider { .. }
| Self::Unsupported(_)
| Self::Headers(_)
| Self::Http(_)
| Self::Secret(_)
| Self::Host(_)
| Self::PostCallHook(_) => false,
}
}
pub fn is_unsupported(&self) -> bool {
matches!(self, Self::Unsupported(_))
match self {
Self::Unsupported(_) => true,
Self::InvalidProvider(_)
| Self::UnresolvedProvider { .. }
| Self::Llm(_)
| Self::Auth(_)
| Self::Headers(_)
| Self::Http(_)
| Self::Secret(_)
| Self::Host(_)
| Self::PostCallHook(_) => false,
}
}
pub fn dispatched(&self) -> bool {
match self {
Self::Llm(error) => error.dispatched(),
Self::Http(error) => error.dispatched(),
Self::Host(_) | Self::PostCallHook(_) => true,
Self::InvalidProvider(_)
| Self::UnresolvedProvider { .. }
| Self::Unsupported(_)
| Self::Auth(_)
| Self::Headers(_)
| Self::Secret(_) => false,
}
}
}
impl Error {
pub fn upstream_response(&self) -> Option<(u16, &str)> {
match self {
Self::Llm(error) => error.upstream_response(),
Self::Http(error) => error.upstream_response(),
_ => None,
Self::InvalidProvider(_)
| Self::UnresolvedProvider { .. }
| Self::Unsupported(_)
| Self::Auth(_)
| Self::Headers(_)
| Self::Secret(_)
| Self::Host(_)
| Self::PostCallHook(_) => None,
}
}
}
@ -97,13 +150,13 @@ impl From<litellm_http::Error> for Error {
impl From<LlmError> for Error {
fn from(error: LlmError) -> Self {
match error {
LlmError::InvalidType { expected, actual } => Self::InvalidType { expected, actual },
LlmError::MissingField(field) => Self::MissingField(field),
LlmError::InvalidRequest(message) => Self::InvalidRequest(message),
LlmError::InvalidResponse(message) => Self::InvalidResponse(message),
LlmError::Unsupported(reason) => Self::Unsupported(reason),
LlmError::Auth(error) => Self::Auth(error),
LlmError::Http(error) => Self::from(error),
LlmError::Auth(error) => Self::Auth(error),
error @ (LlmError::InvalidType { .. }
| LlmError::MissingField(_)
| LlmError::InvalidRequest(_)
| LlmError::InvalidResponse(_)) => Self::Llm(error),
}
}
}
@ -135,45 +188,120 @@ impl Eq for SecretError {}
#[cfg(test)]
mod tests {
use super::Error;
use litellm_host::machine::MachineFault;
use litellm_llms::{Error as LlmError, ErrorDetail};
use rstest::rstest;
#[test]
fn a_missing_api_key_is_the_environment_not_the_request() {
assert!(
!Error::Auth(litellm_auth::Error::MissingApiKey {
provider: "Anthropic",
environment_variable: "ANTHROPIC_API_KEY",
})
.is_request()
);
assert!(Error::Auth(litellm_auth::Error::InvalidHeader).is_request());
assert!(
Error::InvalidRequest(ErrorDetail::Expected {
subject: "top_k",
expected: "an integer",
})
.is_request()
);
assert!(
!Error::InvalidResponse(ErrorDetail::MissingField {
subject: "response",
field: "usage",
})
.is_request()
);
#[rstest]
#[case::missing_api_key(
litellm_auth::Error::MissingApiKey {
provider: "Anthropic",
environment_variable: "ANTHROPIC_API_KEY",
}
.into(),
false
)]
#[case::invalid_auth_header(litellm_auth::Error::InvalidHeader.into(), true)]
#[case::transformation_missing_api_key(
LlmError::Auth(litellm_auth::Error::MissingApiKey {
provider: "Anthropic",
environment_variable: "ANTHROPIC_API_KEY",
})
.into(),
false
)]
#[case::invalid_request(
LlmError::InvalidRequest(ErrorDetail::Expected {
subject: "top_k",
expected: "an integer",
})
.into(),
true
)]
#[case::invalid_response(
LlmError::InvalidResponse(ErrorDetail::MissingField {
subject: "response",
field: "usage",
})
.into(),
false
)]
#[case::host_driver(MachineFault::Abandoned.into(), false)]
#[case::post_call_hook_over_a_request_fault(
Error::post_call(LlmError::MissingField("usage").into()),
false
)]
fn only_the_callers_input_is_a_request_fault(#[case] error: Error, #[case] is_request: bool) {
assert_eq!(error.is_request(), is_request);
}
#[test]
fn a_header_the_signer_computes_is_left_to_the_host_on_every_route() {
let error = Error::from(litellm_http::Error::computed_header("authorization"));
assert!(matches!(error, Error::Unsupported(_)), "{error:?}");
#[rstest]
#[case::own_auth(litellm_auth::Error::MissingApiKey {
provider: "Anthropic",
environment_variable: "ANTHROPIC_API_KEY",
}
.into())]
#[case::transformation_auth(
LlmError::Auth(litellm_auth::Error::MissingApiKey {
provider: "Anthropic",
environment_variable: "ANTHROPIC_API_KEY",
})
.into()
)]
fn a_missing_api_key_is_a_missing_credential_wherever_it_is_raised(#[case] error: Error) {
assert!(error.is_missing_credential());
}
#[rstest]
#[case::direct(litellm_http::Error::computed_header("authorization").into())]
#[case::through_a_transformation(
LlmError::Http(litellm_http::Error::computed_header("authorization")).into()
)]
fn a_header_the_signer_computes_is_left_to_the_host_on_every_route(#[case] error: Error) {
assert!(error.is_unsupported(), "{error:?}");
assert!(error.is_request());
}
#[test]
fn a_transformation_unsupported_stays_a_decline() {
assert!(Error::from(LlmError::Unsupported("test capability")).is_unsupported());
}
#[test]
fn an_upstream_failure_through_a_transformation_keeps_the_provider_answer() {
let error = Error::from(LlmError::Http(litellm_http::Error::upstream(
429,
"slow down".into(),
)));
assert_eq!(error.upstream_response(), Some((429, "slow down")));
assert!(error.dispatched());
}
#[test]
fn a_post_call_hook_failure_is_not_the_providers_answer() {
let error =
Error::post_call(litellm_http::Error::upstream(500, "hook backend down".into()).into());
assert_eq!(error.upstream_response(), None);
assert!(!error.is_request());
let report = litellm_error::Report::new(&error).to_string();
assert!(report.starts_with("post-call hook failed: "), "{report}");
assert!(report.contains("hook backend down"), "{report}");
}
#[test]
fn a_host_driver_fault_keeps_the_fault_as_its_source() {
let error = Error::from(MachineFault::Abandoned);
assert_eq!(
litellm_error::find::<MachineFault>(&error),
Some(&MachineFault::Abandoned)
);
assert!(error.dispatched());
}
#[rstest]
#[case::request(true)]
#[case::response(false)]
fn contextual_errors_render_their_cause_and_route_classification(#[case] request: bool) {
fn contextual_errors_keep_their_cause_and_route_classification(#[case] request: bool) {
let source = serde_json::from_str::<serde_json::Value>("{").unwrap_err();
let source_message = source.to_string();
let detail = ErrorDetail::invalid("test payload", source);
@ -183,11 +311,10 @@ mod tests {
LlmError::InvalidResponse(detail)
});
assert_eq!(error.is_request(), request);
let category = if request { "request" } else { "response" };
assert_eq!(
litellm_error::Report::new(&error).to_string(),
format!("invalid {category}: invalid test payload: {source_message}")
);
assert_eq!(error.is_response(), !request);
let report = litellm_error::Report::new(&error).to_string();
assert!(report.contains("test payload"), "{report}");
assert_eq!(report.matches(&source_message).count(), 1, "{report}");
assert!(litellm_error::find::<serde_json::Error>(&error).is_some());
}
}

6
litellm-rust/crates/inference/src/provider.rs
View file

@ -25,11 +25,7 @@ pub fn resolve_llm_provider<'a>(
custom_llm_provider: provider,
})
})
.ok_or_else(|| {
Error::InvalidProvider(format!(
"unable to resolve custom_llm_provider for {route} request"
))
})?;
.ok_or(Error::UnresolvedProvider { route })?;
let provider = custom_llm_provider
.parse()
.map_err(|_| Error::InvalidProvider(custom_llm_provider.to_string()))?;

64
litellm-rust/crates/llms/src/error.rs
View file

@ -21,6 +21,70 @@ pub enum Error {
Http(#[from] litellm_http::Error),
}
impl Error {
pub fn is_request(&self) -> bool {
match self {
Self::InvalidType { .. }
| Self::MissingField(_)
| Self::InvalidRequest(_)
| Self::Unsupported(_) => true,
Self::Auth(error) => !error.is_missing_credential(),
Self::Http(error) => error.is_request(),
Self::InvalidResponse(_) => false,
}
}
pub fn is_response(&self) -> bool {
match self {
Self::InvalidResponse(_) => true,
Self::Http(error) => error.is_response(),
Self::InvalidType { .. }
| Self::MissingField(_)
| Self::InvalidRequest(_)
| Self::Unsupported(_)
| Self::Auth(_) => false,
}
}
pub fn is_missing_credential(&self) -> bool {
match self {
Self::Auth(error) => error.is_missing_credential(),
Self::InvalidType { .. }
| Self::MissingField(_)
| Self::InvalidRequest(_)
| Self::InvalidResponse(_)
| Self::Unsupported(_)
| Self::Http(_) => false,
}
}
/// A transformation runs on both sides of the send, so only the transport and credential
/// resolution, which always runs before it, can tell whether the request reached the provider.
pub fn dispatched(&self) -> bool {
match self {
Self::Http(error) => error.dispatched(),
Self::Auth(_) => false,
Self::InvalidType { .. }
| Self::MissingField(_)
| Self::InvalidRequest(_)
| Self::InvalidResponse(_)
| Self::Unsupported(_) => true,
}
}
pub fn upstream_response(&self) -> Option<(u16, &str)> {
match self {
Self::Http(error) => error.upstream_response(),
Self::InvalidType { .. }
| Self::MissingField(_)
| Self::InvalidRequest(_)
| Self::InvalidResponse(_)
| Self::Unsupported(_)
| Self::Auth(_) => None,
}
}
}
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum ErrorDetail {
#[error("invalid {subject}")]

4
litellm-rust/crates/python-bridge/src/errors.rs
View file

@ -45,7 +45,7 @@ mod tests {
#[case::unsupported(Error::Unsupported("test capability"), true)]
#[case::invalid_provider(Error::InvalidProvider("unknown".into()), true)]
#[case::invalid_request(
Error::InvalidRequest(litellm_llms::ErrorDetail::Expected {
Error::invalid_request(litellm_llms::ErrorDetail::Expected {
subject: "messages",
expected: "non-empty",
}),
@ -56,7 +56,7 @@ mod tests {
false
)]
#[case::invalid_response(
Error::InvalidResponse(litellm_llms::ErrorDetail::MissingField {
Error::invalid_response(litellm_llms::ErrorDetail::MissingField {
subject: "response",
field: "usage",
}),

2
litellm-rust/crates/python-bridge/src/routes/chat_completions/host.rs
View file

@ -79,7 +79,7 @@ impl PythonBinding for ChatCompletionsPythonHost {
self.0.error(py, error)
}
fn host_error(_: &PyErr) -> Error {
Error::InvalidRequest(litellm_llms::ErrorDetail::HostCall)
Error::invalid_request(litellm_llms::ErrorDetail::HostCall)
}
}

12
litellm-rust/crates/python-bridge/src/routes/messages/host.rs
View file

@ -76,12 +76,12 @@ fn native_error(py: Python<'_>, error: Error) -> PyResult<PyErr> {
return Ok(error);
}
match error {
Error::InvalidRequest(message) => {
Error::Llm(litellm_llms::Error::InvalidRequest(message)) => {
let error = PyValueError::new_err(litellm_error::Report::new(&message).to_string());
error.value(py).setattr(REQUEST_ERROR_MARKER, true)?;
Ok(error)
}
Error::MissingField(field) => {
Error::Llm(litellm_llms::Error::MissingField(field)) => {
let error = PyValueError::new_err(format!("missing required field: {field}"));
error.value(py).setattr(REQUEST_ERROR_MARKER, true)?;
Ok(error)
@ -284,7 +284,7 @@ impl PythonBinding for MessagesPythonHost {
}
fn host_error(_: &PyErr) -> Error {
Error::InvalidRequest(litellm_llms::ErrorDetail::HostCall)
Error::invalid_request(litellm_llms::ErrorDetail::HostCall)
}
}
@ -458,7 +458,7 @@ mod tests {
projected,
litellm_params(serde_json::from_value(fields).unwrap()).unwrap()
),
(Err(error), Err(())) => assert!(matches!(error, Error::InvalidRequest(_))),
(Err(error), Err(())) => assert!(error.is_request(), "{error:?}"),
(projected, expected) => panic!("got {projected:?}, expected {expected:?}"),
}
});
@ -466,7 +466,7 @@ mod tests {
#[rstest]
#[case::rejected_request(
Error::InvalidRequest(litellm_llms::ErrorDetail::UnsupportedParameter {
Error::invalid_request(litellm_llms::ErrorDetail::UnsupportedParameter {
model: "claude".into(),
param: "top_k".into(),
value: "5".into(),
@ -474,7 +474,7 @@ mod tests {
}),
true
)]
#[case::missing_field(Error::MissingField("max_tokens"), true)]
#[case::missing_field(litellm_llms::Error::MissingField("max_tokens").into(), true)]
#[case::unresolvable_provider(Error::InvalidProvider("openai".into()), false)]
#[case::upstream_failure(
litellm_http::Error::upstream(400, "bad".into()).into(),

2
litellm-rust/crates/python-bridge/src/routes/responses/host.rs
View file

@ -161,7 +161,7 @@ impl PythonBinding for ResponsesPythonHost {
self.host.error(py, error)
}
fn host_error(_: &PyErr) -> Error {
Error::InvalidRequest(litellm_llms::ErrorDetail::HostCall)
Error::invalid_request(litellm_llms::ErrorDetail::HostCall)
}
}