scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88

refactor(rust): localize route errors

Browse source
This commit is contained in:
Yujong Lee 2026-09-15 11:27:09 -07:00
parent 55fe8d40d0
commit 8cfb59082a
81 changed files with 1070 additions and 834 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

21
litellm-rust/AGENTS.md Normal file
View file

@ -0,0 +1,21 @@
# Rust error conventions
Small crates define their public `Error` enum in `src/error.rs` and reexport it with `mod error; pub use error::Error;` from `lib.rs`. A crate implementing another crate's interface may reuse that interface's error instead of inventing a wrapper
In `core`, each route defines `Error` in `src/<route>/error.rs` and reexports it from the route module. Route entrypoints return their route error. The root `core::Error` is a thin enum wrapping route errors for consumers that handle multiple routes
Create error types at boundaries with different ownership, possible failures, or caller handling. Do not create one per file or provider by default. Keep related request, response, and polling enums together in the route's `error.rs` when narrower function contracts justify them. Reexport public payload types from the owning module; keep implementation-only errors private or `pub(crate)`
Shared subsystem errors live beside their implementation and do not depend on routes or the root error. Provider implementations normally return route errors. Add a provider error only when distinct typed handling requires one. Keep small private helper errors inline rather than creating a directory solely for an error file
Import dependency errors from their owning crate, for example `litellm_auth::Error`. Do not reexport them from `core` as `AuthError`. A local import alias such as `use litellm_auth::Error as AuthError;` is appropriate when multiple error types are in scope
Derive `Debug` and `thiserror::Error`. Add `Clone`, `PartialEq`, or `Eq` only when needed; do not stringify causes to enable those derives
Use `#[from]` for unambiguous, context-free wrapping. It also marks the source. Use `#[error(transparent)]` when intentionally forwarding display and source behavior. Use `#[source]` with explicit context when the wrapper adds meaning. Use manual `From` only for infallible, context-free conversions, defined beside the receiving error when practical. Use `map_err` or a named constructor when conversion needs operation, provider, field, or dispatch context
Conversions flow from narrower errors into broader errors. Do not wrap the root error inside a route error, add reverse conversions, or add every transitive conversion solely to make `?` compile. Shared lifecycle interfaces carry the caller's error type
Preserve typed causes and structured status/dispatch information through Rust layers. Sanitize sensitive diagnostics deliberately. Convert to presentation strings and map Python exception classes or compatibility HTTP statuses at host boundaries. Never infer retry or fallback safety from display text; timeouts do not prove that no request was dispatched
Test caller-visible behavior: typed failure details survive propagation, host exception/status mappings remain correct, and post-dispatch failures cannot authorize replay. Do not test filesystem structure alone

2
litellm-rust/Cargo.lock generated
View file

@ -1845,6 +1845,7 @@ dependencies = [
"base64 0.22.1",
"futures-channel",
"futures-util",
"litellm-auth",
"litellm-config",
"litellm-core",
"reqwest 0.12.28",
@ -1854,6 +1855,7 @@ dependencies = [
"serde_json",
"sha2 0.10.9",
"subtle",
"thiserror 2.0.19",
"tokio",
"tokio-tungstenite",
"tower",

26
litellm-rust/crates/core/src/audio_transcription/error.rs Normal file
View file

@ -0,0 +1,26 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("expected {expected}, got {actual}")]
InvalidType {
expected: &'static str,
actual: &'static str,
},
#[error("missing required field: {0}")]
MissingField(&'static str),
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Transport(#[from] crate::transport::Error),
#[error(transparent)]
Params(#[from] crate::params::Error),
#[error(transparent)]
Headers(#[from] crate::http_utils::HeaderError),
#[error(transparent)]
Aws(#[from] litellm_auth_aws::Error),
}

10
litellm-rust/crates/core/src/audio_transcription/handler.rs
View file

@ -1,6 +1,6 @@
use serde_json::Value;
use crate::error::Error;
use crate::audio_transcription::Error;
use crate::http_utils::{http_request, truncate_error_body};
use super::client::http_client;
@ -21,17 +21,17 @@ pub async fn execute_audio_transcription_provider_call(
}
let response = http_request(request_builder)
.await
.map_err(|error| Error::Network(error.to_string()))?;
.map_err(|error| Error::Transport(crate::transport::Error::Network(error.to_string())))?;
let status = response.status();
let text = response
.text()
.await
.map_err(|error| Error::Network(error.to_string()))?;
.map_err(|error| Error::Transport(crate::transport::Error::Network(error.to_string())))?;
if !status.is_success() {
return Err(Error::Http {
return Err(Error::Transport(crate::transport::Error::Http {
status: status.as_u16(),
body: truncate_error_body(&text),
});
}));
}
let response_json = serde_json::from_str(&text)
.map_err(|error| Error::InvalidResponse(format!("invalid audio response JSON: {error}")))?;

3
litellm-rust/crates/core/src/audio_transcription/mod.rs
View file

@ -1,4 +1,5 @@
use crate::Error;
mod error;
pub use error::Error;
mod client;
mod handler;
mod prepare;

2
litellm-rust/crates/core/src/audio_transcription/prepare.rs
View file

@ -1,4 +1,4 @@
use crate::error::Error;
use crate::audio_transcription::Error;
use crate::http_utils::{has_header, string_headers};
use crate::providers::bedrock::audio_transcription::BEDROCK_AUDIO_TRANSCRIPTION_CONFIG;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};

2
litellm-rust/crates/core/src/audio_transcription/transformation.rs
View file

@ -1,4 +1,4 @@
use crate::Error;
use crate::audio_transcription::Error;
use serde_json::Value;
use super::types::{AudioTranscriptionRequestData, AudioTranscriptionResponseData};

19
litellm-rust/crates/core/src/call_lifecycle/host.rs
View file

@ -6,10 +6,11 @@ pub enum HostCallStep<O, C> {
Complete(C),
}
pub type HostCallFuture<'a, O, C> =
Pin<Box<dyn Future<Output = Result<HostCallStep<O, C>, crate::Error>> + Send + 'a>>;
pub type HostCallFuture<'a, O, C, E> =
Pin<Box<dyn Future<Output = Result<HostCallStep<O, C>, E>> + Send + 'a>>;
pub trait HostCall: Send + Sync {
type Error: Send + Sync + 'static;
type Operation: Send + 'static;
type Result: Send + 'static;
type Complete: Send + 'static;
@ -17,12 +18,12 @@ pub trait HostCall: Send + Sync {
fn resume(
&mut self,
result: Option<Self::Result>,
) -> HostCallFuture<'_, Self::Operation, Self::Complete>;
) -> HostCallFuture<'_, Self::Operation, Self::Complete, Self::Error>;
fn interrupt(
&mut self,
failure: HostFailure,
) -> HostCallFuture<'_, Self::Operation, Self::Complete>;
failure: HostFailure<Self::Error>,
) -> HostCallFuture<'_, Self::Operation, Self::Complete, Self::Error>;
}
pub enum HostStep<V, S> {
@ -48,9 +49,9 @@ pub enum HostPhase {
}
#[derive(Clone, Debug)]
pub enum HostFailure {
Error(crate::Error),
Cancelled(crate::Error),
pub enum HostFailure<E> {
Error(E),
Cancelled(E),
}
pub struct HostLifecycle {
@ -70,7 +71,7 @@ impl HostLifecycle {
self.phase
}
pub fn accept(&mut self, result: Result<(), HostFailure>) -> Option<crate::Error> {
pub fn accept<E>(&mut self, result: Result<(), HostFailure<E>>) -> Option<E> {
if let Err(failure) = result {
if self.phase == HostPhase::DeploymentFailure {
self.phase = HostPhase::Failure;

33
litellm-rust/crates/core/src/call_lifecycle/mod.rs
View file

@ -1,8 +1,6 @@
use std::future::Future;
use std::time::{Instant, SystemTime, UNIX_EPOCH};
use crate::Error;
pub mod host;
#[cfg(test)]
#[path = "../../tests/host_lifecycle.rs"]
@ -15,14 +13,15 @@ pub use types::{
};
pub trait CallLifecycleHooks<InitialReq, ProviderReq, Resp>: Send + Sync {
type PreCallFuture<'a>: Future<Output = Result<InitialReq, Error>> + Send + 'a
type Error: Send + Sync;
type PreCallFuture<'a>: Future<Output = Result<InitialReq, Self::Error>> + Send + 'a
where
Self: 'a,
InitialReq: 'a,
ProviderReq: 'a,
Resp: 'a;
type DuringCallFuture<'a>: Future<Output = Result<ProviderReq, Error>> + Send + 'a
type DuringCallFuture<'a>: Future<Output = Result<ProviderReq, Self::Error>> + Send + 'a
where
Self: 'a,
InitialReq: 'a,
@ -60,7 +59,7 @@ pub trait CallLifecycleHooks<InitialReq, ProviderReq, Resp>: Send + Sync {
fn async_log_failure_event<'a>(
&'a self,
context: &'a CallLifecycleContext,
error: &'a Error,
error: &'a Self::Error,
timing: &'a CallLifecycleTiming,
) -> Self::FailureFuture<'a>;
}
@ -90,12 +89,12 @@ impl<'a> CallLifecycle<'a> {
request: InitialReq,
hooks: &Hooks,
provider_call: ProviderCall,
) -> Result<Resp, Error>
) -> Result<Resp, Hooks::Error>
where
InitialReq: CallLifecycleRequest,
Hooks: CallLifecycleHooks<InitialReq, ProviderReq, Resp>,
ProviderCall: FnOnce(ProviderReq) -> ProviderFuture,
ProviderFuture: Future<Output = Result<Resp, Error>>,
ProviderFuture: Future<Output = Result<Resp, Hooks::Error>>,
{
let context = request.lifecycle_context();
self.run(context, request, hooks, provider_call).await
@ -107,11 +106,11 @@ impl<'a> CallLifecycle<'a> {
request: InitialReq,
hooks: &Hooks,
provider_call: ProviderCall,
) -> Result<Resp, Error>
) -> Result<Resp, Hooks::Error>
where
Hooks: CallLifecycleHooks<InitialReq, ProviderReq, Resp>,
ProviderCall: FnOnce(ProviderReq) -> ProviderFuture,
ProviderFuture: Future<Output = Result<Resp, Error>>,
ProviderFuture: Future<Output = Result<Resp, Hooks::Error>>,
{
let call_start = epoch_seconds();
let mut phases = Vec::new();
@ -170,7 +169,7 @@ impl<'a> CallLifecycle<'a> {
&self,
context: &CallLifecycleContext,
hooks: &Hooks,
error: &Error,
error: &Hooks::Error,
call_start: f64,
phases: &mut Vec<CallLifecyclePhaseTiming>,
) where
@ -230,6 +229,7 @@ fn epoch_seconds() -> f64 {
#[cfg(test)]
mod tests {
use super::*;
use crate::messages::Error;
use std::pin::Pin;
use std::sync::Mutex;
@ -255,6 +255,7 @@ mod tests {
}
impl CallLifecycleHooks<String, String, String> for RecordingHooks {
type Error = Error;
type PreCallFuture<'a> = BoxFuture<'a, Result<String, Error>>;
type DuringCallFuture<'a> = BoxFuture<'a, Result<String, Error>>;
type SuccessFuture<'a> = BoxFuture<'a, ()>;
@ -308,6 +309,7 @@ mod tests {
}
impl CallLifecycleHooks<RecordingRequest, String, String> for RecordingHooks {
type Error = Error;
type PreCallFuture<'a> = BoxFuture<'a, Result<RecordingRequest, Error>>;
type DuringCallFuture<'a> = BoxFuture<'a, Result<String, Error>>;
type SuccessFuture<'a> = BoxFuture<'a, ()>;
@ -387,13 +389,20 @@ mod tests {
"request".to_string(),
&hooks,
|_request| async move {
Err::<String, Error>(Error::Network("provider down".to_string()))
Err::<String, Error>(Error::Transport(crate::transport::Error::Network(
"provider down".to_string(),
)))
},
)
.await
.expect_err("call fails");
assert_eq!(error, Error::Network("provider down".to_string()));
assert_eq!(
error,
Error::Transport(crate::transport::Error::Network(
"provider down".to_string()
))
);
assert_eq!(hooks.events(), vec!["pre_call", "during_call", "failure"]);
}

4
litellm-rust/crates/core/src/chat_completions/common_utils.rs
View file

@ -1,4 +1,4 @@
use crate::Error;
use crate::chat_completions::Error;
use crate::http_utils::string_headers as shared_string_headers;
use crate::providers::anthropic::chat_completions::transformation::ANTHROPIC_CHAT_COMPLETIONS_CONFIG;
use serde_json::{Map, Value};
@ -22,5 +22,5 @@ pub(super) fn chat_completions_provider_config(
pub(super) fn string_headers(
extra_headers: Option<Map<String, Value>>,
) -> Result<Vec<(String, String)>, Error> {
shared_string_headers(HEADER_CONTEXT, extra_headers)
shared_string_headers(HEADER_CONTEXT, extra_headers).map_err(Error::from)
}

30
litellm-rust/crates/core/src/chat_completions/error.rs Normal file
View file

@ -0,0 +1,30 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("expected {expected}, got {actual}")]
InvalidType {
expected: &'static str,
actual: &'static str,
},
#[error("missing required field: {0}")]
MissingField(&'static str),
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error("invalid response: {0}")]
ResponseTransform(#[source] Box<Error>),
#[error("unsupported by the rust path: {0}")]
Unsupported(&'static str),
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Transport(#[from] crate::transport::Error),
#[error(transparent)]
Params(#[from] crate::params::Error),
#[error(transparent)]
Headers(#[from] crate::http_utils::HeaderError),
#[error(transparent)]
Aws(#[from] litellm_auth_aws::Error),
}

27
litellm-rust/crates/core/src/chat_completions/handler.rs
View file

@ -1,6 +1,6 @@
use serde_json::Value;
use crate::error::Error;
use crate::chat_completions::Error;
use crate::http_utils::{http_request, truncate_error_body};
use super::client::http_client;
@ -30,28 +30,21 @@ pub(super) async fn execute_chat_completions_provider_call(
request_builder = request_builder.timeout(duration);
}
let response = http_request(request_builder).await.map_err(|err| {
// Failing to establish the connection means the request never went out,
// so the host can still serve it. Everything else here, a timeout
// above all, may have reached the provider and been answered.
if err.is_connect() || err.is_builder() {
Error::Connect(err.to_string())
} else {
Error::Network(err.to_string())
}
})?;
let response = http_request(request_builder)
.await
.map_err(crate::transport::Error::from_reqwest_before_dispatch)?;
let status = response.status();
let text = response
.text()
.await
.map_err(|err| Error::Network(err.to_string()))?;
.map_err(|err| Error::Transport(crate::transport::Error::Network(err.to_string())))?;
if !status.is_success() {
return Err(Error::Http {
return Err(Error::Transport(crate::transport::Error::Http {
status: status.as_u16(),
body: truncate_error_body(&text),
});
}));
}
let body: Value = serde_json::from_str(&text).map_err(|err| {
@ -74,8 +67,10 @@ pub(super) async fn execute_chat_completions_provider_call(
/// can only mean the provider was already called.
pub(super) fn as_response_error(err: Error) -> Error {
match err {
already @ (Error::InvalidResponse(_) | Error::Http { .. }) => already,
other => Error::InvalidResponse(other.to_string()),
already @ (Error::InvalidResponse(_)
| Error::ResponseTransform(_)
| Error::Transport(crate::transport::Error::Http { .. })) => already,
other => Error::ResponseTransform(Box::new(other)),
}
}

3
litellm-rust/crates/core/src/chat_completions/mod.rs
View file

@ -6,7 +6,8 @@
//! credentials, and it resolves the provider, translates the conversation,
//! calls the provider, and returns a typed OpenAI-shaped response.
use crate::Error;
mod error;
pub use error::Error;
mod client;
mod common_utils;
pub mod conversation;

2
litellm-rust/crates/core/src/chat_completions/prepare.rs
View file

@ -1,6 +1,6 @@
use serde_json::Value;
use crate::error::Error;
use crate::chat_completions::Error;
use crate::http_utils::has_header;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};

33
litellm-rust/crates/core/src/chat_completions/tests.rs
View file

@ -1,6 +1,6 @@
use serde_json::{Map, Value, json};
use crate::error::Error;
use crate::chat_completions::Error;
use super::prepare::{prepare_provider_request, resolve_request};
use super::transformation::ChatCompletionsAuth;
@ -264,9 +264,11 @@ fn rejects_non_string_extra_headers() {
call.extra_headers = Some(Map::from_iter([("x-trace".to_string(), json!(7))]));
assert_eq!(
decline(call),
Error::InvalidRequest(
"chat completions extra_headers.x-trace must be a string, got number".to_string()
)
Error::Headers(crate::http_utils::HeaderError {
context: "chat completions",
name: "x-trace".to_string(),
actual: "number",
})
);
}
@ -728,7 +730,7 @@ mod round_trip {
.expect_err("response cannot be normalized");
handle.await.expect("server task");
assert!(
matches!(err, Error::InvalidResponse(_)),
matches!(err, Error::InvalidResponse(_) | Error::ResponseTransform(_)),
"expected a post-send error, got {err:?}"
);
}
@ -746,7 +748,7 @@ mod round_trip {
.expect_err("response cannot be normalized");
handle.await.expect("server task");
assert!(
matches!(err, Error::InvalidResponse(_)),
matches!(err, Error::InvalidResponse(_) | Error::ResponseTransform(_)),
"expected a post-send error, got {err:?}"
);
}
@ -764,7 +766,10 @@ mod round_trip {
.expect_err("upstream rejects");
handle.await.expect("server task");
assert!(
matches!(err, Error::Http { status: 429, .. }),
matches!(
err,
Error::Transport(crate::transport::Error::Http { status: 429, .. })
),
"expected a 429, got {err:?}"
);
}
@ -788,7 +793,7 @@ mod round_trip {
.await
.expect_err("nothing is listening");
assert!(
matches!(err, Error::Connect(_)),
matches!(err, Error::Transport(crate::transport::Error::Connect(_))),
"expected a pre-send connect failure, got {err:?}"
);
}
@ -801,21 +806,23 @@ mod round_trip {
Error::MissingField("usage"),
Error::Unsupported("non-text response content block"),
Error::InvalidRequest("whatever".to_string()),
Error::Auth("whatever".to_string()),
Error::Auth(litellm_auth::Error::ProviderAuthentication(
"whatever".to_string(),
)),
] {
let label = format!("{original:?}");
assert!(
matches!(as_response_error(original), Error::InvalidResponse(_)),
matches!(as_response_error(original.clone()), Error::ResponseTransform(source) if *source == original),
"{label} must not stay retryable once the provider has answered"
);
}
// An upstream status is already unambiguous, so it survives intact.
assert!(matches!(
as_response_error(Error::Http {
as_response_error(Error::Transport(crate::transport::Error::Http {
status: 500,
body: "boom".to_string()
}),
Error::Http { status: 500, .. }
})),
Error::Transport(crate::transport::Error::Http { status: 500, .. })
));
}
}

2
litellm-rust/crates/core/src/chat_completions/transformation.rs
View file

@ -1,4 +1,4 @@
use crate::Error;
use crate::chat_completions::Error;
use serde_json::Value;
use super::types::{

257
litellm-rust/crates/core/src/error.rs
View file

@ -1,246 +1,15 @@
use thiserror::Error as ThisError;
#[derive(Clone, Debug, ThisError, PartialEq, Eq)]
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("expected {expected}, got {actual}")]
InvalidType {
expected: &'static str,
actual: &'static str,
},
#[error("missing required field: {0}")]
MissingField(&'static str),
#[error("Document URL is required")]
MissingDocumentUrl,
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("{0}")]
Auth(String),
#[error(
"Missing {provider} API Key - Set `api_key` or the {environment_variable} environment variable"
)]
MissingApiKey {
provider: &'static str,
environment_variable: &'static str,
},
#[error(
"invalid authentication configuration: Missing Azure AI credentials - set AZURE_AI_API_KEY or configure Entra ID"
)]
MissingAzureAiCredentials,
#[error(
"invalid authentication configuration: Missing Azure Document Intelligence credentials - set AZURE_DOCUMENT_INTELLIGENCE_API_KEY or configure Entra ID"
)]
MissingAzureDocumentIntelligenceCredentials,
#[error(
"Missing REDUCTO_API_KEY - set it in the environment or pass api_key to litellm.ocr()/litellm.aocr()"
)]
MissingReductoApiKey,
#[error("upstream request failed with status {status}: {body}")]
Http { status: u16, body: String },
#[error("upstream network error: {0}")]
Network(String),
/// The provider was never reached: DNS, TCP, TLS or proxy setup failed
/// before any byte of the request went out. Nothing was billed, so a host
/// that keeps a reference implementation can serve the request itself.
/// A timeout is deliberately not this, since the provider may have received
/// and answered the request already.
#[error("could not reach the provider: {0}")]
Connect(String),
#[error("routing error: {0}")]
Routing(String),
/// The request is outside the surface this route covers in Rust. Hosts that
/// keep a reference implementation treat this as "fall back", not "fail".
#[error("unsupported by the rust path: {0}")]
Unsupported(&'static str),
}
impl Error {
pub const fn http_status_code(&self) -> Option<u16> {
match self {
Self::InvalidRequest(_) => Some(400),
Self::MissingDocumentUrl => Some(500),
Self::Http { status, .. } => Some(*status),
_ => None,
}
}
}
#[derive(Debug, ThisError)]
pub(crate) enum MediaError {
#[error("media URL rejected by network policy")]
BlockedUrl,
#[error("media download is disabled")]
DownloadDisabled,
#[error("media download exceeds the maximum size")]
DownloadTooLarge,
#[error("too many redirects while fetching media")]
TooManyRedirects,
#[error("media redirect is missing a Location header")]
MissingRedirectLocation,
#[error("invalid media redirect")]
InvalidRedirect,
#[error("media download failed with status {0}")]
Http(u16),
#[error("media download timed out")]
Timeout,
#[error("{0}")]
Transport(#[from] TransportError),
}
#[derive(Clone, Debug, ThisError, PartialEq, Eq)]
pub enum TransportError {
#[error("upstream request failed with status {status}: {body}")]
Http { status: u16, body: String },
#[error("upstream network error: {0}")]
Network(String),
#[error("could not reach the provider: {0}")]
Connect(String),
}
impl TransportError {
pub fn from_reqwest_before_dispatch(error: reqwest::Error) -> Self {
let before_dispatch = !error.is_timeout() && (error.is_connect() || error.is_builder());
let message = error.without_url().to_string();
if before_dispatch {
Self::Connect(message)
} else {
Self::Network(message)
}
}
}
impl From<reqwest::Error> for TransportError {
fn from(error: reqwest::Error) -> Self {
Self::Network(error.without_url().to_string())
}
}
impl From<crate::ocr::error::OcrRequestError> for Error {
fn from(error: crate::ocr::error::OcrRequestError) -> Self {
match error {
crate::ocr::error::OcrRequestError::MissingField(field) => Self::MissingField(field),
crate::ocr::error::OcrRequestError::MissingDocumentUrl => Self::MissingDocumentUrl,
error => Self::InvalidRequest(error.to_string()),
}
}
}
impl From<crate::ocr::error::OcrResponseError> for Error {
fn from(error: crate::ocr::error::OcrResponseError) -> Self {
Self::InvalidResponse(error.to_string())
}
}
impl From<TransportError> for Error {
fn from(error: TransportError) -> Self {
match error {
TransportError::Http { status, body } => Self::Http { status, body },
TransportError::Network(message) => Self::Network(message),
TransportError::Connect(message) => Self::Connect(message),
}
}
}
impl From<crate::AuthError> for Error {
fn from(error: crate::AuthError) -> Self {
match error {
crate::AuthError::MissingApiKey {
provider,
environment_variable,
} => Self::MissingApiKey {
provider,
environment_variable,
},
error => Self::Auth(error.to_string()),
}
}
}
impl From<litellm_auth_aws::Error> for Error {
fn from(error: litellm_auth_aws::Error) -> Self {
Self::from(crate::AuthError::from(error))
}
}
pub fn json_type_name(value: &serde_json::Value) -> &'static str {
match value {
serde_json::Value::Null => "null",
serde_json::Value::Bool(_) => "bool",
serde_json::Value::Number(_) => "number",
serde_json::Value::String(_) => "string",
serde_json::Value::Array(_) => "array",
serde_json::Value::Object(_) => "object",
}
}
#[cfg(test)]
mod transport_tests {
use super::*;
#[test]
fn missing_auth_key_preserves_guidance_in_public_error() {
let error = Error::from(crate::AuthError::MissingApiKey {
provider: "Vertex",
environment_variable: "GOOGLE_APPLICATION_CREDENTIALS",
});
assert_eq!(
error,
Error::MissingApiKey {
provider: "Vertex",
environment_variable: "GOOGLE_APPLICATION_CREDENTIALS",
}
);
assert_eq!(
error.to_string(),
"Missing Vertex API Key - Set `api_key` or the GOOGLE_APPLICATION_CREDENTIALS environment variable"
);
}
#[tokio::test]
async fn transport_errors_remove_urls_and_keep_dispatch_context() {
let error = reqwest::Client::builder()
.no_proxy()
.build()
.expect("client")
.get("http://localhost:invalid/private?api_key=secret")
.send()
.await
.expect_err("invalid port");
let error = TransportError::from_reqwest_before_dispatch(error);
assert!(matches!(error, TransportError::Connect(_)));
assert!(!error.to_string().contains("secret"));
assert!(!error.to_string().contains("private"));
}
#[tokio::test]
async fn request_timeout_is_not_safe_to_retry_as_a_connect_failure() {
use std::time::Duration;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let request = reqwest::Client::builder()
.no_proxy()
.build()
.expect("client")
.get(format!("http://{address}"))
.timeout(Duration::from_millis(200))
.send();
let (response, accepted) = tokio::join!(
request,
tokio::time::timeout(Duration::from_secs(2), listener.accept())
);
let _connection = accepted
.expect("accept deadline")
.expect("accepted connection");
let error = response.expect_err("server does not respond");
assert!(error.is_timeout());
assert!(matches!(
TransportError::from_reqwest_before_dispatch(error),
TransportError::Network(_)
));
}
#[error(transparent)]
Ocr(#[from] crate::ocr::Error),
#[error(transparent)]
Messages(#[from] crate::messages::Error),
#[error(transparent)]
ChatCompletions(#[from] crate::chat_completions::Error),
#[error(transparent)]
AudioTranscription(#[from] crate::audio_transcription::Error),
#[error(transparent)]
Responses(#[from] crate::responses::Error),
#[error(transparent)]
Realtime(#[from] crate::realtime::Error),
}

39
litellm-rust/crates/core/src/http_utils.rs
View file

@ -1,7 +1,14 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
#[error("invalid request: {context} extra_headers.{name} must be a string, got {actual}")]
pub struct HeaderError {
pub context: &'static str,
pub name: String,
pub actual: &'static str,
}
use serde_json::{Map, Value};
use crate::constants::UPSTREAM_ERROR_BODY_MAX_CHARS;
use crate::error::{Error, json_type_name};
#[allow(
dead_code,
@ -63,7 +70,7 @@ pub fn truncate_error_body(body: &str) -> String {
pub fn string_headers(
context: &'static str,
extra_headers: Option<Map<String, Value>>,
) -> Result<Vec<(String, String)>, Error> {
) -> Result<Vec<(String, String)>, HeaderError> {
extra_headers
.unwrap_or_default()
.into_iter()
@ -71,11 +78,10 @@ pub fn string_headers(
value
.as_str()
.map(|value| (key.clone(), value.to_string()))
.ok_or_else(|| {
Error::InvalidRequest(format!(
"{context} extra_headers.{key} must be a string, got {}",
json_type_name(&value)
))
.ok_or_else(|| HeaderError {
context,
name: key,
actual: json_type_name(&value),
})
})
.collect()
@ -113,6 +119,17 @@ where
<Option<T> as serde::Deserialize>::deserialize(deserializer).map(Some)
}
pub fn json_type_name(value: &serde_json::Value) -> &'static str {
match value {
serde_json::Value::Null => "null",
serde_json::Value::Bool(_) => "bool",
serde_json::Value::Number(_) => "number",
serde_json::Value::String(_) => "string",
serde_json::Value::Array(_) => "array",
serde_json::Value::Object(_) => "object",
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -192,9 +209,11 @@ mod tests {
let err = string_headers("chat completions", Some(headers)).expect_err("non-string value");
assert_eq!(
err,
Error::InvalidRequest(
"chat completions extra_headers.x-trace must be a string, got number".to_string()
)
HeaderError {
context: "chat completions",
name: "x-trace".into(),
actual: "number"
}
);
}

5
litellm-rust/crates/core/src/lib.rs
View file

@ -2,7 +2,7 @@ pub mod audio_transcription;
pub mod call_lifecycle;
pub mod chat_completions;
pub mod constants;
pub mod error;
mod error;
pub mod http_utils;
pub(crate) mod llms;
mod media;
@ -14,4 +14,5 @@ pub mod responses;
mod url_utils;
pub use error::Error;
pub use litellm_auth::Error as AuthError;
pub mod transport;

21
litellm-rust/crates/core/src/llms/azure_ai/ocr/cohere_parse_transformation.rs
View file

@ -1,10 +1,9 @@
use crate::Error;
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::llms::cohere::ocr::transformation::CohereParseConfig;
use crate::llms::cohere::ocr::{CohereParams, CohereResponse, validate_document};
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::document::{inline_remote_document, validate_inline_document};
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{credential_env, transform_request_body};
use crate::ocr::types::{LiteLLMOcrRequest, LiteLLMOcrResponse};
use crate::url_utils::ApiUrl;
@ -26,7 +25,7 @@ impl BaseOcrConfig for AzureAICohereParseConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let params = crate::ocr::wire::decode_request_value::<CohereParams>(
serde_json::Value::Object(request.optional_params.clone().into()),
"optional_params",
@ -46,9 +45,9 @@ impl BaseOcrConfig for AzureAICohereParseConfig {
.or_else(|| credential_env(AZURE_AI_API_BASE_ENV))
.filter(|base| !base.trim().is_empty())
.ok_or_else(|| {
Error::Auth(
Error::Auth(litellm_auth::Error::ProviderAuthentication(
"Missing Azure AI API Base - Set AZURE_AI_API_BASE or pass api_base".into(),
)
))
})?;
let headers = super::transformation::validate_environment(
&request.connection,
@ -85,15 +84,15 @@ impl BaseOcrConfig for AzureAICohereParseConfig {
&self,
request: &LiteLLMOcrRequest,
response: CohereResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
CohereParseConfig.transform_ocr_response(request, response)
}
}
fn complete_url(base: &str) -> Result<String, OcrError> {
fn complete_url(base: &str) -> Result<String, Error> {
let mut url = reqwest::Url::parse(base).map_err(|_| invalid_api_base())?;
if !matches!(url.scheme(), "http" | "https") {
return Err(invalid_api_base().into());
return Err(invalid_api_base());
}
let path = url.path().trim_end_matches('/').to_string();
if path.ends_with("/v2/parse") {
@ -104,11 +103,11 @@ fn complete_url(base: &str) -> Result<String, OcrError> {
ApiUrl::parse(url.as_str())
.and_then(|url| url.complete_path(&["providers", "cohere", "v2", "parse"]))
.map(|url| url.into_string())
.map_err(|_| invalid_api_base().into())
.map_err(|_| invalid_api_base())
}
fn invalid_api_base() -> OcrRequestError {
OcrRequestError::RequestField {
fn invalid_api_base() -> Error {
Error::RequestField {
path: "api_base".into(),
}
}

9
litellm-rust/crates/core/src/llms/azure_ai/ocr/common_utils.rs
View file

@ -1,7 +1,6 @@
use std::sync::OnceLock;
use crate::Error;
use crate::ocr::error::OcrError;
use crate::ocr::Error;
use crate::ocr::types::OcrConnection;
use litellm_auth::{InputSource, Sourced};
use litellm_auth_azure::{AzureAuthInputs, AzureAuthService};
@ -16,7 +15,7 @@ pub(super) async fn resolve_entra(
.get_azure_ad_token(config, env_lookup)
.await
.or_else(|error| match error {
crate::AuthError::EmptyAzureToken => Ok(None),
litellm_auth::Error::EmptyAzureToken => Ok(None),
other => Err(other),
})
.map(|credential| {
@ -32,12 +31,12 @@ pub(super) async fn resolve_entra(
pub(super) fn validate_destination(
connection: &OcrConnection,
credential_source: InputSource,
) -> Result<(), OcrError> {
) -> Result<(), Error> {
if connection.api_base.is_some()
&& connection.api_base_source == InputSource::Request
&& credential_source != InputSource::Request
{
return Err(Error::from(crate::AuthError::RequestAzureCredentialDestination).into());
return Err(litellm_auth::Error::RequestAzureCredentialDestination.into());
}
Ok(())
}

99
litellm-rust/crates/core/src/llms/azure_ai/ocr/document_intelligence/transformation.rs
View file

@ -11,16 +11,15 @@ use tokio::time::Instant;
use litellm_auth::{InputSource, Sourced};
use litellm_auth_azure::AzureAuthInputs;
use crate::Error;
use crate::constants::{
AZURE_DI_API_VERSION, AZURE_DI_DEFAULT_DPI, AZURE_DI_DEFAULT_HEIGHT, AZURE_DI_DEFAULT_WIDTH,
AZURE_DI_SUBSCRIPTION_HEADER, OCR_POLL_RETRY_SECS,
};
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::client::read_json_response;
use crate::ocr::document::InlineDocument;
use crate::ocr::error::{OcrError, OcrPollingError, OcrRequestError, OcrResponseError};
use crate::ocr::hooks::OcrHooks;
use crate::ocr::prepare::{ParsedProviderParams, credential_env, transform_request_body};
use crate::ocr::types::{
@ -173,19 +172,19 @@ fn optional_f64<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Option<f64
fn decode_input_params(
params: Map<String, Value>,
prefix: &str,
) -> Result<ParsedProviderParams<DocumentIntelligenceInputParams>, OcrRequestError> {
) -> Result<ParsedProviderParams<DocumentIntelligenceInputParams>, Error> {
if let Some(Value::Array(pages)) = params.get("pages") {
if pages.iter().any(Value::is_boolean) {
return Err(OcrRequestError::Pages("boolean page index".into()));
return Err(Error::Pages("boolean page index".into()));
}
if pages
.iter()
.any(|page| page.is_number() && page.as_i64().is_none())
{
return Err(OcrRequestError::Pages("page index is out of range".into()));
return Err(Error::Pages("page index is out of range".into()));
}
if !pages.iter().all(Value::is_i64) && !pages.iter().all(Value::is_string) {
return Err(OcrRequestError::Pages("mixed page element types".into()));
return Err(Error::Pages("mixed page element types".into()));
}
}
crate::ocr::wire::decode_request_value(Value::Object(params), prefix)
@ -193,7 +192,7 @@ fn decode_input_params(
fn normalize_ocr_params(
params: DocumentIntelligenceInputParams,
) -> Result<DocumentIntelligenceParams, OcrRequestError> {
) -> Result<DocumentIntelligenceParams, Error> {
Ok(DocumentIntelligenceParams {
pages: params.pages.map(normalize_pages).transpose()?.flatten(),
features: params
@ -204,7 +203,7 @@ fn normalize_ocr_params(
})
}
fn normalize_pages(pages: PagesInput) -> Result<Option<String>, OcrRequestError> {
fn normalize_pages(pages: PagesInput) -> Result<Option<String>, Error> {
let normalized = match pages {
PagesInput::ZeroBasedIndices(indices) => {
if indices.is_empty() {
@ -214,10 +213,10 @@ fn normalize_pages(pages: PagesInput) -> Result<Option<String>, OcrRequestError>
.into_iter()
.map(|page| {
if page < 0 {
return Err(OcrRequestError::Pages("negative page index".into()));
return Err(Error::Pages("negative page index".into()));
}
page.checked_add(1)
.ok_or_else(|| OcrRequestError::Pages("page index is out of range".into()))
.ok_or_else(|| Error::Pages("page index is out of range".into()))
})
.collect::<Result<BTreeSet<_>, _>>()?
.into_iter()
@ -242,7 +241,7 @@ fn normalize_pages(pages: PagesInput) -> Result<Option<String>, OcrRequestError>
.join(","),
};
if !normalized.split(',').all(valid_page_token) {
return Err(OcrRequestError::Pages("invalid native page range".into()));
return Err(Error::Pages("invalid native page range".into()));
}
Ok(Some(normalized))
}
@ -263,7 +262,7 @@ fn valid_page_token(token: &str) -> bool {
}
}
fn normalize_features(features: FeaturesInput) -> Result<Option<String>, OcrRequestError> {
fn normalize_features(features: FeaturesInput) -> Result<Option<String>, Error> {
let tokens = match features {
FeaturesInput::Names(names) => names,
FeaturesInput::CommaSeparated(names) => names.split(',').map(str::to_string).collect(),
@ -278,18 +277,16 @@ fn normalize_features(features: FeaturesInput) -> Result<Option<String>, OcrRequ
};
first.is_ascii_alphabetic() && rest.iter().all(u8::is_ascii_alphanumeric)
}) {
return Err(OcrRequestError::Features);
return Err(Error::Features);
}
Ok(Some(normalized.join(",")))
}
#[tracing::instrument(target = "litellm::function_trace", level = "trace", skip_all)]
fn transform_ocr_request(
document: OcrDocument,
) -> Result<DocumentIntelligenceRequest, OcrRequestError> {
fn transform_ocr_request(document: OcrDocument) -> Result<DocumentIntelligenceRequest, Error> {
let source = document.source();
if source.is_empty() {
return Err(OcrRequestError::MissingDocumentUrl);
return Err(Error::MissingDocumentUrl);
}
Ok(if let Some(document) = InlineDocument::parse(source)? {
DocumentIntelligenceRequest::Base64Source {
@ -306,9 +303,9 @@ fn transform_ocr_request(
fn transform_ocr_response(
model: &str,
response: AzureDocumentIntelligenceOperation,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
if response.status != Some(OperationStatus::Succeeded) {
return Err(OcrResponseError::OperationStatus(
return Err(Error::OperationStatus(
response
.status
.map(|status| status.to_string())
@ -337,12 +334,12 @@ fn transform_ocr_response(
})
}
fn normalize_page(page: AzureDocumentIntelligencePage) -> Result<Value, OcrResponseError> {
fn normalize_page(page: AzureDocumentIntelligencePage) -> Result<Value, Error> {
let index = page
.page_number
.unwrap_or(1)
.checked_sub(1)
.ok_or(OcrResponseError::NumericRange("page.pageNumber"))?;
.ok_or(Error::NumericRange("page.pageNumber"))?;
let scale = if page.unit.as_deref().unwrap_or("inch") == "inch" {
AZURE_DI_DEFAULT_DPI as f64
} else {
@ -372,10 +369,10 @@ fn normalize_page(page: AzureDocumentIntelligencePage) -> Result<Value, OcrRespo
}))
}
fn pixel_dimension(value: f64, scale: f64, field: &'static str) -> Result<i64, OcrResponseError> {
fn pixel_dimension(value: f64, scale: f64, field: &'static str) -> Result<i64, Error> {
let value = value * scale;
if !value.is_finite() || value < i64::MIN as f64 || value > i64::MAX as f64 {
return Err(OcrResponseError::NumericRange(field));
return Err(Error::NumericRange(field));
}
Ok(value.trunc() as i64)
}
@ -393,27 +390,27 @@ async fn read_operation_response(
connection: &OcrConnection,
native: bool,
hooks: &Arc<dyn OcrHooks>,
) -> Result<DecodedOcrResponse<AzureDocumentIntelligenceOperation>, OcrError> {
) -> Result<DecodedOcrResponse<AzureDocumentIntelligenceOperation>, Error> {
if response.status() != reqwest::StatusCode::ACCEPTED {
let bytes =
crate::ocr::client::read_response_bytes(response, connection.max_response_bytes)
.await?;
crate::ocr::handler::post_call(hooks, &bytes).await?;
return Ok(crate::ocr::wire::decode_response(&bytes, native)?);
return crate::ocr::wire::decode_response(&bytes, native);
}
let location = response
.headers()
.get("operation-location")
.and_then(|value| value.to_str().ok())
.ok_or(OcrPollingError::PollLocation)?
.ok_or(Error::PollLocation)?
.to_string();
let original = Url::parse(original_url).map_err(|_| OcrPollingError::PollOrigin)?;
let operation = Url::parse(&location).map_err(|_| OcrPollingError::PollOrigin)?;
let original = Url::parse(original_url).map_err(|_| Error::PollOrigin)?;
let operation = Url::parse(&location).map_err(|_| Error::PollOrigin)?;
if original.origin() != operation.origin()
|| !operation.username().is_empty()
|| operation.password().is_some()
{
return Err(OcrPollingError::PollOrigin.into());
return Err(Error::PollOrigin);
}
let bytes =
crate::ocr::client::read_response_bytes(response, connection.max_response_bytes).await?;
@ -427,16 +424,16 @@ async fn poll_operation(
headers: &[(String, String)],
connection: &OcrConnection,
native: bool,
) -> Result<DecodedOcrResponse<AzureDocumentIntelligenceOperation>, OcrError> {
) -> Result<DecodedOcrResponse<AzureDocumentIntelligenceOperation>, Error> {
let deadline = Instant::now()
.checked_add(connection.poll_timeout)
.ok_or(OcrPollingError::PollTimeout)?;
.ok_or(Error::PollTimeout)?;
loop {
let remaining = deadline
.checked_duration_since(Instant::now())
.filter(|remaining| !remaining.is_zero())
.ok_or(OcrPollingError::PollTimeout)?;
.ok_or(Error::PollTimeout)?;
let builder = http_client
.get(url.clone())
.timeout(remaining.min(connection.timeout));
@ -447,8 +444,8 @@ async fn poll_operation(
);
let response = tokio::time::timeout_at(deadline, crate::http_utils::http_request(builder))
.await
.map_err(|_| OcrPollingError::PollTimeout)?
.map_err(crate::error::TransportError::from)?;
.map_err(|_| Error::PollTimeout)?
.map_err(crate::transport::Error::from)?;
let retry = response
.headers()
.get(reqwest::header::RETRY_AFTER)
@ -465,22 +462,21 @@ async fn poll_operation(
),
)
.await
.map_err(|_| OcrPollingError::PollTimeout)??;
.map_err(|_| Error::PollTimeout)??;
match &decoded.data.status {
Some(OperationStatus::Succeeded) => return Ok(decoded),
Some(OperationStatus::Running | OperationStatus::NotStarted) => {
tokio::time::timeout_at(deadline, tokio::time::sleep(Duration::from_secs(retry)))
.await
.map_err(|_| OcrPollingError::PollTimeout)?;
.map_err(|_| Error::PollTimeout)?;
}
status => {
return Err(OcrResponseError::OperationStatus(
return Err(Error::OperationStatus(
status
.as_ref()
.map(ToString::to_string)
.unwrap_or_else(|| "None".into()),
)
.into());
));
}
}
}
@ -503,7 +499,7 @@ impl BaseOcrConfig for AzureDocumentIntelligenceOCRConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let params = map_ocr_params(request)?;
let config = AzureAuthInputs {
azure_ad_token_provider: request.azure_ad_token_provider.clone(),
@ -516,7 +512,7 @@ impl BaseOcrConfig for AzureDocumentIntelligenceOCRConfig {
let headers = validate_environment(&request.connection, &config, &credential_env).await?;
let endpoint = nonblank(request.connection.api_base.clone())
.or_else(|| nonblank(credential_env(AZURE_DI_ENDPOINT_ENV)))
.ok_or_else(|| Error::Auth("Missing Azure Document Intelligence API Base - Set AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT or pass api_base".into()))?;
.ok_or_else(|| Error::Auth(litellm_auth::Error::ProviderAuthentication("Missing Azure Document Intelligence API Base - Set AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT or pass api_base".into())))?;
let url = get_complete_url(&endpoint, &request.model, &params)?;
let body = transform_ocr_request(request.document.clone())?;
transform_request_body(client, request, &url, &headers, false, body, |_| Ok(())).await
@ -526,7 +522,7 @@ impl BaseOcrConfig for AzureDocumentIntelligenceOCRConfig {
&self,
request: &LiteLLMOcrRequest,
response: AzureDocumentIntelligenceOperation,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
transform_ocr_response(&request.model, response)
}
@ -537,7 +533,7 @@ impl BaseOcrConfig for AzureDocumentIntelligenceOCRConfig {
url: &str,
headers: &[(String, String)],
request: &LiteLLMOcrRequest,
) -> Result<crate::ocr::wire::DecodedOcrResponse<AzureDocumentIntelligenceOperation>, OcrError>
) -> Result<crate::ocr::wire::DecodedOcrResponse<AzureDocumentIntelligenceOperation>, Error>
{
read_operation_response(
client.polling_http(),
@ -553,9 +549,7 @@ impl BaseOcrConfig for AzureDocumentIntelligenceOCRConfig {
}
#[tracing::instrument(target = "litellm::function_trace", level = "trace", skip_all)]
fn map_ocr_params(
request: &LiteLLMOcrRequest,
) -> Result<DocumentIntelligenceParams, OcrRequestError> {
fn map_ocr_params(request: &LiteLLMOcrRequest) -> Result<DocumentIntelligenceParams, Error> {
let params = decode_input_params(request.optional_params.clone().into(), "optional_params")?;
let crate::ocr::prepare::ParsedProviderParams {
known: params,
@ -568,7 +562,7 @@ fn get_complete_url(
endpoint: &str,
model: &str,
params: &DocumentIntelligenceParams,
) -> Result<String, OcrError> {
) -> Result<String, Error> {
let model = format!("{}:analyze", model_id(model)?);
ApiUrl::parse(endpoint)
.and_then(|url| url.complete_path(&["documentintelligence", "documentModels", &model]))
@ -586,17 +580,16 @@ fn get_complete_url(
)
.into_string()
})
.map_err(|_| OcrRequestError::RequestField {
.map_err(|_| Error::RequestField {
path: "api_base".into(),
})
.map_err(OcrError::from)
}
async fn validate_environment(
connection: &OcrConnection,
config: &AzureAuthInputs,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, OcrError> {
) -> Result<Vec<(String, String)>, Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization")
|| crate::http_utils::has_header(&connection.extra_headers, AZURE_DI_SUBSCRIPTION_HEADER)
{
@ -631,10 +624,10 @@ async fn validate_environment(
)
}
fn model_id(model: &str) -> Result<&str, OcrRequestError> {
fn model_id(model: &str) -> Result<&str, Error> {
let model = model.rsplit('/').next().unwrap_or(model);
if matches!(model, "." | "..") {
return Err(OcrRequestError::DotModel);
return Err(Error::DotModel);
}
Ok(model)
}
@ -651,7 +644,7 @@ mod tests {
use rstest::rstest;
use serde_json::{Value, json};
fn map(value: Value) -> Result<DocumentIntelligenceParams, OcrRequestError> {
fn map(value: Value) -> Result<DocumentIntelligenceParams, Error> {
let fields = value.as_object().unwrap().clone();
normalize_ocr_params(decode_input_params(fields, "optional_params")?.known)
}

22
litellm-rust/crates/core/src/llms/azure_ai/ocr/transformation.rs
View file

@ -1,11 +1,10 @@
use crate::Error;
use crate::constants::AZURE_AI_OCR_PATH;
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::llms::mistral::ocr::MistralOcrResponse;
use crate::llms::mistral::ocr::transformation::MistralOCRConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::document::{inline_remote_document, validate_inline_document};
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{credential_env, transform_request_body};
use crate::ocr::types::{LiteLLMOcrRequest, LiteLLMOcrResponse, OcrConnection};
use crate::url_utils::ApiUrl;
@ -29,7 +28,7 @@ impl BaseOcrConfig for AzureAIOCRConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let params = self.map_ocr_params(&request.model, &request.optional_params);
let config = AzureAuthInputs {
azure_ad_token_provider: request.azure_ad_token_provider.clone(),
@ -66,7 +65,7 @@ impl BaseOcrConfig for AzureAIOCRConfig {
&self,
request: &LiteLLMOcrRequest,
response: MistralOcrResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
MistralOCRConfig.transform_ocr_response(request, response)
}
}
@ -74,21 +73,16 @@ impl BaseOcrConfig for AzureAIOCRConfig {
fn get_complete_url(
api_base: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, OcrError> {
) -> Result<String, Error> {
let base = nonblank(api_base.map(str::to_string))
.or_else(|| nonblank(env_lookup(AZURE_AI_API_BASE_ENV)))
.ok_or_else(|| Error::Auth(
"Missing Azure AI API Base - Set AZURE_AI_API_BASE environment variable or pass api_base parameter".into(),
))?;
.ok_or_else(|| Error::Auth(litellm_auth::Error::ProviderAuthentication("Missing Azure AI API Base - Set AZURE_AI_API_BASE environment variable or pass api_base parameter".into())))?;
let path: Vec<&str> = AZURE_AI_OCR_PATH.trim_matches('/').split('/').collect();
ApiUrl::parse(&base)
.and_then(|url| url.complete_path(&path))
.map(|url| url.into_string())
.map_err(|_| {
OcrRequestError::RequestField {
path: "api_base".into(),
}
.into()
.map_err(|_| Error::RequestField {
path: "api_base".into(),
})
}
@ -96,7 +90,7 @@ pub(super) async fn validate_environment(
connection: &OcrConnection,
config: &AzureAuthInputs,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, OcrError> {
) -> Result<Vec<(String, String)>, Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization") {
if config.azure_ad_token_provider.is_some() {
super::common_utils::resolve_entra(config, env_lookup).await?;

12
litellm-rust/crates/core/src/llms/base_llm/ocr/transformation.rs
View file

@ -2,8 +2,8 @@ use std::future::Future;
use serde::de::DeserializeOwned;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::error::{OcrError, OcrResponseError};
use crate::ocr::types::{LiteLLMOcrRequest, LiteLLMOcrResponse, OcrResponseFormat};
use crate::ocr::wire::DecodedOcrResponse;
use crate::params::OpaqueParams;
@ -23,13 +23,13 @@ pub(crate) trait BaseOcrConfig: Send + Sync + Sized + 'static {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> impl Future<Output = Result<reqwest::Request, OcrError>> + Send;
) -> impl Future<Output = Result<reqwest::Request, Error>> + Send;
fn transform_ocr_response(
&self,
request: &LiteLLMOcrRequest,
response: Self::ProviderResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError>;
) -> Result<LiteLLMOcrResponse, Error>;
fn read_response(
&self,
@ -38,7 +38,7 @@ pub(crate) trait BaseOcrConfig: Send + Sync + Sized + 'static {
_url: &str,
_headers: &[(String, String)],
request: &LiteLLMOcrRequest,
) -> impl Future<Output = Result<DecodedOcrResponse<Self::ProviderResponse>, OcrError>> + Send
) -> impl Future<Output = Result<DecodedOcrResponse<Self::ProviderResponse>, Error>> + Send
{
async move {
let bytes = crate::ocr::client::read_response_bytes(
@ -47,10 +47,10 @@ pub(crate) trait BaseOcrConfig: Send + Sync + Sized + 'static {
)
.await?;
crate::ocr::handler::post_call(&request.hooks, &bytes).await?;
Ok(crate::ocr::wire::decode_response(
crate::ocr::wire::decode_response(
&bytes,
request.response_format()? == OcrResponseFormat::Native,
)?)
)
}
}
}

45
litellm-rust/crates/core/src/llms/cohere/ocr/transformation.rs
View file

@ -1,12 +1,11 @@
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value, json};
use crate::Error;
use crate::constants::{COHERE_API_KEY_ENV, COHERE_PARSE_API_BASE};
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::document::InlineDocument;
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{credential_env, transform_request_body};
use crate::ocr::types::{LiteLLMOcrRequest, LiteLLMOcrResponse, OcrConnection, OcrDocument};
use crate::url_utils::ApiUrl;
@ -32,16 +31,16 @@ pub(crate) struct CohereRequest {
pub output_format: OutputFormat,
}
pub(crate) fn validate_document(document: &OcrDocument) -> Result<(), OcrRequestError> {
pub(crate) fn validate_document(document: &OcrDocument) -> Result<(), Error> {
let OcrDocument::ImageUrl { image_url, .. } = document else {
return Err(OcrRequestError::CohereImageOnly);
return Err(Error::CohereImageOnly);
};
if image_url.is_empty() {
return Err(OcrRequestError::CohereImageOnly);
return Err(Error::CohereImageOnly);
}
if let Some(inline) = InlineDocument::parse(image_url)? {
if !inline.mime_type().type_.eq_ignore_ascii_case("image") {
return Err(OcrRequestError::CohereImageOnly);
return Err(Error::CohereImageOnly);
}
inline.decode(crate::constants::OCR_INLINE_MAX_BYTES)?;
}
@ -82,14 +81,14 @@ struct CohereBilledUnits {
pub(crate) fn transform_response(
model: &str,
response: CohereResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
let pages_processed = response
.meta
.and_then(|meta| meta.billed_units)
.and_then(|units| units.pages)
.map(Ok)
.unwrap_or_else(|| {
i64::try_from(response.pages.len()).map_err(|_| OcrResponseError::NumericRange("pages"))
i64::try_from(response.pages.len()).map_err(|_| Error::NumericRange("pages"))
})?;
let pages = response
.pages
@ -97,7 +96,7 @@ pub(crate) fn transform_response(
.enumerate()
.map(|(position, page)| {
let index = page.index.map(Ok).unwrap_or_else(|| {
i64::try_from(position).map_err(|_| OcrResponseError::NumericRange("page index"))
i64::try_from(position).map_err(|_| Error::NumericRange("page index"))
})?;
let (content, images) = page
.markdown
@ -128,7 +127,7 @@ pub(crate) fn transform_response(
}
Ok(normalized)
})
.collect::<Result<Vec<_>, OcrResponseError>>()?;
.collect::<Result<Vec<_>, Error>>()?;
Ok(LiteLLMOcrResponse {
pages,
model: model.into(),
@ -149,7 +148,7 @@ impl CohereParseConfig {
model: &str,
document: OcrDocument,
params: CohereParams,
) -> Result<CohereRequest, OcrRequestError> {
) -> Result<CohereRequest, Error> {
validate_document(&document)?;
Ok(CohereRequest {
model: model.into(),
@ -170,7 +169,7 @@ impl BaseOcrConfig for CohereParseConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let params = crate::ocr::wire::decode_request_value::<CohereParams>(
serde_json::Value::Object(request.optional_params.clone().into()),
"optional_params",
@ -194,24 +193,24 @@ impl BaseOcrConfig for CohereParseConfig {
&self,
request: &LiteLLMOcrRequest,
response: CohereResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
transform_response(&request.model, response)
}
}
fn complete_url(base: &str) -> Result<String, OcrError> {
fn complete_url(base: &str) -> Result<String, Error> {
let parsed = reqwest::Url::parse(base).map_err(|_| invalid_api_base())?;
if !matches!(parsed.scheme(), "http" | "https") {
return Err(invalid_api_base().into());
return Err(invalid_api_base());
}
ApiUrl::parse(base)
.and_then(|url| url.complete_path(&["v2", "parse"]))
.map(|url| url.into_string())
.map_err(|_| invalid_api_base().into())
.map_err(|_| invalid_api_base())
}
fn invalid_api_base() -> OcrRequestError {
OcrRequestError::RequestField {
fn invalid_api_base() -> Error {
Error::RequestField {
path: "api_base".into(),
}
}
@ -219,7 +218,7 @@ fn invalid_api_base() -> OcrRequestError {
fn validate_environment(
connection: &OcrConnection,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, OcrError> {
) -> Result<Vec<(String, String)>, Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization") {
return Ok(connection.extra_headers.clone());
}
@ -231,7 +230,9 @@ fn validate_environment(
.map(str::to_string)
.or_else(|| env_lookup(COHERE_API_KEY_ENV).filter(|key| !key.trim().is_empty()))
.ok_or_else(|| {
Error::Auth("Missing COHERE_API_KEY - set it in the environment or pass api_key".into())
Error::Auth(litellm_auth::Error::ProviderAuthentication(
"Missing COHERE_API_KEY - set it in the environment or pass api_key".into(),
))
})?;
Ok(
std::iter::once(("Authorization".into(), format!("Bearer {key}")))
@ -320,7 +321,7 @@ mod tests {
] {
assert_eq!(
validate_document(&serde_json::from_value(value).unwrap()),
Err(OcrRequestError::CohereImageOnly)
Err(Error::CohereImageOnly)
);
}
assert!(serde_json::from_value::<CohereParams>(json!({"output_format":"html"})).is_err());
@ -368,7 +369,7 @@ mod tests {
},
&|_| None,
),
Err(OcrError::Public(Error::Auth(_)))
Err(Error::Auth(_))
));
}
}

28
litellm-rust/crates/core/src/llms/mistral/ocr/transformation.rs
View file

@ -1,11 +1,10 @@
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
use crate::Error;
use crate::constants::MISTRAL_OCR_API_BASE;
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{credential_env, transform_request_body};
use crate::ocr::types::{LiteLLMOcrRequest, LiteLLMOcrResponse, OcrConnection, OcrDocument};
use crate::params::OpaqueParams;
@ -34,7 +33,7 @@ pub(crate) struct MistralOcrResponse {
pub(crate) fn transform_ocr_response(
model: &str,
response: MistralOcrResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
Ok(LiteLLMOcrResponse {
pages: response.pages,
model: response.model.unwrap_or_else(|| model.to_string()),
@ -56,7 +55,7 @@ impl MistralOCRConfig {
model: &str,
document: OcrDocument,
params: &OpaqueParams,
) -> Result<MistralOcrRequest, OcrRequestError> {
) -> Result<MistralOcrRequest, Error> {
Ok(MistralOcrRequest {
model: model.to_string(),
document,
@ -90,7 +89,7 @@ impl BaseOcrConfig for MistralOCRConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let params = self.map_ocr_params(&request.model, &request.optional_params);
let headers = validate_environment(&request.connection, &credential_env)?;
let url = get_complete_url(request.connection.api_base.as_deref())?;
@ -102,12 +101,12 @@ impl BaseOcrConfig for MistralOCRConfig {
&self,
request: &LiteLLMOcrRequest,
response: MistralOcrResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
transform_ocr_response(&request.model, response)
}
}
pub(crate) fn get_complete_url(api_base: Option<&str>) -> Result<String, OcrError> {
pub(crate) fn get_complete_url(api_base: Option<&str>) -> Result<String, Error> {
let base = api_base
.map(str::trim)
.filter(|base| !base.is_empty())
@ -115,18 +114,15 @@ pub(crate) fn get_complete_url(api_base: Option<&str>) -> Result<String, OcrErro
ApiUrl::parse(base)
.and_then(|url| url.complete_path(&["v1", "ocr"]))
.map(|url| url.into_string())
.map_err(|_| {
OcrRequestError::RequestField {
path: "api_base".into(),
}
.into()
.map_err(|_| Error::RequestField {
path: "api_base".into(),
})
}
fn validate_environment(
connection: &OcrConnection,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, OcrError> {
) -> Result<Vec<(String, String)>, Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization") {
return Ok(connection.extra_headers.clone());
}
@ -137,8 +133,9 @@ fn validate_environment(
.filter(|key| !key.is_empty())
.map(str::to_string)
.or_else(|| env_lookup(MISTRAL_API_KEY_ENV).filter(|key| !key.trim().is_empty()))
.ok_or(Error::MissingApiKey {
.ok_or(litellm_auth::Error::MissingApiKey {
provider: "Mistral",
environment_variable: MISTRAL_API_KEY_ENV,
})?;
Ok(
std::iter::once(("Authorization".into(), format!("Bearer {api_key}")))
@ -433,8 +430,9 @@ mod tests {
fn environment_rejects_missing_key() {
assert!(matches!(
validate_environment(&OcrConnection::default(), &|_| None),
Err(OcrError::Public(Error::MissingApiKey {
Err(Error::Auth(litellm_auth::Error::MissingApiKey {
provider: "Mistral",
environment_variable: MISTRAL_API_KEY_ENV,
}))
));
}

46
litellm-rust/crates/core/src/llms/reducto/ocr/transformation.rs
View file

@ -3,12 +3,11 @@ use std::collections::BTreeMap;
use serde::{Deserialize, Deserializer, Serialize};
use serde_json::{Map, Value, json};
use crate::Error;
use crate::constants::{REDUCTO_API_BASE, REDUCTO_API_KEY_ENV, REDUCTO_ID_PREFIX};
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::document::InlineDocument;
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{
_prepare_ocr_request, ParsedProviderParams, build_http_request, credential_env,
guardrail_document, merge_extra_params,
@ -152,7 +151,7 @@ fn transform_v3_ocr_request(
_model: &str,
document: OcrDocument,
params: &ReductoV3Params,
) -> Result<ReductoV3Request, OcrRequestError> {
) -> Result<ReductoV3Request, Error> {
Ok(ReductoV3Request {
input: document.source().to_string(),
params: params.clone(),
@ -169,7 +168,7 @@ fn transform_legacy_ocr_request(
_model: &str,
document: OcrDocument,
params: &ReductoLegacyParams,
) -> Result<ReductoLegacyRequest, OcrRequestError> {
) -> Result<ReductoLegacyRequest, Error> {
Ok(ReductoLegacyRequest {
document_url: document.source().to_string(),
options: params.enhance.as_ref().map(|_| params.clone()),
@ -179,7 +178,7 @@ fn transform_legacy_ocr_request(
pub(crate) fn transform_ocr_response(
model: &str,
response: ReductoResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
let result = match response.result {
Some(result) => result.unwrap_or_default(),
None => ReductoResult {
@ -249,7 +248,7 @@ fn page(index: i64, markdown: String, blocks: Option<Value>) -> Value {
}
result
}
fn get_complete_url(api_base: Option<&str>, path: &str) -> Result<String, OcrError> {
fn get_complete_url(api_base: Option<&str>, path: &str) -> Result<String, Error> {
let base = api_base
.map(str::trim)
.filter(|base| !base.is_empty())
@ -257,18 +256,15 @@ fn get_complete_url(api_base: Option<&str>, path: &str) -> Result<String, OcrErr
ApiUrl::parse(base)
.and_then(|url| url.complete_path(&[path]))
.map(|url| url.into_string())
.map_err(|_| {
OcrRequestError::RequestField {
path: "api_base".into(),
}
.into()
.map_err(|_| Error::RequestField {
path: "api_base".into(),
})
}
fn validate_environment(
connection: &OcrConnection,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, OcrError> {
) -> Result<Vec<(String, String)>, Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization") {
return Ok(connection.extra_headers.clone());
}
@ -296,26 +292,25 @@ async fn prepare_document(
document: OcrDocument,
connection: &OcrConnection,
headers: &[(String, String)],
) -> Result<OcrDocument, OcrError> {
) -> Result<OcrDocument, Error> {
if document.source().starts_with(REDUCTO_ID_PREFIX) {
if document.source()[REDUCTO_ID_PREFIX.len()..]
.trim()
.is_empty()
{
return Err(OcrRequestError::RequestField {
return Err(Error::RequestField {
path: "document file id".into(),
}
.into());
});
}
return Ok(document);
}
let inline = InlineDocument::parse(document.source())?.ok_or(OcrRequestError::ReductoSource)?;
let inline = InlineDocument::parse(document.source())?.ok_or(Error::ReductoSource)?;
let mime = inline.mime_type().to_string();
let bytes = inline.decode(crate::constants::OCR_INLINE_MAX_BYTES)?;
let part = reqwest::multipart::Part::bytes(bytes)
.file_name("document")
.mime_str(&mime)
.map_err(|_| OcrRequestError::InvalidDataUri)?;
.map_err(|_| Error::InvalidDataUri)?;
let builder = client
.provider_http()
.post(get_complete_url(connection.api_base.as_deref(), "upload")?)
@ -328,7 +323,7 @@ async fn prepare_document(
);
let response = crate::http_utils::http_request(builder)
.await
.map_err(crate::error::TransportError::from)?;
.map_err(crate::transport::Error::from)?;
let uploaded = crate::ocr::client::read_json_response::<ReductoUploadResponse>(
response,
false,
@ -342,10 +337,9 @@ async fn prepare_document(
.map(str::trim)
.filter(|id| !id.is_empty());
let Some(file_id) = file_id else {
return Err(OcrResponseError::ResponseField {
return Err(Error::ResponseField {
path: "file_id".into(),
}
.into());
});
};
Ok(document.with_source(file_id.to_string()))
}
@ -364,7 +358,7 @@ impl BaseOcrConfig for ReductoParseLegacyConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let ParsedProviderParams {
known: params,
extra_params,
@ -382,7 +376,7 @@ impl BaseOcrConfig for ReductoParseLegacyConfig {
&self,
request: &LiteLLMOcrRequest,
response: ReductoResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
transform_ocr_response(&request.model, response)
}
}
@ -400,7 +394,7 @@ impl BaseOcrConfig for ReductoParseV3Config {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let ParsedProviderParams {
known: params,
extra_params,
@ -418,7 +412,7 @@ impl BaseOcrConfig for ReductoParseV3Config {
&self,
request: &LiteLLMOcrRequest,
response: ReductoResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
transform_ocr_response(&request.model, response)
}
}

7
litellm-rust/crates/core/src/llms/vertex_ai/ocr/common_utils.rs
View file

@ -1,11 +1,10 @@
use crate::Error;
use crate::ocr::error::OcrError;
use crate::ocr::Error;
use crate::ocr::types::OcrConnection;
use litellm_auth::InputSource;
pub(super) fn validate_destination(connection: &OcrConnection) -> Result<(), OcrError> {
pub(super) fn validate_destination(connection: &OcrConnection) -> Result<(), Error> {
if connection.api_base.is_some() && connection.api_base_source == InputSource::Request {
return Err(Error::from(crate::AuthError::RequestVertexCredentialDestination).into());
return Err(litellm_auth::Error::RequestVertexCredentialDestination.into());
}
Ok(())
}

45
litellm-rust/crates/core/src/llms/vertex_ai/ocr/deepseek_transformation.rs
View file

@ -110,7 +110,7 @@ mod mapping {
use super::DeepSeekAi;
use super::types::*;
use crate::ocr::error::{OcrRequestError, OcrResponseError};
use crate::ocr::Error;
use crate::ocr::types::{LiteLLMOcrResponse, OcrDocument};
use crate::providers::model::ProviderModel;
@ -119,9 +119,9 @@ mod mapping {
provider_model: ProviderModel<DeepSeekAi>,
document: OcrDocument,
params: &DeepSeekOcrParams,
) -> Result<DeepSeekOcrRequest, OcrRequestError> {
) -> Result<DeepSeekOcrRequest, Error> {
if document.source().is_empty() {
return Err(OcrRequestError::MissingDocumentUrl);
return Err(Error::MissingDocumentUrl);
}
let content = OcrDocument::ImageUrl {
image_url: document.source().to_string(),
@ -141,13 +141,13 @@ mod mapping {
pub(crate) fn transform_ocr_response(
model: &str,
response: DeepSeekOcrResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
let content = response
.choices
.into_iter()
.next()
.and_then(|choice| choice.message.content)
.ok_or(OcrResponseError::EmptyContent)?;
.ok_or(Error::EmptyContent)?;
let decoded = decode_content(content)?;
let pages = match decoded.result.pages {
Some(pages) if !pages.is_empty() => pages
@ -176,18 +176,17 @@ mod mapping {
fallback_markdown: String,
}
fn decode_content(content: DeepSeekContent) -> Result<DecodedContent, OcrResponseError> {
fn decode_content(content: DeepSeekContent) -> Result<DecodedContent, Error> {
let (result, fallback_markdown) = match content {
DeepSeekContent::Text(text) if text.is_empty() => {
return Err(OcrResponseError::EmptyContent);
return Err(Error::EmptyContent);
}
DeepSeekContent::Text(text) => (decode_json_content(&text)?, text),
DeepSeekContent::Object(object) => {
let fallback = serde_json::to_string(&object).map_err(|_| {
OcrResponseError::ResponseField {
let fallback =
serde_json::to_string(&object).map_err(|_| Error::ResponseField {
path: "choices[0].message.content".into(),
}
})?;
})?;
(Some(object), fallback)
}
};
@ -197,7 +196,7 @@ mod mapping {
})
}
fn decode_json_content(text: &str) -> Result<Option<DeepSeekOcrResult>, OcrResponseError> {
fn decode_json_content(text: &str) -> Result<Option<DeepSeekOcrResult>, Error> {
if !text.trim_start().starts_with('{') {
return Ok(None);
}
@ -207,7 +206,7 @@ mod mapping {
};
serde_path_to_error::deserialize(value.into_deserializer())
.map(Some)
.map_err(|error| OcrResponseError::ResponseField {
.map_err(|error| Error::ResponseField {
path: format!("choices[0].message.content.{}", error.path()),
})
}
@ -217,10 +216,9 @@ mod mapping {
pub(crate) use mapping::{transform_ocr_request, transform_ocr_response};
use super::common_utils::validate_destination;
use crate::Error;
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{
_prepare_ocr_request, ParsedProviderParams, credential_env, transform_request_body,
};
@ -253,7 +251,7 @@ impl BaseOcrConfig for VertexAIDeepSeekOCRConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
validate_destination(&request.connection)?;
let ParsedProviderParams {
known: params,
@ -300,15 +298,15 @@ impl BaseOcrConfig for VertexAIDeepSeekOCRConfig {
&self,
request: &LiteLLMOcrRequest,
response: DeepSeekOcrResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
mapping::transform_ocr_response(&request.model, response)
}
}
pub(crate) fn provider_model(model: &str) -> Result<ProviderModel<DeepSeekAi>, OcrRequestError> {
pub(crate) fn provider_model(model: &str) -> Result<ProviderModel<DeepSeekAi>, Error> {
RoutedModel::new(model)
.and_then(RoutedModel::into_provider::<DeepSeekAi>)
.map_err(|_| OcrRequestError::RequestField {
.map_err(|_| Error::RequestField {
path: "model".into(),
})
}
@ -317,7 +315,7 @@ fn get_complete_url(
api_base: Option<&str>,
project: &str,
location: &str,
) -> Result<String, OcrError> {
) -> Result<String, Error> {
let base = api_base
.map(str::trim)
.filter(|base| !base.is_empty())
@ -337,11 +335,8 @@ fn get_complete_url(
])
})
.map(|url| url.into_string())
.map_err(|_| {
OcrRequestError::RequestField {
path: "api_base".into(),
}
.into()
.map_err(|_| Error::RequestField {
path: "api_base".into(),
})
}

23
litellm-rust/crates/core/src/llms/vertex_ai/ocr/transformation.rs
View file

@ -1,11 +1,10 @@
use super::common_utils::validate_destination;
use crate::Error;
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::llms::mistral::ocr::MistralOcrResponse;
use crate::llms::mistral::ocr::transformation::MistralOCRConfig;
use crate::ocr::Error;
use crate::ocr::OcrClient;
use crate::ocr::document::{inline_remote_document, validate_inline_document};
use crate::ocr::error::{OcrError, OcrRequestError, OcrResponseError};
use crate::ocr::prepare::{credential_env, transform_request_body};
use crate::ocr::types::{LiteLLMOcrRequest, LiteLLMOcrResponse};
use crate::url_utils::ApiUrl;
@ -26,7 +25,7 @@ impl BaseOcrConfig for VertexAIOCRConfig {
&self,
request: &LiteLLMOcrRequest,
client: &OcrClient,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
validate_destination(&request.connection)?;
let params = self.map_ocr_params(&request.model, &request.optional_params);
let config = VertexConfig::from_sourced_optional_params(
@ -77,7 +76,7 @@ impl BaseOcrConfig for VertexAIOCRConfig {
&self,
request: &LiteLLMOcrRequest,
response: MistralOcrResponse,
) -> Result<LiteLLMOcrResponse, OcrResponseError> {
) -> Result<LiteLLMOcrResponse, Error> {
MistralOCRConfig.transform_ocr_response(request, response)
}
}
@ -87,7 +86,7 @@ fn get_complete_url(
project: &str,
location: &str,
model: &str,
) -> Result<String, OcrError> {
) -> Result<String, Error> {
validate_location(location)?;
let default_base = format!("https://{location}-aiplatform.googleapis.com");
let base = api_base
@ -110,15 +109,12 @@ fn get_complete_url(
])
})
.map(|url| url.into_string())
.map_err(|_| {
OcrRequestError::RequestField {
path: "api_base".into(),
}
.into()
.map_err(|_| Error::RequestField {
path: "api_base".into(),
})
}
fn validate_location(location: &str) -> Result<(), OcrError> {
fn validate_location(location: &str) -> Result<(), Error> {
let valid = !location.is_empty()
&& location
.bytes()
@ -134,10 +130,9 @@ fn validate_location(location: &str) -> Result<(), OcrError> {
if valid {
return Ok(());
}
Err(OcrRequestError::RequestField {
Err(Error::RequestField {
path: "vertex_location".into(),
}
.into())
})
}
#[cfg(test)]

72
litellm-rust/crates/core/src/media.rs
View file

@ -9,7 +9,29 @@ use reqwest::Url;
use reqwest::dns::{Addrs, Name, Resolve, Resolving};
use crate::constants::MEDIA_CONNECT_TIMEOUT_SECS;
use crate::error::{MediaError, TransportError};
use crate::transport::Error as TransportError;
#[derive(Debug, thiserror::Error)]
pub(crate) enum Error {
#[error("media URL rejected by network policy")]
BlockedUrl,
#[error("media download is disabled")]
DownloadDisabled,
#[error("media download exceeds the maximum size")]
DownloadTooLarge,
#[error("too many redirects while fetching media")]
TooManyRedirects,
#[error("media redirect is missing a Location header")]
MissingRedirectLocation,
#[error("invalid media redirect")]
InvalidRedirect,
#[error("media download failed with status {0}")]
Http(u16),
#[error("media download timed out")]
Timeout,
#[error("{0}")]
Transport(#[from] TransportError),
}
#[derive(Clone)]
pub(crate) struct MediaFetcher {
@ -75,20 +97,20 @@ impl MediaFetcher {
&self,
url: Url,
policy: DownloadPolicy,
) -> Result<DownloadedMedia, MediaError> {
) -> Result<DownloadedMedia, Error> {
if policy.max_bytes == 0 {
return Err(MediaError::DownloadDisabled);
return Err(Error::DownloadDisabled);
}
tokio::time::timeout(policy.timeout, self.fetch_before_deadline(url, policy))
.await
.map_err(|_| MediaError::Timeout)?
.map_err(|_| Error::Timeout)?
}
async fn fetch_before_deadline(
&self,
mut url: Url,
policy: DownloadPolicy,
) -> Result<DownloadedMedia, MediaError> {
) -> Result<DownloadedMedia, Error> {
let mut redirects_followed = 0;
loop {
self.validate_url(&url).await?;
@ -100,21 +122,19 @@ impl MediaFetcher {
.map_err(TransportError::from)?;
if response.status().is_redirection() {
if redirects_followed == policy.max_redirects {
return Err(MediaError::TooManyRedirects);
return Err(Error::TooManyRedirects);
}
let location = response
.headers()
.get(reqwest::header::LOCATION)
.and_then(|value| value.to_str().ok())
.ok_or(MediaError::MissingRedirectLocation)?;
url = url
.join(location)
.map_err(|_| MediaError::InvalidRedirect)?;
.ok_or(Error::MissingRedirectLocation)?;
url = url.join(location).map_err(|_| Error::InvalidRedirect)?;
redirects_followed += 1;
continue;
}
if !response.status().is_success() {
return Err(MediaError::Http(response.status().as_u16()));
return Err(Error::Http(response.status().as_u16()));
}
enforce_download_size(response.content_length().unwrap_or(0), policy.max_bytes)?;
let content_type = response
@ -138,23 +158,21 @@ impl MediaFetcher {
}
}
async fn validate_url(&self, url: &Url) -> Result<(), MediaError> {
async fn validate_url(&self, url: &Url) -> Result<(), Error> {
if !matches!(url.scheme(), "http" | "https")
|| !url.username().is_empty()
|| url.password().is_some()
{
return Err(MediaError::BlockedUrl);
return Err(Error::BlockedUrl);
}
let host = url.host_str().ok_or(MediaError::BlockedUrl)?;
let host = url.host_str().ok_or(Error::BlockedUrl)?;
if self.allow_private_network {
return Ok(());
}
if let Ok(ip) = host.parse::<IpAddr>() {
return (!is_blocked_ip(ip))
.then_some(())
.ok_or(MediaError::BlockedUrl);
return (!is_blocked_ip(ip)).then_some(()).ok_or(Error::BlockedUrl);
}
let port = url.port_or_known_default().ok_or(MediaError::BlockedUrl)?;
let port = url.port_or_known_default().ok_or(Error::BlockedUrl)?;
let addresses = self
.address_resolver
.resolve(host, port)
@ -164,16 +182,16 @@ impl MediaFetcher {
}
}
fn enforce_download_size(length: u64, max_bytes: u64) -> Result<(), MediaError> {
fn enforce_download_size(length: u64, max_bytes: u64) -> Result<(), Error> {
if length > max_bytes {
return Err(MediaError::DownloadTooLarge);
return Err(Error::DownloadTooLarge);
}
Ok(())
}
fn validate_addresses(addresses: &[SocketAddr]) -> Result<(), MediaError> {
fn validate_addresses(addresses: &[SocketAddr]) -> Result<(), Error> {
if addresses.is_empty() || addresses.iter().any(|address| is_blocked_ip(address.ip())) {
return Err(MediaError::BlockedUrl);
return Err(Error::BlockedUrl);
}
Ok(())
}
@ -415,7 +433,7 @@ mod tests {
.await
.expect_err("oversize body is rejected");
server.await.expect("server completes");
assert!(matches!(error, MediaError::DownloadTooLarge));
assert!(matches!(error, Error::DownloadTooLarge));
}
#[tokio::test]
@ -433,7 +451,7 @@ mod tests {
.await
.expect_err("stream crossing limit is rejected");
server.await.expect("server completes");
assert!(matches!(error, MediaError::DownloadTooLarge));
assert!(matches!(error, Error::DownloadTooLarge));
}
#[tokio::test]
@ -469,7 +487,7 @@ mod tests {
.expect_err("private redirect is rejected");
let requests = server.await.expect("server completes");
assert_eq!(requests.len(), 1);
assert!(matches!(error, MediaError::BlockedUrl));
assert!(matches!(error, Error::BlockedUrl));
}
#[tokio::test]
@ -496,7 +514,7 @@ mod tests {
.await
.expect_err("fetch times out");
server.await.expect("server completes");
assert!(matches!(error, MediaError::Timeout));
assert!(matches!(error, Error::Timeout));
}
#[tokio::test]
@ -522,7 +540,7 @@ mod tests {
Url::parse("https://user:password@8.8.8.8/document").expect("credentialed URL parses");
assert!(matches!(
fetcher.validate_url(&url).await,
Err(MediaError::BlockedUrl)
Err(Error::BlockedUrl)
));
}
}

4
litellm-rust/crates/core/src/messages/common_utils.rs
View file

@ -1,5 +1,5 @@
use crate::Error;
use crate::http_utils::string_headers as shared_string_headers;
use crate::messages::Error;
use crate::providers::anthropic::messages::transformation::ANTHROPIC_MESSAGES_CONFIG;
use crate::providers::azure_ai::messages::transformation::AZURE_ANTHROPIC_MESSAGES_CONFIG;
use serde_json::{Map, Value};
@ -23,5 +23,5 @@ pub(super) fn messages_provider_config(
pub(super) fn string_headers(
extra_headers: Option<Map<String, Value>>,
) -> Result<Vec<(String, String)>, Error> {
shared_string_headers(HEADER_CONTEXT, extra_headers)
shared_string_headers(HEADER_CONTEXT, extra_headers).map_err(Error::from)
}

19
litellm-rust/crates/core/src/messages/error.rs Normal file
View file

@ -0,0 +1,19 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error("routing error: {0}")]
Routing(String),
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Transport(#[from] crate::transport::Error),
#[error(transparent)]
Params(#[from] crate::params::Error),
#[error(transparent)]
Headers(#[from] crate::http_utils::HeaderError),
}

18
litellm-rust/crates/core/src/messages/handler.rs
View file

@ -1,6 +1,6 @@
use crate::constants::ANTHROPIC_MESSAGES_PROVIDER;
use crate::error::Error;
use crate::http_utils::http_request;
use crate::messages::Error;
use super::client::http_client;
use super::common_utils::truncate_error_body;
@ -21,19 +21,19 @@ pub(super) async fn execute_messages_provider_call(
let response = http_request(request_builder)
.await
.map_err(|err| Error::Network(err.to_string()))?;
.map_err(|err| Error::Transport(crate::transport::Error::Network(err.to_string())))?;
let status = response.status();
let text = response
.text()
.await
.map_err(|err| Error::Network(err.to_string()))?;
.map_err(|err| Error::Transport(crate::transport::Error::Network(err.to_string())))?;
if !status.is_success() {
return Err(Error::Http {
return Err(Error::Transport(crate::transport::Error::Http {
status: status.as_u16(),
body: truncate_error_body(&text),
});
}));
}
let response = serde_json::from_str(&text)
@ -61,17 +61,17 @@ pub(super) async fn execute_messages_provider_stream(
let response = http_request(request_builder)
.await
.map_err(|err| Error::Network(err.to_string()))?;
.map_err(|err| Error::Transport(crate::transport::Error::Network(err.to_string())))?;
let status = response.status();
if !status.is_success() {
let text = response
.text()
.await
.map_err(|err| Error::Network(err.to_string()))?;
return Err(Error::Http {
.map_err(|err| Error::Transport(crate::transport::Error::Network(err.to_string())))?;
return Err(Error::Transport(crate::transport::Error::Http {
status: status.as_u16(),
body: truncate_error_body(&text),
});
}));
}
Ok(response)
}

3
litellm-rust/crates/core/src/messages/mod.rs
View file

@ -7,7 +7,8 @@
//! is the streaming variant; it hands the raw upstream response back so a host
//! can splice the event stream to its own caller.
use crate::Error;
mod error;
pub use error::Error;
mod client;
mod common_utils;
mod handler;

4
litellm-rust/crates/core/src/messages/prepare.rs
View file

@ -1,5 +1,5 @@
use crate::error::Error;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};
use crate::messages::Error;
use crate::routing_utils::provider::{CustomLlmProvider, get_custom_llm_provider};
use super::common_utils::{has_bearer_auth, has_header, messages_provider_config, string_headers};
use super::transformation::{AnthropicMessagesProviderConfig, MessagesAuthStrategy};

16
litellm-rust/crates/core/src/messages/tests.rs
View file

@ -4,7 +4,7 @@ use serde_json::{Map, Value, json};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use crate::error::Error;
use crate::messages::Error;
use super::common_utils::{
has_bearer_auth, has_header, messages_provider_config, string_headers, truncate_error_body,
@ -77,7 +77,14 @@ fn truncate_error_body_caps_long_payloads() {
fn string_headers_rejects_non_string_values() {
let headers = json!({"x-count": 3}).as_object().unwrap().clone();
let err = string_headers(Some(headers)).expect_err("non-string header rejected");
assert!(matches!(err, Error::InvalidRequest(_)));
assert_eq!(
err,
Error::Headers(crate::http_utils::HeaderError {
context: "messages",
name: "x-count".to_string(),
actual: "number",
})
);
}
#[test]
@ -431,7 +438,10 @@ async fn messages_maps_provider_error_status_to_http_error() {
.await
.expect_err("provider error propagates");
assert!(matches!(err, Error::Http { status: 401, .. }));
assert!(matches!(
err,
Error::Transport(crate::transport::Error::Http { status: 401, .. })
));
}
#[tokio::test]

2
litellm-rust/crates/core/src/messages/transformation.rs
View file

@ -1,5 +1,5 @@
use super::types::{AnthropicMessagesRequest, AnthropicMessagesResponse};
use crate::Error;
use crate::messages::Error;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MessagesAuthStrategy {

25
litellm-rust/crates/core/src/ocr/client.rs
View file

@ -4,13 +4,12 @@ use std::time::Duration;
use bytes::{Bytes, BytesMut};
use serde::de::DeserializeOwned;
use super::error::{OcrError, OcrResponseError};
use super::Error;
use super::types::{LiteLLMOcrRequest, LiteLLMOcrResponse};
use super::wire::{DecodedOcrResponse, decode_response};
use crate::Error;
use crate::constants::OCR_CONNECT_TIMEOUT_SECS;
use crate::error::TransportError;
use crate::media::MediaFetcher;
use crate::transport::Error as TransportError;
use litellm_auth_gcp::VertexAuth;
#[derive(Clone)]
@ -124,15 +123,15 @@ pub async fn read_json_response<T: DeserializeOwned>(
response: reqwest::Response,
native: bool,
max_response_bytes: usize,
) -> Result<DecodedOcrResponse<T>, OcrError> {
) -> Result<DecodedOcrResponse<T>, Error> {
let bytes = read_response_bytes(response, max_response_bytes).await?;
Ok(decode_response(&bytes, native)?)
decode_response(&bytes, native)
}
pub(crate) async fn read_response_bytes(
mut response: reqwest::Response,
max_response_bytes: usize,
) -> Result<Bytes, OcrError> {
) -> Result<Bytes, Error> {
let status = response.status();
let limit = if status.is_success() {
max_response_bytes
@ -144,13 +143,13 @@ pub(crate) async fn read_response_bytes(
.content_length()
.is_some_and(|length| length > limit as u64)
{
return Err(OcrResponseError::TooLarge { limit }.into());
return Err(Error::TooLarge { limit });
}
let mut bytes = BytesMut::new();
while let Some(chunk) = response.chunk().await.map_err(transport_error)? {
let remaining = limit.saturating_sub(bytes.len());
if status.is_success() && chunk.len() > remaining {
return Err(OcrResponseError::TooLarge { limit }.into());
return Err(Error::TooLarge { limit });
}
bytes.extend_from_slice(&chunk[..chunk.len().min(remaining)]);
if !status.is_success() && bytes.len() == limit {
@ -158,7 +157,7 @@ pub(crate) async fn read_response_bytes(
}
}
if !status.is_success() {
return Err(crate::error::TransportError::Http {
return Err(crate::transport::Error::Http {
status: status.as_u16(),
body: crate::http_utils::truncate_error_body(&String::from_utf8_lossy(&bytes)),
}
@ -169,12 +168,12 @@ pub(crate) async fn read_response_bytes(
pub(crate) fn transport_error(error: reqwest::Error) -> Error {
if error.is_timeout() {
return Error::Http {
return Error::Transport(crate::transport::Error::Http {
status: 408,
body: "OCR request timed out".into(),
};
});
}
crate::error::TransportError::from(error).into()
crate::transport::Error::from(error).into()
}
#[cfg(test)]
@ -197,7 +196,7 @@ mod tests {
.unwrap_err();
assert!(matches!(
transport_error(error),
Error::Http { status: 408, .. }
Error::Transport(crate::transport::Error::Http { status: 408, .. })
));
server.abort();
}

50
litellm-rust/crates/core/src/ocr/document.rs
View file

@ -4,27 +4,28 @@ use data_url::{DataUrl, DataUrlError, forgiving_base64::DecodeError};
use reqwest::Url;
use serde_json::Map;
use super::error::{OcrError, OcrRequestError, OcrResponseError};
use super::Error;
use super::types::{OcrConnection, OcrDocument};
use crate::constants::{OCR_INLINE_MAX_BYTES, OCR_MAX_FETCH_REDIRECTS};
use crate::error::{MediaError, TransportError};
use crate::media::Error as MediaError;
use crate::media::{DownloadPolicy, MediaFetcher};
use crate::transport::Error as TransportError;
pub fn encode_file_document(
bytes: &[u8],
file_name: Option<&str>,
mime_type: Option<&str>,
) -> Result<OcrDocument, OcrRequestError> {
) -> Result<OcrDocument, Error> {
if bytes.is_empty() {
return Err(OcrRequestError::EmptyFile);
return Err(Error::EmptyFile);
}
if bytes.len() > OCR_INLINE_MAX_BYTES {
return Err(OcrRequestError::InlineDocumentTooLarge);
return Err(Error::InlineDocumentTooLarge);
}
if let Some(value) = mime_type
&& !valid_mime_type(value)
{
return Err(OcrRequestError::InvalidMimeType(value.into()));
return Err(Error::InvalidMimeType(value.into()));
}
let mime_type = mime_type
.map(str::to_string)
@ -89,11 +90,11 @@ pub fn upload_mime_type<'a>(file_name: Option<&str>, content_type: Option<&'a st
pub(crate) struct InlineDocument<'a>(DataUrl<'a>);
impl<'a> InlineDocument<'a> {
pub(crate) fn parse(source: &'a str) -> Result<Option<Self>, OcrRequestError> {
pub(crate) fn parse(source: &'a str) -> Result<Option<Self>, Error> {
match DataUrl::process(source) {
Ok(url) => Ok(Some(Self(url))),
Err(DataUrlError::NotADataUrl) => Ok(None),
Err(DataUrlError::NoComma) => Err(OcrRequestError::InvalidDataUri),
Err(DataUrlError::NoComma) => Err(Error::InvalidDataUri),
}
}
@ -101,27 +102,26 @@ impl<'a> InlineDocument<'a> {
self.0.mime_type()
}
pub(crate) fn decode(&self, max_bytes: usize) -> Result<Vec<u8>, OcrRequestError> {
pub(crate) fn decode(&self, max_bytes: usize) -> Result<Vec<u8>, Error> {
let mut body = Vec::new();
self.0
.decode(|bytes| {
if bytes.len() > max_bytes.saturating_sub(body.len()) {
return Err(OcrRequestError::InlineDocumentTooLarge);
return Err(Error::InlineDocumentTooLarge);
}
body.extend_from_slice(bytes);
Ok(())
})
.map_err(|error| match error {
DecodeError::InvalidBase64(_) => OcrRequestError::InvalidDataUri,
DecodeError::InvalidBase64(_) => Error::InvalidDataUri,
DecodeError::WriteError(error) => error,
})?;
Ok(body)
}
}
pub(crate) fn validate_inline_document(document: &OcrDocument) -> Result<(), OcrRequestError> {
let inline =
InlineDocument::parse(document.source())?.ok_or(OcrRequestError::InvalidDataUri)?;
pub(crate) fn validate_inline_document(document: &OcrDocument) -> Result<(), Error> {
let inline = InlineDocument::parse(document.source())?.ok_or(Error::InvalidDataUri)?;
inline.decode(crate::constants::OCR_INLINE_MAX_BYTES)?;
Ok(())
}
@ -130,13 +130,13 @@ pub(crate) async fn inline_remote_document(
fetcher: &MediaFetcher,
document: OcrDocument,
connection: &OcrConnection,
) -> Result<OcrDocument, OcrError> {
) -> Result<OcrDocument, Error> {
let source = document.source();
if !source.starts_with("http://") && !source.starts_with("https://") {
validate_inline_document(&document)?;
return Ok(document);
}
let url = Url::parse(source).map_err(|_| OcrRequestError::RequestField {
let url = Url::parse(source).map_err(|_| Error::RequestField {
path: "document URL".into(),
})?;
let downloaded = fetcher
@ -159,14 +159,14 @@ pub(crate) async fn inline_remote_document(
Ok(result)
}
fn map_media_error(error: MediaError) -> OcrError {
fn map_media_error(error: MediaError) -> Error {
match error {
MediaError::BlockedUrl => OcrRequestError::BlockedDocumentUrl.into(),
MediaError::DownloadDisabled => OcrRequestError::DownloadDisabled.into(),
MediaError::DownloadTooLarge => OcrRequestError::DownloadTooLarge.into(),
MediaError::TooManyRedirects => OcrRequestError::TooManyRedirects.into(),
MediaError::MissingRedirectLocation => OcrResponseError::MissingRedirectLocation.into(),
MediaError::InvalidRedirect => OcrResponseError::InvalidRedirect.into(),
MediaError::BlockedUrl => Error::BlockedDocumentUrl,
MediaError::DownloadDisabled => Error::DownloadDisabled,
MediaError::DownloadTooLarge => Error::DownloadTooLarge,
MediaError::TooManyRedirects => Error::TooManyRedirects,
MediaError::MissingRedirectLocation => Error::MissingRedirectLocation,
MediaError::InvalidRedirect => Error::InvalidRedirect,
MediaError::Http(status) => TransportError::Http {
status,
body: "OCR document download failed".into(),
@ -254,7 +254,7 @@ mod tests {
let bytes = vec![b'a'; OCR_INLINE_MAX_BYTES + 1];
assert_eq!(
encode_file_document(&bytes, None, None),
Err(OcrRequestError::InlineDocumentTooLarge)
Err(Error::InlineDocumentTooLarge)
);
let document = encode_file_document(&bytes[..OCR_INLINE_MAX_BYTES], None, None).unwrap();
let inline = InlineDocument::parse(document.source()).unwrap().unwrap();
@ -288,7 +288,7 @@ mod tests {
assert_eq!(inline.decode(expected.len()).unwrap(), expected);
assert_eq!(
inline.decode(expected.len() - 1),
Err(OcrRequestError::InlineDocumentTooLarge)
Err(Error::InlineDocumentTooLarge)
);
}
}

107
litellm-rust/crates/core/src/ocr/error.rs
View file

@ -1,9 +1,5 @@
use thiserror::Error;
use crate::error::TransportError;
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum OcrRequestError {
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("File is empty or could not be read")]
EmptyFile,
#[error("Invalid MIME type: {0}")]
@ -42,10 +38,6 @@ pub enum OcrRequestError {
Features,
#[error("OCR model cannot be a dot segment")]
DotModel,
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum OcrResponseError {
#[error("OCR response exceeds the size limit of {limit} bytes")]
TooLarge { limit: usize },
#[error("invalid OCR response field: {path}")]
@ -60,40 +52,83 @@ pub enum OcrResponseError {
OperationStatus(String),
#[error("OCR response numeric value is out of range: {0}")]
NumericRange(&'static str),
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum OcrPollingError {
#[error("OCR accepted response is missing a valid operation-location")]
PollLocation,
#[error("OCR operation-location must use the submission origin without credentials")]
PollOrigin,
#[error("OCR polling timed out")]
PollTimeout,
#[error("unsupported by the rust path: {0}")]
Unsupported(&'static str),
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error(
"invalid authentication configuration: Missing Azure AI credentials - set AZURE_AI_API_KEY or configure Entra ID"
)]
MissingAzureAiCredentials,
#[error(
"invalid authentication configuration: Missing Azure Document Intelligence credentials - set AZURE_DOCUMENT_INTELLIGENCE_API_KEY or configure Entra ID"
)]
MissingAzureDocumentIntelligenceCredentials,
#[error(
"Missing REDUCTO_API_KEY - set it in the environment or pass api_key to litellm.ocr()/litellm.aocr()"
)]
MissingReductoApiKey,
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Transport(#[from] crate::transport::Error),
#[error(transparent)]
Params(#[from] crate::params::Error),
#[error(transparent)]
Headers(#[from] crate::http_utils::HeaderError),
}
#[derive(Debug, Error)]
pub enum OcrError {
#[error("{0}")]
Request(#[from] OcrRequestError),
#[error("{0}")]
Response(#[from] OcrResponseError),
#[error("{0}")]
Transport(#[from] TransportError),
#[error("{0}")]
Polling(#[from] OcrPollingError),
#[error("{0}")]
Public(#[from] crate::Error),
}
impl Error {
pub fn is_request(&self) -> bool {
matches!(
self,
Self::EmptyFile
| Self::InvalidMimeType(_)
| Self::CohereImageOnly
| Self::RequestFormat
| Self::RequestField { .. }
| Self::MissingField(_)
| Self::MissingDocumentUrl
| Self::InvalidDataUri
| Self::ReductoSource
| Self::InlineDocumentTooLarge
| Self::BlockedDocumentUrl
| Self::DownloadDisabled
| Self::DownloadTooLarge
| Self::TooManyRedirects
| Self::Pages(_)
| Self::Features
| Self::DotModel
| Self::InvalidRequest(_)
| Self::Params(_)
| Self::Headers(_)
)
}
impl From<OcrError> for crate::Error {
fn from(error: OcrError) -> Self {
match error {
OcrError::Request(error) => error.into(),
OcrError::Response(error) => error.into(),
OcrError::Transport(error) => error.into(),
OcrError::Polling(error) => crate::Error::InvalidResponse(error.to_string()),
OcrError::Public(error) => error,
}
pub fn is_response(&self) -> bool {
matches!(
self,
Self::TooLarge { .. }
| Self::ResponseField { .. }
| Self::EmptyContent
| Self::MissingRedirectLocation
| Self::InvalidRedirect
| Self::OperationStatus(_)
| Self::NumericRange(_)
| Self::PollLocation
| Self::PollOrigin
| Self::PollTimeout
| Self::InvalidResponse(_)
)
}
}

5
litellm-rust/crates/core/src/ocr/handler.rs
View file

@ -5,7 +5,6 @@ use super::hooks::{OcrHooks, OcrLifecycleHooks, OcrPostCallRequest};
use super::provider_config::OcrConfigKind;
use super::types::{LiteLLMOcrRequest, LiteLLMOcrResponse};
use super::wire::DecodedOcrResponse;
use crate::Error;
use crate::call_lifecycle::{CallLifecycle, CallLifecycleContext};
use crate::llms::azure_ai::ocr::cohere_parse_transformation::AzureAICohereParseConfig;
use crate::llms::azure_ai::ocr::document_intelligence::AzureDocumentIntelligenceOperation;
@ -22,6 +21,7 @@ use crate::llms::vertex_ai::ocr::deepseek_transformation::{
DeepSeekOcrResponse, VertexAIDeepSeekOCRConfig,
};
use crate::llms::vertex_ai::ocr::transformation::VertexAIOCRConfig;
use crate::ocr::Error;
pub(crate) async fn perform_ocr_request(
client: &OcrClient,
@ -170,10 +170,9 @@ fn request_headers(request: &reqwest::Request) -> Result<Vec<(String, String)>,
value
.to_str()
.map(|value| (name.to_string(), value.to_string()))
.map_err(|_| super::error::OcrRequestError::RequestField {
.map_err(|_| super::Error::RequestField {
path: "headers".into(),
})
.map_err(Error::from)
})
.collect()
}

8
litellm-rust/crates/core/src/ocr/hooks.rs
View file

@ -3,8 +3,8 @@ use std::pin::Pin;
use std::sync::Arc;
use super::types::{LiteLLMOcrRequest, LiteLLMOcrResponse, OcrDocument};
use crate::Error;
use crate::call_lifecycle::{CallLifecycleContext, CallLifecycleHooks, CallLifecycleTiming};
use crate::ocr::Error;
use serde::Serialize;
use serde_json::Value;
@ -80,6 +80,7 @@ pub(crate) struct OcrLifecycleHooks {
impl CallLifecycleHooks<LiteLLMOcrRequest, LiteLLMOcrRequest, LiteLLMOcrResponse>
for OcrLifecycleHooks
{
type Error = Error;
type PreCallFuture<'a> = OcrHookFuture<'a, LiteLLMOcrRequest>;
type DuringCallFuture<'a> = OcrHookFuture<'a, LiteLLMOcrRequest>;
type SuccessFuture<'a> = OcrLogFuture<'a>;
@ -104,10 +105,9 @@ impl CallLifecycleHooks<LiteLLMOcrRequest, LiteLLMOcrRequest, LiteLLMOcrResponse
})
.await?;
let Value::Object(optional_params) = changed.optional_params else {
return Err(super::error::OcrRequestError::RequestField {
return Err(super::Error::RequestField {
path: "guardrail.optional_params".into(),
}
.into());
});
};
Ok(LiteLLMOcrRequest {
document: changed.document,

19
litellm-rust/crates/core/src/ocr/lifecycle.rs
View file

@ -10,12 +10,12 @@ use super::hooks::{
OcrPreCallRequest,
};
use super::{LiteLLMOcrRequest, LiteLLMOcrResponse, OcrClient};
use crate::AuthError;
use crate::Error;
use crate::call_lifecycle::host::{
HostCall, HostCallFuture, HostCallStep, HostFailure, HostLifecycle, HostPhase,
};
use crate::call_lifecycle::{CallLifecycleContext, CallLifecycleTiming};
use crate::ocr::Error;
use litellm_auth::Error as AuthError;
use litellm_auth::{ResolvedCredential, TokenFuture, TokenProvider, TokenProviderHandle};
pub type NativeResult<T> = Result<NativeOutcome<T>, Error>;
@ -84,7 +84,7 @@ impl OcrHostOperation {
pub enum OcrHostResult {
Request(Result<(Box<LiteLLMOcrRequest>, bool), Error>),
Lifecycle(Result<(), HostFailure>),
Lifecycle(Result<(), HostFailure<Error>>),
AzureAdToken(Result<ResolvedCredential, AuthError>),
PreCall(Result<OcrPreCallRequest, Error>),
DuringCall(Result<OcrDuringCallRequest, Error>),
@ -256,7 +256,7 @@ impl OcrCall {
Ok(self.host_step(operation))
}
fn accept(&mut self, result: Result<(), HostFailure>) {
fn accept(&mut self, result: Result<(), HostFailure<Error>>) {
let cancelled = matches!(&result, Err(HostFailure::Cancelled(_)));
if let Some(error) = self.lifecycle.accept(result) {
if cancelled {
@ -268,7 +268,7 @@ impl OcrCall {
}
}
pub async fn interrupt(&mut self, failure: HostFailure) -> Result<OcrCallStep, Error> {
pub async fn interrupt(&mut self, failure: HostFailure<Error>) -> Result<OcrCallStep, Error> {
if self.completed {
return Err(Error::InvalidRequest(
"OCR call cannot be interrupted after completion".into(),
@ -286,6 +286,7 @@ impl OcrCall {
}
impl HostCall for OcrCall {
type Error = Error;
type Operation = OcrHostOperation;
type Result = OcrHostResult;
type Complete = LiteLLMOcrResponse;
@ -293,14 +294,14 @@ impl HostCall for OcrCall {
fn resume(
&mut self,
result: Option<Self::Result>,
) -> HostCallFuture<'_, Self::Operation, Self::Complete> {
) -> HostCallFuture<'_, Self::Operation, Self::Complete, Self::Error> {
Box::pin(OcrCall::resume(self, result))
}
fn interrupt(
&mut self,
failure: HostFailure,
) -> HostCallFuture<'_, Self::Operation, Self::Complete> {
failure: HostFailure<Error>,
) -> HostCallFuture<'_, Self::Operation, Self::Complete, Self::Error> {
Box::pin(OcrCall::interrupt(self, failure))
}
}
@ -376,7 +377,7 @@ impl OcrExecution {
self.execution = None;
self.completed = true;
result
.map_err(|error| Error::Network(format!("OCR execution task failed: {error}")))?
.map_err(|error| Error::Transport(crate::transport::Error::Network(format!("OCR execution task failed: {error}"))))?
.map(OcrCallStep::Complete)
}
}

3
litellm-rust/crates/core/src/ocr/mod.rs
View file

@ -1,6 +1,7 @@
mod error;
pub use error::Error;
pub mod client;
pub(crate) mod document;
pub mod error;
pub(crate) mod handler;
pub mod hooks;
mod lifecycle;

30
litellm-rust/crates/core/src/ocr/prepare.rs
View file

@ -1,8 +1,8 @@
use serde::{Serialize, de::DeserializeOwned};
use serde_json::Value;
use super::Error;
use super::OcrClient;
use super::error::{OcrError, OcrRequestError};
use super::hooks::OcrDuringCallRequest;
use super::types::{LiteLLMOcrRequest, OcrDocument};
@ -10,7 +10,7 @@ pub(crate) use crate::params::{ParsedProviderParams, merge_extra_params};
pub(crate) fn _prepare_ocr_request<T: DeserializeOwned>(
request: &LiteLLMOcrRequest,
) -> Result<ParsedProviderParams<T>, OcrRequestError> {
) -> Result<ParsedProviderParams<T>, Error> {
super::wire::decode_request_value(
Value::Object(request.optional_params.provider_params().into()),
"optional_params",
@ -24,8 +24,8 @@ pub(crate) async fn transform_request_body<B>(
headers: &[(String, String)],
retains_document: bool,
body: B,
validate: impl Fn(&B) -> Result<(), OcrRequestError>,
) -> Result<reqwest::Request, OcrError>
validate: impl Fn(&B) -> Result<(), Error>,
) -> Result<reqwest::Request, Error>
where
B: Serialize + DeserializeOwned,
{
@ -36,7 +36,7 @@ where
let composed = OcrWireBody::<B>::decode(composed, "body")?;
validate(&composed.body)?;
let (body, headers) = if request.hooks.intercepts_requests() {
let body = serde_json::to_value(composed).map_err(|_| OcrRequestError::RequestField {
let body = serde_json::to_value(composed).map_err(|_| Error::RequestField {
path: "body".into(),
})?;
let retained_fields = request
@ -79,7 +79,7 @@ pub(crate) fn build_http_request<B: Serialize>(
url: &str,
headers: &[(String, String)],
body: &B,
) -> Result<reqwest::Request, OcrError> {
) -> Result<reqwest::Request, Error> {
let builder = client
.provider_http()
.post(url)
@ -87,15 +87,15 @@ pub(crate) fn build_http_request<B: Serialize>(
.timeout(request.connection.timeout);
crate::http_utils::with_headers(builder, headers, crate::http_utils::HeaderPolicy::All)
.build()
.map_err(crate::error::TransportError::from)
.map_err(OcrError::from)
.map_err(crate::transport::Error::from)
.map_err(Error::from)
}
pub(crate) async fn guardrail_document(
request: &LiteLLMOcrRequest,
url: &str,
headers: &[(String, String)],
) -> Result<(OcrDocument, Vec<(String, String)>), OcrError> {
) -> Result<(OcrDocument, Vec<(String, String)>), Error> {
if !request.hooks.intercepts_requests() {
return Ok((request.document.clone(), headers.to_vec()));
}
@ -106,10 +106,8 @@ pub(crate) async fn guardrail_document(
custom_llm_provider: request.config.provider().as_str().into(),
url: url.into(),
headers: headers.to_vec(),
body: serde_json::to_value(&request.document).map_err(|_| {
OcrRequestError::RequestField {
path: "document".into(),
}
body: serde_json::to_value(&request.document).map_err(|_| Error::RequestField {
path: "document".into(),
})?,
retained_fields: Vec::new(),
})
@ -127,14 +125,14 @@ struct OcrWireBody<B> {
}
impl<B: Serialize + DeserializeOwned> OcrWireBody<B> {
fn decode(value: Value, prefix: &str) -> Result<Self, OcrRequestError> {
fn decode(value: Value, prefix: &str) -> Result<Self, Error> {
let body: B = super::wire::decode_request_value(value.clone(), prefix)?;
let Value::Object(fields) = value else {
return Err(OcrRequestError::RequestField {
return Err(Error::RequestField {
path: prefix.into(),
});
};
let known = serde_json::to_value(&body).map_err(|_| OcrRequestError::RequestField {
let known = serde_json::to_value(&body).map_err(|_| Error::RequestField {
path: prefix.into(),
})?;
let extra = fields

4
litellm-rust/crates/core/src/ocr/provider_config.rs
View file

@ -1,4 +1,3 @@
use crate::Error;
use crate::llms::azure_ai::ocr::cohere_parse_transformation::AzureAICohereParseConfig;
use crate::llms::azure_ai::ocr::document_intelligence::transformation::AzureDocumentIntelligenceOCRConfig;
use crate::llms::azure_ai::ocr::transformation::AzureAIOCRConfig;
@ -8,7 +7,8 @@ use crate::llms::mistral::ocr::transformation::MistralOCRConfig;
use crate::llms::reducto::ocr::transformation::{ReductoParseLegacyConfig, ReductoParseV3Config};
use crate::llms::vertex_ai::ocr::deepseek_transformation::VertexAIDeepSeekOCRConfig;
use crate::llms::vertex_ai::ocr::transformation::VertexAIOCRConfig;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};
use crate::ocr::Error;
use crate::routing_utils::provider::{CustomLlmProvider, get_custom_llm_provider};
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum OcrConfigKind {

9
litellm-rust/crates/core/src/ocr/types.rs
View file

@ -7,8 +7,8 @@ use serde_json::{Map, Value};
use super::hooks::{NoopOcrHooks, OcrHooks};
use super::provider_config::{OcrConfigKind, resolve_provider_config};
use crate::Error;
use crate::constants::OCR_HTTP_TIMEOUT_SECS;
use crate::ocr::Error;
use crate::params::OpaqueParams;
use litellm_auth::{InputSource, TokenProviderHandle};
@ -124,14 +124,11 @@ impl LiteLLMOcrRequest {
})
}
pub(crate) fn response_format(
&self,
) -> Result<OcrResponseFormat, super::error::OcrRequestError> {
pub(crate) fn response_format(&self) -> Result<OcrResponseFormat, super::Error> {
self.optional_params
.get("req_format")
.map(|value| {
serde_json::from_value(value.clone())
.map_err(|_| super::error::OcrRequestError::RequestFormat)
serde_json::from_value(value.clone()).map_err(|_| super::Error::RequestFormat)
})
.transpose()
.map(|format| format.unwrap_or_default())

59
litellm-rust/crates/core/src/ocr/wire.rs
View file

@ -1,10 +1,8 @@
use crate::ocr::error::OcrRequestError;
use crate::ocr::error::OcrResponseError;
use std::collections::BTreeMap;
use std::time::Duration;
use super::types::{LiteLLMOcrRequest, OcrConnection, OcrDocument};
use crate::Error;
use crate::ocr::Error;
use crate::params::OpaqueParams;
use litellm_auth::InputSource;
use serde::{
@ -123,18 +121,16 @@ pub fn decode_request(wire: OcrWireRequest) -> Result<LiteLLMOcrRequest, Error>
.unwrap_or_default()
.into_iter()
.map(|(name, value)| {
let value = value
.as_str()
.ok_or_else(|| OcrRequestError::RequestField {
path: format!("extra_headers.{name}"),
})?;
let value = value.as_str().ok_or_else(|| Error::RequestField {
path: format!("extra_headers.{name}"),
})?;
Ok((name, value.to_string()))
})
.collect::<Result<Vec<_>, OcrRequestError>>()?;
.collect::<Result<Vec<_>, Error>>()?;
let timeout = wire
.timeout_seconds
.map(|seconds| {
Duration::try_from_secs_f64(seconds).map_err(|_| OcrRequestError::RequestField {
Duration::try_from_secs_f64(seconds).map_err(|_| Error::RequestField {
path: "timeout_seconds".into(),
})
})
@ -148,7 +144,7 @@ pub fn decode_request(wire: OcrWireRequest) -> Result<LiteLLMOcrRequest, Error>
.as_u64()
.and_then(|value| usize::try_from(value).ok())
.filter(|value| *value > 0 && *value <= defaults.max_response_bytes)
.ok_or_else(|| OcrRequestError::RequestField {
.ok_or_else(|| Error::RequestField {
path: "max_response_bytes".into(),
})
})
@ -182,12 +178,12 @@ pub fn decode_request(wire: OcrWireRequest) -> Result<LiteLLMOcrRequest, Error>
})
}
fn decode_document(value: Value) -> Result<OcrDocument, OcrRequestError> {
fn decode_document(value: Value) -> Result<OcrDocument, Error> {
let kind = value.get("type").and_then(Value::as_str);
let missing_url = matches!(kind, Some("document_url")) && value.get("document_url").is_none()
|| matches!(kind, Some("image_url")) && value.get("image_url").is_none();
if missing_url {
return Err(OcrRequestError::MissingDocumentUrl);
return Err(Error::MissingDocumentUrl);
}
decode_request_value(value, "document")
}
@ -201,12 +197,9 @@ fn nonblank(value: Option<String>) -> Option<String> {
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
}
pub fn decode_request_value<T: DeserializeOwned>(
value: Value,
prefix: &str,
) -> Result<T, OcrRequestError> {
pub fn decode_request_value<T: DeserializeOwned>(value: Value, prefix: &str) -> Result<T, Error> {
serde_path_to_error::deserialize(value.into_deserializer()).map_err(|error| {
OcrRequestError::RequestField {
Error::RequestField {
path: format!("{prefix}.{}", error.path()),
}
})
@ -215,21 +208,19 @@ pub fn decode_request_value<T: DeserializeOwned>(
pub fn decode_response<T: DeserializeOwned>(
bytes: &[u8],
native: bool,
) -> Result<DecodedOcrResponse<T>, OcrResponseError> {
) -> Result<DecodedOcrResponse<T>, Error> {
let mut deserializer = serde_json::Deserializer::from_slice(bytes);
let data = serde_path_to_error::deserialize(&mut deserializer).map_err(|error| {
OcrResponseError::ResponseField {
Error::ResponseField {
path: error.path().to_string(),
}
})?;
deserializer
.end()
.map_err(|_| OcrResponseError::ResponseField {
path: "response".into(),
})?;
deserializer.end().map_err(|_| Error::ResponseField {
path: "response".into(),
})?;
let native = if native {
Some(
serde_json::from_slice(bytes).map_err(|_| OcrResponseError::ResponseField {
serde_json::from_slice(bytes).map_err(|_| Error::ResponseField {
path: "response".into(),
})?,
)
@ -307,10 +298,18 @@ mod tests {
serde_json::json!({"type": "document_url"}),
serde_json::json!({"type": "image_url"}),
] {
assert_eq!(
decode_document(document),
Err(OcrRequestError::MissingDocumentUrl)
);
let wire = serde_json::from_value(serde_json::json!({
"model": "mistral/model",
"document": document,
}))
.unwrap();
let error = decode_request(wire).err().expect("missing document URL");
assert_eq!(error, Error::MissingDocumentUrl);
let error = crate::Error::from(error);
assert!(matches!(
error,
crate::Error::Ocr(Error::MissingDocumentUrl)
));
}
}
}

24
litellm-rust/crates/core/src/params.rs
View file

@ -1,3 +1,11 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("invalid request: extra_body must be an object")]
ExtraBody,
#[error("invalid request: body must be a JSON object")]
Body,
}
use std::ops::{Deref, DerefMut};
use serde::{Deserialize, Serialize};
@ -85,15 +93,13 @@ impl OpaqueParams {
.collect()
}
pub fn into_provider_body(self) -> Result<Map<String, Value>, crate::Error> {
pub fn into_provider_body(self) -> Result<Map<String, Value>, Error> {
let mut fields = self.0;
let overrides = match fields.remove("extra_body") {
None | Some(Value::Null) => Map::new(),
Some(Value::Object(fields)) => fields,
Some(_) => {
return Err(crate::Error::InvalidRequest(
"extra_body must be an object".into(),
));
return Err(Error::ExtraBody);
}
};
Ok(fields
@ -107,13 +113,9 @@ impl OpaqueParams {
pub(crate) fn merge_extra_params<B: Serialize>(
body: &B,
extra_params: OpaqueParams,
) -> Result<Value, crate::Error> {
let Value::Object(fields) = serde_json::to_value(body)
.map_err(|_| crate::Error::InvalidRequest("body must be a JSON object".into()))?
else {
return Err(crate::Error::InvalidRequest(
"body must be a JSON object".into(),
));
) -> Result<Value, Error> {
let Value::Object(fields) = serde_json::to_value(body).map_err(|_| Error::Body)? else {
return Err(Error::Body);
};
Ok(Value::Object(
fields

2
litellm-rust/crates/core/src/providers/anthropic/chat_completions/tests.rs
View file

@ -1,5 +1,5 @@
use super::*;
use crate::Error;
use crate::chat_completions::Error;
use serde_json::json;
fn messages(value: Value) -> Vec<ChatMessage> {

2
litellm-rust/crates/core/src/providers/anthropic/chat_completions/transformation.rs
View file

@ -1,5 +1,6 @@
use serde_json::{Map, Value, json};
use crate::chat_completions::Error;
use crate::chat_completions::conversation::{Conversation, build_conversation};
use crate::chat_completions::transformation::{
ChatCompletionsAuth, ChatCompletionsProviderConfig, Unsupported, unsupported_message,
@ -10,7 +11,6 @@ use crate::chat_completions::types::{
ProviderChatRequestData, ProviderChatResponseData,
};
use crate::constants::ANTHROPIC_OAUTH_TOKEN_PREFIX;
use crate::error::Error;
use crate::params::OpaqueParams;
use crate::providers::anthropic::messages::transformation::{
complete_anthropic_url, resolve_anthropic_api_key,

14
litellm-rust/crates/core/src/providers/anthropic/messages/transformation.rs
View file

@ -1,4 +1,4 @@
use crate::error::Error;
use crate::messages::Error;
use crate::messages::transformation::{AnthropicMessagesProviderConfig, MessagesAuthStrategy};
const ANTHROPIC_API_KEY_ENV: &str = "ANTHROPIC_API_KEY";
@ -17,15 +17,13 @@ pub fn non_empty(value: Option<&str>) -> Option<&str> {
pub fn resolve_anthropic_api_key(
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
) -> Result<String, litellm_auth::Error> {
non_empty(api_key)
.map(str::to_string)
.or_else(|| env_lookup(ANTHROPIC_API_KEY_ENV).filter(|value| !value.trim().is_empty()))
.ok_or_else(|| {
Error::from(crate::AuthError::MissingApiKey {
provider: "Anthropic",
environment_variable: ANTHROPIC_API_KEY_ENV,
})
.ok_or(litellm_auth::Error::MissingApiKey {
provider: "Anthropic",
environment_variable: ANTHROPIC_API_KEY_ENV,
})
}
@ -60,7 +58,7 @@ impl AnthropicMessagesProviderConfig for AnthropicMessagesConfig {
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
resolve_anthropic_api_key(api_key, env_lookup)
resolve_anthropic_api_key(api_key, env_lookup).map_err(Error::from)
}
fn auth_strategy(&self) -> MessagesAuthStrategy {

6
litellm-rust/crates/core/src/providers/azure_ai/messages/transformation.rs
View file

@ -1,4 +1,4 @@
use crate::error::Error;
use crate::messages::Error;
use crate::messages::transformation::{AnthropicMessagesProviderConfig, MessagesAuthStrategy};
use crate::messages::types::{
AnthropicMessage, AnthropicMessagesRequest, AnthropicMessagesResponse, ContentBlock,
@ -33,7 +33,7 @@ pub fn resolve_azure_api_key(
.map(str::to_string)
.or_else(|| env_lookup(AZURE_API_KEY_ENV).filter(|value| !value.trim().is_empty()))
.ok_or_else(|| {
Error::from(crate::AuthError::MissingApiKey {
Error::from(litellm_auth::Error::MissingApiKey {
provider: "Azure",
environment_variable: AZURE_API_KEY_ENV,
})
@ -47,7 +47,7 @@ pub fn complete_azure_anthropic_url(
let api_base = non_empty(api_base)
.map(str::to_string)
.or_else(|| env_lookup(AZURE_API_BASE_ENV).filter(|value| !value.trim().is_empty()))
.ok_or_else(|| Error::from(crate::AuthError::MissingAzureApiBase))?;
.ok_or_else(|| Error::from(litellm_auth::Error::MissingAzureApiBase))?;
let api_base = api_base.trim_end_matches('/');

3
litellm-rust/crates/core/src/providers/bedrock/audio_transcription.rs
View file

@ -1,12 +1,13 @@
use serde_json::{Map, Value, json};
use crate::audio_transcription::Error;
use crate::audio_transcription::transformation::{
AudioTranscriptionAuth, AudioTranscriptionProviderConfig,
};
use crate::audio_transcription::types::{
AudioTranscriptionRequestData, AudioTranscriptionResponseData,
};
use crate::error::{Error, json_type_name};
use crate::http_utils::json_type_name;
use crate::params::OpaqueParams;
pub use super::aws_base::{aws_auth_config, bedrock_model_id_and_region, resolve_bedrock_region};

2
litellm-rust/crates/core/src/providers/bedrock/chat_completions/tests.rs
View file

@ -1,5 +1,5 @@
use super::*;
use crate::Error;
use crate::chat_completions::Error;
use serde_json::json;
fn messages(value: Value) -> Vec<ChatMessage> {

2
litellm-rust/crates/core/src/providers/bedrock/chat_completions/transformation.rs
View file

@ -1,5 +1,6 @@
use serde_json::{Map, Value, json};
use crate::chat_completions::Error;
use crate::chat_completions::conversation::{Conversation, TurnRole, build_conversation};
use crate::chat_completions::response_utils::{finish_reason_for, unix_now, usage_from_parts};
use crate::chat_completions::transformation::{
@ -11,7 +12,6 @@ use crate::chat_completions::types::{
ChatCompletionsUsage, ChatMessage, ChatMessageContent, ProviderChatRequestData,
ProviderChatResponseData,
};
use crate::error::Error;
use crate::params::OpaqueParams;
use super::super::aws_base::{bedrock_model_id_and_region, resolve_bedrock_region};

2
litellm-rust/crates/core/src/providers/openai/responses/transformation.rs
View file

@ -1,4 +1,4 @@
use crate::Error;
use crate::responses::Error;
use crate::responses::types::{ResponsesWsEvent, ResponsesWsTransformResult};
use crate::responses::websocket::{ResponsesWebSocketProviderConfig, enforce_model};

19
litellm-rust/crates/core/src/realtime/error.rs Normal file
View file

@ -0,0 +1,19 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error("routing error: {0}")]
Routing(String),
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Transport(#[from] crate::transport::Error),
#[error(transparent)]
Params(#[from] crate::params::Error),
#[error(transparent)]
Headers(#[from] crate::http_utils::HeaderError),
}

19
litellm-rust/crates/core/src/responses/error.rs Normal file
View file

@ -0,0 +1,19 @@
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("invalid provider: {0}")]
InvalidProvider(String),
#[error("invalid request: {0}")]
InvalidRequest(String),
#[error("invalid response: {0}")]
InvalidResponse(String),
#[error("routing error: {0}")]
Routing(String),
#[error(transparent)]
Auth(#[from] litellm_auth::Error),
#[error(transparent)]
Transport(#[from] crate::transport::Error),
#[error(transparent)]
Params(#[from] crate::params::Error),
#[error(transparent)]
Headers(#[from] crate::http_utils::HeaderError),
}

3
litellm-rust/crates/core/src/responses/instrumentation.rs
View file

@ -5,8 +5,8 @@ use std::time::{SystemTime, UNIX_EPOCH};
use serde_json::Value;
use crate::Error;
use crate::call_lifecycle::{CallLifecycleContext, CallLifecycleHooks, CallLifecycleTiming};
use crate::responses::Error;
use crate::responses::types::{ResponsesWsEvent, ResponsesWsEventType};
#[derive(Clone, Debug, Default, PartialEq, Eq)]
@ -208,6 +208,7 @@ impl ResponsesWsInstrumentation {
type LifecycleFuture<'a, T> = Pin<Box<dyn Future<Output = Result<T, Error>> + Send + 'a>>;
impl CallLifecycleHooks<(), (), ()> for ResponsesWsInstrumentation {
type Error = Error;
type PreCallFuture<'a> = LifecycleFuture<'a, ()>;
type DuringCallFuture<'a> = LifecycleFuture<'a, ()>;
type SuccessFuture<'a> = Pin<Box<dyn Future<Output = ()> + Send + 'a>>;

2
litellm-rust/crates/core/src/responses/mod.rs
View file

@ -1,3 +1,5 @@
mod error;
pub use error::Error;
pub mod instrumentation;
pub mod types;
pub mod websocket;

45
litellm-rust/crates/core/src/responses/websocket.rs
View file

@ -16,8 +16,8 @@ use tokio_tungstenite::{
Connector, MaybeTlsStream, WebSocketStream, connect_async_tls_with_config,
};
use crate::Error;
use crate::constants::{OPENAI_RESPONSES_DEFAULT_API_BASE, OPENAI_RESPONSES_PATH};
use crate::responses::Error;
use crate::responses::types::{ResponsesWsEvent, ResponsesWsEventType, ResponsesWsTransformResult};
pub trait ResponsesWebSocketProviderConfig: Sync {
@ -204,9 +204,9 @@ impl ResponsesWebSocketConnection {
headers: &HashMap<String, String>,
timeout: Option<Duration>,
) -> Result<Self, Error> {
let mut request = url
.into_client_request()
.map_err(|error| Error::Network(error.to_string()))?;
let mut request = url.into_client_request().map_err(|error| {
Error::Transport(crate::transport::Error::Network(error.to_string()))
})?;
for (name, value) in headers {
let header_name = name
.parse::<HeaderName>()
@ -217,17 +217,21 @@ impl ResponsesWebSocketConnection {
}
let connect = connect_upstream(request);
let result = match timeout {
Some(timeout) => tokio::time::timeout(timeout, connect)
.await
.map_err(|_| Error::Network("Responses WebSocket connection timed out".into()))?,
Some(timeout) => tokio::time::timeout(timeout, connect).await.map_err(|_| {
Error::Transport(crate::transport::Error::Network(
"Responses WebSocket connection timed out".into(),
))
})?,
None => connect.await,
};
let (socket, _) = result.map_err(|error| match *error {
tokio_tungstenite::tungstenite::Error::Http(response) => Error::Http {
status: response.status().as_u16(),
body: String::new(),
},
other => Error::Network(other.to_string()),
tokio_tungstenite::tungstenite::Error::Http(response) => {
Error::Transport(crate::transport::Error::Http {
status: response.status().as_u16(),
body: String::new(),
})
}
other => Error::Transport(crate::transport::Error::Network(other.to_string())),
})?;
Ok(Self {
socket: Arc::new(Mutex::new(Some(socket))),
@ -237,12 +241,14 @@ impl ResponsesWebSocketConnection {
pub async fn send_text(&self, text: String) -> Result<(), Error> {
let mut socket = self.socket.lock().await;
let Some(socket) = socket.as_mut() else {
return Err(Error::Network("Responses WebSocket is closed".into()));
return Err(Error::Transport(crate::transport::Error::Network(
"Responses WebSocket is closed".into(),
)));
};
socket
.send(Message::Text(text))
.await
.map_err(|error| Error::Network(error.to_string()))
.map_err(|error| Error::Transport(crate::transport::Error::Network(error.to_string())))
}
pub async fn recv_text(&self) -> Result<Option<String>, Error> {
@ -257,17 +263,18 @@ impl ResponsesWebSocketConnection {
.map_err(|error| Error::InvalidResponse(error.to_string())),
Some(Ok(Message::Close(_))) | None => Ok(None),
Some(Ok(_)) => Ok(None),
Some(Err(error)) => Err(Error::Network(error.to_string())),
Some(Err(error)) => Err(Error::Transport(crate::transport::Error::Network(
error.to_string(),
))),
}
}
pub async fn close(&self) -> Result<(), Error> {
let mut socket = self.socket.lock().await;
if let Some(socket) = socket.as_mut() {
socket
.close(None)
.await
.map_err(|error| Error::Network(error.to_string()))?;
socket.close(None).await.map_err(|error| {
Error::Transport(crate::transport::Error::Network(error.to_string()))
})?;
}
*socket = None;
Ok(())

76
litellm-rust/crates/core/src/transport/error.rs Normal file
View file

@ -0,0 +1,76 @@
#[derive(Clone, Debug, thiserror::Error, PartialEq, Eq)]
pub enum Error {
#[error("upstream request failed with status {status}: {body}")]
Http { status: u16, body: String },
#[error("upstream network error: {0}")]
Network(String),
#[error("could not reach the provider: {0}")]
Connect(String),
}
impl Error {
pub fn from_reqwest_before_dispatch(error: reqwest::Error) -> Self {
let before_dispatch = !error.is_timeout() && (error.is_connect() || error.is_builder());
let message = error.without_url().to_string();
if before_dispatch {
Self::Connect(message)
} else {
Self::Network(message)
}
}
}
impl From<reqwest::Error> for Error {
fn from(error: reqwest::Error) -> Self {
Self::Network(error.without_url().to_string())
}
}
#[cfg(test)]
mod tests {
use super::Error;
#[tokio::test]
async fn transport_errors_remove_urls_and_keep_dispatch_context() {
let error = reqwest::Client::builder()
.no_proxy()
.build()
.expect("client")
.get("http://localhost:invalid/private?api_key=secret")
.send()
.await
.expect_err("invalid port");
let error = Error::from_reqwest_before_dispatch(error);
assert!(matches!(error, Error::Connect(_)));
assert!(!error.to_string().contains("secret"));
assert!(!error.to_string().contains("private"));
}
#[tokio::test]
async fn request_timeout_is_not_safe_to_retry_as_a_connect_failure() {
use std::time::Duration;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind");
let address = listener.local_addr().expect("address");
let request = reqwest::Client::builder()
.no_proxy()
.build()
.expect("client")
.get(format!("http://{address}"))
.timeout(Duration::from_millis(200))
.send();
let (response, accepted) = tokio::join!(
request,
tokio::time::timeout(Duration::from_secs(2), listener.accept())
);
let _connection = accepted
.expect("accept deadline")
.expect("accepted connection");
let error = response.expect_err("server does not respond");
assert!(error.is_timeout());
assert!(matches!(
Error::from_reqwest_before_dispatch(error),
Error::Network(_)
));
}
}

2
litellm-rust/crates/core/src/transport/mod.rs Normal file
View file

@ -0,0 +1,2 @@
mod error;
pub use error::Error;

7
litellm-rust/crates/core/tests/host_lifecycle.rs
View file

@ -1,10 +1,11 @@
use crate::Error;
use crate::call_lifecycle::host::{HostFailure, HostLifecycle, HostPhase};
use crate::ocr::Error;
fn run(fail_at: Option<HostPhase>, asynchronous: bool) -> (Vec<HostPhase>, Vec<Error>) {
let mut lifecycle = HostLifecycle::new(asynchronous);
let mut events = Vec::new();
let mut failures = Vec::new();
while lifecycle.phase() != HostPhase::Complete {
let phase = lifecycle.phase();
events.push(phase);
@ -80,14 +81,14 @@ fn only_provider_and_response_construction_failures_use_provider_mapping() {
fn failure_handler_errors_do_not_replace_selected_failure_or_suppress_async_dispatch() {
let mut lifecycle = HostLifecycle::new(true);
while lifecycle.phase() != HostPhase::Execute {
lifecycle.accept(Ok(()));
lifecycle.accept::<Error>(Ok(()));
}
let selected = Error::InvalidRequest("provider".into());
assert_eq!(
lifecycle.accept(Err(HostFailure::Error(selected.clone()))),
Some(selected)
);
lifecycle.accept(Ok(()));
lifecycle.accept::<Error>(Ok(()));
for phase in [
HostPhase::DeploymentFailure,
HostPhase::Failure,

44
litellm-rust/crates/core/tests/ocr.rs
View file

@ -196,7 +196,7 @@ impl OcrHooks for RecordingHooks {
Box::pin(async move {
self.events.lock().unwrap().push("pre");
if self.block {
return Err(crate::Error::InvalidRequest("blocked".into()));
return Err(crate::ocr::Error::InvalidRequest("blocked".into()));
}
Ok(request)
})
@ -233,7 +233,7 @@ impl OcrHooks for RecordingHooks {
fn failure<'a>(
&'a self,
_context: &'a CallLifecycleContext,
_error: &'a crate::Error,
_error: &'a crate::ocr::Error,
_timing: &'a CallLifecycleTiming,
) -> OcrLogFuture<'a> {
Box::pin(async move {
@ -307,7 +307,7 @@ async fn lifecycle_blocking_prevents_execution_and_emits_one_failure() {
..request
};
let error = perform_ocr(request).await.unwrap_err();
assert!(matches!(error, crate::Error::InvalidRequest(_)));
assert!(matches!(error, crate::ocr::Error::InvalidRequest(_)));
assert_eq!(*events.lock().unwrap(), ["pre", "failure"]);
}
@ -414,7 +414,7 @@ async fn fallible_host_phases_do_not_replay_or_reach_transport() {
OcrHostOperation::PreCall(request) => {
phases.push("pre");
result = Some(OcrHostResult::PreCall(if failure_phase == "pre" {
Err(crate::Error::InvalidRequest("pre failed".into()))
Err(crate::ocr::Error::InvalidRequest("pre failed".into()))
} else {
Ok(request)
}));
@ -422,7 +422,7 @@ async fn fallible_host_phases_do_not_replay_or_reach_transport() {
OcrHostOperation::DuringCall(request) => {
phases.push("during");
result = Some(OcrHostResult::DuringCall(if failure_phase == "during" {
Err(crate::Error::InvalidRequest("during failed".into()))
Err(crate::ocr::Error::InvalidRequest("during failed".into()))
} else {
Ok(request)
}));
@ -433,7 +433,7 @@ async fn fallible_host_phases_do_not_replay_or_reach_transport() {
Ok(OcrCallStep::Complete(_)) => panic!("failed call completed"),
}
};
assert!(matches!(error, crate::Error::InvalidRequest(_)));
assert!(matches!(error, crate::ocr::Error::InvalidRequest(_)));
assert_eq!(
phases
.iter()
@ -476,7 +476,10 @@ async fn invalid_provider_response_runs_post_call_before_normalization_failure()
}
};
server.await.unwrap();
assert!(matches!(error, crate::Error::InvalidResponse(_)));
assert!(matches!(
error,
crate::ocr::Error::ResponseField { ref path } if path == "pages"
));
assert_eq!(seen.lock().unwrap().len(), 1);
assert_eq!(post_calls, [json!(r#"{"pages":"invalid"}"#)]);
}
@ -553,7 +556,7 @@ async fn direct_native_host_drives_the_same_state_machine() {
);
assert!(matches!(
call.resume(None).await,
Err(crate::Error::InvalidRequest(_))
Err(crate::ocr::Error::InvalidRequest(_))
));
}
@ -572,7 +575,7 @@ async fn public_finalization_failure_never_dispatches_success_or_replays_provide
) else {
panic!("supported call declined")
};
let selected = crate::Error::InvalidRequest("public metadata failed".into());
let selected = crate::ocr::Error::InvalidRequest("public metadata failed".into());
let host = NoopOcrHost;
let mut result = None;
let mut failures = Vec::new();
@ -587,7 +590,7 @@ async fn public_finalization_failure_never_dispatches_success_or_replays_provide
assert_eq!(error, selected);
failures.push("sync");
OcrHostResult::Lifecycle(Err(HostFailure::Error(
crate::Error::InvalidRequest("failure callback failed".into()),
crate::ocr::Error::InvalidRequest("failure callback failed".into()),
)))
}
OcrHostOperation::Lifecycle(HostPhase::AsyncFailure) => {
@ -646,7 +649,7 @@ async fn cancellation_at_provider_hook_prevents_execution_and_further_resumption
OcrCallStep::Complete(_) => panic!("provider executed before pre-call result"),
}
}
let selected = crate::Error::InvalidRequest("cancelled".into());
let selected = crate::ocr::Error::InvalidRequest("cancelled".into());
assert!(matches!(
call.interrupt(HostFailure::Cancelled(selected.clone())).await,
Err(error) if error == selected
@ -683,7 +686,7 @@ async fn missing_host_result_preserves_pending_operation() {
async fn read_bounded_response(
response: Vec<u8>,
limit: usize,
) -> Result<bytes::Bytes, super::error::OcrError> {
) -> Result<bytes::Bytes, super::Error> {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
@ -712,7 +715,7 @@ async fn read_bounded_response(
#[tokio::test]
async fn response_limit_accepts_exact_size_and_rejects_declared_and_chunked_overflow() {
use super::error::{OcrError, OcrResponseError};
use super::Error;
for response in [
"HTTP/1.1 200 OK\r\nContent-Length: 8\r\n\r\nabcdefgh",
@ -731,7 +734,7 @@ async fn response_limit_accepts_exact_size_and_rejects_declared_and_chunked_over
] {
assert!(matches!(
read_bounded_response(response.as_bytes().to_vec(), 8).await,
Err(OcrError::Response(OcrResponseError::TooLarge { limit: 8 }))
Err(Error::TooLarge { limit: 8 })
));
}
}
@ -750,10 +753,7 @@ async fn oversized_error_retains_http_status_and_bounded_diagnostics_without_dra
.await
.unwrap_err();
match error {
super::error::OcrError::Transport(crate::error::TransportError::Http {
status,
body,
}) => {
super::Error::Transport(crate::transport::Error::Http { status, body }) => {
assert_eq!(status, 429);
assert_eq!(
body,
@ -811,8 +811,8 @@ impl Drop for TokenFutureDrop {
}
}
impl crate::auth::TokenProvider for PendingToken {
fn acquire(&self) -> crate::auth::TokenFuture<'_> {
impl litellm_auth::TokenProvider for PendingToken {
fn acquire(&self) -> litellm_auth::TokenFuture<'_> {
Box::pin(async move {
let _guard = TokenFutureDrop(self.dropped.clone());
self.entered.notify_one();
@ -837,7 +837,7 @@ async fn cancellation_waits_for_provider_capture_drop_even_when_acknowledgement_
extra_headers: vec![("authorization".into(), "Bearer test-key".into())],
..request.connection
},
azure_ad_token_provider: Some(crate::auth::TokenProviderHandle::new(Arc::new(
azure_ad_token_provider: Some(litellm_auth::TokenProviderHandle::new(Arc::new(
PendingToken {
entered: entered.clone(),
dropped: dropped.clone(),
@ -867,7 +867,7 @@ async fn cancellation_waits_for_provider_capture_drop_even_when_acknowledgement_
}
}).await.unwrap();
assert!(!dropped.load(Ordering::SeqCst));
let selected = crate::Error::InvalidRequest("cancelled".into());
let selected = crate::ocr::Error::InvalidRequest("cancelled".into());
if interrupt_acknowledgement {
let mut acknowledgement =
Box::pin(call.interrupt(HostFailure::Cancelled(selected.clone())));

2
litellm-rust/crates/core/tests/ocr/support.rs
View file

@ -17,7 +17,7 @@ pub(crate) fn ocr_client() -> OcrClient {
pub(crate) async fn perform_ocr(
request: LiteLLMOcrRequest,
) -> Result<LiteLLMOcrResponse, crate::Error> {
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
ocr_client().perform(request).await
}

2
litellm-rust/crates/core/tests/vertex_ai_deepseek_ocr.rs
View file

@ -1,7 +1,7 @@
use serde_json::{Value, json};
use super::test_support::{MockResponse, mock_server, perform_ocr, wire_request};
use crate::auth::InputSource;
use litellm_auth::InputSource;
fn request_body(request: &str) -> Value {
serde_json::from_str(request.split_once("\r\n\r\n").unwrap().1).unwrap()

2
litellm-rust/crates/core/tests/vertex_ai_ocr.rs
View file

@ -1,7 +1,7 @@
use serde_json::{Value, json};
use super::test_support::{MockResponse, mock_server, perform_ocr, wire_request};
use crate::auth::InputSource;
use litellm_auth::InputSource;
fn request_body(request: &str) -> Value {
serde_json::from_str(request.split_once("\r\n\r\n").unwrap().1).unwrap()

179
litellm-rust/crates/python-bridge/src/errors.rs
View file

@ -1,4 +1,5 @@
use litellm_core::error::Error;
use litellm_core::transport::Error as TransportError;
use litellm_core::{Error, audio_transcription, chat_completions, messages, realtime, responses};
use pyo3::exceptions::{PyRuntimeError, PyValueError};
use pyo3::prelude::*;
@ -16,43 +17,98 @@ pyo3::create_exception!(
"The provider call was already issued and failed. Args are (status, message); status is 0 when there was no HTTP response."
);
pub(crate) fn core_error_to_pyerr(err: Error) -> PyErr {
match err {
Error::Auth(message) => PyValueError::new_err(message),
Error::InvalidProvider(_)
| Error::InvalidRequest(_)
| Error::InvalidType { .. }
| Error::MissingField(_)
| Error::MissingDocumentUrl => PyValueError::new_err(err.to_string()),
other => PyRuntimeError::new_err(other.to_string()),
pub(crate) fn core_error_to_pyerr(error: impl Into<Error>) -> PyErr {
let error = error.into();
let value_error = match &error {
Error::Ocr(error) => {
error.is_request()
|| matches!(error, litellm_core::ocr::Error::InvalidProvider(_))
|| matches!(error, litellm_core::ocr::Error::Auth(source) if !matches!(source, litellm_auth::Error::MissingApiKey { .. }))
}
Error::Messages(error) => match error {
messages::Error::Auth(source) => {
!matches!(source, litellm_auth::Error::MissingApiKey { .. })
}
messages::Error::InvalidProvider(_)
| messages::Error::InvalidRequest(_)
| messages::Error::Params(_)
| messages::Error::Headers(_) => true,
_ => false,
},
Error::AudioTranscription(error) => match error {
audio_transcription::Error::Auth(source) => {
!matches!(source, litellm_auth::Error::MissingApiKey { .. })
}
audio_transcription::Error::InvalidProvider(_)
| audio_transcription::Error::InvalidRequest(_)
| audio_transcription::Error::Params(_)
| audio_transcription::Error::Headers(_)
| audio_transcription::Error::InvalidType { .. }
| audio_transcription::Error::MissingField(_)
| audio_transcription::Error::Aws(_) => true,
_ => false,
},
Error::ChatCompletions(error) => match error {
chat_completions::Error::Auth(source) => {
!matches!(source, litellm_auth::Error::MissingApiKey { .. })
}
chat_completions::Error::InvalidProvider(_)
| chat_completions::Error::InvalidRequest(_)
| chat_completions::Error::Params(_)
| chat_completions::Error::Headers(_)
| chat_completions::Error::InvalidType { .. }
| chat_completions::Error::MissingField(_)
| chat_completions::Error::Aws(_) => true,
_ => false,
},
Error::Realtime(error) => match error {
realtime::Error::Auth(source) => {
!matches!(source, litellm_auth::Error::MissingApiKey { .. })
}
realtime::Error::InvalidProvider(_)
| realtime::Error::InvalidRequest(_)
| realtime::Error::Params(_)
| realtime::Error::Headers(_) => true,
_ => false,
},
Error::Responses(error) => match error {
responses::Error::Auth(source) => {
!matches!(source, litellm_auth::Error::MissingApiKey { .. })
}
responses::Error::InvalidProvider(_)
| responses::Error::InvalidRequest(_)
| responses::Error::Params(_)
| responses::Error::Headers(_) => true,
_ => false,
},
};
if value_error {
PyValueError::new_err(error.to_string())
} else {
PyRuntimeError::new_err(error.to_string())
}
}
/// Map a core error for a route whose host keeps a Python implementation.
///
/// The distinction the host needs is whether the provider was already called.
/// Everything raised before the request goes out is safe for the host to retry
/// on its own path; anything after it is not, because the provider has already
/// done the work and billed for it.
pub(crate) fn chat_completions_error_to_pyerr(err: Error) -> PyErr {
match err {
pub(crate) fn chat_completions_error_to_pyerr(error: chat_completions::Error) -> PyErr {
use chat_completions::Error;
match error {
Error::Unsupported(_)
| Error::Auth(_)
| Error::Aws(_)
| Error::InvalidProvider(_)
| Error::InvalidRequest(_)
| Error::InvalidType { .. }
| Error::MissingField(_)
| Error::MissingDocumentUrl
| Error::MissingApiKey { .. }
| Error::MissingAzureAiCredentials
| Error::MissingAzureDocumentIntelligenceCredentials
| Error::MissingReductoApiKey
| Error::Routing(_)
// Nothing reached the provider, so serving it on Python cannot double
// bill and is the only way the caller gets an answer at all.
| Error::Connect(_) => RustBridgeDeclined::new_err(err.to_string()),
Error::Http { status, body } => RustUpstreamError::new_err((status, body)),
Error::Network(message) | Error::InvalidResponse(message) => {
| Error::Params(_)
| Error::Headers(_)
| Error::Transport(TransportError::Connect(_)) => {
RustBridgeDeclined::new_err(error.to_string())
}
Error::ResponseTransform(source) => RustUpstreamError::new_err((0u16, source.to_string())),
Error::Transport(TransportError::Http { status, body }) => {
RustUpstreamError::new_err((status, body))
}
Error::Transport(TransportError::Network(message)) | Error::InvalidResponse(message) => {
RustUpstreamError::new_err((0u16, message))
}
}
@ -63,3 +119,68 @@ pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {
module.add("RustBridgeDeclined", py.get_type::<RustBridgeDeclined>())?;
module.add("RustUpstreamError", py.get_type::<RustUpstreamError>())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn response_normalization_failure_never_declines_or_loses_the_cause() {
use std::error::Error as _;
Python::initialize();
let error = chat_completions::Error::ResponseTransform(Box::new(
chat_completions::Error::MissingField("usage"),
));
assert!(matches!(
error
.source()
.unwrap()
.downcast_ref::<Box<chat_completions::Error>>(),
Some(source) if **source == chat_completions::Error::MissingField("usage")
));
let mapped = chat_completions_error_to_pyerr(error);
Python::attach(|py| {
assert!(mapped.is_instance_of::<RustUpstreamError>(py));
assert!(!mapped.is_instance_of::<RustBridgeDeclined>(py));
assert_eq!(
mapped
.value(py)
.getattr("args")
.unwrap()
.extract::<(u16, String)>()
.unwrap(),
(0, "missing required field: usage".into())
);
});
}
#[test]
fn transport_status_and_dispatch_certainty_survive_python_mapping() {
Python::initialize();
Python::attach(|py| {
let connect = chat_completions_error_to_pyerr(
TransportError::Connect("unreachable".into()).into(),
);
assert!(connect.is_instance_of::<RustBridgeDeclined>(py));
let network =
chat_completions_error_to_pyerr(TransportError::Network("timed out".into()).into());
assert!(network.is_instance_of::<RustUpstreamError>(py));
let upstream = chat_completions_error_to_pyerr(
TransportError::Http {
status: 429,
body: "slow down".into(),
}
.into(),
);
assert_eq!(
upstream
.value(py)
.getattr("args")
.unwrap()
.extract::<(u16, String)>()
.unwrap(),
(429, "slow down".into())
);
});
}
}

2
litellm-rust/crates/python-bridge/src/execution.rs
View file

@ -165,7 +165,7 @@ mod tests {
use std::thread;
use std::time::Instant;
use litellm_core::error::Error;
use litellm_core::messages::Error;
use pyo3::panic::PanicException;
use pyo3::types::{PyDict, PyModule};
use rstest::{fixture, rstest};

27
litellm-rust/crates/python-bridge/src/lifecycle/mod.rs
View file

@ -35,7 +35,8 @@ pub(crate) trait PythonRoute: Send + Sync {
fn state_mut(&mut self) -> &mut PythonCallState;
fn classify(operation: &<Self::Call as NativeCall>::Operation) -> OperationClass;
fn lifecycle_result() -> <Self::Call as NativeCall>::Result;
fn map_error(error: litellm_core::Error) -> PyErr;
fn map_error(error: <Self::Call as NativeCall>::Error) -> PyErr;
fn host_error(message: String) -> <Self::Call as NativeCall>::Error;
fn invoke(
&mut self,
py: Python<'_>,
@ -46,7 +47,8 @@ pub(crate) trait PythonRoute: Send + Sync {
}
type NativeStep<C> = NativeCallStep<<C as NativeCall>::Operation, <C as NativeCall>::Complete>;
type NativeResult<C> = Result<NativeStep<C>, litellm_core::Error>;
type NativeResult<C> = Result<NativeStep<C>, <C as NativeCall>::Error>;
type HostResult<C> = Result<<C as NativeCall>::Result, HostFailure<<C as NativeCall>::Error>>;
type HostResumeStep<R> = HostStep<NativeStep<<R as PythonRoute>::Call>, Py<PyAny>>;
struct NativeCallState<C: NativeCall> {
@ -102,7 +104,7 @@ impl<R: PythonRoute> PythonLifecycle<R> {
fn resume_core(
&mut self,
py: Python<'_>,
result: Option<Result<<R::Call as NativeCall>::Result, HostFailure>>,
result: Option<HostResult<R::Call>>,
) -> PyResult<HostResumeStep<R>> {
let call = Arc::clone(self.call.as_ref().ok_or_else(missing_state)?);
let future = async move {
@ -154,8 +156,8 @@ impl<R: PythonRoute> PythonLifecycle<R> {
py: Python<'_>,
error: PyErr,
phase: Option<HostPhase>,
) -> HostFailure {
let native = litellm_core::Error::InvalidRequest(error.to_string());
) -> HostFailure<<R::Call as NativeCall>::Error> {
let native = R::host_error(error.to_string());
let cancelled = !error.is_instance_of::<PyException>(py);
let failure = if !cancelled {
HostFailure::Error(native)
@ -667,6 +669,7 @@ mod tests {
struct SyntheticCall(bool);
impl NativeCall for SyntheticCall {
type Error = litellm_core::messages::Error;
type Operation = ();
type Result = ();
type Complete = ();
@ -674,7 +677,7 @@ mod tests {
fn resume(
&mut self,
result: Option<Self::Result>,
) -> HostCallFuture<'_, Self::Operation, Self::Complete> {
) -> HostCallFuture<'_, Self::Operation, Self::Complete, Self::Error> {
Box::pin(async move {
match (self.0, result) {
(false, None) => {
@ -682,7 +685,7 @@ mod tests {
Ok(NativeCallStep::Host(()))
}
(true, Some(())) => Ok(NativeCallStep::Complete(())),
_ => Err(litellm_core::Error::InvalidRequest(
_ => Err(litellm_core::messages::Error::InvalidRequest(
"invalid synthetic lifecycle state".into(),
)),
}
@ -691,8 +694,8 @@ mod tests {
fn interrupt(
&mut self,
_: HostFailure,
) -> HostCallFuture<'_, Self::Operation, Self::Complete> {
_: HostFailure<Self::Error>,
) -> HostCallFuture<'_, Self::Operation, Self::Complete, Self::Error> {
Box::pin(async { Ok(NativeCallStep::Complete(())) })
}
}
@ -716,7 +719,11 @@ mod tests {
fn lifecycle_result() {}
fn map_error(error: litellm_core::Error) -> PyErr {
fn host_error(message: String) -> litellm_core::messages::Error {
litellm_core::messages::Error::InvalidRequest(message)
}
fn map_error(error: litellm_core::messages::Error) -> PyErr {
crate::errors::core_error_to_pyerr(error)
}

2
litellm-rust/crates/python-bridge/src/routes/audio_transcription/value.rs
View file

@ -1,4 +1,4 @@
use litellm_core::Error;
use litellm_core::audio_transcription::Error;
use std::future::Future;
use litellm_core::audio_transcription::{

2
litellm-rust/crates/python-bridge/src/routes/chat_completions/value.rs
View file

@ -1,4 +1,4 @@
use litellm_core::Error;
use litellm_core::chat_completions::Error;
use std::future::Future;
use litellm_core::chat_completions::types::{ChatCompletionsRequest, ChatCompletionsResponse};

2
litellm-rust/crates/python-bridge/src/routes/definition.rs
View file

@ -79,7 +79,7 @@ mod tests {
use std::ffi::CString;
use std::sync::atomic::{AtomicBool, Ordering};
use litellm_core::error::Error;
use litellm_core::messages::Error;
use pyo3::exceptions::PyLookupError;
use pyo3::types::{PyDict, PyList};

2
litellm-rust/crates/python-bridge/src/routes/messages/value.rs
View file

@ -1,4 +1,4 @@
use litellm_core::Error;
use litellm_core::messages::Error;
use litellm_core::messages::messages as run_messages;
use litellm_core::messages::types::{AnthropicMessagesResponse, MessagesRequest};
use pyo3::prelude::*;

64
litellm-rust/crates/python-bridge/src/routes/ocr/errors.rs
View file

@ -1,13 +1,32 @@
use litellm_core::error::Error;
use litellm_core::ocr::Error;
use litellm_core::transport::Error as TransportError;
use pyo3::exceptions::{PyRuntimeError, PyValueError};
use pyo3::prelude::*;
use crate::errors::{RustUpstreamError, core_error_to_pyerr};
pub(super) fn to_pyerr(error: Error) -> PyErr {
let status = error.http_status_code();
let mapped = match error {
Error::Http { status, body } => RustUpstreamError::new_err((status, body)),
other => core_error_to_pyerr(other),
let (mapped, status) = match error {
Error::MissingDocumentUrl => (
PyValueError::new_err(Error::MissingDocumentUrl.to_string()),
Some(500),
),
error @ Error::MissingField(_) => (PyValueError::new_err(error.to_string()), None),
error if error.is_request() => (
PyValueError::new_err(format!("invalid request: {error}")),
Some(400),
),
error if error.is_response() => (
PyRuntimeError::new_err(format!("invalid response: {error}")),
None,
),
error @ (Error::InvalidRequest(_) | Error::Params(_) | Error::Headers(_)) => {
(PyValueError::new_err(error.to_string()), Some(400))
}
Error::Transport(TransportError::Http { status, body }) => {
(RustUpstreamError::new_err((status, body)), Some(status))
}
other => (core_error_to_pyerr(other), None),
};
attach_status(mapped, status)
}
@ -44,10 +63,10 @@ mod tests {
.unwrap(),
500
);
let mapped = to_pyerr(Error::Http {
let mapped = to_pyerr(Error::Transport(litellm_core::transport::Error::Http {
status: 429,
body: r#"{"message":"rate limited"}"#.to_string(),
});
}));
assert!(mapped.is_instance_of::<RustUpstreamError>(py));
let args: (u16, String) = mapped
.value(py)
@ -69,4 +88,35 @@ mod tests {
);
});
}
#[test]
fn typed_request_and_response_failures_keep_python_contracts() {
Python::initialize();
Python::attach(|py| {
let request = to_pyerr(Error::RequestField {
path: "document.type".into(),
});
assert!(request.is_instance_of::<PyValueError>(py));
assert_eq!(
request
.value(py)
.getattr("status_code")
.unwrap()
.extract::<u16>()
.unwrap(),
400
);
assert_eq!(
request.value(py).to_string(),
"invalid request: invalid OCR request field: document.type"
);
let response = to_pyerr(Error::EmptyContent);
assert!(response.is_instance_of::<PyRuntimeError>(py));
assert_eq!(
response.value(py).to_string(),
"invalid response: OCR response is missing non-empty content"
);
assert!(!response.value(py).hasattr("status_code").unwrap());
});
}
}

6
litellm-rust/crates/python-bridge/src/routes/ocr/lifecycle.rs
View file

@ -179,7 +179,11 @@ impl PythonRoute for PythonOcrHost {
OcrHostResult::Lifecycle(Ok(()))
}
fn map_error(error: litellm_core::Error) -> PyErr {
fn host_error(message: String) -> litellm_core::ocr::Error {
litellm_core::ocr::Error::InvalidRequest(message)
}
fn map_error(error: litellm_core::ocr::Error) -> PyErr {
ocr_error_to_pyerr(error)
}

2
litellm-rust/crates/python-bridge/src/routes/ocr/project.rs
View file

@ -177,7 +177,7 @@ pub(super) fn admitted_call(outcome: NativeOutcome<OcrCall>) -> PyResult<OcrCall
#[cfg(test)]
mod tests {
use litellm_core::Error;
use litellm_core::ocr::Error;
use litellm_core::ocr::OcrDecline;
use pyo3::exceptions::{PyKeyError, PyTypeError, PyValueError};

2
litellm-rust/crates/python-bridge/src/routes/ocr/value.rs
View file

@ -1,4 +1,4 @@
use litellm_core::Error;
use litellm_core::ocr::Error;
use std::future::Future;
use litellm_core::ocr::wire::{OcrWireRequest, decode_request};