scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-20 00:11:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 134

providers folder is gone

Browse source
This commit is contained in:
Yujong Lee 2026-09-17 07:52:14 -07:00
parent 27ccf7326b
commit b063ffe883
59 changed files with 1002 additions and 973 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
litellm-rust/crates/core/AGENTS.md
View file

@ -1,6 +1,6 @@
litellm-core is the LiteLLM SDK in Rust — it makes the LLM call. Each top-level call is a module under `src/<route>/` exposing a public entrypoint named after the route (`messages::messages()`, the Rust equivalent of `litellm.messages()`): you call it and get a typed non-streaming response back.
A route module owns the call entrypoint, runtime types, provider/auth/URL resolution, and the handler that performs the HTTP call. OCR provider transforms and the base provider trait live under `src/llms/`, mirroring their Python source paths. Other routes still use `src/providers/` and route-local traits. Handlers belong in core, never in a host crate
A route module owns the call entrypoint, runtime types, provider/auth/URL resolution, and the handler that performs the HTTP call. Provider code and base config traits live under `src/llms/`, mirroring their Python source paths. This applies to every API surface: shared orchestration stays in its route module (`ocr/`, `chat_completions/`, `messages/`, `audio_transcription/`, or `responses/`), while provider transformations live under the corresponding Python-mirrored `llms/<provider>/` path. Import implementations directly from their canonical paths; do not add a `src/providers/` layer or compatibility re-exports. Shared provider resolution lives under `src/litellm_core_utils/get_llm_provider_logic.rs`. Handlers belong in core, never in a host crate
Not here: serving HTTP (axum routes, extractors), config file reading, rollout state, databases, or host-specific callback execution. Core owns lifecycle sequencing and callback payload construction; hosts execute the selected integrations. Env reads are limited to credential fallback in a route's `prepare.rs`.
@ -21,3 +21,7 @@ Use named `#[rstest]` cases for independent input/output scenarios instead of lo
For base OCR, Python response models correspond to `src/ocr/types.rs`; Rust context/environment types support the runtime. `BaseOcrConfig::prepare_request` corresponds to Python's HTTP-handler preparation rather than a `BaseOCRConfig` method, and `validate_request_body` is a Rust-only hook
For Mistral, `async_transform_ocr_request` uses the base default in both languages. `resolve_headers` and `build_ocr_url` implement the respective environment and URL operations, and `normalize_response` implements the typed part of response transformation. Existing auth key/header handling and top-level response-extra preservation differ between languages; layout refactors must preserve those behaviors and verify them with the existing tests
For non-OCR pairs, order corresponding methods as parameter support/mapping, environment validation, URL construction, request transformation, and response transformation, followed by Rust-only runtime hooks. Auth resolution remains split between configs and route preparation. Chat `supported_openai_param_mappings` describes accepted OpenAI/provider name pairs, unlike Python's `get_supported_openai_params` name list. Audio `map_transcription_params` remains a Rust filtering helper
Azure Messages maps to `llms/azure_ai/anthropic/messages_transformation.py`; Bedrock Converse maps to `llms/bedrock/chat/converse_transformation.py`. `AnthropicConfig`, `AmazonConverseConfig`, and the non-OCR base traits are partial ports. `OpenAiResponsesApiConfig` currently implements only the WebSocket surface. Preserve their acceptance gates, passthrough behavior, and host fallback contracts when aligning layout

7
litellm-rust/crates/core/src/audio_transcription/handler.rs
View file

@ -36,7 +36,7 @@ pub async fn execute_audio_transcription_provider_call(
.map_err(|error| Error::InvalidResponse(format!("invalid audio response JSON: {error}")))?;
Ok(request
.config
.transform_transcription_response(&request.model, response_json)?
.transform_audio_transcription_response(&request.model, response_json)?
.into_json())
}
@ -47,9 +47,8 @@ async fn signed_headers(
use std::collections::BTreeMap;
use std::time::SystemTime;
use crate::audio_transcription::transformation::AudioTranscriptionAuth;
use crate::providers::bedrock::audio_transcription::aws_auth_config;
use crate::providers::bedrock::aws_base::{resolve_credentials, sign_bedrock_post};
use crate::llms::base_llm::audio_transcription::transformation::AudioTranscriptionAuth;
use litellm_auth_aws::{aws_auth_config, resolve_credentials, sign_bedrock_post};
let AudioTranscriptionAuth::AwsSigV4 { region, .. } = &request.auth else {
return Ok(request.upstream_headers.clone());

1
litellm-rust/crates/core/src/audio_transcription/mod.rs
View file

@ -3,7 +3,6 @@ pub use error::Error;
mod client;
mod handler;
mod prepare;
pub mod transformation;
pub mod types;
pub use handler::execute_audio_transcription_provider_call;

16
litellm-rust/crates/core/src/audio_transcription/prepare.rs
View file

@ -1,11 +1,15 @@
use super::Error;
use super::transformation::{AudioTranscriptionAuth, AudioTranscriptionProviderConfig};
use super::types::{AudioTranscriptionRequest, ProviderAudioTranscriptionRequest};
use crate::http_utils::{has_header, string_headers};
use crate::providers::bedrock::audio_transcription::BEDROCK_AUDIO_TRANSCRIPTION_CONFIG;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};
use crate::litellm_core_utils::get_llm_provider_logic::{
CustomLlmProvider, get_custom_llm_provider,
};
use crate::llms::base_llm::audio_transcription::transformation::{
AudioTranscriptionAuth, BaseAudioTranscriptionConfig,
};
use crate::llms::bedrock::audio_transcription::BEDROCK_AUDIO_TRANSCRIPTION_CONFIG;
fn provider_config(provider: &str) -> Option<&'static dyn AudioTranscriptionProviderConfig> {
fn provider_config(provider: &str) -> Option<&'static dyn BaseAudioTranscriptionConfig> {
if provider == "bedrock" {
return Some(&BEDROCK_AUDIO_TRANSCRIPTION_CONFIG);
}
@ -45,7 +49,7 @@ pub fn prepare_audio_transcription_provider_call(
if !has_header(&headers, "content-type") {
headers.push(("Content-Type".to_string(), "application/json".to_string()));
}
let url = config.complete_url(
let url = config.get_complete_url(
request.api_base,
&model,
&request.optional_params,
@ -53,7 +57,7 @@ pub fn prepare_audio_transcription_provider_call(
)?;
let filtered_params = config.map_transcription_params(&request.optional_params);
let transformed =
config.transform_transcription_request(&model, request.audio, filtered_params)?;
config.transform_audio_transcription_request(&model, request.audio, filtered_params)?;
Ok(ProviderAudioTranscriptionRequest {
model,
custom_llm_provider: provider_info.custom_llm_provider.to_string(),

6
litellm-rust/crates/core/src/audio_transcription/types.rs
View file

@ -3,7 +3,9 @@ use std::time::Duration;
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
use super::transformation::{AudioTranscriptionAuth, AudioTranscriptionProviderConfig};
use crate::llms::base_llm::audio_transcription::transformation::{
AudioTranscriptionAuth, BaseAudioTranscriptionConfig,
};
pub struct AudioTranscriptionRequest<'a> {
pub model: &'a str,
@ -20,7 +22,7 @@ pub struct AudioTranscriptionRequest<'a> {
pub struct ProviderAudioTranscriptionRequest {
pub(super) model: String,
pub(super) custom_llm_provider: String,
pub(super) config: &'static dyn AudioTranscriptionProviderConfig,
pub(super) config: &'static dyn BaseAudioTranscriptionConfig,
pub(super) url: String,
pub(super) body: Value,
pub(super) upstream_headers: Vec<(String, String)>,

10
litellm-rust/crates/core/src/chat_completions/common_utils.rs
View file

@ -1,19 +1,17 @@
use serde_json::{Map, Value};
use super::Error;
use super::transformation::ChatCompletionsProviderConfig;
use crate::http_utils::string_headers as shared_string_headers;
use crate::providers::anthropic::chat_completions::transformation::ANTHROPIC_CHAT_COMPLETIONS_CONFIG;
use crate::llms::anthropic::chat::transformation::ANTHROPIC_CHAT_COMPLETIONS_CONFIG;
use crate::llms::base_llm::chat::transformation::BaseConfig;
const HEADER_CONTEXT: &str = "chat completions";
pub(super) fn chat_completions_provider_config(
provider: &str,
) -> Option<&'static dyn ChatCompletionsProviderConfig> {
pub(super) fn chat_completions_provider_config(provider: &str) -> Option<&'static dyn BaseConfig> {
match provider {
"anthropic" => Some(&ANTHROPIC_CHAT_COMPLETIONS_CONFIG),
"bedrock" => Some(
&crate::providers::bedrock::chat_completions::transformation::BEDROCK_CHAT_COMPLETIONS_CONFIG,
&crate::llms::bedrock::chat::converse_transformation::BEDROCK_CHAT_COMPLETIONS_CONFIG,
),
_ => None,
}

4
litellm-rust/crates/core/src/chat_completions/handler.rs
View file

@ -3,12 +3,12 @@ use serde_json::Value;
use super::Error;
use super::client::http_client;
use super::prepare::prepare_provider_request;
use super::transformation::ChatCompletionsAuth;
use super::types::{
ChatCompletionsResponse, ProviderChatCompletionsRequest, ProviderChatResponseData,
ResolvedChatCompletionsRequest,
};
use crate::http_utils::{http_request, truncate_error_body};
use crate::llms::base_llm::chat::transformation::ChatCompletionsAuth;
pub(super) async fn execute_chat_completions_provider_call(
request: ResolvedChatCompletionsRequest<'_>,
@ -86,7 +86,7 @@ pub(super) async fn signed_headers(
use std::collections::BTreeMap;
use std::time::SystemTime;
use crate::providers::bedrock::aws_base::{
use litellm_auth_aws::{
aws_auth_config, aws_signature_headers, host_supplied_credentials,
is_sigv4_computed_header, resolve_credentials, sign_bedrock_post,
};

1
litellm-rust/crates/core/src/chat_completions/mod.rs
View file

@ -14,7 +14,6 @@ pub mod conversation;
pub(crate) mod handler;
mod prepare;
pub mod response_utils;
pub mod transformation;
pub mod types;
use handler::execute_chat_completions_provider_call;

12
litellm-rust/crates/core/src/chat_completions/prepare.rs
View file

@ -2,18 +2,20 @@ use serde_json::Value;
use super::Error;
use super::common_utils::{chat_completions_provider_config, string_headers};
use super::transformation::{ChatCompletionsAuth, ChatCompletionsProviderConfig};
use super::types::{
ChatCompletionsRequest, ChatMessage, ProviderChatCompletionsRequest,
ResolvedChatCompletionsRequest,
};
use crate::http_utils::has_header;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};
use crate::litellm_core_utils::get_llm_provider_logic::{
CustomLlmProvider, get_custom_llm_provider,
};
use crate::llms::base_llm::chat::transformation::{BaseConfig, ChatCompletionsAuth};
pub(super) fn resolve_provider_config<'a>(
model: &'a str,
custom_llm_provider: Option<&'a str>,
) -> Result<(String, &'static dyn ChatCompletionsProviderConfig), Error> {
) -> Result<(String, &'static dyn BaseConfig), Error> {
let provider_info = get_custom_llm_provider(model, custom_llm_provider)
.or_else(|| {
custom_llm_provider.map(|provider| CustomLlmProvider {
@ -64,7 +66,7 @@ pub(super) fn resolve_request(
fn validate_environment(
request: &ResolvedChatCompletionsRequest<'_>,
model: &str,
config: &dyn ChatCompletionsProviderConfig,
config: &dyn BaseConfig,
) -> Result<(Vec<(String, String)>, ChatCompletionsAuth), Error> {
let env_lookup = |key: &str| std::env::var(key).ok();
let mut headers = string_headers(request.extra_headers.clone())?;
@ -121,7 +123,7 @@ pub(super) fn prepare_provider_request(
let model = request.model;
let config = request.config;
let env_lookup = |key: &str| std::env::var(key).ok();
let url = config.complete_url(
let url = config.get_complete_url(
request.api_base,
&model,
&request.optional_params,

2
litellm-rust/crates/core/src/chat_completions/tests.rs
View file

@ -2,8 +2,8 @@ use serde_json::{Map, Value, json};
use super::Error;
use super::prepare::{prepare_provider_request, resolve_request};
use super::transformation::ChatCompletionsAuth;
use super::types::{ChatCompletionsRequest, ProviderChatCompletionsRequest};
use crate::llms::base_llm::chat::transformation::ChatCompletionsAuth;
fn prepare_chat_completions_call(
request: ChatCompletionsRequest<'_>,

6
litellm-rust/crates/core/src/chat_completions/types.rs
View file

@ -3,7 +3,7 @@ use std::time::Duration;
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
use super::transformation::{ChatCompletionsAuth, ChatCompletionsProviderConfig};
use crate::llms::base_llm::chat::transformation::{BaseConfig, ChatCompletionsAuth};
/// A `/chat/completions` call as it crosses into the core.
///
@ -24,7 +24,7 @@ pub struct ChatCompletionsRequest<'a> {
pub(super) struct ResolvedChatCompletionsRequest<'a> {
pub(super) model: String,
pub(super) config: &'static dyn ChatCompletionsProviderConfig,
pub(super) config: &'static dyn BaseConfig,
pub(super) messages: Vec<ChatMessage>,
pub(super) optional_params: Map<String, Value>,
pub(super) api_key: Option<&'a str>,
@ -35,7 +35,7 @@ pub(super) struct ResolvedChatCompletionsRequest<'a> {
pub(super) struct ProviderChatCompletionsRequest {
pub(super) model: String,
pub(super) config: &'static dyn ChatCompletionsProviderConfig,
pub(super) config: &'static dyn BaseConfig,
pub(super) url: String,
pub(super) body: Value,
pub(super) upstream_headers: Vec<(String, String)>,

4
litellm-rust/crates/core/src/lib.rs
View file

@ -5,12 +5,12 @@ pub mod chat_completions;
pub mod constants;
pub mod error;
pub mod http_utils;
pub(crate) mod llms;
pub mod litellm_core_utils;
pub mod llms;
mod media;
pub mod messages;
pub mod ocr;
pub mod params;
pub mod providers;
pub mod responses;
mod serde_compat;
pub mod transport;

0
litellm-rust/crates/core/src/providers/custom_llm_provider.rs → litellm-rust/crates/core/src/litellm_core_utils/get_llm_provider_logic.rs
View file

1
litellm-rust/crates/core/src/litellm_core_utils/mod.rs Normal file
View file

@ -0,0 +1 @@
pub mod get_llm_provider_logic;

0
litellm-rust/crates/core/src/providers/anthropic/chat_completions/mod.rs → litellm-rust/crates/core/src/llms/anthropic/chat/mod.rs
View file

2
litellm-rust/crates/core/src/providers/anthropic/chat_completions/tests.rs → litellm-rust/crates/core/src/llms/anthropic/chat/tests.rs
View file

@ -420,7 +420,7 @@ fn resolves_the_messages_url_and_x_api_key_auth() {
let config = &ANTHROPIC_CHAT_COMPLETIONS_CONFIG;
assert_eq!(
config
.complete_url(None, "claude-sonnet-4-5", &Map::new(), &|_| None)
.get_complete_url(None, "claude-sonnet-4-5", &Map::new(), &|_| None)
.expect("url builds"),
"https://api.anthropic.com/v1/messages"
);

194
litellm-rust/crates/core/src/providers/anthropic/chat_completions/transformation.rs → litellm-rust/crates/core/src/llms/anthropic/chat/transformation.rs
View file

@ -3,18 +3,17 @@ use serde_json::{Map, Value, json};
use crate::chat_completions::Error;
use crate::chat_completions::conversation::{Conversation, build_conversation};
use crate::chat_completions::response_utils::{finish_reason_for, unix_now, usage_from_parts};
use crate::chat_completions::transformation::{
ChatCompletionsAuth, ChatCompletionsProviderConfig, Unsupported, unsupported_message,
unsupported_param,
};
use crate::chat_completions::types::{
ChatCompletionsChoice, ChatCompletionsChoiceMessage, ChatCompletionsResponse, ChatMessage,
ProviderChatRequestData, ProviderChatResponseData,
};
use crate::constants::ANTHROPIC_OAUTH_TOKEN_PREFIX;
use crate::providers::anthropic::messages::transformation::{
use crate::llms::anthropic::experimental_pass_through::messages::transformation::{
complete_anthropic_url, resolve_anthropic_api_key,
};
use crate::llms::base_llm::chat::transformation::{
BaseConfig, ChatCompletionsAuth, Unsupported, unsupported_message, unsupported_param,
};
/// Anthropic parameter names, post `map_openai_params`, that the Rust path can
/// place verbatim in the Messages body.
@ -33,46 +32,16 @@ const SUPPORTED_PARAMS: &[(&str, &str)] = &[
("stop", "stop_sequences"),
];
pub struct AnthropicChatCompletionsConfig;
pub struct AnthropicConfig;
pub const ANTHROPIC_CHAT_COMPLETIONS_CONFIG: AnthropicChatCompletionsConfig =
AnthropicChatCompletionsConfig;
pub const ANTHROPIC_CHAT_COMPLETIONS_CONFIG: AnthropicConfig = AnthropicConfig;
fn text_block(text: &str) -> Value {
json!({"type": "text", "text": text})
}
impl BaseConfig for AnthropicConfig {
fn supported_openai_param_mappings(&self) -> &'static [(&'static str, &'static str)] {
SUPPORTED_PARAMS
}
fn anthropic_body(model: &str, conversation: &Conversation, params: Map<String, Value>) -> Value {
let messages: Vec<Value> = conversation
.turns
.iter()
.map(|turn| {
json!({
"role": turn.role.as_str(),
"content": turn.texts.iter().map(|text| text_block(text)).collect::<Vec<_>>(),
})
})
.collect();
let system: Vec<Value> = conversation.system.iter().map(|s| text_block(s)).collect();
let body = Map::from_iter(
[
("model".to_string(), json!(model)),
("messages".to_string(), json!(messages)),
]
.into_iter()
// Python builds `{"model", "messages", **optional_params}` with
// `system` already folded into optional_params, so a caller-supplied
// key of the same name wins here too.
.chain((!system.is_empty()).then(|| ("system".to_string(), json!(system))))
.chain(params),
);
Value::Object(body)
}
impl ChatCompletionsProviderConfig for AnthropicChatCompletionsConfig {
fn complete_url(
fn get_complete_url(
&self,
api_base: Option<&str>,
_model: &str,
@ -82,60 +51,6 @@ impl ChatCompletionsProviderConfig for AnthropicChatCompletionsConfig {
Ok(complete_anthropic_url(api_base, env_lookup))
}
fn auth(
&self,
api_key: Option<&str>,
_model: &str,
_optional_params: &Map<String, Value>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<ChatCompletionsAuth, Error> {
Ok(ChatCompletionsAuth::Header {
name: "x-api-key",
value: resolve_anthropic_api_key(api_key, env_lookup)?,
})
}
fn default_headers(&self) -> &'static [(&'static str, &'static str)] {
&[
("anthropic-version", "2023-06-01"),
("content-type", "application/json"),
]
}
/// An OAuth bearer is the whole credential: Python's `validate_environment`
/// authenticates with it and drops `x-api-key` rather than resolving one, so
/// the resolved key must not be applied over the top. Any other forwarded
/// `authorization` is unrelated to this header and does not defer, which is
/// also what Python does: it sends the deployment's `x-api-key` alongside.
fn defers_to_forwarded_auth(&self, headers: &[(String, String)]) -> bool {
headers.iter().any(|(name, value)| {
name.eq_ignore_ascii_case("authorization")
&& value
.strip_prefix("Bearer ")
.is_some_and(|token| token.starts_with(ANTHROPIC_OAUTH_TOKEN_PREFIX))
})
}
fn supported_openai_params(&self) -> &'static [(&'static str, &'static str)] {
SUPPORTED_PARAMS
}
fn unsupported_reason(
&self,
messages: &[ChatMessage],
optional_params: &Map<String, Value>,
) -> Option<Unsupported> {
unsupported_param(self.supported_openai_params(), &[], optional_params)
.or_else(|| messages.iter().find_map(unsupported_message))
// Anthropic rejects a request whose first turn is not a user turn.
// Python only repairs that under `litellm.modify_params`, which the
// core cannot observe, so decline instead of guessing.
.or_else(|| {
(!build_conversation(messages).opens_on_user_turn())
.then_some(Unsupported("conversation does not open on a user turn"))
})
}
fn transform_request(
&self,
model: &str,
@ -209,6 +124,93 @@ impl ChatCompletionsProviderConfig for AnthropicChatCompletionsConfig {
),
})
}
fn auth(
&self,
api_key: Option<&str>,
_model: &str,
_optional_params: &Map<String, Value>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<ChatCompletionsAuth, Error> {
Ok(ChatCompletionsAuth::Header {
name: "x-api-key",
value: resolve_anthropic_api_key(api_key, env_lookup)?,
})
}
fn default_headers(&self) -> &'static [(&'static str, &'static str)] {
&[
("anthropic-version", "2023-06-01"),
("content-type", "application/json"),
]
}
/// An OAuth bearer is the whole credential: Python's `validate_environment`
/// authenticates with it and drops `x-api-key` rather than resolving one, so
/// the resolved key must not be applied over the top. Any other forwarded
/// `authorization` is unrelated to this header and does not defer, which is
/// also what Python does: it sends the deployment's `x-api-key` alongside.
fn defers_to_forwarded_auth(&self, headers: &[(String, String)]) -> bool {
headers.iter().any(|(name, value)| {
name.eq_ignore_ascii_case("authorization")
&& value
.strip_prefix("Bearer ")
.is_some_and(|token| token.starts_with(ANTHROPIC_OAUTH_TOKEN_PREFIX))
})
}
fn unsupported_reason(
&self,
messages: &[ChatMessage],
optional_params: &Map<String, Value>,
) -> Option<Unsupported> {
unsupported_param(self.supported_openai_param_mappings(), &[], optional_params)
.or_else(|| messages.iter().find_map(unsupported_message))
// Anthropic rejects a request whose first turn is not a user turn.
// Python only repairs that under `litellm.modify_params`, which the
// core cannot observe, so decline instead of guessing.
.or_else(|| {
(!build_conversation(messages).opens_on_user_turn())
.then_some(Unsupported("conversation does not open on a user turn"))
})
}
}
fn text_block(text: &str) -> Value {
json!({"type": "text", "text": text})
}
fn anthropic_body(
model: &str,
conversation: &Conversation,
optional_params: Map<String, Value>,
) -> Value {
let messages: Vec<Value> = conversation
.turns
.iter()
.map(|turn| {
json!({
"role": turn.role.as_str(),
"content": turn.texts.iter().map(|text| text_block(text)).collect::<Vec<_>>(),
})
})
.collect();
let system: Vec<Value> = conversation.system.iter().map(|s| text_block(s)).collect();
let body = Map::from_iter(
[
("model".to_string(), json!(model)),
("messages".to_string(), json!(messages)),
]
.into_iter()
// Python builds `{"model", "messages", **optional_params}` with
// `system` already folded into optional_params, so a caller-supplied
// key of the same name wins here too.
.chain((!system.is_empty()).then(|| ("system".to_string(), json!(system))))
.chain(optional_params),
);
Value::Object(body)
}
#[cfg(test)]

0
litellm-rust/crates/core/src/providers/anthropic/messages/mod.rs → litellm-rust/crates/core/src/llms/anthropic/experimental_pass_through/messages/mod.rs
View file

44
litellm-rust/crates/core/src/providers/anthropic/messages/transformation.rs → litellm-rust/crates/core/src/llms/anthropic/experimental_pass_through/messages/transformation.rs
View file

@ -1,5 +1,5 @@
use crate::llms::base_llm::anthropic_messages::transformation::BaseAnthropicMessagesConfig;
use crate::messages::Error;
use crate::messages::transformation::{AnthropicMessagesProviderConfig, MessagesAuthStrategy};
const ANTHROPIC_API_KEY_ENV: &str = "ANTHROPIC_API_KEY";
const ANTHROPIC_API_BASE_ENV: &str = "ANTHROPIC_API_BASE";
@ -10,6 +10,25 @@ pub struct AnthropicMessagesConfig;
pub const ANTHROPIC_MESSAGES_CONFIG: AnthropicMessagesConfig = AnthropicMessagesConfig;
impl BaseAnthropicMessagesConfig for AnthropicMessagesConfig {
fn get_complete_url(
&self,
api_base: Option<&str>,
_model: &str,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
Ok(complete_anthropic_url(api_base, env_lookup))
}
fn resolve_api_key(
&self,
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
resolve_anthropic_api_key(api_key, env_lookup).map_err(Error::from)
}
}
pub fn non_empty(value: Option<&str>) -> Option<&str> {
value.map(str::trim).filter(|value| !value.is_empty())
}
@ -43,29 +62,6 @@ pub fn complete_anthropic_url(
format!("{api_base}{MESSAGES_PATH_SUFFIX}")
}
impl AnthropicMessagesProviderConfig for AnthropicMessagesConfig {
fn complete_url(
&self,
api_base: Option<&str>,
_model: &str,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
Ok(complete_anthropic_url(api_base, env_lookup))
}
fn resolve_api_key(
&self,
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
resolve_anthropic_api_key(api_key, env_lookup).map_err(Error::from)
}
fn auth_strategy(&self) -> MessagesAuthStrategy {
MessagesAuthStrategy::Header("x-api-key")
}
}
#[cfg(test)]
mod tests {
use super::*;

0
litellm-rust/crates/core/src/providers/azure_ai/mod.rs → litellm-rust/crates/core/src/llms/anthropic/experimental_pass_through/mod.rs
View file

2
litellm-rust/crates/core/src/llms/anthropic/mod.rs Normal file
View file

@ -0,0 +1,2 @@
pub mod chat;
pub mod experimental_pass_through;

135
litellm-rust/crates/core/src/providers/azure_ai/messages/transformation.rs → litellm-rust/crates/core/src/llms/azure_ai/anthropic/messages_transformation.rs
View file

@ -1,14 +1,16 @@
use serde_json::{Map, Value};
use crate::llms::anthropic::experimental_pass_through::messages::transformation::{
ANTHROPIC_MESSAGES_CONFIG, AnthropicMessagesConfig, non_empty,
};
use crate::llms::base_llm::anthropic_messages::transformation::{
BaseAnthropicMessagesConfig, MessagesAuthStrategy,
};
use crate::messages::Error;
use crate::messages::transformation::{AnthropicMessagesProviderConfig, MessagesAuthStrategy};
use crate::messages::types::{
AnthropicMessage, AnthropicMessagesRequest, AnthropicMessagesResponse, ContentBlock,
MessageContent, SystemPrompt,
};
use crate::providers::anthropic::messages::transformation::{
ANTHROPIC_MESSAGES_CONFIG, AnthropicMessagesConfig, non_empty,
};
const AZURE_API_KEY_ENV: &str = "AZURE_API_KEY";
const AZURE_API_BASE_ENV: &str = "AZURE_API_BASE";
@ -26,6 +28,61 @@ pub const AZURE_ANTHROPIC_MESSAGES_CONFIG: AzureAnthropicMessagesConfig =
anthropic: ANTHROPIC_MESSAGES_CONFIG,
};
impl BaseAnthropicMessagesConfig for AzureAnthropicMessagesConfig {
fn get_complete_url(
&self,
api_base: Option<&str>,
_model: &str,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
complete_azure_anthropic_url(api_base, env_lookup)
}
fn transform_anthropic_messages_request(
&self,
request: AnthropicMessagesRequest,
) -> Result<AnthropicMessagesRequest, Error> {
let mut request = fold_system_role_messages(request);
if let Some(system) = request.system.as_mut() {
strip_scope_from_system(system);
}
request
.messages
.iter_mut()
.for_each(strip_scope_from_message);
self.anthropic.transform_anthropic_messages_request(request)
}
fn transform_anthropic_messages_response(
&self,
model: &str,
response: AnthropicMessagesResponse,
) -> Result<AnthropicMessagesResponse, Error> {
self.anthropic
.transform_anthropic_messages_response(model, response)
}
fn resolve_api_key(
&self,
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
resolve_azure_api_key(api_key, env_lookup)
}
fn auth_strategy(&self) -> MessagesAuthStrategy {
self.anthropic.auth_strategy()
}
fn accepts_bearer_auth(&self) -> bool {
true
}
fn default_headers(&self) -> &'static [(&'static str, &'static str)] {
self.anthropic.default_headers()
}
}
pub fn resolve_azure_api_key(
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
@ -136,60 +193,6 @@ fn fold_system_role_messages(request: AnthropicMessagesRequest) -> AnthropicMess
}
}
impl AnthropicMessagesProviderConfig for AzureAnthropicMessagesConfig {
fn complete_url(
&self,
api_base: Option<&str>,
_model: &str,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
complete_azure_anthropic_url(api_base, env_lookup)
}
fn resolve_api_key(
&self,
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error> {
resolve_azure_api_key(api_key, env_lookup)
}
fn auth_strategy(&self) -> MessagesAuthStrategy {
self.anthropic.auth_strategy()
}
fn accepts_bearer_auth(&self) -> bool {
true
}
fn default_headers(&self) -> &'static [(&'static str, &'static str)] {
self.anthropic.default_headers()
}
fn transform_request(
&self,
request: AnthropicMessagesRequest,
) -> Result<AnthropicMessagesRequest, Error> {
let mut request = fold_system_role_messages(request);
if let Some(system) = request.system.as_mut() {
strip_scope_from_system(system);
}
request
.messages
.iter_mut()
.for_each(strip_scope_from_message);
self.anthropic.transform_request(request)
}
fn transform_response(
&self,
model: &str,
response: AnthropicMessagesResponse,
) -> Result<AnthropicMessagesResponse, Error> {
self.anthropic.transform_response(model, response)
}
}
#[cfg(test)]
mod tests {
use serde_json::json;
@ -339,7 +342,7 @@ mod tests {
let transformed = to_value(
AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(request)
.transform_anthropic_messages_request(request)
.expect("request transforms"),
);
@ -366,10 +369,10 @@ mod tests {
"messages": [{"role": "user", "content": "hi"}]
}));
let once = AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(request)
.transform_anthropic_messages_request(request)
.expect("request transforms");
let twice = AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(once.clone())
.transform_anthropic_messages_request(once.clone())
.expect("request transforms");
assert_eq!(once, twice);
assert_eq!(to_value(once)["system"], json!("plain string system"));
@ -403,7 +406,7 @@ mod tests {
});
let transformed = to_value(
AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(request_from(body.clone()))
.transform_anthropic_messages_request(request_from(body.clone()))
.expect("request transforms"),
);
assert_eq!(transformed, body);
@ -423,7 +426,7 @@ mod tests {
let transformed = to_value(
AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(request)
.transform_anthropic_messages_request(request)
.expect("request transforms"),
);
@ -453,7 +456,7 @@ mod tests {
let transformed = to_value(
AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(request)
.transform_anthropic_messages_request(request)
.expect("request transforms"),
);
@ -480,7 +483,7 @@ mod tests {
});
let transformed = to_value(
AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_request(request_from(body.clone()))
.transform_anthropic_messages_request(request_from(body.clone()))
.expect("request transforms"),
);
assert_eq!(transformed, body);
@ -507,7 +510,7 @@ mod tests {
}))
.expect("valid response");
let transformed = AZURE_ANTHROPIC_MESSAGES_CONFIG
.transform_response("claude-sonnet-4-5", response)
.transform_anthropic_messages_response("claude-sonnet-4-5", response)
.expect("response transforms");
let value = serde_json::to_value(transformed).expect("serializable");
assert_eq!(value["stop_reason"], json!("end_turn"));

1
litellm-rust/crates/core/src/llms/azure_ai/anthropic/mod.rs Normal file
View file

@ -0,0 +1 @@
pub mod messages_transformation;

1
litellm-rust/crates/core/src/llms/azure_ai/mod.rs
View file

@ -1 +1,2 @@
pub mod anthropic;
pub(crate) mod ocr;

6
litellm-rust/crates/core/src/llms/azure_ai/ocr/cohere_parse_transformation.rs
View file

@ -18,7 +18,7 @@ impl BaseOcrConfig for AzureAICohereParseConfig {
type Environment = Vec<(String, String)>;
fn get_api_key_env_var(&self) -> Option<&'static str> {
super::transformation::AzureAIOCRConfig.get_api_key_env_var()
super::transformation::AzureAiOcrConfig.get_api_key_env_var()
}
fn get_health_check_document(&self) -> OcrDocument {
@ -31,7 +31,7 @@ impl BaseOcrConfig for AzureAICohereParseConfig {
client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
BaseOcrConfig::validate_environment(
&super::transformation::AzureAIOCRConfig,
&super::transformation::AzureAiOcrConfig,
request,
client,
)
@ -44,7 +44,7 @@ impl BaseOcrConfig for AzureAICohereParseConfig {
_params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
let base = super::transformation::AzureAIOCRConfig::resolve_api_base(
let base = super::transformation::AzureAiOcrConfig::resolve_api_base(
request.connection.api_base.as_deref(),
&crate::ocr::prepare::credential_env,
)?;

288
litellm-rust/crates/core/src/llms/azure_ai/ocr/document_intelligence/transformation.rs
View file

@ -17,7 +17,7 @@ use crate::constants::{
AZURE_DI_SUBSCRIPTION_HEADER, OCR_POLL_RETRY_SECS,
};
use crate::llms::base_llm::ocr::transformation::{
BaseOcrConfig, OcrRequestContext, OcrResponseContext,
BaseOcrConfig, OcrResponseContext, decode_and_normalize_response,
};
use crate::ocr::OcrClient;
use crate::ocr::client::read_json_response;
@ -32,6 +32,9 @@ use crate::ocr::types::{
use crate::serde_compat::{FiniteF64, LaxI64};
use crate::url_utils::ApiUrl;
const AZURE_DI_API_KEY_ENV: &str = "AZURE_DOCUMENT_INTELLIGENCE_API_KEY";
const AZURE_DI_ENDPOINT_ENV: &str = "AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT";
#[derive(Clone, Debug, PartialEq, Serialize)]
pub(crate) struct DocumentIntelligenceParams {
#[serde(skip_serializing_if = "Option::is_none")]
@ -123,7 +126,126 @@ struct AzureDocumentIntelligenceLine {
pub content: Option<String>,
}
fn normalize_pages(pages: Option<&Value>) -> Result<Option<String>, crate::ocr::Error> {
#[derive(Clone, Debug)]
pub(crate) struct AzureDocumentIntelligenceOcrConfig;
impl BaseOcrConfig for AzureDocumentIntelligenceOcrConfig {
type OcrParams = DocumentIntelligenceParams;
type ProviderRequest = DocumentIntelligenceRequest;
type Environment = Vec<(String, String)>;
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["pages", "features", "req_format"]
}
fn get_api_key_env_var(&self) -> Option<&'static str> {
Some(AZURE_DI_API_KEY_ENV)
}
fn resolve_connection_params(&self, inputs: OcrCredentialInputs) -> ResolvedOcrCredentials {
ResolvedOcrCredentials {
api_key: inputs.api_key.and_then(|key| {
inputs
.dynamic_api_key
.filter(|value| !value.value().is_empty())
.or(Some(key))
}),
api_base: inputs.api_base.and_then(|base| {
inputs
.dynamic_api_base
.filter(|value| !value.value().is_empty())
.or(Some(base))
}),
}
}
fn map_ocr_params(
&self,
non_default_params: &CallArguments,
_model: &str,
) -> Result<DocumentIntelligenceParams, crate::ocr::Error> {
Ok(DocumentIntelligenceParams {
pages: normalize_pages_param(non_default_params.get("pages"))?,
features: normalize_features_param(non_default_params.get("features"))?,
})
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
_client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
let config = AzureAuthInputs {
azure_ad_token_provider: request.azure_ad_token_provider.clone(),
..AzureAuthInputs::from_sourced_optional_params(
&request.optional_params,
&request.input_sources,
)?
};
self.resolve_headers(&request.connection, &config, &credential_env)
.await
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
optional_params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
let endpoint = nonblank(request.connection.api_base.clone())
.or_else(|| nonblank(credential_env(AZURE_DI_ENDPOINT_ENV)))
.ok_or_else(|| crate::ocr::Error::Auth(litellm_auth::Error::ProviderAuthentication("Missing Azure Document Intelligence API Base - Set AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT or pass api_base".into())))?;
self.build_ocr_url(&endpoint, &request.model, optional_params)
}
fn transform_ocr_request(
&self,
_model: &str,
document: OcrDocument,
_optional_params: &DocumentIntelligenceParams,
_headers: &[(String, String)],
) -> Result<DocumentIntelligenceRequest, crate::ocr::Error> {
build_request(document)
}
fn transform_ocr_response(
&self,
model: &str,
raw_response: &[u8],
request_format: OcrResponseFormat,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
decode_and_normalize_response(
model,
raw_response,
request_format,
transform_completed_response,
)
}
async fn async_transform_ocr_response(
&self,
model: &str,
raw_response: reqwest::Response,
context: OcrResponseContext<'_>,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
let decoded = read_operation_response(
context.client.polling_http(),
raw_response,
context.url,
context.headers,
context.connection,
context.request_format == OcrResponseFormat::Native,
context.hooks,
)
.await?;
Ok(LiteLLMOcrResponse {
provider_native_response: decoded.native,
..transform_completed_response(model, decoded.data)?
})
}
}
fn normalize_pages_param(pages: Option<&Value>) -> Result<Option<String>, crate::ocr::Error> {
let normalized = match pages {
None | Some(Value::Null) => return Ok(None),
Some(Value::Array(pages)) if pages.is_empty() => return Ok(None),
@ -186,7 +308,7 @@ fn valid_page_token(token: &str) -> bool {
}
}
fn normalize_features(features: Option<&Value>) -> Result<Option<String>, crate::ocr::Error> {
fn normalize_features_param(features: Option<&Value>) -> Result<Option<String>, crate::ocr::Error> {
let tokens = match features {
None | Some(Value::Null) => return Ok(None),
Some(Value::Array(names)) => names
@ -414,140 +536,8 @@ async fn poll_operation(
}
}
const AZURE_DI_API_KEY_ENV: &str = "AZURE_DOCUMENT_INTELLIGENCE_API_KEY";
const AZURE_DI_ENDPOINT_ENV: &str = "AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT";
#[derive(Clone, Debug)]
pub(crate) struct AzureDocumentIntelligenceOCRConfig;
impl BaseOcrConfig for AzureDocumentIntelligenceOCRConfig {
type OcrParams = DocumentIntelligenceParams;
type ProviderRequest = DocumentIntelligenceRequest;
type Environment = Vec<(String, String)>;
fn get_api_key_env_var(&self) -> Option<&'static str> {
Some(AZURE_DI_API_KEY_ENV)
}
fn resolve_connection_params(&self, inputs: OcrCredentialInputs) -> ResolvedOcrCredentials {
ResolvedOcrCredentials {
api_key: inputs.api_key.and_then(|key| {
inputs
.dynamic_api_key
.filter(|value| !value.value().is_empty())
.or(Some(key))
}),
api_base: inputs.api_base.and_then(|base| {
inputs
.dynamic_api_base
.filter(|value| !value.value().is_empty())
.or(Some(base))
}),
}
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
_client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
let config = AzureAuthInputs {
azure_ad_token_provider: request.azure_ad_token_provider.clone(),
..AzureAuthInputs::from_sourced_optional_params(
&request.optional_params,
&request.input_sources,
)?
};
self.validate_environment(&request.connection, &config, &credential_env)
.await
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
let endpoint = nonblank(request.connection.api_base.clone())
.or_else(|| nonblank(credential_env(AZURE_DI_ENDPOINT_ENV)))
.ok_or_else(|| crate::ocr::Error::Auth(litellm_auth::Error::ProviderAuthentication("Missing Azure Document Intelligence API Base - Set AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT or pass api_base".into())))?;
self.get_complete_url(&endpoint, &request.model, params)
}
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["pages", "features", "req_format"]
}
fn map_ocr_params(
&self,
arguments: &CallArguments,
_model: &str,
) -> Result<DocumentIntelligenceParams, crate::ocr::Error> {
Ok(DocumentIntelligenceParams {
pages: normalize_pages(arguments.get("pages"))?,
features: normalize_features(arguments.get("features"))?,
})
}
async fn async_transform_ocr_request(
&self,
model: &str,
document: OcrDocument,
optional_params: &DocumentIntelligenceParams,
headers: &[(String, String)],
_context: OcrRequestContext<'_>,
) -> Result<DocumentIntelligenceRequest, crate::ocr::Error> {
self.transform_ocr_request(model, document, optional_params, headers)
}
fn transform_ocr_response(
&self,
model: &str,
raw_response: &[u8],
request_format: OcrResponseFormat,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
crate::llms::base_llm::ocr::transformation::decode_and_normalize_response(
model,
raw_response,
request_format,
transform_completed_response,
)
}
async fn async_transform_ocr_response(
&self,
model: &str,
raw_response: reqwest::Response,
context: OcrResponseContext<'_>,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
let decoded = read_operation_response(
context.client.polling_http(),
raw_response,
context.url,
context.headers,
context.connection,
context.request_format == OcrResponseFormat::Native,
context.hooks,
)
.await?;
Ok(LiteLLMOcrResponse {
provider_native_response: decoded.native,
..transform_completed_response(model, decoded.data)?
})
}
fn transform_ocr_request(
&self,
_model: &str,
document: OcrDocument,
_optional_params: &DocumentIntelligenceParams,
_headers: &[(String, String)],
) -> Result<DocumentIntelligenceRequest, crate::ocr::Error> {
build_request(document)
}
}
impl AzureDocumentIntelligenceOCRConfig {
fn get_complete_url(
impl AzureDocumentIntelligenceOcrConfig {
fn build_ocr_url(
&self,
endpoint: &str,
model: &str,
@ -575,7 +565,7 @@ impl AzureDocumentIntelligenceOCRConfig {
})
}
async fn validate_environment(
async fn resolve_headers(
&self,
connection: &OcrConnection,
config: &AzureAuthInputs,
@ -642,7 +632,7 @@ mod tests {
fn map(value: Value) -> Result<DocumentIntelligenceParams, crate::ocr::Error> {
let arguments = serde_json::from_value(value).unwrap();
AzureDocumentIntelligenceOCRConfig.map_ocr_params(&arguments, "model")
AzureDocumentIntelligenceOcrConfig.map_ocr_params(&arguments, "model")
}
#[test]
@ -650,7 +640,7 @@ mod tests {
let overrides =
serde_json::from_value(json!({"pages":[], "features":null, "req_format":"native"}))
.unwrap();
let mapped = AzureDocumentIntelligenceOCRConfig
let mapped = AzureDocumentIntelligenceOcrConfig
.map_ocr_params(&overrides, "model")
.unwrap();
assert_eq!(serde_json::to_value(mapped).unwrap(), json!({}));
@ -664,7 +654,7 @@ mod tests {
"extra_body": {"provider_option": "value"}
}))
.unwrap();
let mapped = AzureDocumentIntelligenceOCRConfig
let mapped = AzureDocumentIntelligenceOcrConfig
.map_ocr_params(&arguments, "model")
.unwrap();
assert_eq!(mapped.pages.as_deref(), Some("1"));
@ -680,7 +670,7 @@ mod tests {
"pages":"4", "features":"languages", "extension":true
}))
.unwrap();
let mapped = AzureDocumentIntelligenceOCRConfig
let mapped = AzureDocumentIntelligenceOcrConfig
.map_ocr_params(&arguments, "model")
.unwrap();
assert_eq!(
@ -694,7 +684,7 @@ mod tests {
#[test]
fn response_numbers_follow_python_validation_before_dimension_conversion() {
let response = AzureDocumentIntelligenceOCRConfig.transform_ocr_response(
let response = AzureDocumentIntelligenceOcrConfig.transform_ocr_response(
"model",
br#"{"status":"succeeded","analyzeResult":{"pages":[{"pageNumber":2.0,"width":" 8.5 ","height":true}]}}"#,
OcrResponseFormat::Litellm,
@ -703,6 +693,10 @@ mod tests {
let dimensions = response.pages[0].dimensions.as_ref().unwrap();
assert_eq!(dimensions.width, Some(816));
assert_eq!(dimensions.height, Some(96));
}
#[test]
fn pixel_dimension_rejects_out_of_range_value() {
assert!(pixel_dimension(9_223_372_036_854_775_808.0, 1.0, "width").is_err());
}
@ -776,8 +770,8 @@ mod tests {
..Default::default()
};
let error = AzureDocumentIntelligenceOCRConfig
.validate_environment(&connection, &Default::default(), &|name| {
let error = AzureDocumentIntelligenceOcrConfig
.resolve_headers(&connection, &Default::default(), &|name| {
(name == AZURE_DI_API_KEY_ENV).then(|| "environment-key".into())
})
.await
@ -800,8 +794,8 @@ mod tests {
..Default::default()
};
let headers = AzureDocumentIntelligenceOCRConfig
.validate_environment(&connection, &Default::default(), &|_| None)
let headers = AzureDocumentIntelligenceOcrConfig
.resolve_headers(&connection, &Default::default(), &|_| None)
.await
.unwrap();

143
litellm-rust/crates/core/src/llms/azure_ai/ocr/transformation.rs
View file

@ -17,17 +17,29 @@ const AZURE_AI_API_KEY_ENV: &str = "AZURE_AI_API_KEY";
const AZURE_AI_API_BASE_ENV: &str = "AZURE_AI_API_BASE";
#[derive(Clone, Debug, Default)]
pub(crate) struct AzureAIOCRConfig;
pub(crate) struct AzureAiOcrConfig;
impl BaseOcrConfig for AzureAIOCRConfig {
impl BaseOcrConfig for AzureAiOcrConfig {
type OcrParams = OpaqueParams;
type ProviderRequest = MistralOcrRequest;
type Environment = Vec<(String, String)>;
fn get_supported_ocr_params(&self, model: &str) -> &'static [&'static str] {
MistralOcrConfig.get_supported_ocr_params(model)
}
fn get_api_key_env_var(&self) -> Option<&'static str> {
Some(AZURE_AI_API_KEY_ENV)
}
fn map_ocr_params(
&self,
non_default_params: &CallArguments,
model: &str,
) -> Result<OpaqueParams, crate::ocr::Error> {
MistralOcrConfig.map_ocr_params(non_default_params, model)
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
@ -40,39 +52,27 @@ impl BaseOcrConfig for AzureAIOCRConfig {
&request.input_sources,
)?
};
self.validate_environment(&request.connection, &config, &credential_env)
self.resolve_headers(&request.connection, &config, &credential_env)
.await
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
_params: &Self::OcrParams,
_optional_params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
self.get_complete_url(request.connection.api_base.as_deref(), &credential_env)
self.build_ocr_url(request.connection.api_base.as_deref(), &credential_env)
}
fn transform_ocr_request(
&self,
model: &str,
document: OcrDocument,
params: &OpaqueParams,
optional_params: &OpaqueParams,
headers: &[(String, String)],
) -> Result<MistralOcrRequest, crate::ocr::Error> {
MistralOcrConfig.transform_ocr_request(model, document, params, headers)
}
fn get_supported_ocr_params(&self, model: &str) -> &'static [&'static str] {
MistralOcrConfig.get_supported_ocr_params(model)
}
fn map_ocr_params(
&self,
arguments: &CallArguments,
model: &str,
) -> Result<OpaqueParams, crate::ocr::Error> {
MistralOcrConfig.map_ocr_params(arguments, model)
MistralOcrConfig.transform_ocr_request(model, document, optional_params, headers)
}
async fn async_transform_ocr_request(
@ -106,7 +106,7 @@ impl BaseOcrConfig for AzureAIOCRConfig {
}
}
impl AzureAIOCRConfig {
impl AzureAiOcrConfig {
/// Python `AzureAIOCRConfig.validate_environment` requires the endpoint
/// before it resolves credentials; keep that order so a missing base is
/// reported without invoking any token provider.
@ -124,22 +124,7 @@ impl AzureAIOCRConfig {
))
}
fn get_complete_url(
&self,
api_base: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, crate::ocr::Error> {
let base = Self::resolve_api_base(api_base, env_lookup)?;
let path: Vec<&str> = AZURE_AI_OCR_PATH.trim_matches('/').split('/').collect();
ApiUrl::parse(&base)
.and_then(|url| url.complete_path(&path))
.map(|url| url.into_string())
.map_err(|_| crate::ocr::Error::RequestField {
path: "api_base".into(),
})
}
pub(super) async fn validate_environment(
async fn resolve_headers(
&self,
connection: &OcrConnection,
config: &AzureAuthInputs,
@ -169,6 +154,21 @@ impl AzureAIOCRConfig {
super::common_utils::validate_destination(connection, key.source())?;
Ok(bearer_headers(connection, key.value()))
}
fn build_ocr_url(
&self,
api_base: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, crate::ocr::Error> {
let base = Self::resolve_api_base(api_base, env_lookup)?;
let path: Vec<&str> = AZURE_AI_OCR_PATH.trim_matches('/').split('/').collect();
ApiUrl::parse(&base)
.and_then(|url| url.complete_path(&path))
.map(|url| url.into_string())
.map_err(|_| crate::ocr::Error::RequestField {
path: "api_base".into(),
})
}
}
fn bearer_headers(connection: &OcrConnection, key: &str) -> Vec<(String, String)> {
@ -185,31 +185,41 @@ fn nonblank(value: Option<String>) -> Option<String> {
#[cfg(test)]
mod tests {
use rstest::{fixture, rstest};
use super::*;
#[test]
fn completes_azure_path_and_preserves_query() {
#[fixture]
fn connection() -> OcrConnection {
OcrConnection {
api_key: Some("request-key".into()),
api_base: Some("https://example.com".into()),
..Default::default()
}
}
#[rstest]
#[case::base_with_query(
"https://example.com/?tenant=a",
"https://example.com/providers/mistral/azure/ocr?tenant=a"
)]
#[case::complete_endpoint(
"https://example.com/providers/mistral/azure/ocr",
"https://example.com/providers/mistral/azure/ocr"
)]
fn completes_azure_path_and_preserves_query(#[case] api_base: &str, #[case] expected: &str) {
assert_eq!(
AzureAIOCRConfig
.get_complete_url(Some("https://example.com/?tenant=a"), &|_| None)
AzureAiOcrConfig
.build_ocr_url(Some(api_base), &|_| None)
.unwrap(),
"https://example.com/providers/mistral/azure/ocr?tenant=a"
);
assert_eq!(
AzureAIOCRConfig
.get_complete_url(
Some("https://example.com/providers/mistral/azure/ocr"),
&|_| None
)
.unwrap(),
"https://example.com/providers/mistral/azure/ocr"
expected
);
}
#[test]
fn missing_api_base_is_structured() {
assert!(matches!(
AzureAIOCRConfig::resolve_api_base(None, &|_| None),
AzureAiOcrConfig::resolve_api_base(None, &|_| None),
Err(crate::ocr::Error::Auth(
litellm_auth::Error::MissingApiBase {
provider: "Azure AI",
@ -219,17 +229,16 @@ mod tests {
));
}
#[rstest]
#[tokio::test]
async fn supplied_authorization_precedes_keys() {
async fn supplied_authorization_precedes_keys(connection: OcrConnection) {
let connection = OcrConnection {
api_key: Some("request-key".into()),
api_base: Some("https://example.com".into()),
extra_headers: vec![("authorization".into(), "Bearer prepared".into())],
..Default::default()
..connection
};
assert_eq!(
AzureAIOCRConfig
.validate_environment(&connection, &Default::default(), &|_| {
AzureAiOcrConfig
.resolve_headers(&connection, &Default::default(), &|_| {
Some("environment-key".into())
})
.await
@ -238,16 +247,12 @@ mod tests {
);
}
#[rstest]
#[tokio::test]
async fn request_key_precedes_environment_key() {
let connection = OcrConnection {
api_key: Some("request-key".into()),
api_base: Some("https://example.com".into()),
..Default::default()
};
async fn request_key_precedes_environment_key(connection: OcrConnection) {
assert_eq!(
AzureAIOCRConfig
.validate_environment(&connection, &Default::default(), &|_| {
AzureAiOcrConfig
.resolve_headers(&connection, &Default::default(), &|_| {
Some("environment-key".into())
})
.await
@ -264,8 +269,8 @@ mod tests {
..Default::default()
};
let error = AzureAIOCRConfig
.validate_environment(&connection, &Default::default(), &|name| {
let error = AzureAiOcrConfig
.resolve_headers(&connection, &Default::default(), &|name| {
(name == AZURE_AI_API_KEY_ENV).then(|| "environment-key".into())
})
.await
@ -288,8 +293,8 @@ mod tests {
..Default::default()
};
let headers = AzureAIOCRConfig
.validate_environment(&connection, &Default::default(), &|_| None)
let headers = AzureAiOcrConfig
.resolve_headers(&connection, &Default::default(), &|_| None)
.await
.unwrap();

0
litellm-rust/crates/core/src/providers/azure_ai/messages/mod.rs → litellm-rust/crates/core/src/llms/base_llm/anthropic_messages/mod.rs
View file

38
litellm-rust/crates/core/src/messages/transformation.rs → litellm-rust/crates/core/src/llms/base_llm/anthropic_messages/transformation.rs
View file

@ -1,5 +1,5 @@
use super::Error;
use super::types::{AnthropicMessagesRequest, AnthropicMessagesResponse};
use crate::messages::Error;
use crate::messages::types::{AnthropicMessagesRequest, AnthropicMessagesResponse};
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MessagesAuthStrategy {
@ -16,14 +16,29 @@ impl MessagesAuthStrategy {
}
}
pub trait AnthropicMessagesProviderConfig: Sync {
fn complete_url(
pub trait BaseAnthropicMessagesConfig: Sync {
fn get_complete_url(
&self,
api_base: Option<&str>,
model: &str,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error>;
fn transform_anthropic_messages_request(
&self,
request: AnthropicMessagesRequest,
) -> Result<AnthropicMessagesRequest, Error> {
Ok(request)
}
fn transform_anthropic_messages_response(
&self,
_model: &str,
response: AnthropicMessagesResponse,
) -> Result<AnthropicMessagesResponse, Error> {
Ok(response)
}
fn resolve_api_key(
&self,
api_key: Option<&str>,
@ -44,19 +59,4 @@ pub trait AnthropicMessagesProviderConfig: Sync {
("content-type", "application/json"),
]
}
fn transform_request(
&self,
request: AnthropicMessagesRequest,
) -> Result<AnthropicMessagesRequest, Error> {
Ok(request)
}
fn transform_response(
&self,
_model: &str,
response: AnthropicMessagesResponse,
) -> Result<AnthropicMessagesResponse, Error> {
Ok(response)
}
}

0
litellm-rust/crates/core/src/providers/bedrock/chat_completions/mod.rs → litellm-rust/crates/core/src/llms/base_llm/audio_transcription/mod.rs
View file

50
litellm-rust/crates/core/src/audio_transcription/transformation.rs → litellm-rust/crates/core/src/llms/base_llm/audio_transcription/transformation.rs
View file

@ -1,7 +1,9 @@
use serde_json::{Map, Value};
use super::Error;
use super::types::{AudioTranscriptionRequestData, AudioTranscriptionResponseData};
use crate::audio_transcription::Error;
use crate::audio_transcription::types::{
AudioTranscriptionRequestData, AudioTranscriptionResponseData,
};
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum AudioTranscriptionAuth {
@ -12,34 +14,21 @@ pub enum AudioTranscriptionAuth {
},
}
pub trait AudioTranscriptionProviderConfig: Sync {
fn supported_transcription_params(&self) -> &'static [&'static str];
pub trait BaseAudioTranscriptionConfig: Sync {
fn get_supported_openai_params(&self) -> &'static [&'static str];
fn map_transcription_params(&self, params: &Map<String, Value>) -> Map<String, Value> {
params
fn map_transcription_params(
&self,
non_default_params: &Map<String, Value>,
) -> Map<String, Value> {
non_default_params
.iter()
.filter(|(key, _)| {
self.supported_transcription_params()
.contains(&key.as_str())
})
.filter(|(key, _)| self.get_supported_openai_params().contains(&key.as_str()))
.map(|(key, value)| (key.clone(), value.clone()))
.collect()
}
fn transform_transcription_request(
&self,
model: &str,
audio: Value,
optional_params: Map<String, Value>,
) -> Result<AudioTranscriptionRequestData, Error>;
fn transform_transcription_response(
&self,
model: &str,
response_json: Value,
) -> Result<AudioTranscriptionResponseData, Error>;
fn complete_url(
fn get_complete_url(
&self,
api_base: Option<&str>,
model: &str,
@ -47,6 +36,19 @@ pub trait AudioTranscriptionProviderConfig: Sync {
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error>;
fn transform_audio_transcription_request(
&self,
model: &str,
audio: Value,
optional_params: Map<String, Value>,
) -> Result<AudioTranscriptionRequestData, Error>;
fn transform_audio_transcription_response(
&self,
model: &str,
response_json: Value,
) -> Result<AudioTranscriptionResponseData, Error>;
fn auth_strategy(
&self,
model: &str,

0
litellm-rust/crates/core/src/providers/openai/responses/mod.rs → litellm-rust/crates/core/src/llms/base_llm/chat/mod.rs
View file

52
litellm-rust/crates/core/src/chat_completions/transformation.rs → litellm-rust/crates/core/src/llms/base_llm/chat/transformation.rs
View file

@ -1,11 +1,17 @@
use serde_json::{Map, Value};
use super::Error;
use super::types::{
use crate::chat_completions::Error;
use crate::chat_completions::types::{
ChatCompletionsResponse, ChatMessage, ChatMessageContent, ProviderChatRequestData,
ProviderChatResponseData,
};
pub const STREAM_PARAM: &str = "stream";
/// Message fields that carry no meaning for the upstream body, so their
/// presence does not make a request untranslatable.
const IGNORABLE_MESSAGE_FIELDS: &[&str] = &["name"];
/// How the upstream call is authenticated. API-key strategies are resolved in
/// `prepare`; SigV4 needs the serialized body, so the handler signs it.
#[derive(Clone, Debug, PartialEq, Eq)]
@ -25,14 +31,11 @@ pub enum ChatCompletionsAuth {
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Unsupported(pub &'static str);
pub const STREAM_PARAM: &str = "stream";
pub trait BaseConfig: Sync {
/// Supported OpenAI parameter names paired with their provider names.
fn supported_openai_param_mappings(&self) -> &'static [(&'static str, &'static str)];
/// Message fields that carry no meaning for the upstream body, so their
/// presence does not make a request untranslatable.
const IGNORABLE_MESSAGE_FIELDS: &[&str] = &["name"];
pub trait ChatCompletionsProviderConfig: Sync {
fn complete_url(
fn get_complete_url(
&self,
api_base: Option<&str>,
model: &str,
@ -40,6 +43,19 @@ pub trait ChatCompletionsProviderConfig: Sync {
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<String, Error>;
fn transform_request(
&self,
model: &str,
messages: Vec<ChatMessage>,
optional_params: Map<String, Value>,
) -> Result<ProviderChatRequestData, Error>;
fn transform_response(
&self,
model: &str,
response: ProviderChatResponseData,
) -> Result<ChatCompletionsResponse, Error>;
fn auth(
&self,
api_key: Option<&str>,
@ -62,9 +78,6 @@ pub trait ChatCompletionsProviderConfig: Sync {
false
}
/// Supported OpenAI parameter names paired with their provider names.
fn supported_openai_params(&self) -> &'static [(&'static str, &'static str)];
/// Parameters consumed as call configuration (credentials, endpoints)
/// rather than placed in the body. Accepted, never serialized.
fn config_params(&self) -> &'static [&'static str] {
@ -77,25 +90,12 @@ pub trait ChatCompletionsProviderConfig: Sync {
optional_params: &Map<String, Value>,
) -> Option<Unsupported> {
unsupported_param(
self.supported_openai_params(),
self.supported_openai_param_mappings(),
self.config_params(),
optional_params,
)
.or_else(|| messages.iter().find_map(unsupported_message))
}
fn transform_request(
&self,
model: &str,
messages: Vec<ChatMessage>,
optional_params: Map<String, Value>,
) -> Result<ProviderChatRequestData, Error>;
fn transform_response(
&self,
model: &str,
response: ProviderChatResponseData,
) -> Result<ChatCompletionsResponse, Error>;
}
pub fn unsupported_param(

3
litellm-rust/crates/core/src/llms/base_llm/mod.rs
View file

@ -1 +1,4 @@
pub mod anthropic_messages;
pub mod audio_transcription;
pub mod chat;
pub(crate) mod ocr;

28
litellm-rust/crates/core/src/providers/bedrock/audio_transcription.rs → litellm-rust/crates/core/src/llms/bedrock/audio_transcription/mod.rs
View file

@ -1,15 +1,15 @@
use serde_json::{Map, Value, json};
pub use super::aws_base::{aws_auth_config, bedrock_model_id_and_region, resolve_bedrock_region};
use super::constants::{BEDROCK_RUNTIME_ENDPOINT_TEMPLATE, BEDROCK_SERVICE};
use crate::audio_transcription::Error;
use crate::audio_transcription::transformation::{
AudioTranscriptionAuth, AudioTranscriptionProviderConfig,
};
use crate::audio_transcription::types::{
AudioTranscriptionRequestData, AudioTranscriptionResponseData,
};
use crate::http_utils::json_type_name;
use crate::llms::base_llm::audio_transcription::transformation::{
AudioTranscriptionAuth, BaseAudioTranscriptionConfig,
};
use litellm_auth_aws::constants::{BEDROCK_RUNTIME_ENDPOINT_TEMPLATE, BEDROCK_SERVICE};
use litellm_auth_aws::{bedrock_model_id_and_region, resolve_bedrock_region};
const SUPPORTED_PARAMS: &[&str] = &["language", "prompt", "temperature", "response_format"];
@ -45,12 +45,12 @@ fn optional_string<'a>(params: &'a Map<String, Value>, key: &str) -> Option<&'a
.filter(|value| !value.is_empty())
}
impl AudioTranscriptionProviderConfig for BedrockAudioTranscriptionConfig {
fn supported_transcription_params(&self) -> &'static [&'static str] {
impl BaseAudioTranscriptionConfig for BedrockAudioTranscriptionConfig {
fn get_supported_openai_params(&self) -> &'static [&'static str] {
SUPPORTED_PARAMS
}
fn transform_transcription_request(
fn transform_audio_transcription_request(
&self,
_model: &str,
audio: Value,
@ -83,7 +83,7 @@ impl AudioTranscriptionProviderConfig for BedrockAudioTranscriptionConfig {
})
}
fn transform_transcription_response(
fn transform_audio_transcription_response(
&self,
_model: &str,
response_json: Value,
@ -105,7 +105,7 @@ impl AudioTranscriptionProviderConfig for BedrockAudioTranscriptionConfig {
Ok(AudioTranscriptionResponseData { text })
}
fn complete_url(
fn get_complete_url(
&self,
api_base: Option<&str>,
model: &str,
@ -160,7 +160,7 @@ mod tests {
]);
let params = BEDROCK_AUDIO_TRANSCRIPTION_CONFIG.map_transcription_params(&params);
let result = BEDROCK_AUDIO_TRANSCRIPTION_CONFIG
.transform_transcription_request(
.transform_audio_transcription_request(
"mistral.voxtral-mini-3b-2507",
json!({"data": "AQI=", "format": "wav", "filename": "sample.wav"}),
params,
@ -185,7 +185,7 @@ mod tests {
#[test]
fn response_concatenates_content_blocks() {
let result = BEDROCK_AUDIO_TRANSCRIPTION_CONFIG
.transform_transcription_response(
.transform_audio_transcription_response(
"model",
json!({"output": {"message": {"content": [{"text": "hello "}, {"text": "world"}]}}}),
)
@ -196,7 +196,7 @@ mod tests {
#[test]
fn invalid_audio_is_rejected() {
let result = BEDROCK_AUDIO_TRANSCRIPTION_CONFIG.transform_transcription_request(
let result = BEDROCK_AUDIO_TRANSCRIPTION_CONFIG.transform_audio_transcription_request(
"model",
json!({"data": "AQI="}),
Map::new(),
@ -208,7 +208,7 @@ mod tests {
fn region_and_url_precedence_match_python() {
let params = Map::from_iter([("aws_region_name".to_string(), json!("eu-west-1"))]);
let url = BEDROCK_AUDIO_TRANSCRIPTION_CONFIG
.complete_url(
.get_complete_url(
None,
"bedrock/us-east-1/mistral.voxtral-mini-3b-2507",
&params,

266
litellm-rust/crates/core/src/providers/bedrock/chat_completions/transformation.rs → litellm-rust/crates/core/src/llms/bedrock/chat/converse_transformation.rs
View file

@ -1,19 +1,18 @@
use serde_json::{Map, Value, json};
use super::super::aws_base::{bedrock_model_id_and_region, resolve_bedrock_region};
use super::super::constants::{AWS_BEARER_TOKEN_BEDROCK, BEDROCK_RUNTIME_ENDPOINT_TEMPLATE};
use crate::chat_completions::Error;
use crate::chat_completions::conversation::{Conversation, TurnRole, build_conversation};
use crate::chat_completions::response_utils::{finish_reason_for, unix_now, usage_from_parts};
use crate::chat_completions::transformation::{
ChatCompletionsAuth, ChatCompletionsProviderConfig, Unsupported, unsupported_message,
unsupported_param,
};
use crate::chat_completions::types::{
ChatCompletionsChoice, ChatCompletionsChoiceMessage, ChatCompletionsResponse,
ChatCompletionsUsage, ChatMessage, ChatMessageContent, ProviderChatRequestData,
ProviderChatResponseData,
};
use crate::llms::base_llm::chat::transformation::{
BaseConfig, ChatCompletionsAuth, Unsupported, unsupported_message, unsupported_param,
};
use litellm_auth_aws::constants::{AWS_BEARER_TOKEN_BEDROCK, BEDROCK_RUNTIME_ENDPOINT_TEMPLATE};
use litellm_auth_aws::{bedrock_model_id_and_region, resolve_bedrock_region};
/// Converse parameter names, post `map_openai_params`, that the Rust path can
/// place verbatim in `inferenceConfig`.
@ -49,62 +48,16 @@ const CONFIG_PARAMS: &[&str] = &[
const CONVERSE_PATH_SUFFIX: &str = "/converse";
pub struct BedrockChatCompletionsConfig;
pub struct AmazonConverseConfig;
pub const BEDROCK_CHAT_COMPLETIONS_CONFIG: BedrockChatCompletionsConfig =
BedrockChatCompletionsConfig;
pub const BEDROCK_CHAT_COMPLETIONS_CONFIG: AmazonConverseConfig = AmazonConverseConfig;
fn converse_body(conversation: &Conversation, params: &Map<String, Value>) -> Value {
let messages: Vec<Value> = conversation
.turns
.iter()
.map(|turn| {
json!({
"role": turn.role.as_str(),
"content": turn.texts.iter().map(|text| json!({"text": text})).collect::<Vec<_>>(),
})
})
.collect();
let inference_config = Map::from_iter(SUPPORTED_PARAMS.iter().filter_map(|(_, name)| {
params
.get(*name)
.map(|value| ((*name).to_string(), value.clone()))
}));
let system: Vec<Value> = conversation
.system
.iter()
.map(|text| json!({"text": text}))
.collect();
Value::Object(Map::from_iter(
[
(
"inferenceConfig".to_string(),
Value::Object(inference_config),
),
("messages".to_string(), json!(messages)),
]
.into_iter()
.chain((!system.is_empty()).then(|| ("system".to_string(), json!(system)))),
))
}
fn has_blank_text(message: &ChatMessage) -> bool {
match &message.content {
None => false,
Some(ChatMessageContent::Text(text)) => text.trim().is_empty(),
Some(ChatMessageContent::Parts(parts)) => parts.iter().any(|part| {
part.get("text")
.and_then(Value::as_str)
.is_none_or(|text| text.trim().is_empty())
}),
impl BaseConfig for AmazonConverseConfig {
fn supported_openai_param_mappings(&self) -> &'static [(&'static str, &'static str)] {
SUPPORTED_PARAMS
}
}
impl ChatCompletionsProviderConfig for BedrockChatCompletionsConfig {
fn complete_url(
fn get_complete_url(
&self,
api_base: Option<&str>,
model: &str,
@ -131,82 +84,6 @@ impl ChatCompletionsProviderConfig for BedrockChatCompletionsConfig {
Ok(format!("{endpoint}/model/{model_id}{CONVERSE_PATH_SUFFIX}"))
}
fn auth(
&self,
api_key: Option<&str>,
model: &str,
optional_params: &Map<String, Value>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<ChatCompletionsAuth, Error> {
// Python reads `api_key` as the Bedrock bearer token and consults the
// env only when the caller passed none, so a caller-supplied empty key
// falls through to SigV4 without reaching for the environment. An
// all-whitespace token stays a bearer token here because Python sends
// it too: treating it as absent would sign as the host principal
// instead, which is the identity swap this branch exists to prevent.
let bearer = match api_key {
Some(key) => Some(key.to_string()),
None => env_lookup(AWS_BEARER_TOKEN_BEDROCK),
}
.filter(|token| !token.is_empty());
if let Some(token) = bearer {
return Ok(ChatCompletionsAuth::Bearer { token });
}
let (_, model_region) = bedrock_model_id_and_region(model);
Ok(ChatCompletionsAuth::AwsSigV4 {
region: resolve_bedrock_region(model_region.as_deref(), optional_params, env_lookup),
})
}
fn default_headers(&self) -> &'static [(&'static str, &'static str)] {
&[("Content-Type", "application/json")]
}
fn supported_openai_params(&self) -> &'static [(&'static str, &'static str)] {
SUPPORTED_PARAMS
}
fn config_params(&self) -> &'static [&'static str] {
CONFIG_PARAMS
}
fn unsupported_reason(
&self,
messages: &[ChatMessage],
optional_params: &Map<String, Value>,
) -> Option<Unsupported> {
unsupported_param(
self.supported_openai_params(),
CONFIG_PARAMS,
optional_params,
)
.or_else(|| messages.iter().find_map(unsupported_message))
// Python's Converse translation drops blank text blocks instead of
// substituting the placeholder the shared conversation builder
// applies, so decline blank text rather than diverge.
.or_else(|| {
messages
.iter()
.any(has_blank_text)
.then_some(Unsupported("blank message text"))
})
// Converse has no assistant prefill: Python inserts a continue turn
// when a conversation opens or closes on an assistant message, and
// only under `litellm.modify_params`, which the core cannot see.
// Declining both ends also keeps the shared builder's final
// assistant right-strip (an Anthropic rule) unreachable here.
.or_else(|| {
let conversation = build_conversation(messages);
let ends_on_assistant = conversation
.turns
.last()
.is_some_and(|turn| turn.role == TurnRole::Assistant);
(!conversation.opens_on_user_turn() || ends_on_assistant).then_some(Unsupported(
"conversation does not run user turn to user turn",
))
})
}
fn transform_request(
&self,
_model: &str,
@ -295,6 +172,127 @@ impl ChatCompletionsProviderConfig for BedrockChatCompletionsConfig {
usage,
})
}
fn auth(
&self,
api_key: Option<&str>,
model: &str,
optional_params: &Map<String, Value>,
env_lookup: &dyn Fn(&str) -> Option<String>,
) -> Result<ChatCompletionsAuth, Error> {
// Python reads `api_key` as the Bedrock bearer token and consults the
// env only when the caller passed none, so a caller-supplied empty key
// falls through to SigV4 without reaching for the environment. An
// all-whitespace token stays a bearer token here because Python sends
// it too: treating it as absent would sign as the host principal
// instead, which is the identity swap this branch exists to prevent.
let bearer = match api_key {
Some(key) => Some(key.to_string()),
None => env_lookup(AWS_BEARER_TOKEN_BEDROCK),
}
.filter(|token| !token.is_empty());
if let Some(token) = bearer {
return Ok(ChatCompletionsAuth::Bearer { token });
}
let (_, model_region) = bedrock_model_id_and_region(model);
Ok(ChatCompletionsAuth::AwsSigV4 {
region: resolve_bedrock_region(model_region.as_deref(), optional_params, env_lookup),
})
}
fn default_headers(&self) -> &'static [(&'static str, &'static str)] {
&[("Content-Type", "application/json")]
}
fn config_params(&self) -> &'static [&'static str] {
CONFIG_PARAMS
}
fn unsupported_reason(
&self,
messages: &[ChatMessage],
optional_params: &Map<String, Value>,
) -> Option<Unsupported> {
unsupported_param(
self.supported_openai_param_mappings(),
CONFIG_PARAMS,
optional_params,
)
.or_else(|| messages.iter().find_map(unsupported_message))
// Python's Converse translation drops blank text blocks instead of
// substituting the placeholder the shared conversation builder
// applies, so decline blank text rather than diverge.
.or_else(|| {
messages
.iter()
.any(has_blank_text)
.then_some(Unsupported("blank message text"))
})
// Converse has no assistant prefill: Python inserts a continue turn
// when a conversation opens or closes on an assistant message, and
// only under `litellm.modify_params`, which the core cannot see.
// Declining both ends also keeps the shared builder's final
// assistant right-strip (an Anthropic rule) unreachable here.
.or_else(|| {
let conversation = build_conversation(messages);
let ends_on_assistant = conversation
.turns
.last()
.is_some_and(|turn| turn.role == TurnRole::Assistant);
(!conversation.opens_on_user_turn() || ends_on_assistant).then_some(Unsupported(
"conversation does not run user turn to user turn",
))
})
}
}
fn converse_body(conversation: &Conversation, optional_params: &Map<String, Value>) -> Value {
let messages: Vec<Value> = conversation
.turns
.iter()
.map(|turn| {
json!({
"role": turn.role.as_str(),
"content": turn.texts.iter().map(|text| json!({"text": text})).collect::<Vec<_>>(),
})
})
.collect();
let inference_config = Map::from_iter(SUPPORTED_PARAMS.iter().filter_map(|(_, name)| {
optional_params
.get(*name)
.map(|value| ((*name).to_string(), value.clone()))
}));
let system: Vec<Value> = conversation
.system
.iter()
.map(|text| json!({"text": text}))
.collect();
Value::Object(Map::from_iter(
[
(
"inferenceConfig".to_string(),
Value::Object(inference_config),
),
("messages".to_string(), json!(messages)),
]
.into_iter()
.chain((!system.is_empty()).then(|| ("system".to_string(), json!(system)))),
))
}
fn has_blank_text(message: &ChatMessage) -> bool {
match &message.content {
None => false,
Some(ChatMessageContent::Text(text)) => text.trim().is_empty(),
Some(ChatMessageContent::Parts(parts)) => parts.iter().any(|part| {
part.get("text")
.and_then(Value::as_str)
.is_none_or(|text| text.trim().is_empty())
}),
}
}
#[cfg(test)]

1
litellm-rust/crates/core/src/llms/bedrock/chat/mod.rs Normal file
View file

@ -0,0 +1 @@
pub mod converse_transformation;

12
litellm-rust/crates/core/src/providers/bedrock/chat_completions/tests.rs → litellm-rust/crates/core/src/llms/bedrock/chat/tests.rs
View file

@ -226,7 +226,7 @@ fn builds_the_converse_url_from_the_region_in_the_model_id() {
let config = &BEDROCK_CHAT_COMPLETIONS_CONFIG;
assert_eq!(
config
.complete_url(None, "us-east-1/anthropic.claude-v2", &Map::new(), &|_| {
.get_complete_url(None, "us-east-1/anthropic.claude-v2", &Map::new(), &|_| {
None
})
.expect("url builds"),
@ -240,13 +240,13 @@ fn falls_back_to_the_region_env_then_the_default_region() {
let with_env = |key: &str| (key == "AWS_REGION_NAME").then(|| "eu-west-1".to_string());
assert_eq!(
config
.complete_url(None, "anthropic.claude-v2", &Map::new(), &with_env)
.get_complete_url(None, "anthropic.claude-v2", &Map::new(), &with_env)
.expect("url builds"),
"https://bedrock-runtime.eu-west-1.amazonaws.com/model/anthropic.claude-v2/converse"
);
assert_eq!(
config
.complete_url(None, "anthropic.claude-v2", &Map::new(), &|_| None)
.get_complete_url(None, "anthropic.claude-v2", &Map::new(), &|_| None)
.expect("url builds"),
"https://bedrock-runtime.us-west-2.amazonaws.com/model/anthropic.claude-v2/converse"
);
@ -258,7 +258,7 @@ fn prefers_an_explicit_runtime_endpoint_over_the_api_base() {
let overrides = params(json!({"aws_bedrock_runtime_endpoint": "https://vpce.internal/"}));
assert_eq!(
config
.complete_url(
.get_complete_url(
Some("https://ignored.example"),
"anthropic.claude-v2",
&overrides,
@ -540,7 +540,7 @@ fn leaves_a_complete_converse_url_untouched() {
"https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-v2%3A0/converse";
assert_eq!(
config
.complete_url(
.get_complete_url(
Some(already_built),
"anthropic.claude-v2",
&Map::new(),
@ -554,7 +554,7 @@ fn leaves_a_complete_converse_url_untouched() {
#[test]
fn host_supplied_credentials_outrank_ambient_profile_and_role_state() {
use crate::providers::bedrock::aws_base::host_supplied_credentials;
use litellm_auth_aws::host_supplied_credentials;
let supplied = params(json!({
"aws_access_key_id": "AKIAHOST",

2
litellm-rust/crates/core/src/llms/bedrock/mod.rs Normal file
View file

@ -0,0 +1,2 @@
pub mod audio_transcription;
pub mod chat;

298
litellm-rust/crates/core/src/llms/cohere/ocr/transformation.rs
View file

@ -4,13 +4,13 @@ use serde_with::serde_as;
use crate::call_arguments::{CallArguments, parse_options};
use crate::constants::{COHERE_API_KEY_ENV, COHERE_PARSE_API_BASE};
use crate::llms::base_llm::ocr::transformation::{BaseOcrConfig, OcrRequestContext};
use crate::llms::base_llm::ocr::transformation::{BaseOcrConfig, decode_and_normalize_response};
use crate::ocr::OcrClient;
use crate::ocr::document::InlineDocument;
use crate::ocr::prepare::credential_env;
use crate::ocr::types::{
LiteLLMOcrResponse, OcrConnection, OcrDocument, OcrPage, OcrPageImage, OcrUsageInfo,
PreparedOcrRequest,
LiteLLMOcrResponse, OcrConnection, OcrDocument, OcrPage, OcrPageImage, OcrResponseFormat,
OcrUsageInfo, PreparedOcrRequest,
};
use crate::serde_compat::LaxI64;
use crate::url_utils::ApiUrl;
@ -88,6 +88,10 @@ impl BaseOcrConfig for CohereParseConfig {
type ProviderRequest = CohereRequest;
type Environment = Vec<(String, String)>;
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["output_format", "req_format"]
}
fn get_api_key_env_var(&self) -> Option<&'static str> {
Some(COHERE_API_KEY_ENV)
}
@ -99,21 +103,29 @@ impl BaseOcrConfig for CohereParseConfig {
}
}
fn map_ocr_params(
&self,
non_default_params: &CallArguments,
_model: &str,
) -> Result<CohereOptions, crate::ocr::Error> {
Ok(parse_options(non_default_params)?)
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
_client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
self.validate_environment(&request.connection, &credential_env)
self.resolve_headers(&request.connection, &credential_env)
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
_params: &Self::OcrParams,
_optional_params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
self.get_complete_url(
self.build_ocr_url(
request
.connection
.api_base
@ -133,41 +145,13 @@ impl BaseOcrConfig for CohereParseConfig {
Ok(build_request(model, image_url, optional_params))
}
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["output_format", "req_format"]
}
fn map_ocr_params(
&self,
arguments: &CallArguments,
_model: &str,
) -> Result<CohereOptions, crate::ocr::Error> {
Ok(parse_options(arguments)?)
}
async fn async_transform_ocr_request(
&self,
model: &str,
document: OcrDocument,
optional_params: &CohereOptions,
headers: &[(String, String)],
_context: OcrRequestContext<'_>,
) -> Result<CohereRequest, crate::ocr::Error> {
self.transform_ocr_request(model, document, optional_params, headers)
}
fn transform_ocr_response(
&self,
model: &str,
raw_response: &[u8],
request_format: crate::ocr::types::OcrResponseFormat,
request_format: OcrResponseFormat,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
crate::llms::base_llm::ocr::transformation::decode_and_normalize_response(
model,
raw_response,
request_format,
normalize_response,
)
decode_and_normalize_response(model, raw_response, request_format, normalize_response)
}
fn validate_request_body(&self, body: &Value) -> Result<(), crate::ocr::Error> {
@ -175,6 +159,50 @@ impl BaseOcrConfig for CohereParseConfig {
}
}
impl CohereParseConfig {
fn resolve_headers(
&self,
connection: &OcrConnection,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, crate::ocr::Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization") {
return Ok(connection.extra_headers.clone());
}
let key = connection
.api_key
.as_deref()
.map(str::trim)
.filter(|key| !key.is_empty())
.map(str::to_string)
.or_else(|| {
self.get_api_key_env_var()
.and_then(env_lookup)
.filter(|key| !key.trim().is_empty())
})
.ok_or_else(|| {
crate::ocr::Error::Auth(litellm_auth::Error::ProviderAuthentication(
"Missing COHERE_API_KEY - set it in the environment or pass api_key".into(),
))
})?;
Ok(
std::iter::once(("Authorization".into(), format!("Bearer {key}")))
.chain(connection.extra_headers.clone())
.collect(),
)
}
fn build_ocr_url(&self, api_base: &str) -> Result<String, crate::ocr::Error> {
let parsed = reqwest::Url::parse(api_base).map_err(|_| invalid_api_base())?;
if !matches!(parsed.scheme(), "http" | "https") {
return Err(invalid_api_base());
}
ApiUrl::parse(api_base)
.and_then(|url| url.complete_path(&["v2", "parse"]))
.map(|url| url.into_string())
.map_err(|_| invalid_api_base())
}
}
pub(crate) fn validate_document(document: &OcrDocument) -> Result<(), crate::ocr::Error> {
let OcrDocument::ImageUrl { image_url, .. } = document else {
return Err(crate::ocr::Error::CohereImageOnly);
@ -292,50 +320,6 @@ fn billed_pages(response: &CohereResponse) -> Option<i64> {
response.meta.as_ref()?.billed_units.as_ref()?.pages
}
impl CohereParseConfig {
fn get_complete_url(&self, base: &str) -> Result<String, crate::ocr::Error> {
let parsed = reqwest::Url::parse(base).map_err(|_| invalid_api_base())?;
if !matches!(parsed.scheme(), "http" | "https") {
return Err(invalid_api_base());
}
ApiUrl::parse(base)
.and_then(|url| url.complete_path(&["v2", "parse"]))
.map(|url| url.into_string())
.map_err(|_| invalid_api_base())
}
fn validate_environment(
&self,
connection: &OcrConnection,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, crate::ocr::Error> {
if crate::http_utils::has_header(&connection.extra_headers, "authorization") {
return Ok(connection.extra_headers.clone());
}
let key = connection
.api_key
.as_deref()
.map(str::trim)
.filter(|key| !key.is_empty())
.map(str::to_string)
.or_else(|| {
self.get_api_key_env_var()
.and_then(env_lookup)
.filter(|key| !key.trim().is_empty())
})
.ok_or_else(|| {
crate::ocr::Error::Auth(litellm_auth::Error::ProviderAuthentication(
"Missing COHERE_API_KEY - set it in the environment or pass api_key".into(),
))
})?;
Ok(
std::iter::once(("Authorization".into(), format!("Bearer {key}")))
.chain(connection.extra_headers.clone())
.collect(),
)
}
}
fn invalid_api_base() -> crate::ocr::Error {
crate::ocr::Error::RequestField {
path: "api_base".into(),
@ -385,27 +369,31 @@ mod tests {
);
}
#[test]
fn options_read_known_fields_without_changing_arguments() {
#[rstest]
#[case::cohere(false)]
#[case::azure(true)]
fn options_read_known_fields_without_changing_arguments(#[case] azure: bool) {
let arguments = serde_json::from_value(json!({
"output_format":"blocks", "req_format":"native", "extension":false
}))
.unwrap();
for config in [false, true] {
let mapped = if config {
crate::llms::azure_ai::ocr::cohere_parse_transformation::AzureAICohereParseConfig
.map_ocr_params(&arguments, "parse")
} else {
CohereParseConfig.map_ocr_params(&arguments, "parse")
}
.unwrap();
assert_eq!(
serde_json::to_value(mapped).unwrap(),
json!({"output_format":"blocks"})
);
let mapped = if azure {
crate::llms::azure_ai::ocr::cohere_parse_transformation::AzureAICohereParseConfig
.map_ocr_params(&arguments, "parse")
} else {
CohereParseConfig.map_ocr_params(&arguments, "parse")
}
.unwrap();
assert_eq!(
serde_json::to_value(mapped).unwrap(),
json!({"output_format":"blocks"})
);
assert_eq!(arguments["req_format"], "native");
assert_eq!(arguments["extension"], false);
}
#[test]
fn options_reject_invalid_output_format() {
let invalid = serde_json::from_value(json!({"output_format":"html"})).unwrap();
assert!(matches!(
CohereParseConfig.map_ocr_params(&invalid, "parse"),
@ -415,13 +403,17 @@ mod tests {
}
#[test]
fn billed_pages_accept_integral_doubles_and_reject_fractional_counts() {
fn billed_pages_accept_integral_doubles() {
let response = serde_json::from_str::<CohereResponse>(
r#"{"pages":[],"meta":{"billed_units":{"pages":1.0}}}"#,
)
.unwrap();
let normalized = normalize_response("parse", response).unwrap();
assert_eq!(normalized.usage_info.unwrap().pages_processed, Some(1));
}
#[test]
fn billed_pages_reject_fractional_counts() {
assert!(
serde_json::from_str::<CohereResponse>(
r#"{"pages":[],"meta":{"billed_units":{"pages":1.5}}}"#,
@ -590,25 +582,27 @@ mod tests {
assert_eq!(normalized.usage_info.unwrap().pages_processed, Some(3));
}
#[rstest]
#[case::empty(json!({}))]
#[case::null_meta(json!({"meta":null}))]
#[case::null_billed_units(json!({"pages":[],"meta":{"billed_units":null}}))]
fn response_defaults(#[case] value: Value) {
let normalized =
normalize_response("parse", serde_json::from_value(value).unwrap()).unwrap();
assert!(normalized.pages.is_empty());
assert_eq!(normalized.usage_info.unwrap().pages_processed, Some(0));
}
#[rstest]
#[case::null_pages(json!({"pages":null}))]
#[case::invalid_markdown(json!({"pages":[{"markdown":"text"}]}))]
#[case::invalid_index(json!({"pages":[{"index":"bad"}]}))]
fn response_rejects_invalid_fields(#[case] value: Value) {
assert!(serde_json::from_value::<CohereResponse>(value).is_err());
}
#[test]
fn response_defaults_and_invalid_fields() {
for value in [
json!({}),
json!({"meta":null}),
json!({"pages":[],"meta":{"billed_units":null}}),
] {
let normalized =
normalize_response("parse", serde_json::from_value(value).unwrap()).unwrap();
assert!(normalized.pages.is_empty());
assert_eq!(normalized.usage_info.unwrap().pages_processed, Some(0));
}
for value in [
json!({"pages":null}),
json!({"pages":[{"markdown":"text"}]}),
json!({"pages":[{"index":"bad"}]}),
] {
assert!(serde_json::from_value::<CohereResponse>(value).is_err());
}
fn null_markdown_uses_page_defaults() {
let normalized = normalize_response(
"parse",
serde_json::from_value(json!({"pages":[{"markdown":null}]})).unwrap(),
@ -710,24 +704,30 @@ mod tests {
);
}
#[rstest]
#[case::document_url(json!({"type":"document_url","document_url":"https://example.com/a.pdf"}))]
#[case::empty_image_url(json!({"type":"image_url","image_url":""}))]
#[case::pdf_data_uri(json!({"type":"image_url","image_url":"data:application/pdf;base64,YQ=="}))]
fn request_requires_image(#[case] value: Value) {
assert!(matches!(
validate_document(&serde_json::from_value(value).unwrap()),
Err(crate::ocr::Error::CohereImageOnly)
));
}
#[rstest]
#[case::markdown("markdown", true)]
#[case::blocks("blocks", true)]
#[case::unsupported("html", false)]
fn request_requires_supported_output_format(#[case] format: &str, #[case] valid: bool) {
assert_eq!(
serde_json::from_value::<CohereOptions>(json!({"output_format":format})).is_ok(),
valid
);
}
#[test]
fn request_requires_image_and_supported_output_format() {
for value in [
json!({"type":"document_url","document_url":"https://example.com/a.pdf"}),
json!({"type":"image_url","image_url":""}),
json!({"type":"image_url","image_url":"data:application/pdf;base64,YQ=="}),
] {
assert!(matches!(
validate_document(&serde_json::from_value(value).unwrap()),
Err(crate::ocr::Error::CohereImageOnly)
));
}
assert!(serde_json::from_value::<CohereOptions>(json!({"output_format":"html"})).is_err());
for format in ["markdown", "blocks"] {
assert!(
serde_json::from_value::<CohereOptions>(json!({"output_format":format})).is_ok()
);
}
fn request_defaults_to_markdown() {
let request = CohereParseConfig
.transform_ocr_request(
"parse-v5.0",
@ -746,28 +746,30 @@ mod tests {
);
}
#[test]
fn completes_provider_urls_without_duplicate_paths_and_preserves_queries() {
for suffix in ["", "/v2", "/v2/parse"] {
assert_eq!(
CohereParseConfig
.get_complete_url(&format!("https://example.com{suffix}?tenant=a"))
.unwrap(),
"https://example.com/v2/parse?tenant=a"
);
}
#[rstest]
#[case::base("")]
#[case::version("/v2")]
#[case::complete("/v2/parse")]
fn completes_provider_urls_without_duplicate_paths_and_preserves_queries(#[case] suffix: &str) {
assert_eq!(
CohereParseConfig
.build_ocr_url(&format!("https://example.com{suffix}?tenant=a"))
.unwrap(),
"https://example.com/v2/parse?tenant=a"
);
}
#[rstest]
#[case::relative("relative/path")]
#[case::unsupported_scheme("ftp://example.com")]
fn rejects_invalid_urls(#[case] api_base: &str) {
assert!(CohereParseConfig.build_ocr_url(api_base).is_err());
}
#[test]
fn rejects_invalid_urls_and_blank_keys() {
assert!(CohereParseConfig.get_complete_url("relative/path").is_err());
assert!(
CohereParseConfig
.get_complete_url("ftp://example.com")
.is_err()
);
fn rejects_blank_keys() {
assert!(matches!(
CohereParseConfig.validate_environment(
CohereParseConfig.resolve_headers(
&OcrConnection {
api_key: Some(" ".into()),
..Default::default()

7
litellm-rust/crates/core/src/llms/mod.rs
View file

@ -1,6 +1,9 @@
pub(crate) mod azure_ai;
pub(crate) mod base_llm;
pub mod anthropic;
pub mod azure_ai;
pub mod base_llm;
pub mod bedrock;
pub(crate) mod cohere;
pub(crate) mod mistral;
pub mod openai;
pub(crate) mod reducto;
pub(crate) mod vertex_ai;

0
litellm-rust/crates/core/src/providers/openai/mod.rs → litellm-rust/crates/core/src/llms/openai/mod.rs
View file

1
litellm-rust/crates/core/src/llms/openai/responses/mod.rs Normal file
View file

@ -0,0 +1 @@
pub mod transformation;

6
litellm-rust/crates/core/src/providers/openai/responses/transformation.rs → litellm-rust/crates/core/src/llms/openai/responses/transformation.rs
View file

@ -2,11 +2,11 @@ use crate::responses::Error;
use crate::responses::types::{ResponsesWsEvent, ResponsesWsTransformResult};
use crate::responses::websocket::{ResponsesWebSocketProviderConfig, enforce_model};
pub struct OpenAIResponsesWsConfig;
pub struct OpenAiResponsesApiConfig;
pub const OPENAI_RESPONSES_WS_CONFIG: OpenAIResponsesWsConfig = OpenAIResponsesWsConfig;
pub const OPENAI_RESPONSES_WS_CONFIG: OpenAiResponsesApiConfig = OpenAiResponsesApiConfig;
impl ResponsesWebSocketProviderConfig for OpenAIResponsesWsConfig {
impl ResponsesWebSocketProviderConfig for OpenAiResponsesApiConfig {
fn supports_native_websocket(&self) -> bool {
true
}

131
litellm-rust/crates/core/src/llms/reducto/ocr/transformation.rs
View file

@ -5,12 +5,15 @@ use serde_json::{Map, Value, json};
use crate::call_arguments::{CallArguments, compose_body};
use crate::constants::{REDUCTO_API_BASE, REDUCTO_API_KEY_ENV, REDUCTO_ID_PREFIX};
use crate::llms::base_llm::ocr::transformation::{BaseOcrConfig, OcrRequestContext};
use crate::llms::base_llm::ocr::transformation::{
BaseOcrConfig, OcrRequestContext, decode_and_normalize_response,
};
use crate::ocr::OcrClient;
use crate::ocr::document::InlineDocument;
use crate::ocr::prepare::{build_http_request, credential_env, guardrail_document};
use crate::ocr::types::{
LiteLLMOcrResponse, OcrConnection, OcrDocument, OcrPage, OcrUsageInfo, PreparedOcrRequest,
LiteLLMOcrResponse, OcrConnection, OcrDocument, OcrPage, OcrResponseFormat, OcrUsageInfo,
PreparedOcrRequest,
};
use crate::params::OpaqueParams;
use crate::url_utils::ApiUrl;
@ -83,50 +86,50 @@ impl BaseOcrConfig for ReductoParseV3Config {
type ProviderRequest = ReductoV3Request;
type Environment = Vec<(String, String)>;
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
_client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
validate_environment(&request.connection, &credential_env)
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
_params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
get_complete_url(request.connection.api_base.as_deref())
}
fn transform_ocr_request(
&self,
_model: &str,
document: OcrDocument,
params: &Self::OcrParams,
_headers: &[(String, String)],
) -> Result<Self::ProviderRequest, crate::ocr::Error> {
Ok(ReductoV3Request {
input: uploaded_file_id(document)?,
params: params.clone(),
})
}
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["formatting", "retrieval", "settings"]
}
fn map_ocr_params(
&self,
arguments: &CallArguments,
non_default_params: &CallArguments,
model: &str,
) -> Result<ReductoV3Params, crate::ocr::Error> {
Ok(arguments
Ok(non_default_params
.select(self.get_supported_ocr_params(model))
.into())
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
_client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
resolve_headers(&request.connection, &credential_env)
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
_optional_params: &Self::OcrParams,
_environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
build_ocr_url(request.connection.api_base.as_deref())
}
fn transform_ocr_request(
&self,
_model: &str,
document: OcrDocument,
optional_params: &Self::OcrParams,
_headers: &[(String, String)],
) -> Result<Self::ProviderRequest, crate::ocr::Error> {
Ok(ReductoV3Request {
input: uploaded_file_id(document)?,
params: optional_params.clone(),
})
}
async fn async_transform_ocr_request(
&self,
_model: &str,
@ -146,14 +149,9 @@ impl BaseOcrConfig for ReductoParseV3Config {
&self,
model: &str,
raw_response: &[u8],
request_format: crate::ocr::types::OcrResponseFormat,
request_format: OcrResponseFormat,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
crate::llms::base_llm::ocr::transformation::decode_and_normalize_response(
model,
raw_response,
request_format,
normalize_response,
)
decode_and_normalize_response(model, raw_response, request_format, normalize_response)
}
async fn prepare_request(
@ -173,6 +171,20 @@ impl BaseOcrConfig for ReductoParseLegacyConfig {
type ProviderRequest = ReductoLegacyRequest;
type Environment = Vec<(String, String)>;
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["enhance"]
}
fn map_ocr_params(
&self,
non_default_params: &CallArguments,
model: &str,
) -> Result<ReductoLegacyParams, crate::ocr::Error> {
Ok(non_default_params
.select(self.get_supported_ocr_params(model))
.into())
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
@ -186,34 +198,23 @@ impl BaseOcrConfig for ReductoParseLegacyConfig {
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
params: &Self::OcrParams,
optional_params: &Self::OcrParams,
environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
ReductoParseV3Config.get_complete_url(request, params, environment)
ReductoParseV3Config.get_complete_url(request, optional_params, environment)
}
fn transform_ocr_request(
&self,
_model: &str,
document: OcrDocument,
params: &Self::OcrParams,
optional_params: &Self::OcrParams,
_headers: &[(String, String)],
) -> Result<Self::ProviderRequest, crate::ocr::Error> {
Ok(build_legacy_body(uploaded_file_id(document)?, params))
}
fn get_supported_ocr_params(&self, _model: &str) -> &'static [&'static str] {
&["enhance"]
}
fn map_ocr_params(
&self,
arguments: &CallArguments,
model: &str,
) -> Result<ReductoLegacyParams, crate::ocr::Error> {
Ok(arguments
.select(self.get_supported_ocr_params(model))
.into())
Ok(build_legacy_body(
uploaded_file_id(document)?,
optional_params,
))
}
async fn async_transform_ocr_request(
@ -232,7 +233,7 @@ impl BaseOcrConfig for ReductoParseLegacyConfig {
&self,
model: &str,
raw_response: &[u8],
request_format: crate::ocr::types::OcrResponseFormat,
request_format: OcrResponseFormat,
) -> Result<LiteLLMOcrResponse, crate::ocr::Error> {
ReductoParseV3Config.transform_ocr_response(model, raw_response, request_format)
}
@ -403,7 +404,7 @@ fn page(index: i64, markdown: String, blocks: Option<Value>) -> OcrPage {
..Default::default()
}
}
fn get_complete_url(api_base: Option<&str>) -> Result<String, crate::ocr::Error> {
fn build_ocr_url(api_base: Option<&str>) -> Result<String, crate::ocr::Error> {
complete_endpoint_url(api_base, "parse")
}
@ -420,7 +421,7 @@ fn complete_endpoint_url(api_base: Option<&str>, path: &str) -> Result<String, c
})
}
fn validate_environment(
fn resolve_headers(
connection: &OcrConnection,
env_lookup: &(dyn Fn(&str) -> Option<String> + Sync),
) -> Result<Vec<(String, String)>, crate::ocr::Error> {
@ -655,7 +656,7 @@ mod tests {
api_key: Some("passed-key".into()),
..Default::default()
};
let headers = validate_environment(&connection, &|_| Some("env-key".into())).unwrap();
let headers = resolve_headers(&connection, &|_| Some("env-key".into())).unwrap();
assert_eq!(headers[0].1, "Bearer passed-key");
}
@ -665,7 +666,7 @@ mod tests {
api_key: Some(" ".into()),
..Default::default()
};
let headers = validate_environment(&connection, &|_| Some(" env-key ".into())).unwrap();
let headers = resolve_headers(&connection, &|_| Some(" env-key ".into())).unwrap();
assert_eq!(headers[0].1, "Bearer env-key");
}
@ -676,7 +677,7 @@ mod tests {
..Default::default()
};
assert_eq!(
validate_environment(&connection, &|_| None).unwrap(),
resolve_headers(&connection, &|_| None).unwrap(),
connection.extra_headers
);
}

8
litellm-rust/crates/core/src/llms/vertex_ai/ocr/deepseek_transformation.rs
View file

@ -2,7 +2,7 @@ use litellm_auth_gcp::{self as vertex, VertexConfig};
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
use super::transformation::VertexAIOCRConfig;
use super::transformation::VertexAiOcrConfig;
use crate::call_arguments::CallArguments;
use crate::llms::base_llm::ocr::transformation::{BaseOcrConfig, OcrRequestContext};
use crate::ocr::OcrClient;
@ -95,7 +95,7 @@ impl BaseOcrConfig for VertexAIDeepSeekOCRConfig {
type Environment = vertex::VertexEnvironment;
fn get_api_key_env_var(&self) -> Option<&'static str> {
VertexAIOCRConfig.get_api_key_env_var()
VertexAiOcrConfig.get_api_key_env_var()
}
fn map_ocr_params(
@ -111,7 +111,9 @@ impl BaseOcrConfig for VertexAIDeepSeekOCRConfig {
request: &PreparedOcrRequest,
client: &OcrClient,
) -> Result<Self::Environment, crate::ocr::Error> {
BaseOcrConfig::validate_environment(&VertexAIOCRConfig, request, client).await
VertexAiOcrConfig
.validate_environment(request, client)
.await
}
fn get_complete_url(

110
litellm-rust/crates/core/src/llms/vertex_ai/ocr/transformation.rs
View file

@ -17,17 +17,29 @@ use crate::url_utils::ApiUrl;
const DEFAULT_LOCATION: &str = "us-central1";
#[derive(Clone, Debug, Default)]
pub(crate) struct VertexAIOCRConfig;
pub(crate) struct VertexAiOcrConfig;
impl BaseOcrConfig for VertexAIOCRConfig {
impl BaseOcrConfig for VertexAiOcrConfig {
type OcrParams = OpaqueParams;
type ProviderRequest = MistralOcrRequest;
type Environment = vertex::VertexEnvironment;
fn get_supported_ocr_params(&self, model: &str) -> &'static [&'static str] {
MistralOcrConfig.get_supported_ocr_params(model)
}
fn get_api_key_env_var(&self) -> Option<&'static str> {
Some("VERTEX_AI_API_KEY")
}
fn map_ocr_params(
&self,
non_default_params: &CallArguments,
model: &str,
) -> Result<OpaqueParams, crate::ocr::Error> {
MistralOcrConfig.map_ocr_params(non_default_params, model)
}
async fn validate_environment(
&self,
request: &PreparedOcrRequest,
@ -37,14 +49,14 @@ impl BaseOcrConfig for VertexAIOCRConfig {
&request.optional_params,
&request.input_sources,
)?;
self.validate_environment(&request.connection, &config, client)
self.resolve_environment(&request.connection, &config, client)
.await
}
fn get_complete_url(
&self,
request: &PreparedOcrRequest,
_params: &Self::OcrParams,
_optional_params: &Self::OcrParams,
environment: &Self::Environment,
) -> Result<String, crate::ocr::Error> {
let config = VertexConfig::from_sourced_optional_params(
@ -53,7 +65,7 @@ impl BaseOcrConfig for VertexAIOCRConfig {
)?;
let location = vertex::get_vertex_ai_location(&config, &credential_env)
.unwrap_or_else(|| DEFAULT_LOCATION.to_string());
self.get_complete_url(
self.build_ocr_url(
request.connection.api_base.as_deref(),
&environment.project_id,
&location,
@ -65,22 +77,10 @@ impl BaseOcrConfig for VertexAIOCRConfig {
&self,
model: &str,
document: OcrDocument,
params: &OpaqueParams,
optional_params: &OpaqueParams,
headers: &[(String, String)],
) -> Result<MistralOcrRequest, crate::ocr::Error> {
MistralOcrConfig.transform_ocr_request(model, document, params, headers)
}
fn get_supported_ocr_params(&self, model: &str) -> &'static [&'static str] {
MistralOcrConfig.get_supported_ocr_params(model)
}
fn map_ocr_params(
&self,
arguments: &CallArguments,
model: &str,
) -> Result<OpaqueParams, crate::ocr::Error> {
MistralOcrConfig.map_ocr_params(arguments, model)
MistralOcrConfig.transform_ocr_request(model, document, optional_params, headers)
}
async fn async_transform_ocr_request(
@ -120,8 +120,8 @@ impl OcrEnvironment for vertex::VertexEnvironment {
}
}
impl VertexAIOCRConfig {
pub(super) async fn validate_environment(
impl VertexAiOcrConfig {
async fn resolve_environment(
&self,
connection: &OcrConnection,
config: &VertexConfig,
@ -140,7 +140,7 @@ impl VertexAIOCRConfig {
.map_err(crate::ocr::Error::from)
}
fn get_complete_url(
fn build_ocr_url(
&self,
api_base: Option<&str>,
project: &str,
@ -198,19 +198,24 @@ fn validate_location(location: &str) -> Result<(), crate::ocr::Error> {
#[cfg(test)]
mod tests {
use super::VertexAIOCRConfig;
use super::VertexAiOcrConfig;
use rstest::rstest;
#[test]
fn endpoint_uses_location_project_and_model() {
assert_eq!(
VertexAIOCRConfig
.get_complete_url(None, "proj-1", "europe-west4", "mistral-ocr-maas")
VertexAiOcrConfig
.build_ocr_url(None, "proj-1", "europe-west4", "mistral-ocr-maas")
.unwrap(),
"https://europe-west4-aiplatform.googleapis.com/v1/projects/proj-1/locations/europe-west4/publishers/mistralai/models/mistral-ocr-maas:rawPredict"
);
}
#[test]
fn endpoint_rejects_invalid_location() {
assert!(
VertexAIOCRConfig
.get_complete_url(None, "proj-1", "attacker.example/path", "model")
VertexAiOcrConfig
.build_ocr_url(None, "proj-1", "attacker.example/path", "model")
.is_err()
);
}
@ -315,13 +320,18 @@ mod tests {
);
}
#[rstest]
#[case::mistral(false)]
#[case::vertex(true)]
#[tokio::test]
async fn configs_build_complete_requests_and_share_mistral_normalization() {
async fn configs_build_complete_requests_and_share_mistral_normalization(
#[case] use_vertex: bool,
) {
use std::time::Duration;
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::llms::mistral::ocr::transformation::MistralOcrConfig;
use crate::llms::vertex_ai::ocr::transformation::VertexAIOCRConfig;
use crate::llms::vertex_ai::ocr::transformation::VertexAiOcrConfig;
use crate::ocr::test_support::ocr_client;
let client = ocr_client();
@ -348,7 +358,7 @@ mod tests {
.prepare_request(&direct, &client)
.await
.unwrap();
let vertex_http = VertexAIOCRConfig
let vertex_http = VertexAiOcrConfig
.prepare_request(&vertex, &client)
.await
.unwrap();
@ -357,24 +367,26 @@ mod tests {
vertex_http.url().as_str(),
"https://vertex.test/v1/projects/project-1/locations/us-central1/publishers/mistralai/models/mistral-ocr-maas:rawPredict"
);
for http in [&direct_http, &vertex_http] {
assert_eq!(http.method(), reqwest::Method::POST);
assert_eq!(http.headers()["authorization"], "Bearer test-key");
assert_eq!(http.headers()["content-type"], "application/json");
assert_eq!(http.timeout(), Some(&Duration::from_secs(2)));
let body: Value =
serde_json::from_slice(http.body().unwrap().as_bytes().unwrap()).unwrap();
assert_eq!(
body,
json!({
"model": "mistral-ocr-maas",
"document": {"type": "document_url", "document_url": "data:application/pdf;base64,YWJj"},
"pages": [0, 2],
"include_image_base64": true,
"unknown": "preserved"
})
);
}
let http = if use_vertex {
&vertex_http
} else {
&direct_http
};
assert_eq!(http.method(), reqwest::Method::POST);
assert_eq!(http.headers()["authorization"], "Bearer test-key");
assert_eq!(http.headers()["content-type"], "application/json");
assert_eq!(http.timeout(), Some(&Duration::from_secs(2)));
let body: Value = serde_json::from_slice(http.body().unwrap().as_bytes().unwrap()).unwrap();
assert_eq!(
body,
json!({
"model": "mistral-ocr-maas",
"document": {"type": "document_url", "document_url": "data:application/pdf;base64,YWJj"},
"pages": [0, 2],
"include_image_base64": true,
"unknown": "preserved"
})
);
let payload = serde_json::to_vec(
&json!({"pages": [{"index": 0, "markdown": "hello"}], "extra": "preserved"}),
)
@ -383,7 +395,7 @@ mod tests {
.transform_ocr_response(&direct.model, &payload, Default::default())
.unwrap()
.into_json();
let vertex_response = VertexAIOCRConfig
let vertex_response = VertexAiOcrConfig
.transform_ocr_response(&vertex.model, &payload, Default::default())
.unwrap()
.into_json();

8
litellm-rust/crates/core/src/messages/common_utils.rs
View file

@ -1,17 +1,17 @@
use serde_json::{Map, Value};
use super::Error;
use super::transformation::AnthropicMessagesProviderConfig;
use crate::http_utils::string_headers as shared_string_headers;
pub(super) use crate::http_utils::{has_bearer_auth, has_header, truncate_error_body};
use crate::providers::anthropic::messages::transformation::ANTHROPIC_MESSAGES_CONFIG;
use crate::providers::azure_ai::messages::transformation::AZURE_ANTHROPIC_MESSAGES_CONFIG;
use crate::llms::anthropic::experimental_pass_through::messages::transformation::ANTHROPIC_MESSAGES_CONFIG;
use crate::llms::azure_ai::anthropic::messages_transformation::AZURE_ANTHROPIC_MESSAGES_CONFIG;
use crate::llms::base_llm::anthropic_messages::transformation::BaseAnthropicMessagesConfig;
const HEADER_CONTEXT: &str = "messages";
pub(super) fn messages_provider_config(
provider: &str,
) -> Option<&'static dyn AnthropicMessagesProviderConfig> {
) -> Option<&'static dyn BaseAnthropicMessagesConfig> {
match provider {
"anthropic" => Some(&ANTHROPIC_MESSAGES_CONFIG),
"azure_ai" => Some(&AZURE_ANTHROPIC_MESSAGES_CONFIG),

4
litellm-rust/crates/core/src/messages/handler.rs
View file

@ -37,7 +37,9 @@ pub(super) async fn execute_messages_provider_call(
let response = serde_json::from_str(&text)
.map_err(|err| Error::InvalidResponse(format!("invalid messages response JSON: {err}")))?;
request.config.transform_response(&request.model, response)
request
.config
.transform_anthropic_messages_response(&request.model, response)
}
pub(super) async fn execute_messages_provider_stream(

1
litellm-rust/crates/core/src/messages/mod.rs
View file

@ -13,7 +13,6 @@ mod client;
mod common_utils;
mod handler;
mod prepare;
pub mod transformation;
pub mod types;
use handler::{execute_messages_provider_call, execute_messages_provider_stream};

14
litellm-rust/crates/core/src/messages/prepare.rs
View file

@ -2,9 +2,13 @@ use serde_json::{Map, Value};
use super::Error;
use super::common_utils::{has_bearer_auth, has_header, messages_provider_config, string_headers};
use super::transformation::{AnthropicMessagesProviderConfig, MessagesAuthStrategy};
use super::types::{MessagesRequest, ProviderMessagesRequest};
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};
use crate::litellm_core_utils::get_llm_provider_logic::{
CustomLlmProvider, get_custom_llm_provider,
};
use crate::llms::base_llm::anthropic_messages::transformation::{
BaseAnthropicMessagesConfig, MessagesAuthStrategy,
};
pub(super) fn prepare_provider_request(
request: MessagesRequest<'_>,
@ -36,14 +40,14 @@ pub(super) fn prepare_provider_request(
let typed_request = serde_json::from_value(request.body).map_err(|err| {
Error::InvalidRequest(format!("invalid Anthropic messages request: {err}"))
})?;
let transformed = config.transform_request(typed_request)?;
let transformed = config.transform_anthropic_messages_request(typed_request)?;
let body = serde_json::to_value(transformed).map_err(|err| {
Error::InvalidRequest(format!(
"failed to serialize Anthropic messages request: {err}"
))
})?;
let url = config.complete_url(request.api_base, &model, &env_lookup)?;
let url = config.get_complete_url(request.api_base, &model, &env_lookup)?;
Ok(ProviderMessagesRequest {
provider: provider.to_string(),
@ -57,7 +61,7 @@ pub(super) fn prepare_provider_request(
}
fn validate_environment(
config: &dyn AnthropicMessagesProviderConfig,
config: &dyn BaseAnthropicMessagesConfig,
extra_headers: Option<Map<String, Value>>,
api_key: Option<&str>,
env_lookup: &dyn Fn(&str) -> Option<String>,

4
litellm-rust/crates/core/src/messages/types.rs
View file

@ -3,7 +3,7 @@ use std::time::Duration;
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
use super::transformation::AnthropicMessagesProviderConfig;
use crate::llms::base_llm::anthropic_messages::transformation::BaseAnthropicMessagesConfig;
pub struct MessagesRequest<'a> {
pub model: &'a str,
@ -18,7 +18,7 @@ pub struct MessagesRequest<'a> {
pub(super) struct ProviderMessagesRequest {
pub(super) provider: String,
pub(super) model: String,
pub(super) config: &'static dyn AnthropicMessagesProviderConfig,
pub(super) config: &'static dyn BaseAnthropicMessagesConfig,
pub(super) url: String,
pub(super) body: Value,
pub(super) upstream_headers: Vec<(String, String)>,

16
litellm-rust/crates/core/src/ocr/provider_config.rs
View file

@ -5,16 +5,18 @@ use super::types::{
LiteLLMOcrResponse, OcrCredentialInputs, OcrDocument, PreparedOcrRequest,
ResolvedOcrCredentials,
};
use crate::litellm_core_utils::get_llm_provider_logic::{
CustomLlmProvider, get_custom_llm_provider,
};
use crate::llms::azure_ai::ocr::cohere_parse_transformation::AzureAICohereParseConfig;
use crate::llms::azure_ai::ocr::document_intelligence::transformation::AzureDocumentIntelligenceOCRConfig;
use crate::llms::azure_ai::ocr::transformation::AzureAIOCRConfig;
use crate::llms::azure_ai::ocr::document_intelligence::transformation::AzureDocumentIntelligenceOcrConfig;
use crate::llms::azure_ai::ocr::transformation::AzureAiOcrConfig;
use crate::llms::base_llm::ocr::transformation::{BaseOcrConfig, OcrResponseContext};
use crate::llms::cohere::ocr::transformation::CohereParseConfig;
use crate::llms::mistral::ocr::transformation::MistralOcrConfig;
use crate::llms::reducto::ocr::transformation::{ReductoParseLegacyConfig, ReductoParseV3Config};
use crate::llms::vertex_ai::ocr::deepseek_transformation::VertexAIDeepSeekOCRConfig;
use crate::llms::vertex_ai::ocr::transformation::VertexAIOCRConfig;
use crate::providers::custom_llm_provider::{CustomLlmProvider, get_custom_llm_provider};
use crate::llms::vertex_ai::ocr::transformation::VertexAiOcrConfig;
macro_rules! dispatch_config {
($config:expr, $method:ident($($argument:expr),* $(,)?)) => {
@ -27,12 +29,12 @@ macro_rules! dispatch_config {
match $config {
OcrConfigKind::Cohere => CohereParseConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::Mistral => MistralOcrConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::AzureAi => AzureAIOCRConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::AzureAi => AzureAiOcrConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::AzureCohere => AzureAICohereParseConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::AzureDocumentIntelligence => AzureDocumentIntelligenceOCRConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::AzureDocumentIntelligence => AzureDocumentIntelligenceOcrConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::ReductoLegacy => ReductoParseLegacyConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::ReductoV3 => ReductoParseV3Config.$method($($argument),*)$($suffix)*,
OcrConfigKind::VertexAi => VertexAIOCRConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::VertexAi => VertexAiOcrConfig.$method($($argument),*)$($suffix)*,
OcrConfigKind::VertexDeepSeek => VertexAIDeepSeekOCRConfig.$method($($argument),*)$($suffix)*,
}
};

2
litellm-rust/crates/core/src/providers/anthropic/mod.rs
View file

@ -1,2 +0,0 @@
pub mod chat_completions;
pub mod messages;

1
litellm-rust/crates/core/src/providers/bedrock/aws_base.rs
View file

@ -1 +0,0 @@
pub use litellm_auth_aws::*;

1
litellm-rust/crates/core/src/providers/bedrock/constants.rs
View file

@ -1 +0,0 @@
pub use litellm_auth_aws::constants::*;

8
litellm-rust/crates/core/src/providers/bedrock/mod.rs
View file

@ -1,8 +0,0 @@
//! User-directed exception: this base provider owns AWS auth I/O for parity
//! with Python's `BaseAWSLLM`; the broader core purity guidance is reconciled
//! separately.
pub mod audio_transcription;
pub mod aws_base;
pub mod chat_completions;
mod constants;

5
litellm-rust/crates/core/src/providers/mod.rs
View file

@ -1,5 +0,0 @@
pub mod anthropic;
pub mod azure_ai;
pub mod bedrock;
pub mod custom_llm_provider;
pub mod openai;

6
litellm-rust/crates/core/tests/vertex_ai_ocr.rs
View file

@ -104,7 +104,7 @@ async fn adapters_build_complete_requests_and_share_mistral_normalization() {
use crate::llms::base_llm::ocr::transformation::BaseOcrConfig;
use crate::llms::mistral::ocr::transformation::MistralOcrConfig;
use crate::llms::vertex_ai::ocr::transformation::VertexAIOCRConfig;
use crate::llms::vertex_ai::ocr::transformation::VertexAiOcrConfig;
use crate::ocr::test_support::ocr_client;
let client = ocr_client();
@ -129,7 +129,7 @@ async fn adapters_build_complete_requests_and_share_mistral_normalization() {
.prepare_request(&direct, &client)
.await
.unwrap();
let vertex_http = VertexAIOCRConfig
let vertex_http = VertexAiOcrConfig
.prepare_request(&vertex, &client)
.await
.unwrap();
@ -165,7 +165,7 @@ async fn adapters_build_complete_requests_and_share_mistral_normalization() {
)
.unwrap()
.into_json();
let vertex_response = VertexAIOCRConfig
let vertex_response = VertexAiOcrConfig
.transform_ocr_response(
&vertex.model,
&raw,