From a41061be43e8c77ec57cf208f3dbf2ecd46a9553 Mon Sep 17 00:00:00 2001 From: Yujong Lee Date: Mon, 21 Sep 2026 12:49:42 -0700 Subject: [PATCH 1/6] docs(rust): plan Python interop foundation --- PYTHON_INTEROP_PLAN.md | 88 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 PYTHON_INTEROP_PLAN.md diff --git a/PYTHON_INTEROP_PLAN.md b/PYTHON_INTEROP_PLAN.md new file mode 100644 index 00000000000..b456d795cfd --- /dev/null +++ b/PYTHON_INTEROP_PLAN.md @@ -0,0 +1,88 @@ +# Python interop foundation PR plan + +Proposed implementation PR title: `fix(rust): preserve Python settings semantics at the native boundary` + +Base: `main` at `457b01e96d131f88df8cace8832f8e044ee5f167`. Planning branch: `litellm_python_interop_foundation` + +This plan follows `migration/tdd/sections/rust/python-interop/{pyo3-contract,boundary,coercion}.typ` in the sibling `litellm-typst` checkout. The first PR establishes conversion contracts and exercises them through existing HTTP, URL policy, and OCR provider-default consumers. Implementation, runtime validation, and PR creation remain future work + +**What already exists** + +`host-python/src/marshal.rs` already uses `pythonize` directly, separates internal conversion from public argument errors, and contains serializer panics in `Pythonized`. Keep those entrypoints. `Pythonized` currently stringifies conversion errors, unlike `from_py` and `to_py`, so its error transfer needs correction + +`core-utils/src/serde_compat.rs` already provides composable `LaxI64` and `FiniteF64` adapters. They first deserialize into `serde_json::Value`. Preserve their accepted input contracts while moving scalar decoding to Serde visitors, avoiding an intermediate JSON representation and making behavior through `pythonize` explicit + +`python-bridge/src/http.rs` currently uses strict derived Boolean and string extraction for mutable settings, converts extraction errors into `RustBridgeDeclined`, and silently drops unsupported `ssl_verify` values. OCR provider defaults have the same strict-extraction/decline pattern. `python_settings.json` checks field names only. These are the initial production consumers and regressions for the foundation + +**Ownership and placement** + +Keep interpreter attachment, structured-data conversion, panic containment, and execution adapters in `host-python`. Keep field paths, configuration error policy, named settings coercion, and product-specific tagged inputs in `python-bridge`. Keep pure parsing and Serde adapters in `core-utils`. This requires no new crate and no domain dependency from `host-python` into `core-utils` + +Add one focused `python-bridge/src/coercion.rs` module containing the field reader, semantic wrappers, and projection errors. Use `core-utils/src/serde_compat.rs` for shared token/numeric parsing and Serde decoding initially, splitting only if its size warrants it. The bridge can call those pure helpers through its existing dependency. Do not introduce a generic coercion registry, runtime manifest dispatch, or a new conversion framework + +Settings snapshots hold raw `Bound<'py, PyAny>` values only during attached projection. Successful adapters return owned values. Python snapshot dataclass annotations use `object` for arbitrary mutable globals, retaining strict annotations for accessor-owned fields such as `user_agent` and `readable`. No live settings object, iterator, or borrowed Python value enters native HTTP state + +**Serde and structured conversion** + +Keep `from_py` and `to_py` as the internal, exception-preserving conversion path. Make `Pythonized` use the same standard `PythonizeError` to `PyErr` conversion without losing its panic guard. Keep the explicitly argument-focused `ValueError` contract of `from_py_argument`; settings projection never passes through that helper + +Implement scalar visitors behind the existing `LaxI64` and `FiniteF64` adapter names. Preserve integer precision beyond the exact f64 range, signed bounds, supported decimal/underscore strings, Boolean numeric behavior, fractional rejection for integers, and nonfinite rejection for floats. Preserve composition inside `Option` and sequences, missing/null behavior at the field boundary, and ordinary numeric serialization. Use the existing [serde_with DeserializeAs contract](https://docs.rs/serde_with/3.16.1/serde_with/trait.DeserializeAs.html) + +Add the pure `parse_str_bool(&str) -> Option` parser used by bridge `StrBool`, HTTP `SslVerify::parse`, and the environment switch helper. It recognizes trimmed, case-insensitive true/false only. The environment helper retains its separate rule that only `Some(true)` contributes an enabled layer. Numeric helpers are shared only where a second actual consumer needs them + +Run common ordinary-value fixtures through JSON deserialization and direct Python-to-typed-Serde conversion. Restrict equivalence claims to their overlapping input domain. Live descriptors, identity, truthiness, iteration, and stringification are exercised through PyO3 separately. Do not add unused Serde counterparts for Python-only semantics or convert live settings through JSON text, `serde_json::Value`, `repr`, or `py_literal` + +**Named settings semantics** + +| Adapter | Contract and first consumer | +| --- | --- | +| `Truthy` | Execute Python truth testing and preserve its exception; IPv4, URL validation, and trust-env globals | +| `ExactTrue` | Compare identity with the True singleton without equality or truth testing; HTTP2, transport disable, and token refresh | +| `StrBool` | None or an actual string parsed by the shared parser; no arbitrary stringification | +| `OptionalStrictString` | None or an actual string, including the empty string; client certificate | +| `FalsyOptionalString` | Test truthiness first, treat falsey as absent, reject truthy non-strings; provider project and location | +| `TuningString` | Test truthiness first, then retain actual strings and ignore other values; TLS tuning | +| `StringCollection` | Apply the field's explicit container/member policy and return owned strings; URL allowlist | +| `SslVerifyInput` | Classify None, actual Boolean, Boolean string, CA path, live SSLContext, and invalid types separately | + +A single generic Boolean or optional-string coercer cannot implement these contracts. Accept string subclasses by their Unicode contents without calling overridden convenience methods. Use no `.ok()` or default value to discard an error from a Python protocol operation + +For URL hosts, a direct string represents one host. Otherwise test container truthiness and iterate, test member truthiness, skip falsey members, and reject truthy non-string members. Normalize with the existing URL-policy rules, deduplicate, and sort only where membership makes order irrelevant. Keep origin/scheme/port parsing in its existing domain helper rather than applying hostname normalization blindly to arbitrary URL strings + +**Errors and first production adoption** + +Represent projection failures as a tagged result separating original Python exceptions, invalid configuration, unsupported live objects, and internal accessor/schema failures. Map it once at the bridge: preserve original `PyErr`, use field-focused `ValueError` for invalid or unsupported configuration, and `RuntimeError` for genuine internal schema failures. Diagnostics contain group, field, expected forms, and actual type, never the supplied value or its representation + +Attribute, truthiness, iteration, and explicit stringification exceptions retain identity, traceback, cause, and context. In particular, do not relabel an AttributeError deliberately raised by a descriptor as a missing-field schema failure. Contract validation must distinguish schema drift from errors executing Python behavior + +Convert HTTP and URL snapshots, per-call `ssl_verify`, and OCR provider defaults through the named adapters. Preserve existing call/environment/global precedence and projection timing. Finish projection before constructing the native client or starting provider I/O. Invalid values and live SSLContext must raise configuration errors rather than disappear or authorize fallback + +Keep certificate paths through projection and validate empty, missing, or unusable client-certificate paths before I/O. The current native layer filters empty client-certificate paths, so correcting that narrow downstream behavior is part of adoption. Keep the existing CA-bundle missing-file policy explicit and separately tested; do not silently conflate it with client-certificate validation + +Classify the existing Secret Manager `readable` field as a strict accessor Boolean. Full Secret Manager client/system/settings snapshots and callback execution remain a separate PR. The existing readable-manager capability gap must be documented and must not be reported as fixed by this foundation + +**Semantic manifest** + +Extend `python_settings.json` with a stable group version and field records containing adapter ID, requiredness, precedence role, sensitivity, and specialized accepted/unsupported shapes. Include only the snapshot fields that exist in this PR. Update the Python contract test and a static Rust `SettingSpec` table to agree with the manifest + +The manifest checks declared contracts; behavioral tests prove the adapters implement them. Projection stays direct typed code. A manifest row alone is never evidence that a coercion works + +**Behavioral validation** + +Extend the existing mapped Python settings tests and Rust marshal/HTTP tests. A new coercion module may have its own focused Rust tests. Use the existing installed-extension OCR suites for public-route regressions. Do not add source-text assertions or class-attribute monkeypatching + +The acceptance matrix covers None, Boolean values, integer zero/one, strings, containers, subclasses, and arbitrary objects. Protocol fixtures raise pre-created exceptions from descriptors, `__bool__`, `__len__`, `__iter__`, and `__next__`; assert identity and exception chains. Verify ExactTrue never invokes hostile equality/truthiness. Verify falsey provider defaults preserve fallback, HTTP2 does not accept integer one, and a false/unknown environment token cannot switch off a true global + +Cover a real SSLContext, unsupported objects, Boolean strings, certificate path failures, direct-string hosts, sets, generators, duplicates, mixed members, and protocol failures. Mutate globals and source collections between calls: the next snapshot observes changes and an already projected value stays unchanged. At the installed public OCR boundary, projection failures must cause zero provider requests and zero Python fallback calls under required-native execution + +Run focused crate tests first, then the workspace Rust checks used by CI, `make test-rust-extension`, relevant Python settings tests, and `make check`. Use the fresh installed wheel and verify native provenance. Review the saved `make check` log rather than rerunning it to inspect output + +Target mutation tests at truthiness versus strict extraction, identity versus equality, swallowed versus preserved exceptions, string-as-one versus character iteration, normalization/deduplication, numeric bounds, and terminal errors versus fallback. Aim for more than 90% killed non-equivalent mutants in the changed coercion paths + +Before opening the implementation PR for maintainer review, provide a reproducible localhost proxy curl request with a real provider and positive native-execution evidence. Record the configured settings and user-visible result without credentials. Unit tests belong in validation, not the proof-of-fix section. Require the current tip's CI and coverage, Greptile confidence of at least 4/5, and acceptable Veria/Bugbot results; pending or unavailable results remain explicitly unresolved + +**Commit sequence and follow-ups** + +Start with the Serde visitor/error-transfer changes and their regression tests. Follow with field adapters and the shared token parser. Adopt them in HTTP, URL policy, and provider defaults together with the semantic manifest and installed-extension regressions. Keep these as reviewable commits in one foundational implementation PR + +Follow-up PRs can add `OptionalRedisBool`, cache-specific stringification and collection rules, and full Secret Manager bindings using the same field/error machinery. Redis accepts a different token set from StrBool, so do not share their Boolean semantics. Runtime redesign, callback lifecycle changes, free-threaded support, wholesale request serialization, and unrelated cache work are outside this PR From 52216df1ff3dffd8c97ee8ea459ebcd4cd20b17f Mon Sep 17 00:00:00 2001 From: Yujong Lee Date: Mon, 21 Sep 2026 13:01:57 -0700 Subject: [PATCH 2/6] fix(rust): preserve structured conversion semantics --- litellm-rust/Cargo.lock | 1 + .../crates/core-utils/src/serde_compat.rs | 112 +++++++++++++++--- .../crates/host-python/src/marshal.rs | 15 ++- litellm-rust/crates/python-bridge/Cargo.toml | 2 + .../crates/python-bridge/src/marshal.rs | 54 +++++++++ 5 files changed, 164 insertions(+), 20 deletions(-) diff --git a/litellm-rust/Cargo.lock b/litellm-rust/Cargo.lock index ea4f3d848c4..911cc571df9 100644 --- a/litellm-rust/Cargo.lock +++ b/litellm-rust/Cargo.lock @@ -2688,6 +2688,7 @@ dependencies = [ "rstest", "serde", "serde_json", + "serde_with", "tokio", "tokio-tungstenite", ] diff --git a/litellm-rust/crates/core-utils/src/serde_compat.rs b/litellm-rust/crates/core-utils/src/serde_compat.rs index bb2648eb0be..c767c709f50 100644 --- a/litellm-rust/crates/core-utils/src/serde_compat.rs +++ b/litellm-rust/crates/core-utils/src/serde_compat.rs @@ -1,33 +1,91 @@ -use serde::{Deserialize, Deserializer, de::Error}; -use serde_json::Value; +use serde::{ + Deserializer, + de::{Error, Visitor}, +}; use serde_with::DeserializeAs; pub struct LaxI64; pub struct FiniteF64; +pub fn parse_str_bool(value: &str) -> Option { + let token = value.trim_matches(|character: char| { + character.is_whitespace() || matches!(character, '\u{1c}'..='\u{1f}') + }); + if token.eq_ignore_ascii_case("true") { + return Some(true); + } + token.eq_ignore_ascii_case("false").then_some(false) +} + impl<'de> DeserializeAs<'de, i64> for LaxI64 { fn deserialize_as>(deserializer: D) -> Result { - match Value::deserialize(deserializer)? { - Value::Number(number) if number.is_f64() => number.as_f64().and_then(integral_float), - Value::Number(number) => number.as_i64(), - Value::String(value) => integer_string(value.trim()), - Value::Bool(value) => Some(i64::from(value)), - _ => None, - } - .ok_or_else(|| D::Error::custom("expected an integer in the i64 range")) + deserializer.deserialize_any(Self) + } +} + +impl<'de> Visitor<'de> for LaxI64 { + type Value = i64; + + fn expecting(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter.write_str("an integer in the i64 range") + } + + fn visit_i64(self, value: i64) -> Result { + Ok(value) + } + + fn visit_u64(self, value: u64) -> Result { + i64::try_from(value).map_err(E::custom) + } + + fn visit_f64(self, value: f64) -> Result { + integral_float(value).ok_or_else(|| E::custom("expected an integer in the i64 range")) + } + + fn visit_str(self, value: &str) -> Result { + integer_string(value.trim()) + .ok_or_else(|| E::custom("expected an integer in the i64 range")) + } + + fn visit_bool(self, value: bool) -> Result { + Ok(i64::from(value)) } } impl<'de> DeserializeAs<'de, f64> for FiniteF64 { fn deserialize_as>(deserializer: D) -> Result { - match Value::deserialize(deserializer)? { - Value::Number(number) => number.as_f64(), - Value::String(value) => value.trim().parse::().ok(), - Value::Bool(value) => Some(f64::from(value)), - _ => None, - } - .filter(|value| value.is_finite()) - .ok_or_else(|| D::Error::custom("expected a finite number")) + deserializer.deserialize_any(Self) + } +} + +impl<'de> Visitor<'de> for FiniteF64 { + type Value = f64; + + fn expecting(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter.write_str("a finite number") + } + + fn visit_i64(self, value: i64) -> Result { + Ok(value as f64) + } + + fn visit_u64(self, value: u64) -> Result { + Ok(value as f64) + } + + fn visit_f64(self, value: f64) -> Result { + value + .is_finite() + .then_some(value) + .ok_or_else(|| E::custom("expected a finite number")) + } + + fn visit_str(self, value: &str) -> Result { + self.visit_f64(value.trim().parse::().map_err(E::custom)?) + } + + fn visit_bool(self, value: bool) -> Result { + Ok(f64::from(value)) } } @@ -66,7 +124,7 @@ fn integral_float(value: f64) -> Option { #[cfg(test)] mod tests { - use serde::Serialize; + use serde::{Deserialize, Serialize}; use serde_json::json; use serde_with::serde_as; @@ -81,6 +139,22 @@ mod tests { float: Option, } + #[test] + fn boolean_tokens_follow_python_string_trimming_without_redis_tokens() { + for (input, expected) in [ + (" True ", Some(true)), + ("\u{1c}TRUE\u{1f}", Some(true)), + ("\u{a0}False\u{2003}", Some(false)), + ("true\u{200b}", None), + ("yes", None), + ("1", None), + ("", None), + ("unknown", None), + ] { + assert_eq!(parse_str_bool(input), expected, "{input:?}"); + } + } + #[test] fn adapters_compose_and_serialize_as_numbers() { let numbers: Numbers = serde_json::from_value(json!({ diff --git a/litellm-rust/crates/host-python/src/marshal.rs b/litellm-rust/crates/host-python/src/marshal.rs index 881ad0e0389..8f284abf9dd 100644 --- a/litellm-rust/crates/host-python/src/marshal.rs +++ b/litellm-rust/crates/host-python/src/marshal.rs @@ -45,7 +45,7 @@ where fn into_pyobject(self, py: Python<'py>) -> PyResult { catch_unwind(AssertUnwindSafe(|| pythonize::pythonize(py, &self.0))) .map_err(panic_to_pyerr)? - .map_err(|error| PyValueError::new_err(error.to_string())) + .map_err(PyErr::from) } } @@ -87,6 +87,19 @@ mod tests { }); } + #[test] + fn pythonized_preserves_python_serialization_error_types() { + crate::initialize_python(); + Python::attach(|py| { + let value = std::collections::BTreeMap::from([(vec![1], "value")]); + let direct = to_py(py, &value).unwrap_err(); + let wrapped = Pythonized(value).into_pyobject(py).unwrap_err(); + assert!(direct.is_instance_of::(py)); + assert!(wrapped.is_instance_of::(py)); + assert_eq!(wrapped.to_string(), direct.to_string()); + }); + } + #[test] fn pythonized_maps_serializer_panics_to_a_base_exception() { crate::initialize_python(); diff --git a/litellm-rust/crates/python-bridge/Cargo.toml b/litellm-rust/crates/python-bridge/Cargo.toml index 1eb2ec28036..1bba83922f9 100644 --- a/litellm-rust/crates/python-bridge/Cargo.toml +++ b/litellm-rust/crates/python-bridge/Cargo.toml @@ -41,6 +41,8 @@ serde_json.workspace = true tokio = { workspace = true, features = ["sync"] } [dev-dependencies] +serde.workspace = true +serde_with.workspace = true criterion.workspace = true futures-util.workspace = true rstest.workspace = true diff --git a/litellm-rust/crates/python-bridge/src/marshal.rs b/litellm-rust/crates/python-bridge/src/marshal.rs index 2aba51cc4ff..fe5d551a931 100644 --- a/litellm-rust/crates/python-bridge/src/marshal.rs +++ b/litellm-rust/crates/python-bridge/src/marshal.rs @@ -172,6 +172,60 @@ mod tests { request_input_sources(&kwargs, names.iter().copied()) } + #[serde_with::serde_as] + #[derive(Debug, serde::Deserialize, serde::Serialize, PartialEq)] + struct Numbers { + #[serde_as(deserialize_as = "Option>")] + integers: Option>, + #[serde_as(deserialize_as = "Option")] + float: Option, + } + + #[test] + fn numeric_adapters_agree_across_json_and_python_boundaries() { + Python::initialize(); + Python::attach(|py| { + for input in [ + json!({}), + json!({"integers": null, "float": null}), + json!({"integers": [i64::MIN, i64::MAX, "9007199254740993.0", " +1_000.00 ", true, 3.0], "float": " 1.25 "}), + json!({"integers": [u64::MAX]}), + json!({"integers": ["1.0000000000000001"]}), + json!({"integers": [2.5]}), + json!({"float": "NaN"}), + json!({"float": "inf"}), + json!({"float": "1e999"}), + json!({"float": true}), + json!({"float": u64::MAX}), + ] { + let expected = serde_json::from_value::(input.clone()); + let python = litellm_host_python::to_py(py, &input).unwrap(); + let actual = from_py::(python.bind(py)); + match (expected, actual) { + (Ok(expected), Ok(actual)) => { + assert_eq!(actual, expected); + let serialized = litellm_host_python::to_py(py, &actual).unwrap(); + assert_eq!( + from_py::(serialized.bind(py)).unwrap(), + serde_json::to_value(expected).unwrap() + ); + } + (Err(_), Err(_)) => {} + mismatch => panic!("boundary mismatch for {input}: {mismatch:?}"), + } + } + for source in [ + c"{'float': float('nan')}", + c"{'float': float('inf')}", + c"{'integers': [float('inf')]}", + c"{'integers': [2 ** 100]}", + ] { + let value = py.eval(source, None, None).unwrap(); + assert!(from_py::(&value).is_err()); + } + }); + } + #[test] fn argument_converters_keep_nested_values_and_accept_explicit_none() { Python::initialize(); From 5aeb367d2a42ec4050c7db17813dd95b1e9e5839 Mon Sep 17 00:00:00 2001 From: Yujong Lee Date: Mon, 21 Sep 2026 13:01:57 -0700 Subject: [PATCH 3/6] fix(rust): preserve Python settings coercion at the native boundary --- PYTHON_INTEROP_PLAN.md | 4 +- .../crates/core-utils/src/settings.rs | 4 +- litellm-rust/crates/http/src/media.rs | 2 +- litellm-rust/crates/http/src/settings.rs | 19 +- .../crates/python-bridge/python_settings.json | 176 +++++++-- .../crates/python-bridge/src/coercion.rs | 231 +++++++++++ .../python-bridge/src/coercion/tests.rs | 372 ++++++++++++++++++ litellm-rust/crates/python-bridge/src/http.rs | 192 +++++---- litellm-rust/crates/python-bridge/src/lib.rs | 1 + .../python-bridge/src/python_settings.rs | 216 ++++++++-- .../python-bridge/src/routes/ocr/mod.rs | 73 ++-- litellm/rust_bridge/settings.py | 29 +- .../test_litellm/rust_bridge/test_settings.py | 29 +- tests/test_litellm_rust/ocr/test_requests.py | 150 ++++++- 14 files changed, 1309 insertions(+), 189 deletions(-) create mode 100644 litellm-rust/crates/python-bridge/src/coercion.rs create mode 100644 litellm-rust/crates/python-bridge/src/coercion/tests.rs diff --git a/PYTHON_INTEROP_PLAN.md b/PYTHON_INTEROP_PLAN.md index b456d795cfd..a7783f81472 100644 --- a/PYTHON_INTEROP_PLAN.md +++ b/PYTHON_INTEROP_PLAN.md @@ -4,9 +4,9 @@ Proposed implementation PR title: `fix(rust): preserve Python settings semantics Base: `main` at `457b01e96d131f88df8cace8832f8e044ee5f167`. Planning branch: `litellm_python_interop_foundation` -This plan follows `migration/tdd/sections/rust/python-interop/{pyo3-contract,boundary,coercion}.typ` in the sibling `litellm-typst` checkout. The first PR establishes conversion contracts and exercises them through existing HTTP, URL policy, and OCR provider-default consumers. Implementation, runtime validation, and PR creation remain future work +This plan follows `migration/tdd/sections/rust/python-interop/{pyo3-contract,boundary,coercion}.typ` in the sibling `litellm-typst` checkout. The first PR establishes conversion contracts and exercises them through existing HTTP, URL policy, and OCR provider-default consumers. The foundation is implemented on this branch. No PR is being created for this task -**What already exists** +**Baseline before implementation** `host-python/src/marshal.rs` already uses `pythonize` directly, separates internal conversion from public argument errors, and contains serializer panics in `Pythonized`. Keep those entrypoints. `Pythonized` currently stringifies conversion errors, unlike `from_py` and `to_py`, so its error transfer needs correction diff --git a/litellm-rust/crates/core-utils/src/settings.rs b/litellm-rust/crates/core-utils/src/settings.rs index 59c76ce3015..293dc71871c 100644 --- a/litellm-rust/crates/core-utils/src/settings.rs +++ b/litellm-rust/crates/core-utils/src/settings.rs @@ -1,5 +1,7 @@ use std::str::FromStr; +use crate::serde_compat::parse_str_bool; + pub trait Lookup { fn get(&self, name: &str) -> Option; @@ -9,7 +11,7 @@ pub trait Lookup { fn enabled(&self, name: &str) -> Option { self.get(name) - .is_some_and(|value| value.trim().eq_ignore_ascii_case("true")) + .is_some_and(|value| parse_str_bool(&value) == Some(true)) .then_some(true) } diff --git a/litellm-rust/crates/http/src/media.rs b/litellm-rust/crates/http/src/media.rs index 3b29c9e28a7..753f25c29de 100644 --- a/litellm-rust/crates/http/src/media.rs +++ b/litellm-rust/crates/http/src/media.rs @@ -62,7 +62,7 @@ impl UrlPolicy { } } -fn normalize_host(host: &str) -> String { +pub fn normalize_host(host: &str) -> String { host.to_ascii_lowercase().trim_end_matches('.').to_owned() } diff --git a/litellm-rust/crates/http/src/settings.rs b/litellm-rust/crates/http/src/settings.rs index a6397f1e8e3..e1edc6d37e1 100644 --- a/litellm-rust/crates/http/src/settings.rs +++ b/litellm-rust/crates/http/src/settings.rs @@ -3,7 +3,10 @@ use std::{ time::Duration, }; -use litellm_core_utils::settings::{Layer, Lookup, merge}; +use litellm_core_utils::{ + serde_compat::parse_str_bool, + settings::{Layer, Lookup, merge}, +}; use crate::proxy::EnvironmentProxies; @@ -16,9 +19,9 @@ pub enum SslVerify { impl SslVerify { pub fn parse(value: &str) -> Self { - match value.trim().to_ascii_lowercase().as_str() { - "true" => Self::Enabled, - "false" => Self::Disabled, + match parse_str_bool(value) { + Some(true) => Self::Enabled, + Some(false) => Self::Disabled, _ => Self::CaBundle(PathBuf::from(value)), } } @@ -152,9 +155,7 @@ impl HttpSettings { Self { ssl_verify: merged.ssl_verify, ssl_cert_file: merged.ssl_cert_file, - ssl_certificate: merged - .ssl_certificate - .filter(|path| !path.as_os_str().is_empty()), + ssl_certificate: merged.ssl_certificate, ssl_security_level: merged.ssl_security_level.filter(|level| !level.is_empty()), ssl_ecdh_curve: merged.ssl_ecdh_curve.filter(|curve| !curve.is_empty()), force_ipv4: merged.force_ipv4.unwrap_or(defaults.force_ipv4), @@ -287,7 +288,7 @@ mod tests { } #[test] - fn empty_environment_values_clear_the_setting_like_python_truthiness() { + fn empty_certificate_is_retained_for_validation_while_empty_tuning_is_absent() { let configured = HttpSettingsLayer { ssl_certificate: Some("/configured/client.pem".into()), ssl_security_level: Some("configured".into()), @@ -300,7 +301,7 @@ mod tests { ("SSL_ECDH_CURVE", ""), ])); let settings = HttpSettings::from_layers([environment, configured]); - assert_eq!(settings.ssl_certificate, None); + assert_eq!(settings.ssl_certificate, Some(PathBuf::new())); assert_eq!(settings.ssl_security_level, None); assert_eq!(settings.ssl_ecdh_curve, None); } diff --git a/litellm-rust/crates/python-bridge/python_settings.json b/litellm-rust/crates/python-bridge/python_settings.json index 0af55083bef..ea53d1d2025 100644 --- a/litellm-rust/crates/python-bridge/python_settings.json +++ b/litellm-rust/crates/python-bridge/python_settings.json @@ -1,26 +1,154 @@ { - "http_settings": [ - "ssl_verify", - "ssl_certificate", - "ssl_security_level", - "ssl_ecdh_curve", - "force_ipv4", - "http2", - "aiohttp_trust_env", - "disable_aiohttp_trust_env", - "disable_aiohttp_transport", - "user_agent" - ], - "url_policy": [ - "user_url_validation", - "user_url_allowed_hosts" - ], - "provider_defaults": [ - "vertex_project", - "vertex_location", - "enable_azure_ad_token_refresh" - ], - "secret_manager": [ - "readable" - ] + "http_settings": { + "version": 1, + "fields": { + "ssl_verify": { + "adapter": "SslVerifyInput", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [ + "none", + "bool", + "str" + ], + "unsupported_live": "configuration_error" + }, + "ssl_certificate": { + "adapter": "OptionalStrictString", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "ssl_security_level": { + "adapter": "TuningString", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "ssl_ecdh_curve": { + "adapter": "TuningString", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "force_ipv4": { + "adapter": "Truthy", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "http2": { + "adapter": "ExactTrue", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "aiohttp_trust_env": { + "adapter": "Truthy", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "disable_aiohttp_trust_env": { + "adapter": "Truthy", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "disable_aiohttp_transport": { + "adapter": "ExactTrue", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "user_agent": { + "adapter": "StrictString", + "required": true, + "precedence": "accessor", + "sensitive": false, + "shapes": [], + "unsupported_live": null + } + } + }, + "url_policy": { + "version": 1, + "fields": { + "user_url_validation": { + "adapter": "Truthy", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + }, + "user_url_allowed_hosts": { + "adapter": "HostCollection", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + } + } + }, + "provider_defaults": { + "version": 1, + "fields": { + "vertex_project": { + "adapter": "FalsyOptionalString", + "required": true, + "precedence": "module_global", + "sensitive": true, + "shapes": [], + "unsupported_live": null + }, + "vertex_location": { + "adapter": "FalsyOptionalString", + "required": true, + "precedence": "module_global", + "sensitive": true, + "shapes": [], + "unsupported_live": null + }, + "enable_azure_ad_token_refresh": { + "adapter": "ExactTrue", + "required": true, + "precedence": "module_global", + "sensitive": false, + "shapes": [], + "unsupported_live": null + } + } + }, + "secret_manager": { + "version": 1, + "fields": { + "readable": { + "adapter": "StrictBool", + "required": true, + "precedence": "accessor", + "sensitive": false, + "shapes": [], + "unsupported_live": null + } + } + } } diff --git a/litellm-rust/crates/python-bridge/src/coercion.rs b/litellm-rust/crates/python-bridge/src/coercion.rs new file mode 100644 index 00000000000..bb5b8b2d454 --- /dev/null +++ b/litellm-rust/crates/python-bridge/src/coercion.rs @@ -0,0 +1,231 @@ +use std::collections::BTreeSet; + +use litellm_core_utils::serde_compat::parse_str_bool; +use litellm_http::SslVerify; +use pyo3::{ + exceptions::{PyAttributeError, PyRuntimeError, PyValueError}, + prelude::*, + types::{PyBool, PyString}, +}; + +#[derive(Debug)] +pub(crate) enum ProjectionError { + Python(PyErr), + InvalidConfiguration(String), + UnsupportedLiveObject(String), + InternalSchemaFailure(String), +} + +impl From for ProjectionError { + fn from(error: PyErr) -> Self { + Self::Python(error) + } +} + +impl From for PyErr { + fn from(error: ProjectionError) -> Self { + match error { + ProjectionError::Python(error) => error, + ProjectionError::InvalidConfiguration(message) + | ProjectionError::UnsupportedLiveObject(message) => PyValueError::new_err(message), + ProjectionError::InternalSchemaFailure(message) => PyRuntimeError::new_err(message), + } + } +} + +pub(crate) struct Truthy(pub bool); +pub(crate) struct ExactTrue(pub bool); +pub(crate) struct StrBool(pub Option); +pub(crate) struct OptionalStrictString(pub Option); +pub(crate) struct FalsyOptionalString(pub Option); +pub(crate) struct TuningString(pub Option); +pub(crate) struct StringCollection(pub Vec); +pub(crate) struct SslVerifyInput(pub Option); + +pub(crate) struct Field<'py> { + path: &'static str, + value: Bound<'py, PyAny>, +} + +impl<'py> Field<'py> { + pub(crate) fn new(path: &'static str, value: Bound<'py, PyAny>) -> Self { + Self { path, value } + } + + pub(crate) fn read( + snapshot: &Bound<'py, PyAny>, + path: &'static str, + ) -> Result { + let name = path.rsplit('.').next().unwrap_or(path); + match snapshot.getattr(name) { + Ok(value) => Ok(Self::new(path, value)), + Err(error) if error.is_instance_of::(snapshot.py()) => { + match Self::missing_field(snapshot, name) { + Ok(true) => Err(ProjectionError::InternalSchemaFailure(format!( + "{path}: missing snapshot field" + ))), + _ => Err(error.into()), + } + } + Err(error) => Err(error.into()), + } + } + + fn missing_field(snapshot: &Bound<'_, PyAny>, name: &str) -> PyResult { + let py = snapshot.py(); + let object = py.import("builtins")?.getattr("object")?; + let missing = object.call0()?; + let lookup = py.import("inspect")?.getattr("getattr_static")?; + let declared = lookup.call1((snapshot, name, &missing))?; + let fallback = lookup.call1((snapshot.get_type(), "__getattr__", &missing))?; + let getter = lookup.call1((snapshot.get_type(), "__getattribute__"))?; + Ok(declared.is(&missing) + && fallback.is(&missing) + && getter.is(object.getattr("__getattribute__")?)) + } + + fn expected(&self, expected: &'static str) -> Result { + Ok(format!( + "{}: expected {expected}, got {}", + self.path, + self.value.get_type().name()? + )) + } + + fn invalid(&self, expected: &'static str) -> ProjectionError { + match self.expected(expected) { + Ok(message) => ProjectionError::InvalidConfiguration(message), + Err(error) => error, + } + } + + pub(crate) fn truthy(&self) -> Result { + Ok(Truthy(self.value.is_truthy()?)) + } + + pub(crate) fn exact_true(&self) -> ExactTrue { + ExactTrue(self.value.is(PyBool::new(self.value.py(), true))) + } + + pub(crate) fn strict_string(&self) -> Result { + let value = self + .value + .cast::() + .map_err(|_| self.invalid("a string"))?; + Ok(value.to_str()?.to_owned()) + } + + pub(crate) fn schema_string(&self) -> Result { + if !self.value.is_instance_of::() { + return Err(ProjectionError::InternalSchemaFailure( + self.expected("a string")?, + )); + } + self.strict_string() + } + + pub(crate) fn schema_bool(&self) -> Result { + if !self.value.is_instance_of::() { + return Err(ProjectionError::InternalSchemaFailure( + self.expected("a Boolean")?, + )); + } + Ok(self.exact_true().0) + } + + pub(crate) fn str_bool(&self) -> Result { + if self.value.is_none() { + return Ok(StrBool(None)); + } + Ok(StrBool(parse_str_bool(&self.strict_string()?))) + } + + pub(crate) fn optional_strict_string(&self) -> Result { + if self.value.is_none() { + return Ok(OptionalStrictString(None)); + } + self.strict_string().map(Some).map(OptionalStrictString) + } + + pub(crate) fn falsy_optional_string(&self) -> Result { + if !self.truthy()?.0 { + return Ok(FalsyOptionalString(None)); + } + self.strict_string().map(Some).map(FalsyOptionalString) + } + + pub(crate) fn tuning_string(&self) -> Result { + if !self.truthy()?.0 || !self.value.is_instance_of::() { + return Ok(TuningString(None)); + } + self.strict_string().map(Some).map(TuningString) + } + + pub(crate) fn string_collection(&self) -> Result { + if !self.truthy()?.0 { + return Ok(StringCollection(Vec::new())); + } + if self.value.is_instance_of::() { + return self + .strict_string() + .map(|value| StringCollection(vec![value])); + } + let values = self + .value + .try_iter()? + .filter_map(|item| { + let member = match item { + Ok(value) => Self::new(self.path, value), + Err(error) => return Some(Err(error.into())), + }; + match member.truthy() { + Ok(Truthy(false)) => None, + Ok(Truthy(true)) => Some(member.strict_string()), + Err(error) => Some(Err(error)), + } + }) + .collect::, ProjectionError>>()?; + Ok(StringCollection(values)) + } + + pub(crate) fn host_collection(&self) -> Result { + let values = self + .string_collection()? + .0 + .into_iter() + .map(|host| litellm_http::media::normalize_host(&host)) + .collect::>(); + Ok(StringCollection(values.into_iter().collect())) + } + + pub(crate) fn ssl_verify(&self) -> Result { + if self.value.is_none() { + return Ok(SslVerifyInput(None)); + } + if self.value.is_instance_of::() { + return Ok(SslVerifyInput(Some(if self.exact_true().0 { + SslVerify::Enabled + } else { + SslVerify::Disabled + }))); + } + if self.value.is_instance_of::() { + let parsed = match self.str_bool()?.0 { + Some(true) => SslVerify::Enabled, + Some(false) => SslVerify::Disabled, + None => SslVerify::CaBundle(self.strict_string()?.into()), + }; + return Ok(SslVerifyInput(Some(parsed))); + } + let context = self.value.py().import("ssl")?.getattr("SSLContext")?; + if self.value.is_instance(&context)? { + return Err(ProjectionError::UnsupportedLiveObject(self.expected( + "a Boolean, Boolean string, CA path, or None; live SSLContext is unsupported", + )?)); + } + Err(self.invalid("a Boolean, Boolean string, CA path, or None")) + } +} + +#[cfg(test)] +mod tests; diff --git a/litellm-rust/crates/python-bridge/src/coercion/tests.rs b/litellm-rust/crates/python-bridge/src/coercion/tests.rs new file mode 100644 index 00000000000..5ed237c3c64 --- /dev/null +++ b/litellm-rust/crates/python-bridge/src/coercion/tests.rs @@ -0,0 +1,372 @@ +use std::ffi::CString; + +use pyo3::{ + exceptions::{PyLookupError, PyRuntimeError, PyValueError}, + types::PyDict, +}; +use rstest::rstest; + +use super::*; + +fn evaluate<'py>(py: Python<'py>, source: &str) -> Bound<'py, PyAny> { + py.eval(&CString::new(source).unwrap(), None, None).unwrap() +} + +#[rstest] +#[case("None", false, false)] +#[case("False", false, false)] +#[case("True", true, true)] +#[case("0", false, false)] +#[case("1", true, false)] +#[case("''", false, false)] +#[case("'false'", true, false)] +#[case("[]", false, false)] +#[case("[0]", true, false)] +#[case("{}", false, false)] +#[case("object()", true, false)] +fn boolean_operations_have_distinct_python_semantics( + #[case] source: &str, + #[case] truth: bool, + #[case] exact: bool, +) { + Python::initialize(); + Python::attach(|py| { + let value = evaluate(py, source); + let field = Field::new("test.flag", value.clone()); + assert_eq!(field.truthy().unwrap().0, truth); + assert_eq!(field.exact_true().0, exact); + assert_eq!( + field.truthy().unwrap().0, + py.import("builtins") + .unwrap() + .getattr("bool") + .unwrap() + .call1((value,)) + .unwrap() + .extract::() + .unwrap() + ); + }); +} + +#[rstest] +#[case("None", Ok(None), Ok(None), Ok(None))] +#[case("''", Ok(Some("")), Ok(None), Ok(None))] +#[case( + "' value '", + Ok(Some(" value ")), + Ok(Some(" value ")), + Ok(Some(" value ")) +)] +#[case("[]", Err(()), Ok(None), Ok(None))] +#[case("0", Err(()), Ok(None), Ok(None))] +#[case("1", Err(()), Err(()), Ok(None))] +#[case("object()", Err(()), Err(()), Ok(None))] +fn string_operations_do_not_conflate_absence_and_type_checks( + #[case] source: &str, + #[case] strict: Result, ()>, + #[case] fallback: Result, ()>, + #[case] tuning: Result, ()>, +) { + Python::initialize(); + Python::attach(|py| { + let field = Field::new("test.string", evaluate(py, source)); + let owned = + |expected: Result, ()>| expected.map(|value| value.map(str::to_owned)); + assert_eq!( + field + .optional_strict_string() + .map(|value| value.0) + .map_err(|_| ()), + owned(strict) + ); + assert_eq!( + field + .falsy_optional_string() + .map(|value| value.0) + .map_err(|_| ()), + owned(fallback) + ); + assert_eq!( + field.tuning_string().map(|value| value.0).map_err(|_| ()), + owned(tuning) + ); + }); +} + +#[rstest] +#[case("None", None)] +#[case("' True '", Some(true))] +#[case("' fAlSe '", Some(false))] +#[case("'yes'", None)] +#[case("'1'", None)] +#[case("'unknown'", None)] +fn string_boolean_tokens_remain_separate_from_truthiness( + #[case] source: &str, + #[case] expected: Option, +) { + Python::initialize(); + Python::attach(|py| { + assert_eq!( + Field::new("test.flag", evaluate(py, source)) + .str_bool() + .unwrap() + .0, + expected + ); + }); +} + +#[rstest] +#[case("'EXAMPLE.TEST.'", vec!["example.test"])] +#[case("['B.test', '', None, 0, [], 'A.test.', 'b.test']", vec!["a.test", "b.test"])] +#[case("('B.test', 'a.test')", vec!["a.test", "b.test"])] +#[case("{'B.test', 'a.test'}", vec!["a.test", "b.test"])] +#[case("(host for host in ['B.test', 'a.test'])", vec!["a.test", "b.test"])] +#[case("None", vec![])] +#[case("False", vec![])] +fn host_collection_is_owned_normalized_and_deterministic( + #[case] source: &str, + #[case] expected: Vec<&str>, +) { + Python::initialize(); + Python::attach(|py| { + assert_eq!( + Field::new("url_policy.user_url_allowed_hosts", evaluate(py, source)) + .host_collection() + .unwrap() + .0, + expected + ); + }); +} + +#[test] +fn protocol_errors_preserve_exception_identity_traceback_cause_and_context() { + Python::initialize(); + Python::attach(|py| { + let locals = PyDict::new(py); + py.run( + c" +failure = LookupError('protocol failed') +cause = ValueError('cause') +context = RuntimeError('context') +def fail(): + try: + raise context + except RuntimeError: + raise failure from cause +class Bool: + def __bool__(self): return fail() +class Length: + def __len__(self): return fail() +class Iter: + def __iter__(self): return fail() +class Next: + def __iter__(self): return self + def __next__(self): return fail() +class Descriptor: + @property + def flag(self): return fail() +values = (Bool(), Length(), Iter(), Next(), [Bool()]) +descriptor = Descriptor() +", + Some(&locals), + Some(&locals), + ) + .unwrap(); + let values = locals.get_item("values").unwrap().unwrap(); + for value in values.try_iter().unwrap() { + let error = Field::new("test.flag", value.unwrap()) + .host_collection() + .err() + .unwrap(); + let error = PyErr::from(error); + assert!( + error + .value(py) + .is(locals.get_item("failure").unwrap().unwrap()) + ); + assert!(error.is_instance_of::(py)); + assert!(error.traceback(py).is_some()); + assert!( + error + .value(py) + .getattr("__cause__") + .unwrap() + .is(locals.get_item("cause").unwrap().unwrap()) + ); + assert!( + error + .value(py) + .getattr("__context__") + .unwrap() + .is(locals.get_item("context").unwrap().unwrap()) + ); + } + let error = Field::read( + &locals.get_item("descriptor").unwrap().unwrap(), + "test.flag", + ) + .err() + .unwrap(); + assert!( + PyErr::from(error) + .value(py) + .is(locals.get_item("failure").unwrap().unwrap()) + ); + }); +} + +#[test] +fn identity_and_string_contents_do_not_invoke_unrelated_protocols() { + Python::initialize(); + Python::attach(|py| { + let locals = PyDict::new(py); + py.run( + c" +class Hostile: + def __bool__(self): raise AssertionError('bool called') + def __eq__(self, other): raise AssertionError('eq called') + def __str__(self): raise AssertionError('str called') +class Text(str): + def __str__(self): raise AssertionError('str called') + def strip(self): raise AssertionError('strip called') + def lower(self): raise AssertionError('lower called') +hostile = Hostile() +text = Text(' False ') +", + Some(&locals), + Some(&locals), + ) + .unwrap(); + let hostile = Field::new("test.flag", locals.get_item("hostile").unwrap().unwrap()); + assert!(!hostile.exact_true().0); + assert!(matches!( + hostile.strict_string(), + Err(ProjectionError::InvalidConfiguration(_)) + )); + let text = Field::new("test.flag", locals.get_item("text").unwrap().unwrap()); + assert_eq!(text.strict_string().unwrap(), " False "); + assert_eq!(text.str_bool().unwrap().0, Some(false)); + }); +} + +#[test] +fn missing_snapshot_fields_and_descriptor_attribute_errors_are_distinct() { + Python::initialize(); + Python::attach(|py| { + let locals = PyDict::new(py); + py.run( + c" +failure = AttributeError('descriptor failed') +class Snapshot: + @property + def flag(self): raise failure +snapshot = Snapshot() +class Dynamic: + def __getattr__(self, name): raise failure +class Intercepted: + def __getattribute__(self, name): raise failure +dynamic = Dynamic() +intercepted = Intercepted() +", + Some(&locals), + Some(&locals), + ) + .unwrap(); + let snapshot = locals.get_item("snapshot").unwrap().unwrap(); + let descriptor = PyErr::from(Field::read(&snapshot, "test.flag").err().unwrap()); + assert!( + descriptor + .value(py) + .is(locals.get_item("failure").unwrap().unwrap()) + ); + for name in ["dynamic", "intercepted"] { + let value = locals.get_item(name).unwrap().unwrap(); + let error = PyErr::from(Field::read(&value, "test.flag").err().unwrap()); + assert!( + error + .value(py) + .is(locals.get_item("failure").unwrap().unwrap()) + ); + } + let missing = PyErr::from(Field::read(&snapshot, "test.missing").err().unwrap()); + assert!(missing.is_instance_of::(py)); + assert!(missing.to_string().contains("test.missing")); + }); +} + +#[test] +fn configuration_errors_name_fields_without_exposing_values() { + Python::initialize(); + Python::attach(|py| { + for source in [ + "{'secret': 'do-not-print'}", + "['host.test', {'secret': 'do-not-print'}]", + ] { + let field = Field::new("test.setting", evaluate(py, source)); + let error = PyErr::from(field.falsy_optional_string().err().unwrap()); + assert!(error.is_instance_of::(py)); + assert!(error.to_string().contains("test.setting")); + assert!(!error.to_string().contains("do-not-print")); + } + let hosts = Field::new( + "url_policy.user_url_allowed_hosts", + evaluate(py, "['host.test', 1]"), + ); + assert!(matches!( + hosts.host_collection(), + Err(ProjectionError::InvalidConfiguration(_)) + )); + assert!(matches!( + Field::new("test.flag", evaluate(py, "1")).str_bool(), + Err(ProjectionError::InvalidConfiguration(_)) + )); + }); +} + +#[test] +fn projection_releases_the_source_collection() { + Python::initialize(); + Python::attach(|py| { + let source = evaluate(py, "['A.test']"); + let projected = Field::new("test.hosts", source.clone()) + .host_collection() + .unwrap() + .0; + source.call_method1("append", ("b.test",)).unwrap(); + assert_eq!(projected, ["a.test"]); + assert_eq!( + Field::new("test.hosts", source) + .host_collection() + .unwrap() + .0, + ["a.test", "b.test"] + ); + }); +} + +#[rstest] +#[case("True", Some(true))] +#[case("False", Some(false))] +#[case("1", None)] +#[case("None", None)] +#[case("[]", None)] +fn accessor_booleans_are_strict_schema_values( + #[case] source: &str, + #[case] expected: Option, +) { + Python::initialize(); + Python::attach(|py| { + let result = Field::new("secret_manager.readable", evaluate(py, source)).schema_bool(); + match expected { + Some(expected) => assert_eq!(result.unwrap(), expected), + None => { + let error = PyErr::from(result.unwrap_err()); + assert!(error.is_instance_of::(py)); + assert!(error.to_string().contains("secret_manager.readable")); + } + } + }); +} diff --git a/litellm-rust/crates/python-bridge/src/http.rs b/litellm-rust/crates/python-bridge/src/http.rs index 7e9a5f093b4..859d579129f 100644 --- a/litellm-rust/crates/python-bridge/src/http.rs +++ b/litellm-rust/crates/python-bridge/src/http.rs @@ -10,9 +10,9 @@ use litellm_http::{ Unsupported, media::{PublicDnsResolver, UrlPolicy}, }; -use pyo3::{prelude::*, types::PyDict}; +use pyo3::{exceptions::PyValueError, prelude::*, types::PyDict}; -use crate::{errors::RustBridgeDeclined, python_settings::PythonSettings}; +use crate::{coercion::Field, python_settings::PythonSettings}; static POOL: LazyLock = LazyLock::new(|| HttpClientPool::new(Arc::new(PublicDnsResolver))); @@ -41,6 +41,22 @@ pub(crate) fn call_config( Ok(resolution.config) } +pub(crate) fn client_error(error: litellm_http::Error, config: &HttpClientConfig) -> PyErr { + match error { + litellm_http::Error::Read { path, .. } | litellm_http::Error::InvalidPem { path, .. } + if config.client_certificate.as_ref() == Some(&path) => + { + PyValueError::new_err( + "http_settings.ssl_certificate: expected a readable PEM certificate and private key", + ) + } + litellm_http::Error::Read { .. } | litellm_http::Error::InvalidPem { .. } => { + PyValueError::new_err("http_settings.ssl_verify: expected a readable PEM CA bundle") + } + _ => PyValueError::new_err("http_settings: native HTTP client configuration is invalid"), + } +} + fn unreported( reported: &Mutex>, unsupported: Vec, @@ -53,25 +69,25 @@ fn unreported( } pub(crate) fn url_policy(py: Python<'_>) -> PyResult { - let policy: PythonUrlPolicy = - PythonSettings::UrlPolicy - .read(py)? - .extract() - .map_err(|error: PyErr| { - RustBridgeDeclined::new_err(format!( - "litellm URL policy cannot be used by the Rust route: {error}" - )) - })?; + project_url_policy(&PythonSettings::UrlPolicy.read(py)?) +} + +fn project_url_policy(value: &Bound<'_, PyAny>) -> PyResult { Ok(UrlPolicy { - validate: policy.user_url_validation, - allowed_hosts: policy.user_url_allowed_hosts, + validate: Field::read(value, "url_policy.user_url_validation")? + .truthy()? + .0, + allowed_hosts: Field::read(value, "url_policy.user_url_allowed_hosts")? + .host_collection()? + .0, }) } fn call_ssl_verify(kwargs: &Bound<'_, PyDict>) -> PyResult> { - Ok(kwargs - .get_item("ssl_verify")? - .and_then(|value| ssl_verify(&value))) + match kwargs.get_item("ssl_verify")? { + Some(value) => Ok(Field::new("request.ssl_verify", value).ssl_verify()?.0), + None => Ok(None), + } } fn for_call(call_ssl_verify: Option, asynchronous: bool) -> HttpSettingsLayer { @@ -82,64 +98,47 @@ fn for_call(call_ssl_verify: Option, asynchronous: bool) -> HttpSetti } } -#[derive(FromPyObject)] -struct PythonUrlPolicy { - user_url_validation: bool, - user_url_allowed_hosts: Vec, -} - -#[derive(FromPyObject)] -struct PythonHttpSettings<'py> { - ssl_verify: Bound<'py, PyAny>, - ssl_certificate: Option, - ssl_security_level: Option, - ssl_ecdh_curve: Option, - force_ipv4: bool, - http2: bool, - aiohttp_trust_env: bool, - disable_aiohttp_trust_env: bool, - disable_aiohttp_transport: bool, - user_agent: String, -} - fn configured(value: &Bound<'_, PyAny>) -> PyResult { - let python: PythonHttpSettings = value.extract().map_err(|error: PyErr| { - RustBridgeDeclined::new_err(format!( - "litellm HTTP settings cannot be used by the Rust route: {error}" - )) - })?; Ok(HttpSettingsLayer { - ssl_verify: ssl_verify(&python.ssl_verify), - ssl_certificate: python.ssl_certificate.map(PathBuf::from), - ssl_security_level: python.ssl_security_level, - ssl_ecdh_curve: python.ssl_ecdh_curve, - force_ipv4: Some(python.force_ipv4), - http2: Some(python.http2), - aiohttp_trust_env: Some(python.aiohttp_trust_env), - disable_aiohttp_trust_env: Some(python.disable_aiohttp_trust_env), - disable_aiohttp_transport: Some(python.disable_aiohttp_transport), - user_agent: Some(python.user_agent), + ssl_verify: Field::read(value, "http_settings.ssl_verify")? + .ssl_verify()? + .0, + ssl_certificate: Field::read(value, "http_settings.ssl_certificate")? + .optional_strict_string()? + .0 + .map(PathBuf::from), + ssl_security_level: Field::read(value, "http_settings.ssl_security_level")? + .tuning_string()? + .0, + ssl_ecdh_curve: Field::read(value, "http_settings.ssl_ecdh_curve")? + .tuning_string()? + .0, + force_ipv4: Some(Field::read(value, "http_settings.force_ipv4")?.truthy()?.0), + http2: Some(Field::read(value, "http_settings.http2")?.exact_true().0), + aiohttp_trust_env: Some( + Field::read(value, "http_settings.aiohttp_trust_env")? + .truthy()? + .0, + ), + disable_aiohttp_trust_env: Some( + Field::read(value, "http_settings.disable_aiohttp_trust_env")? + .truthy()? + .0, + ), + disable_aiohttp_transport: Some( + Field::read(value, "http_settings.disable_aiohttp_transport")? + .exact_true() + .0, + ), + user_agent: Some(Field::read(value, "http_settings.user_agent")?.schema_string()?), ..HttpSettingsLayer::default() }) } -fn ssl_verify(value: &Bound<'_, PyAny>) -> Option { - if let Ok(enabled) = value.extract::() { - return Some(if enabled { - SslVerify::Enabled - } else { - SslVerify::Disabled - }); - } - value - .extract::() - .ok() - .map(|path| SslVerify::parse(&path)) -} - #[cfg(test)] mod tests { use litellm_http::Verify; + use pyo3::exceptions::PyRuntimeError; use rstest::rstest; use super::*; @@ -163,7 +162,7 @@ defaults = dict( user_agent='litellm/test', ) defaults.update(dict({overrides})) -settings = types.SimpleNamespace(**{{name: defaults[name] for name in json.loads(contract)['http_settings']}}) +settings = types.SimpleNamespace(**{{name: defaults[name] for name in json.loads(contract)['http_settings']['fields']}}) " ); let locals = PyDict::new(py); @@ -259,12 +258,16 @@ user_agent='litellm/9.9.9', }); } - #[test] - fn ssl_context_global_is_ignored_so_environment_and_defaults_apply() { + #[rstest] + #[case("ssl_verify=object()")] + #[case("ssl_verify=__import__('ssl').SSLContext(__import__('ssl').PROTOCOL_TLS_CLIENT)")] + #[case("ssl_certificate=1")] + fn invalid_http_configuration_is_terminal(#[case] overrides: &str) { Python::initialize(); Python::attach(|py| { - let layer = configured(&python_settings(py, "ssl_verify=object()")).unwrap(); - assert_eq!(layer.ssl_verify, None); + let error = configured(&python_settings(py, overrides)).unwrap_err(); + assert!(error.is_instance_of::(py)); + assert!(error.to_string().contains("http_settings.ssl_")); }); } @@ -281,11 +284,21 @@ user_agent='litellm/9.9.9', } #[test] - fn mistyped_python_settings_decline_instead_of_raising() { + fn mutable_globals_use_their_consumer_operations() { Python::initialize(); Python::attach(|py| { - let error = configured(&python_settings(py, "force_ipv4='yes'")).unwrap_err(); - assert!(error.is_instance_of::(py)); + let layer = configured(&python_settings(py, + "force_ipv4='yes', http2=1, disable_aiohttp_transport=1, aiohttp_trust_env=[1], disable_aiohttp_trust_env=[], ssl_security_level=1, ssl_ecdh_curve=[]" + )).unwrap(); + assert_eq!(layer.force_ipv4, Some(true)); + assert_eq!(layer.http2, Some(false)); + assert_eq!(layer.disable_aiohttp_transport, Some(false)); + assert_eq!(layer.aiohttp_trust_env, Some(true)); + assert_eq!(layer.disable_aiohttp_trust_env, Some(false)); + assert_eq!(layer.ssl_security_level, None); + assert_eq!(layer.ssl_ecdh_curve, None); + let error = configured(&python_settings(py, "user_agent=1")).unwrap_err(); + assert!(error.is_instance_of::(py)); }); } @@ -323,17 +336,36 @@ user_agent='litellm/9.9.9', } #[test] - fn live_ssl_context_argument_is_ignored_so_the_configured_value_applies() { + fn live_ssl_context_argument_raises_instead_of_using_another_layer() { Python::initialize(); Python::attach(|py| { let kwargs = PyDict::new(py); - kwargs - .set_item("ssl_verify", py.eval(c"object()", None, None).unwrap()) + let ssl = py.import("ssl").unwrap(); + let context = ssl + .getattr("SSLContext") + .unwrap() + .call1((ssl.getattr("PROTOCOL_TLS_CLIENT").unwrap(),)) .unwrap(); - let call = for_call(call_ssl_verify(&kwargs).unwrap(), true); - let settings = - HttpSettings::from_layers([call, configured_ssl_verify(SslVerify::Disabled)]); - assert_eq!(settings.ssl_verify, Some(SslVerify::Disabled)); + kwargs.set_item("ssl_verify", context).unwrap(); + let error = call_ssl_verify(&kwargs).unwrap_err(); + assert!(error.is_instance_of::(py)); + assert!(error.to_string().contains("request.ssl_verify")); + assert!(error.to_string().contains("SSLContext")); + }); + } + + #[test] + fn url_policy_uses_truthiness_and_normalized_owned_hosts() { + Python::initialize(); + Python::attach(|py| { + let value = py.eval(c"__import__('types').SimpleNamespace(user_url_validation=[], user_url_allowed_hosts=['B.test', 'a.test.', 'b.test'])", None, None).unwrap(); + assert_eq!( + project_url_policy(&value).unwrap(), + UrlPolicy { + validate: false, + allowed_hosts: vec!["a.test".into(), "b.test".into()], + } + ); }); } diff --git a/litellm-rust/crates/python-bridge/src/lib.rs b/litellm-rust/crates/python-bridge/src/lib.rs index bd62c5aadf1..f13a3ad433f 100644 --- a/litellm-rust/crates/python-bridge/src/lib.rs +++ b/litellm-rust/crates/python-bridge/src/lib.rs @@ -1,4 +1,5 @@ mod cache; +mod coercion; mod credentials; mod diagnostics; mod errors; diff --git a/litellm-rust/crates/python-bridge/src/python_settings.rs b/litellm-rust/crates/python-bridge/src/python_settings.rs index 7ac23a05542..bdc6d14356d 100644 --- a/litellm-rust/crates/python-bridge/src/python_settings.rs +++ b/litellm-rust/crates/python-bridge/src/python_settings.rs @@ -43,32 +43,204 @@ pub(crate) const CONTRACT: &str = include_str!("../python_settings.json"); #[cfg(test)] mod tests { - use std::{collections::BTreeSet, ffi::CString}; - - use pyo3::{prelude::*, types::PyDict}; - use super::{CONTRACT, PythonSettings}; + use pyo3::prelude::*; + use serde_json::{Value, json}; + + struct SettingSpec { + group: &'static str, + name: &'static str, + adapter: &'static str, + precedence: &'static str, + sensitive: bool, + shapes: &'static [&'static str], + unsupported_live: Option<&'static str>, + } + + const SETTINGS: &[SettingSpec] = &[ + SettingSpec { + group: "http_settings", + name: "ssl_verify", + adapter: "SslVerifyInput", + precedence: "module_global", + sensitive: false, + shapes: &["none", "bool", "str"], + unsupported_live: Some("configuration_error"), + }, + SettingSpec { + group: "http_settings", + name: "ssl_certificate", + adapter: "OptionalStrictString", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "ssl_security_level", + adapter: "TuningString", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "ssl_ecdh_curve", + adapter: "TuningString", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "force_ipv4", + adapter: "Truthy", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "http2", + adapter: "ExactTrue", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "aiohttp_trust_env", + adapter: "Truthy", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "disable_aiohttp_trust_env", + adapter: "Truthy", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "disable_aiohttp_transport", + adapter: "ExactTrue", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "http_settings", + name: "user_agent", + adapter: "StrictString", + precedence: "accessor", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "url_policy", + name: "user_url_validation", + adapter: "Truthy", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "url_policy", + name: "user_url_allowed_hosts", + adapter: "HostCollection", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "provider_defaults", + name: "vertex_project", + adapter: "FalsyOptionalString", + precedence: "module_global", + sensitive: true, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "provider_defaults", + name: "vertex_location", + adapter: "FalsyOptionalString", + precedence: "module_global", + sensitive: true, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "provider_defaults", + name: "enable_azure_ad_token_refresh", + adapter: "ExactTrue", + precedence: "module_global", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + SettingSpec { + group: "secret_manager", + name: "readable", + adapter: "StrictBool", + precedence: "accessor", + sensitive: false, + shapes: &[], + unsupported_live: None, + }, + ]; #[test] - fn every_settings_group_is_in_the_python_contract() { - Python::initialize(); - Python::attach(|py| { - let locals = PyDict::new(py); - locals.set_item("contract", CONTRACT).unwrap(); - let source = CString::new("import json\nkeys = list(json.loads(contract))").unwrap(); - py.run(&source, Some(&locals), Some(&locals)).unwrap(); - let declared: BTreeSet = locals - .get_item("keys") + fn settings_manifest_matches_the_semantic_contract() { + pyo3::Python::initialize(); + let manifest: Value = pyo3::Python::attach(|py| { + let value = py + .import("json") .unwrap() - .unwrap() - .extract::>() - .unwrap() - .into_iter() - .collect(); - let read: BTreeSet = PythonSettings::ALL - .map(|group| group.name().to_owned()) - .into(); - assert_eq!(read, declared); + .call_method1("loads", (CONTRACT,)) + .unwrap(); + litellm_host_python::from_py(&value).unwrap() }); + let expected: serde_json::Map = PythonSettings::ALL + .into_iter() + .map(|group| { + let fields: serde_json::Map = SETTINGS + .iter() + .filter(|spec| spec.group == group.name()) + .map(|spec| { + ( + spec.name.to_owned(), + json!({ + "adapter": spec.adapter, + "required": true, + "precedence": spec.precedence, + "sensitive": spec.sensitive, + "shapes": spec.shapes, + "unsupported_live": spec.unsupported_live, + }), + ) + }) + .collect(); + ( + group.name().to_owned(), + json!({"version": 1, "fields": fields}), + ) + }) + .collect(); + assert_eq!(manifest, Value::Object(expected)); } } diff --git a/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs b/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs index e518f972bac..ce6f04c321b 100644 --- a/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs +++ b/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs @@ -19,7 +19,7 @@ use pyo3::{ types::{PyDict, PyTuple}, }; -use crate::{errors::RustBridgeDeclined, http, python_settings::PythonSettings}; +use crate::{coercion::Field, errors::RustBridgeDeclined, http, python_settings::PythonSettings}; const SURFACE: LegacySurface = LegacySurface { call_type: "ocr", @@ -51,7 +51,7 @@ fn run_ocr( ocr_settings(py)?, secrets, ) - .map_err(|error| RustBridgeDeclined::new_err(error.to_string()))?; + .map_err(|error| http::client_error(error, &config))?; run_legacy_call( py, if asynchronous { ASYNC_SURFACE } else { SURFACE }, @@ -62,14 +62,8 @@ fn run_ocr( ) } -#[derive(FromPyObject)] -struct PythonSecretManager { - readable: bool, -} - fn process_environment_secrets(secret_manager: &Bound<'_, PyAny>) -> PyResult { - let manager: PythonSecretManager = secret_manager.extract()?; - if manager.readable { + if Field::read(secret_manager, "secret_manager.readable")?.schema_bool()? { return Err(RustBridgeDeclined::new_err( "a readable secret manager is configured and the Rust route only reads the process environment", )); @@ -77,26 +71,24 @@ fn process_environment_secrets(secret_manager: &Bound<'_, PyAny>) -> PyResult, - vertex_location: Option, - enable_azure_ad_token_refresh: Option, +fn ocr_settings(py: Python<'_>) -> PyResult { + project_provider_defaults(&PythonSettings::ProviderDefaults.read(py)?) } -fn ocr_settings(py: Python<'_>) -> PyResult { - let defaults: PythonProviderDefaults = PythonSettings::ProviderDefaults - .read(py)? - .extract() - .map_err(|error: PyErr| { - RustBridgeDeclined::new_err(format!( - "litellm provider defaults cannot be used by the Rust route: {error}" - )) - })?; +fn project_provider_defaults(value: &Bound<'_, PyAny>) -> PyResult { Ok(OcrSettings { - vertex_project: defaults.vertex_project, - vertex_location: defaults.vertex_location, - enable_azure_ad_token_refresh: defaults.enable_azure_ad_token_refresh == Some(true), + vertex_project: Field::read(value, "provider_defaults.vertex_project")? + .falsy_optional_string()? + .0, + vertex_location: Field::read(value, "provider_defaults.vertex_location")? + .falsy_optional_string()? + .0, + enable_azure_ad_token_refresh: Field::read( + value, + "provider_defaults.enable_azure_ad_token_refresh", + )? + .exact_true() + .0, ..OcrSettings::from_environment(&ProcessEnvironment) }) } @@ -140,6 +132,35 @@ mod tests { locals.get_item("manager").unwrap().unwrap() } + #[test] + fn provider_defaults_distinguish_falsey_values_and_exact_true() { + Python::initialize(); + Python::attach(|py| { + let value = py.eval(c"__import__('types').SimpleNamespace(vertex_project=[], vertex_location=0, enable_azure_ad_token_refresh=1)", None, None).unwrap(); + let projected = super::project_provider_defaults(&value).unwrap(); + assert_eq!(projected.vertex_project, None); + assert_eq!(projected.vertex_location, None); + assert!(!projected.enable_azure_ad_token_refresh); + value.setattr("vertex_project", "project").unwrap(); + value.setattr("vertex_location", "region").unwrap(); + value + .setattr("enable_azure_ad_token_refresh", true) + .unwrap(); + let next = super::project_provider_defaults(&value).unwrap(); + assert_eq!(next.vertex_project.as_deref(), Some("project")); + assert_eq!(next.vertex_location.as_deref(), Some("region")); + assert!(next.enable_azure_ad_token_refresh); + value.setattr("vertex_project", 1).unwrap(); + let error = super::project_provider_defaults(&value).err().unwrap(); + assert!(error.is_instance_of::(py)); + assert!( + error + .to_string() + .contains("provider_defaults.vertex_project") + ); + }); + } + #[test] fn a_readable_secret_manager_sends_the_call_back_to_python() { Python::initialize(); diff --git a/litellm/rust_bridge/settings.py b/litellm/rust_bridge/settings.py index 3aa2d742862..9a5cf49f298 100644 --- a/litellm/rust_bridge/settings.py +++ b/litellm/rust_bridge/settings.py @@ -1,34 +1,33 @@ from __future__ import annotations -from collections.abc import Sequence from dataclasses import dataclass @dataclass(frozen=True, slots=True) class HttpSettings: - ssl_verify: bool | str - ssl_certificate: str | None - ssl_security_level: str | None - ssl_ecdh_curve: str | None - force_ipv4: bool - http2: bool - aiohttp_trust_env: bool - disable_aiohttp_trust_env: bool - disable_aiohttp_transport: bool + ssl_verify: object + ssl_certificate: object + ssl_security_level: object + ssl_ecdh_curve: object + force_ipv4: object + http2: object + aiohttp_trust_env: object + disable_aiohttp_trust_env: object + disable_aiohttp_transport: object user_agent: str @dataclass(frozen=True, slots=True) class UrlPolicy: - user_url_validation: bool - user_url_allowed_hosts: Sequence[str] + user_url_validation: object + user_url_allowed_hosts: object @dataclass(frozen=True, slots=True) class ProviderDefaults: - vertex_project: str | None - vertex_location: str | None - enable_azure_ad_token_refresh: bool | None + vertex_project: object + vertex_location: object + enable_azure_ad_token_refresh: object @dataclass(frozen=True, slots=True) diff --git a/tests/test_litellm/rust_bridge/test_settings.py b/tests/test_litellm/rust_bridge/test_settings.py index 6b78ddad44b..023f02cffbb 100644 --- a/tests/test_litellm/rust_bridge/test_settings.py +++ b/tests/test_litellm/rust_bridge/test_settings.py @@ -6,6 +6,7 @@ from typing import Final import httpx import pytest from pydantic import TypeAdapter +from typing_extensions import ReadOnly, TypedDict import litellm from litellm.integrations.custom_secret_manager import CustomSecretManager @@ -17,14 +18,28 @@ from litellm.types.secret_managers.main import KeyManagementSettings, KeyManagem CONTRACT_PATH: Final = Path(__file__).parents[3] / "litellm-rust/crates/python-bridge/python_settings.json" -def test_the_rust_contract_matches_the_returned_fields() -> None: - contract: Final = TypeAdapter(dict[str, list[str]]).validate_json(CONTRACT_PATH.read_text()) +class SettingSpec(TypedDict): + adapter: ReadOnly[str] + required: ReadOnly[bool] + precedence: ReadOnly[str] + sensitive: ReadOnly[bool] + shapes: ReadOnly[list[str]] + unsupported_live: ReadOnly[str | None] - assert contract == { - "http_settings": [field.name for field in dataclasses.fields(settings.http_settings())], - "url_policy": [field.name for field in dataclasses.fields(settings.url_policy())], - "provider_defaults": [field.name for field in dataclasses.fields(settings.provider_defaults())], - "secret_manager": [field.name for field in dataclasses.fields(settings.secret_manager())], + +class SettingsGroup(TypedDict): + version: ReadOnly[int] + fields: ReadOnly[dict[str, SettingSpec]] + + +def test_the_rust_contract_matches_the_returned_fields() -> None: + contract: Final = TypeAdapter(dict[str, SettingsGroup]).validate_json(CONTRACT_PATH.read_text()) + + assert {name: tuple(group["fields"]) for name, group in contract.items()} == { + "http_settings": tuple(field.name for field in dataclasses.fields(settings.http_settings())), + "url_policy": tuple(field.name for field in dataclasses.fields(settings.url_policy())), + "provider_defaults": tuple(field.name for field in dataclasses.fields(settings.provider_defaults())), + "secret_manager": tuple(field.name for field in dataclasses.fields(settings.secret_manager())), } diff --git a/tests/test_litellm_rust/ocr/test_requests.py b/tests/test_litellm_rust/ocr/test_requests.py index 5e9d2c78808..51815651eb4 100644 --- a/tests/test_litellm_rust/ocr/test_requests.py +++ b/tests/test_litellm_rust/ocr/test_requests.py @@ -3,14 +3,13 @@ from collections.abc import Callable from dataclasses import dataclass from io import BytesIO from pathlib import Path -from typing import Final +from typing import Final, NoReturn import httpx import pytest from pydantic import JsonValue import litellm -from litellm.llms.base_llm.ocr.transformation import OCRResponse from tests.test_litellm_rust.support.callback_recorder import RecordingLogger from tests.test_litellm_rust.support.recording_server import RecordingServer, ResponseSpec from tests.test_litellm_rust.support.requests import ( @@ -503,3 +502,150 @@ async def test_native_failures_raise_the_public_exception_class( assert len(ocr_server.requests) == failure.provider_requests if failure.cause is not None: assert isinstance(caught.value.__context__, failure.cause) + + +@pytest.mark.asyncio +@pytest.mark.parametrize("asynchronous", [False, True], ids=["sync", "async"]) +@pytest.mark.parametrize( + "name,value", + [ + ("ssl_verify", object()), + ("ssl_certificate", 1), + ("ssl_certificate", ""), + ("vertex_project", 1), + ("vertex_location", ["region"]), + ("user_url_allowed_hosts", ["example.test", 1]), + ], +) +async def test_native_settings_fail_before_provider_io( + ocr_server: RecordingServer, + monkeypatch: pytest.MonkeyPatch, + asynchronous: bool, + name: str, + value: object, +) -> None: + ocr_server.expected_requests = 0 + monkeypatch.setattr(litellm, name, value) + with pytest.raises(ValueError, match=r"http_settings|provider_defaults|url_policy"): + await call_native(ocr_server, asynchronous, num_retries=0) + assert ocr_server.requests == [] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("asynchronous", [False, True], ids=["sync", "async"]) +async def test_native_ssl_context_is_terminal_configuration(ocr_server: RecordingServer, asynchronous: bool) -> None: + import ssl + + ocr_server.expected_requests = 0 + context: Final = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + with pytest.raises(ValueError, match=r"request\.ssl_verify.*SSLContext"): + await call_native(ocr_server, asynchronous, ssl_verify=context, num_retries=0) + assert ocr_server.requests == [] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("asynchronous", [False, True], ids=["sync", "async"]) +async def test_native_settings_preserve_protocol_failures( + ocr_server: RecordingServer, monkeypatch: pytest.MonkeyPatch, asynchronous: bool +) -> None: + ocr_server.expected_requests = 0 + failure: Final = LookupError("settings truth test failed") + cause: Final = RuntimeError("settings cause") + + class RaisesBool: + def __bool__(self) -> bool: + raise failure from cause + + monkeypatch.setattr(litellm, "force_ipv4", RaisesBool()) + with pytest.raises(LookupError) as caught: + await call_native(ocr_server, asynchronous, num_retries=0) + assert caught.value is failure + assert caught.value.__cause__ is cause + assert caught.value.__traceback__ is not None + assert ocr_server.requests == [] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("asynchronous", [False, True], ids=["sync", "async"]) +async def test_native_settings_observe_mutation_between_calls( + ocr_server: RecordingServer, monkeypatch: pytest.MonkeyPatch, asynchronous: bool +) -> None: + monkeypatch.setattr(litellm, "force_ipv4", "yes") + monkeypatch.setattr(litellm, "http2", 1) + monkeypatch.setattr(litellm, "vertex_project", []) + monkeypatch.setattr(litellm, "vertex_location", 0) + monkeypatch.setattr(litellm, "user_url_allowed_hosts", "EXAMPLE.TEST.") + response: Final = await call_native(ocr_server, asynchronous, num_retries=0) + assert response.pages[0].markdown == "native OCR response" + assert_native_request(ocr_server) + monkeypatch.setattr(litellm, "ssl_certificate", 1) + with pytest.raises(ValueError, match=r"http_settings\.ssl_certificate"): + await call_native(ocr_server, asynchronous, num_retries=0) + assert len(ocr_server.requests) == 1 + + +@pytest.mark.parametrize("required", [False, True]) +@pytest.mark.parametrize("failure", ["invalid", "live", "schema"]) +def test_native_projection_errors_never_select_python( + ocr_server: RecordingServer, monkeypatch: pytest.MonkeyPatch, required: bool, failure: str +) -> None: + import dataclasses + import ssl + + from litellm.rust_bridge import runtime, settings + from litellm.rust_bridge.catalog import Context, Route, Rule + from litellm.rust_bridge.configuration import Rollout + from litellm.rust_bridge.ocr.entrypoints import NATIVE_OCR, LiteLLMOcrRequest + + ocr_server.expected_requests = 0 + snapshot: Final = dataclasses.replace(settings.http_settings(), user_agent=1) + if failure == "schema": + monkeypatch.setattr(settings, "http_settings", lambda: snapshot) + else: + monkeypatch.setattr( + litellm, "ssl_verify", ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) if failure == "live" else object() + ) + request: Final = LiteLLMOcrRequest( + model="mistral/mistral-ocr-latest", + document=OCR_DOCUMENT, + api_key="test-key", + api_base=ocr_server.base_url, + timeout=None, + custom_llm_provider="mistral", + extra_headers=None, + kwargs={}, + ) + + def python_fallback() -> NoReturn: + pytest.fail("projection failures must not select Python") + + with pytest.raises(RuntimeError if failure == "schema" else ValueError, match="http_settings"): + runtime.run( + Context(Route.OCR, provider="mistral"), + binding=NATIVE_OCR, + native=lambda native: native(request, (), {}), + python=python_fallback, + rules=(Rule(Route.OCR, Rollout.RUST_REQUIRED if required else Rollout.RUST_OPT_OUT),), + ) + assert ocr_server.requests == [] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("asynchronous", [False, True], ids=["sync", "async"]) +@pytest.mark.parametrize("present", [False, True], ids=["missing", "invalid-pem"]) +async def test_native_client_certificate_is_validated_before_io( + ocr_server: RecordingServer, + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + asynchronous: bool, + present: bool, +) -> None: + ocr_server.expected_requests = 0 + certificate: Final = tmp_path / "client.pem" + if present: + certificate.write_text("invalid certificate") + monkeypatch.setattr(litellm, "ssl_certificate", str(certificate)) + with pytest.raises(ValueError, match=r"http_settings\.ssl_certificate.*PEM") as caught: + await call_native(ocr_server, asynchronous, num_retries=0) + assert str(certificate) not in str(caught.value) + assert ocr_server.requests == [] From 27808b51a0d82602457fef61e8befb6ecb847694 Mon Sep 17 00:00:00 2001 From: Yujong Lee Date: Mon, 21 Sep 2026 20:09:50 +0000 Subject: [PATCH 4/6] chore(rust): drop interop planning note Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- PYTHON_INTEROP_PLAN.md | 88 ------------------------------------------ 1 file changed, 88 deletions(-) delete mode 100644 PYTHON_INTEROP_PLAN.md diff --git a/PYTHON_INTEROP_PLAN.md b/PYTHON_INTEROP_PLAN.md deleted file mode 100644 index a7783f81472..00000000000 --- a/PYTHON_INTEROP_PLAN.md +++ /dev/null @@ -1,88 +0,0 @@ -# Python interop foundation PR plan - -Proposed implementation PR title: `fix(rust): preserve Python settings semantics at the native boundary` - -Base: `main` at `457b01e96d131f88df8cace8832f8e044ee5f167`. Planning branch: `litellm_python_interop_foundation` - -This plan follows `migration/tdd/sections/rust/python-interop/{pyo3-contract,boundary,coercion}.typ` in the sibling `litellm-typst` checkout. The first PR establishes conversion contracts and exercises them through existing HTTP, URL policy, and OCR provider-default consumers. The foundation is implemented on this branch. No PR is being created for this task - -**Baseline before implementation** - -`host-python/src/marshal.rs` already uses `pythonize` directly, separates internal conversion from public argument errors, and contains serializer panics in `Pythonized`. Keep those entrypoints. `Pythonized` currently stringifies conversion errors, unlike `from_py` and `to_py`, so its error transfer needs correction - -`core-utils/src/serde_compat.rs` already provides composable `LaxI64` and `FiniteF64` adapters. They first deserialize into `serde_json::Value`. Preserve their accepted input contracts while moving scalar decoding to Serde visitors, avoiding an intermediate JSON representation and making behavior through `pythonize` explicit - -`python-bridge/src/http.rs` currently uses strict derived Boolean and string extraction for mutable settings, converts extraction errors into `RustBridgeDeclined`, and silently drops unsupported `ssl_verify` values. OCR provider defaults have the same strict-extraction/decline pattern. `python_settings.json` checks field names only. These are the initial production consumers and regressions for the foundation - -**Ownership and placement** - -Keep interpreter attachment, structured-data conversion, panic containment, and execution adapters in `host-python`. Keep field paths, configuration error policy, named settings coercion, and product-specific tagged inputs in `python-bridge`. Keep pure parsing and Serde adapters in `core-utils`. This requires no new crate and no domain dependency from `host-python` into `core-utils` - -Add one focused `python-bridge/src/coercion.rs` module containing the field reader, semantic wrappers, and projection errors. Use `core-utils/src/serde_compat.rs` for shared token/numeric parsing and Serde decoding initially, splitting only if its size warrants it. The bridge can call those pure helpers through its existing dependency. Do not introduce a generic coercion registry, runtime manifest dispatch, or a new conversion framework - -Settings snapshots hold raw `Bound<'py, PyAny>` values only during attached projection. Successful adapters return owned values. Python snapshot dataclass annotations use `object` for arbitrary mutable globals, retaining strict annotations for accessor-owned fields such as `user_agent` and `readable`. No live settings object, iterator, or borrowed Python value enters native HTTP state - -**Serde and structured conversion** - -Keep `from_py` and `to_py` as the internal, exception-preserving conversion path. Make `Pythonized` use the same standard `PythonizeError` to `PyErr` conversion without losing its panic guard. Keep the explicitly argument-focused `ValueError` contract of `from_py_argument`; settings projection never passes through that helper - -Implement scalar visitors behind the existing `LaxI64` and `FiniteF64` adapter names. Preserve integer precision beyond the exact f64 range, signed bounds, supported decimal/underscore strings, Boolean numeric behavior, fractional rejection for integers, and nonfinite rejection for floats. Preserve composition inside `Option` and sequences, missing/null behavior at the field boundary, and ordinary numeric serialization. Use the existing [serde_with DeserializeAs contract](https://docs.rs/serde_with/3.16.1/serde_with/trait.DeserializeAs.html) - -Add the pure `parse_str_bool(&str) -> Option` parser used by bridge `StrBool`, HTTP `SslVerify::parse`, and the environment switch helper. It recognizes trimmed, case-insensitive true/false only. The environment helper retains its separate rule that only `Some(true)` contributes an enabled layer. Numeric helpers are shared only where a second actual consumer needs them - -Run common ordinary-value fixtures through JSON deserialization and direct Python-to-typed-Serde conversion. Restrict equivalence claims to their overlapping input domain. Live descriptors, identity, truthiness, iteration, and stringification are exercised through PyO3 separately. Do not add unused Serde counterparts for Python-only semantics or convert live settings through JSON text, `serde_json::Value`, `repr`, or `py_literal` - -**Named settings semantics** - -| Adapter | Contract and first consumer | -| --- | --- | -| `Truthy` | Execute Python truth testing and preserve its exception; IPv4, URL validation, and trust-env globals | -| `ExactTrue` | Compare identity with the True singleton without equality or truth testing; HTTP2, transport disable, and token refresh | -| `StrBool` | None or an actual string parsed by the shared parser; no arbitrary stringification | -| `OptionalStrictString` | None or an actual string, including the empty string; client certificate | -| `FalsyOptionalString` | Test truthiness first, treat falsey as absent, reject truthy non-strings; provider project and location | -| `TuningString` | Test truthiness first, then retain actual strings and ignore other values; TLS tuning | -| `StringCollection` | Apply the field's explicit container/member policy and return owned strings; URL allowlist | -| `SslVerifyInput` | Classify None, actual Boolean, Boolean string, CA path, live SSLContext, and invalid types separately | - -A single generic Boolean or optional-string coercer cannot implement these contracts. Accept string subclasses by their Unicode contents without calling overridden convenience methods. Use no `.ok()` or default value to discard an error from a Python protocol operation - -For URL hosts, a direct string represents one host. Otherwise test container truthiness and iterate, test member truthiness, skip falsey members, and reject truthy non-string members. Normalize with the existing URL-policy rules, deduplicate, and sort only where membership makes order irrelevant. Keep origin/scheme/port parsing in its existing domain helper rather than applying hostname normalization blindly to arbitrary URL strings - -**Errors and first production adoption** - -Represent projection failures as a tagged result separating original Python exceptions, invalid configuration, unsupported live objects, and internal accessor/schema failures. Map it once at the bridge: preserve original `PyErr`, use field-focused `ValueError` for invalid or unsupported configuration, and `RuntimeError` for genuine internal schema failures. Diagnostics contain group, field, expected forms, and actual type, never the supplied value or its representation - -Attribute, truthiness, iteration, and explicit stringification exceptions retain identity, traceback, cause, and context. In particular, do not relabel an AttributeError deliberately raised by a descriptor as a missing-field schema failure. Contract validation must distinguish schema drift from errors executing Python behavior - -Convert HTTP and URL snapshots, per-call `ssl_verify`, and OCR provider defaults through the named adapters. Preserve existing call/environment/global precedence and projection timing. Finish projection before constructing the native client or starting provider I/O. Invalid values and live SSLContext must raise configuration errors rather than disappear or authorize fallback - -Keep certificate paths through projection and validate empty, missing, or unusable client-certificate paths before I/O. The current native layer filters empty client-certificate paths, so correcting that narrow downstream behavior is part of adoption. Keep the existing CA-bundle missing-file policy explicit and separately tested; do not silently conflate it with client-certificate validation - -Classify the existing Secret Manager `readable` field as a strict accessor Boolean. Full Secret Manager client/system/settings snapshots and callback execution remain a separate PR. The existing readable-manager capability gap must be documented and must not be reported as fixed by this foundation - -**Semantic manifest** - -Extend `python_settings.json` with a stable group version and field records containing adapter ID, requiredness, precedence role, sensitivity, and specialized accepted/unsupported shapes. Include only the snapshot fields that exist in this PR. Update the Python contract test and a static Rust `SettingSpec` table to agree with the manifest - -The manifest checks declared contracts; behavioral tests prove the adapters implement them. Projection stays direct typed code. A manifest row alone is never evidence that a coercion works - -**Behavioral validation** - -Extend the existing mapped Python settings tests and Rust marshal/HTTP tests. A new coercion module may have its own focused Rust tests. Use the existing installed-extension OCR suites for public-route regressions. Do not add source-text assertions or class-attribute monkeypatching - -The acceptance matrix covers None, Boolean values, integer zero/one, strings, containers, subclasses, and arbitrary objects. Protocol fixtures raise pre-created exceptions from descriptors, `__bool__`, `__len__`, `__iter__`, and `__next__`; assert identity and exception chains. Verify ExactTrue never invokes hostile equality/truthiness. Verify falsey provider defaults preserve fallback, HTTP2 does not accept integer one, and a false/unknown environment token cannot switch off a true global - -Cover a real SSLContext, unsupported objects, Boolean strings, certificate path failures, direct-string hosts, sets, generators, duplicates, mixed members, and protocol failures. Mutate globals and source collections between calls: the next snapshot observes changes and an already projected value stays unchanged. At the installed public OCR boundary, projection failures must cause zero provider requests and zero Python fallback calls under required-native execution - -Run focused crate tests first, then the workspace Rust checks used by CI, `make test-rust-extension`, relevant Python settings tests, and `make check`. Use the fresh installed wheel and verify native provenance. Review the saved `make check` log rather than rerunning it to inspect output - -Target mutation tests at truthiness versus strict extraction, identity versus equality, swallowed versus preserved exceptions, string-as-one versus character iteration, normalization/deduplication, numeric bounds, and terminal errors versus fallback. Aim for more than 90% killed non-equivalent mutants in the changed coercion paths - -Before opening the implementation PR for maintainer review, provide a reproducible localhost proxy curl request with a real provider and positive native-execution evidence. Record the configured settings and user-visible result without credentials. Unit tests belong in validation, not the proof-of-fix section. Require the current tip's CI and coverage, Greptile confidence of at least 4/5, and acceptable Veria/Bugbot results; pending or unavailable results remain explicitly unresolved - -**Commit sequence and follow-ups** - -Start with the Serde visitor/error-transfer changes and their regression tests. Follow with field adapters and the shared token parser. Adopt them in HTTP, URL policy, and provider defaults together with the semantic manifest and installed-extension regressions. Keep these as reviewable commits in one foundational implementation PR - -Follow-up PRs can add `OptionalRedisBool`, cache-specific stringification and collection rules, and full Secret Manager bindings using the same field/error machinery. Redis accepts a different token set from StrBool, so do not share their Boolean semantics. Runtime redesign, callback lifecycle changes, free-threaded support, wholesale request serialization, and unrelated cache work are outside this PR From 9c48e137dcf63853c4ae75f2f060945ef44e2839 Mon Sep 17 00:00:00 2001 From: Yujong Lee Date: Mon, 21 Sep 2026 20:21:23 +0000 Subject: [PATCH 5/6] fix(rust): preserve HTTP host and TLS error context Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- litellm-rust/crates/http/src/config.rs | 13 ++++- litellm-rust/crates/http/src/error.rs | 18 +++++- litellm-rust/crates/http/src/lib.rs | 2 +- litellm-rust/crates/http/src/media.rs | 46 ++++++++++++++-- litellm-rust/crates/http/src/tls.rs | 49 ++++++++++------- litellm-rust/crates/python-bridge/src/http.rs | 55 +++++++++++++++---- .../python-bridge/src/routes/ocr/mod.rs | 2 +- 7 files changed, 146 insertions(+), 39 deletions(-) diff --git a/litellm-rust/crates/http/src/config.rs b/litellm-rust/crates/http/src/config.rs index bf8ecef85a8..cb0173369d5 100644 --- a/litellm-rust/crates/http/src/config.rs +++ b/litellm-rust/crates/http/src/config.rs @@ -129,6 +129,7 @@ mod tests { use rstest::rstest; use super::*; + use crate::TlsSource; fn settings(ssl_verify: Option, ssl_cert_file: Option<&str>) -> HttpSettings { HttpSettings { @@ -298,7 +299,11 @@ mod tests { }; assert!(matches!( reqwest::ClientBuilder::try_from(&config), - Err(Error::Read { path: reported, .. }) if reported == path + Err(Error::Read { + path: reported, + tls_source: TlsSource::CaBundle, + .. + }) if reported == path )); } @@ -315,7 +320,11 @@ mod tests { std::fs::remove_file(&path).unwrap(); assert!(matches!( result, - Err(Error::InvalidPem { path: reported, .. }) if reported == path + Err(Error::InvalidPem { + path: reported, + tls_source: TlsSource::CaBundle, + .. + }) if reported == path )); } } diff --git a/litellm-rust/crates/http/src/error.rs b/litellm-rust/crates/http/src/error.rs index e06f7c00cf5..eafb4d2976b 100644 --- a/litellm-rust/crates/http/src/error.rs +++ b/litellm-rust/crates/http/src/error.rs @@ -1,11 +1,25 @@ use std::path::PathBuf; +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum TlsSource { + CaBundle, + ClientIdentity, +} + #[derive(Clone, Debug, thiserror::Error, PartialEq, Eq)] pub enum Error { #[error("could not read {}: {message}", path.display())] - Read { path: PathBuf, message: String }, + Read { + path: PathBuf, + message: String, + tls_source: TlsSource, + }, #[error("{} is not a PEM file: {message}", path.display())] - InvalidPem { path: PathBuf, message: String }, + InvalidPem { + path: PathBuf, + message: String, + tls_source: TlsSource, + }, #[error("could not build the HTTP client: {0}")] Client(String), #[error("request body could not be serialized: {0}")] diff --git a/litellm-rust/crates/http/src/lib.rs b/litellm-rust/crates/http/src/lib.rs index 6f62a00175c..a1456208bb3 100644 --- a/litellm-rust/crates/http/src/lib.rs +++ b/litellm-rust/crates/http/src/lib.rs @@ -10,7 +10,7 @@ mod tls; pub mod transport; pub use config::{HttpClientConfig, Resolution, Verify}; -pub use error::Error; +pub use error::{Error, TlsSource}; pub use pool::{ClientVariant, HttpClientPool}; pub use proxy::EnvironmentProxies; pub use settings::{HttpSettings, HttpSettingsLayer, SslVerify, TcpKeepalive}; diff --git a/litellm-rust/crates/http/src/media.rs b/litellm-rust/crates/http/src/media.rs index 753f25c29de..1b9159973ef 100644 --- a/litellm-rust/crates/http/src/media.rs +++ b/litellm-rust/crates/http/src/media.rs @@ -54,16 +54,39 @@ impl Default for UrlPolicy { impl UrlPolicy { fn allows(&self, host: &str, port: u16) -> bool { let host = normalize_host(host); - let with_port = format!("{host}:{port}"); self.allowed_hosts .iter() - .map(|entry| normalize_host(entry)) - .any(|entry| entry == host || entry == with_port) + .filter_map(|entry| parse_allowed_host(entry)) + .any(|(entry_host, entry_port)| { + entry_host == host && entry_port.is_none_or(|entry_port| entry_port == port) + }) } } pub fn normalize_host(host: &str) -> String { - host.to_ascii_lowercase().trim_end_matches('.').to_owned() + let host = host.trim().trim_end_matches('.'); + let host = host + .strip_prefix('[') + .and_then(|host| host.strip_suffix(']')) + .unwrap_or(host); + host.to_ascii_lowercase() +} + +fn parse_allowed_host(entry: &str) -> Option<(String, Option)> { + let entry = entry.trim(); + if let Some(entry) = entry.strip_prefix('[') { + let (host, suffix) = entry.split_once(']')?; + let port = match suffix { + "" => None, + suffix => Some(suffix.strip_prefix(':')?.parse().ok()?), + }; + return Some((normalize_host(host), port)); + } + let (host, port) = match entry.rsplit_once(':') { + Some((host, port)) if !host.contains(':') => (host, Some(port.parse().ok()?)), + _ => (entry, None), + }; + Some((normalize_host(host), port)) } type ProxyMatch = Arc bool + Send + Sync>; @@ -670,6 +693,21 @@ mod tests { assert!(matches!(result, Err(Error::BlockedUrl))); } + #[test] + fn allowlist_matches_bracketed_ipv6_hosts_and_ports() { + let policy = UrlPolicy { + validate: true, + allowed_hosts: vec!["[2001:db8::1]".into(), "[2001:db8::1]:8443".into()], + }; + assert!(policy.allows("2001:db8::1", 443)); + assert!(policy.allows("2001:db8::1", 8443)); + let port_specific = UrlPolicy { + validate: true, + allowed_hosts: vec!["[2001:db8::1]:8443".into()], + }; + assert!(!port_specific.allows("2001:db8::1", 9443)); + } + #[tokio::test] async fn validation_off_fetches_private_hosts_and_follows_redirects() { let (url, server, _) = serve_named( diff --git a/litellm-rust/crates/http/src/tls.rs b/litellm-rust/crates/http/src/tls.rs index aaae2b659e3..e2e6d27cd54 100644 --- a/litellm-rust/crates/http/src/tls.rs +++ b/litellm-rust/crates/http/src/tls.rs @@ -9,7 +9,7 @@ use rustls::{ use crate::{ config::{HttpClientConfig, Verify}, - error::Error, + error::{Error, TlsSource}, }; #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] @@ -197,15 +197,17 @@ impl TryFrom<&HttpClientConfig> for ClientConfig { Verify::BuiltInRoots => builder.with_root_certificates(RootCertStore { roots: webpki_roots::TLS_SERVER_ROOTS.to_vec(), }), - Verify::CaBundle(path) => builder.with_root_certificates(bundle_roots(path)?), + Verify::CaBundle(path) => { + builder.with_root_certificates(bundle_roots(path, TlsSource::CaBundle)?) + } }; let mut tls = match &config.client_certificate { None => verified.with_no_client_auth(), Some(path) => { - let (chain, key) = identity(path)?; + let (chain, key) = identity(path, TlsSource::ClientIdentity)?; verified .with_client_auth_cert(chain, key) - .map_err(|error| invalid_pem(path, error))? + .map_err(|error| invalid_pem(path, TlsSource::ClientIdentity, error))? } }; tls.alpn_protocols = if config.http2 { @@ -217,47 +219,52 @@ impl TryFrom<&HttpClientConfig> for ClientConfig { } } -fn bundle_roots(path: &Path) -> Result { - let certificates = certificates(path)?; +fn bundle_roots(path: &Path, source: TlsSource) -> Result { + let certificates = certificates(path, source)?; if certificates.is_empty() { - return Err(invalid_pem(path, "no certificates found")); + return Err(invalid_pem(path, source, "no certificates found")); } let mut store = RootCertStore::empty(); for certificate in certificates { store .add(certificate) - .map_err(|error| invalid_pem(path, error))?; + .map_err(|error| invalid_pem(path, source, error))?; } Ok(store) } -fn identity(path: &Path) -> Result<(Vec>, PrivateKeyDer<'static>), Error> { - let chain = certificates(path)?; +fn identity( + path: &Path, + source: TlsSource, +) -> Result<(Vec>, PrivateKeyDer<'static>), Error> { + let chain = certificates(path, source)?; if chain.is_empty() { - return Err(invalid_pem(path, "no certificates found")); + return Err(invalid_pem(path, source, "no certificates found")); } - let key = - PrivateKeyDer::from_pem_slice(&read(path)?).map_err(|error| invalid_pem(path, error))?; + let key = PrivateKeyDer::from_pem_slice(&read(path, source)?) + .map_err(|error| invalid_pem(path, source, error))?; Ok((chain, key)) } -fn certificates(path: &Path) -> Result>, Error> { - CertificateDer::pem_slice_iter(&read(path)?) +fn certificates(path: &Path, source: TlsSource) -> Result>, Error> { + CertificateDer::pem_slice_iter(&read(path, source)?) .collect::>() - .map_err(|error| invalid_pem(path, error)) + .map_err(|error| invalid_pem(path, source, error)) } -fn read(path: &Path) -> Result, Error> { +fn read(path: &Path, source: TlsSource) -> Result, Error> { std::fs::read(path).map_err(|error| Error::Read { path: path.to_path_buf(), message: error.to_string(), + tls_source: source, }) } -fn invalid_pem(path: &Path, message: impl fmt::Display) -> Error { +fn invalid_pem(path: &Path, source: TlsSource, message: impl fmt::Display) -> Error { Error::InvalidPem { path: path.to_path_buf(), message: message.to_string(), + tls_source: source, } } @@ -405,7 +412,11 @@ mod tests { std::fs::remove_file(&path).unwrap(); assert!(matches!( result, - Err(Error::InvalidPem { path: reported, .. }) if reported == path + Err(Error::InvalidPem { + path: reported, + tls_source: TlsSource::ClientIdentity, + .. + }) if reported == path )); } } diff --git a/litellm-rust/crates/python-bridge/src/http.rs b/litellm-rust/crates/python-bridge/src/http.rs index 859d579129f..596a89a73d7 100644 --- a/litellm-rust/crates/python-bridge/src/http.rs +++ b/litellm-rust/crates/python-bridge/src/http.rs @@ -7,7 +7,7 @@ use std::{ use litellm_core_utils::settings::ProcessEnvironment; use litellm_http::{ HttpClientConfig, HttpClientPool, HttpSettings, HttpSettingsLayer, Resolution, SslVerify, - Unsupported, + TlsSource, Unsupported, media::{PublicDnsResolver, UrlPolicy}, }; use pyo3::{exceptions::PyValueError, prelude::*, types::PyDict}; @@ -41,18 +41,26 @@ pub(crate) fn call_config( Ok(resolution.config) } -pub(crate) fn client_error(error: litellm_http::Error, config: &HttpClientConfig) -> PyErr { +pub(crate) fn client_error(error: litellm_http::Error) -> PyErr { match error { - litellm_http::Error::Read { path, .. } | litellm_http::Error::InvalidPem { path, .. } - if config.client_certificate.as_ref() == Some(&path) => - { - PyValueError::new_err( - "http_settings.ssl_certificate: expected a readable PEM certificate and private key", - ) + litellm_http::Error::Read { + tls_source: TlsSource::ClientIdentity, + .. } - litellm_http::Error::Read { .. } | litellm_http::Error::InvalidPem { .. } => { - PyValueError::new_err("http_settings.ssl_verify: expected a readable PEM CA bundle") + | litellm_http::Error::InvalidPem { + tls_source: TlsSource::ClientIdentity, + .. + } => PyValueError::new_err( + "http_settings.ssl_certificate: expected a readable PEM certificate and private key", + ), + litellm_http::Error::Read { + tls_source: TlsSource::CaBundle, + .. } + | litellm_http::Error::InvalidPem { + tls_source: TlsSource::CaBundle, + .. + } => PyValueError::new_err("http_settings.ssl_verify: expected a readable PEM CA bundle"), _ => PyValueError::new_err("http_settings: native HTTP client configuration is invalid"), } } @@ -188,6 +196,33 @@ settings = types.SimpleNamespace(**{{name: defaults[name] for name in json.loads }); } + #[test] + fn client_error_uses_tls_source_when_paths_match() { + Python::initialize(); + Python::attach(|py| { + let path = PathBuf::from("/shared.pem"); + let ca_error = client_error(litellm_http::Error::InvalidPem { + path: path.clone(), + message: "invalid".into(), + tls_source: TlsSource::CaBundle, + }); + assert_eq!( + ca_error.to_string(), + "ValueError: http_settings.ssl_verify: expected a readable PEM CA bundle" + ); + let client_error = client_error(litellm_http::Error::InvalidPem { + path, + message: "invalid".into(), + tls_source: TlsSource::ClientIdentity, + }); + assert!(client_error.is_instance_of::(py)); + assert_eq!( + client_error.to_string(), + "ValueError: http_settings.ssl_certificate: expected a readable PEM certificate and private key" + ); + }); + } + #[test] fn python_settings_flow_into_the_configured_layer() { Python::initialize(); diff --git a/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs b/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs index ce6f04c321b..d0b13e5056a 100644 --- a/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs +++ b/litellm-rust/crates/python-bridge/src/routes/ocr/mod.rs @@ -51,7 +51,7 @@ fn run_ocr( ocr_settings(py)?, secrets, ) - .map_err(|error| http::client_error(error, &config))?; + .map_err(http::client_error)?; run_legacy_call( py, if asynchronous { ASYNC_SURFACE } else { SURFACE }, From c7c0afb1f0b0c737b6f561065c67563a929b5ae0 Mon Sep 17 00:00:00 2001 From: Yujong Lee Date: Mon, 21 Sep 2026 21:21:26 +0000 Subject: [PATCH 6/6] ci(rust): raise the native wheel size gate to 40 MB Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- .github/scripts/verify_linux_native_wheel.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.github/scripts/verify_linux_native_wheel.py b/.github/scripts/verify_linux_native_wheel.py index 0adbc015ad0..ea6d2401084 100644 --- a/.github/scripts/verify_linux_native_wheel.py +++ b/.github/scripts/verify_linux_native_wheel.py @@ -205,7 +205,7 @@ def main( native_module: Final = load_native_module(native_path) native_module_loads: Final = native_module is not None panic_test_hook_absent: Final = native_module is not None and not hasattr(native_module, "_panic_for_test") - native_size_limit: Final = 30_000_000 + native_size_limit: Final = 40_000_000 native_size_within_limit: Final = native_member.file_size <= native_size_limit validations: Final = ( (f"Python tag is {EXPECTED_PYTHON_TAG}", python_tag == EXPECTED_PYTHON_TAG), @@ -222,7 +222,7 @@ def main( ("Python extension entry point is present", extension_entry_point_present), ("Native module loads", native_module_loads), ("Production module omits the panic test hook", panic_test_hook_absent), - ("Native extension does not exceed 30 MB", native_size_within_limit), + (f"Native extension does not exceed {native_size_limit / 1_000_000:.0f} MB", native_size_within_limit), ("Wheel contents are valid", not unexpected_members), ) @@ -267,7 +267,8 @@ def main( ), ( not native_size_within_limit, - f"native extension exceeds 30 MB: {native_member.file_size / 1_000_000:.2f} MB", + f"native extension exceeds {native_size_limit / 1_000_000:.0f} MB: " + f"{native_member.file_size / 1_000_000:.2f} MB", ), (bool(unexpected_members), f"wheel contains unexpected build artifacts: {', '.join(unexpected_members)}"), )