Commit graph

9 commits

Author SHA1 Message Date
Yujong Lee
bae4f22d3a refactor(rust): merge http settings from per-source layers
Each source (per-call kwargs, environment variables, the Python module) now
builds an HttpSettingsLayer, and HttpSettings::from_layers merges them with
explicit precedence. The aiohttp and httpx proxy-env rule is resolved once in
the merge, so HttpSettings carries a single trust_proxy_env flag
2026-09-18 19:53:14 -07:00
Yujong Lee
80dbb2a28a refactor(rust): resolve the http client config through From and TryFrom
HttpClientConfig::resolve becomes From<&HttpSettings> for Resolution and client_builder becomes TryFrom<&HttpClientConfig> for reqwest::ClientBuilder, matching the rustls conversion. The verify decision moves into From<&HttpSettings> for Verify, and the proxy environment rule moves next to its flags as HttpSettings::trusts_proxy_env. The curve and cipher results are read with transpose and a default selection, which removes the tuple destructuring
2026-09-18 19:48:34 -07:00
Yujong Lee
51010ea486 feat(rust): serve every gateway HTTP setting natively instead of declining to Python
litellm-http now builds the rustls config itself, so one route-neutral place covers roots, the client certificate, ALPN, ssl_ecdh_curve and ssl_security_level. A curve picks the single key exchange group. A cipher string restricts the TLS 1.2 suites it names, and entries rustls cannot express, such as @SECLEVEL=1, are logged once and skipped.

user_url_validation and user_url_allowed_hosts are applied by the media fetcher. Document downloads honor the environment proxy whenever provider calls do, keeping the per-hop address check, and stay on the pinned resolver when no proxy applies.

AIOHTTP_SO_KEEPALIVE, AIOHTTP_TCP_KEEPIDLE, AIOHTTP_TCP_KEEPINTVL, AIOHTTP_TCP_KEEPCNT and AIOHTTP_KEEPALIVE_TIMEOUT map onto the client. A client= argument and a live SSLContext are ignored
2026-09-18 19:39:07 -07:00
Yujong Lee
c635c35b3d fix(rust): keep native OCR on the proxy by declining only a supplied client
The proxy attaches its shared aiohttp session to every request as shared_session, so declining on it sent every proxy OCR call to Python, which never uses that session for OCR. aclient_session is a litellm global and never a call argument, so that check could not match. The proxy-shaped lifecycle test now asserts the call was served by Rust
2026-09-18 18:47:36 -07:00
Yujong Lee
157fa58947 fix(rust): leave calls with a custom URL policy on the Python route
litellm.user_url_validation and litellm.user_url_allowed_hosts are only implemented by the Python document fetcher, so an allowlisted internal document was rejected by the Rust route's network policy. The bridge now declines when either is changed from its default
2026-09-18 18:09:14 -07:00
Yujong Lee
8d2476465f fix(rust): honor environment proxies by default and name the cause in transport errors
Python's aiohttp transport reads HTTP(S)_PROXY on every request unless disable_aiohttp_trust_env is set, so the Rust clients now do the same instead of requiring aiohttp_trust_env. Transport error messages include reqwest's source chain, so a rejected certificate or refused connection is no longer reported as just 'error sending request'
2026-09-18 17:58:07 -07:00
Yujong Lee
a3aceec2f8 fix(rust): match Python proxy, ssl_verify and client expiry behavior in the http pool
Honor environment proxies whenever Python would use httpx (sync calls, HTTP/2, aiohttp disabled), apply the per-call ssl_verify argument, ignore empty or missing SSL env values the way http_handler.py does, expire pooled clients after an hour so rotated certificates reload, keep the client certificate off media downloads, and decline instead of raising when a litellm global has an unexpected type
2026-09-18 17:37:08 -07:00
Yujong Lee
b2d6cd1fcf refactor(rust): read litellm HTTP globals through one Python shim and tighten the http pool
Drop the core ocr() facade so VertexAuth and the http pool stay out of litellm-core's
public API, move the http Error enum to error.rs, and inject the media DNS resolver into
HttpClientPool instead of a per-call builder hook the cache key ignored.

The bridge now reads litellm.* HTTP settings only through litellm/rust_bridge/settings.py,
pinned by python_settings.json, while env overrides stay in Rust. This adds the Python
default User-Agent, parses string ssl_verify globals like get_ssl_verify, drops per-call
ssl_verify that Python OCR never honored, and removes the unused request_timeout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 17:23:26 -07:00
Yujong Lee
abb9618971 feat(rust): add litellm-http client pool and inject it into the OCR route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 23:28:11 +00:00