* refactor(messages): take the provider client from the injected HTTP pool
The messages route kept its own process-wide reqwest client, so it ignored
ssl_verify, CA bundles, client certs, proxies and every other setting that
litellm-http resolves. The machine now takes the HttpClientPool and the
call's HttpClientConfig, as OCR does, and the bridge passes its shared pool.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(http): hand out an owned Client and move chat, audio and OIDC onto the pool
HttpClientPool now returns litellm_http::Client, a newtype only crates/http
can build, so every provider client carries the resolved TLS, proxy and
timeout settings. Chat completions and audio transcription drop their
process-wide reqwest clients and take the pool and call config like
messages; their 600s ceiling moves to the request. OidcResolver takes its
client instead of building one, and the bridge hands it the pooled one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(secrets): build Google, Azure and CyberArk manager clients from the pool
The native secret managers built bare reqwest clients, so they ignored the
host's TLS and proxy settings. load_native_manager now takes the pool and
the host config and hands each manager a pooled client.
CyberArk's CYBERARK_SSL_VERIFY and CYBERARK_CLIENT_CERT/KEY become an
override on the host config instead of a hand-built client. To express a
certificate and key in separate files, HttpClientConfig::client_certificate
is now a ClientIdentity that is either one PEM or a split pair.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(clippy): only crates/http may build a reqwest client
Fence reqwest::Client, ClientBuilder and the TLS builder methods with
disallowed-types and disallowed-methods so new code takes a
litellm_http::Client from the pool. crates/http is exempt as the one place
clients are built, and testkit as a dev-only installer. Tests move to
litellm_http::Client::plain_for_test or a pooled client.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(secrets-cyberark): keep verifying certificates when the host disables it
Python hands CyberArk its own ssl_verify, which wins over the global
setting, so CYBERARK_SSL_VERIFY unset or true still verifies even when the
host sets ssl_verify false. The pooled client copied the host's Disabled
and would send the API key unverified; fall back to the built-in roots
instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(python-bridge): treat a missing litellm package as no host HTTP settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): move tests.rs files inline or under tests/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): move tests.rs files inline or under tests/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): inline path-included test files into their owning src files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(rust): drop stray proptest regression file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): cover lowercase, empty and non-authorization headers in bearer detection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add an aws_textract OCR provider on the Rust route, with no Python path. The
detect-document-text model returns plain lines and analyze-document renders
layout and tables as markdown. Both use Textract's synchronous API, so a
multi-page PDF or TIFF is rejected with an error that names the single-page
limit. A call with no region fails instead of falling back to Bedrock's default
SigV4 covers the request body, and host hooks can rewrite that body before it
is sent. litellm-http now has OutboundRequest, which serializes the body once,
shows those bytes to a RequestSigner and is the only thing a route can send.
Chat, audio transcription and OCR build it after their hooks ran, so a callback
that redacts the body still produces a valid Bedrock or Textract signature
ChatCompletionsAuth and AudioTranscriptionAuth are replaced by
litellm_auth::RequestAuth, and one helper in core turns it into a signed or
unsigned request. Audio transcription now signs only the AWS header set and
rejects a forwarded header that SigV4 computes, the same as chat
The OCR catalog routes aws_textract as Rust required, and the dispatch context
reads the provider from the model prefix so a provider scoped rule can match
EnvironmentProxies holds raw proxy URLs, which can carry user:password, and
it sits inside HttpSettings and HttpClientConfig, so any {:?} of those would
print the password. Derive veil's Redact like the auth crate does. NO_PROXY
stays readable because it holds no credentials.
The media fetcher also rebuilt the hyper-util matcher for every URL and
redirect hop. Build it once when the fetcher is created
Python resolves proxies through urllib.request.getproxies_environment: the
lowercase variable wins, an empty value is unset, an empty lowercase value
clears the uppercase one, and under CGI only the uppercase HTTP_PROXY is
forgotten because a client can set it with a Proxy header. The Rust route
took the uppercase variable even when empty and dropped every proxy under
CGI, so provider calls could skip a required egress proxy
custom_httpx mirrored a Python module that mixes transport plumbing with
OCR orchestration. The transport half (media fetcher, transport errors,
request and header helpers) now lives in litellm-http next to the pool,
TLS, proxies and settings, and the OCR request handler moves to
base_llm/ocr/handler.rs. Drops the unused deserialize_optional_param and
stale dead_code allows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
reqwest and hyper each read HTTP(S)_PROXY, ALL_PROXY and NO_PROXY from the
process on their own, so tests could not inject them and the pooled client
key ignored proxy changes. EnvironmentProxies now reads them through
Lookup with the same precedence hyper used, the resolved config carries
them (empty when the transport does not trust the env), and both the
provider clients and the media fetcher build from that one value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Settings sources beyond HTTP (media fetch, Azure Document Intelligence,
Vertex, timeouts) need the same env lookup and precedence merge, so move
them out of litellm-http into core_utils::settings. Lookup readers name the
Python idiom they mirror: get keeps a present empty value like
os.getenv(X, fallback), truthy drops it like an `or` chain, enabled only
switches on for "true". SSL_CERT_FILE now reads through truthy, matching
Python's `if ssl_cert_file and ...` check.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each source (per-call kwargs, environment variables, the Python module) now
builds an HttpSettingsLayer, and HttpSettings::from_layers merges them with
explicit precedence. The aiohttp and httpx proxy-env rule is resolved once in
the merge, so HttpSettings carries a single trust_proxy_env flag
HttpClientConfig::resolve becomes From<&HttpSettings> for Resolution and client_builder becomes TryFrom<&HttpClientConfig> for reqwest::ClientBuilder, matching the rustls conversion. The verify decision moves into From<&HttpSettings> for Verify, and the proxy environment rule moves next to its flags as HttpSettings::trusts_proxy_env. The curve and cipher results are read with transpose and a default selection, which removes the tuple destructuring
KeyExchangeGroup and Tls12CipherSuite parse with FromStr and fail with Unsupported, so a setting rustls cannot honor is a typed error instead of a missing value. The cipher string conversions cannot fail and use From. The rustls ClientConfig is built with TryFrom<&HttpClientConfig>, and the built-in root store is constructed in one expression
litellm-http now builds the rustls config itself, so one route-neutral place covers roots, the client certificate, ALPN, ssl_ecdh_curve and ssl_security_level. A curve picks the single key exchange group. A cipher string restricts the TLS 1.2 suites it names, and entries rustls cannot express, such as @SECLEVEL=1, are logged once and skipped.
user_url_validation and user_url_allowed_hosts are applied by the media fetcher. Document downloads honor the environment proxy whenever provider calls do, keeping the per-hop address check, and stay on the pinned resolver when no proxy applies.
AIOHTTP_SO_KEEPALIVE, AIOHTTP_TCP_KEEPIDLE, AIOHTTP_TCP_KEEPINTVL, AIOHTTP_TCP_KEEPCNT and AIOHTTP_KEEPALIVE_TIMEOUT map onto the client. A client= argument and a live SSLContext are ignored
Python OCR passes the call timeout per request, so its connect timeout is the call timeout and never the 5s handler default. 10s is what every Rust route uses on main. The media client never uses a proxy, so trust_proxy_env no longer splits its pool key
Python's aiohttp transport reads HTTP(S)_PROXY on every request unless disable_aiohttp_trust_env is set, so the Rust clients now do the same instead of requiring aiohttp_trust_env. Transport error messages include reqwest's source chain, so a rejected certificate or refused connection is no longer reported as just 'error sending request'
Honor environment proxies whenever Python would use httpx (sync calls, HTTP/2, aiohttp disabled), apply the per-call ssl_verify argument, ignore empty or missing SSL env values the way http_handler.py does, expire pooled clients after an hour so rotated certificates reload, keep the client certificate off media downloads, and decline instead of raising when a litellm global has an unexpected type
Drop the core ocr() facade so VertexAuth and the http pool stay out of litellm-core's
public API, move the http Error enum to error.rs, and inject the media DNS resolver into
HttpClientPool instead of a per-call builder hook the cache key ignored.
The bridge now reads litellm.* HTTP settings only through litellm/rust_bridge/settings.py,
pinned by python_settings.json, while env overrides stay in Rust. This adds the Python
default User-Agent, parses string ssl_verify globals like get_ssl_verify, drops per-call
ssl_verify that Python OCR never honored, and removes the unused request_timeout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
