* refactor(messages): take the provider client from the injected HTTP pool
The messages route kept its own process-wide reqwest client, so it ignored
ssl_verify, CA bundles, client certs, proxies and every other setting that
litellm-http resolves. The machine now takes the HttpClientPool and the
call's HttpClientConfig, as OCR does, and the bridge passes its shared pool.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(http): hand out an owned Client and move chat, audio and OIDC onto the pool
HttpClientPool now returns litellm_http::Client, a newtype only crates/http
can build, so every provider client carries the resolved TLS, proxy and
timeout settings. Chat completions and audio transcription drop their
process-wide reqwest clients and take the pool and call config like
messages; their 600s ceiling moves to the request. OidcResolver takes its
client instead of building one, and the bridge hands it the pooled one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(secrets): build Google, Azure and CyberArk manager clients from the pool
The native secret managers built bare reqwest clients, so they ignored the
host's TLS and proxy settings. load_native_manager now takes the pool and
the host config and hands each manager a pooled client.
CyberArk's CYBERARK_SSL_VERIFY and CYBERARK_CLIENT_CERT/KEY become an
override on the host config instead of a hand-built client. To express a
certificate and key in separate files, HttpClientConfig::client_certificate
is now a ClientIdentity that is either one PEM or a split pair.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(clippy): only crates/http may build a reqwest client
Fence reqwest::Client, ClientBuilder and the TLS builder methods with
disallowed-types and disallowed-methods so new code takes a
litellm_http::Client from the pool. crates/http is exempt as the one place
clients are built, and testkit as a dev-only installer. Tests move to
litellm_http::Client::plain_for_test or a pooled client.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(secrets-cyberark): keep verifying certificates when the host disables it
Python hands CyberArk its own ssl_verify, which wins over the global
setting, so CYBERARK_SSL_VERIFY unset or true still verifies even when the
host sets ssl_verify false. The pooled client copied the host's Disabled
and would send the API key unverified; fall back to the built-in roots
instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(python-bridge): treat a missing litellm package as no host HTTP settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): group cache tests under cache/ and fold test_ocr.py into ocr/
The two failure cases in test_ocr.py duplicated the upstream-500 and timeout rows
of PUBLIC_FAILURES, so only the file-input encoding case moves to ocr/test_requests.py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): split the response cache suite into one file per backend
test_response_cache.py grew to 2400 lines. Each backend now has its own file, shared
fixtures live in cache/conftest.py and shared helpers in support/cache.py. The helpers
alias the private native test handles once, dropping the per-call reportPrivateUsage hits
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* split tokenizer test
* test(core): consolidate route integration tests under tests/ with rstest and wiremock
Moves the public-API OCR route tests out of src/ocr/route.rs and document.rs into
tests/ocr/, split per provider plus lifecycle, machine, and document tests, merging
the duplicated pairs. Messages, audio transcription, and chat completions share one
wiremock-based upstream and recording secret source in tests/support, and gain
table-driven cases for auth, routing, upstream errors, streaming, and declines.
Tests of litellm-llms items move to that crate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(messages): keep the stream relay test independent of the stream head contents
The stream head carries no headers on main, so the relay test asserts the
open-then-deliver order and the relayed body instead of header hand-off.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): move tests.rs files inline or under tests/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): move tests.rs files inline or under tests/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): inline path-included test files into their owning src files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(rust): drop stray proptest regression file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): cover lowercase, empty and non-authorization headers in bearer detection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
custom_httpx mirrored a Python module that mixes transport plumbing with
OCR orchestration. The transport half (media fetcher, transport errors,
request and header helpers) now lives in litellm-http next to the pool,
TLS, proxies and settings, and the OCR request handler moves to
base_llm/ocr/handler.rs. Drops the unused deserialize_optional_param and
stale dead_code allows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every legacy callback call from callbacks-legacy now goes through one typed
Python shim, litellm.rust_bridge.legacy_callbacks, the only Python module
the crate reaches. Before, the crate called Logging methods, litellm.utils
hooks, the logging worker, the executor and several litellm globals
directly, and its tests retyped those signatures by hand, so an outdated
fake could accept a call the real code rejects. python_contract.json lists
each shim function's parameters: a Python test pins it to the real
signatures and a Rust test pins it to the Rust enum.
The lifecycle contract changes to match the Python @client wrapper:
- the driver emits CallEvent::Started before begin, so every host sees one
start time
- RequestContext carries the route-resolved api_key, so legacy pre_call and
post_call receive it, and post_call's additional_args match the Python OCR
path
- Passthrough and its re-aliasing are gone
- async deployment hooks always run, and the "no callbacks" shortcut that
skipped the logging payload is removed, as in the Python path
The OCR api_key is a SecretValue from the wire request onward, so Debug
output upstream of the callback contract cannot leak it.
host-python's RouteHost now classifies native failures once through
classify, and host ops return HostOpError. The OCR route host keeps main's
public errors by sending both through the existing Python map_failure.
OCR transformations, BaseOcrConfig with its response and connection types,
the OCR error, and the HTTP pieces (custom_httpx: http_handler, transport,
media, llm_http_handler with OcrClient and the request/response handler)
now live in litellm-llms at their Python paths. Provider code no longer
reaches into the route: it gets the caller's hooks through a route-neutral
CallHooks trait that core implements over its host, and core dispatches to
llm_http_handler::ocr with the concrete config, the way Python calls
base_llm_http_handler.ocr(provider_config=...).
Core keeps the route: entrypoint, request types, credential fallback,
provider dispatch, the machine and hook glue. ocr/mod.rs no longer
re-exports anything, provider constants moved next to their only users,
and provider tests that drive the whole route moved to core's route test
files. Twenty-two of those were exact copies of tests already there and
were dropped; every one still runs once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
litellm-types mirrors litellm/types (chat, Anthropic Messages and Responses
websocket data) and litellm-core-utils mirrors litellm/litellm_core_utils
(provider resolution, prompt factory, core helpers, call arguments). Route
request types move up to their core route module, the transform contracts
move into base_llm, and the three duplicated provider error enums become one
Error in base_llm/chat/transformation.rs, mirroring BaseLLMException.
The empty-text placeholder goes back to the value Python's factory.py uses;
the provider extraction had changed it to a single space.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Extracted from #41733 without the router loop, the cache machine layer, streaming, or the
error, timeout and route-pruning work that moved to #41745
litellm-callbacks holds the contract a native call and its host share: Machine, HostOp,
CallEvent, the in-process run loop, and Passthrough, which is built only by comparing the
caller's inputs with the body the route sends, so a route can never mark a key it rewrote.
litellm-host-python (formerly python-interop) owns the CPython driver and the Execution
handle, and litellm-callbacks-legacy is the @client wrapper as the native call sees it:
function_setup, the deployment hooks, pre_call and post_call, the success and failure fan-out
and the deferred proxy release. OCR is the one route on it, and the old core and bridge
lifecycles are gone
The passthrough rule is the structural fix for the bug #41719 patched in core and #41716
reworks: an inlined remote document no longer counts as the caller's value, so the legacy
adapter never hands the caller's URL back into the body. core/tests/ocr/passthrough.rs pins
it for every route and document source, including that unchanged values stay passthrough,
and callbacks-legacy/tests/payload.rs pins the adapter side with a real pre_call callback
Python OCR integration tests that only exercised core behavior now live as Rust tests, so
tests/test_litellm_rust keeps the cases that need the full Python stack
