mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-03 07:23:33 +00:00
9 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
yujonglee
|
a1c238ff4a
|
refactor(rust): extract SecretValue and use it for Redis credentials (#49186)
* refactor(rust): move SecretValue into litellm-secret-value * docs(rust): add rust-redaction skill * refactor(redis-client): carry credentials as SecretValue --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yujonglee
|
7b60fd061a
|
refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042)
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http The workspace moves to reqwest 0.13, so litellm-http no longer carries a second reqwest just for rmcp. The mcp feature, its pool and its proxy conversion go away, and the gateway hands rmcp a pooled NoRedirect client that already carries the TLS, proxy and timeout settings The Client test helpers collapse into for_test, a no-redirect client without TLS or proxy settings, and wrap_for_test for the few tests that need a hand-built reqwest client Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(http): honor transport settings for Rust WebSockets The Responses WebSocket connected through tokio-tungstenite with its own native-roots TLS, so it ignored the configured CA bundle, verify flag, client certificate, proxies and timeouts. The handshake now goes through a pooled reqwest client (ClientVariant::WebSocket, HTTP/1.1, no redirects) and upgrades the response, so it shares the settings every other outbound request uses The upgrade response is checked for the Sec-WebSocket-Accept key, the Upgrade and Connection headers and a subprotocol that was offered Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(http): build outbound requests as reqwest::Request OutboundRequest was a hand-rolled copy of reqwest::Request. Routes now build a reqwest::Request directly and RequestSigner reads the body bytes back out of it, so the signer still covers exactly what is sent. SigV4 signing moves onto HeaderMap and takes the method, which drops the BTreeMap plumbing and the header casing table Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(http): make litellm_http::Error the one opaque error transport::Error and media::Error fold into litellm_http::Error, an Arc<Repr> newtype whose private variants keep the typed reqwest, io, rustls and tungstenite causes and own every message. Callers can no longer build an error with a message of their own. They ask through dispatched, is_request, status, upstream_response and tls_source One From<reqwest::Error> now classifies connect, network and timeout and strips the URL. The WebSocket handshake reports each failed check as its own variant and owns its connect timeout. OCR drops the media variants it mirrored, and inference::Error maps a header the signer computes to Unsupported for every route, not only chat The error rules move out of AGENTS.md into the rust-error skill, which also records the decisions behind this design Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(rust): split rust-error skill into thin SKILL.md with references Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): render causes once and ask predicates at the boundaries The four WebSocket handshake variants collapse into Handshake(HandshakeFault), and media download failures no longer answer dispatched() since they happen before the provider call ErrorDetail in llms and auth-types, the inference error and the OCR error now store their cause in a field named cause with no #[source], so to_string() carries it exactly once and source() is None. Tests that walked the chain now destructure the variant and downcast the cause The gateway asks is_unsupported() and is_missing_credential() on the inference and OCR errors instead of matching through nested enums, and litellm_auth::Error answers is_missing_credential() for both The rust-error skill gains axum and pyo3 references, boundaries.md describes the real predicate order and the known divergences, the testing example asserts dispatched() correctly, and shape, classification, conversions, messages-and-causes and recipes agree with the code again * docs(rust): focus the rust-error gateway page on the error flow and add a rust-gateway skill The former axum page toured gateway-inference: its status match, the envelope table, an extractor, the unsupported route and the Sse closure. Those details belong to the gateway crates and go stale with them. The page is now gateway.md and follows an error from the provider failure through each From into IntoResponse, where it is classified once and rendered by surface and framing. The gateway-structure material moves to a new rust-gateway skill: envelopes per API surface, extractors and path registration, SSE relay, and the router test harness. * test(inference): pin provider URL redaction on every route's transport error Chat, Messages, Responses and transcription each get a route-level test that fails if a reqwest error reaches the caller with its URL. The Messages stream decoder also takes body errors through litellm_http::Error like every other reqwest error Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(rust-error): adopt std source chains and add litellm-error Report Messages describe their own layer and the cause is the source, boundaries render the chain once with litellm_error::Report. Corrects skill claims the audit found stale against the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): keep causes as sources across http, auth, secrets, cache and utility crates Messages describe their own layer and every cause is a typed source, predicates match exhaustively, and boundaries render with litellm_error::Report. Media downloads no longer count as dispatched, Error::upstream bounds the provider body itself, and a forwarded header SigV4 cannot sign is the caller's fault. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): keep causes as sources in llms and the gateway crates ErrorDetail loses its string variant, OCR provider variants become data, host faults stop counting as the caller's, a tunnelled transport error classifies the same on every stream path, and each gateway crate has one Error with one status() and extractors that reject in its envelope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): nest llms errors in inference and ask predicates in route tests litellm_inference::Error nests litellm_llms::Error instead of mirroring its variants, host driver faults get their own variant, every predicate matches exhaustively, and route tests ask predicates instead of matching transport variants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): map bridge and gateway-inference errors once and render them with Report RustUpstreamError has one constructor that always sets headers, argument checks raise typed errors mapped once, OCR asks upstream_response(), the Responses stream error event comes from the error type, gateway-inference loses its string variants, and auth faults classify through one litellm_auth predicate on every route. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(rust-error): carry OCR provider headers as a HeaderMap and keep outages off the caller The OCR provider error and get_error_class take reqwest's HeaderMap, the bridge turns it into Python's (name, value) pairs once. Credential acquisition and provider SDK failures are no longer the caller's fault, the UI hides auth and backend causes, and chat keeps a provider answer as is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(rust-error): correct skill claims against the aligned code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): fix the remaining sweep findings in gateway, llms, auth and cache crates Auth rejections hide their backend cause on every inference renderer, path parameters reject in the surface envelope, Textract's unsupported document and Azure DI's unparseable poll URL get typed variants, the credential chain exposes its last failure as the source, gateway-mcp asks predicates over rmcp errors, and the S3 and Azure blob transports classify through litellm_http::Error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(rust-error): fix the remaining sweep findings on the Python side host-python keeps a native error's chain and returns blocking-pool cancellation instead of panicking, Vault transport failures convert through litellm_http::Error so no URL reaches Python, the bridge asks predicates instead of matching three levels into Vault errors, and the last string-built PyErrs become typed errors mapped once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): refresh compatible dependencies flagged by OSV * refactor(rust): remove redundant client construction comment * test(rust): cover configured MCP proxy and CA bundle * fix(http): preserve native WebSocket trust and optional protocols * fix(http): allow plain WebSockets without native roots * fix(http): normalize provider URLs before building outbound requests * fix(rust): preserve OCR download status and reduce error size * fix(rust): preserve upstream bodies and bound diagnostics only * test(rust): preserve bridge serialization boundary in body regression --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yujonglee
|
6c0585b8f1
|
build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http (#48039)
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http The workspace moves to reqwest 0.13, so litellm-http no longer carries a second reqwest just for rmcp. The mcp feature, its pool and its proxy conversion go away, and the gateway hands rmcp a pooled NoRedirect client that already carries the TLS, proxy and timeout settings The Client test helpers collapse into for_test, a no-redirect client without TLS or proxy settings, and wrap_for_test for the few tests that need a hand-built reqwest client Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): refresh compatible dependencies flagged by OSV * refactor(rust): remove redundant client construction comment * test(rust): cover configured MCP proxy and CA bundle --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yujonglee
|
ec605826d4
|
feat: improve trace ingestion and trace details (#43975)
* refactor: separate OTLP HTTP decoding from trace codec * feat: complete trace ingestion and read paths * fix: encode OTLP protobuf errors in Rust * fix: raise OTLP body limit to 16 MiB * test: cover OTLP auth body parsing boundary * refactor: parse OTLP media type into enum * fix: enforce OTLP body size at HTTP boundary * perf: preserve shared OTLP metadata across ingestion * bench: compare owned and shared trace resource fanout * refactor: extract shared storage and Python conversion caches * refactor: keep shared storage owned by traces * test: keep trace loopback coverage in Rust * test(proxy): adapt trace coverage to injected access context * fix(tracing): satisfy stacked branch lint checks * refactor(tracing): use immutable ingestion payloads * fix(tracing): declare native error encoder export * test(proxy): resolve trace access through dependency * fix(tracing): align merged normalizer types and bridge tests * fix(tracing): address ingestion and diagnostic review findings * fix(proxy): preserve body parsing for partial request scopes * test(proxy): use valid HTTP scopes in request fixtures * test(proxy): complete auth request flow scopes |
||
|
devin-ai-integration[bot]
|
1a58162630
|
refactor(http): hand out an owned Client and route all providers through the pool (#43245)
* refactor(messages): take the provider client from the injected HTTP pool The messages route kept its own process-wide reqwest client, so it ignored ssl_verify, CA bundles, client certs, proxies and every other setting that litellm-http resolves. The machine now takes the HttpClientPool and the call's HttpClientConfig, as OCR does, and the bridge passes its shared pool. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(http): hand out an owned Client and move chat, audio and OIDC onto the pool HttpClientPool now returns litellm_http::Client, a newtype only crates/http can build, so every provider client carries the resolved TLS, proxy and timeout settings. Chat completions and audio transcription drop their process-wide reqwest clients and take the pool and call config like messages; their 600s ceiling moves to the request. OidcResolver takes its client instead of building one, and the bridge hands it the pooled one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(secrets): build Google, Azure and CyberArk manager clients from the pool The native secret managers built bare reqwest clients, so they ignored the host's TLS and proxy settings. load_native_manager now takes the pool and the host config and hands each manager a pooled client. CyberArk's CYBERARK_SSL_VERIFY and CYBERARK_CLIENT_CERT/KEY become an override on the host config instead of a hand-built client. To express a certificate and key in separate files, HttpClientConfig::client_certificate is now a ClientIdentity that is either one PEM or a split pair. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(clippy): only crates/http may build a reqwest client Fence reqwest::Client, ClientBuilder and the TLS builder methods with disallowed-types and disallowed-methods so new code takes a litellm_http::Client from the pool. crates/http is exempt as the one place clients are built, and testkit as a dev-only installer. Tests move to litellm_http::Client::plain_for_test or a pooled client. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(secrets-cyberark): keep verifying certificates when the host disables it Python hands CyberArk its own ssl_verify, which wins over the global setting, so CYBERARK_SSL_VERIFY unset or true still verifies even when the host sets ssl_verify false. The pooled client copied the host's Disabled and would send the API key unverified; fall back to the built-in roots instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python-bridge): treat a missing litellm package as no host HTTP settings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
19556952d9
|
feat(secrets): route secret resolution through native Rust backends (#42619)
* fix(secrets): verify provider API request and payload contracts * wip * fix(secrets): unify backend reads and route secret resolution * feat(secrets): bind built-in managers to retained Rust backends * refactor(secrets): centralize catalog dispatch and native binding * test(secrets): split provider integration tests * refactor(secrets): enforce cache and rotation contracts * test(secrets): stub parent packages in failing resolver fixture Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(secrets): pass manager settings through the interop boundary Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): align cloud KMS auth and harden provider reads * ci(rust): raise native wheel size gate to 40 MB for secrets backends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): treat unset google kms flag as disabled like the old loader Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve certificate credentials and disabled KMS flags * test(secrets): cover certificate validation and bounded auth retries * test(secrets): cover Python dispatch without the native extension * test(proxy): skip legacy secret manager cases when the optional SDK is missing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): port Python parity tests and preserve provider behavior * fix(secrets): store the captured native config without setattr to satisfy the strict lint budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve missing Azure manager values Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(secrets): pin typed values and recovery failure precedence * refactor(secrets): organize provider internals and behavioral test suites * refactor(secrets): simplify recovery and isolate Python compatibility * fix(secrets): distinguish Azure callback absence from HTTP not found * fix(secrets): preserve Python AWS read results at the bridge * fix(secrets): route public reads through the native catalog bridge * fix(secrets): keep JSON selection outside the bridge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve provider JSON reads at the bridge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve Python primary JSON semantics Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve CyberArk mutation behavior through the native bridge * docs(secrets): record public API replacement gaps * refactor(secrets): share Vault write payload preparation * feat(secrets): route Vault mutations through the native bridge * fix(secrets): preserve typed Vault rotation failures * refactor(secrets): move Python dispatch into bridge * refactor(secrets): move CyberArk Python policy into bridge * refactor(secrets): move Vault Python policy into bridge * test(secrets): assert Vault rotation request paths * fix(secrets): keep bridge JSON interop centralized Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
d6ffc554ec
|
feat(rust): align secret manager operation contexts (#42480)
* refactor(rust): simplify Python secret callbacks * feat(rust): align secret manager operation contexts * test(rust): port secret name validation cases * fix(rust): restore secret manager CI checks * fix(rust): validate Vault contexts and preserve rotation timeouts --------- Co-authored-by: Yujong Lee <yujong@berri.ai> |
||
|
Yujong Lee
|
bfb4a8a2b3 |
refactor(rust): build Azure Key Vault auth inputs directly and keep credential tests offline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yujong Lee
|
ae69a8c79a |
feat(rust): add Azure Key Vault secret manager backend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |