Commit graph

81 commits

Author SHA1 Message Date
Yujong Lee
aaace234bb fix(rust): type driver initialization errors 2026-09-07 23:42:53 -07:00
Yujong Lee
87d1997155 test(rust): expand callback lifecycle coverage 2026-09-07 23:38:20 -07:00
Yujong Lee
b02c53cebc fix(rust): satisfy CI quality gates 2026-09-07 23:31:04 -07:00
Yujong Lee
6f8f07433d refactor(ocr): inject upstream transport 2026-09-07 23:07:25 -07:00
Yujong Lee
aa8d8bf84b wip 2026-09-07 23:03:27 -07:00
Yujong Lee
b1310a1bf8 refactor(rust): remove generic core cache 2026-09-07 22:38:45 -07:00
Yujong Lee
5b7ba86c30 wip 2026-09-07 22:02:46 -07:00
Yujong Lee
0a53a00edd refactor(ocr): fold execution into shared lifecycle 2026-09-07 21:38:57 -07:00
Yujong Lee
969fa34968 wip 2026-09-07 20:58:41 -07:00
Yujong Lee
d02ec68372 wip 2026-09-07 20:34:07 -07:00
Yujong Lee
e1f009df86 remove bunch of wrong code 2026-09-07 19:50:22 -07:00
Yujong Lee
5afda69ca5 refactor test 2026-09-07 19:49:54 -07:00
Yujong Lee
993ddf7214 done 2026-09-07 19:49:54 -07:00
Yujong Lee
b1f74b3603 wip 2026-09-07 19:49:54 -07:00
Yujong Lee
ff958cc30b lint 2026-09-07 19:49:54 -07:00
Yujong Lee
5cb55a5efd wip 2026-09-07 19:49:54 -07:00
Yujong Lee
5da66a7290 refactor(rust): harden retained callback adapter and dedupe test fixtures 2026-09-07 19:49:37 -07:00
Yujong Lee
bb3ac0fb74 fix(rust): install proxy extra for retained callback integration tests
The integration fixtures import litellm.proxy.* guardrails (crowdstrike,
purview, proxy.utils), which import fastapi at module load. The install
target only synced base SDK deps, so those 12 tests failed with a
ModuleNotFoundError on fastapi in CI.
2026-09-07 19:49:37 -07:00
Yujong Lee
036007a4e9 cleanup 2026-09-07 19:49:37 -07:00
Yujong Lee
415f6894a8 fix(rust): avoid retained adapter initialization deadlocks 2026-09-07 19:49:37 -07:00
Yujong Lee
391bb4cc74 wip 2026-09-07 19:49:37 -07:00
Yujong Lee
2ca2d106f7 wip 2026-09-07 19:49:37 -07:00
Yujong Lee
c6cd873266 wip 2026-09-07 19:49:19 -07:00
Yujong Lee
f1553ae9c3 wip 2026-09-07 19:49:05 -07:00
Yujong Lee
f854c38f36 await test 2026-09-07 19:49:05 -07:00
Yujong Lee
fc1877718d refactor(rust): complete retained callback lifecycle foundation 2026-09-07 19:49:05 -07:00
Yujong Lee
d4fb0921e1 refactor(rust): add retained callback invocation 2026-09-07 19:49:05 -07:00
Mateo Wang
642a0f68ae
Merge pull request #39530 from BerriAI/litellm_fix_gateway_rustls_provider
fix(ai-gateway): dial upstream WebSockets over an explicit rustls provider
2026-09-07 10:55:15 -07:00
yujonglee
8da735410c
test(ocr): trace callback lifecycle parity (#40063) 2026-09-07 10:44:22 -07:00
mateo-berri
836c20a4b6 test(ai-gateway): drop provider fingerprint and cache identity assertions 2026-09-04 22:31:06 -07:00
mateo-berri
ff856080c5 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_gateway_rustls_provider 2026-09-04 20:08:06 -07:00
mateo-berri
6463994f78 fix(ai-gateway): dial upstream WebSockets over an explicit rustls provider
Build one ClientConfig that names ring and loads the native roots once, and hand it to every tokio-tungstenite dial as its connector instead of installing a process-wide default from the dial path. Each of the three dial sites gets a wss:// test that reproduces the panic if its connector is dropped.
2026-09-04 17:29:10 -07:00
yujonglee
b75ac5cf52
feat(python): rename Rust rollout API (#39704) 2026-09-04 08:40:44 -07:00
yujonglee
7276caecd4
refactor(rust): extract config crate (#39706)
* refactor(rust): extract config crate

* refactor(config): split crate modules

* refactor(gateway): remove gil health counter
2026-09-04 08:17:07 -07:00
yujonglee
eb67e5402b
test(ocr): complete Rust unit test parity (#39689)
* feat(ocr): complete Rust unit test parity

* test(ocr): keep parity changes harness-only

* test(ocr): share azure DI native fixture across response tests

* refactor(ocr): colocate gateway unit tests and extract lifecycle integration tests
2026-09-03 21:15:01 -07:00
yujonglee
ee08c36fc0
refactor(tests): restructure rust python harness around strategy definitions (#39628)
* wip

* refactor(tests): move sdk function tracing into rust python harness

* dead code

* fix: handle harness keyboard interrupts

* refactor(tests): deduplicate rust python harness helpers

* fix(harness): expose validated strategy choices

* wip

* refactor(harness): let strategies own parity reports

* docs(harness): update strategy structure

* refactor(harness): localize strategy report views

* wip

* fix(harness): satisfy mapping runner type checks

* fix(harness): clarify trace parity output

* wip

* fix(harness): clarify unit mapping report

* fix(harness): finalize trace parity contracts

* refactor(harness): structure parity contracts

* feat: derive unit test mapping from traces

* feat(harness): map rstest test families

* feat(ocr): port Azure document intelligence tests

* feat(harness): enforce complete unit mappings

* feat(ocr): add reducto core transforms

* feat(harness): classify host-only unit tests

* fix(ocr): complete Rust provider plumbing

* fix(harness): reuse OCR parity workers
2026-09-03 21:15:01 -07:00
Mateo Wang
c99149ac0f
Merge pull request #39493 from BerriAI/litellm_ci_rust_server_feature
ci(rust): build and test the ai-gateway server feature
2026-09-03 14:35:53 -07:00
yujonglee
2c30fe16b0
Merge pull request #38765 from BerriAI/litellm_ocr_sdk_parity_tests
test(harness): add OCR parity with migration strategy runners
2026-09-03 10:16:35 -07:00
mateo-berri
2c08e7abf8 test(ai-gateway): pin the crypto provider to ring and prove the dial installs it
The two tests that shipped with the fix both called ensure_crypto_provider
themselves, so deleting the call from connect_upstream left the whole suite
green, and swapping ring for aws-lc-rs did too.

Adds an integration test, which gets its own process, that dials wss:// at a
local plain-TCP listener through the public Responses WebSocket entrypoint and
asserts an Err plus an installed provider. Without the install in the dial it
panics with the original CryptoProvider message. A unit test now compares the
installed provider's cipher suites and key-exchange groups against ring's, so
the choice of backend is pinned rather than assumed.

Also names tls12 in the workspace rustls features: it already arrives through
reqwest and tokio-rustls, so the graph is unchanged, but a direct dependency
should say it needs TLS 1.2 rather than inherit it.
2026-09-03 02:27:00 -07:00
mateo-berri
c79d1d12ae fix(ai-gateway): install a rustls crypto provider before dialing upstream WebSockets
The gateway's dependency graph turns on two rustls crypto backends at once:
reqwest's rustls-tls pulls in ring, and litellm-core's bedrock-auth pulls in
aws-lc-rs through aws-config. rustls 0.23 refuses to guess between them, so
ClientConfig::builder panics, and that is exactly how tokio-tungstenite builds
its TLS config. Every outbound WebSocket dial killed its tokio worker and the
client saw the socket vanish with no close frame.

reqwest and the AWS SDK both pick a provider explicitly, so only the tungstenite
path was affected. Route all three dial sites through one helper that installs
ring once per process before connecting.
2026-09-03 02:15:57 -07:00
mateo-berri
5120590890 docs(litellm-rust): fix the gateway run commands and point ADDING_A_PROVIDER at the one checks runbook
Both `cargo run` invocations in the ai-gateway README fail with "requires the
features: `server`", the same root cause as the missing CI coverage.
2026-09-03 00:07:19 -07:00
mateo-berri
70dc0a69a4 test(ai-gateway): cover the blank session id and model fallbacks
The emptiness guards in on_session had no test, so the let-chain rewrite
could have dropped them unnoticed.
2026-09-02 22:16:41 -07:00
mateo-berri
f986fc52f0 ci(rust): lint every gateway feature and keep one checks runbook
Clippy never links, so python-config's pyo3/auto-initialize needs no
libpython and the gateway clippy step can cover every feature at once.
The test step stays on --features server because cargo test does link
and this job installs no Python.

The check list existed in three places that had already drifted apart;
CLAUDE.md is now the only copy and the other two point at it.
2026-09-02 22:16:40 -07:00
mateo-berri
6fdd3128d9 fix(ai-gateway): build the gateway binary in the release image
The Dockerfile asked cargo for --features python-config, which cannot
select the litellm-ai-gateway bin target: that target carries
required-features = ["server"], so cargo silently built nothing and the
later COPY of /build/litellm-rust/target/release/litellm-ai-gateway had
no file to copy. Turn the server feature on and name the bin explicitly
so a future required-features drift fails at the cargo step instead of
silently producing an empty release dir.
2026-09-02 22:16:39 -07:00
mateo-berri
cf958c0e6f ci(rust): build and test the ai-gateway server feature
litellm-ai-gateway's server feature is off by default and nothing in the workspace turns it on, so the workspace clippy and test steps never compiled src/auth, src/routes, src/state, src/realtime or the gateway binary. 43 tests ran instead of 57.

Adds the two steps CLAUDE.md already documents as the local gate, and fixes the three collapsible_if violations that had accumulated behind the flag.
2026-09-02 21:56:23 -07:00
ishaan-berri
e058aa68c4
test: add mistral ocr transformation parity coverage (#39482)
* test: cover mistral ocr transformation parity

Co-Authored-By: Claude Code <noreply@anthropic.com>

* test: map mistral ocr parity contracts

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-02 20:40:00 -07:00
yujonglee
34b45f3d79
refactor(python-bridge): declare sync and async routes once (#39333)
* refactor(rust): move audio transcription into core

* refactor(python-bridge): split non-streaming bridge modules

* fix(python-bridge): harden sync and async route boundaries

* refactor(python-bridge): declare sync and async routes once
2026-09-02 16:26:36 -07:00
yujonglee
62e318de8e
fix(python-bridge): harden sync and async route boundaries (#39332) 2026-09-02 16:26:35 -07:00
yujonglee
198906495f
refactor(python-bridge): split routes and add shared function tracing (#39031)
* refactor(python-bridge): split non-streaming bridge modules

* refactor(python-bridge): bring shared function tracing into route layer

* feat(dev): list Python route functions and call sites

* feat(dev): list Rust route functions and call sites

* docs(dev): record OCR parity gaps across Python and Rust

* feat(dev): list executed SDK calls with runtime tracing

* feat(dev): report Python vs Rust SDK pipeline steps in one CLI

* feat(dev): side-by-side pipeline step report in compare CLI

* fix(dev): drop invalid Final annotations in compare cell loop

* feat(dev): blue python-only and yellow rust-only steps in compare CLI

* feat(dev): vertical layout with section spacing in compare CLI

* fix(dev): validate SDK trace stages across sync and async routes

* refactor(rust): align SDK route call structure with Python

* refactor(python-bridge): share sync and async route call wrappers

* refactor(dev): split compare CLI into fixtures, runtime, and report modules

* fix(ci): run SDK trace tests and satisfy test lint
2026-09-02 16:26:35 -07:00
yujonglee
0c0b432cf1
refactor(rust): move audio transcription into core (#39126) 2026-09-02 15:20:54 -07:00