Commit graph

12 commits

Author SHA1 Message Date
Yujong Lee
6675e1fd9c fix: preserve Python 3.10 harness compatibility 2026-09-04 09:13:05 -07:00
Yujong Lee
fae3d224eb Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_python_version_ci
# Conflicts:
#	basedpyright-code-budget.json
#	tests/sdk_function_trace/profiler.py
#	tests/sdk_function_trace/test_profiler.py
2026-09-04 09:01:13 -07:00
yujonglee
b75ac5cf52
feat(python): rename Rust rollout API (#39704) 2026-09-04 08:40:44 -07:00
yujonglee
eb67e5402b
test(ocr): complete Rust unit test parity (#39689)
* feat(ocr): complete Rust unit test parity

* test(ocr): keep parity changes harness-only

* test(ocr): share azure DI native fixture across response tests

* refactor(ocr): colocate gateway unit tests and extract lifecycle integration tests
2026-09-03 21:15:01 -07:00
yujonglee
ee08c36fc0
refactor(tests): restructure rust python harness around strategy definitions (#39628)
* wip

* refactor(tests): move sdk function tracing into rust python harness

* dead code

* fix: handle harness keyboard interrupts

* refactor(tests): deduplicate rust python harness helpers

* fix(harness): expose validated strategy choices

* wip

* refactor(harness): let strategies own parity reports

* docs(harness): update strategy structure

* refactor(harness): localize strategy report views

* wip

* fix(harness): satisfy mapping runner type checks

* fix(harness): clarify trace parity output

* wip

* fix(harness): clarify unit mapping report

* fix(harness): finalize trace parity contracts

* refactor(harness): structure parity contracts

* feat: derive unit test mapping from traces

* feat(harness): map rstest test families

* feat(ocr): port Azure document intelligence tests

* feat(harness): enforce complete unit mappings

* feat(ocr): add reducto core transforms

* feat(harness): classify host-only unit tests

* fix(ocr): complete Rust provider plumbing

* fix(harness): reuse OCR parity workers
2026-09-03 21:15:01 -07:00
Yujong Lee
8c180afd4b test(ocr): record provider fixtures in the migration harness 2026-09-03 10:16:40 -07:00
yujonglee
2c30fe16b0
Merge pull request #38765 from BerriAI/litellm_ocr_sdk_parity_tests
test(harness): add OCR parity with migration strategy runners
2026-09-03 10:16:35 -07:00
ishaan-berri
e058aa68c4
test: add mistral ocr transformation parity coverage (#39482)
* test: cover mistral ocr transformation parity

Co-Authored-By: Claude Code <noreply@anthropic.com>

* test: map mistral ocr parity contracts

Co-Authored-By: Claude Code <noreply@anthropic.com>

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-02 20:40:00 -07:00
ishaan-berri
bcd3e2d94d
feat(rust-python-harness): wire existing e2e SDK tests into the matrix (#39463)
Adds chat_completions and transcription as SDK function columns, backed
by the existing rust_bridge test files. Adds a fourth strategy folder,
existing_e2e_test_sdk, that points at already-existing live-API SDK
tests (tests/ocr_tests/ as a whole folder, plus chat completion and
Whisper transcription tests) instead of writing new parity tests.
Extends selector_matches_node with trailing-slash folder selectors so
a whole test folder can back one matrix cell.
2026-09-03 02:50:08 +00:00
ishaan-berri
a5639b8e2a
test: add OCR python-to-rust test parity ledger (WIP) (#39434)
* test: add OCR python-to-rust test parity ledger

* feat: add ledger loader for OCR test parity data

* feat: add drift audit for OCR test parity ledger (WIP, untested)

* fix: correct drift in OCR test-parity ledger

Two entries referenced a typo'd Python test name, four duplicated entries already tracked under TestProxySecurityGuard, five real Python tests in test_rust_bridge.py were untracked, and three real Rust custom_logger tests were missing from rust_only_tests. Found by running validate_ledger.py's audit against the live repo.

* test: add regression coverage for the OCR ledger and audit script

Covers schema validation, AST/regex test enumeration, drift detection on both the Python and Rust sides, LedgerDriftError content, and a live-repo clean-audit guard against future drift.

* test: simplify ledger test to one drift-guard assertion

Replace the ledger-internals unit tests with a single test that runs the real audit against the live repo and asserts every OCR test is accounted for (mapped, unmapped-with-reason, or rust_only), printing the exact diff on failure.

* chore: move OCR test-parity ledger to core/ocr

validate_sub_methods/ mixes strategy-catalog metadata with the ledger. Ledger data belongs under a per-function core/<function>/ path instead.

* fix: point LEDGER_PATH at the new core/ocr location
2026-09-02 19:22:52 -07:00
yujonglee
87618b7da8
doc (#39456) 2026-09-02 18:12:03 -07:00
ishaan-berri
4c51ca72d6
test: add interactive Rust Python parity harness (#39419)
* test: add interactive Rust Python parity harness

* test: simplify Rust Python parity harness structure

* test: show parity confidence by SDK section

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-09-02 15:51:08 -07:00