Commit graph

7 commits

Author SHA1 Message Date
mateo-berri
f80cb5cb46 fix(router): ignore planted request_retry_count seeds and cover the rust OCR cap path
The router clamps a negative request_retry_count found in request metadata before counting a failure, and the proxy strips a client-supplied request_retry_count with the other router-reserved metadata fields. The rust OCR lifecycle test that trips the per-request cap now plants request_retry_count instead of attempted_retries, which the cap no longer reads since the previous commit
2026-09-15 00:04:02 -07:00
yassin
555e321cf1 fix(router): record flat retry attempts and cap retries from attempted_retries
Router.log_retry used to copy the failed attempt's kwargs and metadata into
metadata.previous_models. Nothing downstream read those copies, but they carried
client credentials into spend logs and grew the payload on every retry. Each
attempt now leaves a flat record (model group, deployment id, exception type and
string, attempt number), which drops RETRY_BREADCRUMB_EXCLUDED_KWARGS and the
per-retry credential masking.

num_retries_per_request was enforced from len(previous_models), which only
looked at the metadata bucket and never exceeded four records. The sync and
async client wrappers and the Rust lifecycle guard now read attempted_retries
from whichever metadata bucket the call carries.

Resolves LIT-7505

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 01:05:51 +00:00
mateo-berri
d7158ba795 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ocr_custom_pricing 2026-09-12 15:42:59 -07:00
yujonglee
347b642bdd
refactor(ocr): complete native lifecycle and preserve Azure auth (#40734)
* refactor(ocr): extract call completion boundary

* fix(ocr): release completion state after dispatch

* test(ocr): prove wrapper completion handoff

* test(ocr): narrow mapped failure assertion

* fix(ocr): preserve wrapper invocation kwargs

* fix(ocr): retain completion through finalization

* fix(ocr): make completion ownership explicit

* refactor(ocr): resolve logging executor explicitly

* fix(callbacks): preserve completion lifecycle behavior

* refactor(ocr): move public OCR into native lifecycle

* refactor(ocr): remove unused rust bridge capability

* wip

* wip

* refactor

* wip

* fix(ocr): preserve reducto native compatibility

* wip

* fix(ocr): document native callable casts

* perf(ocr): bound responses and reduce native scheduling overhead

* refactor(python-bridge): organize placeholder routes

* refactor test

* fix(ocr): normalize DeepSeek document content

* perf(ocr): skip unused callback work and benchmark callback overhead

* fix(ocr): align conversion contracts

* test(ocr): cover official provider response shapes

* fix(ocr): restore Python fallback and honor Rust opt-out

* fixes and refactor

* fix(ocr): preserve Azure Document Intelligence authentication

* fix(rust): enforce OCR response limits and lint contracts

* test(rust): align native OCR contract coverage

* test(ocr): isolate Azure auth precedence coverage
2026-09-12 11:56:49 -07:00
yujonglee
89f1f9567d
refactor(ocr): route native requests through core (#40532)
* refactor(ocr): route native Mistral through core

* fix(ocr): preserve Azure API base resolution

* chore(ocr): document bridge boundary casts

* fix(ocr): keep Azure environment resolution in Rust

* fix(ocr): centralize native execution and isolate request logging

* refactor(ocr): narrow native migration to bridge routing

---------

Co-authored-by: Stack Plan <stack-plan@example.invalid>
2026-09-11 12:37:19 -07:00
yujonglee
61b0def867
test(ocr): isolate native Rust bridge contracts (#40410)
* test(rust): add retained callback suite as expected failures

* fix(tests): narrow retained callback xfails

* test(rust): clarify retained callback contracts

* test(ocr): clarify retained Rust contracts

* test(ocr): restore guardrail contracts

* test(ocr): require Rust file input parity

* test(ocr): isolate native bridge contracts

* fix(ci): repair Rust dispatch and OSV checks

* test(ocr): assert explicit backend dispatch
2026-09-10 16:51:37 -07:00
yujonglee
13df85cceb
test: add Rust extension pytest contract (#40181)
* test: add Rust extension pytest contract

* test: prove native OCR execution

* test: isolate Rust extension pytest collection

* ci: register Rust extension test coverage

* test: prove native OCR at wire boundary
2026-09-07 18:46:29 -07:00