litellm/litellm/ocr
Nate Armstrong 1467b13fd2
fix(rust): run native async calls through python's async wrapper steps (#50423)
Python's async client wrapper stamps the effective timeout and num_retries on every failure, which the proxy reads for x-litellm-timeout and the router reads for retries. The native async paths skipped that wrapper, so error responses from Rust-served async calls had no x-litellm-timeout header

Python's async chat completions, responses, embeddings and messages entries hand the call to run_in_executor before they build the provider body, so a during_call guardrail started next to the call masks the request first. The native paths built their request without yielding, so masked text never reached the provider

The shared native_acall_hook now yields once and stamps the failure, and every async dispatch passes it as its native hook. OCR passes native_acall_hook_on_loop, which only stamps the failure, since its Python entry went through the client wrapper but built its request on the loop
2026-11-10 00:24:20 +00:00
..
__init__.py cleanup 2026-09-16 14:16:15 -07:00
dispatch.py fix(rust): run native async calls through python's async wrapper steps (#50423) 2026-11-10 00:24:20 +00:00