Commit graph

2 commits

Author SHA1 Message Date
devin-ai-integration[bot]
a9bb556111 feat(proxy): lift every earlier billed timed-out retry attempt onto the failed request (#47490)
* feat(proxy): lift every earlier billed timed-out retry attempt onto the failed request

_billed_earlier_attempts reads the router's previous_models records, keeps the Timeout ones other than the final attempt (matched on deployment_id plus attempted_retries, since the count restarts after a fallback hop and the fallback handler re-logs the exhausting attempt) and prices each at the deployment that served it. post_call_failure_hook lifts the tuple as billed_failed_attempts; nothing writes it yet. previous_models joins the router-reserved metadata fields so a client-planted record never reaches it

* fix(proxy): price earlier timed-out attempts from the prompt estimate only

The final row may lift a mid-stream recovered usage (prompt plus delivered chunks) or a failed_attempts_usage sum; earlier attempts never produced output, so they are priced from _estimated_prompt_usage, and when failed_attempts_usage already bills them into the final row none is lifted again

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Backport of #47490 to stable/1.105.x.
Cherry-picked from 7a4a255d1d (main).
2026-10-21 00:57:49 +00:00
devin-ai-integration[bot]
a76b59db9f
test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 18:23:31 +00:00
Renamed from tests/test_litellm/proxy/test_litellm_pre_call_utils.py (Browse further)