mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-14 09:03:16 +00:00
917fd11966
* fix(rate-limit): settle a fallback's tokens on the model group that served it The v3 limiter pinned post-call TPM settlement to the model reserved at admission, so when the router fell back to another model group the requested model's counter took the usage and the serving model's counter stayed empty. When the served model group differs from the admitted one, charge the served group and refund the abandoned group's per-model reservation. Same-model retries and alias reloads keep the admission target. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(rate-limit): refund an abandoned fallback reservation only on counters the served model does not settle Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(rate-limit): wait for the primary refund and the fallback charge before reading counters Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(rate-limit): only treat a limiter-admitted fallback target as the served model at settlement Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(rate-limit): drop unrelated formatting from the v3 limiter unit test file Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(rate-limit): remove helper copies duplicated while appending the fallback settlement tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| litellm_skills | ||
| __init__.py | ||
| test_async_post_call_streaming_iterator_hook.py | ||
| test_autorouter_baseline_cache.py | ||
| test_banned_keyword_list.py | ||
| test_batch_enqueued_tokens.py | ||
| test_batch_file_validation.py | ||
| test_batch_rate_limiter.py | ||
| test_dynamic_rate_limiter.py | ||
| test_dynamic_rate_limiter_v3.py | ||
| test_image_generation_guardrails.py | ||
| test_key_management_event_hooks.py | ||
| test_max_budget_per_session_limiter.py | ||
| test_max_iterations_limiter.py | ||
| test_message_thread_ownership.py | ||
| test_model_budget_shared_windows.py | ||
| test_model_max_budget_limiter.py | ||
| test_parallel_request_limiter.py | ||
| test_parallel_request_limiter_v3.py | ||
| test_post_call_failure_hook_integration.py | ||
| test_post_call_response_headers_hook.py | ||
| test_post_call_streaming_hook_integration.py | ||
| test_post_call_success_hook_integration.py | ||
| test_prompt_cache_observer.py | ||
| test_prompt_injection_detection.py | ||
| test_proxy_hooks_init.py | ||
| test_proxy_rate_limit_provider_field.py | ||
| test_proxy_track_cost_callback.py | ||
| test_rate_limiter_toctou.py | ||
| test_search_free_tier.py | ||
| test_send_invite_email.py | ||
| test_sensitive_data_routing.py | ||
| test_tpm_concurrent.py | ||
| test_unit_test_max_model_budget_limiter.py | ||
| test_user_management_event_hooks.py | ||