mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-08 22:21:35 +00:00
33e9f54dc8
The TPM pre-call reservation read `max_tokens or max_completion_tokens`, so a request declaring both was charged for whichever field came first. A caller sending `max_tokens=1` with `max_completion_tokens=10000` reserved 2 tokens and was then free to consume ten thousand, since the provider honours the modern field and litellm's own param mapping drops the legacy one for the gpt-5 and o-series families. Reserve against the larger of the declared budgets instead. Over-reserving is the safe direction for a limiter: post-call reconciliation refunds the difference between the reservation and actual usage, while under-reserving lets the window be exceeded before anything notices. |
||
|---|---|---|
| .. | ||
| litellm_skills | ||
| test_async_post_call_streaming_iterator_hook.py | ||
| test_batch_file_validation.py | ||
| test_dynamic_rate_limiter.py | ||
| test_dynamic_rate_limiter_v3.py | ||
| test_image_generation_guardrails.py | ||
| test_key_management_event_hooks.py | ||
| test_max_budget_limiter.py | ||
| test_max_budget_per_session_limiter.py | ||
| test_max_iterations_limiter.py | ||
| test_parallel_request_limiter.py | ||
| test_parallel_request_limiter_v3.py | ||
| test_post_call_failure_hook_integration.py | ||
| test_post_call_response_headers_hook.py | ||
| test_post_call_streaming_hook_integration.py | ||
| test_post_call_success_hook_integration.py | ||
| test_prompt_injection_detection.py | ||
| test_proxy_hooks_init.py | ||
| test_proxy_rate_limit_provider_field.py | ||
| test_proxy_track_cost_callback.py | ||
| test_rate_limiter_toctou.py | ||
| test_send_invite_email.py | ||
| test_sensitive_data_routing.py | ||
| test_tpm_concurrent.py | ||
| test_user_management_event_hooks.py | ||