litellm/tests/unit/proxy/hooks
norkelbyte ac1cc5111c
fix(proxy): run baggage tag mappings and budget checks on /queue/chat/completions (#51753)
* fix(proxy): run baggage tag mappings and budget checks on /queue/chat/completions

* fix(proxy): check the key's max_budget on /queue/chat/completions

* fix(proxy): run budget, tag, and request policy checks on queued chat calls

* fix(proxy): bill a dropped queued stream and run the trace ID and failure logging checks on queued calls

A queued stream now hands its logging object to the stream generator, so a
client that leaves mid-stream is billed for the reply it already got, and a
stream left before the first model chunk releases its budget reservation.
A team's require_trace_id and the failure logging for rejected calls now
use the same route check as the other inference checks.

* fix(proxy): settle a dropped queued stream's budget hold through the first-chunk step chat uses

* fix(proxy): keep the trace ID, call ID, unpriced fallback filter, and JWT admin team header on queued chat calls

* fix(proxy): send the call ID on a queued call the provider rejects before streaming

* fix(proxy): release a dropped stream's budget hold only when nothing bills it

A guardrail that buffers the stream can pull model chunks the client never
saw. The early-drop release settled the hold at the input cost first, then
the partial bill for those chunks added its full cost on top. The release
now runs after partial billing and only when no bill fired

* fix(proxy): fold the caller's litellm_metadata into a queued call's metadata

A queued call that sent a litellm_metadata bucket, even an empty one, made tag
routing and logging read that bucket, so the key and team tags the handler
merged into metadata no longer narrowed deployments. The queue handler now
folds the bucket into metadata and drops it, as chat does, after stripping the
proxy-owned user_api_key_* and internal control fields from both buckets, so a
caller can no longer set the queued spend row's organization ID either

* fix(proxy): release a queued call's budget hold when the call is cancelled before it answers

* fix(proxy): refund a cancelled queued call's hold in full until the model call claims it

* fix(proxy): keep a cancelled queued call's hold release running when the call is cancelled again

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 08:54:54 -05:00
..
litellm_skills test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
__init__.py ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903) 2026-09-24 22:59:11 +00:00
test_async_post_call_streaming_iterator_hook.py fix(guardrails): run the end-of-stream post_call scan when the client disconnects mid-stream (#43839) 2026-10-05 01:44:43 -07:00
test_autorouter_baseline_cache.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_banned_keyword_list.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_batch_enqueued_tokens.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_batch_file_validation.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_batch_rate_limiter.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_dynamic_rate_limiter.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_dynamic_rate_limiter_v3.py fix(dynamic_rate_limiter_v3): publish a team's share gauges for requests served while the model is idle (#51709) 2026-11-13 14:58:54 -08:00
test_image_generation_guardrails.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_key_management_event_hooks.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_max_budget_per_session_limiter.py fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473) 2026-11-07 17:50:06 +09:00
test_max_iterations_limiter.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_message_thread_ownership.py feat(anthropic): forward message threads on native /v1/messages (#50976) 2026-11-10 18:33:54 -08:00
test_model_budget_shared_windows.py feat(proxy)!: model family and access group entries in model_max_budget (#47722) 2026-10-28 15:40:57 -07:00
test_model_max_budget_limiter.py feat: add OpenRouter-style -latest model aliases for first-party providers (#50315) 2026-11-06 18:22:38 +08:00
test_parallel_request_limiter.py fix(proxy): count native Responses WebSocket turn tokens toward key and tag TPM limits (#49257) 2026-10-31 02:53:27 -07:00
test_parallel_request_limiter_v3.py feat(proxy): enforce customer org, team and member budgets and limits on the request path (#51489) 2026-11-12 20:43:44 -08:00
test_post_call_failure_hook_integration.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_post_call_response_headers_hook.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_post_call_streaming_hook_integration.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_post_call_success_hook_integration.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_priority_class_share.py feat(dynamic_rate_limiter_v3): priority_class_share_splits_equally_among_active_teams splits a priority class share among the teams active on a model (#51366) 2026-11-12 06:38:07 -08:00
test_prompt_cache_observer.py fix(proxy): keep a retried /v1/messages stream's spend log by logging the superseded attempt as a failure (#45136) 2026-10-16 21:19:58 +00:00
test_prompt_injection_detection.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_proxy_hooks_init.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_proxy_rate_limit_provider_field.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_proxy_track_cost_callback.py fix(proxy): run baggage tag mappings and budget checks on /queue/chat/completions (#51753) 2026-11-14 08:54:54 -05:00
test_rate_limiter_toctou.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_search_free_tier.py fix(proxy): scope the search free-tier counter to the request's provider (#47646) 2026-10-20 22:01:53 +00:00
test_send_invite_email.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
test_sensitive_data_routing.py fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473) 2026-11-07 17:50:06 +09:00
test_tpm_concurrent.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_unit_test_max_model_budget_limiter.py fix(router): inspect image content without translating routed requests (#49371) 2026-10-31 10:44:24 -07:00
test_user_management_event_hooks.py test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00