litellm/tests/unit/litellm_core_utils
yucheng-berri dd511a2b22
fix(logging)!: fire one CustomLogger failure hook per async call outside the async flag list (#47124)
* fix(logging): fire one CustomLogger hook per async call outside the async flag list

aspeech, atranscription, aimage_generation, aimage_edit, aocr and asearch are
missing from is_sync_litellm_request's flag list, so the sync success and
failure handlers called a CustomLogger's sync hook on top of the async hook the
async handler had already called for the same request.

The sync handlers now skip a CustomLogger's log_success_event and
log_failure_event when that run is paired with an async handler run
(for_async_request) and the async handler gives the logger its own async hook.
The async client wrapper, dispatch_failure_handlers, the proxy-only failure
dispatch and the native bridge's sync failure step for an async call mark their
sync failure run as paired.

Resolves LIT-9486

* fix(logging): keep the sync CustomLogger hook when no async success runs

The async client wrapper skips async success logging for calls carrying a
fallbacks kwarg, and the native bridge skips it for internal and fallbacks
calls, yet the paired sync success run still dropped a both-hooks logger's
sync hook, so that logger got no success event at all.

The sync success run now suppresses the sync hook only when the async success
handler was actually scheduled for the call: the client wrapper passes
async_handler_runs=not is_completion_with_fallbacks, and the native host
records per call whether it enqueued or deferred the async success.

Tests that pinned the doubled failure hook on native aocr, messages and
responses calls and on proxy-rejected requests now expect the async hook
alone. The integration test adds /v1/messages and /v1/responses cells and
runs the key-auth rejection on every route.

* fix(logging): type dispatch_success_handlers kwargs

The new async_handler_runs keyword on success_handler made the untyped
**kwargs that dispatch_success_handlers forwards an unknown argument for it,
which grew the reportUnknownArgumentType count past its merge-base ceiling.
Typing the forwarded kwargs as Any, as _success_handler_body already does,
removes those diagnostics without changing what is forwarded.

* test(integration): cover CustomLogger hook counts across modes, chaos and router strategies

Moves the hook count rig into tests/integration/_support/hook_counts.py and
extends the cells it drives:

- streamed chat, Messages and Responses requests, success and failure
- loggers the async handler never reaches (sync-only overrides, sync event
  lists only) keep one sync hook per request on every surface outside the
  async flag list, key-auth rejections included
- a concurrent burst while the logger's destination stalls and then fails,
  and a burst while a worker is killed, never give a request two hooks
- least-busy keeps counting a request in flight while async speech requests
  finish around it, and usage-based routing v2 counts an async image
  request's tokens once

* test(integration): Datadog and LangSmith deliver one record per async request

Points the built-in Datadog, LangSmith and OpenTelemetry loggers at a local
sink and checks every async surface outside the flag list: one Datadog log
entry and one LangSmith run per request (two on the merge base), and never two
OpenTelemetry spans, which its own emit-once guard already ensured.

* fix(logging): give a both-hooks logger its sync hook when the async run never reaches it

The sync hook was skipped whenever the logger sat in the async callback list, so a
logger the async run never reached got no hook at all: a sibling stalled past the
logging worker timeout, a batch poll that skips default logging, or an async
failure run cut off. The sync run now holds the hook and the async run hands it
back unless it reached the logger

* fix(logging): run the held sync success hooks when the proxy drops a blocked response's success

A post-call guardrail block drops the parked async success, which left the sync
success hook held for it unrun. Base gave a both-hooks logger that hook on these
call types, so the drop now ends the pairing and the held hook runs as before

* fix(logging): run held sync hooks only after every sync run of the request has closed

A held hook released by the async run could run on another thread while the sync
run was still walking its other callbacks. The pairing now waits until the async
run has ended and every sync run has closed, and the last sync run to close runs
the held hooks itself

* fix(logging): end the async run when its coroutine is dropped before it starts

A logging queue that drops the async success coroutine, or a deferred success
that fails to schedule, left the sync hooks held for it unrun. The enqueued async
run now ends when its coroutine is discarded, and the proxy ends it when firing
the deferred success raises

* fix(logging): leave a still-queued async run to the exit flush

The discard finalizer ran at interpreter exit before the logging worker's exit
flush, so a queued request could get its held sync hook and then its async hook

* test: keep the router and file-stream tests in step with main

Main gives each owned proxy a private Redis by default, so the router cells
share the worker's Redis to read its counters, and the file-stream stub takes
the hook pairing the stream now passes

* fix(logging): run held sync hooks when a during_call guardrail raises or the exit flush gives up

A during_call guardrail that raised after the LLM call parked its deferred
success skipped the proxy's flush, and async runs the exit flush never started
were never ended, so a both-hooks logger lost its sync hook in both cases

* fix(logging): skip the sync failure hook when the async failure run logs to the logger

An async call outside is_sync_litellm_request's flag list ran both failure
handlers, so a CustomLogger with both failure hooks got both. The sync run
of an async request now skips log_failure_event for a logger that overrides
async_log_failure_event and is registered for async failures. The four sync
runs of async failures (dispatch_failure_handlers, the @client async wrapper,
the proxy-only failure thread and the Rust host) pass for_async_request=True.

Drops the HookPairing fallback. Successes are left to #47192, which already
gives such a logger only async_log_success_event.

---------

Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-20 15:19:14 -07:00
..
audio_utils
llm_cost_calc fix(bedrock): resolve profile pricing before applying custom rates (#47419) 2026-10-20 00:10:49 -07:00
llm_response_utils fix(caching): keep prompt cache usage on streamed cache hits (#47073) 2026-10-16 23:28:11 -07:00
prompt_templates fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
specialty_caches test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
__init__.py
conftest.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
event_loop_lag.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
fake_secret_vault.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
messages_with_counts.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_agentic_followup_kwargs.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_anthropic_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_api_route_to_call_types.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_app_crypto.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_audio_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_aws_partition.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_bedrock_converse_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_bug_report.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_chat_completion_agentic_loop.py fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967) 2026-10-16 18:06:01 -07:00
test_classifier_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cli_token_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cloud_storage_security.py fix(proxy)!: let proxy admin keys read raw cloud storage file ids on GET /files/{id}/content (#40621) 2026-10-20 10:58:42 +00:00
test_codestral_provider_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_core_helpers.py perf(streaming): cut per-chunk pydantic round trips on openai-compatible streams (#47030) 2026-10-20 02:42:32 +00:00
test_coroutine_checker.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cost_discount.py feat(cost): support provider/model-pattern keys in cost_discount_config (#43335) 2026-10-12 14:59:18 -07:00
test_custom_logger_registry.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_dd_tracing.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_decisions_pricing.py feat(decisions): bill decisions and systemone from _decisions cost map fields (#45904) 2026-10-12 14:31:12 -07:00
test_decode_special_tokens.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_default_encoding.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_dot_notation_indexing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_duration_parser.py feat(proxy): report when each per-model budget window resets (#45797) 2026-10-15 10:56:45 -07:00
test_env_utils.py fix(config): treat blank env values as unset so the proxy neither crash-loops nor silently drops a guardrail (#46341) 2026-10-14 12:49:31 -07:00
test_error_normalization.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_exception_mapping_utils.py fix(streaming)!: map stream read timeouts to litellm.Timeout on chat, messages, and responses (#47198) 2026-10-18 13:58:34 +00:00
test_extract_base64_image.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_fallback_generalizations.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_fallback_utils.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_get_litellm_params.py feat(github_copilot): support a GHE.com host on per-user Copilot credentials (#47610) 2026-10-20 14:16:35 -07:00
test_get_llm_provider_endpoint_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_get_llm_provider_logic.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_get_model_cost_map.py test: catch regressions in the validation lines #44798 added (#47420) 2026-10-20 01:06:37 +00:00
test_get_supported_openai_params.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_health_check_helpers.py fix(azure): cache the remaining per-request Entra credentials and keep token refresh off the event loop (#47348) 2026-10-19 20:35:27 +00:00
test_hidden_params.py perf(streaming): cut per-chunk pydantic round trips on openai-compatible streams (#47030) 2026-10-20 02:42:32 +00:00
test_image_handling.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_initialize_dynamic_callback_params.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_internal_call_metadata.py fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703) 2026-10-15 11:00:14 -07:00
test_json_fragment_accumulator.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_json_schema_validation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_litellm_logging.py fix(logging)!: fire one CustomLogger failure hook per async call outside the async flag list (#47124) 2026-10-20 15:19:14 -07:00
test_llm_judge.py fix(guardrails): resolve decision models in caller team scope (#46085) 2026-10-15 11:43:36 -07:00
test_llm_request_utils.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_logging_callback_manager.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_logging_utils.py fix(logging): bound data URI regex so base64 truncation stays linear (#45132) 2026-10-07 12:32:04 -07:00
test_logging_worker.py fix(router): preserve native baseline identity and accounting (#44960) 2026-10-07 11:00:31 -07:00
test_max_streaming_duration.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_model_param_helper.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_model_response_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_moderation_standard_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_oauth_token_exchange.py feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158) 2026-10-09 15:19:07 -07:00
test_optional_imports.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_path_utils.py refactor(core): move path_utils from proxy common_utils to litellm_core_utils (#45775) 2026-10-10 19:07:22 +00:00
test_private_json.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_affinity.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_specific_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_ptu_pricing.py feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
test_realtime_errors.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_realtime_streaming.py fix(bedrock)!: block realtime replay after an input-first start and reject malformed Nova Sonic session settings (#46936) 2026-10-17 00:29:50 +00:00
test_redact_messages.py fix(logging)!: redact Responses instructions and callback output (#43315) 2026-10-20 20:38:17 +00:00
test_request_timeout_resolver.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_retry_after_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_divide_seconds.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_json_dumps.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_sensitive_data_masker.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_sentry_scrubbing.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_served_output_texts.py fix: handle missing choice indices in streamed output recording (#46947) 2026-10-17 14:17:46 -07:00
test_stream_usage_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_streaming_chunk_builder_cursor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_server_tool_use.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_utils.py fix(streaming): keep finish_reason and provider usage when rebuilding streamed text completions (#44349) 2026-10-20 05:31:39 +00:00
test_streaming_handler.py fix(streaming): restore the token estimate for streams that send no usage (#44230) 2026-10-20 08:50:29 +00:00
test_streaming_overhead.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_text_completion_conversion.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_thread_pool_executor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter.py fix(token_counter): count DeepSeek V4 text with DeepSeek's own tokenizer (#47353) 2026-10-20 03:43:35 +00:00
test_token_counter_tool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter_tool_data.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_tokenizer.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_tool_fields.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_tool_search_spend_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_url_utils.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_xai_oauth_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00