mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
bd15250960
Fixes #16486 ## Problem Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith) were not being invoked for litellm.acompletion() in short-lived scripts. The callbacks worked correctly for synchronous completions but async completions would queue callbacks that were lost when the script exited before GLOBAL_LOGGING_WORKER could process them. Root cause: asyncio.run() closes the event loop immediately after the async function completes, preventing the background worker from processing queued callbacks. ## Solution Implemented a two-level atexit handler approach: 1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py): - Processes remaining callbacks from queue before exit - Creates new event loop to run pending coroutines synchronously - Applies time and iteration limits to prevent blocking shutdown 2. Integration-specific atexit handlers (posthog.py as example): - Flushes internal queue to external service - Uses synchronous HTTP client for reliable delivery - Each integration needs its own handler due to varying sync APIs ## Changes - litellm/litellm_core_utils/logging_worker.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events - Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds - litellm/integrations/posthog.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Groups events by credentials for batch sending - Uses sync_client for reliable HTTP delivery - tests/logging_callback_tests/test_posthog.py: - Added test_async_callback_atexit_handler_exists() - Added test_posthog_atexit_flushes_internal_queue() - Added test_sync_callback_not_affected_by_atexit() ## Testing - All existing tests pass - Manual end-to-end testing confirms fix: - Async events now arrive in PostHog - Sync events continue working (no regression) - Unit tests verify atexit handlers registered and functional ## Impact - Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER - No breaking changes - only adds missing functionality - Sync path unchanged - no performance impact |
||
|---|---|---|
| .. | ||
| gcs_pub_sub_body | ||
| langfuse_expected_request_body | ||
| base_test.py | ||
| conftest.py | ||
| create_mock_standard_logging_payload.py | ||
| gettysburg.wav | ||
| log.txt | ||
| test_alerting.py | ||
| test_amazing_s3_logs.py | ||
| test_assemble_streaming_responses.py | ||
| test_azure_blob_storage.py | ||
| test_bedrock_knowledgebase_hook.py | ||
| test_built_in_tools_cost_tracking.py | ||
| test_custom_callback_router.py | ||
| test_datadog.py | ||
| test_datadog_llm_obs.py | ||
| test_gcs_pub_sub.py | ||
| test_generic_api_callback.py | ||
| test_humanloop_unit_tests.py | ||
| test_langfuse_e2e_test.py | ||
| test_langfuse_unit_tests.py | ||
| test_langsmith_unit_test.py | ||
| test_log_db_redis_services.py | ||
| test_logging_redaction_e2e_test.py | ||
| test_moderations_api_logging.py | ||
| test_opentelemetry_unit_tests.py | ||
| test_otel_logging.py | ||
| test_pagerduty_alerting.py | ||
| test_posthog.py | ||
| test_spend_logs.py | ||
| test_sqs_logger.py | ||
| test_standard_logging_payload.py | ||
| test_token_counting.py | ||
| test_unit_test_litellm_logging.py | ||
| test_unit_tests_init_callbacks.py | ||
| test_view_request_resp_logs.py | ||