mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-13 03:59:12 +00:00
157d96dcd0
* fix(logging): claim the async success dedup flag before the first await Nested @client wrappers on the chat -> Responses bridge schedule two async success tasks on one logging object. The dedup flag was checked at the top of _async_success_handler_body but only set after several awaits; the worker-thread base64 offload for messages >= 256 KiB yields there, so the second task passed the same check and the request was logged twice (two SpendLogs rows, key/team/daily spend counted twice). Fixes #44500 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(logging): serialise async success handlers per logging object instead of claiming early Claiming the dedup flag before the first await (previous commit) let a first handler that is cancelled before its callbacks run (e.g. by the logging worker's per-coroutine timeout) suppress the second one, losing the request's success accounting. Serialise non-streaming handlers on the same Logging object with an asyncio.Lock kept in a module-level WeakKeyDictionary: the second handler waits, then skips if the first logged, or logs itself if the first raised or was cancelled first. Streaming chunk logging is not locked. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(integration): pin one spend row and one key charge for large bridged chat completions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): wait for the large request's spend row before asserting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(logging): use fixed timestamps in async success dedup tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(logging): trim the async success dedup lock comment Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(logging): reset background interaction dedupe under the success dedup lock Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(logging): use a fixed start time in the interactions logging helper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: claim the async success log at schedule time instead of locking the handler Nested @client wrappers (chat over the Responses bridge, Anthropic Messages over the chat adapter) exit with one shared Logging object and each queued their own async success handler. The handler checked has_logged_async_success before its awaited work and set it after, so both could pass the check and the request was logged and billed twice. _schedule_async_success_logging now claims the log on the Logging object synchronously through claim_async_success_log, and a later wrapper returns without enqueuing. There is no await between the check and the decision, so no lock is needed. The per-object asyncio.Lock, the WeakKeyDictionary that held it and the locked block in async_log_background_interaction_completion are gone. Tests: the regression test moves to tests/unit/test_utils.py and drives two _dispatch_success_logging exits with one Logging object through the logging worker, asserting one log with the inner result. It fails without the claim. The cancelled-handler test is dropped since a second wrapper no longer retries. The background interaction completion test stays. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(logging): pin the background completion event order and name the innermost wrapper in the claim docstrings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): send one bridged request and assert the inner Responses result is the logged one Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| fixtures | ||
| golden | ||
| _daily_activity_fixtures.py | ||
| test_background_interaction_settlement.py | ||
| test_batch_completion_accounting.py | ||
| test_batch_enqueued_tokens_redis_lua.py | ||
| test_batch_observability.py | ||
| test_batch_poll_starvation.py | ||
| test_cache_and_quota.py | ||
| test_chaos_burst_spend_once.py | ||
| test_daily_activity_aggregated_breakdowns.py | ||
| test_daily_activity_key_alias_probes.py | ||
| test_daily_activity_key_metadata_query_timeout.py | ||
| test_daily_activity_key_owner.py | ||
| test_daily_activity_key_owner_faults.py | ||
| test_daily_activity_key_owner_traffic.py | ||
| test_daily_activity_repository.py | ||
| test_daily_activity_routes.py | ||
| test_daily_rollup_retry.py | ||
| test_daily_tag_spend_retention.py | ||
| test_disconnected_bedrock_messages_stream_billing.py | ||
| test_end_user_spend_without_proxy_user.py | ||
| test_failed_dispatch_tokens.py | ||
| test_filtered_ledger.py | ||
| test_global_spend_report.py | ||
| test_image_generation_key_spend.py | ||
| test_key_budget_lockout.py | ||
| test_key_metadata_recovery_dropped_connection.py | ||
| test_key_metadata_recovery_probe_bounds.py | ||
| test_legacy_spend_logs_row_cap.py | ||
| test_lens_billing.py | ||
| test_messages_stream_usage_cost.py | ||
| test_model_router_selected_model.py | ||
| test_normalized_error_long_message.py | ||
| test_org_budget_cli_session_token.py | ||
| test_passthrough_budget_reservation.py | ||
| test_passthrough_request_tags.py | ||
| test_prompt_caching_requests_pagination.py | ||
| test_redis_ttl_preserving_token_increment.py | ||
| test_reset_budget_leader_election.py | ||
| test_responses_bridge_spend_once.py | ||
| test_responses_cache_write_itemization.py | ||
| test_roi_branch_spend.py | ||
| test_service_tier_stream_billing.py | ||
| test_session_total_spend.py | ||
| test_shutdown_flush.py | ||
| test_spend_calculate.py | ||
| test_spend_capture_rate_captured_spend.py | ||
| test_spend_log_read_scope.py | ||
| test_spend_log_tool_payload_content.py | ||
| test_spend_log_write_batching.py | ||
| test_spend_rollup_accuracy.py | ||
| test_stream_alias_billing.py | ||
| test_tag_budget_enforcement.py | ||
| test_team_budget_enforcement.py | ||
| test_team_daily_activity_aggregated.py | ||
| test_team_member_budget_alerts.py | ||
| test_team_member_spend.py | ||
| test_team_member_spend_flush.py | ||
| test_user_budget_on_team_keys.py | ||
| test_vertex_image_regional_uplift.py | ||
| test_vertex_regional_uplift_spend.py | ||