litellm/tests/integration/observability
yuneng-jiang 822ccfc11d
fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) (#43354)
* fix(s3_v2): drop terminal upload failures, bound retries per flush and enforce the queue cap at enqueue



* fix(s3_v2): keep retrying credential-rotation 403s, only AccessDenied-style errors are terminal



* test(env_keys): exclude DEFAULT_S3_MAX_FLUSH_ATTEMPTS as an internal tuning var



* fix(s3_v2): retry every 5xx, warn on first queue overflow, validate the flush budget



* fix(s3_v2): read the queue cap defensively so un-initialized loggers still enqueue



* fix(s3_v2): drop the getattr in _enqueue and tighten the retry tests



* fix(s3_v2): keep the constructor flush budget when the callback override is invalid



* feat(s3_v2): adapt per-object upload concurrency to sink latency and throttling



* refactor(s3_v2): tidy adaptive limiter



* perf(s3_v2): wake one waiter per released upload slot



* feat(s3_v2): make the enqueue queue cap configurable with s3_max_queue_size



* test(s3_v2): audit cells for cache hits, coded 403 and callback modes



* fix(s3_v2): retry bucket-wide failures by default, age-budget requeues and make terminal drops and adaptive concurrency opt-in



* refactor(s3_v2): suppress the missing-waiter ValueError explicitly in the adaptive limiter



* fix(s3_v2): count oldest events trimmed after a failed flush as callback failures



* style(s3_v2): move the mutable-ok marker onto the list literal it suppresses



* fix(s3_v2): report post-flush overflow drops once and grow adaptive concurrency above the floor before asserting back-off



* test(s3_v2): fail the SlowDown back-off test when the measured window sees no PUTs



* fix(s3_v2): restore base retry defaults, opt-in age budget, no enqueue cap, back off outside the limiter slot



* fix(s3_v2): hoist the default no-op upload slot to a module constant



* fix(s3_v2): drop unused mutable-ok suppressions on queue appends



* fix(s3_v2): keep the retry queue oldest-first and prioritise fresh events at upload time



* style(s3_v2): keep the mutable-ok marker on the queue list literal



* fix(s3_v2): build request bodies inside the upload slot and keep the sync retry set at base parity



* test(s3_v2): drop wall-clock sleeps from the unit tests



* fix(s3_v2): rebuild the request body inside the slot on every retry attempt



* test(s3_v2): anchor the backoff window on the first observed failure and tighten shard assertions



* fix(s3_v2): default the upload slot to the logger limiter so monkeypatched doubles keep working



* test(s3_v2): clear ambient AWS env credentials so the rotating profile signs the sync retry test



* test(s3_v2): shrink the linear send-batch perf test to 2k/8k elements



* test(s3_v2): fix stale batch sizes in the perf test assert message



* fix(s3_v2): keep async in-call retries on the base 403/500/503 set



* fix(s3_v2): hold the upload slot across retries, restore the bool upload contract, and fail safe on bool config



* fix(s3_v2): mark dropped uploads by element identity so a shared key cannot mask a retryable sibling



* fix(s3_v2): make the per-flush drop lookup constant time



* fix(s3_v2): take the upload slot in the caller like base, build the body once per attempt loop



* fix(s3_v2): match base retry, logging and hook behaviour unless the new options are opted in



* test(s3_v2): assert the wire key in the init-bypassed sync upload test



* test(s3_v2): assert the signed headers and wire key in the init-bypassed sync upload test



* fix(s3_v2): default the upload limiter at class level instead of reading it with getattr



* fix(s3_v2): drop the duplicate annotations that redeclare the class-level counters



* feat(s3_v2): drop terminal-failed uploads by default and bound retry age to one hour



* fix(s3_v2): fall back to the configured retry age on invalid values



* test(s3_v2): drive retry-age tests from a fixed clock



* test(s3_v2): audit cells for retry-age opt-out and 429 single-put parity



---------



(cherry picked from commit e47b1f2a3f)

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-26 15:46:08 -07:00
..
_s3_v2_support.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) (#43354) 2026-09-26 15:46:08 -07:00
test_bedrock_error_request_id.py test(integration): cover customer reported cache key, cache_control, bedrock request id, responses schema, scim and tag budget contracts (#42785) 2026-09-23 13:19:38 -07:00
test_cache_hit_guardrail_metrics.py fix(proxy): keep deployment labels on cache-hit post_call guardrail rejections (#42780) 2026-09-23 22:48:04 -07:00
test_cache_hit_guardrail_metrics_chaos.py test(guardrails): run the cache-hit redis outage test on the shared owned_redis helper (#42925) 2026-09-24 12:00:10 -07:00
test_callback_delivery.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_guardrail_effects.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_langfuse_delivery.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_otel_conversation_id.py feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486) 2026-09-23 00:49:29 -07:00
test_otel_text_completion_choices.py fix(otel): keep text completion choice fields beside the synthesized message (#42537) 2026-09-22 14:24:32 -07:00
test_passthrough_upstream_error_chaos.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_passthrough_upstream_error_visibility.py fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook (#42695) 2026-09-23 23:42:24 -07:00
test_presidio_streaming_output.py fix(presidio): mask streamed /v1/messages output when the first upstream read is a keepalive, a data-less ping, or a split utf8 character (#43023) 2026-09-25 00:48:42 -07:00
test_s3_v2_flush_surfaces.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) (#43354) 2026-09-26 15:46:08 -07:00
test_s3_v2_upload_fanout.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) (#43354) 2026-09-26 15:46:08 -07:00
test_straiker_v3_platform.py test(straiker): deterministic integration audit of the v3 platform relay (#42781) 2026-09-23 15:57:51 -07:00