* test(e2e): event_loop_stalls charges a hold to the longest gc pass that fits it, not to the sum of every pass since the last hold
* test(e2e): event_loop_stalls lists the pass-through post-stream logging step seen once at 0.105 s
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): event_loop_stalls perf case reads the gateway's asyncio debug log and fails on a loop hold over 100 ms that is not on its allowlist
The gateway runs the stdlib loop in debug mode (new LITELLM_EVENT_LOOP=asyncio knob next to
PYTHONASYNCIODEBUG=1), the case replays the mixed body mix at 1 rps and groups asyncio's
'Executing <Task ...> took X seconds' lines by coroutine and location. Known holds are listed with
the count they show today; anything else, or a known one past its count, fails the case. Opt-in
through E2E_PERF_EVENT_LOOP=1 so the nightly build skips it
* test(e2e): event_loop_stalls reads the pod list and pod logs through e2e_http instead of a raw httpx client
* test(e2e): event_loop_stalls charges a hold to garbage collection when the gateway's debug-mode gc log accounts for it
The proxy registers a gc.callbacks hook only when the event loop runs in debug mode and logs every
collection over the loop's slow-callback threshold (generation, collected count, seconds). The case
parser pairs such a pass with the hold line that follows it and reports the hold under a bounded
garbage collection bucket instead of the task asyncio happened to blame, which made 7 of 10 local
runs fail on a different task name each time. Holds the collector does not account for keep the
strict allowlist, which gains the prisma first-query warm-up and the second pub/sub subscriber
* test(e2e): event_loop_stalls allowlist caps set at twice the ten-run maximum
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): mixed_size_requests_2rps perf case (coding-agent bodies, JWT and key callers, two guardrails)
* test(e2e_harness): test_observe builds DbCounters with statements and expects the background Postgres line
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): attach perf case metrics and limits as JUnit properties
The perf case only printed its metrics table, so a green run left no numbers
behind. Each observed metric and its limit now land as <property> entries on
the test's JUnit testcase, named perf_<metric>[_p<percentile>] and
perf_<metric>[_p<percentile>]_limit, so project-releaser can collect them
* test(e2e): compare perf properties exactly instead of through pytest.approx
basedpyright flags pytest.approx as partially unknown in the e2e lint step,
so the harness test now uses metric values that are exact in binary floating
point
* test(e2e): rename the load suite to perf, one E2E_PERF_TESTS opt-in, and a mixed-provider Locust perf test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): drop the mixed-provider perf test, keep the rename and the E2E_PERF_TESTS consolidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: run every CircleCI check from the integration workflow
* test(ci): keep the startup test CircleCI-owned
* test(ui): find reset log rows by request id so concurrent workers cannot page them out
* test(ui): retry the key budget window spec until the draft refresh fix lands
* ci(ui): set Playwright workers and retries on the CircleCI command only
The shared playwright.config.ts also drives the Buildkite UI suite, where 3 workers
ran out of memory on one of two runs. Buildkite keeps main's 1 worker and 2 retries
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* ci: restructure lint, unit and smoke workflows
* ci: preserve source formatting in tier workflows
* ci: retire per-shard coverage flags and tighten the tier workflow guards
* test(ci): freeze the workflow startup safety models
* ci: keep the current required check names running until the ruleset moves to the tier collectors
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test(e2e): move the harness self-tests out of tests/e2e
The nightly Buildkite run copies tests/e2e into the runner image and runs
bare pytest, so the 672 tests of the harness itself (fixture parsing, JUnit
properties, the stack lock, the load aggregators, the Claude Code driver)
counted as e2e tests on the status page even though none of them reaches a
proxy. They now live in tests/e2e_harness, mirroring the tests/e2e layout,
and run in the GitHub Actions lint job and the CircleCI
provider_replay_harness job instead
* fix(ci): point the providers replay controls at tests/e2e_harness
The providers integration job still selected the four replay-control
tests under tests/e2e/test_provider_edge.py, so pytest exited before
they ran. The raw-HTTP check's file walk also drops to one loop per
comprehension
* style(tests): mark the raw-HTTP check's bindings Final
