Commit graph

9 commits

Author SHA1 Message Date
mateo-berri
a9e918577b ci(e2e): run the access_control canary on harness changes and name failed tests
A harness-only change (proxy_client.py, conftest.py, pytest.ini, the gateway
config, .github/e2e-stack, or the workflow) selected nothing, so the stack was
never exercised by the change that touched it. select_tests.py keeps the
changed-file rule and adds the access_control suite whenever a harness file
changes. The run step now reports the pytest exit code before the evidence
check, prints pytest's summary line per pass so the rerun count is visible,
and assert_tests_ran.py names each failed or errored test as classname::name
2026-09-05 18:46:51 -07:00
mateo-berri
65d8bbb8ac fix(e2e): wait for every gateway before using a new model and keep the network rerun
The changed-tests workflow overrode the suite's `--reruns 1` with `--reruns 0`, so a
transport blip failed a pass that pytest.ini already scopes to network errors and
5xx responses. Pass 2 of run 33692484803 also went red 15s after a model write with
"no healthy deployments": the barrier only polled /v1/models through nginx, which
proves one gateway converged, and the next request rolled the other. The stack now
exports LITELLM_PROXY_REPLICA_URLS, the barrier polls every replica with the full
budget before settling, and up.sh refuses to boot without DD_API_KEY, since the
gateway config enables the datadog callback on every run
2026-09-05 16:10:40 -07:00
Yuneng Jiang
0f59b6fb7a
ci(e2e): refine changed-test selection and runner lifecycle 2026-09-05 12:03:42 -07:00
Yuneng Jiang
a3ee6b2566
ci(e2e): fail a pass whose every collected test was skipped 2026-09-02 17:50:35 -07:00
Yuneng Jiang
1af9c229fa
ci(e2e): leave the managed-files opt-in file to its own lane instead of failing on an empty collection 2026-09-02 17:29:56 -07:00
Yuneng Jiang
c62643f0cf
ci(e2e): drop trailing newlines from fetched secret values before writing the env 2026-09-02 15:39:21 -07:00
Yuneng Jiang
743f94f82a
ci(e2e): fail when nothing collects and ignore lanes with their own checks in the smoke trigger 2026-09-02 14:59:15 -07:00
Yuneng Jiang
b348ed7f09
ci(e2e): only tail stack logs when the stack actually booted 2026-09-02 14:56:52 -07:00
Yuneng Jiang
4a646dd9a0
ci(e2e): run a PR's changed e2e tests three times behind a human-approved environment
Adds a required-check candidate that selects the tests/e2e test files a PR added or
modified, boots a stage-mirror stack on the runner (migrations, backend, two gateway
processes behind nginx, Postgres, Jaeger, TLS cluster Valkey), and runs those files
three times with retries off. The run job sits behind the e2e-changed GitHub
environment, so a reviewer approves each run before the OIDC token that reads the
provider keys from AWS Secrets Manager exists. Supersedes #34981
2026-09-02 14:53:40 -07:00