Commit graph

5 commits

Author SHA1 Message Date
mateo-berri
468f74c628 ci(e2e): fix the stage-mirror batch reds and keep a redacted pytest log
The changed-test gate booted its stage-mirror stack without files_settings
or finetune_settings, so every raw upload with a custom_llm_provider hit a
500, and it exported the whole provider env into the gateways, so the
AWS_ROLE_NAME the assume-role test needs made the GovCloud deployment run
an AssumeRole with its static keys. The gate also deleted its pytest output,
so a red run left nothing to read. The mirror config now carries the
openai, azure, and vertex_ai file settings, gateways start without
AWS_ROLE_NAME, and the workflow uploads the pass logs and junit files with
every secret value, every field of a JSON-valued secret, and their
XML-escaped forms replaced before the raw files are removed.
2026-09-19 22:37:07 -07:00
Yuneng Jiang
77f406dc00
test(e2e): start persistent Keycloak in the changed-test runner 2026-09-11 17:09:35 -07:00
mateo-berri
65d8bbb8ac fix(e2e): wait for every gateway before using a new model and keep the network rerun
The changed-tests workflow overrode the suite's `--reruns 1` with `--reruns 0`, so a
transport blip failed a pass that pytest.ini already scopes to network errors and
5xx responses. Pass 2 of run 33692484803 also went red 15s after a model write with
"no healthy deployments": the barrier only polled /v1/models through nginx, which
proves one gateway converged, and the next request rolled the other. The stack now
exports LITELLM_PROXY_REPLICA_URLS, the barrier polls every replica with the full
budget before settling, and up.sh refuses to boot without DD_API_KEY, since the
gateway config enables the datadog callback on every run
2026-09-05 16:10:40 -07:00
Yuneng Jiang
0f59b6fb7a
ci(e2e): refine changed-test selection and runner lifecycle 2026-09-05 12:03:42 -07:00
Yuneng Jiang
4a646dd9a0
ci(e2e): run a PR's changed e2e tests three times behind a human-approved environment
Adds a required-check candidate that selects the tests/e2e test files a PR added or
modified, boots a stage-mirror stack on the runner (migrations, backend, two gateway
processes behind nginx, Postgres, Jaeger, TLS cluster Valkey), and runs those files
three times with retries off. The run job sits behind the e2e-changed GitHub
environment, so a reviewer approves each run before the OIDC token that reads the
provider keys from AWS Secrets Manager exists. Supersedes #34981
2026-09-02 14:53:40 -07:00