litellm/tests/e2e
ryan-crabbe-berri 048a1500df
test(e2e): move the harness self-tests out of tests/e2e (#45172)
* test(e2e): move the harness self-tests out of tests/e2e

The nightly Buildkite run copies tests/e2e into the runner image and runs
bare pytest, so the 672 tests of the harness itself (fixture parsing, JUnit
properties, the stack lock, the load aggregators, the Claude Code driver)
counted as e2e tests on the status page even though none of them reaches a
proxy. They now live in tests/e2e_harness, mirroring the tests/e2e layout,
and run in the GitHub Actions lint job and the CircleCI
provider_replay_harness job instead

* fix(ci): point the providers replay controls at tests/e2e_harness

The providers integration job still selected the four replay-control
tests under tests/e2e/test_provider_edge.py, so pytest exited before
they ran. The raw-HTTP check's file walk also drops to one loop per
comprehension

* style(tests): mark the raw-HTTP check's bindings Final
2026-10-07 22:20:02 +00:00
..
a2a test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
access_control test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
batches test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
claude_code test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
coverage_registry test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
gateway test(e2e): add secret manager lanes for HashiCorp Vault and CyberArk Conjur (#42503) 2026-09-22 18:02:32 -07:00
guardrails test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
llm_translation test(e2e): cover ollama and ollama_chat on chat completions, responses and messages (#45079) 2026-10-07 20:17:35 +00:00
load test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
logging test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
management test(e2e): tag management tests with Subject metadata and record management client steps (#44962) 2026-10-07 10:32:09 -07:00
mcp test(e2e): tag router, batches and mcp tests with Subject metadata and record client steps (#44964) 2026-10-07 10:32:58 -07:00
migrations test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
other test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
quota_management test(e2e): tag the remaining quota_management tests and record budget and spend client steps (#44966) 2026-10-07 10:33:12 -07:00
router test(e2e): tag router, batches and mcp tests with Subject metadata and record client steps (#44964) 2026-10-07 10:32:58 -07:00
secret_manager test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
ui feat(ui): configure Anthropic workload identity federation from the dashboard (#44889) 2026-10-07 13:38:58 -07:00
AGENTS.md test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
conftest.py feat(jwt): auto_register_map_existing_key maps JWT to the user's existing virtual key (#42375) 2026-10-02 12:11:33 -07:00
CONTRIBUTING.md test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
e2e_config.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py feat(vertex): native batch JSONL passthrough with cost tracking (#42810) 2026-09-24 12:35:34 -07:00
e2e_metadata.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
fixture_bundle.py test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
fixture_canonical.py feat(e2e): key the provider cache per test and mount Bedrock behind it 2026-09-16 02:15:46 -07:00
fixture_mode.py fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
fixture_profile.py test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
idp.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
idp_realm.json test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
junit_properties.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
lifecycle.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
memory_readings.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
models.py test(e2e): cover ollama and ollama_chat on chat completions, responses and messages (#45079) 2026-10-07 20:17:35 +00:00
otel_client.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
PROVIDER_CACHE.md test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
provider_cache.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
provider_cache_redis.py chore(e2e): report the key components behind a mount that never converges 2026-09-16 15:01:08 -07:00
provider_cache_routing.py revert(e2e): unmount Gemini, its api_base means two things 2026-09-16 08:37:25 -07:00
provider_edge.py fix(proxy): send a real error event when a /v1/messages stream fails (#41826) 2026-09-25 15:36:35 -07:00
provider_edge_bedrock.py feat(e2e): cache the responses and embeddings endpoints behind the edge 2026-09-16 02:34:09 -07:00
proxy_client.py feat(e2e): record each e2e test's steps, starting with ProxyClient (#42393) 2026-09-30 19:33:53 -07:00
pytest.ini feat(jwt): auto_register_map_existing_key maps JWT to the user's existing virtual key (#42375) 2026-10-02 12:11:33 -07:00
stack_lock.py test(e2e): run the memory cell alone on the shared stack (#42518) 2026-09-22 15:13:37 -07:00
transport.py fix(e2e): route credential, cost map, and UI login calls to the control plane (#42506) 2026-09-22 12:59:56 -07:00