litellm/tests/e2e
yuneng-jiang 9b2fc9db40
test: remove checks that pin implementation details and add a shadcn testing rule (#48663)
* test: remove e2e checks and a CI line count that pin implementation details

Deletes the Auto Router template select anchoring spec, the 390px logs pagination variant, and the call id tooltip and clipboard step in the logs spec. Each one tests library or test helper behavior rather than what users see, and each has been red or flaky on main without a product regression

Also drops the exact line count on the fan-in requirements export in the integration coverage job. It went from 6 to 10 when uv.lock gained a Python 3.15 fork, while the install was still correct. A package the filter drops still fails the job, either at the hash-checked install or at the next step's import

* docs(agents): do not test what shadcn and Base UI already guarantee

Adds the boundary to the dashboard AGENTS.md next to the test tiers, and points the Playwright suite's entry in tests/e2e/AGENTS.md at it
2026-10-27 22:26:15 +00:00
..
a2a test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
access_control fix(proxy/auth): expand all-team-models sentinel for project model access checks (#43734) 2026-10-10 15:49:27 -07:00
batches test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
claude_code test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
coverage_registry refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
gateway test(perf): add chat_completions_2rps, responses_2rps and messages_2rps direct vs bridge cases (#47752) 2026-10-21 02:32:03 +00:00
guardrails test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
llm_translation fix(completion): send prompt_cache_retention to OpenAI instead of silently dropping it (#47480) 2026-10-20 09:41:22 -07:00
logging test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
management test(e2e): repair fallback management test for config-owned fallbacks (#47023) 2026-10-17 01:27:39 +00:00
mcp test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
migrations fix(proxy-extras): hold only Prisma's lock key during migrate deploy so it never waits on the request log index build (#46660) 2026-10-15 14:47:33 -07:00
other fix(proxy): enforce the personal max_budget on proxy-admin JWTs (#43070) 2026-10-12 16:04:12 -07:00
perf test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
quota_management ci: run every CircleCI check from the integration workflow (#45938) 2026-10-11 07:39:16 -07:00
router test(e2e): hold failing-request RSS to a loose ceiling over two measured batches (#47170) 2026-10-17 14:05:42 -07:00
secret_manager test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
ui test: remove checks that pin implementation details and add a shadcn testing rule (#48663) 2026-10-27 22:26:15 +00:00
AGENTS.md test: remove checks that pin implementation details and add a shadcn testing rule (#48663) 2026-10-27 22:26:15 +00:00
conftest.py test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
CONTRIBUTING.md test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
e2e_config.py test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): get_text_external accepts JSON next to text so a pod-log error reads as the Kubernetes Status, not a 406 (#47150) 2026-10-17 13:25:29 -07:00
e2e_metadata.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
fixture_bundle.py test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
fixture_canonical.py feat(e2e): key the provider cache per test and mount Bedrock behind it 2026-09-16 02:15:46 -07:00
fixture_mode.py fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
fixture_profile.py test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
idp.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
idp_realm.json test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
junit_properties.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
lifecycle.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
memory_readings.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
models.py fix(completion): send prompt_cache_retention to OpenAI instead of silently dropping it (#47480) 2026-10-20 09:41:22 -07:00
otel_client.py test(e2e): record steps for raw transport calls and poll helpers (#45150) 2026-10-07 15:21:26 -07:00
PROVIDER_CACHE.md test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
provider_cache.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
provider_cache_redis.py chore(e2e): report the key components behind a mount that never converges 2026-09-16 15:01:08 -07:00
provider_cache_routing.py revert(e2e): unmount Gemini, its api_base means two things 2026-09-16 08:37:25 -07:00
provider_edge.py test(e2e): record steps for raw transport calls and poll helpers (#45150) 2026-10-07 15:21:26 -07:00
provider_edge_bedrock.py fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
proxy_client.py feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
pytest.ini test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
stack_lock.py test(e2e): run the memory cell alone on the shared stack (#42518) 2026-09-22 15:13:37 -07:00
transport.py test(e2e): record steps for raw transport calls and poll helpers (#45150) 2026-10-07 15:21:26 -07:00