litellm/tests/e2e
Kerry Lu a6245c1bbb
test(perf): measure SpendLog-backed UI query latency (#46955)
* test(perf): measure SpendLog-backed UI query latency

* test(perf): validate every numeric row and both counter snapshots

* test(perf): isolate read API cases and report request metrics

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-17 19:20:14 -07:00
..
a2a test(e2e): tag a2a, access_control, other, secret_manager and migrations tests with Subject metadata (#44965) 2026-10-07 12:11:55 -07:00
access_control fix(proxy/auth): expand all-team-models sentinel for project model access checks (#43734) 2026-10-10 15:49:27 -07:00
batches test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
claude_code test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
coverage_registry test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
gateway test(e2e): enable prometheus_system on the redis chaos proxy config (#47160) 2026-10-17 13:51:38 -07:00
guardrails test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
llm_translation feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
logging test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
management test(e2e): repair fallback management test for config-owned fallbacks (#47023) 2026-10-17 01:27:39 +00:00
mcp test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
migrations fix(proxy-extras): hold only Prisma's lock key during migrate deploy so it never waits on the request log index build (#46660) 2026-10-15 14:47:33 -07:00
other fix(proxy): enforce the personal max_budget on proxy-admin JWTs (#43070) 2026-10-12 16:04:12 -07:00
perf test(perf): measure SpendLog-backed UI query latency (#46955) 2026-10-17 19:20:14 -07:00
quota_management ci: run every CircleCI check from the integration workflow (#45938) 2026-10-11 07:39:16 -07:00
router test(e2e): hold failing-request RSS to a loose ceiling over two measured batches (#47170) 2026-10-17 14:05:42 -07:00
secret_manager test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
ui fix(ui): preserve key settings drafts during background refreshes (#41350) 2026-10-16 17:20:46 -07:00
AGENTS.md test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
conftest.py test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
CONTRIBUTING.md test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
e2e_config.py test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): get_text_external accepts JSON next to text so a pod-log error reads as the Kubernetes Status, not a 406 (#47150) 2026-10-17 13:25:29 -07:00
e2e_metadata.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
fixture_bundle.py test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
fixture_canonical.py feat(e2e): key the provider cache per test and mount Bedrock behind it 2026-09-16 02:15:46 -07:00
fixture_mode.py fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
fixture_profile.py test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
idp.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
idp_realm.json test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
junit_properties.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
lifecycle.py test(e2e): add enum values, auto-discovering label gates and secret hiding for e2e metadata (#44949) 2026-10-06 17:03:35 -07:00
memory_readings.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
models.py feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
otel_client.py test(e2e): record steps for raw transport calls and poll helpers (#45150) 2026-10-07 15:21:26 -07:00
PROVIDER_CACHE.md test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
provider_cache.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
provider_cache_redis.py chore(e2e): report the key components behind a mount that never converges 2026-09-16 15:01:08 -07:00
provider_cache_routing.py revert(e2e): unmount Gemini, its api_base means two things 2026-09-16 08:37:25 -07:00
provider_edge.py test(e2e): record steps for raw transport calls and poll helpers (#45150) 2026-10-07 15:21:26 -07:00
provider_edge_bedrock.py fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
proxy_client.py feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
pytest.ini test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
stack_lock.py test(e2e): run the memory cell alone on the shared stack (#42518) 2026-09-22 15:13:37 -07:00
transport.py test(e2e): record steps for raw transport calls and poll helpers (#45150) 2026-10-07 15:21:26 -07:00