litellm/tests/e2e
devin-ai-integration[bot] f7ae9efad2
test(e2e): hold every worker under an idle RSS budget before any traffic (#42552)
* test(e2e): hold every worker under an idle RSS budget before any traffic

The harness reads /debug/memory/summary on every replica once at collection
time, right after the readiness gate and before this pytest process sends any
traffic, and the memory suite's first test fails when any worker idles past
E2E_MEMORY_IDLE_RSS_BUDGET_MB (768 MB by default) or gives no reading at all.

A v1.100.x worker with a database idled at 836-886 MB where v1.101.0rc1 idled
at 544 MB on the same database: prisma-client-py's default recursive type
depth generated 91k TypedDict classes that v1.101.0's recursive_type_depth = -1
cut to 19k. The budget starts at the rc1 reading plus headroom.

* test(e2e): read idle RSS only when the idle budget test is selected

Gate the collection-time /debug/memory/summary read on a selected test using
the idle_rss fixture and skip it under --collect-only, so sessions that never
run the idle budget test pay no round trip. Drop the markerless unit test file
the e2e guide bans and assert live that every configured replica was measured

* test(e2e): take the idle RSS read after collection settles

Read every replica's RSS from a tryfirst pytest_collection_finish hook so -k
and -m deselection has already run, and only when a selected test still asks
for the idle_rss fixture and the run is not --collect-only

* test(e2e): record the heaviest idle RSS reading as junit properties

* test(e2e): attach the idle RSS properties from the harness's setup hook

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 14:47:14 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control Merge remote-tracking branch 'github/main' into litellm_rust_bridge_declarative_route_catalog 2026-09-17 11:08:08 -07:00
batches fix(bedrock): sign batch S3 requests with s3_access_key_id and s3_secret_access_key (#42342) 2026-09-21 19:01:18 -07:00
claude_code chore: consolidate CLAUDE.md into AGENTS.md 2026-09-19 02:30:35 +00:00
coverage_registry test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
gateway test(e2e): one request lands the same spend on every surface (#42540) 2026-09-22 13:42:00 -07:00
guardrails feat(proxy): opt-in include_guardrail_response returns guardrail_information in the response (#42327) 2026-09-22 12:43:30 -07:00
llm_translation fix(e2e): route credential, cost map, and UI login calls to the control plane (#42506) 2026-09-22 12:59:56 -07:00
load chore: merge litellm_internal_staging into litellm_e2e_reliability_module_cells 2026-09-12 13:05:02 -07:00
logging fix(otel): honor SSL_CERT_FILE and ssl_verify in OTLP HTTP exporters (#42106) 2026-09-22 20:50:53 +00:00
management fix(proxy): write key deleted audit logs for cascade and alias key deletions (#42446) 2026-09-22 11:52:08 -07:00
mcp test(e2e): restore LIT-3467 implementation for rework 2026-09-19 16:21:53 -07:00
migrations test(migrations): close the gaps the upgrade assertions left open 2026-09-21 13:15:46 -07:00
other fix(jwt): accept a team alias in x-litellm-team-id (#42445) 2026-09-22 02:22:00 -07:00
quota_management fix(proxy): attribute provider and model_info on pre_call_hook rejections (#41077) 2026-09-22 14:14:13 -07:00
router test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
ui test(e2e-ui): check the MCP Tools tab against the upstream's own tools/list (#42397) 2026-09-21 20:35:30 -07:00
AGENTS.md test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
conftest.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
CONTRIBUTING.md fix(otel v2): map completions, images, speech, transcription and moderation output onto the Langfuse generation output (#42394) 2026-09-21 21:57:44 -07:00
e2e_config.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py fix(websearch_interception): keep intercepted searches under the parent request's session and trace (#41711) 2026-09-22 12:35:27 -07:00
fixture_bundle.py test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
fixture_canonical.py feat(e2e): key the provider cache per test and mount Bedrock behind it 2026-09-16 02:15:46 -07:00
fixture_mode.py fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
fixture_profile.py test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
idp.py test(e2e): restore LIT-3467 implementation for rework 2026-09-19 16:21:53 -07:00
idp_realm.json test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
junit_properties.py test: bind management E2E callers and isolate JWT actors 2026-09-12 13:29:04 -07:00
lifecycle.py fix(e2e): clean up batch files reliably and expire Azure inputs 2026-09-07 12:15:06 -07:00
memory_readings.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
models.py fix(proxy): attribute provider and model_info on pre_call_hook rejections (#41077) 2026-09-22 14:14:13 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
PROVIDER_CACHE.md fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
provider_cache.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
provider_cache_redis.py chore(e2e): report the key components behind a mount that never converges 2026-09-16 15:01:08 -07:00
provider_cache_routing.py revert(e2e): unmount Gemini, its api_base means two things 2026-09-16 08:37:25 -07:00
provider_edge.py fix(responses): forward safety_identifier through the chat completion bridge 2026-09-21 04:09:59 +00:00
provider_edge_bedrock.py feat(e2e): cache the responses and embeddings endpoints behind the edge 2026-09-16 02:34:09 -07:00
proxy_client.py test(e2e): hold every worker under an idle RSS budget before any traffic (#42552) 2026-09-22 14:47:14 -07:00
pytest.ini fix(otel): honor SSL_CERT_FILE and ssl_verify in OTLP HTTP exporters (#42106) 2026-09-22 20:50:53 +00:00
test_e2e_http.py test: bind management E2E callers and isolate JWT actors 2026-09-12 13:29:04 -07:00
test_fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_idp.py test: enforce isolated actors and stop OIDC process groups 2026-09-12 13:49:49 -07:00
test_junit_properties.py test(e2e): read JUnit properties off the real collected pytest Item 2026-09-01 19:12:07 -07:00
test_provider_edge.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
test_proxy_client.py test(e2e): assert a cooldown reaches a sibling replica within the 1s Redis read interval (#42422) 2026-09-22 12:42:41 -07:00
transport.py fix(e2e): route credential, cost map, and UI login calls to the control plane (#42506) 2026-09-22 12:59:56 -07:00