litellm/tests/integration/AGENTS.md
yuneng-jiang 9b53ee878f
perf(ci): isolate and shard integration suites (#44146)
* perf(ci): distribute unit cases and reuse security sweep connections

* revert(ci): reject first speed trial after new failures

* perf(ci): avoid repeated test discovery and parsing

* perf(ci): isolate parallel security workers

* revert(ci): restore measured security execution isolation

* perf(tests): reuse source scans and deduplicate cache cleanup

* perf(ci): restore parallel security with safe log batching

* test(ci): isolate fixture state and schema comparisons

* test(ci): make schema and scheduler fixtures deterministic

* test(ci): drain queued logging before fixture teardown

* test(ci): preserve logging delivery without idle worker restarts

* test(ci): keep guardrail sinks available during parallel bursts

* test(ci): isolate full proxy lifecycle from gateway imports

* test(ci): preserve first failure responses and sweep evidence

* test(ci): restore documentation URL settings between cases

* test(ci): isolate open MCP tools in private gateways

* test(ci): group private MCP cases and handle exited workers

* test(ci): retain proxy exits and MCP failure details

* test(ci): repair merge interactions with main

Key the schema shard's full clone on artifact-name, since main removed the
unit-flag input. Give each owned Redis its own log file again. Hand the
realtime transcription tests main added a logging object whose dispatch
handlers are awaitable, so the per-test logging drain no longer waits on a
MagicMock task forever

* perf(ci): shard owned integration cases and close lifecycle leaks

* fix(ci): qualify shard execution and harden interrupted cleanup

* fix(ci): bound worker supervision and own fault traffic

* perf(ci): reuse unparametrized fixtures across bounded case groups

* fix(ci): preserve boot fault allowances and complete exit records

* test(ci): tolerate startup retries and retain readiness details

* perf(ci): balance integration shards by recorded duration

* fix(ci): load migration database helpers only for workers
2026-10-10 11:15:19 -07:00

1.1 KiB

tests/integration

Real proxy, Postgres, Redis, scripted upstream. README.md has shards and CI wiring

What good looks like

The root example is from here. The spend row lands async: poll, never sleep

rows = eventually(
    lambda: read_rows('SELECT spend FROM "LiteLLM_SpendLogs" WHERE request_id=%s', (body["id"],)),
    lambda values: len(values) == 1,
    seconds=70,
)
assert float(rows[0]["spend"]) == pytest.approx(10 * 0.001 + 5 * 0.0001 + 7 * 0.002 + 4 * 0.002)

sleep(3) fails on a slow runner and taxes every fast one. Assert the outbound body in the upstream handler. A leaked field is invisible from the response. monkeypatch.setenv is fine. Patching our own function in a full stack is not

Where it goes

By the domain a user would name: pricing, spend, routing, mcp. A file only needs to live in a directory that a GROUPS entry in run.py selects. There is no manifest and no covers marker on new tests. A product bug the test exposes is pytest.skip("BUG: <symptom>") at the top of the body, not a fix in the test and not a deletion. Needs no network, proxy, DB or Redis: tests/unit