litellm/tests/integration/AGENTS.md

tests/integration

Real proxy, Postgres, Redis, scripted upstream. README.md has shards and CI wiring

What good looks like

The root example is from here. The spend row lands async: poll, never sleep

rows = eventually(
    lambda: read_rows('SELECT spend FROM "LiteLLM_SpendLogs" WHERE request_id=%s', (body["id"],)),
    lambda values: len(values) == 1,
    seconds=70,
)
assert float(rows[0]["spend"]) == pytest.approx(10 * 0.001 + 5 * 0.0001 + 7 * 0.002 + 4 * 0.002)

sleep(3) fails on a slow runner and taxes every fast one. Assert the outbound body in the upstream handler. A leaked field is invisible from the response. monkeypatch.setenv is fine. Patching our own function in a full stack is not

Where it goes

By the domain a user would name: pricing, spend, routing, mcp. A file only needs to live in a directory that a GROUPS entry in run.py selects. There is no manifest and no covers marker on new tests. A product bug the test exposes is pytest.skip("BUG: <symptom>") at the top of the body, not a fix in the test and not a deletion. Needs no network, proxy, DB or Redis: tests/unit

GitHub Actions integration groups run every file here except the OCI and sandbox tests, which no CI runs, and the files in GITHUB_FILES in run.py. test-integration.yml runs the same groups through the same run_integration.sh on GitHub Actions, on main every three hours from 8am to 8pm Pacific Monday to Saturday, and on a pull request when someone adds the run-ci label. These jobs do not run every pull request, so a new file here first runs on main after its PR merges, or earlier if the PR gets run-ci. Wanting a file to run before merge is no reason to change CI, and a new dedicated workflow for it is the same mistake as a shared shard. Write the file, leave GITHUB_FILES and every workflow alone, and tell the user it first runs on main. GITHUB_FILES is only for a file that needs what GitHub Actions groups cannot give it, like the images image-scan.yml builds. test_roi_observed.py previously needed a dedicated plain pytest job and now has no CI owner. It is listed explicitly in the coverage allowlist. Add a file to GITHUB_FILES only when needed, together with its dedicated job, and list a new dedicated GitHub workflow with the directory it owns in DEDICATED_INTEGRATION_WORKFLOWS in .github/scripts/assert_ci_coverage.py. The ci-coverage check fails a file here that another workflow selects unless that workflow owns its path in DEDICATED_INTEGRATION_WORKFLOWS (#46780).

Fixtures that boot a proxy

Scope them to the module or class, or pin a session one to a single worker with an xdist_group mark as _llm_gateway in mcp/test_mcp_llm_endpoints.py does, and request them as arguments, never through request.getfixturevalue. The harness puts tests that request the same set of module or class fixtures on one xdist worker, so the proxy they share boots there once and stops when that worker moves to another file. A test that requests one more of those fixtures than its neighbors gets its own group and can boot the proxy again on another worker, so keep that set the same across the tests that share a proxy. An unpinned session fixture boots a copy on every worker that runs one of its tests and keeps them all until the node ends, and those copies ran GitHub Actions integration-extensions nodes out of memory