scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-18 04:44:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 123
litellm/tests/integration/AGENTS.md
Mateo Wang 57e69ce713
docs(agents): keep directory rules in nested AGENTS.md and pin integration tests to CircleCI (#46796) 
* docs(agents): keep directory rules in nested AGENTS.md and pin integration tests to CircleCI

* docs(agents): scope the CircleCI rule to the files it covers and ask agents to read nested AGENTS.md

* ci(coverage): reject GITHUB_FILES integration contracts run from a test-unit.yml shard

* ci(coverage): report integration ownership findings under their own header

* ci(coverage): flag GITHUB_FILES contracts run from any non-dedicated workflow

* docs(integration): say a pre-merge run is no reason to route an integration file to GitHub

* ci(coverage): tie each dedicated integration workflow to the directory it owns
2026-10-16 14:57:49 -07:00

2.2 KiB
Raw Blame History

tests/integration

Real proxy, Postgres, Redis, scripted upstream. README.md has shards and CI wiring

What good looks like

The root example is from here. The spend row lands async: poll, never sleep

rows = eventually(
    lambda: read_rows('SELECT spend FROM "LiteLLM_SpendLogs" WHERE request_id=%s', (body["id"],)),
    lambda values: len(values) == 1,
    seconds=70,
)
assert float(rows[0]["spend"]) == pytest.approx(10 * 0.001 + 5 * 0.0001 + 7 * 0.002 + 4 * 0.002)

sleep(3) fails on a slow runner and taxes every fast one. Assert the outbound body in the upstream handler. A leaked field is invisible from the response. monkeypatch.setenv is fine. Patching our own function in a full stack is not

Where it goes

By the domain a user would name: pricing, spend, routing, mcp. A file only needs to live in a directory that a GROUPS entry in run.py selects. There is no manifest and no covers marker on new tests. A product bug the test exposes is pytest.skip("BUG: <symptom>") at the top of the body, not a fix in the test and not a deletion. Needs no network, proxy, DB or Redis: tests/unit

CircleCI's integration groups run every file here except the OCI and sandbox tests, which no CI runs, and the files in GITHUB_FILES in run.py. CircleCI runs on main, not on pull requests, so a new file here first runs after its PR merges. Wanting a file to run before merge is no reason to change CI, and a new dedicated workflow for it is the same mistake as a shared shard. Write the file, leave GITHUB_FILES and every workflow alone, and tell the user it first runs on main. GITHUB_FILES is only for a file that needs what CircleCI's groups cannot give it, like the images image-scan.yml builds or the plain pytest run CircleCI's roi-database job gives test_roi_observed.py. Add a file there only in that case, together with its dedicated job, and list a new dedicated GitHub workflow with the directory it owns in DEDICATED_INTEGRATION_WORKFLOWS in .github/scripts/assert_ci_coverage.py. The ci-coverage check fails a file here that a GitHub workflow runs unless it is in GITHUB_FILES and that workflow owns its directory (#46780)