3069a467d0
* docs(agents): ban semicolons and colons in human-facing text Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "." * docs(agents): rewrite semicolons in nested AGENTS.md files * docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines * docs(agents): point at the PR template path that exists * docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
2 KiB
Tests
Nothing on the other side of the call: tests/unit. A proxy we start with an upstream we script:
tests/integration. Someone else's service with real credentials: tests/e2e. Tests of that harness
itself, no proxy at all: tests/e2e_harness. Two fit, split it
What good looks like
Red when the claim in the name is broken. Prove it: mutate the behaviour, red. Restore, green. Put the mutation in the PR body
def test_custom_price_is_reported_and_charged(gateway: Gateway) -> None:
with gateway.scenario() as scenario:
model = scenario.model(input_cost_per_token=0.001, output_cost_per_token=0.002)
response = gateway.request("POST", "/v1/chat/completions", {"model": model, "messages": [{"role": "user", "content": "price control"}]})
assert response.status_code == 200, response.text
assert float(response.headers["x-litellm-response-cost"]) == pytest.approx(20 * 0.001 + 20 * 0.002)
Rates in the test, expected computed by hand, one call, response.text in the assert
Assert the whole value. Iterating expected_body.items() (test_responses_api_request_body.py) cannot
see an extra key. That is the shape of stream_options.include_usage (#19777, #28553)
The linter catches no-assert, mock-echo and credential skips. It cannot see an assert
behind an if (a poll that ends in pytest.fail is fine), except Exception around the call
(test_router.py: except Exception as e: print(f"FAILED TEST")), or blanket --reruns
Where it goes
What the assertion depends on goes in the test. Everything else in conftest. A rate in a fixture three directories up makes a failed assertion unreadable. Extend the file that already covers the behaviour
Writing it so a human can read it
Name says what broke: test_send_batched_with_valid_data says nothing. Build, one call, assert, on one
screen. Helpers named for what they return, _pii_prompt(marker, email), not _setup(). Context in the
assert message, not a comment