litellm/tests/e2e
mubashir1osmani c4ed75206d test(e2e): guard that no provider logs a negative spend row
Add a live spend-tracking e2e that sweeps every chat provider the proxy
exposes (non-streaming and streaming) plus an embedding on one key, then
asserts every logged spend row is non-negative. A negative cost is a real
billing regression; cache-token accounting that lets a derived token count
fall below zero (BerriAI/litellm#25846) surfaces here as spend < 0, and the
streaming leg is exercised because that is the path the regression rode in on.
The per-provider positive-row check keeps the guard non-vacuous, so a pipeline
that silently logs 0 or drops the row fails instead of sliding past a bare
non-negative check.

Register the covering cell quota_management.spend_tracking.non_negative.
never_negative and note the test in the coverage matrix.
2026-07-14 16:03:19 -07:00
..
access_control test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
batches fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window 2026-07-14 12:56:35 -07:00
coverage_registry test(e2e): guard that no provider logs a negative spend row 2026-07-14 16:03:19 -07:00
llm_translation Merge pull request #32918 from BerriAI/litellm_e2e_basedpyright_gate 2026-07-11 12:01:50 -07:00
logging test(e2e): otel trace completeness on /v1/messages (#33133) 2026-07-13 20:19:06 -07:00
management ci: gate tests/e2e on zero basedpyright errors in pre-commit and lint CI 2026-07-11 10:25:22 -07:00
quota_management test(e2e): guard that no provider logs a negative spend row 2026-07-14 16:03:19 -07:00
bob_the_builder.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
CLAUDE.md refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
conftest.py refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
CONTRIBUTING.md ci: gate tests/e2e on zero basedpyright errors in pre-commit and lint CI 2026-07-11 10:25:22 -07:00
docker-compose.yml test(e2e): otel trace completeness on /chat/completions (#33132) 2026-07-13 19:06:12 -07:00
e2e_config.py test(e2e): otel trace completeness on /v1/messages (#33133) 2026-07-13 20:19:06 -07:00
e2e_gateway.py fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window 2026-07-14 12:56:35 -07:00
e2e_http.py test(e2e): cover Langfuse logging.yaml P0 logs_spend cells (#32857) 2026-07-11 14:11:00 -04:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window 2026-07-14 12:56:35 -07:00
pytest.ini refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
test_e2e_gateway.py fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window 2026-07-14 12:56:35 -07:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00