litellm/tests/e2e
mubashir1osmani 1bf98c0687
test(e2e): cover Langfuse logging.yaml P0 logs_spend cells (#32857)
* test(e2e): cover Langfuse logging.yaml P0 logs_spend cells

Team, user/key, and org-scoped dynamic Langfuse callbacks drive real chat
traffic and assert calculatedTotalCost matches StandardLogging response_cost
and proxy spend. Also assert tool calls and applied guardrails land on the
trace. Missing env or proxy is a hard failure, never a skip

* test(e2e): use langfuse_otel callback for Langfuse spend coverage

Team and key dynamic logging attach callback_name=langfuse_otel (OTLP to
Langfuse) instead of the classic langfuse SDK. Match generations named
litellm_request by prompt marker and user_api_key_alias

* test(e2e): require Langfuse spend assert; drop AGENTS.md

Guardrail path no longer soft-gates logs_spend. Non-stream responses must
return positive x-litellm-response-cost; remove tests/e2e/AGENTS.md

* test(e2e): fail when Langfuse spend is missing on guardrail path

Always run logs_spend assertions for tool_permission; require positive
x-litellm-response-cost on non-stream and positive /spend/logs spend

* test(e2e): do not fall back to unmatched spend log rows

poll_proxy_spend_for_key returns None when response_id or positive-spend
filters match nothing, instead of silently using rows[0]
2026-07-11 14:11:00 -04:00
..
access_control test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
batches fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
budgets fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
coverage_registry test: emit e2e coverage lines for loki (#32513) 2026-07-08 11:06:56 -07:00
llm_translation fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
logging test(e2e): cover Langfuse logging.yaml P0 logs_spend cells (#32857) 2026-07-11 14:11:00 -04:00
management fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
spend_tracking test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
bob_the_builder.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
CLAUDE.md test: emit e2e coverage lines for loki (#32513) 2026-07-08 11:06:56 -07:00
conftest.py test(e2e): cover Langfuse logging.yaml P0 logs_spend cells (#32857) 2026-07-11 14:11:00 -04:00
CONTRIBUTING.md fix(ui): scope key models dropdown options to the key's team (#32382) 2026-07-07 18:54:19 -07:00
docker-compose.yml fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
e2e_config.py fix(ui): scope key models dropdown options to the key's team (#32382) 2026-07-07 18:54:19 -07:00
e2e_gateway.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
e2e_http.py test(e2e): cover Langfuse logging.yaml P0 logs_spend cells (#32857) 2026-07-11 14:11:00 -04:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py test(e2e): cover Langfuse logging.yaml P0 logs_spend cells (#32857) 2026-07-11 14:11:00 -04:00
pytest.ini test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_e2e_gateway.py test(e2e): make dynamic model provisioning robust on split deployments (#32670) 2026-07-09 13:39:47 -07:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00