litellm/tests/e2e
Mubashir Osmani d0db267f40 test(e2e): add datadog log-delivery e2e scenarios
Drive real traffic through the proxy and read the events back out of the
Datadog Logs Search API to prove delivery instead of trusting the proxy. The
DatadogClient reads DD_API_KEY, DD_SITE and DD_APP_KEY from the environment and
polls the search API for a per-run marker stamped into each request

Three scenarios cover the contract: a batch of gpt-5.5 chat completions shipped
with no drops, the /v1/responses path on a claude-haiku-4-5 deployment
exercising the Responses-to-Anthropic translation, and a failed request that
must land as an error event. The e2e compose config enables the datadog (and
prometheus) callback so the delivery path is actually exercised, and the suite
skips unless all three credentials are set
2026-07-07 22:32:41 -04:00
..
access_control test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
batches Merge pull request #32166 from BerriAI/litellm_e2e_batches_ocr_model_registration 2026-07-05 02:15:10 +00:00
budgets feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
coverage_registry test(e2e): add coverage registry and collector (#32304) 2026-07-07 15:51:20 -04:00
llm_translation test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
logging test(e2e): add datadog log-delivery e2e scenarios 2026-07-07 22:32:41 -04:00
management test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
realtime test: add realtime proxy e2e suite across providers (#30960) 2026-06-26 09:36:49 -07:00
spend_tracking test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
CLAUDE.md test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
conftest.py test(e2e): add coverage registry and collector (#32304) 2026-07-07 15:51:20 -04:00
CONTRIBUTING.md chore(e2e): untrack gateway config and document e2e test location (#31914) 2026-07-02 19:22:02 -07:00
docker-compose.yml test(e2e): add datadog log-delivery e2e scenarios 2026-07-07 22:32:41 -04:00
e2e_config.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
e2e_gateway.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
e2e_http.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
pytest.ini test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_e2e_gateway.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00