litellm/tests/e2e
mateo-berri 187cab5049 test(e2e): guard Azure reasoning_effort=none via base_model and xfail the dual token-param bug
Adds two customer-regression rows to the Azure chat e2e suite. The first
sends reasoning_effort=none to a custom-named deployment (gpt-5.6-sol-e2e,
swappable via E2E_AZURE_CUSTOM_MODEL) whose capabilities resolve through
base_model, asserting the request completes with zero reasoning tokens on
a prompt that reasons at default effort, so both a gate 400 (GH #31243,
SDK fix in PR #28490) and a silently dropped param fail the row. The
second, a strict xfail until GH #31614 is fixed, sends a client
max_completion_tokens to a gpt-4o deployment carrying a config-level
max_tokens default; the proxy forwards both and Azure rejects the pair
2026-07-16 15:41:00 -07:00
..
access_control test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
batches fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window (#33265) 2026-07-14 13:32:10 -07:00
claude_code test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
coverage_registry test(e2e): guard Azure reasoning_effort=none via base_model and xfail the dual token-param bug 2026-07-16 15:41:00 -07:00
llm_translation test(e2e): guard Azure reasoning_effort=none via base_model and xfail the dual token-param bug 2026-07-16 15:41:00 -07:00
logging test(e2e): failed request error span carries the full untruncated message and status (LIT-4179) (#33304) 2026-07-14 22:13:13 -07:00
management test(e2e): cover key rpm/tpm rate limiting, window reset, and pacing headers 2026-07-11 16:15:16 -07:00
quota_management refactor: make the code easier to read 2026-07-14 13:58:23 -07:00
bob_the_builder.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
CLAUDE.md test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
conftest.py refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
CONTRIBUTING.md ci: gate tests/e2e on zero basedpyright errors in pre-commit and lint CI 2026-07-11 10:25:22 -07:00
docker-compose.yml test(e2e): guard Azure reasoning_effort=none via base_model and xfail the dual token-param bug 2026-07-16 15:41:00 -07:00
e2e_config.py test(e2e): guard Azure reasoning_effort=none via base_model and xfail the dual token-param bug 2026-07-16 15:41:00 -07:00
e2e_gateway.py fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window (#33265) 2026-07-14 13:32:10 -07:00
e2e_http.py test(e2e): cover /chat/completions on Azure OpenAI, streaming and non-streaming 2026-07-15 16:11:45 -07:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py test(e2e): guard Azure reasoning_effort=none via base_model and xfail the dual token-param bug 2026-07-16 15:41:00 -07:00
pytest.ini refactor(e2e): move budgets and spend_tracking suites under quota_management 2026-07-11 16:14:21 -07:00
test_e2e_gateway.py fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window (#33265) 2026-07-14 13:32:10 -07:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00