scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 82
litellm/tests/e2e
History
Exact
mateo-berri 116f88b023 fix(e2e-changed): keep the gate off suites the stack cannot run 
The selector picked up two suites that can never pass in this stack, so
editing either one turned the check permanently red: the presidio masking
suite calls pytest.fail without an analyzer and anonymizer that up.sh
never starts, and the pipecat audio suite skips itself at import time
unless the NLTK punkt_tab data is present, which nothing installs.

tests/e2e/coverage_registry/test_collector.py had the same problem for a
different reason. Its nested pytest.main autoloads pytest-retry from the
ci group the workflow installs and dies with "INTERNALERROR: no option
named 'filtered_exceptions'", so the collect-only pass now disables that
plugin. The plugin's entry point is pytest-retry, not retry, so the same
one-word fix lands on mutmut's pytest_add_cli_args, where "-p no:retry"
was disabling nothing.

Two smaller holes in the harness: a canary argument the shell never
expanded used to select nothing and let the gate pass green, and a secret
that cannot be represented in both bash and dotenv was rejected without
naming the key.
2026-09-05 21:03:50 -07:00
..
a2a test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00
access_control test(e2e): require a 200 inside the regenerate grace window and drop the helper docstrings 2026-09-05 11:53:22 -07:00
batches test(e2e/batches): run the list assertion when a batch completes before cancel 2026-09-04 18:50:47 -07:00
claude_code fix(e2e): wait for every gateway before using a new model and keep the network rerun 2026-09-05 16:10:40 -07:00
coverage_registry fix(e2e-changed): keep the gate off suites the stack cannot run 2026-09-05 21:03:50 -07:00
gateway ci(e2e): declare the embedding model the access_control canary calls 2026-09-05 18:58:07 -07:00
guardrails test(e2e): cover presidio post_call, tool_permission, and weave logging cells (#39279) 2026-09-05 13:03:28 -07:00
llm_translation Merge pull request #39953 from BerriAI/litellm_/litellm-e2e-flaky-test-2159ae 2026-09-05 16:04:45 -07:00
load test(e2e): move load/perf testing out of the main suite and drop the vllm passthrough test (#35820) 2026-08-04 14:12:25 -07:00
logging test(e2e): cover presidio post_call, tool_permission, and weave logging cells (#39279) 2026-09-05 13:03:28 -07:00
management test(e2e): require a 200 inside the regenerate grace window and drop the helper docstrings 2026-09-05 11:53:22 -07:00
mcp test(e2e): drop coverage registry cell for the alias-grant test 2026-09-01 09:03:47 -07:00
other test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149) 2026-07-21 14:06:29 -07:00
quota_management test(e2e): cover model access group budgets against a live proxy 2026-08-29 15:46:49 -07:00
router test(e2e): repair two suites broken by intentional behaviour changes 2026-09-04 13:48:42 -07:00
ui test: repair two CI tests broken by intentional changes 2026-09-05 12:02:48 -07:00
CLAUDE.md test(e2e): judge /v1/messages streaming on the clock, not on the provider's delta count 2026-09-05 14:44:21 -07:00
conftest.py ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
CONTRIBUTING.md fix(e2e-changed): keep the gate off suites the stack cannot run 2026-09-05 21:03:50 -07:00
e2e_config.py chore: merge litellm_internal_staging into litellm_/e2e-test-performance-7d53be 2026-09-05 16:15:11 -07:00
e2e_db.py test(e2e): guard destructive spend-log truncate behind an explicit opt-in (#33751) 2026-07-20 08:47:39 -07:00
e2e_http.py test(e2e): drop the explanatory sentence from the StreamingResponse docstring 2026-09-05 14:53:12 -07:00
fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
fixture_canonical.py test(e2e): key multipart uploads by structured part identity 2026-08-21 19:32:02 -07:00
fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
junit_properties.py Refuse a source path carrying a colon 2026-09-01 16:02:54 -07:00
lifecycle.py test(e2e): pin openai_passthrough routing, cost logging, and file list isolation 2026-08-20 03:03:35 -07:00
models.py Merge pull request #39938 from BerriAI/litellm_e2e_vertex_cache_first_call 2026-09-05 15:10:15 -07:00
otel_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
provider_edge.py fix(e2e): record a streamed chunk only after its downstream write lands 2026-08-24 13:26:58 -07:00
proxy_client.py chore: merge litellm_internal_staging into litellm_/e2e-test-performance-7d53be 2026-09-05 16:15:11 -07:00
pytest.ini ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
test_e2e_http.py test(e2e): annotate new stream-timing locals as Final and trim the docstrings 2026-09-05 14:49:56 -07:00
test_fixture_bundle.py feat(e2e): record and replay streamed provider responses chunk-for-chunk 2026-08-24 12:51:44 -07:00
test_fixture_canonical.py test(e2e): pin query params and multipart form fields as replay match-key identity 2026-08-20 15:59:34 -04:00
test_fixture_mode.py feat(e2e): move record/replay to the provider edge (LIT-5745) 2026-08-19 18:39:15 -07:00
test_junit_properties.py test(e2e): read JUnit properties off the real collected pytest Item 2026-09-01 19:12:07 -07:00
test_provider_edge.py fix(e2e): record a streamed chunk only after its downstream write lands 2026-08-24 13:26:58 -07:00
test_proxy_client.py fix(e2e): wait for every gateway before using a new model and keep the network rerun 2026-09-05 16:10:40 -07:00
transport.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00