scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

fix(ci): unblock the mutation run's stats phase

Browse source 
With the coverage fix in place the run generates mutants, then dies before
testing any of them: "1 failed ... failed to collect stats. runner returned 1".

The offender is one test. google_login is called inside a bare
`except Exception: pass` and the assertion then reads the mock's call_args, so
an early raise inside mutmut's mutants/ sandbox surfaces as
"'NoneType' object has no attribute 'kwargs'" rather than as the real error.

Deselected rather than ignored, so the other 248 tests in test_ui_sso.py still
contribute to the score.

This is measured rather than guessed. mutmut's stats phase hardcodes -x, so a
failing run only ever names its first offender, which is why deselecting looked
like whack-a-mole before. pytest_add_cli_args is appended after -x, and a later
--maxfail wins, so overriding it once let the whole folder run inside the
sandbox: 1 failed, 2901 passed. That one test is the only one that cannot run
there.

What is still not known is why it raises early in the sandbox. It is not the
suite and not the copied tree: the same folder passes outside mutants/ on the
runner image (2930), passes on a copied tree put first on PYTHONPATH (2902),
and passes with and without the test_saml_sso.py ignore. What is left is
mutmut's trampolines.
This commit is contained in:
Yuneng Jiang 2026-08-26 05:10:43 -07:00
parent ac1eb1029a
commit 5bbbbff9dd
No known key found for this signature in database
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
pyproject.toml
View file

@ -369,11 +369,21 @@ mutate_only_covered_lines = true
# rejects the SHA256 instance the fixture builds with "Algorithm must be a
# registered hash algorithm". Nothing to do with mutation coverage, and one
# erroring test is enough to end the stats phase before any mutant runs.
# test_google_login_only_threads_user_code_when_enabled is the same story, one
# test rather than a whole file. It calls google_login inside a bare
# `except Exception: pass` and then reads a mock's call_args, so any early raise
# inside the sandbox surfaces as `'NoneType' object has no attribute 'kwargs'`.
# Deselected rather than ignored so the rest of test_ui_sso.py still counts.
# Measured, not guessed: running the folder in the sandbox with the -x of the
# stats phase overridden gives 1 failed, 2901 passed, so this is the only test
# that cannot run there.
pytest_add_cli_args = [
"-p", "no:retry",
"-p", "no:rerunfailures",
"-p", "no:xdist",
"--ignore=tests/test_litellm/proxy/management_endpoints/test_saml_sso.py",
"--deselect",
"tests/test_litellm/proxy/management_endpoints/test_ui_sso.py::TestCLIKeyRegenerationFlow::test_google_login_only_threads_user_code_when_enabled",
]
[tool.coverage.run]