diff --git a/MUTATIONS.md b/MUTATIONS.md new file mode 100644 index 00000000000..b507e54ecc1 --- /dev/null +++ b/MUTATIONS.md @@ -0,0 +1,188 @@ +# Mutation evidence for litellm_sail_tests_integration + +Review evidence for the tests/integration cell of the PR 42840 regression sweep. Not for merge + +Tip: d7bc17fa47ee9acacd441e6b3349c7aa9e1c6c62 (litellm_add_sail_provider). Base: 0c1c3e18d5250ec3a0e1e3f287e0b93e2906d900 + +Every run below used the existing integration harness against a real proxy (`LITELLM_LOCAL_MODEL_COST_MAP=True`, +`STORE_MODEL_IN_DB=True`, `num_retries: 0`), Postgres, Redis and the scripted upstream from +`tests/integration/_support/upstream.py`. Two stacks ran side by side: tip (upstream 8190, proxy 4000, database +`integration_tip`) and base (upstream 8191, proxy 4001, database `integration_base`). The proxy was restarted after +every apply and after every restore so the mutated module was actually the one serving + +Runner used for every pytest invocation (paths shortened below as `run