* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
A name bound twice keeps only the second binding. In `tests/` that is nearly
always a repeated import, harmless but misleading, and the same rule is what
catches the cases that are not harmless: a local that shadows an import the
module still calls, and a second `def test_x` that quietly replaces the first.
311 of the 344 sites were repeated imports and came out with ruff's own fix.
The remaining 33 needed a decision. Four modules imported a name they never
used because a local definition below already shadowed it. Two comprehensions
bound `call` over `unittest.mock.call`, which those modules import and use.
One test rebound the two module handles its nested reload closure had captured.
One class attribute shadowed an unused `status` import.
The load-test fixtures move to a conftest, which is how pytest is meant to share
them, so the test module no longer imports three fixture names it never calls.
The nine `prisma_client` parameters keep a narrow `noqa`: pytest resolves that
fixture by name before the body runs, so the parameter never shadows anything.
The revert of 9711e3adfe left xdist tests without proper state isolation.
Module-level assignments like `litellm.num_retries = 3` in 12+ test files
pollute shared globals, and the fixture was saving/restoring contaminated
values instead of resetting to true defaults.
- Capture true litellm defaults at conftest import time and reset before
each test (local_testing + llm_translation)
- Make llm_translation/conftest.py xdist-safe (skip reload under xdist,
add state isolation)
- Replace asyncio.sleep(2) with polling in cooldown handler tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The conftest fixtures were saving/restoring the current (potentially
contaminated) values of litellm globals like num_retries instead of
resetting to true defaults. Under xdist, module-level assignments
(e.g. `litellm.num_retries = 3` in 12+ test files) pollute the
shared module state and leak across tests in the same worker.
- Capture true litellm defaults at conftest import time and reset
before each test (local_testing + llm_translation)
- Make llm_translation/conftest.py xdist-safe (skip reload, add
state isolation)
- Replace asyncio.sleep(2) with polling in cooldown handler tests
- Add @pytest.mark.flaky to tests making real API calls under xdist
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test used fallbacks=[{"gpt-3.5-turbo": ["123"]}] where "123" is a
model_id, but the fallback mechanism treats values as model group names.
This caused a ValueError since no model group "123" exists. Additionally,
mock_response propagates to fallback calls, making mock-based fallback
tests unreliable.
Simplified the test to verify that a RateLimitError doesn't permanently
cool down a deployment for subsequent requests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_async_fallbacks, test_async_fallbacks_streaming, test_sync_fallbacks:
update previous_models assertion from 4 to 3 (fallback not counted)
- test_ausage_based_routing_fallbacks: update deprecated model
claude-3-5-haiku-20241022 to claude-haiku-4-5-20251001
- test_router_fallbacks_with_cooldowns_and_model_id: increase RPM from
1 to 2 so second request isn't blocked by RPM consumed during failed
first request
- test_sync_in_memory_spend_with_redis: add delay after constructing
RouterBudgetLimiting to let background init tasks complete before
overwriting Redis values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_router_cooldown_handlers: add mock_response to avoid real API call requiring OPENAI_API_KEY
- test_router_timeout: update deprecated claude-3-5-haiku-20241022 to claude-haiku-4-5
- test_router_fallbacks: relax assertion from == 4 to >= 3 to handle cooldown timing variance
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
