Add client cache flush before setting disable_aiohttp_transport to prevent
cache pollution. Previous tests may cache HTTP clients with aiohttp enabled,
which bypass respx mocks and cause real API calls to fail.
This follows the same pattern as test_openai_env_base which also uses respx.
Related to #8425
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Follow-up to PR #21248 addressing greptile code review feedback.
Removes hasattr checks for non-existent attributes that were identified
as dead code by greptile automated code review.
Changes:
- Remove hasattr check for LangFuseLogger._langfuse_clients (class attribute doesn't exist)
- Remove hasattr check for self.logger._langfuse_client_cache (instance attribute doesn't exist)
- Update comments to be more accurate about what cleanup is being done
The core fix from PR #21248 (nulling Langfuse reference and deleting
logger instance) remains unchanged and effective. This just removes
misleading dead code that serves no purpose.
Context:
These checks were added defensively but reference attributes that don't
actually exist on the LangFuseLogger class, making them always no-ops.
Greptile correctly identified these as dead code in PR #21248 review,
but the PR was merged before the cleanup could be applied.
Related: #21248
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes test isolation issue where test_video_content_handler_uses_get_for_openai
was making real HTTP requests to OpenAI API instead of using the mock client.
Changes:
- Patch _get_httpx_client to ensure it returns the mock client
- Prevents creation of real HTTP client even if isinstance check fails
- Wraps handler call in context manager for proper cleanup
Root Cause:
When run after other tests, the isinstance(mock_client, HTTPHandler) check
in video_content_handler() could fail due to state pollution, causing the
handler to create a real HTTP client via _get_httpx_client(). This resulted in:
- Real API calls to https://api.openai.com/v1/videos/video_abc/content
- 401 errors: "Incorrect API key provided: sk-test"
- Test expecting b'mp4-bytes' but getting actual error response
Impact:
Test passes in isolation but fails when run with other tests, especially
in CI environments with parallel execution.
Fixes: Test isolation for test_video_content_handler_uses_get_for_openai
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes test isolation issue where test_extra_body_with_fallback was setting
litellm.disable_aiohttp_transport = True but never resetting it, causing
state pollution that affected other tests.
Changes:
- Save original value of disable_aiohttp_transport before modifying
- Wrap test logic in try/finally block
- Restore original value in finally to ensure cleanup even on failure
Root Cause:
The test was modifying global litellm state without cleanup. When run in
certain orders with other tests:
- If this test ran first, it left disable_aiohttp_transport=True globally
- If other tests ran first, their state could interfere with this test
- Result: "All fallback attempts failed" error in CI
Impact:
Test passes in isolation but fails when run with other tests, especially
in parallel execution or CI environments.
Fixes: Test isolation for test_extra_body_with_fallback
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enhances test isolation in TestLangfuseUsageDetails by ensuring the
logger instance is completely fresh for each test and properly cleaned
up afterward.
Changes:
- Clear any class-level cached Langfuse clients before creating logger
- Reset logger's cached client instances in setUp
- Properly clean up logger instance and its state in tearDown
Root Cause:
The test_log_langfuse_v2_handles_null_usage_values test was failing
when run after other tests due to lingering state in the logger instance.
While the test passes in isolation, test ordering issues caused it to
fail with "Expected 'generation' to have been called once. Called 0 times."
This builds on PR #21214 which added sys.modules cleanup, but that wasn't
sufficient to prevent all state leakage between tests.
Fixes: Test isolation issues in test_log_langfuse_v2_handles_null_usage_values
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The test_end_to_end_rerank_flow mock for _ensure_access_token was not
being applied because conftest reloads litellm, causing the
VertexAIRerankConfig class to be a different object than what's patched.
Fix: Reload the transformation module in setup_method and re-import the
class to ensure the patch targets the same class object used by tests.
The tests were making real API calls instead of using mocks because
conftest.py reloads litellm at module scope, causing the HTTPHandler
class reference in the HuggingFace embedding handler to become stale.
The patches were applied to the new class, but the handler used the old one.
Fix: Add a reload_huggingface_modules fixture that reloads the relevant
modules BEFORE the mock fixtures apply their patches. This ensures all
references point to the same class object.
The test was making real API calls instead of using mocks because the
conftest.py reloads litellm at module scope, causing stale module
references. The mock was patching the old reference while the actual
code used the new one.
Fix: Reload litellm.containers.main inside the test to get a fresh
reference to base_llm_http_handler, then re-import create_container
after the reload.
The setup_and_teardown fixture was failing with "ImportError: module
litellm not in sys.modules" during parallel test execution. This occurs
because another worker might have removed/modified litellm from
sys.modules before this test tries to reload it.
Fix: Check if litellm is in sys.modules before attempting reload.
- Reload litellm_pre_call_utils module inside test to get fresh litellm reference
- Use string-based patch("litellm.model_group_settings") instead of patch.object
- These changes ensure the patch targets the correct module after conftest reloads litellm
Fix several tests that fail in CI due to parallel test execution and
module reloading in conftest.py.
1. test_empty_assistant_message_handling:
- Use patch.object on factory_module.litellm instead of direct assignment
- Ensures the correct litellm reference is modified after conftest reloads
2. test_embedding_header_forwarding_with_model_group:
- Use patch.object on pre_call_utils_module.litellm instead of direct assignment
- Same fix for module reloading issue
3. test_embedding_input_array_of_tokens:
- Move mock inside test function (after fixture initializes router)
- Add skip condition if llm_router is None
- Fixes "AttributeError: None does not have 'aembedding'" in parallel execution
Root cause: conftest.py reloads litellm at module scope, which can cause:
- Different litellm references between test code and library code
- Global state (like llm_router) being None at decorator execution time
- isinstance checks failing due to class identity mismatches
1. test_bedrock_converse_budget_tokens_preserved:
- Fixed mocking at the correct level (litellm.acompletion instead of client.post)
- The previous mock didn't work because the code runs through run_in_executor
and the passed client parameter was not being used
2. test_error_class_returns_volcengine_error:
- Changed isinstance check to class name comparison
- This avoids issues when module reloading (in conftest.py) causes class
identity mismatches during parallel test execution
1. test_acompletion_with_mcp_streaming_metadata_in_correct_chunks:
- Moved stream consumption inside patch context to avoid real API calls
- The previous implementation had assertions outside the `with patch(...)`
block, causing real OpenAI API calls when consuming the stream
2. TestCheckResponsesCost tests:
- Added skip condition when litellm_enterprise module is not available
- These tests import from litellm_enterprise.proxy.common_utils.check_responses_cost
which is only available in the enterprise version
The test_sso_key_generate_shows_deprecation_banner test was failing in CI
with a 403 Forbidden error because the SSO endpoint checks for premium_user
at line 297 in ui_sso.py.
The fix adds a monkeypatch for premium_user at its source location
(litellm.proxy.proxy_server.premium_user) to bypass the enterprise check
during testing.
Fixes the intermittent test failure where the endpoint would return 403
instead of the expected 200 status code.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit addresses two issues:
1. **Merge conflict resolution**: Resolved merge conflict in litellm/integrations/opentelemetry.py
that was preventing imports from working. The conflict was in the OpenTelemetry SDK
LogRecord import section.
2. **Test flakiness fix**: Fixed intermittent failures in test_bedrock_converse_budget_tokens_preserved
by properly configuring mock objects to avoid unawaited coroutine warnings.
The test was failing in CI with "Expected 'post' to have been called once. Called 0 times."
The root cause was improper mock setup where AsyncMock was creating async child methods
(raise_for_status, json) that returned unawaited coroutines, causing unreliable behavior
across different Python versions and test environments.
**Changes:**
- Set raise_for_status() and json() as explicit MagicMock instances on the response
- Use AsyncMock explicitly for the post() method via patch.object's 'new' parameter
- This ensures response methods are synchronous while the HTTP call remains async
**Testing:**
- Test now passes consistently across 5 consecutive runs
- RuntimeWarnings about unawaited coroutines eliminated (18 warnings → 16 warnings)
- Request JSON verification shows budget_tokens correctly preserved
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The manual sys.modules restoration code was redundant because
patch.dict.stop() automatically handles the cleanup. This simplifies
the tearDown method and removes the now-unused _original_langfuse_module
instance variable.
Addresses review comment: https://github.com/BerriAI/litellm/pull/21214#pullrequestreview-3802348462
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes test_log_langfuse_v2_handles_null_usage_values flaky test failure
by properly cleaning up sys.modules['langfuse'] in tearDown.
Changes:
- Store original langfuse module in setUp before mocking
- Restore original or remove mock in tearDown to prevent state pollution
- Remove invalid print_verbose parameter from log_event_on_langfuse
Root Cause:
The tearDown method was not cleaning up sys.modules['langfuse'] after
each test, causing mock state to leak between tests. This caused
intermittent failures in CI, especially when tests run in parallel or
in different orders.
Impact:
This test has a long history of flakiness with multiple attempted fixes
(#20475, #17599, #17594, #17591, #17588). The missing sys.modules cleanup
was the underlying issue causing continued failures despite those patches.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(content_filter.py): fix filter on toxic keywords
* feat: improve toxic/abusive language detection
* fix: additional improvements to nsfw filters
* feat: more improvements to nsfw filter
* feat(content_filter.json): add new australia specific nsfw content filter
ensure complete coverage for australia nsfw
* fix: cleanup policy templates
* fix(index.tsx): alert notice
* fix(index.tsx): add disclaimer notice
* feat(harmful_child_safety.yaml): new child safety content filter
ensure we catch inappropriate, child-specific content
* feat(policy_templates.json): add child safety and self harm filters
* fix(content_filter.py): improve racial bias filter to use a similar identifier + block word pattern and cover a wider range of ethnicities
* feat(policy_templates.json): add racial bias to nsfw policy template
* feat: add json content viewer
* fix(model_cost): add missing supports_system_messages and supports_tool_choice to bedrock/moonshotai.kimi-k2.5
* fix(streaming): ensure role=assistant is set on first streaming chunk via strip_role_from_delta
* fix(vertex_ai): ensure role=assistant on first streaming chunk for Llama models
Add VertexAILlama3StreamingHandler that injects role='assistant' into the
first streaming chunk delta when the Vertex AI Llama API omits it.
