Prevents silent incomplete releases by checking if Docker nightly image exists when PyPI package is already published. If both exist, the release was completed successfully and the workflow halts gracefully. If only PyPI exists, fail with an error to indicate the release needs investigation.
Increased retry attempts from 3 to 6 with longer delay (2s) for the Vertex AI Gemini function calling test. Added explicit timeout parameter and improved exception handling to skip (rather than fail) on transient API issues like rate limits, service unavailability, and timeouts. This prevents legitimate external API issues from blocking releases.
Added pre-flight checks in _safe_log to verify logger handlers are valid and their streams are open before attempting to log. This prevents 'I/O operation on closed file' errors during pytest-xdist worker shutdown when atexit handlers run after Python has closed file handles.
Renamed coverage files from litellm_router_unit_testing job to avoid conflict with litellm_router_testing job. Both jobs were persisting files with identical names causing workspace layer collision during parallel execution.
Enhanced the Docker deployment workflow to automatically detect and use the current branch for building containers when running on release day branches, improving deployment consistency.
Adjusted test job requirements in CI workflow to optimize build pipeline performance and reduce dependency complexity for coverage and publishing tasks.
Added autouse fixture that reloads constants and auth_checks modules before and after each test. This ensures complete test isolation and prevents any test from polluting module state for subsequent tests, regardless of test execution order.
The constant CLI_JWT_EXPIRATION_HOURS now checks both:
1. CLI_JWT_EXPIRATION_HOURS (without LITELLM_ prefix)
2. LITELLM_CLI_JWT_EXPIRATION_HOURS (with prefix, for backwards compatibility)
This ensures the test_all_numeric_constants_can_be_overridden test passes while maintaining backwards compatibility for users who set LITELLM_CLI_JWT_EXPIRATION_HOURS.
The custom expiration test was setting LITELLM_CLI_JWT_EXPIRATION_HOURS=48 and reloading modules, but not cleaning up afterwards. This caused subsequent tests to use the cached 48-hour value instead of the default 24 hours.
Added request.addfinalizer() to properly reload modules after the test completes, ensuring test isolation.
Changed from integration test to unit test by mocking the Lakera API response. This makes the test:
- Deterministic and not dependent on external API behavior
- Consistent with other tests in the file which are all mocked
- Able to test the masking logic regardless of Lakera's detection of test data
- Faster and more reliable in CI/CD
The mock response includes both credit card and email in the payload with proper start/end positions so we can verify the masking logic works correctly.
Lakera's API may not consistently return test credit card numbers (like 4111-1111-1111-1111) in the payload field for masking. Updated test to focus on email masking which works reliably, and made assertions more flexible.
The test was reloading the constants module but not auth_checks, which imports CLI_JWT_EXPIRATION_HOURS at module level. This caused the test to use the old cached value instead of the new environment variable value.
Addresses GHSA-wp53-j4wj-2cfg (CVE with CVSS 8.6) - Arbitrary File Write vulnerability. Updated python-multipart to version 0.0.22 which includes the security fix.
* Fix broken mocks in 6 flaky tests to prevent real API calls
Added network-level HTTP blocking using respx to prevent tests from making real API calls when Python-level mocks fail. This makes tests more reliable and retryable in CI.
Changes:
- Azure OIDC test: Added Azure Identity SDK mock to prevent real Azure calls
- Vector store test: Added @respx.mock decorator to block HTTP requests
- Resend email tests (3): Added @respx.mock decorator for all 3 test functions
- SendGrid email test: Added @respx.mock decorator
All test assertions and verification logic remain unchanged - only added safety nets to catch leaked API calls.
* Fix failing OIDC secret manager tests
Fixed two test failures in test_secret_managers_main.py:
1. test_oidc_azure_ad_token_success: Corrected the patch path for get_bearer_token_provider from 'litellm.secret_managers.get_azure_ad_token_provider.get_bearer_token_provider' to 'azure.identity.get_bearer_token_provider' since the function is imported from azure.identity.
2. test_oidc_google_success: Added @patch('httpx.Client') decorator to prevent any real HTTP connections during test execution, resolving httpx.ConnectError issues.
Both tests now pass successfully.
* Fix PLR0915: Extract system message handling to reduce statement count
* fix mypy
* fix: add host_progress_callback parameter to mock_call_tool in test
The test_call_tool_without_broken_pipe_error was failing because the mock function did not accept the host_progress_callback keyword argument that the actual implementation passes to client.call_tool(). Updated the mock to accept this parameter to match the real implementation signature.
* fixing flaky tests around oidc and email
* Add documentation comment to test file
* add retry
* add dependency
* increase retry
---------
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
* cache control for user messages and system messages
* add cache createion tokens in reponse
* cache controls in tool calls and assistant turns
* refactor with _should_preserve_cache_control
* add cache control unit tests
* use simpler cache creation token count logic
* use helper function
* remove unused function
* fix unit tests
* fix: make HTTPHandler mockable in OIDC secret manager tests
- Add _get_oidc_http_handler() factory function to make HTTPHandler
easily mockable in tests
- Update test_oidc_github_success to patch factory function instead
of HTTPHandler directly
- Update Google OIDC tests for consistency
- Fixes test_oidc_github_success failure where mock was bypassed
This change allows tests to properly mock HTTPHandler instances used
for OIDC token requests, fixing the test failure where the mock was
not being used.
* fix: patch base_llm_http_handler method directly in container tests
- Use patch.object to patch container_create_handler method directly
on the base_llm_http_handler instance instead of patching the module
- Fixes test_provider_support[openai] failure where mock wasn't applied
- Also fixes test_error_handling_integration with same approach
The issue was that patching 'litellm.containers.main.base_llm_http_handler'
didn't work because the module imports it with 'from litellm.main import',
creating a local reference. Using patch.object patches the method on the
actual object instance, which works regardless of import style.
* fix: resolve flaky test_openai_env_base by clearing cache
- Add cache clearing at start of test_openai_env_base to prevent cache pollution
- Ensures no cached clients from previous tests interfere with respx mocks
- Fixes intermittent failures where aiohttp transport was used instead of httpx
- Test-only change with low risk, no production code modifications
Resolves flaky test marked with @pytest.mark.flaky(retries=3, delay=1)
Both parametrized versions (OPENAI_API_BASE and OPENAI_BASE_URL) now pass consistently
* test: add explicit mock verification in test_provider_support
- Capture mock handler with 'as mock_handler' for explicit validation
- Add assert_called_once() to verify mock was actually used
- Ensures test verifies no real API calls are made
- Follows same pattern as test_openai_env_base validation
Mock _create_mcp_client to avoid network calls in health checks.
This prevents asyncio.CancelledError when the test teardown closes
the event loop while health checks are still pending.
The test focuses on conversion logic (access_groups, description)
not health check functionality, so mocking the network call is appropriate.
- Add test_get_valid_args in test_router_helper_utils.py to cover get_valid_args
- Use encoding='utf-8' in router_code_coverage.py for cross-platform file reads
- Create a test user with auto_create_key=False to ensure known starting state
- Filter get_users by user_ids to target only the test user
- Verify initial key count is 0 before creating a key
- Clean up test user after test completes
- This ensures consistent behavior across CI and local environments
- Added a Pydantic validator to convert empty string inputs for max_budget to None, preventing float parsing errors from the frontend.
- Modified the internal user update logic to explicitly allow max_budget to be None, ensuring the value isn't filtered out and can be reset to unlimited in the database.
- Added unit tests for validation and logic.
Closes#19781
* fix(proxy): use return value from CustomLogger.async_post_call_success_hook
Previously the return value was ignored for CustomLogger callbacks,
preventing users from modifying responses. Now the return value is
captured and used to replace the response (if not None), consistent
with CustomGuardrail and streaming iterator hook behavior.
Fixes issue with custom_callbacks not being able to inject data into
LLM responses.
* fix(proxy): also fix async_post_call_streaming_hook to use return value
Previously the streaming hook only used return values that started with
"data: " (SSE format). Now any non-None return value is used, consistent
with async_post_call_success_hook and streaming iterator hook behavior.
Added tests for streaming hook transformation.
---------
Co-authored-by: Gabriele Michelli <michelligabriele0@gmail.com>
