Commit graph

11 commits

Author SHA1 Message Date
Alexsander Hamir
f95572e3ed
Fix broken mocks in 6 flaky tests to prevent real API calls (#19829)
* Fix broken mocks in 6 flaky tests to prevent real API calls

Added network-level HTTP blocking using respx to prevent tests from making real API calls when Python-level mocks fail. This makes tests more reliable and retryable in CI.

Changes:

- Azure OIDC test: Added Azure Identity SDK mock to prevent real Azure calls

- Vector store test: Added @respx.mock decorator to block HTTP requests

- Resend email tests (3): Added @respx.mock decorator for all 3 test functions

- SendGrid email test: Added @respx.mock decorator

All test assertions and verification logic remain unchanged - only added safety nets to catch leaked API calls.

* Fix failing OIDC secret manager tests

Fixed two test failures in test_secret_managers_main.py:

1. test_oidc_azure_ad_token_success: Corrected the patch path for get_bearer_token_provider from 'litellm.secret_managers.get_azure_ad_token_provider.get_bearer_token_provider' to 'azure.identity.get_bearer_token_provider' since the function is imported from azure.identity.

2. test_oidc_google_success: Added @patch('httpx.Client') decorator to prevent any real HTTP connections during test execution, resolving httpx.ConnectError issues.

Both tests now pass successfully.
2026-01-26 17:39:40 -08:00
yuneng-jiang
33b4b444ac fixing flaky tests around oidc and email 2026-01-26 15:23:17 -08:00
Alexsander Hamir
f2c3a01a57
fix: make HTTPHandler mockable in OIDC secret manager tests (#19803)
* fix: make HTTPHandler mockable in OIDC secret manager tests

- Add _get_oidc_http_handler() factory function to make HTTPHandler
  easily mockable in tests
- Update test_oidc_github_success to patch factory function instead
  of HTTPHandler directly
- Update Google OIDC tests for consistency
- Fixes test_oidc_github_success failure where mock was bypassed

This change allows tests to properly mock HTTPHandler instances used
for OIDC token requests, fixing the test failure where the mock was
not being used.

* fix: patch base_llm_http_handler method directly in container tests

- Use patch.object to patch container_create_handler method directly
  on the base_llm_http_handler instance instead of patching the module
- Fixes test_provider_support[openai] failure where mock wasn't applied
- Also fixes test_error_handling_integration with same approach

The issue was that patching 'litellm.containers.main.base_llm_http_handler'
didn't work because the module imports it with 'from litellm.main import',
creating a local reference. Using patch.object patches the method on the
actual object instance, which works regardless of import style.

* fix: resolve flaky test_openai_env_base by clearing cache

- Add cache clearing at start of test_openai_env_base to prevent cache pollution
- Ensures no cached clients from previous tests interfere with respx mocks
- Fixes intermittent failures where aiohttp transport was used instead of httpx
- Test-only change with low risk, no production code modifications

Resolves flaky test marked with @pytest.mark.flaky(retries=3, delay=1)
Both parametrized versions (OPENAI_API_BASE and OPENAI_BASE_URL) now pass consistently

* test: add explicit mock verification in test_provider_support

- Capture mock handler with 'as mock_handler' for explicit validation
- Add assert_called_once() to verify mock was actually used
- Ensures test verifies no real API calls are made
- Follows same pattern as test_openai_env_base validation
2026-01-26 11:00:42 -08:00
Ishaan Jaffer
f148207b11 fix patch reliability mock tests 2026-01-24 17:37:17 -08:00
Ishaan Jaffer
a7e26460d0 fix unstable tests 2026-01-24 11:15:16 -08:00
Ishaan Jaff
630a746c84
[Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297)
* add CustomSecretManager class

* docs custom secret manager

* add TestCustomSecretManager

* add KeyManagementSystem.CUSTOM

* add get_secret_from_manager

* add custom secret manager

* Potential fix for code scanning alert no. 3662: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* docs fix

* load_custom_secret_manager

* initialize_secret_manager

* add custom_secret_manager

* fix add custom secret manager

* add custom secret manager to KeyManagementSystem

* fix KeyManagementSystem.CUSTOM

* fix custom secret manager within cookbook

* fix link for custom secret manager

* Potential fix for code scanning alert no. 3663: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-05 17:20:26 -08:00
Krrish Dholakia
d0732f55b3 test(test_get_azure_ad_token_provider.py): add unit test to ensure default azure credentials used in the right context 2025-09-11 16:07:32 -07:00
Felix Burmester
51bbd12116
Added handling for pwd protected cert files in AOAI CertificateCredential auth (#12995) 2025-07-28 21:34:54 -07:00
Shuai Zhang
712e042aa4
fix(secret-managers): Break AzureCredentialType restriction on AZURE_CREDENTIAL (#11272) 2025-05-31 01:03:08 -07:00
Krish Dholakia
1995c7aad5
fix(utils.py): support non default params for audio transcription (#11212)
* fix(utils.py): support non default params for audio transcription

allows passing provider specific params straight through on transcription calls

* fix(gpt_transformation.py): fix o_series model routing

call _transform_request on async event

* refactor: refactor tests

* test(test_azure_chat_o_series_transformation.py): add unit test for azure o series error

* test: update test

* test: update json

* fix: fix mutiple keyword error
2025-05-28 22:24:02 -07:00
Krish Dholakia
ef42461c1e
Litellm fix GitHub action testing (#11163)
* test: add __init__.py files

* refactor: rename test folder to avoid naming conflict

* test: update workflows

* test: update tests

* test: update imports

* test: update tests

* test: remove unused import

* ci(test-litellm.yml): add pytest retry to github workflow

* test: fix test
2025-05-26 14:41:42 -07:00