* fix: prevent HTTP client memory leaks in Presidio and OpenAI wrappers
Fixes multiple memory leak issues reported in #14540 and related tickets:
**Presidio Guardrail Fix (#14540)**
- Problem: Every guardrail check created a new aiohttp.ClientSession
- Impact: High-traffic proxies accumulated thousands of unclosed sessions
- Solution: Share a single session across all guardrail checks
- Added `self._http_session` instance variable
- Lazy session creation via `_get_http_session()`
- Proper cleanup via `_close_http_session()` and `__del__()`
- Files: litellm/proxy/guardrails/guardrail_hooks/presidio.py
**OpenAI HTTP Client Caching (#14540)**
- Problem: `_get_async_http_client()` created new httpx.AsyncClient on each call
- Impact: OpenAI/Azure completions bypassed client caching system
- Solution: Route through `get_async_httpx_client()` for TTL-based caching
- Caches clients by provider and SSL config
- Fallback to direct creation if caching fails
- Applied to both async and sync client methods
- Files: litellm/llms/openai/common_utils.py
**Test Script**
- Added validation script to demonstrate fixes
- Counts file descriptors and unclosed session objects
- Files: test_oom_fixes.py
Related issues: #14384, #13251, #12443
* fix(oom): prevent memory leaks in Presidio guardrails and OpenAI client creation
Fixes two high-impact memory leaks:
1. Presidio Guardrail Session Leak (issue #14540)
- Problem: Created new aiohttp.ClientSession on every guardrail check
- Impact: Runs on EVERY proxy request when PII masking enabled
- Fix: Shared session pattern with lifecycle management
- Files: litellm/proxy/guardrails/guardrail_hooks/presidio.py
2. OpenAI HTTP Client Cache Bypass (issue #14540)
- Problem: _get_async_http_client() created new httpx.AsyncClient, bypassing TTL cache
- Impact: Every completion created new client with own connection pool
- Fix: Route through get_async_httpx_client() for proper caching
- Critical: Include SSL config in cache key for correctness
- Files: litellm/llms/openai/common_utils.py
Validation:
- Presidio: 100 requests → 0 new sessions (was 100)
- OpenAI: 100 calls → 1 unique client (was 100)
- test_oom_fixes.py: Automated validation script
* fix(oom): resolve Gemini aiohttp session leak (issue #12443)
Fixes persistent "Unclosed client session" warnings when using Gemini models.
Root Causes:
1. Broken atexit cleanup - get_event_loop() fails at exit time
2. On-demand session creation without reliable cleanup
Changes:
1. Fixed atexit Cleanup (async_client_cleanup.py)
- OLD: Used get_event_loop() which fails when loop is closed
- NEW: Always create fresh event loop at exit time
- Ensures cleanup runs successfully even when main loop is closed
2. Added __del__ Cleanup (aiohttp_handler.py)
- Defense-in-depth: cleanup on garbage collection
- Handles abnormal termination cases
- Similar pattern to Presidio guardrail fix
3. Enhanced Cleanup Scope (async_client_cleanup.py)
- Now closes global base_llm_aiohttp_handler instance
- Previously only checked cache, missed module-level handler
Validation:
- Test 1: __del__ cleanup → 0 sessions leaked ✓
- Test 2: atexit cleanup → 0 sessions leaked ✓
- test_gemini_session_leak.py: Automated validation
Related: #14540 (broader OOM issue tracking)
* fix(types): use LlmProviders enum for get_async_httpx_client
MyPy was failing because llm_provider parameter expects Union[LlmProviders, httpxSpecialProvider], not a string.
Changed from string "openai" to LlmProviders.OPENAI enum value.
* test: move validation tests to proper CI directories
- Move test_oom_fixes.py to tests/test_litellm/llms/
- Move test_gemini_session_leak.py to tests/test_litellm/llms/custom_httpx/
- Fix pytest warning: use pytest.skip() instead of return True
This ensures CI actually runs our OOM fix validation tests.
* fix(oom): add asyncio.Lock to prevent race conditions in Presidio session creation
- Make _get_http_session() async with asyncio.Lock protection
- Prevents multiple concurrent requests from creating orphaned sessions
- Add concurrent load test (50 parallel requests) to validate fix
- Test confirms only 1 session created under concurrent load
Critical fix: Previous implementation had race condition where
concurrent guardrail checks could create multiple sessions,
defeating the shared session pattern and causing memory leaks.
* fix(presidio): eliminate race condition in session lock initialization
Move asyncio.Lock creation from lazy initialization in _get_http_session()
to __init__. The previous lazy init had a race condition where concurrent
coroutines could both see _session_lock as None, both create locks, and
end up with different lock instances - defeating the synchronization.
asyncio.Lock() can be safely created without an event loop; it only
requires one when awaited.
This commit fixes two critical test failures and two test isolation issues
in the SSL configuration tests.
## Critical Test Failures Fixed
### 1. test_get_ssl_configuration
**Problem:** Test was failing with assertion error that ssl.create_default_context
was never called (expected 1 call, got 0).
**Root Cause:** The get_ssl_configuration() function uses a caching mechanism
(_ssl_context_cache) to avoid creating duplicate SSL contexts with the same
configuration. When tests run in sequence, a previous test may have created an
SSL context with the same configuration (same cafile, ssl_security_level,
ssl_ecdh_curve). When this test runs, it retrieves the cached context instead
of creating a new one, so ssl.create_default_context() is never called, causing
the mock assertion to fail.
**Fix:** Clear the SSL context cache at the start of the test to ensure a fresh
context is created, allowing the mock to be called and verified.
### 2. test_ssl_ecdh_curve
**Problem:** Test was failing with assertion error that set_ecdh_curve was
never called (expected 1 call, got 0).
**Root Cause:** Same caching issue as above. Additionally, the test needed to
use a real SSLContext instance instead of a MagicMock because _create_ssl_context
calls methods like set_ciphers() and minimum_version that require a real context.
**Fix:**
- Clear the SSL context cache at the start of the test
- Use a real SSLContext instance and patch set_ecdh_curve on it specifically
- Added explanatory comment about why a real context is needed
## Test Isolation Issues Fixed
### 3. test_ssl_security_level
**Problem:** Test was failing because it expected LiteLLMAiohttpTransport but
got httpx.AsyncHTTPTransport instead.
**Root Cause:** Test isolation issue. Other tests in the file (test_force_ipv4_transport,
test_aiohttp_disabled_transport) set litellm.disable_aiohttp_transport = True
but don't restore the original value. When this test runs after those tests,
aiohttp transport is disabled, causing it to use httpx transport instead.
**Fix:** Explicitly enable aiohttp transport at the start of the test and restore
the original value in a finally block, ensuring the test works regardless of
test execution order.
### 4. test_ssl_verification_with_aiohttp_transport
**Problem:** Same as above - expected LiteLLMAiohttpTransport but got
httpx.AsyncHTTPTransport.
**Root Cause:** Same test isolation issue - aiohttp transport disabled by
previous tests.
**Fix:** Same approach - explicitly enable aiohttp transport and restore
original value in finally block.
## Why These Fixes Work
1. **Cache clearing:** By clearing _ssl_context_cache before each test, we
ensure that get_ssl_configuration() creates a fresh SSL context, allowing
mocks to be properly called and verified.
2. **Test isolation:** By saving and restoring the disable_aiohttp_transport
setting, tests are independent of each other and work correctly regardless
of execution order.
These are minimal, targeted fixes that address the root causes without
modifying production code or affecting other functionality.
* perf: Skip sleep delays in base_mail.py during tests to improve test speed
* perf: Mock datetime.now in parallel_request_limiter_v3.py to improve test speed
* pref: Mock urllib system calls in test_aiohttp_transport.py to improve test speed
* chore: add --durations=50 to visualize slowest tests
* pref: reduce setup phase overhead by widening fixture scope in conftest.py
* test: stabilize flaky tests
* fix: minor issue
Fixes RuntimeError "Session is closed" by:
- Checking session.closed before use and recreating if needed
- Catching RuntimeError during requests and retrying with new session
- Validating newly created sessions aren't already closed
Adds tests for both proactive detection and reactive retry scenarios.
Allow passing aiohttp.ClientSession to acompletion() calls for better
performance and resource management. Includes debug logging, tests,
and documentation. Backward compatible.
- Add optional client_session, transport, and connector parameters to constructor
- Implement session ownership tracking to prevent closing user-provided sessions
- Add comprehensive session resolution hierarchy (dynamic > instance > create new)
- Include transport control for advanced HTTP stack management
- Add 29 comprehensive tests covering all injection scenarios
- Maintain backward compatibility with existing code
This enhancement allows users to inject their own configured aiohttp sessions,
transports, and connectors for fine-grained control over connection pooling,
SSL settings, proxy configurations, and other HTTP stack parameters.
* fix: fixes for transfer encoding error on aiohttp transport
* Update tests/test_litellm/llms/custom_httpx/test_aiohttp_transport.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix: AiohttpResponseStream transport
* fix: use AiohttpResponseStream transport by default
* fix: AiohttpResponseStream transport
* fixes: mapping aiohttp exceptions
* fixes: aiohttp rollout
* fixes: add support ssl_verify for aiohttp
* fixes: add support ssl_verify for aiohttp
* fixes: remove duplicates
