litellm/tests/test_litellm/llms
Anmol Jaiswal bb27bfd9a7
fix(http_handler): dispose aiohttp session when AsyncHTTPHandler is finalized without a running loop (#36670)
* fix(http_handler): dispose aiohttp session when finalized without a running loop

AsyncHTTPHandler.__del__ can only schedule an async close when a running
event loop exists at finalization time; in any other context (worker
threads whose loop has closed, sync contexts, interpreter shutdown) the
RuntimeError from get_running_loop() is swallowed and the underlying
aiohttp ClientSession is abandoned to GC, emitting 'Unclosed client
session' / 'Unclosed connector' warnings.

This is the disposal gap left after the recycle-time fix: clients created
for short-lived event loops (the loop-id-keyed LLM client cache mints one
handler per loop) are never recycled - they live and die with their loop,
and their finalization is precisely the loop-less case.

Fix:
- no running loop: fall back to the connector's synchronous teardown via
  LiteLLMAiohttpTransport._mark_connector_closed - the same finalizer-safe
  path used for dead-loop recycles - honoring _owns_session so a shared
  session is never closed.
- running loop: keep the async close, but hold a strong reference to the
  scheduled task until it completes (a bare create_task() result may be
  collected before running), mirroring _background_close_tasks.

Tests: loop-less finalization closes a dead-loop session; running-loop
finalization registers and drains the close task; the sync fallback
respects session ownership. All three fail without the fix.

* lint: conform new finalizer code to the type-discipline budget

Final on the five never-rebound locals (LIT010); the class-level task
registry keeps its mutable set with the sanctioned mutable-ok reason,
mirroring the aiohttp transport's registry (LIT001).

* lint: reasoned pyright ignore on the cross-class teardown call

The handler deliberately reuses the transport's finalizer-safe connector
teardown; no public seam exists and an async close can never run at
loop-less finalization. Clears the net-new reportPrivateUsage the
basedpyright budget gate flagged once the LIT stage passed.

* fix(http_handler): retrieve exceptions from finalizer close tasks

A bare discard done-callback dropped the task without consuming its
exception, so a failing aclose() emitted "Task exception was never
retrieved" at GC, the same noise class this path exists to remove.
Mirror the transport's _on_close_task_done: discard, early-return on
cancellation, retrieve and debug-log the exception.

* fix(http_handler): dispose foreign-loop sessions instead of scheduling aclose on the live loop

GC on a live loop (e.g. the app's) of a handler whose session belongs to
another, possibly dead, loop scheduled aclose() on the current loop, the
cross-loop path the transport refuses. Route both that case and the
loop-less case through the transport's lifecycle-aware
_close_recycled_session, which picks async close on the session's own
loop, threadsafe handoff, or the synchronous connector teardown.

Regression test: a dead-loop session collected while another loop runs
is disposed without scheduling anything on that loop.

* chore: retrigger CI (test_mcp_logging payload-order flake, also failed on litellm_spendlogs_fallback_metadata minutes earlier)

* test(mcp): select the MCP tool-call payload instead of the last-delivered one

TestMCPLogger kept a single last-writer slot; an async success event from
another call (a mocked acompletion whose log task lands late) races the
MCP event for it, so the cost assertions intermittently read the wrong
payload. This PR's finalizer change shifts task interleaving on the loop
and tips that latent race over (also seen on an unrelated PR minutes
earlier). Collect call_type=call_mcp_tool payloads in their own list and
assert on those.

* test(mcp): MCPLoggerHook inherits the order-independent payload capture

It duplicated TestMCPLogger's init and success handler verbatim; the
hook test reads the same MCP payload selection, so subclass instead.
2026-08-25 08:12:10 -07:00
..
a2a/chat fix(a2a): populate response usage in a2a chat transformation (#31980) 2026-07-03 09:28:36 +05:30
aiml/image_generation test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
amazon_nova/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
anthropic Merge pull request #38108 from BerriAI/litellm_fix_46_thinking_budget 2026-08-24 12:19:37 -07:00
apiserpent test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
azure Merge pull request #38119 from BerriAI/litellm_bing_grounding_search_provider 2026-08-24 19:33:23 -07:00
azure_ai Merge pull request #35415 from BerriAI/litellm_azure_ai_entra_auth 2026-08-24 11:54:48 -07:00
base_llm feat(search): add Grounding with Bing Search (bing_grounding) as a search provider 2026-08-24 11:08:24 -07:00
baseten/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
bedrock Merge branch 'litellm_internal_staging' into litellm_bedrock_converse_no_trailing_empty_chunk 2026-08-24 10:37:32 -07:00
bedrock_mantle test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
black_forest_labs test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
bytez/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
chatgpt test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cloudflare fix(cloudflare): route native Workers AI provider through OpenAI-compatible endpoint (#31053) 2026-06-23 12:09:11 -07:00
cohere test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cometapi/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
compactifai style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
crusoe test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
custom_httpx fix(http_handler): dispose aiohttp session when AsyncHTTPHandler is finalized without a running loop (#36670) 2026-08-25 08:12:10 -07:00
dashscope test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
databricks test: separate gemini entries storing the promo rate from those storing list 2026-08-22 17:54:44 -07:00
datarobot test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
deepgram test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
deepinfra test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
deepseek test: merge three stranded twins into the files that shadow them (#37600) 2026-08-20 14:25:23 -07:00
docker_model_runner test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
elevenlabs test: avoid provider test module name collision 2026-04-29 22:36:57 -07:00
fal_ai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
fastcrw/search chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
featherless_ai/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
fireworks_ai fix(anthropic): stop signing replayed thinking blocks and strip reasoning_content 2026-08-22 14:27:20 -07:00
gdc/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
gemini test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
github_copilot test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
gradient_ai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
groq fix: omit thinking.type=disabled for always-on thinking Claude models (#37510) 2026-08-21 10:27:26 -07:00
heroku style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
hosted_vllm fix(anthropic): stop signing replayed thinking blocks and strip reasoning_content 2026-08-22 14:27:20 -07:00
huggingface test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
inception test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
jina_ai/embedding test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
langflow test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
lemonade test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
litellm_proxy test: remove tests that never execute 2026-08-12 10:45:38 -07:00
llamafile/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
lm_studio style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
manus test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
meta_llama test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
minimax test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
mistral test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
modelscope test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
moonshot test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
nebius test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
neosantara Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
nimble/search feat(search): add Nimble as a search provider (#36347) 2026-08-14 17:09:58 -07:00
novita/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
nscale/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
nvidia_nim/rerank test: drop rerank package marker colliding with voyage test package 2026-08-11 23:37:27 -07:00
nvidia_riva test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
oci test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ocr test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ollama Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
oobabooga/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
openai fix(videos): forward uploaded source file on /v1/videos/edits to the provider 2026-08-24 15:34:21 -07:00
openai_like Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_scx_ai_provider 2026-08-21 17:27:49 +10:00
openrouter test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ovhcloud test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
parallel_ai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
parasail Litellm oss staging 080626 (#29932) 2026-06-08 13:49:52 -07:00
pass_through test: remove four mirror test files that exercise none of their module 2026-07-25 10:57:19 -07:00
perplexity test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
pg_vector/vector_stores test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
publicai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ragflow/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
recraft test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
reducto Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
runwayml fix(runwayml): skip progress scaling when Runway returns a null progress 2026-08-24 12:20:00 -07:00
s3_vectors Fix extra body error 2026-04-29 08:34:31 +05:30
sagemaker test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
sambanova Feat/sambanova embeddings (#13308) 2025-08-12 17:15:26 -07:00
sap style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
scaleway Feature/add audio support for scaleway (#26110) 2026-04-20 14:49:41 -07:00
snowflake chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
soniox chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
stability test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
tencent fix: omit thinking.type=disabled for always-on thinking Claude models (#37510) 2026-08-21 10:27:26 -07:00
tinyfish test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
valkey/vector_stores feat(vector_stores): add Valkey as a managed vector store provider (#37002) 2026-08-18 21:45:22 +00:00
vercel_ai_gateway test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
vertex_ai fix(videos): forward uploaded source file on /v1/videos/edits to the provider 2026-08-24 15:34:21 -07:00
volcengine test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
voyage test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
wandb test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
watsonx Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
xai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
you_com test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
zai test: stop the zai tests from leaking env and litellm globals into the session (#37834) 2026-08-21 22:10:34 -07:00
test_cache_control_and_reasoning.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_file_content_block.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_file_search_responses.py fix(tests): use canonical litellm_enterprise import path (#27699) 2026-05-12 12:32:57 -07:00
test_lifecycle_fix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_polling_url_origin_match.py chore(security): close two unaddressed SSRF cases 2026-05-01 18:43:47 +00:00
test_predibase_transformation.py refactor(predibase): migrate transform_request and transform_response… (#25249) 2026-04-27 08:58:22 +05:30