scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
litellm/tests/test_litellm
History
Exact
Cursor Agent eb1a13f7a7 fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 
Root cause: LLMClientCache._remove_key() was calling aclose()/close() on
httpx clients when their cache entries expired (TTL) or were evicted (LRU).
However, callers of get_async_httpx_client() hold direct references to
these client objects (e.g. litellm.module_level_aclient, streaming handlers,
passthrough request handlers). Closing the underlying httpx client while
these references are still in use causes:

    RuntimeError: Cannot send a request, as the client has been closed.

This was introduced in commits 6931fea9 and bfeed0e5 (Feb 20, 2026) which
added close-on-eviction to prevent connection pool leaks. The fix removes
the close-on-eviction behavior: when a cache entry is evicted, only the
cache's reference is dropped. Python GC will finalize transport/connection
pool resources when all external references are released.

The bug manifests after ~1 hour of traffic (_DEFAULT_TTL_FOR_HTTPX_CLIENTS
= 3600s) when cache entries expire, or earlier under high concurrency if
the 200-entry LRU limit is reached. It affects all providers using the
shared httpx client cache (Vertex AI, Bedrock, etc.).

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-02-26 17:59:20 +00:00
..
a2a_protocol [Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604) 2026-02-06 15:02:34 -08:00
anthropic_interface/exceptions [bug fix] do not fallback to token counter if disable_token_counter is enabled (#19041) 2026-01-13 16:53:38 -08:00
caching fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 2026-02-26 17:59:20 +00:00
completion_extras fix(responses-api): return finish_reason='tool_calls' when response.completed contains function_call items (#19745) 2026-02-16 09:19:57 -08:00
containers fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
enterprise Fixes based on greptile reviews 2026-02-18 12:19:11 +05:30
expected_responses_api_request [Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 2026-02-12 10:00:30 -08:00
experimental_mcp_client fix: FLAKY tests 2026-01-24 11:13:44 -08:00
google_genai litellm_fix_mapped_tests_core: fix test isolation and mock injection issues (#20209) 2026-01-31 17:53:54 -08:00
images fix(image_edit): add drop_params support and fix Vertex AI config (#18077) 2025-12-17 11:28:34 +05:30
integrations Merge pull request #21604 from michelligabriele/fix/websearch-thinking-blocks 2026-02-26 22:26:38 +05:30
interactions fix(test): Update status enum values to match Google Interactions OpenAPI spec (#22061) 2026-02-24 20:26:11 -08:00
litellm_core_utils feat(realtime guardrails): end_session_after_n_fails + Endpoint Settings wizard step (#22165) 2026-02-25 23:49:03 -08:00
llms fix: don't close httpx clients on cache eviction (fixes 'client has been closed' errors) 2026-02-26 17:59:20 +00:00
passthrough fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
proxy fix(tests): update MCP server test mocks to match production API 2026-02-26 13:11:57 -03:00
responses fix(test): add timeout to flush() call to prevent 300s hang in CI (#21819) 2026-02-21 14:10:57 -08:00
router_strategy test(router): add coverage tests for _is_complexity_router_deployment and init_complexity_router_deployment (#21848) 2026-02-21 15:21:10 -08:00
router_utils Fix code qa 2026-02-26 12:43:06 +05:30
secret_managers fix(tests): isolate flaky files endpoint tests from global proxy state (#21788) 2026-02-21 11:20:32 -08:00
test_router Revert "Merge pull request #21957 from jquinter/fix/flaky-rpm-limit-test" 2026-02-24 09:52:50 -03:00
types Add pipeline flow builder UI for guardrail policies (#21188) 2026-02-13 20:06:03 -08:00
vector_stores litellm_fix_mapped_tests_core: fix test isolation and mock injection issues (#20209) 2026-01-31 17:53:54 -08:00
__init__.py
conftest.py fix(tests): restore disable_aiohttp_transport and force_ipv4 in isolate_litellm_state 2026-02-17 21:18:49 -03:00
log.txt fix: fix linting errors 2025-09-27 14:25:05 -07:00
readme.md
test_a2a_registry_lookup.py [Feat] Use A2A registered agents with /chat/completions (#20362) 2026-02-03 15:25:38 -08:00
test_acompletion_session_reuse_e2e.py [Fix] - Sessions not being shared (#15388) 2025-10-09 16:57:40 -07:00
test_add_deployment_no_master_key.py fix: remove strict master_key check in add_deployment (#16453) 2025-11-10 19:23:13 -08:00
test_aembedding_session_reuse_e2e.py [Fix] - shared session parsing and usage issue (#15440) 2025-10-10 19:26:53 -07:00
test_anthropic_beta_headers_filtering.py Make tests run with local beta header mapping json 2026-02-13 22:31:42 +05:30
test_azure_video_router.py Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
test_claude_haiku_4_5_config.py fix haiku-4-5 bedrock configs (#16732) 2025-11-17 19:52:01 -08:00
test_claude_opus_4_6_config.py Fix au.anthropic.claude opus 4 6 v1 (#20731) 2026-02-16 14:15:37 -08:00
test_constants.py [Release - 02/10/2026] v1.81.10-nightly 2026-02-10 16:26:30 -08:00
test_container_router.py Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
test_cost_calculation_log_level.py fix(tests): use record.getMessage() instead of record.message for LogRecord 2026-02-18 11:46:32 -03:00
test_cost_calculator.py perf: optimize completion_cost() — eliminate enum overhead, reduce function call indirection 2026-02-21 12:14:55 -08:00
test_deepseek_model_metadata.py fix(model-info): sync DeepSeek model metadata and add bare-name fallback (#20885) 2026-02-11 12:48:10 +05:30
test_eager_tiktoken_load.py fix(main): use local tiktoken cache in lazy loading (#19774) 2026-01-27 18:16:58 -08:00
test_exception_exports.py fix: export PermissionDeniedError from litellm.__init__ 2026-02-11 13:39:19 +01:00
test_exception_header_preservation.py fix: preserve llm_provider-* headers in error responses (#19020) 2026-01-14 22:49:39 +05:30
test_exception_mapping_request_attribute.py Added test 2025-09-29 21:43:31 +05:30
test_filter_out_litellm_params.py [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
test_get_blog_posts.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_gpt_image_cost_calculator.py Fix gpt-image-1.5 cost calculation not including output image tokens (#19515) 2026-01-22 19:42:15 -08:00
test_groq_streaming_encoding.py [Bug] Fix: Vertex Mistral not working for streaming (#13952) 2025-08-25 17:39:40 -07:00
test_lazy_imports.py Fix: test_token_counter_lazy_imports 2026-01-08 16:44:35 +05:30
test_logging.py fix:Parse embedded JSON in the message field of logs (#20366) 2026-02-10 16:13:33 +05:30
test_lowest_latency_zero_tokens.py fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 2025-09-29 15:49:46 -07:00
test_main.py Fix : test_video_content_handler_uses_get_for_openai 2026-02-17 20:06:08 +05:30
test_model_param_helper.py perf: cache _get_relevant_args_to_use_for_logging() at module level (#20077) 2026-02-02 10:54:49 -08:00
test_model_response_normalization.py Normalize OpenAI SDK BaseModel choices/messages to avoid Pydantic serializer warnings (#18972) 2026-01-14 03:40:11 +05:30
test_nested_drop_params.py feat: Replace jsonpath-ng with custom minimal parser for additional_drop_params 2025-12-09 17:30:30 +05:30
test_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
test_responses_api_bridge_non_stream.py fix: Pydantic will fail to parse it because cached_tokens is required but not provided 2026-01-28 11:51:26 +05:30
test_responses_id_security.py fix(ui): use non-streaming method for endpoint v1/a2a/message/send in… (#19025) 2026-01-14 03:29:10 +05:30
test_router.py Merge origin/main; keep both model_group_info and access_groups cache invalidation 2026-02-24 16:08:29 -08:00
test_router_google_genai.py test(gemini): Add unit tests for Google GenAI adapter 2025-09-29 18:51:35 +08:00
test_router_model_cost_isolation.py [Fix] prevent shared backend model key from being polluted by per-deployment custom pricing (#20679) 2026-02-09 19:38:44 -08:00
test_router_per_deployment_num_retries.py Bugfix/19481 num retries env var type (#19507) 2026-01-22 19:39:58 -08:00
test_router_redis_init.py fix: handle deprecated 'redis_db' arg to prevent crash (#19808) 2026-02-02 18:18:05 +05:30
test_router_silent_experiment.py litellm_fix(test): fix router silent experiment tests to properly mock async functions (#20140) 2026-01-31 07:39:05 -08:00
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_shared_session_integration.py feat: Add shared_session parameter for aiohttp ClientSession reuse 2025-09-19 01:46:53 -07:00
test_ssl_verify_unit.py BUMP Enterprise PIP 2026-02-14 13:40:48 -08:00
test_streaming_connection_cleanup.py fix: add debug logging to stream cleanup, improve tests 2026-02-14 17:31:39 -08:00
test_system_message_format_bug.py
test_utils.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_uuid_helper.py Fix: revert fastuuid optional dependency, always use fastuuid in .__uid helper (#14941) 2025-09-26 09:14:20 -07:00
test_video_generation.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_xai_responses_auto_routing.py Add routing of xai chat completions to responses when web search options is present 2026-01-30 14:15:35 +05:30

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py