scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
litellm/tests/test_litellm
History
shin-bot-litellm 37a45a3295
litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks (#20196) 
## Problem
Tests using mocked HTTP clients were hitting real APIs because:
1. HTTP client cache was returning previously cached real clients
2. isinstance checks failed due to module identity issues from sys.path

### Tests affected:
- test_send_email_missing_api_key
- test_send_email_multiple_recipients (resend & sendgrid)
- test_search_uses_registry_credentials
- test_vector_store_create_with_simple_provider_name
- test_vector_store_create_with_provider_api_type
- test_vector_store_create_with_ragflow_provider
- test_image_edit_merges_headers_and_extra_headers
- test_retrieve_container_basic (container API tests)

## Solution
1. Add clear_client_cache fixture (autouse=True) to clear
   litellm.in_memory_llm_clients_cache before each test
2. Fix isinstance checks to use type name comparison
   (avoids module identity issues from sys.path.insert)

## Why not disable_aiohttp_transport
The default transport is aiohttp, so tests should work with it.
Clearing the cache ensures mocks are used instead of cached real clients.

## Regression
PR #19829 (commit f95572e3ed) added @respx.mock but cached clients
from earlier tests were being reused, bypassing the mocks.

Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
2026-01-31 15:42:17 -08:00
..
a2a_protocol [Fix] A2a Gateway - Allow supporting old A2a card formats (#19949) 2026-01-28 15:02:08 -08:00
anthropic_interface/exceptions [bug fix] do not fallback to token counter if disable_token_counter is enabled (#19041) 2026-01-13 16:53:38 -08:00
caching fix(redis): handle float redis_version from AWS ElastiCache Valkey (#16207) 2025-11-04 19:20:00 -08:00
completion_extras Add annotations to completions responses API bridge 2026-01-07 14:30:31 +05:30
containers litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks (#20196) 2026-01-31 15:42:17 -08:00
enterprise litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks (#20196) 2026-01-31 15:42:17 -08:00
experimental_mcp_client fix: FLAKY tests 2026-01-24 11:13:44 -08:00
google_genai test_stream_transformation_error_sync 2026-01-24 17:19:21 -08:00
images fix(image_edit): add drop_params support and fix Vertex AI config (#18077) 2025-12-17 11:28:34 +05:30
integrations fix(proxy): resolve high CPU when router_settings in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger (#20087) 2026-01-30 14:01:45 -08:00
interactions fix: test_create_simple_string_input 2026-01-05 17:15:12 +05:30
litellm_core_utils fix(test): correct prompt_tokens in test_string_cost_values (#20185) 2026-01-31 14:23:10 -08:00
llms Merge pull request #20086 from BerriAI/litellm_watsonx_inte_fix 2026-01-31 09:13:42 -08:00
passthrough fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
proxy test_budget_reset_and_expires_at_first_of_month 2026-01-31 12:36:53 -08:00
responses fix(responses): update local_vars with detected provider (#19782) (#19798) 2026-01-26 20:47:35 -08:00
router_strategy feat(tag-routing): support toggling tag matching between ANY and ALL (#18776) 2026-01-08 23:39:03 +05:30
router_utils [Feat] New Logging Integration - Azure Sentinel Logger (#18146) 2025-12-18 02:04:49 +05:30
secret_managers Fix broken mocks in 6 flaky tests to prevent real API calls (#19829) 2026-01-26 17:39:40 -08:00
types fix: extend case normalization to ALL guardrail types (not just tool_permission) (#18664) 2026-01-05 11:59:00 -08:00
vector_stores litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks (#20196) 2026-01-31 15:42:17 -08:00
__init__.py
conftest.py Perf speed up pytest (#15951) 2025-10-27 19:43:40 -07:00
log.txt fix: fix linting errors 2025-09-27 14:25:05 -07:00
readme.md
test_acompletion_session_reuse_e2e.py [Fix] - Sessions not being shared (#15388) 2025-10-09 16:57:40 -07:00
test_add_deployment_no_master_key.py fix: remove strict master_key check in add_deployment (#16453) 2025-11-10 19:23:13 -08:00
test_aembedding_session_reuse_e2e.py [Fix] - shared session parsing and usage issue (#15440) 2025-10-10 19:26:53 -07:00
test_azure_video_router.py Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
test_claude_haiku_4_5_config.py fix haiku-4-5 bedrock configs (#16732) 2025-11-17 19:52:01 -08:00
test_constants.py tes numeric constants 2025-10-29 18:23:38 -07:00
test_container_router.py Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
test_cost_calculation_log_level.py Enhance logging in cost calculation tests to ensure DEBUG level captures are accurate. Updated tests to set logger level before assertions and restored original logger level after execution. This improves reliability of log level checks in both cost and batch cost calculation tests. 2025-08-14 14:21:22 -07:00
test_cost_calculator.py Merge branch 'main' into litellm_oss_staging_01_28_2026 2026-01-29 17:39:42 +05:30
test_eager_tiktoken_load.py fix(main): use local tiktoken cache in lazy loading (#19774) 2026-01-27 18:16:58 -08:00
test_exception_header_preservation.py fix: preserve llm_provider-* headers in error responses (#19020) 2026-01-14 22:49:39 +05:30
test_exception_mapping_request_attribute.py Added test 2025-09-29 21:43:31 +05:30
test_filter_out_litellm_params.py [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
test_gpt_image_cost_calculator.py Fix gpt-image-1.5 cost calculation not including output image tokens (#19515) 2026-01-22 19:42:15 -08:00
test_groq_streaming_encoding.py [Bug] Fix: Vertex Mistral not working for streaming (#13952) 2025-08-25 17:39:40 -07:00
test_lazy_imports.py Fix: test_token_counter_lazy_imports 2026-01-08 16:44:35 +05:30
test_logging.py
test_lowest_latency_zero_tokens.py fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 2025-09-29 15:49:46 -07:00
test_main.py litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks (#20196) 2026-01-31 15:42:17 -08:00
test_model_response_normalization.py Normalize OpenAI SDK BaseModel choices/messages to avoid Pydantic serializer warnings (#18972) 2026-01-14 03:40:11 +05:30
test_nested_drop_params.py feat: Replace jsonpath-ng with custom minimal parser for additional_drop_params 2025-12-09 17:30:30 +05:30
test_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
test_responses_api_bridge_non_stream.py fix: Pydantic will fail to parse it because cached_tokens is required but not provided 2026-01-28 11:51:26 +05:30
test_responses_id_security.py fix(ui): use non-streaming method for endpoint v1/a2a/message/send in… (#19025) 2026-01-14 03:29:10 +05:30
test_router.py Manual revert #19078 2026-01-17 06:54:08 +09:00
test_router_google_genai.py test(gemini): Add unit tests for Google GenAI adapter 2025-09-29 18:51:35 +08:00
test_router_per_deployment_num_retries.py Bugfix/19481 num retries env var type (#19507) 2026-01-22 19:39:58 -08:00
test_router_silent_experiment.py litellm_fix(test): fix router silent experiment tests to properly mock async functions (#20140) 2026-01-31 07:39:05 -08:00
test_shared_session_integration.py feat: Add shared_session parameter for aiohttp ClientSession reuse 2025-09-19 01:46:53 -07:00
test_ssl_verify_unit.py Fix/per service ssl override v2 (#19538) 2026-01-21 20:10:04 -08:00
test_system_message_format_bug.py
test_utils.py litellm_fix(test): allow comment field in schema and exclude robotics models from tpm check (#20139) 2026-01-31 07:32:33 -08:00
test_uuid_helper.py Fix: revert fastuuid optional dependency, always use fastuuid in .__uid helper (#14941) 2025-09-26 09:14:20 -07:00
test_video_generation.py test fixes 2026-01-24 14:08:48 -08:00
test_xai_responses_auto_routing.py Add routing of xai chat completions to responses when web search options is present 2026-01-30 14:15:35 +05:30

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py