scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102
litellm/tests
History
Yuneng Jiang 4b3f5d7f81
[Fix] conftest: flush cache instances and warn on silent skips 
Addresses review feedback on the snapshot approach:

1. Class-instance mutable state
   The snapshot only covers primitives + collections + None. Class
   instances (DualCache, LLMClientCache) weren't reset between tests,
   so in-place cache mutations could leak. Can't deepcopy these — they
   hold thread locks — but they expose flush_cache(). Collect every
   module attribute whose value implements flush_cache() at conftest
   import, and invoke it per-test alongside the snapshot restore.

2. Silent skips are now warnings
   _snapshot_mutable_state and _restore_mutable_state previously
   swallowed exceptions, so if a future attr gained a property without
   a setter (or other non-round-trippable state), an isolation gap
   would have no signal. Emit warnings.warn on each failure path.

3. Docstring
   Explicitly documents what IS and IS NOT reset, and tells authors to
   use monkeypatch.setattr() for in-place mutations of instances
   without flush_cache() (ProxyLogging, JWTHandler, etc.).
2026-04-20 22:19:36 -07:00
..
agent_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
audio_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
benchmarks Add CodSpeed performance benchmarks (#23676) 2026-03-14 18:44:36 -07:00
code_coverage_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
documentation_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
enterprise fixing backwards compatibility for tests 2026-04-17 18:08:48 +00:00
guardrails_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
image_gen_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_utils_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
llm_responses_api_testing style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
llm_translation style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing replace retired claude-3-haiku-20240307 with claude-haiku-4-5-20251001 in local_testing part1 and router fallback tests 2026-04-20 16:10:45 -07:00
logging_callback_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr17 2026-04-17 17:36:40 -07:00
mcp_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr17 2026-04-17 17:36:40 -07:00
multi_instance_e2e_tests
ocr_tests Merge branch 'litellm_internal_staging' into litellm_pages_support_for_ocr 2026-04-17 19:29:45 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
otel_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
pass_through_tests [Fix] should_create_missing_views returns False for reltuples=0 (falsy zero bug) 2026-04-18 11:00:09 -07:00
pass_through_unit_tests replace retired claude-3-haiku-20240307 with claude-haiku-4-5-20251001 in anthropic messages passthrough test 2026-04-20 15:44:11 -07:00
proxy_admin_ui_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_e2e_anthropic_messages_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_security_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_unit_tests [Fix] conftest: flush cache instances and warn on silent skips 2026-04-20 22:19:36 -07:00
router_unit_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
scim_tests
search_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
spend_tracking_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
store_model_in_db_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm Litellm krrish staging 04 20 2026 (#26138) 2026-04-20 16:22:12 -07:00
unified_google_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
vector_store_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
windows_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py
test_entrypoint.py
test_fallbacks.py Revert "fix: prevent error when max_fallbacks exceeds available models (#20071)" 2026-02-03 15:15:30 +05:30
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py
test_keys.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_proxy_responses_config.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py [Fix] test_ratelimit: skip over-limit cases that race with background RPM tracking 2026-04-11 13:08:59 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py fix(langfuse_otel): prevent empty proxy request spans from being sent to Langfuse 2026-01-28 15:35:35 +01:00
test_spend_logs.py
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py
test_users.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.