mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-08 03:08:45 +00:00
a734afca32
* feat(ci): gate patching of SDK internals in tests as TQ008 TQ002 catches the narrowest symptom of the suite's dominant mocking idiom, patch X then assert only that X was called. The idiom itself is wider: tests reach for litellm's own functions instead of faking the wire, so they pin how the code is wired rather than what it does, and a test that patches internals but makes weak real assertions trips nothing today. TQ008 counts patch targets rooted at `litellm`, both the dotted string form and the attribute chain handed to patch.object, and ratchets like every other rule. Mocking anything outside the SDK is untouched: respx, httpx transports and third-party clients do not trip it, which is the point, since those are the patterns this is meant to move the suite toward. Seeded at 9,643, in line with the ~9.4k patch sites an independent grep found in the mirror. The burn-down horizon is long; the value here is stopping the flow rather than clearing the stock. Five existing rule tests patched `litellm.completion` incidentally and now report TQ008 alongside what they were pinning. Their expected values are updated to the accurate pair rather than loosened, so they keep failing on a regression in either rule. * test: add TQ008 to the shipped-budget rule canary * fix(ci): resolve imported SDK names in TQ008 patch.object(handler.OpenAIChatCompletion, ...) after a from-import reaches the same internal as the dotted string form, but the rule only saw the bare local name and let it through. Import bindings are now resolved to the path they stand for, so the aliased, renamed and from-imported forms all read alike and the reported target is the real one. That is 1,496 patches the ratchet could not see, so the TQ008 limit moves from 9,643 to 11,139. Third-party names and locals with no SDK import behind them stay unflagged. |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm-proxy-extras | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.