litellm/tests/code_coverage_tests
Yuneng Jiang 30c6241e3a
fix(e2e): a null error field is not an error
Every OpenAI Responses body carries `error: null` at the top level, and the
completeness check tested the key's presence rather than its value, so it
rejected every single one. The cost was silent: nothing failed, the endpoint
simply never cached, which is exactly the outcome the endpoint was added for.

Found by driving the edge against the real providers rather than the synthetic
fixtures, which carried no error key at all. Reading the value instead of the
key is also more accurate for chat completions and messages, where a real error
body carries a populated error object.
2026-09-16 03:01:07 -07:00
..
azure_client_usage_test.py fix - correctly re-use azure openai client 2025-03-18 09:51:28 -07:00
ban_constant_numbers.py Squashed commit of the following: (#9709) 2025-04-02 21:24:54 -07:00
ban_copy_deepcopy_kwargs.py Fix - using managed files w/ OTEL + UI - add model group alias on UI (#13171) 2025-07-31 21:22:04 -07:00
bedrock_pricing.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
callback_manager_test.py (Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112) 2025-01-30 19:35:50 -08:00
check_data_replace_usage.py Bug fix - String data: stripped from entire content in streamed Gemini responses (#9070) 2025-03-07 21:06:39 -08:00
check_e2e_no_raw_requests.py test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
check_endpoint_coverage.py Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
check_fastuuid_usage.py code cov test script check_fastuuid_usage.py 2025-09-24 10:27:22 +09:00
check_get_model_cost_key_performance.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
check_guardrail_apply_decorator.py content filter test fix 2026-02-12 17:54:16 -08:00
check_licenses.py fix: address cross-version CI failures 2026-09-02 14:17:19 -07:00
check_migrations_no_data_rewrites.py fix: keep a schema-qualified call inside an index expression from reading as a relation 2026-08-24 16:04:38 -07:00
check_prisma_binary_cache.py fix(ci): bound setup steps so pytest always gets its full budget 2026-08-10 16:22:07 +00:00
check_provider_folders_documented.py Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
check_py310_typing_imports.py fix: keep litellm importable on Python 3.10 and guard 3.11-only typing imports in CI (#39448) 2026-09-02 18:27:19 -07:00
check_spanattributes_value_usage.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
check_unsafe_enterprise_import.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
check_workflow_job_name_collisions.py fix(ci): keep an expression matrix directive out of the comparison 2026-09-06 02:59:16 -07:00
check_workflow_startup_safety.py fix(ci): report budgets the startup guard cannot resolve 2026-08-10 18:12:18 +00:00
code_qa_check_tests.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
enforce_llms_folder_style.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ensure_async_clients_test.py fix code QA check 2025-10-07 17:49:57 -07:00
info_log_check.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
liccheck.ini Merge pull request #38765 from BerriAI/litellm_ocr_sdk_parity_tests 2026-09-03 10:16:35 -07:00
license_cache.json fix(deps): update python-multipart to >=0.0.20 in CI and test configs 2026-03-03 15:10:39 -03:00
litellm_logging_code_coverage.py docs(litellm_logging_code_coverage.py): fix check 2025-06-18 21:36:03 -07:00
log.txt Squashed commit of the following: (#9709) 2025-04-02 21:24:54 -07:00
memory_test.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
pass_through_code_coverage.py test: initial commit enforcing testing on all anthropic pass through … (#7794) 2025-01-15 22:02:35 -08:00
prevent_key_leaks_in_exceptions.py fix(main.py): fix key leak error when unknown provider given (#8556) 2025-02-15 14:02:55 -08:00
recursive_detector.py test(ci): allow completion_cost recursion for mixed-tier WS pricing split 2026-09-15 22:50:28 +00:00
router_code_coverage.py feat(router): support percentile-based TTFT routing (#40352) 2026-09-09 10:47:34 -07:00
router_enforce_line_length.py Litellm router code coverage 3 (#6274) 2024-10-16 21:30:25 -07:00
test_aio_http_image_conversion.py fix img URL for tests 2025-11-22 09:41:15 -08:00
test_ban_set_verbose.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_chat_completion_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_e2e_changed_gate.py test: bind management E2E callers and isolate JWT actors 2026-09-12 13:29:04 -07:00
test_e2e_idp_stack.py test(e2e): verify IdP readiness through real HTTP 2026-09-11 17:25:38 -07:00
test_no_hardcoded_secrets.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_provider_cache.py fix(e2e): a null error field is not an error 2026-09-16 03:01:07 -07:00
test_provider_replay_harness.py test: relocate strict replay harness coverage 2026-09-14 17:20:58 -07:00
test_proxy_types_import.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_strategy_async.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_workflow_job_name_collisions.py fix(ci): keep an expression matrix directive out of the comparison 2026-09-06 02:59:16 -07:00
user_api_key_auth_code_coverage.py test: initial test to enforce all functions in user_api_key_auth.py h… (#7797) 2025-01-15 21:52:45 -08:00