litellm/tests/code_coverage_tests
Yuneng Jiang c63d0e6922
fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process
The cache edge keyed every recording on its own process's PYTEST_CURRENT_TEST.
Under xdist that names whatever test the serving worker is in, which is
unrelated to the caller: the proxy is a separate pod, and the Claude Code compat
matrix registered its shared aliases from every worker, each pointing at that
worker's edge, so the router spread one worker's calls across all eight edges.
Builds 234 and 235 of litellm-e2e, same commit, credited the same Bedrock
request to unrelated tests 92% of the time, and Bedrock never converged past a
~20% hit rate while OpenAI, whose deployments are per test, sat at 90%.

A deployment registered from inside a test now carries its test's slug in the
edge URL it is pointed at, `{edge}/{mount}/t/{slug}`, and the edge reads that
segment off every request before forwarding. A request without one is forwarded
live and never cached, and the edge no longer falls back to process state. The
compat aliases are registered with provider_live=True and stay on their real
provider path: no single test owns them, and the matrix exists to prove the real
CLI against real providers.
2026-09-16 17:08:17 -07:00
..
azure_client_usage_test.py fix - correctly re-use azure openai client 2025-03-18 09:51:28 -07:00
ban_constant_numbers.py Squashed commit of the following: (#9709) 2025-04-02 21:24:54 -07:00
ban_copy_deepcopy_kwargs.py Fix - using managed files w/ OTEL + UI - add model group alias on UI (#13171) 2025-07-31 21:22:04 -07:00
bedrock_pricing.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
callback_manager_test.py (Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112) 2025-01-30 19:35:50 -08:00
check_data_replace_usage.py Bug fix - String data: stripped from entire content in streamed Gemini responses (#9070) 2025-03-07 21:06:39 -08:00
check_e2e_no_raw_requests.py test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
check_endpoint_coverage.py Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
check_fastuuid_usage.py code cov test script check_fastuuid_usage.py 2025-09-24 10:27:22 +09:00
check_get_model_cost_key_performance.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
check_guardrail_apply_decorator.py content filter test fix 2026-02-12 17:54:16 -08:00
check_licenses.py fix: address cross-version CI failures 2026-09-02 14:17:19 -07:00
check_migrations_no_data_rewrites.py fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT 2026-09-16 18:14:36 +00:00
check_prisma_binary_cache.py fix(ci): bound setup steps so pytest always gets its full budget 2026-08-10 16:22:07 +00:00
check_provider_folders_documented.py Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
check_py310_typing_imports.py fix: keep litellm importable on Python 3.10 and guard 3.11-only typing imports in CI (#39448) 2026-09-02 18:27:19 -07:00
check_spanattributes_value_usage.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
check_unsafe_enterprise_import.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
check_workflow_job_name_collisions.py fix(ci): keep an expression matrix directive out of the comparison 2026-09-06 02:59:16 -07:00
check_workflow_startup_safety.py fix(ci): report budgets the startup guard cannot resolve 2026-08-10 18:12:18 +00:00
code_qa_check_tests.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
enforce_llms_folder_style.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ensure_async_clients_test.py fix code QA check 2025-10-07 17:49:57 -07:00
info_log_check.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
liccheck.ini Merge pull request #38765 from BerriAI/litellm_ocr_sdk_parity_tests 2026-09-03 10:16:35 -07:00
license_cache.json fix(deps): update python-multipart to >=0.0.20 in CI and test configs 2026-03-03 15:10:39 -03:00
litellm_logging_code_coverage.py docs(litellm_logging_code_coverage.py): fix check 2025-06-18 21:36:03 -07:00
log.txt Squashed commit of the following: (#9709) 2025-04-02 21:24:54 -07:00
memory_test.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
pass_through_code_coverage.py test: initial commit enforcing testing on all anthropic pass through … (#7794) 2025-01-15 22:02:35 -08:00
prevent_key_leaks_in_exceptions.py fix(main.py): fix key leak error when unknown provider given (#8556) 2025-02-15 14:02:55 -08:00
recursive_detector.py test(ci): allow completion_cost recursion for mixed-tier WS pricing split 2026-09-15 22:50:28 +00:00
router_code_coverage.py feat(router): support percentile-based TTFT routing (#40352) 2026-09-09 10:47:34 -07:00
router_enforce_line_length.py Litellm router code coverage 3 (#6274) 2024-10-16 21:30:25 -07:00
test_aio_http_image_conversion.py fix img URL for tests 2025-11-22 09:41:15 -08:00
test_ban_set_verbose.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_chat_completion_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_e2e_changed_gate.py test: bind management E2E callers and isolate JWT actors 2026-09-12 13:29:04 -07:00
test_e2e_idp_stack.py test(e2e): verify IdP readiness through real HTTP 2026-09-11 17:25:38 -07:00
test_no_hardcoded_secrets.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_provider_cache.py fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process 2026-09-16 17:08:17 -07:00
test_provider_replay_harness.py test: relocate strict replay harness coverage 2026-09-14 17:20:58 -07:00
test_proxy_types_import.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_strategy_async.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_workflow_job_name_collisions.py fix(ci): keep an expression matrix directive out of the comparison 2026-09-06 02:59:16 -07:00
user_api_key_auth_code_coverage.py test: initial test to enforce all functions in user_api_key_auth.py h… (#7797) 2025-01-15 21:52:45 -08:00