description: >- Paths deliberately outside CI coverage, each with the reason it is exempt. assert_ci_coverage.py fails when a test file or Dockerfile is neither invoked by a job nor listed here, so every entry below is a decision on the record. test_paths: - reason: >- The caching suite in tests/local_testing, which runs nowhere. Every job that globs that directory either deselects it (local_testing_part1 and part2 carry `-k "... and not caching and not cache"`) or keeps only another keyword (langfuse, router, assistants), and no job names these files the way redis_caching_unit_tests names test_dual_cache.py. Measured 2026-08-20 by collecting the directory under each job's own selector: 118 tests across these eight files are selected by none of them. Listed so the gap is a decision rather than an accident, and so the --slices guard has a baseline to ratchet down from. Revisit when tests/local_testing is ported off CircleCI, where the keyless part of this suite belongs in a real job paths: - tests/local_testing/test_cache_preset_key.py - tests/local_testing/test_caching.py - tests/local_testing/test_caching_handler.py - tests/local_testing/test_disk_cache_unit_tests.py - tests/local_testing/test_gcs_cache_unit_tests.py - tests/local_testing/test_prompt_caching.py - tests/local_testing/test_responses_stream_cache_keys.py - tests/local_testing/test_unit_test_caching.py - reason: >- The end-to-end suite runs against a deployed proxy from its own in-cluster rig rather than from a pull request; it needs a live gateway and provider credentials no PR job holds paths: - tests/e2e - reason: >- The documentation and code-quality workflows execute four files in this directory by name as scripts and pytest never collects the directory, so these six run nowhere; listed individually so a seventh cannot inherit the exemption paths: - tests/documentation_tests/test_exception_types.py - tests/documentation_tests/test_general_setting_keys.py - tests/documentation_tests/test_optional_params.py - tests/documentation_tests/test_readme_providers.py - tests/documentation_tests/test_requests_lib_usage.py - tests/documentation_tests/test_standard_logging_payload.py - reason: >- Named like a test but shaped like a benchmark: it fetches live image URLs, times aiohttp against httpx, prints the ratio, and asserts nothing, so pytest cannot collect it (its functions take arguments, not fixtures) and running it beside its siblings in the code-quality workflow would add a network dependency for a number nothing reads. Exempt as a script rather than as an unresolved gap; revisit by deleting it once the aiohttp choice it informed is settled paths: - tests/code_coverage_tests/test_aio_http_image_conversion.py - reason: >- No job invokes this suite and its files mix pure transformation tests with ones driving live vendor vector stores, so assigning them needs a per-file decision paths: - tests/vector_store_tests/rag/test_rag_bedrock.py - tests/vector_store_tests/rag/test_rag_openai.py - tests/vector_store_tests/rag/test_rag_s3_vectors.py - tests/vector_store_tests/rag/test_rag_vertex_ai.py - tests/vector_store_tests/test_azure_ai_vector_store.py - tests/vector_store_tests/test_azure_vector_store.py - tests/vector_store_tests/test_bedrock_vector_store.py - tests/vector_store_tests/test_gemini_vector_store.py - tests/vector_store_tests/test_milvus_vector_store.py - tests/vector_store_tests/test_openai_vector_store.py - tests/vector_store_tests/test_ragflow_vector_store.py - tests/vector_store_tests/test_s3_vectors_vector_store.py - tests/vector_store_tests/test_vertex_ai_search_api_vector_store.py - tests/vector_store_tests/test_vertex_ai_vector_store.py - reason: >- Throughput and memory-growth measurements whose runtime and variance make them unsuitable for a per-pull-request job paths: - tests/load_tests/test_datadog_load_test.py - tests/load_tests/test_langsmith_load_test.py - tests/load_tests/test_linear_memory_growth.py - tests/load_tests/test_memory_usage.py - tests/load_tests/test_otel_load_test.py - tests/load_tests/test_vertex_embeddings_load_test.py - tests/load_tests/test_vertex_load_tests.py - reason: >- A local-only agent rig: test_a2a_completion_bridge.py needs a LangGraph server on localhost:2024 and test_a2a.py drives a live A2A endpoint, so neither can run in a pull request job. Until 2026-08-20 the CircleCI agent job hid them behind a grep -v that this census could not see; the glob now excludes them structurally and this entry is the decision on the record. Revisit when the A2A bridge gets a recorded-wire fixture paths: - tests/agent_tests/local_only_agent_tests - reason: >- Third-party integration tests that skip themselves without OCI configuration or sandbox credentials, neither of which a pull request job holds paths: - tests/integration/sandbox/test_e2b_sandbox.py - tests/integration/test_oci_integration.py - tests/integration/test_oci_proxy_integration.py - reason: >- A unit test for the proxy-extras package that no job invokes, while the package's other tests live under tests/proxy_migration_tests. Measured 2026-08-20: 24 of its 28 tests pass and the 4 in TestMigrationSQLIdempotency fail, because 13 migrations from 2026-03 onward use bare CREATE TABLE, ADD COLUMN, CREATE INDEX and ADD CONSTRAINT rather than the guarded forms this file requires. It also matches those keywords inside SQL comments, so two further migrations are reported that are in fact fine. Wiring it up means deciding what to do about the 13 first, and they cannot simply be edited: Prisma checksums an applied migration, so a changed one breaks migrate deploy for existing installs paths: - tests/litellm-proxy-extras/test_litellm_proxy_extras_utils.py dockerfiles: - reason: >- The dashboard container is a static Next.js export served by nginx, and the dashboard build and lint workflows already exercise that output, so building the image adds no signal about it paths: - ui/Dockerfile - reason: >- The Rust gateway ships as its own chart and package with a separate release pipeline, so its image is not part of this repo's Python image set paths: - litellm-rust/crates/ai-gateway/Dockerfile - reason: >- An example image under cookbook/ that is documentation rather than a shipped artifact paths: - cookbook/litellm-ollama-docker-image/Dockerfile