litellm/tests/integration/routing
yuneng-jiang b602209ca6
test(integration): hand every harness server a port from one per-worker block (#46761)
* test(integration): hand every harness server a port from one per-worker block

About 45 places in the integration harness picked a port by binding port 0,
reading the number back, closing the socket and starting a server on that
number later. Another process could take the port in between, which surfaced
as "Address already in use" flakes. Earlier fixes retried around the race in
single call sites without removing it.

tests/integration/_support/ports.py now owns every port the harness picks
ahead of a server. Each pytest process claims a 200-port block between 10000
and 32767 by taking an exclusive flock on a per-block lock file in the system
temp directory, so parallel runs on one machine never share a block and the
lock goes away with the process. The range is above the fixed service ports
and below the Linux and macOS ephemeral ranges. Ports go out in order, are
never reused in a run, skip anything an outside process already holds, and
a process that exhausts its block claims the next free one.

The three _free_port copies, worker_endpoints' port-0 binds,
free_loopback_port, the owned proxy's lost-port-race retry and the owned
Redis retry loop are gone. Tests that restart a server on the same port now
allocate that port up front instead of reading it back from the first
server. TQ012 in scripts/check_test_quality.py rejects a port-0 bind read
back with getsockname anywhere under tests/integration.

* ci(integration): record listening sockets and the ephemeral port range

* Revert "ci(integration): record listening sockets and the ephemeral port range"

This reverts commit d3fafc3da2.
2026-10-15 16:56:40 -07:00
..
_prompt_cache_pin_support.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
either_role.json test(integration): add read-replica routing harness to the CircleCI integration suite (#42692) 2026-09-24 00:25:03 -07:00
test_advisor_failure_cooldown.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_audit_any_sweep_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_auto_router_management.py test(integration): wait on the right signal in five flaky integration contracts (#46737) 2026-10-15 13:48:51 -07:00
test_azure_passthrough_spend_row.py fix(vllm): write spend log rows for /vllm passthrough calls to router models (#45733) 2026-10-14 12:40:48 -07:00
test_complexity_router_databricks_classifier.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_complexity_router_llm_classifier.py refactor(types): replace Any with proven types in 7 files (#43844) 2026-10-02 02:47:47 -07:00
test_declared_provider_bare_model_routes.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_deployment_num_retries_generic_routes_wire.py fix(router): retry a /v1/messages stream the provider drops before the first content chunk (#44276) 2026-10-05 22:53:39 +00:00
test_encrypted_content_affinity_no_peer.py fix(router): serve Responses turns from a sibling when the encrypted content origin has no boundary peer (#43015) 2026-09-24 17:31:43 -07:00
test_end_user_region_routing.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_fallback_hop_foreign_encrypted_reasoning.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_fallback_retry_header_contracts.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_include_fallback_errors_wire.py fix(router): keep include_fallback_errors off the provider call on the sync Router path (#45218) 2026-10-07 22:25:12 -07:00
test_key_max_parallel_requests.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_key_tpm_reservation.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_messages_stream_retry_budget_sources_owned_proxy.py fix(router): retry a /v1/messages stream the provider drops before the first content chunk (#44276) 2026-10-05 22:53:39 +00:00
test_model_capability_flags.py fix: keep model capability flags out of provider requests (#46647) 2026-10-15 14:30:10 +00:00
test_observed_routing.py test(integration): cover health_routing endpoint contracts (#44744) 2026-10-14 06:34:22 +00:00
test_payment_required_cooldown.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_payment_required_mapping.py fix(exceptions): map upstream 402 to PaymentRequiredError and cool down 402 deployments (#44879) 2026-10-06 20:35:13 -07:00
test_pdf_document_pre_call_checks_owned_proxy.py test(integration): remove six setup causes of CircleCI integration reds (#46490) 2026-10-14 12:00:35 -07:00
test_per_model_rate_limit_fallbacks.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_per_model_rate_limit_fallbacks_lifecycle.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_priority_model_tpm_enforcement.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_priority_rate_limit_headers.py chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
test_priority_scheduler_queue_cleanup.py test(integration): remove six setup causes of CircleCI integration reds (#46490) 2026-10-14 12:00:35 -07:00
test_prompt_cache_pin_memory_routes.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_prompt_cache_pin_ttl.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_prompt_cache_pin_ttl_chaos.py fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl (#46084) 2026-10-14 21:10:09 -07:00
test_prompt_caching_affinity_tool_call_marks.py fix(caching): count tool_call cache_control marks in the injection census (#43556) 2026-10-03 17:33:48 -07:00
test_ptu_shares.py feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
test_ptu_shares_regex_timing.py feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
test_redis_increment_with_floor.py ci(circleci): test Redis behavior against local Redis and print short tracebacks (#44062) 2026-10-01 13:00:54 -07:00
test_redis_recovery.py test(integration): add read-replica routing harness to the CircleCI integration suite (#42692) 2026-09-24 00:25:03 -07:00
test_stale_cost_map_boot.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_team_tag_routing.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_usage_based_routing_redis_reads.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_user_config_routing.py test: take keys out of the legacy proxy, enterprise and mcp unit tests before moving them (#42901) 2026-09-24 14:56:52 -07:00
test_video_poll_routing_wire.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_vllm_passthrough_spend_row.py fix(vllm): write spend log rows for /vllm passthrough calls to router models (#45733) 2026-10-14 12:40:48 -07:00
test_vllm_passthrough_stream_relay.py fix(vllm): write spend log rows for /vllm passthrough calls to router models (#45733) 2026-10-14 12:40:48 -07:00
test_vllm_passthrough_stream_usage_settings.py fix(vllm): write spend log rows for /vllm passthrough calls to router models (#45733) 2026-10-14 12:40:48 -07:00