litellm/tests/integration/sdk
yuneng-jiang b602209ca6
test(integration): hand every harness server a port from one per-worker block (#46761)
* test(integration): hand every harness server a port from one per-worker block

About 45 places in the integration harness picked a port by binding port 0,
reading the number back, closing the socket and starting a server on that
number later. Another process could take the port in between, which surfaced
as "Address already in use" flakes. Earlier fixes retried around the race in
single call sites without removing it.

tests/integration/_support/ports.py now owns every port the harness picks
ahead of a server. Each pytest process claims a 200-port block between 10000
and 32767 by taking an exclusive flock on a per-block lock file in the system
temp directory, so parallel runs on one machine never share a block and the
lock goes away with the process. The range is above the fixed service ports
and below the Linux and macOS ephemeral ranges. Ports go out in order, are
never reused in a run, skip anything an outside process already holds, and
a process that exhausts its block claims the next free one.

The three _free_port copies, worker_endpoints' port-0 binds,
free_loopback_port, the owned proxy's lost-port-race retry and the owned
Redis retry loop are gone. Tests that restart a server on the same port now
allocate that port up front instead of reading it back from the first
server. TQ012 in scripts/check_test_quality.py rejects a port-0 bind read
back with getsockname anywhere under tests/integration.

* ci(integration): record listening sockets and the ephemeral port range

* Revert "ci(integration): record listening sockets and the ephemeral port range"

This reverts commit d3fafc3da2.
2026-10-15 16:56:40 -07:00
..
conftest.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
test_aiohttp_session_rebuild_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_anthropic_messages_bridge_mid_stream_failure_sdk.py test: extend runtime regression contracts for Messages and SDK (#46641) 2026-10-15 17:51:15 +00:00
test_azure_prompt_shield_tuple_messages.py fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging (#44067) 2026-10-01 23:30:32 -07:00
test_bedrock_converse_redacted_thinking_router_sdk.py fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
test_bedrock_converse_stream_sync_decoder_wire.py fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn (#44112) 2026-10-02 16:06:26 -07:00
test_bedrock_model_discovery_sdk.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
test_cache_control_census_pydantic_messages_sdk.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_caching_redis_sdk.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_callback_logging_cache.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_databricks_json_schema_refs_sdk.py fix(databricks): keep #/$defs refs in json_schema response_format (#45659) 2026-10-09 18:51:36 -07:00
test_device_code_login_guard_sdk.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_dual_cache_redis.py ci(circleci): test Redis behavior against local Redis and print short tracebacks (#44062) 2026-10-01 13:00:54 -07:00
test_gemini_long_decode_sync_sdk.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_generic_api_callback.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_google_genai_adapter_stream_usage_sdk.py fix(vertex_ai): route Model Garden models through the generateContent adapter (#41836) 2026-10-15 08:06:49 -07:00
test_http2_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_idle_upstream_reconnect_sdk.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_interception_converted_stream_options_sdk.py fix(websearch_interception): drop stream_options when forcing stream=False and replay it on the final stream (#45433) 2026-10-09 22:16:02 -07:00
test_langfuse_redaction_sdk.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_ocr_matrix_sdk.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_sdk_client_sessions_wire.py fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
test_owned_redis_ports.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_provider_budget_redis.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_provider_error_and_extra_body_families_wire.py feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
test_redis_breaker_metrics_multiprocess_sdk.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
test_redis_cluster_iam_auth.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_redis_service_metrics.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_response_cache_sync_sdk.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
test_responses_bridge_stream_errors_sdk.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_responses_bridge_tool_search_sdk.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_router_behavior_wire.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_router_budget_limiter_wire.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_router_cooldown_fallback_sdk.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_router_include_fallback_errors_wire.py fix(router): keep include_fallback_errors off the provider call on the sync Router path (#45218) 2026-10-07 22:25:12 -07:00
test_router_lowest_latency_wire.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_router_redis_tls_url.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_router_sync_stream_fallback_wire.py fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493) 2026-10-14 14:19:25 -07:00
test_router_tool_call_stream_fallback_budget_wire.py fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493) 2026-10-14 14:19:25 -07:00
test_router_unsupported_params_fallback_wire.py fix(router): add fallback_on_unsupported_params so param rejections can raise instead of falling back (#46350) 2026-10-14 14:43:01 -07:00
test_sagemaker_embedding_sdk.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
test_slack_daily_report_redis.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_stream_open_error_provider_headers_sdk.py fix(streaming): keep provider response headers on SDK streaming errors (#41528) 2026-10-15 08:11:54 -07:00
test_sync_stream_connection_release_wire.py fix(streaming): add CustomStreamWrapper.close() and release the connection when a sync stream ends (#46354) 2026-10-14 19:14:04 +00:00
test_tool_loop_server_tool_replay_sdk.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_topaz_image_variation_trailing_slash_sdk.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_usage_based_routing_sdk_redis_reads.py refactor: clean up fresh tech debt from 2026-09-29 (#43830) 2026-09-30 02:40:37 -07:00
test_usage_routing_counter_ttl.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_vertex_batch_row_line_errors_sdk.py fix(vertex_ai): return 400 with file line number for malformed batch JSONL row (#40097) 2026-10-15 03:21:48 -07:00
test_vertex_gemini_image_batch_cost_sdk.py refactor: expose public hidden_params accessors (#44668) 2026-10-07 00:01:33 -07:00
test_vertex_gemini_stream_timeout_wire.py fix(vertex_ai): pass the per-request timeout to Gemini streaming calls (#46349) 2026-10-14 10:51:18 -07:00
test_vertex_partner_count_tokens_sdk.py fix(vertex_ai): forward system and tools to partner model count_tokens (#43900) 2026-10-03 16:36:14 -07:00