* feat(vertex_ai): use HTTP/2 httpx client for search_api vector store
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(httpx): exercise configured request path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(httpx): verify HTTP/2 request negotiation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(httpx): preserve HTTP/2 for sync proxy mounts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): add h2 to proxy extra so vertex search HTTP/2 is on by default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): isolate ssl context for http2 clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http): add an HTTP/2 opt-out and harden the Vertex AI Search HTTP/2 client
litellm.http2 is now unset by default, so http2: false or LITELLM_HTTP2=False
turns the provider's HTTP/2 default off. A caller's own SSLContext keeps the
search on HTTP/1.1, the HTTP/2 async client honors DISABLE_AIOHTTP_TRUST_ENV,
forced IPv4 transports get the CA bundle and client cert, and vector store
search retries once when the server drops the connection with a GOAWAY.
* fix(http): retry vector store search only on HTTP/2 GOAWAY and keep aiohttp pool and proxy behavior
Scope the one retry to a connection Google closes with an HTTP/2 GOAWAY and
log it. When HTTP/2 replaces the aiohttp transport, keep aiohttp's idle
keepalive and connection cap, and only then skip env proxies under
DISABLE_AIOHTTP_TRUST_ENV, so the global LITELLM_HTTP2 toggle keeps HTTPS_PROXY
* test(http): cover the vector store search GOAWAY retry in unit tests and drop duplicate SSL cache comments
* fix(http): open another HTTP/2 connection once 100 streams are in use
httpcore keeps every request to an origin on one HTTP/2 connection and caps it at 100 concurrent streams, so request 101 and later waited with no timeout. The handlers now spread requests over extra connections, opening one only when every open connection has 100 streams in flight. The force_ipv4 wire tests now cover ssl_verify=False and the client certificate.
* fix(vertex_ai): reuse one Vertex search OAuth token across searches
Every vector store search built a new search config, so the token cache it carries was thrown away and each search fetched a new Google OAuth token with a blocking call on the event loop. Under a burst of searches that blocking fetch held back every HTTP/2 response. The provider config is now shared, so searches with the same credentials reuse one token until it expires.
* fix(http): keep the HTTP/2 connection spread within the connection cap
* fix(http): keep sync vertex search on HTTP/1.1, since httpcore's sync HTTP/2 is not thread-safe
httpcore 1.0.9 hands two threads the same HTTP/2 stream id, so threaded sync
searches failed with protocol errors. Only the async client, which the proxy
uses, now defaults to HTTP/2, and the sync connection spread is removed.
* fix(http): split the connection cap across HTTP/2 lanes
Each lane had its own pool with the full connection cap, so an origin answering
HTTP/1.1, or several origins on one client, could exceed max_connections once
more than 100 requests were in flight. Lanes now open only while lanes times
100 streams stay under the cap, and each lane pool gets at most 100 of it.
* test(http2): scope wire-test peer logs to each fixture instance
* fix(http2): keep TE, force_ipv4 trust, evicted-client and opt-out behavior on the HTTP/2 path
- drop TE values other than trailers on https requests through the HTTP/2 spread, since h2 refuses them
- force_ipv4 transports keep httpx's default trust store (SSL_CERT_DIR) unless SSL is configured
- the evicted client closer asks the HTTP/2 spread whether a request is in flight
- LITELLM_HTTP2 set to 0, no or off, and a quoted "false" http2 setting, opt out of HTTP/2
- the wire test's held-search barrier waits on an event instead of polling
* fix(http2): count proxy-mounted transports when checking an evicted client for in-flight requests
* test(http2): move the HTTP/2 eviction cases that open a loopback server into the integration wire tests
* fix(http2): read an unquoted http2: 0 as off and free the stream slot of a response dropped without closing
* fix(http2): free the stream slot of a partly read response dropped without closing
* test(http2): check that the evicted client and its stream close after a partly read response is dropped
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): hand every harness server a port from one per-worker block
About 45 places in the integration harness picked a port by binding port 0,
reading the number back, closing the socket and starting a server on that
number later. Another process could take the port in between, which surfaced
as "Address already in use" flakes. Earlier fixes retried around the race in
single call sites without removing it.
tests/integration/_support/ports.py now owns every port the harness picks
ahead of a server. Each pytest process claims a 200-port block between 10000
and 32767 by taking an exclusive flock on a per-block lock file in the system
temp directory, so parallel runs on one machine never share a block and the
lock goes away with the process. The range is above the fixed service ports
and below the Linux and macOS ephemeral ranges. Ports go out in order, are
never reused in a run, skip anything an outside process already holds, and
a process that exhausts its block claims the next free one.
The three _free_port copies, worker_endpoints' port-0 binds,
free_loopback_port, the owned proxy's lost-port-race retry and the owned
Redis retry loop are gone. Tests that restart a server on the same port now
allocate that port up front instead of reading it back from the first
server. TQ012 in scripts/check_test_quality.py rejects a port-0 bind read
back with getsockname anywhere under tests/integration.
* ci(integration): record listening sockets and the ephemeral port range
* Revert "ci(integration): record listening sockets and the ephemeral port range"
This reverts commit d3fafc3da2.
The check spins up a hypercorn TLS peer and drives the SDK's own httpx
handlers at it, so it needs litellm importable, hypercorn installed and a
loopback socket. It lived under tests/e2e, whose Buildkite runner image
installs neither litellm nor hypercorn by design (the suite drives a
remote proxy over HTTP), so every scheduled e2e build since #230 failed
to import the module and pytest reported it as a collection error. The
unit tree bans sockets, so it does not belong there either
tests/integration is the CircleCI tier built for real TCP against local
protocol peers. This adds an sdk shard to it for cases that exercise the
SDK's clients with no gateway in the path, registers the two HTTP/2
nodes in the contracts manifest, and adds the shard to the CircleCI
matrix. The test now flips the feature through LITELLM_HTTP2 (the user
surface) instead of patching module attributes, and asserts the version
the peer observed on the wire next to the one the client reports
2026-09-17 10:37:07 -07:00
Renamed from tests/e2e/llm_translation/test_outbound_http2_e2e.py (Browse further)
