The suite already waits for a new model or agent to become servable before
handing it back, but that wait returns on the first successful read. Every
request opens a fresh connection (e2e_http calls requests.* with no Session), so
a load-balanced Service routes each one independently: one successful read proves
one replica converged, and the caller's next request re-rolls and can land on a
replica that has not reloaded yet.
At replicaCount: 2 this surfaced as 30 failures on a SHA that is green at 1
replica -- 400 "Invalid model name passed", 404 "Guardrail not found", "no
healthy deployments for this model", and a /model/info listing that contained
one of two models created moments apart.
Add PROPAGATION_TIMEOUT (default 15s, override E2E_PROPAGATION_TIMEOUT) and
settle_propagation(), sized off the proxy's proxy_config_reload_interval_seconds
(30s by default, 7s on the e2e stack) plus margin, and settle after every
control-plane create whose object the suite then uses:
- ProxyClient.create_model and A2AClient.register_agent, after their existing
polls -- the poll still fails loudly if the object never appears at all
- GuardrailsClient.register, which had no barrier; create_content_filter_guardrail
and create_bedrock_guardrail now route through it instead of POSTing directly
- the guardrail creates in mcp_client and logging_client
- the vertex passthrough model, whose body cannot go through create_model
Left alone: the /model/new calls that assert a 403 or read back a status code,
since they never use the model.
The shared proxy wrapper in tests/e2e/e2e_gateway.py was misnamed: Gateway is
not a gateway server, it is the client every suite uses to talk to the proxy
(keys, models, chat/embed/ocr, spend read-backs, poll helpers). Rename the
module to proxy_client.py and the class to ProxyClient, with build_gateway
becoming build_proxy_client and the GatewayProvider protocol becoming
ProxyClientProvider. The .gateway attribute suites held is now .proxy. Only
identifiers changed; prose and string literals that use the word gateway for the
proxy-server concept were left alone.
Each suite previously built its own instance through a per-suite build_client()
that called build_gateway() inside, duplicating the proxy wiring across suites.
There is now one session-scoped proxy fixture in tests/e2e/conftest.py; every
suite's client fixture depends on it and injects it, so the wiring lives in one
place. claude_code keeps building its own client directly since it has its own
harness and does not use the shared fixtures.
Behavior is unchanged: shared transport, data-plane/control-plane split routing,
poll budget, typed request/response models, and resource cleanup all go through
the same object.
* test(e2e): add vertex_ai passthrough spend-log coverage
Port the de-flake of the SDK-based vertex spend test (#31689) into the
tests/e2e/llm_translation harness. The vertexai SDK intermittently ignored the
proxy api_endpoint override and billed Vertex directly, so the request never
reached LiteLLM and no spend was logged; driving native generateContent over the
shared transport always reaches the proxy, which the harness already guarantees.
The vertex deployment is added at runtime through /model/new with
use_in_pass_through rather than declared in the gateway config, and deleted on
teardown. That registers the deployment's service account for the /vertex_ai
route, so the passthrough call sends only its litellm virtual key in
x-litellm-api-key and no upstream bearer, and the proxy mints the Vertex token
itself. The credential is the one the proxy already holds, read from the same
VERTEXAI_CREDENTIALS/VERTEXAI_PROJECT env; the test never mints a token.
Asserts both that the forward succeeds and that a costed SpendLogs row lands
(vertex_ai provider, a gemini model, spend > 0, call_type pass_through_endpoint),
correlated by the x-litellm-call-id header.
* Update tests/e2e/llm_translation/test_vertex_passthrough_e2e.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/e2e/llm_translation/test_vertex_passthrough_e2e.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
