mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-05 08:07:05 +00:00
09a98f5505
The suite already waits for a new model or agent to become servable before handing it back, but that wait returns on the first successful read. Every request opens a fresh connection (e2e_http calls requests.* with no Session), so a load-balanced Service routes each one independently: one successful read proves one replica converged, and the caller's next request re-rolls and can land on a replica that has not reloaded yet. At replicaCount: 2 this surfaced as 30 failures on a SHA that is green at 1 replica -- 400 "Invalid model name passed", 404 "Guardrail not found", "no healthy deployments for this model", and a /model/info listing that contained one of two models created moments apart. Add PROPAGATION_TIMEOUT (default 15s, override E2E_PROPAGATION_TIMEOUT) and settle_propagation(), sized off the proxy's proxy_config_reload_interval_seconds (30s by default, 7s on the e2e stack) plus margin, and settle after every control-plane create whose object the suite then uses: - ProxyClient.create_model and A2AClient.register_agent, after their existing polls -- the poll still fails loudly if the object never appears at all - GuardrailsClient.register, which had no barrier; create_content_filter_guardrail and create_bedrock_guardrail now route through it instead of POSTing directly - the guardrail creates in mcp_client and logging_client - the vertex passthrough model, whose body cannot go through create_model Left alone: the /model/new calls that assert a 403 or read back a status code, since they never use the model. |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| datadog_reader.py | ||
| logging_client.py | ||
| test_datadog_log_e2e.py | ||
| test_otel_trace_e2e.py | ||
| test_prometheus_cardinality_e2e.py | ||