mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
09a98f5505
The suite already waits for a new model or agent to become servable before handing it back, but that wait returns on the first successful read. Every request opens a fresh connection (e2e_http calls requests.* with no Session), so a load-balanced Service routes each one independently: one successful read proves one replica converged, and the caller's next request re-rolls and can land on a replica that has not reloaded yet. At replicaCount: 2 this surfaced as 30 failures on a SHA that is green at 1 replica -- 400 "Invalid model name passed", 404 "Guardrail not found", "no healthy deployments for this model", and a /model/info listing that contained one of two models created moments apart. Add PROPAGATION_TIMEOUT (default 15s, override E2E_PROPAGATION_TIMEOUT) and settle_propagation(), sized off the proxy's proxy_config_reload_interval_seconds (30s by default, 7s on the e2e stack) plus margin, and settle after every control-plane create whose object the suite then uses: - ProxyClient.create_model and A2AClient.register_agent, after their existing polls -- the poll still fails loudly if the object never appears at all - GuardrailsClient.register, which had no barrier; create_content_filter_guardrail and create_bedrock_guardrail now route through it instead of POSTing directly - the guardrail creates in mcp_client and logging_client - the vertex passthrough model, whose body cannot go through create_model Left alone: the /model/new calls that assert a 403 or read back a status code, since they never use the model. |
||
|---|---|---|
| .. | ||
| realtime | ||
| conftest.py | ||
| endpoints_client.py | ||
| LLM_TRANSLATION_COVERAGE_MATRIX.md | ||
| passthrough_client.py | ||
| test_audio_speech_e2e.py | ||
| test_audio_transcriptions_e2e.py | ||
| test_cache_control.py | ||
| test_chat_completions_regression_e2e.py | ||
| test_completions_endpoint_e2e.py | ||
| test_credential_messages_e2e.py | ||
| test_custom_pricing_e2e.py | ||
| test_deepseek_reasoning_e2e.py | ||
| test_embeddings_endpoint_e2e.py | ||
| test_image_edits_e2e.py | ||
| test_image_generation_e2e.py | ||
| test_messages_azure_foundry_e2e.py | ||
| test_messages_e2e.py | ||
| test_messages_mid_conversation_system_e2e.py | ||
| test_messages_mid_conversation_system_native_providers_e2e.py | ||
| test_moderations_e2e.py | ||
| test_ocr_rust_e2e.py | ||
| test_passthrough_e2e.py | ||
| test_passthrough_headers_e2e.py | ||
| test_provider_features_e2e.py | ||
| test_rerank_e2e.py | ||
| test_responses_bridge_streaming_e2e.py | ||
| test_responses_e2e.py | ||
| test_vertex_passthrough_e2e.py | ||