mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
53c9d48bd2
Adds a scheduled GitHub Actions lane on top of the merged record/replay transport. A Saturday cron records the `replayable` e2e tests against the real providers and publishes the fixture bundle as a private `e2e-fixtures-bundle` artifact with a SHA-256 sidecar. Weekday crons pull that artifact by its pinned digest, verify the checksum before extracting, and replay it with provider credentials set to bogus values, so a run that ever reached a real provider fails instead of passing. An egress sentinel pins the provider hostnames to a local sink for the whole replay job and counts every connection that reaches them; the job asserts that count is zero, so hermeticity is proven by measurement. A red Saturday publishes no bundle, so the next weekday finds nothing fresh and fails loudly rather than replaying a week-old recording, and the transport's seven-day freshness gate hard-fails any bundle that has drifted too far. The lane also runs on demand from the Actions tab with a record/replay `mode` input. Tests join the lane with `@pytest.mark.replayable`. The streaming Anthropic test now counts to twenty so its recorded response banks several content deltas, matching the assertion that the stream arrives incrementally. |
||
|---|---|---|
| .. | ||
| realtime | ||
| conftest.py | ||
| endpoints_client.py | ||
| LLM_TRANSLATION_COVERAGE_MATRIX.md | ||
| passthrough_client.py | ||
| test_audio_speech_e2e.py | ||
| test_audio_transcriptions_e2e.py | ||
| test_bedrock_native_e2e.py | ||
| test_bedrock_web_search_server_tool_e2e.py | ||
| test_cache_control.py | ||
| test_chat_completions_contract_e2e.py | ||
| test_chat_completions_regression_e2e.py | ||
| test_chat_stream_contract_e2e.py | ||
| test_completions_endpoint_e2e.py | ||
| test_credential_messages_e2e.py | ||
| test_custom_pricing_e2e.py | ||
| test_deepseek_reasoning_e2e.py | ||
| test_embeddings_endpoint_e2e.py | ||
| test_files_batches_contract_e2e.py | ||
| test_google_native_e2e.py | ||
| test_image_edits_e2e.py | ||
| test_image_generation_e2e.py | ||
| test_messages_azure_foundry_e2e.py | ||
| test_messages_e2e.py | ||
| test_messages_mid_conversation_system_e2e.py | ||
| test_messages_mid_conversation_system_native_providers_e2e.py | ||
| test_moderations_e2e.py | ||
| test_ocr_rust_e2e.py | ||
| test_passthrough_e2e.py | ||
| test_passthrough_headers_e2e.py | ||
| test_provider_features_e2e.py | ||
| test_realtime_http_e2e.py | ||
| test_rerank_e2e.py | ||
| test_responses_bridge_streaming_e2e.py | ||
| test_responses_e2e.py | ||
| test_responses_retrieve_e2e.py | ||
| test_vector_stores_e2e.py | ||
| test_vertex_passthrough_e2e.py | ||