mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-11 22:51:28 +00:00
53c9d48bd2
Adds a scheduled GitHub Actions lane on top of the merged record/replay transport. A Saturday cron records the `replayable` e2e tests against the real providers and publishes the fixture bundle as a private `e2e-fixtures-bundle` artifact with a SHA-256 sidecar. Weekday crons pull that artifact by its pinned digest, verify the checksum before extracting, and replay it with provider credentials set to bogus values, so a run that ever reached a real provider fails instead of passing. An egress sentinel pins the provider hostnames to a local sink for the whole replay job and counts every connection that reaches them; the job asserts that count is zero, so hermeticity is proven by measurement. A red Saturday publishes no bundle, so the next weekday finds nothing fresh and fails loudly rather than replaying a week-old recording, and the transport's seven-day freshness gate hard-fails any bundle that has drifted too far. The lane also runs on demand from the Actions tab with a record/replay `mode` input. Tests join the lane with `@pytest.mark.replayable`. The streaming Anthropic test now counts to twenty so its recorded response banks several content deltas, matching the assertion that the stream arrives incrementally. |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| cost_rows.py | ||
| spend_e2e_client.py | ||
| SPEND_TRACKING_COVERAGE_MATRIX.md | ||
| test_cache_cost_accounting_e2e.py | ||
| test_cost_headers_e2e.py | ||
| test_provider_edge_spend_e2e.py | ||
| test_service_tier_pricing_e2e.py | ||
| test_spend_routes.py | ||
| test_spend_tracking_e2e.py | ||
| test_team_daily_activity_e2e.py | ||