litellm/tests/e2e
mubashir1osmani 54d404ef2c
fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744)
* fix(e2e): wire batch provider secrets for docker and k8s

Point batch deployments at the credential field names and os.environ refs
the gateway actually resolves from process env (compose .env or EKS secret
mounts). Missing secrets skip instead of failing red so a red run means a
product bug. Mirror S3 bucket env aliases in docker-compose for provider_fallback

* fix(e2e): drop batch provider_env unit tests

The batches suite is live e2e only; no monkeypatch or unit-level tests

* fix: batch credentials, provider list, and team db lookup

Keep object-storage fields through CredentialLiteLLMParams and resolve
os.environ/ refs when reading deployment credentials so Vertex/Bedrock
batch file uploads see bucket and AWS keys from K8s/docker env

Skip managed batch list when the request is provider-scoped so
/{provider}/v1/batches list works instead of 500

Force DB on check_db_only team lookups and stop masking non-404 errors
as "team doesn't exist"

Drop e2e runner-side skip helpers; hard-fail on missing gateway secrets

* fix: tag reseed, team window spend, and remaining e2e flakes

Reseed spend:tag counters from LiteLLM_TagTable so cold redis still
enforces after the spend writer flushes

When applying post-call cost to team multi-window counters, load the
team from the DB if it is missing from the management cache so window
spend is not dropped on cache misses

Harden cold-counter reseed e2e (namespace-aware keys, burst success,
poll). Give tag budget more headroom. Retry /key/update on redis DNS
blips. Ensure NLTK punkt_tab is present for pipecat realtime audio

* revert: drop product code changes; e2e-only scope

Reverts all litellm/ and unit-test product edits. This branch is limited
to tests/e2e per contributor instruction

* fix(e2e): harden batch list and team member setup races

provider_fallback list falls back when managed batches reject provider
filtering. Team create waits for /team/info and member_add retries on
transient team-not-found so split control-plane lag does not red the suite

* fix(e2e): remove .env.example

Leave local .env and docker-compose env wiring as the secret source

* fix(e2e): wire files_settings and faster budget rescheduler for compose

OpenAI/Azure batch file uploads need files_settings; budget reset e2e needs a
short rescheduler window. Drop unsupported bedrock-encoded create_batch cells,
tolerate bedrock file.bytes=0, and surface team-info wait failures instead of
hanging silently

* chore(e2e): strip verbose comments from batch capabilities

* fix(e2e): assert managed list fallback before provider_fallback skip

When provider-scoped list is rejected, still fetch the unfiltered list and
check the envelope. Only skip membership when the id is a raw
provider_fallback batch that managed list cannot index
2026-07-10 11:31:40 -07:00
..
access_control test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
batches fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
budgets fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
coverage_registry test: emit e2e coverage lines for loki (#32513) 2026-07-08 11:06:56 -07:00
llm_translation fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
logging test(e2e): close coverage gaps across chat/responses, provider features, batches, prometheus, and langfuse eviction (#32165) 2026-07-04 18:56:52 -07:00
management fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
spend_tracking test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
bob_the_builder.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
CLAUDE.md test: emit e2e coverage lines for loki (#32513) 2026-07-08 11:06:56 -07:00
conftest.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
CONTRIBUTING.md fix(ui): scope key models dropdown options to the key's team (#32382) 2026-07-07 18:54:19 -07:00
docker-compose.yml fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
e2e_config.py fix(ui): scope key models dropdown options to the key's team (#32382) 2026-07-07 18:54:19 -07:00
e2e_gateway.py test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
e2e_http.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
models.py fix(e2e): batch credentials wiring and compose harness for live proxy suite (#32744) 2026-07-10 11:31:40 -07:00
pytest.ini test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_e2e_gateway.py test(e2e): make dynamic model provisioning robust on split deployments (#32670) 2026-07-09 13:39:47 -07:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00