mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
587902becf
The secret manager lanes ran the proxy against a Vault dev container and a Conjur container that we start ourselves, so nothing in them needed a real third-party service. They have not run a single test since late October, because the co-located proxy cannot boot in the lane pod, and they were soft_fail outside the promotion gate, so they only added noise. The AWS test in the folder was skipped on every lane, since no aws lane ever existed. The four coverage registry rows go with the tests, because the registry is the e2e denominator and this behavior belongs in the integration tier if it gets covered again. The Vault and CyberArk clients keep their unit tests.
22 lines
2.7 KiB
INI
22 lines
2.7 KiB
INI
[pytest]
|
|
# Config when any e2e suite under tests/e2e/ is run directly, e.g.
|
|
# uv run pytest tests/e2e/quota_management/spend_tracking/ -v
|
|
# The e2e marker is also registered in conftest.py for runs rooted elsewhere.
|
|
addopts = --strict-markers --strict-config --reruns 1 --only-rerun "kind='network'" --only-rerun "status_code=5[0-9][0-9]"
|
|
markers =
|
|
e2e: live test that requires a running proxy and real provider keys
|
|
meta: typed e2e_metadata.Subject describing what this test drives (domain/route/providers/models/capabilities/mode); attach it with @meta(Subject(...)), never as a bare pytest.mark
|
|
replayable: edge-wired test whose provider traffic replays from a fixture bundle, so it makes zero provider calls in replay mode; the record/replay CI lane selects it with -m replayable
|
|
perf: performance test (throughput, latency, memory); collected last so it never perturbs latency-sensitive suites, and deselected unless E2E_PERF_TESTS is set
|
|
weekly: real-provider anomaly load test that spends real money; deselected unless E2E_WEEKLY_ANOMALY is set
|
|
managed_files: needs a proxy running with require_managed_files enabled; deselected unless E2E_MANAGED_FILES_STACK is set
|
|
prompt_caching_stack: needs a proxy running with router_settings.optional_pre_call_checks including prompt_caching; deselected unless E2E_PROMPT_CACHING_STACK is set
|
|
cli_determinism: drives the real claude CLI for several seconds; deselected unless E2E_CLI_DETERMINISM is set
|
|
redis_chaos: load test that pauses the proxy's Redis outright mid-run; needs a proxy booted from gateway/redis_chaos_ci_config.yml on the same host, and is deselected unless E2E_PERF_TESTS is set
|
|
provider_edge_host: routes provider traffic through the pytest host's edge in every fixture mode, so the gateway must reach the pytest host; deselected unless E2E_PROVIDER_EDGE_HOST_REACHABLE is set
|
|
owned_gateway: boots its own proxy from source against the stack's Postgres, so it needs DATABASE_URL on the pytest host; deselected unless E2E_OWNED_GATEWAY is set
|
|
otel_v2: needs a proxy running with LITELLM_OTEL_V2=true; deselected unless E2E_OTEL_V2 is set
|
|
otel_tls: needs a stack whose gateway exports OTLP over TLS signed by the CA in SSL_CERT_FILE; deselected unless E2E_OTEL_EXPORTER_ENDPOINT is set
|
|
sagemaker_nova: needs a deployed SageMaker Nova endpoint the proxy's AWS credentials can invoke; deselected unless SAGEMAKER_NOVA_ENDPOINT names it
|
|
sagemaker_nova2_lite: needs a deployed SageMaker Nova 2 Lite endpoint the proxy's AWS credentials can invoke; deselected unless SAGEMAKER_NOVA2_LITE_ENDPOINT names it
|
|
proxy_wide_config: turns on a proxy-wide setting through /config/update, so it runs on a stack no other suite is driving; deselected unless E2E_PROXY_WIDE_CONFIG_STACK is set
|