litellm/tests/e2e/gateway
yujonglee 630c4624f6
test(e2e): add secret manager lanes for HashiCorp Vault and CyberArk Conjur (#42503)
* test(e2e): add a HashiCorp Vault secret manager lane

key_management_system had no end-to-end coverage: the Rust crates and the
Python unit tests all run against mocked managers. This adds a secret_manager
suite that drives a proxy configured with hashicorp_vault against a real Vault.

The tests seed a fresh secret name per test with the runner's OPENAI_API_KEY and
register a deployment pointing at os.environ/<name>. The proxy's env never holds
that name, so get_secret's os.environ fallback cannot mask a broken manager, and
a bogus value in Vault must come back as the provider's 401. Virtual keys are
checked written to and removed from Vault under prefix_for_stored_virtual_keys.

The setting is global to the proxy, so the lane has its own config and the
secret_manager_vault opt-in marker, and stays out of the per-PR selector.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(e2e): make the secret manager suite backend-agnostic

One marker and opt-in (secret_manager / E2E_SECRET_MANAGER=<system>) pick the
backend from secret_backends.BACKENDS. The tests reach the manager through a
SecretStore protocol, and each backend contributes a secret_store_<system>.py
module, a registry entry, and gateway/secret_manager_<system>_ci_config.yml.
requires_capability deselects tests a backend cannot support (CyberArk does
not delete), and test_secret_backends.py checks every lane config against its
backend without a live stack.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(e2e): add a CyberArk Conjur secret manager lane

Adds cyberark as the second secret_manager backend: a Conjur store over its
REST API (policy-declared variables, raw-text values, policy-patch teardown),
its lane config, and a registry entry without deletes_stored_keys, since the
proxy's CyberArk delete answers not_supported and Conjur keeps the key.

secret_manager/backend.sh up|down <system> boots any backend in Docker and
writes proxy.env and tests.env, so every lane runs the same way; the registry
test checks the script boots exactly the registered backends. e2e_http gains
send_text_external for APIs that speak raw text rather than JSON.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(e2e): give the secret manager suite a client with .proxy and address review

The shared resources fixture reads client.proxy, so a bare ProxyClient errored every
live test at setup. backend.sh now writes its env under a per-user directory with
umask 077, the markerless unit tests are gone per tests/e2e/AGENTS.md, and routine
comments are trimmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 18:02:32 -07:00
..
record_replay_ci_config.yml test(e2e): one request lands the same spend on every surface (#42540) 2026-09-22 13:42:00 -07:00
redis_chaos_ci_config.yml test(e2e): address Redis chaos PR review, add log-bytes budget 2026-09-11 12:40:25 -07:00
secret_manager_cyberark_ci_config.yml test(e2e): add secret manager lanes for HashiCorp Vault and CyberArk Conjur (#42503) 2026-09-22 18:02:32 -07:00
secret_manager_hashicorp_vault_ci_config.yml test(e2e): add secret manager lanes for HashiCorp Vault and CyberArk Conjur (#42503) 2026-09-22 18:02:32 -07:00
stage_mirror_ci_config.yml fix(proxy): share model rate-limit buckets between a model_group_alias and its target (#42516) 2026-09-22 13:31:58 -07:00