mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
test(e2e): drop the two mgmt registry cells no shared-proxy test can cover
mgmt.cache_settings.update.happy_path and mgmt.config_override.hashicorp_vault.happy_path were the last two uncovered Management/UI cells, and neither can be covered against the shared proxy the e2e suites run on. Both routes reconfigure the whole process rather than a resource the test owns. /cache/settings persists whatever it receives into a row that outranks the YAML cache_params and is re-applied on a timer, so a partial write downgrades a TLS cluster to a plaintext standalone node and every later Redis call hangs. That is what took out 60 of 72 tests on 2026-07-25 and got the original test removed in PR #34664. /config_overrides/hashicorp_vault has the same shape: a POST sets the HCP_VAULT_* env vars, swaps litellm.secret_manager_client process-wide, and writes a row the config-reload poll re-applies, so every os.environ/ lookup on the pod resolves against the test's Vault until the DELETE lands. Its constructor also never dials Vault, so a POST to a bogus address still returns 200 and a smoke test built on it would pass for the wrong reason. Keeping rows we have decided not to cover only inflates the denominator, so drop them and record the reasoning where someone would go to write the test. Filing the isolated-proxy harness they both need separately; the cells come back with it. Management/UI goes 75/77 to 75/75, headline 402/544 to 402/542.
This commit is contained in:
parent
3fadcd7155
commit
f7accc4e29
2 changed files with 7 additions and 5 deletions
|
|
@ -64,14 +64,12 @@
|
|||
- {id: mgmt.budget.list_v1.happy_path, module: mgmt, tier: P1, surface: api, assertions: [happy_path], source: "management_v1/budgets.py:129", rationale: "Budget enumeration the Budgets page can page, sort and filter"}
|
||||
- {id: mgmt.budget.list_v1.admin_only, module: mgmt, tier: P1, surface: api, assertions: [admin_only], source: "management_v1/budgets.py:129", rationale: "A caller without admin view is refused, not served an empty page"}
|
||||
- {id: mgmt.callback.list.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "callback_management_endpoints.py", rationale: "Callback config (smoke)"}
|
||||
- {id: mgmt.cache_settings.update.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "cache_settings_endpoints.py", rationale: "Cache config (smoke). Deliberately uncovered: the previous test read the live settings and wrote them back, which proves nothing (identical values in, so a no-op POST still passes) while being able to break the deployment. /cache/settings persists what it receives and that row outranks YAML cache_params, re-applied on a timer, so a write that omits ssl or redis_startup_nodes turns a TLS cluster into a plaintext standalone node and every later Redis call hangs. That took out 60 of 72 tests on 2026-07-25. GET cannot round-trip it either: it resolves the stored row overlaid with REDIS_* env and never reads YAML, so on a fresh deploy it cannot see YAML ssl to echo back. A safe test needs an isolated proxy, or LIT-4816 fixed so a partial write cannot downgrade transport. Do not re-add a read-then-write-back test against a shared proxy."}
|
||||
- {id: mgmt.cost_tracking.estimate.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "cost_tracking_settings.py", rationale: "Cost estimate (smoke)"}
|
||||
- {id: mgmt.router_settings.update.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "router_settings_endpoints.py", rationale: "Router config (smoke)"}
|
||||
- {id: mgmt.jwt_key_mapping.new.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "jwt_key_mapping_endpoints.py", rationale: "JWT->key mapping (smoke)"}
|
||||
- {id: mgmt.compliance.gdpr.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "compliance_endpoints.py", rationale: "GDPR ops (smoke)"}
|
||||
- {id: mgmt.tool_management.list.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "tool_management_endpoints.py", rationale: "Tool inventory (smoke)"}
|
||||
- {id: mgmt.fallback_management.update.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "fallback_management_endpoints.py", rationale: "Fallback config (smoke)"}
|
||||
- {id: mgmt.config_override.hashicorp_vault.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "config_override_endpoints.py", rationale: "Vault integration (smoke)"}
|
||||
- {id: mgmt.workflow.list.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "workflow_management_endpoints.py", rationale: "Workflow tracking (smoke)"}
|
||||
- {id: mgmt.credential_migration.check.happy_path, module: mgmt, tier: P2, surface: api, assertions: [happy_path], source: "key_management_endpoints.py:4252", rationale: "Encryption migration (smoke)"}
|
||||
- {id: mgmt.credential.new.serves_request, module: mgmt, tier: P1, surface: api, assertions: [serves_request], source: "credential_endpoints/endpoints.py:42", rationale: "Stored credential resolves into a deployment and serves a live /messages request"}
|
||||
|
|
|
|||
|
|
@ -7,9 +7,13 @@ so a read-back reflects the change. Router settings, which mutate global proxy
|
|||
state, are exercised with a benign, self-restoring change so a shared proxy is left
|
||||
as it was found.
|
||||
|
||||
Cache settings are deliberately not covered here; see the rationale on
|
||||
mgmt.cache_settings.update.happy_path in coverage_registry/mgmt.yaml before adding
|
||||
a test for that route.
|
||||
Cache settings and the Vault config override are deliberately not covered here.
|
||||
Both routes reconfigure the whole proxy: /cache/settings persists what it receives
|
||||
into a row that outranks the YAML cache_params and is re-applied on a timer, and
|
||||
/config_overrides/hashicorp_vault swaps the process-wide secret manager. Neither can
|
||||
be exercised safely against the shared proxy the suites run on, so they need an
|
||||
isolated proxy before a test lands. Do not add a read-then-write-back test for
|
||||
either one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue