litellm/tests/code_coverage_tests
Yassin Kortam 8e65265bb4
fix(agents): redact secret litellm_params fields from all /v1/agents responses (#39389)
* fix(agents): redact secret litellm_params fields from all /v1/agents responses

Secret-bearing litellm_params fields (aws_secret_access_key, api_key, and
similar) are now write-only: list, get, create, update, and patch
responses always replace them with a fixed marker, regardless of caller
role. Editing an agent no longer requires resending a real credential --
an update that omits a sensitive field, or echoes the marker back,
preserves the stored value; a real value still rotates it.

* fix(agents): redact secrets nested inside dicts/lists in litellm_params too

Greptile found that a secret nested one level down under a
non-sensitively-named key, or inside a list of per-provider configs, was
neither redacted on read nor restored symmetrically on write (the marker
string could get persisted as the real value). Recurse into lists on the
read side, and mirror that recursion on the write side so restoration
isn't limited to top-level keys. Also fixes a regression the redact
rewrite introduced (a plain string leaf like a model name was being
misinterpreted as a JSON blob and redacted), and suppresses 3 new
test-quality-gate findings on an established repo-wide mocking pattern
this PR's new tests also use.

* fix(agents): guard list-position credential restore against misassignment

Two more real gaps Greptile/veria found in the recursive redact/restore
mechanism, verified directly against the exact reported shape
(litellm_params.model_list, each entry carrying its own nested
litellm_params.api_key/aws_secret_access_key) before fixing:

- Positional restoration inside a list could attach one entry's stored
  credential to a different entry if the list were reordered or resized
  between GET and PUT/PATCH. Restoration by index now only fires when the
  incoming and existing entries match on every non-secret field; otherwise
  the caller's own value is used (never a guessed cross-entry secret).
- A subtree collapsed to the flat REDACTED_BY_LITELM marker by the
  read-side recursion depth cap couldn't be recovered on write (the marker
  string itself would get persisted). Restore now recognizes that shape and
  recovers the whole existing subtree.

Both covered by regression tests mirroring the exact model_list shape
reported, mutation-verified.

* fix(agents): simplify list-entry credential restore to positional matching

The content-match guard from the previous commit fixed one Greptile
finding (cross-entry misassignment on reorder) but introduced a worse one:
it also rejected restoration whenever an entry's own non-secret fields
changed, which is the common case (rename a model_list entry while
leaving its own secret masked) -- silently dropping the stored credential
on an ordinary edit.

There is no stable per-element identity in a plain dict[str, object]
schema, so no rule can satisfy both 'restore whenever the entry itself
only had its secret masked' and 'never restore across a reorder' at once.
Positional correspondence is what every other part of this restore (and
the endpoints' full-replace-on-PUT semantics) already assumes, so drop
the content-match gate and rely on it here too: this fixes the common
case correctly and accepts cross-entry misassignment on a simultaneous
reorder-plus-masked-echo as a known, narrow, documented limitation (not a
leak between different agents or tenants, since it only reshuffles one
agent's own stored values). Tests updated to pin the accepted trade-off
explicitly rather than asserting it away, and to cover the previously
broken ordinary-edit case.
2026-09-02 14:28:44 -07:00
..
azure_client_usage_test.py fix - correctly re-use azure openai client 2025-03-18 09:51:28 -07:00
ban_constant_numbers.py Squashed commit of the following: (#9709) 2025-04-02 21:24:54 -07:00
ban_copy_deepcopy_kwargs.py Fix - using managed files w/ OTEL + UI - add model group alias on UI (#13171) 2025-07-31 21:22:04 -07:00
bedrock_pricing.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
callback_manager_test.py (Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112) 2025-01-30 19:35:50 -08:00
check_data_replace_usage.py Bug fix - String data: stripped from entire content in streamed Gemini responses (#9070) 2025-03-07 21:06:39 -08:00
check_e2e_no_raw_requests.py test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers 2026-07-22 23:29:10 -07:00
check_endpoint_coverage.py Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
check_fastuuid_usage.py code cov test script check_fastuuid_usage.py 2025-09-24 10:27:22 +09:00
check_get_model_cost_key_performance.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
check_guardrail_apply_decorator.py content filter test fix 2026-02-12 17:54:16 -08:00
check_licenses.py refactor(ci): simplify PyPI license retries 2026-08-23 09:23:35 +00:00
check_migrations_no_data_rewrites.py fix: keep a schema-qualified call inside an index expression from reading as a relation 2026-08-24 16:04:38 -07:00
check_prisma_binary_cache.py fix(ci): bound setup steps so pytest always gets its full budget 2026-08-10 16:22:07 +00:00
check_provider_folders_documented.py Revert "[Feature] Add /public/supported_endpoints endpoint" 2026-02-26 17:21:43 -08:00
check_spanattributes_value_usage.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
check_unsafe_enterprise_import.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
check_workflow_startup_safety.py fix(ci): report budgets the startup guard cannot resolve 2026-08-10 18:12:18 +00:00
code_qa_check_tests.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
enforce_llms_folder_style.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ensure_async_clients_test.py fix code QA check 2025-10-07 17:49:57 -07:00
info_log_check.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
liccheck.ini feat(rust): 1:1 port of OpenAI Responses API WebSockets to litellm-rust (#33849) 2026-07-19 01:55:35 +00:00
license_cache.json fix(deps): update python-multipart to >=0.0.20 in CI and test configs 2026-03-03 15:10:39 -03:00
litellm_logging_code_coverage.py docs(litellm_logging_code_coverage.py): fix check 2025-06-18 21:36:03 -07:00
log.txt Squashed commit of the following: (#9709) 2025-04-02 21:24:54 -07:00
memory_test.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
pass_through_code_coverage.py test: initial commit enforcing testing on all anthropic pass through … (#7794) 2025-01-15 22:02:35 -08:00
prevent_key_leaks_in_exceptions.py fix(main.py): fix key leak error when unknown provider given (#8556) 2025-02-15 14:02:55 -08:00
recursive_detector.py fix(agents): redact secret litellm_params fields from all /v1/agents responses (#39389) 2026-09-02 14:28:44 -07:00
router_code_coverage.py test: exempt _resolved_provider in router_code_coverage gate 2026-08-31 12:19:51 -07:00
router_enforce_line_length.py Litellm router code coverage 3 (#6274) 2024-10-16 21:30:25 -07:00
test_aio_http_image_conversion.py fix img URL for tests 2025-11-22 09:41:15 -08:00
test_ban_set_verbose.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_chat_completion_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_no_hardcoded_secrets.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_proxy_types_import.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_strategy_async.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
user_api_key_auth_code_coverage.py test: initial test to enforce all functions in user_api_key_auth.py h… (#7797) 2025-01-15 21:52:45 -08:00