Follow-up to merged #26859 (team-callback audit log). The variant
scan from that PR flagged two more high-risk admin endpoints whose
mutations weren't audit-logged:
- ``POST /cache/settings`` (cache_settings_endpoints.py) — writes the
team / global Redis cache configuration, including credentials. An
admin (or compromised admin) flipping the cache backend is a
data-routing pivot — every subsequent LLM response cache write goes
to the new destination — so the action needs to be traceable.
- ``POST /config_overrides/hashicorp_vault`` and
``DELETE /config_overrides/hashicorp_vault``
(config_override_endpoints.py) — control the proxy's KMS config.
Mutating these affects every secret retrieval going forward.
Each endpoint now emits an ``LiteLLM_AuditLogs`` row gated on
``litellm.store_audit_logs`` (Enterprise feature), mirroring the
shape merged in #26859. Both helpers redact every field value
before serialization (replacing them with ``***REDACTED***`` while
keeping field names) so the audit table cannot itself become a
credential-harvest sink — Redis passwords / vault tokens /
``approle_secret_id`` / ``client_key`` would otherwise be persisted
in plaintext JSON.
Both helpers also attach a ``done_callback`` that surfaces a
``verbose_proxy_logger.warning`` when the fire-and-forget audit-log
task fails, so a transient DB error doesn't silently lose the row.
Adds ``CACHE_CONFIG_TABLE_NAME`` / ``CONFIG_OVERRIDES_TABLE_NAME`` to
``LitellmTableNames`` so the audit rows co-locate with the table they
mutate.
Tests:
- /cache/settings: emits when ``store_audit_logs=True``, no emission
when off, plaintext credentials don't appear in the serialized row.
- /config_overrides/hashicorp_vault POST: same, plus checks the
redaction of ``vault_token`` and ``vault_addr``.
- /config_overrides/hashicorp_vault DELETE: emits with ``action="deleted"``
only when an actual row was removed; idempotent delete of a non-existent
row produces no audit log.
When model_info.id equals model_name (common for batch models), the router
resolves via has_model_id and returns one deployment dict instead of a list.
The dict branch incorrectly iterated deployment keys (model_name,
litellm_params, model_info), producing non-string values that broke
LiteLLM_ManagedFileTable validation on managed file upload.
Normalize list vs dict by wrapping single deployments and extracting
model_info.id for each response pair.
Add regression tests including the batch model id == model_name case.
Made-with: Cursor
litellm's default LiteLLMAiohttpTransport routes requests through aiohttp,
which sits below httpx and is invisible to vcrpy's httpx-stub interception.
Under vcrpy + aiohttp, requests reach the real network but responses come
back through the stubbed httpx transport as empty 200s, surfacing as
'Unable to get json response - Expecting value: line 1 column 1 (char 0)'
in providers like Anthropic, Gemini, and any other path that exercises the
aiohttp transport.
Disabling the aiohttp transport when the VCR persister is registered
forces all calls through pure httpx, which vcrpy can record and replay
correctly.
The async/sync delete_response_api_handler always passed json=data into
httpx.delete, where data is {} from the transformer. httpx serializes that
to a 2-byte body. The Azure Responses DELETE endpoint now rejects any
request body with code: unexpected_body, breaking
test_basic_openai_responses_delete_endpoint on the llm_responses_api_testing
job. Build the kwargs dict and only set json= when data is truthy.
Add unit tests that patch httpx.delete and assert json/data are not in the
captured kwargs for the Azure DELETE path (sync and async).
Two Greptile review findings addressed:
1. (P1, security) The ``litellm_oauth_state`` cookie is the sole
guard against Login-CSRF in the PKCE flow but was set without the
``Secure`` attribute, so a network observer on plain HTTP could
read and replay it — bypassing the protection this PR adds.
Thread the originating ``Request`` down through
``get_sso_login_redirect`` and ``get_generic_sso_redirect_response``
and set ``Secure`` based on ``request.url.scheme == "https"``.
When no request is supplied (programmatic callers / tests) default
to ``Secure=True`` — production-safe. Local HTTP dev still works
because the request scheme is observed at runtime.
2. (P2) The cookie was set unconditionally, but the callback only
validates it inside the PKCE branch. Two concurrent SSO sessions
(one PKCE, one plain) could overwrite each other's state cookie
and produce spurious 400s for the plain-flow user.
Move the ``set_cookie`` call inside the existing
``if code_verifier and "state" in redirect_params`` block so the
cookie is only written when PKCE is active and the validation
will actually fire.
Tests cover both paths: PKCE-on (cookie set with Secure default),
PKCE-off (cookie not set), and HTTP dev request (Secure dropped so
the browser will actually attach the cookie on the callback hop).
``variant`` is user-controlled (passed through from
``litellm.video_content(variant=...)``) and was interpolated raw into
the URL query string. A value like ``thumbnail&extra=1`` would inject
additional query parameters into the upstream request — the same
class of issue this PR's path-segment encoding addresses. Wrap the
value in ``quote(value, safe="")`` so ``&`` / ``=`` / ``#`` cannot
terminate the ``variant`` value or open a new parameter.
Adds a regression test asserting that a malicious ``thumbnail&extra=1``
ends up percent-encoded in the URL, and that the legitimate
``thumbnail`` value still round-trips cleanly.
record_mode='once' refused to add new requests once any cassette
existed in Redis. Combined with filter_non_2xx_response (which drops
non-2xx responses from the saved cassette) and a 24h shared-Redis TTL,
a single transient API failure mid-test left the cassette stuck with
only the leading non-API requests (e.g. the model_prices fetch from
raw.githubusercontent.com), and every subsequent run for the next 24h
errored with 'Can't overwrite existing cassette'.
new_episodes records anything not already present, so partially
populated cassettes recover on the next run instead of poisoning the
suite for a full TTL window.
The Generic SSO PKCE flow used the URL ``state`` parameter as the
cache key for the PKCE ``code_verifier`` without binding the state
to the caller's browser. An attacker who pre-minted a state and
cached a verifier under it could hand the resulting login link to a
victim; the victim's auth code would then be exchanged with the
attacker's verifier on the callback, producing an access token
under the attacker's control (Login CSRF / token theft).
The non-PKCE branch is unaffected because it delegates to
fastapi-sso's ``verify_and_process``, which performs its own
session-cookie check. The PKCE branch bypasses that helper, which
is exactly the gap this commit closes.
Two-part fix in ``ui_sso.py``:
- ``get_generic_sso_redirect_response`` now sets a
``litellm_oauth_state`` cookie (HttpOnly, SameSite=Lax, 10-min TTL)
carrying the state value used in the redirect URL. The cookie is
set on the redirect response just like the existing
``litellm_cp_return_to`` cookie a few lines earlier in the file.
- ``get_generic_sso_response`` validates ``request.cookies.get(
"litellm_oauth_state")`` against ``request.query_params.get(
"state")`` via ``secrets.compare_digest`` before invoking the
PKCE token exchange. Mismatch (or either being missing) raises a
``ProxyException`` with HTTP 400.
The pre-existing TODO above the redirect logic ("state should be a
random string and added to the user session with cookie") is now
addressed and removed.
Tests cover the redirect-side cookie set, the missing-cookie reject
shape, the URL/cookie-mismatch reject shape, and the matching-cookie
happy path.
The proxy's ingress hardening (commit 842eea0131) now strips client-supplied
`mock_response` from the request body unless the calling key or team has the
`allow_client_mock_response: true` admin-metadata flag set. The e2e model
access tests rely on `mock_response` to short-circuit the LLM call, so without
the flag they hit real backends — the bedrock wildcard route fakes out to a
shared example endpoint that now 404s on unsupported paths, causing
`test_model_access_patterns[key_models2-bedrock/anthropic.claude-3-True]`
(and the bedrock/anthropic.* row that pytest -x never reaches) to fail.
Set `allow_client_mock_response: true` on every key and team this test file
provisions so `mock_response` is preserved end-to-end.
Provider SDKs already retry transient 5xx/429 with exponential backoff
(default max_retries=2), and pytest.mark.flaky covers test-level
retries on top of that. Setting litellm.num_retries=3 here just
multiplied the existing layers — worst case 6 (flaky) x 3 (this) x
2 (CI rerunfailures) = 36 attempts on a single test.
Removing it keeps SDK-level network-blip protection intact and
shortens worst-case latency on cache-miss runs.
Removes commentary that restated the code, including:
- module-level banners explaining what the conftest does (covered by
Readme.md and the function bodies)
- docstrings on _scrub_response, _before_record_response, vcr_config,
_vcr_disabled, pytest_recording_configure (function names + bodies
are self-evident)
- inline notes about header filtering, match_on, etc.
- per-test docstrings restating the test name
Keeps the two non-obvious notes that aren't recoverable from the code:
the vcrpy/respx httpx-transport collision rationale on
_RESPX_CONFLICTING_FILES, the vcrpy "return None to skip persisting"
contract on filter_non_2xx_response, and the fixture-ordering
dependency on _vcr_record_retries.
