mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
0ed1c08f02
* feat(anthropic): workload identity federation and pluggable identity sources Backend half of #38818 (internal copy of the fork PR #38013), rebuilt as one commit on top of litellm_internal_staging without the dashboard changes. Deployments on anthropic/ without a static api_key can exchange an OIDC workload assertion for a short-lived sk-ant-oat01 token through a shared RFC 7523 JWT-bearer engine. The assertion comes from a mounted token file, an env token, a LiteLLM-signed issuer, or Keycloak, chosen per deployment, per named credential, or through ANTHROPIC_IDENTITY_SOURCE. The federation fields are server-owned: refused inline in request bodies and on POST /model/new, proxy-admin only on credentials, and the token exchange is pinned to api.anthropic.com unless LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS adds a host. GET /credentials/{name}/jwks exports the public key set of a LiteLLM-signed credential for the Claude Console. The OpenAI federation trio from #39613 rides along on the backend side with the same server-owned handling. Fixes #28607 Resolves LIT-6107 Co-authored-by: derhornspieler <15236687+derhornspieler@users.noreply.github.com> * fix(anthropic): let batch-result downloads mint from deployment params and accept host:port allowlist entries The files handler enabled workload identity on batch-result downloads but never received the deployment's litellm_params, so a deployment authenticating through a named credential could only mint from process-wide env vars. It now threads litellm_params through to the auth header the way the batch retrieve path already does. LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS entries written as host:port were read by urlsplit as a scheme, so the allowlist kept the raw entry while the exchange compared bare hostnames and refused the gateway. Entries are now parsed as network locations whether or not they carry a scheme. * fix(types): move the WIF kwargs key sets to a leaf module so the kwargs funnel imports without a cycle * test(anthropic): pin case-insensitive matching of WIF exchange-host allowlist entries * fix(anthropic): end workload identity federation errors without a period so the router suffix reads cleanly * fix(proxy): decrypt stored litellm_params before the WIF write gate * fix(proxy): hide WIF secret references from /health output * fix(proxy): keep the proxy error shape on credential endpoint refusals * fix(proxy): hide identity token file paths from /health output * fix(anthropic): rename the federation workspace param so Bedrock's anthropic_workspace_id keeps working The Bedrock Claude Platform route already reads anthropic_workspace_id from optional_params, so banning that spelling as a server-owned federation parameter broke a pre-existing client capability. The federation field is now anthropic_federation_workspace_id (env ANTHROPIC_FEDERATION_WORKSPACE_ID), which restores the base branch's behavior for Bedrock callers, drops the Bedrock-specific hint from the refusal message, and deletes the unconditional ban constant that no longer had a reader * fix(auth): share one exchanged token across workers reading the same assertion Anthropic accepts each identity assertion exactly once, so two uvicorn workers reading the same token file both minting from it means the second exchange is denied with jti_reused. Minted tokens now land in a per-user 0700 cache directory guarded by a file lock, so workers on the same host reuse one exchange until the token expires or the assertion rotates. A 401 is only retried when the re-read assertion actually differs, and the denial hint explains jti_reused. LITELLM_TOKEN_EXCHANGE_CACHE_DIR moves the cache and an empty value disables it * fix: keep anthropic federation from being shadowed or leaked An empty or whitespace-only ANTHROPIC_API_KEY counted as set, so a federated deployment sent an empty x-api-key on every call instead of minting a token. Blank values now read as unset, and a real static key on a federated deployment logs once that it outranks federation and nothing is being federated. The exchange-host allowlist matched hostnames only, so a second process on another port of an allowed host was trusted with the workload's identity token. An entry that names a port now trusts that port alone, while a bare host still trusts every port. The shared token store exists so the workers reading one projected token file do not each spend its single-use jti. A source that mints its own assertion per exchange shares nothing with another worker, so it no longer writes a live token to disk for a lookup that can never hit. * fix: unlink a staged token file a failed write leaves behind The 401 denial hint now also says federation ignores ANTHROPIC_WORKSPACE_ID, which the Bedrock Claude platform provider already reads. * refactor: move anthropic jwks derivation behind a provider-owned tagged union * fix: unlink the staged token file when its write fails at close A buffered write only reaches the disk when the handle closes, so a full disk surfaces at close and left the staging file behind holding a usable token. * fix(anthropic): close the staging descriptor before writing the shared token file * fix(wif): judge federation writes by what they set, not what is stored The admin gate read the stored deployment, so a team admin lost edit, delete and Test Connection on any deployment carrying federation params. It now returns early unless the submitted fields touch the federation surface, and a Test Connection probe that points the deployment at its own api_base is still refused, with the 403 no longer wrapped into a 500 The rest of the same review pass: POST /model/new refuses only a blocking value of `blocked`, so a client that always sends `blocked: false` is not turned away; a request body can no longer pick which federated identity to mint as by naming a stored credential; an advisory refresh the executor refuses disarms the entry instead of wedging the identity until the follower timeout; the static-key shadow warning resolves its env fallback inside the cache instead of once per request; credential writes drop nulls before storing them; the token exchange validates the endpoint URL before reading an assertion and keeps refusing redirects across a client heal; /health hides every server-owned federation field from non-admins; and the async create_file and create_batch paths say which setting is missing when the provider resolves no URL * fix(proxy): let a deployment write name a federated credential reject_federated_credential_reference runs from is_request_body_safe, which pre_db_read_auth_checks calls on every route, so it also fired on POST /model/new, /model/update, /model/{id}/update and /health/test_connection. A proxy admin could no longer attach a federated credential to a deployment over the API or the Admin UI, leaving a static config.yaml entry as the only way to configure the feature the rejection told the caller to go configure, and _reject_non_admin_wif_write never got to make the call it exists to make. is_request_body_safe now takes the route and skips only the credential-reference check on the routes that reach can_user_make_model_call. Federation fields typed inline into a body stay refused everywhere, and a call naming a federated credential still cannot pick the identity it mints as. * refactor(proxy): derive health display policy from the federation key sets The health check module hand-copied the five workload identity fields whose value is a credential, so a shared proxy surface named provider-specific parameters and a newly added secret-bearing field would have gone on being displayed until someone remembered both places WIF_SECRET_BEARING_KEYS now sits beside the key sets it splits out of, types/utils derives secret_bearing_wif_litellm_params from it, and the health layer splats that tuple the same way it already splats the admin-only one * fix(anthropic_wif): treat blank identity-source fields as unset * test(proxy): classify the federation params in the credential slot registry main's registry test (#43298) now fails the build for any credential-named deployment param without a classification. The five federation fields that carry a token, a token file path, or a signing or client secret reference are Unplanted, matching WIF_SECRET_BEARING_KEYS; the four remaining Keycloak settings name a URL, a client id, an auth method, or a scope and are NotSecret * fix(anthropic_wif): declare federation params as owned connection leaves and chart their metrics Register the 18 Anthropic and 3 OpenAI federation params as frozen ConnectionSettings leaves so the owned-kwarg registry, the kwargs funnel and the request-body ban list read one declaration. Pass the deployment api_base through to the count-tokens handler instead of a pre-suffixed URL, which doubled the /count_tokens path on main's prompt-cache predictor. Add the five litellm_anthropic_wif_* families to the all-metrics Grafana dashboard. * fix(credentials): gate PATCH on WIF fields resolved from model_id The credential PATCH handler checked server-owned workload identity federation fields only on the values the caller sent, while a body that named a deployment through model_id had its credential values resolved after that check. A non-admin could therefore copy a federated deployment's WIF fields onto an ordinary credential. Resolve the incoming values first and run the non-admin gate on them, matching the POST path * fix(anthropic): count tokens with ANTHROPIC_AUTH_TOKEN through the shared auth header Count-tokens walked its own credential ladder: a static key, else skip minting when ANTHROPIC_AUTH_TOKEN is set, else mint a federated token. With only the auth token set it forwarded nothing and the proxy silently fell back to its local tokenizer while chat on the same deployment authenticated with that token. The handler now takes the auth header that AnthropicModelInfo.aget_auth_header resolves, the same ladder chat, files, batches and skills use, and merges the oauth beta a minted or consumer token carries with the token-counting beta --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: derhornspieler <15236687+derhornspieler@users.noreply.github.com> Co-authored-by: mateo-berri <happymvw@gmail.com>
1401 lines
64 KiB
Python
1401 lines
64 KiB
Python
"""Tests for the credential management endpoints."""
|
|
|
|
import json
|
|
from contextlib import contextmanager
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
import pytest
|
|
from cryptography.hazmat.primitives import serialization
|
|
from cryptography.hazmat.primitives.asymmetric import ec
|
|
from fastapi.testclient import TestClient
|
|
|
|
import litellm
|
|
from litellm.proxy._types import UserAPIKeyAuth
|
|
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
|
|
from litellm.proxy.credential_endpoints.endpoints import get_llm_router
|
|
from litellm.proxy.proxy_server import app
|
|
from litellm.types.utils import CredentialItem
|
|
|
|
client = TestClient(app)
|
|
|
|
|
|
def _as_admin():
|
|
return UserAPIKeyAuth(api_key="test-key", user_role="proxy_admin")
|
|
|
|
|
|
def _as_non_admin():
|
|
return UserAPIKeyAuth(api_key="test-key", user_role="internal_user")
|
|
|
|
|
|
def _call_as(method: str, path: str, json_body: dict | None = None, auth=_as_admin):
|
|
missing = object()
|
|
previous_override = app.dependency_overrides.get(user_api_key_auth, missing)
|
|
app.dependency_overrides[user_api_key_auth] = auth
|
|
try:
|
|
return client.request(method, path, json=json_body, headers={"Authorization": "Bearer test-key"})
|
|
finally:
|
|
if previous_override is missing:
|
|
app.dependency_overrides.pop(user_api_key_auth, None)
|
|
else:
|
|
app.dependency_overrides[user_api_key_auth] = previous_override
|
|
|
|
|
|
def _patch_credential(name: str, body: dict, auth=_as_admin):
|
|
return _call_as("PATCH", f"/credentials/{name}", body, auth)
|
|
|
|
|
|
def _post_credential(body: dict, auth=_as_admin):
|
|
return _call_as("POST", "/credentials", body, auth)
|
|
|
|
|
|
def _delete_credential(name: str, auth=_as_admin):
|
|
return _call_as("DELETE", f"/credentials/{name}", auth=auth)
|
|
|
|
|
|
def _list_credentials():
|
|
return _call_as("GET", "/credentials")
|
|
|
|
|
|
def _prisma_without_credential_rows() -> MagicMock:
|
|
prisma_client = MagicMock()
|
|
prisma_client.db.litellm_credentialstable.find_unique = AsyncMock(return_value=None)
|
|
return prisma_client
|
|
|
|
|
|
@pytest.fixture
|
|
def credential_store():
|
|
"""Stands the credential store up for one test: whether the database is reachable, what
|
|
the proxy is already serving from memory, which router deployments resolve against, and
|
|
what each repository call hands back."""
|
|
|
|
def install(
|
|
*,
|
|
connected: bool = True,
|
|
in_memory: tuple[object, ...] = (),
|
|
llm_router: object | None = None,
|
|
**repository_calls: AsyncMock,
|
|
) -> None:
|
|
patch("litellm.proxy.proxy_server.prisma_client", _prisma_without_credential_rows() if connected else None).start()
|
|
patch("litellm.proxy.proxy_server.master_key", "sk-test-master").start()
|
|
patch.object(litellm, "credential_list", list(in_memory)).start()
|
|
app.dependency_overrides[get_llm_router] = lambda: llm_router
|
|
repository = patch("litellm.proxy.credential_endpoints.endpoints.CredentialsRepository").start()
|
|
repository.return_value.find_by_name = AsyncMock(return_value=None)
|
|
for call_name, result in repository_calls.items():
|
|
setattr(repository.return_value, call_name, result)
|
|
|
|
yield install
|
|
patch.stopall()
|
|
app.dependency_overrides.pop(get_llm_router, None)
|
|
|
|
|
|
@contextmanager
|
|
def _repository_holding(stored: CredentialItem | None):
|
|
"""The credentials repository seam, answering ``find_by_name`` with ``stored`` and recording
|
|
the writes the handler attempts. Patched at both import sites, since the handlers resolve an
|
|
existing credential through ``hydrate_named_credential`` (memory first, then this repository)
|
|
and then write through their own ``CredentialsRepository`` binding."""
|
|
with (
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.master_key", "sk-test-master"
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.credential_endpoints.endpoints.CredentialsRepository"
|
|
) as repository,
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.common_utils.credential_hydration.CredentialsRepository", repository
|
|
),
|
|
):
|
|
repository.return_value.find_by_name = AsyncMock(return_value=stored)
|
|
repository.return_value.create = AsyncMock(return_value=None)
|
|
repository.return_value.update_by_name = AsyncMock(return_value=None)
|
|
repository.return_value.delete_by_name = AsyncMock(return_value=stored)
|
|
yield repository.return_value
|
|
|
|
|
|
def test_create_credential_write_omits_the_patch_only_deletion_field(restore_credential_list):
|
|
"""Regression: CredentialItem.credential_values_to_delete is a PATCH-only field that
|
|
defaults to None on every other construction path. A bare .model_dump() (without
|
|
exclude_none) on the create path put a `credential_values_to_delete: null` key into the
|
|
Prisma write, which litellm_credentialstable has no column for."""
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "new-cred",
|
|
"credential_values": {"api_key": "sk-new"},
|
|
"credential_info": {"custom_llm_provider": "openai"},
|
|
}
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_data = repository.create.await_args.kwargs["data"]
|
|
assert "credential_values_to_delete" not in written_data
|
|
|
|
|
|
def test_update_credential_answers_404_when_the_credential_does_not_exist(credential_store):
|
|
"""Regression: the handler used to ``return handle_exception_on_proxy(e)``, which makes
|
|
the exception the response body and lets FastAPI answer 200, so a write the handler
|
|
rejected read as a success to every caller that checks the status. The dashboard's API
|
|
client branches on the status, so it reported a failed edit as applied."""
|
|
credential_store(find_by_name=AsyncMock(return_value=None))
|
|
|
|
response = _patch_credential(
|
|
"definitely-not-there",
|
|
{"credential_name": "definitely-not-there", "credential_values": {"api_key": "sk-x"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 404, f"rejected write answered {response.status_code}: {response.text}"
|
|
assert "error" in response.json()
|
|
|
|
|
|
def test_update_credential_answers_500_when_the_database_is_not_connected(credential_store):
|
|
"""The other rejection this handler raises must carry its own status too."""
|
|
credential_store(connected=False)
|
|
|
|
response = _patch_credential(
|
|
"any-name",
|
|
{"credential_name": "any-name", "credential_values": {"api_key": "sk-x"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 500, f"rejected write answered {response.status_code}: {response.text}"
|
|
|
|
|
|
def test_update_credential_still_answers_200_on_a_successful_write(credential_store):
|
|
"""The fix must not turn a legitimate update into an error; the dashboard and the
|
|
Playwright credentials spec both assert the success path."""
|
|
stored = CredentialItem(
|
|
credential_name="existing",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "openai"},
|
|
)
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=AsyncMock(return_value=None))
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{"credential_name": "existing", "credential_values": {"api_key": "sk-new"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
assert response.json()["success"] is True
|
|
|
|
|
|
def _get_jwks(name: str):
|
|
return _call_as("GET", f"/credentials/{name}/jwks")
|
|
|
|
|
|
@pytest.fixture
|
|
def restore_credential_list(monkeypatch):
|
|
monkeypatch.setattr(litellm, "credential_list", [])
|
|
|
|
|
|
def test_update_credential_rejects_overlap_between_update_and_delete():
|
|
"""A key in both sets is ambiguous (set to what value, before or after the delete?), so the
|
|
endpoint must reject it outright rather than picking a resolution order silently."""
|
|
response = _patch_credential(
|
|
"any-name",
|
|
{
|
|
"credential_name": "any-name",
|
|
"credential_values": {"api_key": "sk-new"},
|
|
"credential_values_to_delete": ["api_key"],
|
|
"credential_info": {},
|
|
},
|
|
)
|
|
|
|
assert response.status_code == 400, response.text
|
|
assert "api_key" in response.json()["error"]["message"]
|
|
|
|
|
|
def test_update_credential_deletion_removes_the_key_from_the_db_write(restore_credential_list):
|
|
"""The bug this closes: switching WIF identity sources (or WIF -> api_key) left the old
|
|
variant's fields behind in the DB row, which wif.py then rejects by presence."""
|
|
stored = CredentialItem(
|
|
credential_name="wif-cred",
|
|
credential_values={"anthropic_identity_source": "keycloak", "anthropic_keycloak_client_id": "old-client"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with (
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.credential_endpoints.endpoints.CredentialsRepository"
|
|
) as repository, # test-quality-ok: the proxy wiring under test is what this patches
|
|
):
|
|
repository.return_value.find_by_name = AsyncMock(return_value=stored)
|
|
update_mock = AsyncMock(return_value=None)
|
|
repository.return_value.update_by_name = update_mock
|
|
|
|
response = _patch_credential(
|
|
"wif-cred",
|
|
{
|
|
"credential_name": "wif-cred",
|
|
"credential_values": {},
|
|
"credential_values_to_delete": ["anthropic_keycloak_client_id"],
|
|
"credential_info": {},
|
|
},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_values = json.loads(update_mock.await_args.kwargs["data"]["credential_values"])
|
|
assert "anthropic_keycloak_client_id" not in written_values
|
|
assert written_values["anthropic_identity_source"] == "keycloak"
|
|
|
|
|
|
def test_update_credential_deletion_updates_in_memory_credential_list(restore_credential_list, monkeypatch):
|
|
"""The in-memory list is what the request-time auth resolvers read; a deletion that only
|
|
landed in the DB would leave the stale field servable until the next process restart."""
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="wif-cred",
|
|
credential_values={
|
|
"anthropic_identity_source": "keycloak",
|
|
"anthropic_keycloak_client_id": "old-client",
|
|
},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
],
|
|
)
|
|
stored = CredentialItem(
|
|
credential_name="wif-cred",
|
|
credential_values={"anthropic_identity_source": "keycloak", "anthropic_keycloak_client_id": "old-client"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with (
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.credential_endpoints.endpoints.CredentialsRepository"
|
|
) as repository, # test-quality-ok: the proxy wiring under test is what this patches
|
|
):
|
|
repository.return_value.find_by_name = AsyncMock(return_value=stored)
|
|
repository.return_value.update_by_name = AsyncMock(return_value=None)
|
|
|
|
response = _patch_credential(
|
|
"wif-cred",
|
|
{
|
|
"credential_name": "wif-cred",
|
|
"credential_values": {},
|
|
"credential_values_to_delete": ["anthropic_keycloak_client_id"],
|
|
"credential_info": {},
|
|
},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
in_memory = next(c for c in litellm.credential_list if c.credential_name == "wif-cred")
|
|
assert "anthropic_keycloak_client_id" not in in_memory.credential_values
|
|
assert in_memory.credential_values["anthropic_identity_source"] == "keycloak"
|
|
|
|
|
|
def test_update_credential_leaves_untouched_fields_alone():
|
|
"""Regression for the masked-value hazard: GET /credentials masks values, so a PATCH that
|
|
only names the field being changed must not let an untouched field be nulled or overwritten
|
|
by anything a round-tripped (masked) form value could contain."""
|
|
stored = CredentialItem(
|
|
credential_name="existing",
|
|
credential_values={"api_key": "sk-real-value", "api_base": "https://api.anthropic.com"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with (
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.master_key", "sk-test-master"
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.credential_endpoints.endpoints.CredentialsRepository"
|
|
) as repository, # test-quality-ok: the proxy wiring under test is what this patches
|
|
):
|
|
repository.return_value.find_by_name = AsyncMock(return_value=stored)
|
|
update_mock = AsyncMock(return_value=None)
|
|
repository.return_value.update_by_name = update_mock
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{"credential_name": "existing", "credential_values": {"api_key": "sk-rotated"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_values = json.loads(update_mock.await_args.kwargs["data"]["credential_values"])
|
|
assert written_values["api_base"] == "https://api.anthropic.com"
|
|
|
|
|
|
def test_create_credential_never_stores_a_null_credential_value(restore_credential_list):
|
|
"""The dashboard posts a key for every field on the provider's form, and the ones the operator
|
|
left blank arrive as null. A null carries no credential, and the federation resolver refuses a
|
|
foreign variant's field by key, so a stored null wedges every deployment naming this credential."""
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "new-cred",
|
|
"credential_values": {"api_key": "sk-new", "anthropic_issuer_url": None},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
}
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_values = json.loads(repository.create.await_args.kwargs["data"]["credential_values"])
|
|
assert "anthropic_issuer_url" not in written_values
|
|
assert "api_key" in written_values
|
|
|
|
|
|
def test_update_credential_never_stores_a_null_credential_value(restore_credential_list):
|
|
"""Same null on the update path, where the merge writes the whole row back: the field the null
|
|
named keeps whatever it stored, since removing a field is what credential_values_to_delete is for."""
|
|
stored = CredentialItem(
|
|
credential_name="wif-cred",
|
|
credential_values={"anthropic_identity_source": "keycloak", "anthropic_keycloak_client_id": "old-client"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with _repository_holding(stored) as repository:
|
|
response = _patch_credential(
|
|
"wif-cred",
|
|
{
|
|
"credential_name": "wif-cred",
|
|
"credential_values": {"anthropic_keycloak_client_id": None},
|
|
"credential_info": {},
|
|
},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_values = json.loads(repository.update_by_name.await_args.kwargs["data"]["credential_values"])
|
|
assert written_values["anthropic_keycloak_client_id"] == "old-client"
|
|
|
|
|
|
def test_update_credential_never_syncs_a_null_into_the_in_memory_credential(restore_credential_list, monkeypatch):
|
|
"""The in-memory list is what request-time resolution reads, so a null that only got kept out of
|
|
the DB row would still wedge every deployment until the next restart."""
|
|
in_memory = CredentialItem(
|
|
credential_name="plain-cred",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
monkeypatch.setattr(litellm, "credential_list", [in_memory])
|
|
with _repository_holding(
|
|
CredentialItem(
|
|
credential_name="plain-cred",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
):
|
|
response = _patch_credential(
|
|
"plain-cred",
|
|
{
|
|
"credential_name": "plain-cred",
|
|
"credential_values": {"api_key": "sk-rotated", "anthropic_issuer_url": None},
|
|
"credential_info": {},
|
|
},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
synced = next(c for c in litellm.credential_list if c.credential_name == "plain-cred")
|
|
assert "anthropic_issuer_url" not in synced.credential_values
|
|
assert synced.credential_values["api_key"] == "sk-rotated"
|
|
|
|
|
|
def _generate_es256_pem() -> str:
|
|
key = ec.generate_private_key(ec.SECP256R1())
|
|
return key.private_bytes(
|
|
encoding=serialization.Encoding.PEM,
|
|
format=serialization.PrivateFormat.PKCS8,
|
|
encryption_algorithm=serialization.NoEncryption(),
|
|
).decode()
|
|
|
|
|
|
class TestCredentialJwksExport:
|
|
def test_jwks_export_succeeds_for_an_internal_issuer_credential(self, restore_credential_list, monkeypatch):
|
|
monkeypatch.setenv("JWKS_TEST_SIGNING_KEY", _generate_es256_pem())
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="anthropic-issuer",
|
|
credential_values={
|
|
"anthropic_identity_source": "internal_issuer",
|
|
"anthropic_issuer_url": "https://issuer.example.com",
|
|
"anthropic_issuer_subject": "my-workload",
|
|
"anthropic_issuer_signing_key_ref": "os.environ/JWKS_TEST_SIGNING_KEY",
|
|
},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
],
|
|
)
|
|
|
|
response = _get_jwks("anthropic-issuer")
|
|
|
|
assert response.status_code == 200, response.text
|
|
body = response.json()
|
|
assert body["keys"][0]["kty"] == "EC"
|
|
assert body["keys"][0]["crv"] == "P-256"
|
|
# The private key material must never leave the process via this endpoint.
|
|
assert "JWKS_TEST_SIGNING_KEY" not in response.text
|
|
assert "PRIVATE KEY" not in response.text
|
|
|
|
def test_jwks_export_treats_blank_optional_fields_as_unset(self, restore_credential_list, monkeypatch):
|
|
monkeypatch.setenv("JWKS_TEST_SIGNING_KEY", _generate_es256_pem())
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="anthropic-issuer-blanks",
|
|
credential_values={
|
|
"anthropic_identity_source": "internal_issuer",
|
|
"anthropic_issuer_url": "https://issuer.example.com",
|
|
"anthropic_issuer_subject": "my-workload",
|
|
"anthropic_issuer_signing_key_ref": "os.environ/JWKS_TEST_SIGNING_KEY",
|
|
"anthropic_issuer_audience": "",
|
|
"anthropic_issuer_ttl_seconds": "",
|
|
},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
],
|
|
)
|
|
|
|
response = _get_jwks("anthropic-issuer-blanks")
|
|
|
|
assert response.status_code == 200, response.text
|
|
assert response.json()["keys"][0]["kty"] == "EC"
|
|
|
|
def test_jwks_export_accepts_the_dashboard_provider_casing(self, restore_credential_list, monkeypatch):
|
|
monkeypatch.setenv("JWKS_TEST_SIGNING_KEY", _generate_es256_pem())
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="anthropic-from-modal",
|
|
credential_values={
|
|
"anthropic_identity_source": "internal_issuer",
|
|
"anthropic_issuer_url": "https://issuer.example.com",
|
|
"anthropic_issuer_subject": "my-workload",
|
|
"anthropic_issuer_signing_key_ref": "os.environ/JWKS_TEST_SIGNING_KEY",
|
|
},
|
|
credential_info={"custom_llm_provider": "Anthropic"},
|
|
)
|
|
],
|
|
)
|
|
|
|
response = _get_jwks("anthropic-from-modal")
|
|
|
|
assert response.status_code == 200, response.text
|
|
assert response.json()["keys"][0]["kty"] == "EC"
|
|
|
|
def test_jwks_export_404s_for_a_non_anthropic_credential(self, restore_credential_list, monkeypatch):
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="openai-key",
|
|
credential_values={"api_key": "sk-x"},
|
|
credential_info={"custom_llm_provider": "openai"},
|
|
)
|
|
],
|
|
)
|
|
|
|
response = _get_jwks("openai-key")
|
|
|
|
assert response.status_code == 404, response.text
|
|
|
|
def test_jwks_export_404s_for_an_anthropic_credential_without_internal_issuer(
|
|
self, restore_credential_list, monkeypatch
|
|
):
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="anthropic-apikey",
|
|
credential_values={"api_key": "sk-ant"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
],
|
|
)
|
|
|
|
response = _get_jwks("anthropic-apikey")
|
|
|
|
assert response.status_code == 404, response.text
|
|
|
|
def test_jwks_export_404s_for_an_unknown_credential(self, restore_credential_list):
|
|
with patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", None
|
|
): # test-quality-ok: the proxy wiring under test is what this patches
|
|
response = _get_jwks("does-not-exist")
|
|
|
|
assert response.status_code == 404, response.text
|
|
|
|
def test_jwks_export_requires_proxy_admin(self, restore_credential_list, monkeypatch):
|
|
monkeypatch.setenv("JWKS_TEST_SIGNING_KEY", _generate_es256_pem())
|
|
monkeypatch.setattr(
|
|
litellm,
|
|
"credential_list",
|
|
[
|
|
CredentialItem(
|
|
credential_name="anthropic-issuer",
|
|
credential_values={
|
|
"anthropic_identity_source": "internal_issuer",
|
|
"anthropic_issuer_url": "https://issuer.example.com",
|
|
"anthropic_issuer_subject": "my-workload",
|
|
"anthropic_issuer_signing_key_ref": "os.environ/JWKS_TEST_SIGNING_KEY",
|
|
},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
],
|
|
)
|
|
|
|
def _as_internal_user():
|
|
return UserAPIKeyAuth(api_key="test-key", user_role="internal_user")
|
|
|
|
app.dependency_overrides[user_api_key_auth] = _as_internal_user
|
|
try:
|
|
response = client.get("/credentials/anthropic-issuer/jwks", headers={"Authorization": "Bearer test-key"})
|
|
finally:
|
|
app.dependency_overrides.pop(user_api_key_auth, None)
|
|
|
|
assert response.status_code == 403, response.text
|
|
|
|
|
|
class TestNonAdminCannotPersistWifFieldsOnCredential:
|
|
"""A credential's ``credential_values`` feeds the same WIF resolution as a deployment's own
|
|
``litellm_params`` when referenced by ``litellm_credential_name``. A non-admin must not be
|
|
able to create or update a credential carrying a server-owned WIF field such as
|
|
``anthropic_keycloak_token_url`` (destination) or ``anthropic_keycloak_client_secret_ref``
|
|
(which secret to read and send there)."""
|
|
|
|
def test_non_admin_cannot_create_a_credential_with_a_wif_destination(self):
|
|
with patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
): # test-quality-ok: the proxy wiring under test is what this patches
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "attacker-cred",
|
|
"credential_values": {"anthropic_keycloak_token_url": "https://evil.example.com/token"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_keycloak_token_url" in response.json()["error"]["message"]
|
|
|
|
def test_non_admin_cannot_create_a_credential_with_a_wif_secret_ref(self):
|
|
with patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
): # test-quality-ok: the proxy wiring under test is what this patches
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "attacker-cred",
|
|
"credential_values": {"anthropic_keycloak_client_secret_ref": "os.environ/LITELLM_MASTER_KEY"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
|
|
def test_non_admin_can_create_a_credential_without_wif_fields(self, restore_credential_list):
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "ordinary-cred",
|
|
"credential_values": {"api_key": "sk-new"},
|
|
"credential_info": {"custom_llm_provider": "openai"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.create.assert_awaited_once()
|
|
|
|
def test_proxy_admin_can_create_a_credential_with_a_wif_destination(self, restore_credential_list):
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "admin-cred",
|
|
"credential_values": {"anthropic_keycloak_token_url": "https://keycloak.internal/token"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.create.assert_awaited_once()
|
|
|
|
def test_non_admin_cannot_create_a_credential_with_an_openai_token_file(self):
|
|
with patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
):
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "attacker-cred",
|
|
"credential_values": {"openai_identity_token_file": "/var/run/secrets/tokens/attacker"},
|
|
"credential_info": {"custom_llm_provider": "openai"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "openai_identity_token_file" in response.json()["error"]["message"]
|
|
|
|
def test_proxy_admin_can_create_a_credential_with_the_openai_identity_trio(self, restore_credential_list):
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "openai-wif",
|
|
"credential_values": {
|
|
"openai_identity_provider_id": "idp_1",
|
|
"openai_service_account_id": "user-1",
|
|
"openai_identity_token_file": "/var/run/secrets/tokens/openai",
|
|
},
|
|
"credential_info": {"custom_llm_provider": "openai"},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.create.assert_awaited_once()
|
|
|
|
def test_non_admin_cannot_update_a_credential_to_add_a_wif_destination(self):
|
|
stored = CredentialItem(
|
|
credential_name="existing",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with (
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.credential_endpoints.endpoints.CredentialsRepository"
|
|
) as repository, # test-quality-ok: the proxy wiring under test is what this patches
|
|
):
|
|
repository.return_value.find_by_name = AsyncMock(return_value=stored)
|
|
update_mock = AsyncMock(return_value=None)
|
|
repository.return_value.update_by_name = update_mock
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{
|
|
"credential_name": "existing",
|
|
"credential_values": {"anthropic_keycloak_token_url": "https://evil.example.com/token"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
update_mock.assert_not_awaited()
|
|
|
|
def test_non_admin_cannot_patch_wif_fields_onto_a_credential_through_model_id(self, credential_store):
|
|
"""Regression: the PATCH gate read only the submitted ``credential_values``, so a non-admin
|
|
naming a federated deployment through ``model_id`` had its WIF fields copied onto an
|
|
ordinary credential unchecked, while POST already gated the resolved values."""
|
|
stored = CredentialItem(credential_name="existing", credential_values={"api_key": "sk-old"}, credential_info={})
|
|
update_by_name = AsyncMock(return_value=None)
|
|
router = MagicMock()
|
|
router.get_deployment.return_value = {"model_name": "claude-opus-5-5"}
|
|
router.get_deployment_credentials.return_value = {
|
|
"anthropic_keycloak_token_url": "https://keycloak.internal/token",
|
|
"anthropic_keycloak_client_secret_ref": "os.environ/KEYCLOAK_CLIENT_SECRET",
|
|
}
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=update_by_name, llm_router=router)
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{"credential_name": "existing", "model_id": "federated-deployment", "credential_info": {}},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
update_by_name.assert_not_awaited()
|
|
|
|
def test_proxy_admin_can_patch_wif_fields_onto_a_credential_through_model_id(self, credential_store):
|
|
stored = CredentialItem(credential_name="existing", credential_values={"api_key": "sk-old"}, credential_info={})
|
|
update_by_name = AsyncMock(return_value=None)
|
|
router = MagicMock()
|
|
router.get_deployment.return_value = {"model_name": "claude-opus-5-5"}
|
|
router.get_deployment_credentials.return_value = {"anthropic_keycloak_token_url": "https://keycloak.internal/token"}
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=update_by_name, llm_router=router)
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{"credential_name": "existing", "model_id": "federated-deployment", "credential_info": {}},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written = json.loads(update_by_name.await_args.kwargs["data"]["credential_values"])
|
|
assert "anthropic_keycloak_token_url" in written, "the deployment's WIF field reaches the stored credential"
|
|
|
|
def test_proxy_admin_can_update_a_credential_to_add_a_wif_destination(self):
|
|
stored = CredentialItem(
|
|
credential_name="existing",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with (
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.prisma_client", MagicMock()
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.proxy_server.master_key", "sk-test-master"
|
|
),
|
|
patch( # test-quality-ok: the proxy wiring under test is what this patches
|
|
"litellm.proxy.credential_endpoints.endpoints.CredentialsRepository"
|
|
) as repository, # test-quality-ok: the proxy wiring under test is what this patches
|
|
):
|
|
repository.return_value.find_by_name = AsyncMock(return_value=stored)
|
|
update_mock = AsyncMock(return_value=None)
|
|
repository.return_value.update_by_name = update_mock
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{
|
|
"credential_name": "existing",
|
|
"credential_values": {"anthropic_keycloak_token_url": "https://keycloak.internal/token"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
update_mock.assert_awaited_once()
|
|
|
|
|
|
def _wif_credential(name: str = "federated-cred") -> CredentialItem:
|
|
return CredentialItem(
|
|
credential_name=name,
|
|
credential_values={
|
|
"anthropic_keycloak_token_url": "https://keycloak.internal/token",
|
|
"api_key": "sk-old",
|
|
},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
|
|
|
|
def _plain_credential(name: str = "ordinary-cred") -> CredentialItem:
|
|
return CredentialItem(
|
|
credential_name=name,
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "openai"},
|
|
)
|
|
|
|
|
|
class TestNonAdminCannotTouchAStoredWifCredential:
|
|
"""The WIF gate used to read only the incoming ``credential_values``, so a non-admin could
|
|
drop a federation field by naming it in ``credential_values_to_delete`` (breaking every
|
|
deployment that references the credential), or edit a stored admin-owned WIF credential by
|
|
sending a payload carrying no WIF field at all. The gate is evaluated against the effective
|
|
surface of the operation: incoming keys (a ``null`` value still persists the key), deleted
|
|
keys, and the stored credential, wherever it lives (DB row or config-only ``credential_list``
|
|
entry)."""
|
|
|
|
def test_non_admin_cannot_delete_a_wif_field_off_a_credential(self, restore_credential_list):
|
|
with _repository_holding(_plain_credential("some-cred")) as repository:
|
|
response = _patch_credential(
|
|
"some-cred",
|
|
{
|
|
"credential_name": "some-cred",
|
|
"credential_values": {},
|
|
"credential_values_to_delete": ["anthropic_keycloak_token_url"],
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_keycloak_token_url" in response.text
|
|
repository.update_by_name.assert_not_awaited()
|
|
|
|
def test_non_admin_cannot_patch_a_stored_wif_credential(self, restore_credential_list):
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _patch_credential(
|
|
"federated-cred",
|
|
{
|
|
"credential_name": "federated-cred",
|
|
"credential_values": {"api_key": "sk-attacker"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_keycloak_token_url" in response.text
|
|
repository.update_by_name.assert_not_awaited()
|
|
|
|
def test_proxy_admin_can_delete_a_wif_field_off_a_credential(self, restore_credential_list):
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _patch_credential(
|
|
"federated-cred",
|
|
{
|
|
"credential_name": "federated-cred",
|
|
"credential_values": {},
|
|
"credential_values_to_delete": ["anthropic_keycloak_token_url"],
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_values = json.loads(repository.update_by_name.await_args.kwargs["data"]["credential_values"])
|
|
assert "anthropic_keycloak_token_url" not in written_values
|
|
|
|
def test_proxy_admin_can_patch_a_stored_wif_credential(self, restore_credential_list):
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _patch_credential(
|
|
"federated-cred",
|
|
{
|
|
"credential_name": "federated-cred",
|
|
"credential_values": {"api_key": "sk-rotated"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written_values = json.loads(repository.update_by_name.await_args.kwargs["data"]["credential_values"])
|
|
assert written_values["anthropic_keycloak_token_url"] is not None
|
|
|
|
def test_non_admin_can_still_patch_a_credential_with_no_wif_fields_anywhere(self, restore_credential_list):
|
|
with _repository_holding(_plain_credential("ordinary-cred")) as repository:
|
|
response = _patch_credential(
|
|
"ordinary-cred",
|
|
{
|
|
"credential_name": "ordinary-cred",
|
|
"credential_values": {"api_key": "sk-rotated"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.update_by_name.assert_awaited_once()
|
|
|
|
def test_non_admin_cannot_delete_a_stored_wif_credential(self, restore_credential_list):
|
|
"""DELETE takes the whole row, so it drops the admin-owned federation settings as surely
|
|
as a targeted key deletion would."""
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _delete_credential("federated-cred", auth=_as_non_admin)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert response.json()["error"]["param"] == "anthropic_keycloak_token_url"
|
|
repository.delete_by_name.assert_not_awaited()
|
|
|
|
def test_a_stale_in_memory_copy_does_not_authorize_deleting_a_stored_wif_credential(
|
|
self, restore_credential_list, monkeypatch
|
|
):
|
|
"""Resolution reads memory first and stops, which is right when serving a request. A pod
|
|
whose in-memory copy predates an admin adding the federation fields must not read that
|
|
stale object and authorize the delete: the gate takes the union of memory and the row."""
|
|
monkeypatch.setattr(litellm, "credential_list", [_plain_credential("federated-cred")])
|
|
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _delete_credential("federated-cred", auth=_as_non_admin)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert response.json()["error"]["param"] == "anthropic_keycloak_token_url"
|
|
repository.delete_by_name.assert_not_awaited()
|
|
|
|
def test_proxy_admin_can_delete_a_stored_wif_credential(self, restore_credential_list):
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _delete_credential("federated-cred", auth=_as_admin)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.delete_by_name.assert_awaited_once_with("federated-cred")
|
|
|
|
def test_non_admin_can_still_delete_a_credential_with_no_wif_fields(self, restore_credential_list):
|
|
with _repository_holding(_plain_credential("ordinary-cred")) as repository:
|
|
response = _delete_credential("ordinary-cred", auth=_as_non_admin)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.delete_by_name.assert_awaited_once_with("ordinary-cred")
|
|
|
|
def test_non_admin_cannot_null_out_a_wif_field_on_a_credential(self, restore_credential_list):
|
|
"""A JSON ``null`` still lands as a key in ``credential_values``. ``get_litellm_params``
|
|
forwards a WIF kwarg on key presence and the federation resolver rejects a foreign
|
|
variant's field by key, so a value-based gate let a non-admin persist the key and wedge
|
|
every deployment referencing the credential at request time."""
|
|
with _repository_holding(_plain_credential("some-cred")) as repository:
|
|
response = _patch_credential(
|
|
"some-cred",
|
|
{
|
|
"credential_name": "some-cred",
|
|
"credential_values": {"anthropic_issuer_url": None},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_issuer_url" in response.text
|
|
repository.update_by_name.assert_not_awaited()
|
|
|
|
def test_non_admin_cannot_patch_a_credential_storing_a_null_wif_field(self, restore_credential_list):
|
|
stored = CredentialItem(
|
|
credential_name="nulled-cred",
|
|
credential_values={"anthropic_issuer_url": None, "api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
with _repository_holding(stored) as repository:
|
|
response = _patch_credential(
|
|
"nulled-cred",
|
|
{
|
|
"credential_name": "nulled-cred",
|
|
"credential_values": {"api_key": "sk-attacker"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_issuer_url" in response.text
|
|
repository.update_by_name.assert_not_awaited()
|
|
|
|
def test_proxy_admin_can_null_out_a_wif_field_on_a_credential(self, restore_credential_list):
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _patch_credential(
|
|
"federated-cred",
|
|
{
|
|
"credential_name": "federated-cred",
|
|
"credential_values": {"anthropic_keycloak_token_url": None},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.update_by_name.assert_awaited_once()
|
|
|
|
def test_non_admin_cannot_delete_a_config_only_wif_credential(self, restore_credential_list, monkeypatch):
|
|
"""A ``credential_list`` entry from config.yaml has no DB row, so a gate that consulted
|
|
only the DB let a non-admin evict the admin-owned federation settings from memory."""
|
|
config_credential = _wif_credential("config-wif")
|
|
monkeypatch.setattr(litellm, "credential_list", [config_credential])
|
|
with _repository_holding(None) as repository:
|
|
response = _delete_credential("config-wif", auth=_as_non_admin)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert response.json()["error"]["param"] == "anthropic_keycloak_token_url"
|
|
repository.delete_by_name.assert_not_awaited()
|
|
assert litellm.credential_list == [config_credential]
|
|
|
|
def test_proxy_admin_can_delete_a_config_only_wif_credential(self, restore_credential_list, monkeypatch):
|
|
"""The gate lets the admin through to the row delete. The 404 that follows is the rule for
|
|
every config-only credential (no row to delete, the entry is back on the next boot), so the
|
|
in-memory entry stays put too."""
|
|
config_credential = _wif_credential("config-wif")
|
|
monkeypatch.setattr(litellm, "credential_list", [config_credential])
|
|
with _repository_holding(None) as repository:
|
|
response = _delete_credential("config-wif", auth=_as_admin)
|
|
|
|
assert response.status_code == 404, response.text
|
|
repository.delete_by_name.assert_awaited_once_with("config-wif")
|
|
assert litellm.credential_list == [config_credential]
|
|
|
|
def test_non_admin_cannot_shadow_a_config_only_wif_credential(self, restore_credential_list, monkeypatch):
|
|
"""POST with the same name carries no WIF field and collides with no DB row, yet
|
|
``CredentialAccessor.upsert_credentials`` would replace the admin entry in memory and
|
|
the periodic config sync would then make the takeover permanent."""
|
|
config_credential = _wif_credential("config-wif")
|
|
monkeypatch.setattr(litellm, "credential_list", [config_credential])
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "config-wif",
|
|
"credential_values": {"api_key": "sk-attacker"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_keycloak_token_url" in response.text
|
|
repository.create.assert_not_awaited()
|
|
assert litellm.credential_list == [config_credential]
|
|
assert litellm.credential_list[0].credential_values["api_key"] == "sk-old"
|
|
|
|
def test_proxy_admin_can_post_over_a_config_only_wif_credential(self, restore_credential_list, monkeypatch):
|
|
monkeypatch.setattr(litellm, "credential_list", [_wif_credential("config-wif")])
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "config-wif",
|
|
"credential_values": {"api_key": "sk-rotated"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.create.assert_awaited_once()
|
|
assert litellm.credential_list[0].credential_values == {"api_key": "sk-rotated"}
|
|
|
|
def test_non_admin_cannot_rename_a_credential_onto_a_config_only_wif_credential(
|
|
self, restore_credential_list, monkeypatch
|
|
):
|
|
"""PATCH is the other way to shadow: renaming an ordinary credential onto the WIF
|
|
credential's name makes ``_sync_in_memory_credential`` upsert the attacker's values over
|
|
the admin entry, with no WIF field in the payload and no DB row to collide with."""
|
|
config_credential = _wif_credential("config-wif")
|
|
monkeypatch.setattr(litellm, "credential_list", [_plain_credential("mine"), config_credential])
|
|
with _repository_holding(_plain_credential("mine")) as repository:
|
|
response = _patch_credential(
|
|
"mine",
|
|
{
|
|
"credential_name": "config-wif",
|
|
"credential_values": {"api_key": "sk-attacker"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_keycloak_token_url" in response.text
|
|
repository.update_by_name.assert_not_awaited()
|
|
assert config_credential in litellm.credential_list
|
|
assert litellm.credential_list[1].credential_values["api_key"] == "sk-old"
|
|
|
|
def test_proxy_admin_can_rename_a_credential_onto_a_config_only_wif_credential(
|
|
self, restore_credential_list, monkeypatch
|
|
):
|
|
monkeypatch.setattr(litellm, "credential_list", [_plain_credential("mine"), _wif_credential("config-wif")])
|
|
with _repository_holding(_plain_credential("mine")) as repository:
|
|
response = _patch_credential(
|
|
"mine",
|
|
{
|
|
"credential_name": "config-wif",
|
|
"credential_values": {"api_key": "sk-rotated"},
|
|
"credential_info": {},
|
|
},
|
|
auth=_as_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.update_by_name.assert_awaited_once()
|
|
assert [c.credential_name for c in litellm.credential_list] == ["config-wif"]
|
|
|
|
def test_non_admin_cannot_post_a_null_wif_field(self, restore_credential_list):
|
|
"""Same key-presence rule on the create path: ``{"anthropic_issuer_url": null}`` persists
|
|
the key, and the resolver reacts to the key."""
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "nulled-cred",
|
|
"credential_values": {"anthropic_issuer_url": None, "api_key": "sk-new"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
assert "anthropic_issuer_url" in response.text
|
|
repository.create.assert_not_awaited()
|
|
assert litellm.credential_list == []
|
|
|
|
def test_non_admin_cannot_shadow_a_db_stored_wif_credential(self, restore_credential_list):
|
|
"""Same hole for a WIF credential another pod wrote to the DB before this pod's in-memory
|
|
list caught up: the existing-credential lookup falls through to the DB."""
|
|
with _repository_holding(_wif_credential("federated-cred")) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "federated-cred",
|
|
"credential_values": {"api_key": "sk-attacker"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 403, response.text
|
|
repository.create.assert_not_awaited()
|
|
|
|
def test_non_admin_can_still_post_a_credential_with_no_wif_fields_anywhere(self, restore_credential_list):
|
|
with _repository_holding(None) as repository:
|
|
response = _post_credential(
|
|
{
|
|
"credential_name": "ordinary-cred",
|
|
"credential_values": {"api_key": "sk-new"},
|
|
"credential_info": {"custom_llm_provider": "openai"},
|
|
},
|
|
auth=_as_non_admin,
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
repository.create.assert_awaited_once()
|
|
assert litellm.credential_list[0].credential_name == "ordinary-cred"
|
|
|
|
|
|
class TestManagementReadsTheStoredCredential:
|
|
"""Serving a request reads memory first, which is right. A management operation cannot: on a
|
|
pod whose in-memory copy predates another pod's update it would act on superseded values."""
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_authoritative_hydrate_prefers_the_row_over_a_stale_memory_copy(self):
|
|
import litellm
|
|
from litellm.proxy.common_utils.credential_hydration import (
|
|
hydrate_named_credential,
|
|
hydrate_named_credential_authoritative,
|
|
)
|
|
from litellm.types.utils import CredentialItem
|
|
|
|
stale = CredentialItem(
|
|
credential_name="anthropic-wif",
|
|
credential_values={"anthropic_issuer_url": "https://old.example.com"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
row = {
|
|
"credential_name": "anthropic-wif",
|
|
"credential_values": {"anthropic_issuer_url": "https://new.example.com"},
|
|
"credential_info": {"custom_llm_provider": "anthropic"},
|
|
}
|
|
|
|
prisma = MagicMock()
|
|
prisma.db.litellm_credentialstable.find_unique = AsyncMock(return_value=row)
|
|
|
|
with patch.object(litellm, "credential_list", [stale]): # test-quality-ok: the stale copy under test
|
|
served = await hydrate_named_credential("anthropic-wif", prisma)
|
|
managed = await hydrate_named_credential_authoritative("anthropic-wif", prisma)
|
|
|
|
assert served is not None and served.credential_values["anthropic_issuer_url"] == "https://old.example.com"
|
|
assert managed is not None and managed.credential_values["anthropic_issuer_url"] == "https://new.example.com"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_authoritative_hydrate_falls_back_to_memory_when_the_row_is_absent(self):
|
|
import litellm
|
|
from litellm.proxy.common_utils.credential_hydration import hydrate_named_credential_authoritative
|
|
from litellm.types.utils import CredentialItem
|
|
|
|
only_in_memory = CredentialItem(
|
|
credential_name="config-yaml-credential",
|
|
credential_values={"anthropic_issuer_url": "https://configured.example.com"},
|
|
credential_info={"custom_llm_provider": "anthropic"},
|
|
)
|
|
prisma = MagicMock()
|
|
prisma.db.litellm_credentialstable.find_unique = AsyncMock(return_value=None)
|
|
|
|
with patch.object(litellm, "credential_list", [only_in_memory]): # test-quality-ok: the config.yaml fallback under test
|
|
resolved = await hydrate_named_credential_authoritative("config-yaml-credential", prisma)
|
|
|
|
assert resolved is not None
|
|
assert resolved.credential_values["anthropic_issuer_url"] == "https://configured.example.com"
|
|
|
|
|
|
def test_delete_credential_answers_404_when_the_credential_does_not_exist(credential_store):
|
|
"""Regression: prisma's ``delete`` hands back None when the ``where`` clause matched no row
|
|
instead of raising, and the handler never looked. Deleting a name that was never stored
|
|
answered 200 "Credential deleted successfully", so an operator scripting cleanup could not
|
|
tell a real deletion from a typo."""
|
|
credential_store(delete_by_name=AsyncMock(return_value=None))
|
|
|
|
response = _delete_credential("definitely-not-there")
|
|
|
|
assert response.status_code == 404, (
|
|
f"delete of a missing credential answered {response.status_code}: {response.text}"
|
|
)
|
|
assert "definitely-not-there" in response.json()["error"]["message"]
|
|
|
|
|
|
def test_delete_credential_still_answers_200_and_drops_the_credential_from_memory(credential_store):
|
|
"""The fix must not turn a real deletion into an error, and the deleted credential must
|
|
stop being served from the in-memory list the proxy routes on."""
|
|
stored = CredentialItem(
|
|
credential_name="doomed",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={"custom_llm_provider": "openai"},
|
|
)
|
|
survivor = CredentialItem(
|
|
credential_name="keeper",
|
|
credential_values={"api_key": "sk-keep"},
|
|
credential_info={},
|
|
)
|
|
credential_store(in_memory=(stored, survivor), delete_by_name=AsyncMock(return_value=MagicMock()))
|
|
|
|
response = _delete_credential("doomed")
|
|
|
|
assert response.status_code == 200, response.text
|
|
assert response.json()["success"] is True
|
|
assert [credential.credential_name for credential in litellm.credential_list] == ["keeper"]
|
|
|
|
|
|
def test_delete_credential_leaves_a_credential_that_only_exists_in_memory_in_place(credential_store):
|
|
"""A credential declared in the config yaml is never written to the table, so the delete
|
|
matches no row. Reporting success would be the same lie: it comes straight back on the next
|
|
proxy boot. ``PATCH /credentials/{name}`` already answers 404 for that credential."""
|
|
config_only = CredentialItem(
|
|
credential_name="from-config-yaml",
|
|
credential_values={"api_key": "sk-config"},
|
|
credential_info={},
|
|
)
|
|
credential_store(in_memory=(config_only,), delete_by_name=AsyncMock(return_value=None))
|
|
|
|
response = _delete_credential("from-config-yaml")
|
|
|
|
assert response.status_code == 404, response.text
|
|
assert [credential.credential_name for credential in litellm.credential_list] == ["from-config-yaml"]
|
|
|
|
|
|
def test_delete_credential_answers_500_when_the_database_is_not_connected(credential_store):
|
|
"""The handler used to ``return handle_exception_on_proxy(e)``, which makes the exception the
|
|
response body and lets FastAPI answer 200. A DB-less proxy answered its own 500 as a success."""
|
|
credential_store(connected=False)
|
|
|
|
response = _delete_credential("any-name")
|
|
|
|
assert response.status_code == 500, f"rejected delete answered {response.status_code}: {response.text}"
|
|
|
|
|
|
class _CredentialThatCannotBeMasked:
|
|
"""Stands in for anything that fails while ``GET /credentials`` builds its response."""
|
|
|
|
credential_name = "unreadable"
|
|
credential_info: dict = {}
|
|
|
|
@property
|
|
def credential_values(self):
|
|
raise RuntimeError("credential store unreadable")
|
|
|
|
|
|
def test_get_credentials_answers_an_error_status_when_the_listing_fails(credential_store):
|
|
"""Same ``return`` instead of ``raise`` on the list route: a failed listing was serialized as
|
|
a 200 whose body happened to be an error, so a caller reading the status saw an empty success."""
|
|
credential_store(in_memory=(_CredentialThatCannotBeMasked(),))
|
|
|
|
response = _list_credentials()
|
|
|
|
assert response.status_code == 500, f"failed listing answered {response.status_code}: {response.text}"
|
|
assert response.json().get("success") is not True
|
|
|
|
|
|
def _create_credential(body: dict):
|
|
return _call_as("POST", "/credentials", body)
|
|
|
|
|
|
class _UniqueViolation(Exception):
|
|
code = "P2002"
|
|
|
|
|
|
def test_create_credential_answers_409_when_the_name_is_already_taken(credential_store):
|
|
"""Regression: the unique index used to surface as a Prisma 500 that callers string-matched."""
|
|
credential_store(
|
|
create=AsyncMock(side_effect=_UniqueViolation("Unique constraint failed on the fields: (`credential_name`)")),
|
|
)
|
|
|
|
response = _create_credential(
|
|
{"credential_name": "aws_bedrock", "credential_values": {"aws_access_key_id": "new"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 409, f"name collision answered {response.status_code}: {response.text}"
|
|
message = response.json()["error"]["message"]
|
|
assert message == (
|
|
"Credential 'aws_bedrock' already exists. Update it with PATCH /credentials/aws_bedrock, or delete it first."
|
|
), f"the operator reads this message verbatim: {message}"
|
|
assert "Unique constraint" not in response.text, f"the Prisma internals must not leak: {response.text}"
|
|
|
|
|
|
def test_create_credential_still_answers_500_when_the_write_fails_for_another_reason(credential_store):
|
|
credential_store(create=AsyncMock(side_effect=Exception("connection reset by peer")))
|
|
|
|
response = _create_credential(
|
|
{"credential_name": "aws_bedrock", "credential_values": {"aws_access_key_id": "new"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 500, f"database fault answered {response.status_code}: {response.text}"
|
|
|
|
|
|
def test_create_credential_still_answers_200_for_a_name_that_is_free(credential_store):
|
|
find_by_name = AsyncMock()
|
|
credential_store(find_by_name=find_by_name, create=AsyncMock(return_value=None))
|
|
|
|
response = _create_credential(
|
|
{"credential_name": "brand_new", "credential_values": {"aws_access_key_id": "new"}, "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
assert response.json()["success"] is True
|
|
find_by_name.assert_not_awaited(), "the unique index is the guard; create must not add a lookup"
|
|
|
|
|
|
def test_update_credential_resolves_credential_values_from_model_id_like_create(credential_store):
|
|
"""Regression: PATCH dropped ``model_id`` from the body, so an update that named a
|
|
deployment instead of raw values wrote whatever the caller sent, or nothing."""
|
|
stored = CredentialItem(
|
|
credential_name="from-deployment",
|
|
credential_values={"api_key": "sk-old"},
|
|
credential_info={},
|
|
)
|
|
update_by_name = AsyncMock(return_value=None)
|
|
router = MagicMock()
|
|
router.get_deployment.return_value = {"model_name": "gpt-5.2"}
|
|
router.get_deployment_credentials.return_value = {"api_key": "sk-from-deployment"}
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=update_by_name, llm_router=router)
|
|
|
|
response = _patch_credential(
|
|
"from-deployment",
|
|
{"credential_name": "from-deployment", "model_id": "deployment-1", "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
router.get_deployment_credentials.assert_called_once_with("deployment-1")
|
|
written = json.loads(update_by_name.await_args.kwargs["data"]["credential_values"])
|
|
assert set(written) == {"api_key"}
|
|
assert written["api_key"] != "sk-old", "the deployment's values must replace the stored ones"
|
|
assert written["api_key"] != "sk-from-deployment", "values are encrypted before they reach the table"
|
|
|
|
|
|
def test_update_credential_answers_404_when_model_id_names_no_deployment(credential_store):
|
|
stored = CredentialItem(
|
|
credential_name="from-deployment", credential_values={"api_key": "sk-old"}, credential_info={}
|
|
)
|
|
update_by_name = AsyncMock(return_value=None)
|
|
router = MagicMock()
|
|
router.get_deployment.return_value = None
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=update_by_name, llm_router=router)
|
|
|
|
response = _patch_credential(
|
|
"from-deployment",
|
|
{"credential_name": "from-deployment", "model_id": "no-such-deployment", "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 404, response.text
|
|
update_by_name.assert_not_awaited()
|
|
|
|
|
|
def test_update_credential_answers_500_when_model_id_is_given_but_no_router_is_loaded(credential_store):
|
|
stored = CredentialItem(
|
|
credential_name="from-deployment", credential_values={"api_key": "sk-old"}, credential_info={}
|
|
)
|
|
update_by_name = AsyncMock(return_value=None)
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=update_by_name, llm_router=None)
|
|
|
|
response = _patch_credential(
|
|
"from-deployment",
|
|
{"credential_name": "from-deployment", "model_id": "deployment-1", "credential_info": {}},
|
|
)
|
|
|
|
assert response.status_code == 500, response.text
|
|
update_by_name.assert_not_awaited()
|
|
|
|
|
|
def test_update_credential_still_accepts_a_body_without_credential_values(credential_store):
|
|
"""Renaming or re-tagging a credential sends only ``credential_info``; that must not 422."""
|
|
stored = CredentialItem(credential_name="existing", credential_values={"api_key": "sk-old"}, credential_info={})
|
|
update_by_name = AsyncMock(return_value=None)
|
|
credential_store(find_by_name=AsyncMock(return_value=stored), update_by_name=update_by_name)
|
|
|
|
response = _patch_credential(
|
|
"existing",
|
|
{"credential_name": "existing", "credential_info": {"custom_llm_provider": "openai"}},
|
|
)
|
|
|
|
assert response.status_code == 200, response.text
|
|
written = update_by_name.await_args.kwargs["data"]
|
|
assert json.loads(written["credential_info"]) == {"custom_llm_provider": "openai"}
|
|
assert set(json.loads(written["credential_values"])) == {"api_key"}, "stored values survive an info-only patch"
|