mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
* feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents Bump a2a-sdk to 1.x and wire send/stream through compat conversions so the proxy accepts A2A 1.0 JSON-RPC while preserving 0.3 wire clients. Co-authored-by: Cursor <cursoragent@cursor.com> * Add user controlled protocol version in agents * Fix exeception mapping * Fix a2a base url * Add e2e test for a2a * Fix lint * Fix lint * fix(a2a): harden card version detection and header isolation coverage Use protocolVersion when inferring agent card wire format, assert distinct httpx cache keys in the header-isolation test, and suppress targeted basedpyright errors for optional SDK imports. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): suppress reportArgumentType for SDK compat types and fix streaming trace ID - Add pyright: ignore[reportArgumentType] to SendMessageSuccessResponse id= and result= args in _send_message, and SendStreamingMessageResponse root= in _stream_messages, where a2a-sdk compat types diverge from basedpyright's inferred signature, reducing the reportArgumentType count back within budget. - Fix streaming trace ID in astream_a2a_message to use str(request.id) when available instead of always generating a new uuid4(), restoring JSON-RPC request-ID correlation for observability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style(a2a): expand SendStreamingMessageResponse for black formatting Move pyright: ignore comment to the root= argument line so Black accepts the expanded multi-line form. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(a2a): fix 2 reportArgumentType errors without suppression - main.py: narrow logging_obj from object|None to Optional[Logging] via isinstance check before A2AStreamingIterator call, fixing the "Logging | object" argument type mismatch at line 699. - a2a_endpoints.py: extract response_dict with explicit isinstance(dict) guard before passing to normalize_jsonrpc_response, fixing the "LLMResponseTypes | dict[str, Any]" type mismatch at line 835. - Remove spurious pyright: ignore comments added in previous commits that were not suppressing the actual errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(a2a): rewrite upstream URL for 1.0 agent cards in getAuthenticatedExtendedCard 1.0 upstream agent cards store the endpoint URL in supportedInterfaces[0].url rather than a top-level url field. The previous guard only rewrote url when it existed at the top level, so after normalize_agent_card lowered a 1.0 card to 0.3 the upstream internal address leaked into the url field of the 0.3 response. Fix: rewrite both url and supportedInterfaces[0].url to the proxy address before calling normalize_agent_card, ensuring the upstream address is never visible to downstream clients regardless of the upstream card's wire format. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: extend _served_version to all PascalCase methods; add direct httpx-client isolation proof - _served_version now checks `_PASCAL_TO_WIRE` membership instead of two hardcoded names, so GetTask/CancelTask/etc. are promoted to 1.0 wire format alongside SendMessage — prevents mixed wire formats mid-session - test_create_a2a_client_uses_fresh_httpx_client now asserts a2a_client_a._litellm_httpx_client is not a2a_client_b._litellm_httpx_client (direct proof that header bleed cannot occur), in addition to the cache-key inequality check Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: id:0 silently dropped in version_convert; explicit continue in stream retry - version_convert.py: replace `request_id or ""` with `str(request_id) if request_id is not None else ""` in both _send_result_to and _stream_result_to; id=0 is valid JSON-RPC and must not be coerced to "" which breaks response correlation - main.py: add explicit `continue` after the A2ALocalhostURLError retry in _execute_a2a_stream_with_retry so the control flow (retry → next iteration → stream_succeeded guard) is unambiguous Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: preserve a2a retry and discovery card urls * Fix black * Fix test * fix(a2a): avoid KeyError in discovery log after 0.3→1.0 card normalization When a 0.3-style agent card is normalized to 1.0, the top-level url key is replaced by supportedInterfaces; log the already-computed proxy_url instead. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): preserve taskId when lowering push notification config set params Flatten 1.x create envelope fields before parsing into TaskPushNotificationConfig so 1.0 clients forwarding to 0.3 upstream keep taskId and config. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): ignore unknown fields in message/send proto fallback ParseDict in _build_message_send_params now matches other inbound paths so 1.0 clients with extra proto fields are not rejected with -32602. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): normalize tasks/list params and response across protocol versions Convert list task entries on the response path and lower ListTasksRequest params including status filters when forwarding 1.0 clients to 0.3 upstream. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): avoid reportArgumentType in _lower_list_tasks_params; use local var instead of _parse return Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(a2a): drop private SDK symbol in tasks/list status lowering _lower_list_tasks_params imported _CORE_TO_COMPAT_TASK_STATE, a private a2a-sdk symbol that could disappear on a patch release and silently break status-filter lowering. Derive the 0.3 wire string from the public protobuf enum name instead (TASK_STATE_<NAME> maps to the 0.3 value once the prefix is dropped and underscores become dashes) and validate the result against the 0.3 TaskState enum's own values via a fully-typed pure helper. Behavior is unchanged for every state; unspecified or unrecognized states still drop the filter. Adds parametrized regression tests covering dashed wire values (input-required, auth-required) and the unspecified drop. * fix(a2a): drop redundant push-notification envelope key; unify MessageToDict import _flatten_create_push_notification_params used `config or pushNotificationConfig`, which short-circuits so a co-present pushNotificationConfig key was never popped and leaked into the flattened params. Pop both keys unconditionally and prefer config when present. Adds a regression test on the helper that fails on the old leak. Also import MessageToDict from a2a.compat.v0_3.conversions in _lower_list_tasks_params to match every other conversion helper in the module instead of pulling it straight from google.protobuf.json_format. * fix(a2a): reject invalid message/stream params early with -32602 _handle_stream_message built MessageSendParams lazily inside the stream_response() generator, so malformed 1.0 params surfaced as a generic -32603 after the 200 status line was already committed. The non-streaming path validates up front and returns -32602 (Invalid params). Validate eagerly before returning the StreamingResponse and emit -32602 on failure so both paths reject malformed params identically. Adds a regression test asserting the streamed error code is -32602. * fix(a2a): raise clear error when non-streaming send ends on an update event _send_message fed the SDK iterator's last event straight into SendMessageSuccessResponse, whose result only accepts Message or Task. A non-standard upstream whose final event is a TaskStatusUpdateEvent or TaskArtifactUpdateEvent made the response construction raise an opaque pydantic ValidationError. Guard the converted result and raise a clear RuntimeError instead, consistent with the no-response guard above it. Adds regression tests for the Message happy path and the update-event rejection via an injected fake client. * test(a2a): lock in clean merged agent-card URL without PROXY_BASE_URL Regression coverage proving _build_merged_agent_card produces no double slash in supportedInterfaces[0].url when PROXY_BASE_URL is unset and request.base_url carries a trailing slash. get_custom_url routes through join_paths, which rstrips the base, so the f-string join stays clean. * style(a2a): modernize type annotations to satisfy strict ruff budget After merging the black->ruff-format migration from base, the A2A files owned by this PR still used Optional[X]/quoted annotations that pushed UP037/UP045 over their lowered ceilings. Convert to X | None, drop the now-unnecessary quoted local annotation in _send_message, and remove the imports left unused by the rewrite. Type semantics are unchanged. * style(a2a): type a2a_endpoints dict params as dict[str, Any] The merge with the formatter-migration baseline tightened the reportUnknownArgumentType ceiling; bare dict annotations made every value Unknown and pushed the codebase total over cap. Annotate the JSON-RPC params, body, metadata, and litellm_params dicts as dict[str, Any] so their values are typed, dropping the unknown-argument count back under the ceiling. No behavior change. * fix(a2a): guard localhost retry against a missing agent card handle_a2a_localhost_retry rewrote the card URL and called create_client with whatever agent_card it received. The caller resolves the card from the SDK client (Optional), so a None card reached set_agent_card_url and create_client, surfacing an opaque SDK error instead of a clear one. Add an early RuntimeError guard mirroring the httpx-client check, drop the now always-true card None-check on the stash line, and cover it with a regression test. * style(a2a): disable reportUnknownArgumentType in a2a-sdk boundary modules The lint env type-checks without the optional a2a-sdk/protobuf installed, so every call into the protobuf-generated compat conversions counts as an Unknown-typed argument and the new A2A code pushed the codebase reportUnknownArgumentType total over its ceiling. These three modules are the A2A SDK boundary; turn the rule off file-wide with a documented reason instead of scattering dozens of per-line ignores across every SDK call. * fix(a2a): tolerate unknown fields when lowering 1.0->0.3; align streaming trace id Two issues greptile flagged: version_convert: the 1.0->0.3 lowering paths (_send_result_to, _task_to, _stream_result_to) called ParseDict without ignore_unknown_fields=True, so a 1.0 upstream response carrying vendor extensions raised and best-effort fell back to passing the un-lowered 1.0 shape to a 0.3 client. Set the flag to match the agent-card path and every inbound path; unknown fields are now dropped and the result is correctly lowered. main.py: asend_message_streaming derived X-LiteLLM-Trace-Id from the JSON-RPC request id, unlike asend_message which uses the logging object's litellm_trace_id. Prefer the logging trace id (then request id, then a uuid) so streamed and non-streamed calls correlate under the same trace. Adds regression tests for both, including the stream-event lowering path. * style(a2a): apply ruff format to a2a protocol and proxy modules Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
817 lines
29 KiB
Python
817 lines
29 KiB
Python
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
import pytest
|
|
from fastapi import FastAPI
|
|
from fastapi.testclient import TestClient
|
|
|
|
from litellm.proxy._types import LitellmUserRoles, UserAPIKeyAuth
|
|
from litellm.proxy.agent_endpoints import endpoints as agent_endpoints
|
|
from litellm.proxy.agent_endpoints.endpoints import (
|
|
_attach_keys_to_agents,
|
|
_check_agent_management_permission,
|
|
get_agent_daily_activity,
|
|
router,
|
|
user_api_key_auth,
|
|
)
|
|
from litellm.types.agents import AgentResponse
|
|
|
|
|
|
def _sample_agent_card_params() -> dict:
|
|
return {
|
|
"protocolVersion": "1.0",
|
|
"name": "Test Agent",
|
|
"description": "desc",
|
|
"url": "http://localhost",
|
|
"version": "1.0.0",
|
|
"capabilities": {"streaming": True},
|
|
"defaultInputModes": ["text"],
|
|
"defaultOutputModes": ["text"],
|
|
"skills": [],
|
|
}
|
|
|
|
|
|
def _sample_agent_config() -> dict:
|
|
return {
|
|
"agent_name": "Test Agent",
|
|
"agent_card_params": _sample_agent_card_params(),
|
|
"litellm_params": {"make_public": False},
|
|
}
|
|
|
|
|
|
def _sample_agent_response(
|
|
agent_id: str = "agent-123", agent_name: str = "Test Agent"
|
|
) -> AgentResponse:
|
|
return AgentResponse(
|
|
agent_id=agent_id,
|
|
agent_name=agent_name,
|
|
agent_card_params=_sample_agent_card_params(),
|
|
litellm_params={"make_public": False},
|
|
)
|
|
|
|
|
|
def _make_app_with_role(role: LitellmUserRoles) -> TestClient:
|
|
"""Create a TestClient where the auth dependency returns the given role."""
|
|
test_app = FastAPI()
|
|
test_app.include_router(router)
|
|
test_app.dependency_overrides[user_api_key_auth] = lambda: UserAPIKeyAuth(
|
|
user_id="test-user", user_role=role
|
|
)
|
|
return TestClient(test_app)
|
|
|
|
|
|
app = FastAPI()
|
|
app.include_router(router)
|
|
app.dependency_overrides[user_api_key_auth] = lambda: UserAPIKeyAuth(
|
|
user_id="test-user", user_role=LitellmUserRoles.PROXY_ADMIN
|
|
)
|
|
client = TestClient(app)
|
|
|
|
|
|
@pytest.fixture
|
|
def mock_prisma_client():
|
|
with patch("litellm.proxy.proxy_server.prisma_client") as mock:
|
|
yield mock
|
|
|
|
|
|
@pytest.fixture
|
|
def mock_user_api_key_auth():
|
|
with patch("litellm.proxy.agent_endpoints.endpoints.user_api_key_auth") as mock:
|
|
mock.return_value = UserAPIKeyAuth(
|
|
user_id="test-user", user_role=LitellmUserRoles.PROXY_ADMIN
|
|
)
|
|
yield mock
|
|
|
|
|
|
def test_update_agent_success(mock_prisma_client, mock_user_api_key_auth, monkeypatch):
|
|
existing_agent = {
|
|
"agent_id": "agent-123",
|
|
"agent_name": "Existing Agent",
|
|
"agent_card_params": _sample_agent_card_params(),
|
|
}
|
|
mock_prisma_client.db.litellm_agentstable.find_unique = AsyncMock(
|
|
return_value=existing_agent
|
|
)
|
|
|
|
mock_registry = MagicMock()
|
|
mock_registry.update_agent_in_db = AsyncMock(
|
|
return_value=_sample_agent_response(agent_id="agent-123")
|
|
)
|
|
mock_registry.deregister_agent = MagicMock()
|
|
mock_registry.register_agent = MagicMock()
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", mock_registry)
|
|
|
|
response = client.put(
|
|
"/v1/agents/agent-123",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer test-key"},
|
|
)
|
|
|
|
assert response.status_code == 200
|
|
assert response.json()["agent_id"] == "agent-123"
|
|
assert response.json()["agent_name"] == "Test Agent"
|
|
|
|
|
|
def test_update_agent_not_found(
|
|
mock_prisma_client, mock_user_api_key_auth, monkeypatch
|
|
):
|
|
mock_prisma_client.db.litellm_agentstable.find_unique = AsyncMock(return_value=None)
|
|
|
|
mock_registry = MagicMock()
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", mock_registry)
|
|
|
|
response = client.put(
|
|
"/v1/agents/missing-agent",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer test-key"},
|
|
)
|
|
|
|
assert response.status_code == 404
|
|
assert "Agent with ID missing-agent not found" in response.json()["detail"]
|
|
|
|
|
|
def test_get_agent_by_id_not_found(
|
|
mock_prisma_client, mock_user_api_key_auth, monkeypatch
|
|
):
|
|
mock_registry = MagicMock()
|
|
mock_registry.get_agent_by_id = MagicMock(return_value=None)
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", mock_registry)
|
|
mock_prisma_client.db.litellm_agentstable.find_unique = AsyncMock(return_value=None)
|
|
|
|
response = client.get(
|
|
"/v1/agents/missing-agent", headers={"Authorization": "Bearer test-key"}
|
|
)
|
|
|
|
assert response.status_code == 404
|
|
assert "Agent with ID missing-agent not found" in response.json()["detail"]
|
|
|
|
|
|
def test_delete_agent_not_found(
|
|
mock_prisma_client, mock_user_api_key_auth, monkeypatch
|
|
):
|
|
mock_prisma_client.db.litellm_agentstable.find_unique = AsyncMock(return_value=None)
|
|
mock_registry = MagicMock()
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", mock_registry)
|
|
|
|
response = client.delete(
|
|
"/v1/agents/missing-agent", headers={"Authorization": "Bearer test-key"}
|
|
)
|
|
|
|
assert response.status_code == 404
|
|
assert "Agent with ID missing-agent not found in DB." in response.json()["detail"]
|
|
|
|
|
|
def test_agent_error_schema_consistency(
|
|
mock_prisma_client, mock_user_api_key_auth, monkeypatch
|
|
):
|
|
mock_registry = MagicMock()
|
|
mock_registry.get_agent_by_id = MagicMock(return_value=None)
|
|
mock_registry.update_agent_in_db = AsyncMock(
|
|
side_effect=Exception("should not run")
|
|
)
|
|
mock_registry.delete_agent_from_db = AsyncMock(
|
|
side_effect=Exception("should not run")
|
|
)
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", mock_registry)
|
|
|
|
mock_prisma_client.db.litellm_agentstable.find_unique = AsyncMock(return_value=None)
|
|
|
|
missing_agent_id = "missing-agent"
|
|
responses = [
|
|
client.get(
|
|
f"/v1/agents/{missing_agent_id}",
|
|
headers={"Authorization": "Bearer test-key"},
|
|
),
|
|
client.put(
|
|
f"/v1/agents/{missing_agent_id}",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer test-key"},
|
|
),
|
|
client.delete(
|
|
f"/v1/agents/{missing_agent_id}",
|
|
headers={"Authorization": "Bearer test-key"},
|
|
),
|
|
]
|
|
|
|
for resp in responses:
|
|
assert resp.status_code == 404
|
|
detail = resp.json()["detail"]
|
|
assert isinstance(detail, str)
|
|
assert missing_agent_id in detail
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_agent_daily_activity_admin_param_passing(monkeypatch):
|
|
mock_prisma = AsyncMock()
|
|
mock_prisma.db.litellm_agentstable.find_many = AsyncMock(return_value=[])
|
|
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", mock_prisma)
|
|
|
|
mocked_response = MagicMock(name="SpendAnalyticsPaginatedResponse")
|
|
get_daily_activity_mock = AsyncMock(return_value=mocked_response)
|
|
monkeypatch.setattr(agent_endpoints, "get_daily_activity", get_daily_activity_mock)
|
|
|
|
auth = UserAPIKeyAuth(user_role=LitellmUserRoles.PROXY_ADMIN, user_id="admin1")
|
|
result = await get_agent_daily_activity(
|
|
agent_ids="agent-1,agent-2",
|
|
start_date="2024-01-01",
|
|
end_date="2024-01-31",
|
|
model="gpt-4",
|
|
api_key="test-key",
|
|
page=2,
|
|
page_size=5,
|
|
exclude_agent_ids="agent-3",
|
|
user_api_key_dict=auth,
|
|
)
|
|
|
|
get_daily_activity_mock.assert_awaited_once()
|
|
kwargs = get_daily_activity_mock.call_args.kwargs
|
|
assert kwargs["table_name"] == "litellm_dailyagentspend"
|
|
assert kwargs["entity_id_field"] == "agent_id"
|
|
assert kwargs["entity_id"] == ["agent-1", "agent-2"]
|
|
assert kwargs["exclude_entity_ids"] == ["agent-3"]
|
|
assert kwargs["start_date"] == "2024-01-01"
|
|
assert kwargs["end_date"] == "2024-01-31"
|
|
assert kwargs["model"] == "gpt-4"
|
|
assert kwargs["api_key"] == "test-key"
|
|
assert kwargs["page"] == 2
|
|
assert kwargs["page_size"] == 5
|
|
assert result is mocked_response
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_agent_daily_activity_with_agent_names(monkeypatch):
|
|
mock_prisma = AsyncMock()
|
|
mock_agent1 = MagicMock()
|
|
mock_agent1.agent_id = "agent-1"
|
|
mock_agent1.agent_name = "First Agent"
|
|
mock_agent2 = MagicMock()
|
|
mock_agent2.agent_id = "agent-2"
|
|
mock_agent2.agent_name = "Second Agent"
|
|
|
|
mock_prisma.db.litellm_agentstable.find_many = AsyncMock(
|
|
return_value=[mock_agent1, mock_agent2]
|
|
)
|
|
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", mock_prisma)
|
|
|
|
mocked_response = MagicMock(name="SpendAnalyticsPaginatedResponse")
|
|
get_daily_activity_mock = AsyncMock(return_value=mocked_response)
|
|
monkeypatch.setattr(agent_endpoints, "get_daily_activity", get_daily_activity_mock)
|
|
|
|
auth = UserAPIKeyAuth(user_role=LitellmUserRoles.PROXY_ADMIN, user_id="admin1")
|
|
await get_agent_daily_activity(
|
|
agent_ids="agent-1,agent-2",
|
|
start_date="2024-01-01",
|
|
end_date="2024-01-31",
|
|
model=None,
|
|
api_key=None,
|
|
page=1,
|
|
page_size=10,
|
|
exclude_agent_ids=None,
|
|
user_api_key_dict=auth,
|
|
)
|
|
|
|
kwargs = get_daily_activity_mock.call_args.kwargs
|
|
assert kwargs["entity_metadata_field"] == {
|
|
"agent-1": {"agent_name": "First Agent"},
|
|
"agent-2": {"agent_name": "Second Agent"},
|
|
}
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_attach_keys_to_agents_groups_by_agent_and_omits_secret():
|
|
"""
|
|
The agents response must carry each agent's attached virtual keys (derived
|
|
from the key table's agent_id FK), grouped per agent, exposing only
|
|
non-secret summary fields. Agents with no key get None so the UI renders
|
|
"Needs Setup" rather than a stale badge.
|
|
"""
|
|
|
|
class _Row:
|
|
def __init__(self, token, agent_id, key_alias, key_name):
|
|
self.token = token
|
|
self.agent_id = agent_id
|
|
self.key_alias = key_alias
|
|
self.key_name = key_name
|
|
self.user_id = "secret-owner" # extra field that must NOT leak
|
|
|
|
agent_with_keys = _sample_agent_response(agent_id="agent-1")
|
|
agent_without_keys = _sample_agent_response(agent_id="agent-2")
|
|
|
|
mock_prisma = MagicMock()
|
|
mock_prisma.db.litellm_verificationtoken.find_many = AsyncMock(
|
|
return_value=[
|
|
_Row("hash-aaa", "agent-1", "primary", "sk-...aaa"),
|
|
_Row("hash-bbb", "agent-1", "backup", "sk-...bbb"),
|
|
]
|
|
)
|
|
|
|
await _attach_keys_to_agents([agent_with_keys, agent_without_keys], mock_prisma)
|
|
|
|
# Query is scoped to the agents being returned, not the whole key table.
|
|
where = mock_prisma.db.litellm_verificationtoken.find_many.call_args.kwargs["where"]
|
|
assert where == {"agent_id": {"in": ["agent-1", "agent-2"]}}
|
|
|
|
# agent-1 gets both of its keys; agent-2 gets None.
|
|
assert agent_without_keys.keys is None
|
|
assert agent_with_keys.keys is not None
|
|
assert {k.token for k in agent_with_keys.keys} == {"hash-aaa", "hash-bbb"}
|
|
assert {k.key_alias for k in agent_with_keys.keys} == {"primary", "backup"}
|
|
|
|
# Only summary fields are exposed; the row's user_id must not be carried.
|
|
summary = agent_with_keys.keys[0]
|
|
assert set(summary.model_dump().keys()) == {"token", "key_alias", "key_name"}
|
|
|
|
|
|
class TestAgentByIdKeyRedaction:
|
|
"""GET /v1/agents/{id} surfaces attached keys to admins but never to
|
|
non-admins, even when the agent has keys attached."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _setup(self, monkeypatch):
|
|
self.mock_registry = MagicMock()
|
|
self.mock_registry.get_agent_by_id = MagicMock(
|
|
return_value=_sample_agent_response()
|
|
)
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", self.mock_registry)
|
|
|
|
def _get_as(self, role: LitellmUserRoles):
|
|
key_row = MagicMock()
|
|
key_row.token = "hash-aaa"
|
|
key_row.agent_id = "agent-123"
|
|
key_row.key_alias = "primary"
|
|
key_row.key_name = "sk-...aaa"
|
|
|
|
test_client = _make_app_with_role(role)
|
|
with patch("litellm.proxy.proxy_server.prisma_client") as mock_prisma:
|
|
mock_prisma.db.litellm_agentstable.find_unique = AsyncMock(
|
|
return_value=None
|
|
)
|
|
mock_prisma.db.litellm_verificationtoken.find_many = AsyncMock(
|
|
return_value=[key_row]
|
|
)
|
|
return test_client.get(
|
|
"/v1/agents/agent-123", headers={"Authorization": "Bearer k"}
|
|
)
|
|
|
|
def test_admin_sees_attached_keys(self):
|
|
resp = self._get_as(LitellmUserRoles.PROXY_ADMIN)
|
|
assert resp.status_code == 200
|
|
keys = resp.json()["keys"]
|
|
assert keys is not None
|
|
assert keys[0] == {
|
|
"token": "hash-aaa",
|
|
"key_alias": "primary",
|
|
"key_name": "sk-...aaa",
|
|
}
|
|
|
|
def test_non_admin_never_sees_keys(self):
|
|
resp = self._get_as(LitellmUserRoles.INTERNAL_USER)
|
|
assert resp.status_code == 200
|
|
assert resp.json()["keys"] is None
|
|
|
|
|
|
# ---------- RBAC enforcement tests ----------
|
|
|
|
|
|
class TestAgentRBACInternalUser:
|
|
"""Internal users should be able to read agents but not create/update/delete."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _setup(self, monkeypatch):
|
|
self.internal_client = _make_app_with_role(LitellmUserRoles.INTERNAL_USER)
|
|
self.mock_registry = MagicMock()
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", self.mock_registry)
|
|
|
|
def test_should_allow_internal_user_to_list_agents(self, monkeypatch):
|
|
self.mock_registry.get_agent_list = MagicMock(return_value=[])
|
|
resp = self.internal_client.get(
|
|
"/v1/agents", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 200
|
|
|
|
def test_should_allow_internal_user_to_get_agent_by_id(self, monkeypatch):
|
|
self.mock_registry.get_agent_by_id = MagicMock(
|
|
return_value=_sample_agent_response()
|
|
)
|
|
with patch("litellm.proxy.proxy_server.prisma_client") as mock_prisma:
|
|
mock_prisma.db.litellm_agentstable.find_unique = AsyncMock(
|
|
return_value=None
|
|
)
|
|
mock_prisma.db.litellm_verificationtoken.find_many = AsyncMock(
|
|
return_value=[]
|
|
)
|
|
resp = self.internal_client.get(
|
|
"/v1/agents/agent-123", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 200
|
|
|
|
def test_should_block_internal_user_from_creating_agent(self):
|
|
resp = self.internal_client.post(
|
|
"/v1/agents",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 403
|
|
assert "Only proxy admins" in resp.json()["detail"]["error"]
|
|
|
|
def test_should_block_internal_user_from_updating_agent(self):
|
|
resp = self.internal_client.put(
|
|
"/v1/agents/agent-123",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 403
|
|
|
|
def test_should_block_internal_user_from_patching_agent(self):
|
|
resp = self.internal_client.patch(
|
|
"/v1/agents/agent-123",
|
|
json={"agent_name": "new-name"},
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 403
|
|
|
|
def test_should_block_internal_user_from_deleting_agent(self):
|
|
resp = self.internal_client.delete(
|
|
"/v1/agents/agent-123", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 403
|
|
|
|
|
|
class TestAgentRBACInternalUserViewOnly:
|
|
"""View-only internal users should only be able to read agents."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _setup(self, monkeypatch):
|
|
self.viewer_client = _make_app_with_role(
|
|
LitellmUserRoles.INTERNAL_USER_VIEW_ONLY
|
|
)
|
|
self.mock_registry = MagicMock()
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", self.mock_registry)
|
|
|
|
def test_should_allow_view_only_user_to_list_agents(self):
|
|
self.mock_registry.get_agent_list = MagicMock(return_value=[])
|
|
resp = self.viewer_client.get(
|
|
"/v1/agents", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 200
|
|
|
|
def test_should_block_view_only_user_from_creating_agent(self):
|
|
resp = self.viewer_client.post(
|
|
"/v1/agents",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 403
|
|
|
|
def test_should_block_view_only_user_from_deleting_agent(self):
|
|
resp = self.viewer_client.delete(
|
|
"/v1/agents/agent-123", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 403
|
|
|
|
|
|
class TestAgentRBACProxyAdmin:
|
|
"""Proxy admins should have full CRUD access to agents."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _setup(self, monkeypatch):
|
|
self.admin_client = _make_app_with_role(LitellmUserRoles.PROXY_ADMIN)
|
|
self.mock_registry = MagicMock()
|
|
monkeypatch.setattr(agent_endpoints, "AGENT_REGISTRY", self.mock_registry)
|
|
|
|
def test_should_allow_admin_to_create_agent(self, monkeypatch):
|
|
with patch("litellm.proxy.proxy_server.prisma_client") as mock_prisma:
|
|
self.mock_registry.get_agent_by_name = MagicMock(return_value=None)
|
|
self.mock_registry.add_agent_to_db = AsyncMock(
|
|
return_value=_sample_agent_response()
|
|
)
|
|
self.mock_registry.register_agent = MagicMock()
|
|
resp = self.admin_client.post(
|
|
"/v1/agents",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 200
|
|
|
|
def test_create_agent_applies_litellm_merge_to_stored_card(self):
|
|
"""The card stored in the DB must reflect the LiteLLM-fronting merge."""
|
|
with patch("litellm.proxy.proxy_server.prisma_client"):
|
|
self.mock_registry.get_agent_by_name = MagicMock(return_value=None)
|
|
self.mock_registry.add_agent_to_db = AsyncMock(
|
|
return_value=_sample_agent_response()
|
|
)
|
|
self.mock_registry.register_agent = MagicMock()
|
|
|
|
self.admin_client.post(
|
|
"/v1/agents",
|
|
json=_sample_agent_config(),
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
|
|
call_kwargs = self.mock_registry.add_agent_to_db.await_args.kwargs
|
|
stored_card = call_kwargs["agent"]["agent_card_params"]
|
|
new_agent_id = call_kwargs["agent_id"]
|
|
|
|
# Top-level url is retained for runtime A2A invocation (the public
|
|
# well-known endpoint rewrites it before exposing to clients);
|
|
# supportedInterfaces points at the proxy.
|
|
assert stored_card["url"] == "http://localhost"
|
|
assert stored_card["supportedInterfaces"][0]["protocolBinding"] == "JSONRPC"
|
|
assert stored_card["supportedInterfaces"][0]["url"].endswith(
|
|
f"/a2a/{new_agent_id}"
|
|
)
|
|
# Security scheme is the LiteLLM scheme.
|
|
assert "LiteLLMKey" in stored_card["securitySchemes"]
|
|
|
|
def test_should_allow_admin_to_delete_agent(self):
|
|
existing = {
|
|
"agent_id": "agent-123",
|
|
"agent_name": "Existing Agent",
|
|
"agent_card_params": _sample_agent_card_params(),
|
|
}
|
|
with patch("litellm.proxy.proxy_server.prisma_client") as mock_prisma:
|
|
mock_prisma.db.litellm_agentstable.find_unique = AsyncMock(
|
|
return_value=existing
|
|
)
|
|
self.mock_registry.delete_agent_from_db = AsyncMock()
|
|
self.mock_registry.deregister_agent = MagicMock()
|
|
resp = self.admin_client.delete(
|
|
"/v1/agents/agent-123", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 200
|
|
|
|
|
|
class TestCheckAgentManagementPermission:
|
|
"""Unit tests for the _check_agent_management_permission helper."""
|
|
|
|
def test_should_allow_proxy_admin(self):
|
|
auth = UserAPIKeyAuth(user_id="admin", user_role=LitellmUserRoles.PROXY_ADMIN)
|
|
_check_agent_management_permission(auth)
|
|
|
|
@pytest.mark.parametrize(
|
|
"role",
|
|
[
|
|
LitellmUserRoles.INTERNAL_USER,
|
|
LitellmUserRoles.INTERNAL_USER_VIEW_ONLY,
|
|
LitellmUserRoles.PROXY_ADMIN_VIEW_ONLY,
|
|
],
|
|
)
|
|
def test_should_block_non_admin_roles(self, role):
|
|
from fastapi import HTTPException
|
|
|
|
auth = UserAPIKeyAuth(user_id="user", user_role=role)
|
|
with pytest.raises(HTTPException) as exc_info:
|
|
_check_agent_management_permission(auth)
|
|
assert exc_info.value.status_code == 403
|
|
|
|
|
|
class TestAgentRoutesIncludesAgentIdPattern:
|
|
"""Verify that agent_routes includes the {agent_id} pattern for route access."""
|
|
|
|
def test_should_include_agent_id_pattern(self):
|
|
from litellm.proxy._types import LiteLLMRoutes
|
|
|
|
assert "/v1/agents/{agent_id}" in LiteLLMRoutes.agent_routes.value
|
|
|
|
|
|
class TestAgentHealthCheck:
|
|
"""Tests for the health_check query parameter on GET /v1/agents."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _setup(self, monkeypatch):
|
|
from litellm.proxy.agent_endpoints import agent_registry as ar_mod
|
|
|
|
self.admin_client = _make_app_with_role(LitellmUserRoles.PROXY_ADMIN)
|
|
self.mock_registry = MagicMock()
|
|
monkeypatch.setattr(ar_mod, "global_agent_registry", self.mock_registry)
|
|
# Ensure prisma_client is None so the endpoint skips DB queries.
|
|
# In CI with parallel workers, a MagicMock can leak from other test
|
|
# scopes, causing "object MagicMock can't be used in 'await'" errors.
|
|
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", None)
|
|
|
|
def _make_agent(self, agent_id: str, url: str | None = None) -> AgentResponse:
|
|
card = _sample_agent_card_params()
|
|
if url is not None:
|
|
card["url"] = url
|
|
else:
|
|
card.pop("url", None)
|
|
return AgentResponse(
|
|
agent_id=agent_id,
|
|
agent_name=f"Agent {agent_id}",
|
|
agent_card_params=card,
|
|
litellm_params={},
|
|
)
|
|
|
|
def test_should_return_all_agents_when_health_check_disabled(self):
|
|
agents = [
|
|
self._make_agent("a1", "http://reachable"),
|
|
self._make_agent("a2", "http://unreachable"),
|
|
]
|
|
self.mock_registry.get_agent_list = MagicMock(return_value=agents)
|
|
|
|
resp = self.admin_client.get(
|
|
"/v1/agents", headers={"Authorization": "Bearer k"}
|
|
)
|
|
assert resp.status_code == 200
|
|
assert len(resp.json()) == 2
|
|
|
|
def test_should_filter_unhealthy_agents_when_health_check_enabled(
|
|
self, monkeypatch
|
|
):
|
|
agents = [
|
|
self._make_agent("a1", "http://reachable"),
|
|
self._make_agent("a2", "http://unreachable"),
|
|
]
|
|
self.mock_registry.get_agent_list = MagicMock(return_value=agents)
|
|
|
|
results = iter(
|
|
[
|
|
{"agent_id": "a1", "healthy": True},
|
|
{"agent_id": "a2", "healthy": False, "error": "Connection refused"},
|
|
]
|
|
)
|
|
monkeypatch.setattr(
|
|
agent_endpoints,
|
|
"_check_agent_url_health",
|
|
AsyncMock(side_effect=lambda agent: next(results)),
|
|
)
|
|
|
|
resp = self.admin_client.get(
|
|
"/v1/agents?health_check=true",
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 200
|
|
data = resp.json()
|
|
assert len(data) == 1
|
|
assert data[0]["agent_id"] == "a1"
|
|
|
|
def test_should_return_empty_list_when_all_agents_unhealthy(self, monkeypatch):
|
|
agents = [self._make_agent("a1", "http://down")]
|
|
self.mock_registry.get_agent_list = MagicMock(return_value=agents)
|
|
monkeypatch.setattr(
|
|
agent_endpoints,
|
|
"_check_agent_url_health",
|
|
AsyncMock(
|
|
return_value={"agent_id": "a1", "healthy": False, "error": "timeout"}
|
|
),
|
|
)
|
|
|
|
resp = self.admin_client.get(
|
|
"/v1/agents?health_check=true",
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 200
|
|
assert len(resp.json()) == 0
|
|
|
|
def test_should_return_all_agents_when_all_healthy(self, monkeypatch):
|
|
agents = [
|
|
self._make_agent("a1", "http://ok1"),
|
|
self._make_agent("a2", "http://ok2"),
|
|
]
|
|
self.mock_registry.get_agent_list = MagicMock(return_value=agents)
|
|
|
|
results = iter(
|
|
[
|
|
{"agent_id": "a1", "healthy": True},
|
|
{"agent_id": "a2", "healthy": True},
|
|
]
|
|
)
|
|
monkeypatch.setattr(
|
|
agent_endpoints,
|
|
"_check_agent_url_health",
|
|
AsyncMock(side_effect=lambda agent: next(results)),
|
|
)
|
|
|
|
resp = self.admin_client.get(
|
|
"/v1/agents?health_check=true",
|
|
headers={"Authorization": "Bearer k"},
|
|
)
|
|
assert resp.status_code == 200
|
|
assert len(resp.json()) == 2
|
|
|
|
|
|
class TestCheckAgentUrlHealth:
|
|
"""Unit tests for the _check_agent_url_health helper."""
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_should_return_healthy_when_no_url(self):
|
|
from litellm.proxy.agent_endpoints.endpoints import _check_agent_url_health
|
|
|
|
agent = AgentResponse(
|
|
agent_id="no-url",
|
|
agent_name="No URL Agent",
|
|
agent_card_params={"name": "test"},
|
|
litellm_params={},
|
|
)
|
|
result = await _check_agent_url_health(agent)
|
|
assert result["healthy"] is True
|
|
assert "error" not in result
|
|
|
|
@pytest.mark.asyncio
|
|
@patch("litellm.proxy.agent_endpoints.endpoints.get_async_httpx_client")
|
|
async def test_should_return_healthy_for_200(self, mock_get_client):
|
|
from litellm.proxy.agent_endpoints.endpoints import _check_agent_url_health
|
|
|
|
mock_response = MagicMock()
|
|
mock_response.status_code = 200
|
|
mock_client = AsyncMock()
|
|
mock_client.get = AsyncMock(return_value=mock_response)
|
|
mock_get_client.return_value = mock_client
|
|
|
|
agent = AgentResponse(
|
|
agent_id="ok",
|
|
agent_name="OK Agent",
|
|
agent_card_params={"url": "http://example.com"},
|
|
litellm_params={},
|
|
)
|
|
result = await _check_agent_url_health(agent)
|
|
assert result["healthy"] is True
|
|
|
|
@pytest.mark.asyncio
|
|
@patch("litellm.proxy.agent_endpoints.endpoints.get_async_httpx_client")
|
|
async def test_should_return_unhealthy_for_500(self, mock_get_client):
|
|
from litellm.proxy.agent_endpoints.endpoints import _check_agent_url_health
|
|
|
|
mock_response = MagicMock()
|
|
mock_response.status_code = 500
|
|
mock_client = AsyncMock()
|
|
mock_client.get = AsyncMock(return_value=mock_response)
|
|
mock_get_client.return_value = mock_client
|
|
|
|
agent = AgentResponse(
|
|
agent_id="err",
|
|
agent_name="Error Agent",
|
|
agent_card_params={"url": "http://failing.com"},
|
|
litellm_params={},
|
|
)
|
|
result = await _check_agent_url_health(agent)
|
|
assert result["healthy"] is False
|
|
assert "HTTP 500" in result["error"]
|
|
|
|
@pytest.mark.asyncio
|
|
@patch("litellm.proxy.agent_endpoints.endpoints.get_async_httpx_client")
|
|
async def test_should_return_unhealthy_on_connection_error(self, mock_get_client):
|
|
from litellm.proxy.agent_endpoints.endpoints import _check_agent_url_health
|
|
|
|
mock_client = AsyncMock()
|
|
mock_client.get = AsyncMock(side_effect=Exception("Connection refused"))
|
|
mock_get_client.return_value = mock_client
|
|
|
|
agent = AgentResponse(
|
|
agent_id="down",
|
|
agent_name="Down Agent",
|
|
agent_card_params={"url": "http://down.com"},
|
|
litellm_params={},
|
|
)
|
|
result = await _check_agent_url_health(agent)
|
|
assert result["healthy"] is False
|
|
assert "Connection refused" in result["error"]
|
|
|
|
@pytest.mark.asyncio
|
|
@patch("litellm.proxy.agent_endpoints.endpoints.get_async_httpx_client")
|
|
async def test_should_treat_404_as_healthy(self, mock_get_client):
|
|
"""A 404 means the server is reachable, just not the specific path."""
|
|
from litellm.proxy.agent_endpoints.endpoints import _check_agent_url_health
|
|
|
|
mock_response = MagicMock()
|
|
mock_response.status_code = 404
|
|
mock_client = AsyncMock()
|
|
mock_client.get = AsyncMock(return_value=mock_response)
|
|
mock_get_client.return_value = mock_client
|
|
|
|
agent = AgentResponse(
|
|
agent_id="notfound",
|
|
agent_name="NotFound Agent",
|
|
agent_card_params={"url": "http://example.com/missing"},
|
|
litellm_params={},
|
|
)
|
|
result = await _check_agent_url_health(agent)
|
|
assert result["healthy"] is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"base_url",
|
|
["http://0.0.0.0:4000/", "http://localhost:4000/", "https://api.example.com/"],
|
|
)
|
|
def test_merged_agent_card_url_has_no_double_slash_without_proxy_base_url(
|
|
monkeypatch, base_url
|
|
):
|
|
"""Without PROXY_BASE_URL, request.base_url carries a trailing slash; the merged
|
|
card's supportedInterfaces URL must still join cleanly (no `//a2a`)."""
|
|
from litellm.proxy.agent_endpoints.endpoints import _build_merged_agent_card
|
|
|
|
monkeypatch.delenv("PROXY_BASE_URL", raising=False)
|
|
monkeypatch.delenv("SERVER_ROOT_PATH", raising=False)
|
|
|
|
http_request = MagicMock()
|
|
http_request.base_url = base_url
|
|
|
|
merged = _build_merged_agent_card(
|
|
_sample_agent_card_params(),
|
|
agent_id="agent-xyz",
|
|
http_request=http_request,
|
|
agent_name="Test Agent",
|
|
)
|
|
|
|
interface_url = merged["supportedInterfaces"][0]["url"]
|
|
assert interface_url == f"{base_url.rstrip('/')}/a2a/agent-xyz"
|
|
assert "//a2a" not in interface_url
|