scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

fix(proxy): populate access_via_team_ids on /v1/model/info (#30274)

Browse source 
* fix(proxy): populate access_via_team_ids on /v1/model/info

Team metadata enrichment previously only ran on /v2/model/info with
include_team_models=true, leaving /v1/model/info without
access_via_team_ids for project model-picker flows.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(dashboard): sync OpenAPI schema for /v1/model/info query params

Add include_team_models and teamId to the generated schema for /model/info
and /v1/model/info after the proxy endpoint gained team-access filtering.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): always return direct_access on /v1/model/info

Set direct_access to true or false on every enriched model so clients
can filter without treating a missing field as ambiguous.

Co-authored-by: Cursor <cursoragent@cursor.com>

* perf(proxy): fail fast when teamId is set without a connected DB on /v1/model/info

Raise the db_not_connected error before building, enriching, and translating the model list instead of after, so a teamId query against a proxy with no database no longer wastes the full enrichment pipeline.

* fix(proxy): fail fast when include_team_models is set without a database

include_team_models=True relies on _populate_team_access_on_models to set
direct_access/access_via_team_ids, which only runs when a database is connected.
Without one, _filter_models_to_user_accessible discarded every model and the
endpoint returned an empty list with HTTP 200. Mirror the teamId guard so the
request fails fast with a clear db_not_connected error before any model-list work.

* fix(proxy): populate direct_access on single-model /model/info lookup

The /v1/model/info list path populates model_info.direct_access (and
access_via_team_ids) when a database is connected, but the
litellm_model_id single-model lookup returned early without it. This
made the two endpoints disagree, breaking the parity assertion in
test_get_specific_model. Run the same population on the single-model
path so both responses match.

* fix(proxy): apply no-DB fast-fail before litellm_model_id branch

The teamId/include_team_models no-DB guard sat after the litellm_model_id
early return, so ?litellm_model_id=X&teamId=Y with no DB returned 200 with
unpopulated access fields instead of the 500 raised on every other path.
Move the guard ahead of the branch so the fast-fail is uniform.

* fix(proxy): apply teamId/include_team_models filters on single-model lookup

The litellm_model_id early-return branch in model_info_v1 populated the
team access fields but returned before the teamId and include_team_models
filters ran, so a single-model lookup surfaced the deployment regardless
of team access when the DB was connected. Run both filters on the
single-model list before returning so the documented query params behave
the same with and without litellm_model_id.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
This commit is contained in:
Sameer Kankute 2026-06-12 22:19:09 +05:30 • committed by GitHub
parent 729b005e4e
commit 7d1f68e72a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 368 additions and 16 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

116
litellm/proxy/proxy_server.py
View file

@ -11031,16 +11031,26 @@ def get_direct_access_models(
return direct_access_models
async def get_all_team_and_direct_access_models(
def _filter_models_to_user_accessible(all_models: List[Dict]) -> List[Dict]:
"""Keep only deployments the caller can use via direct access or team membership."""
return [
_model
for _model in all_models
if _model.get("model_info", {}).get("direct_access", False)
or _model.get("model_info", {}).get("access_via_team_ids", [])
]
async def _populate_team_access_on_models(
user_api_key_dict: UserAPIKeyAuth,
prisma_client: PrismaClient,
llm_router: Router,
all_models: List[Dict],
) -> List[Dict]:
"""
Get all models across all teams user is in.
Populate `model_info.access_via_team_ids` and `model_info.direct_access`
without filtering the model list.
"""
user_teams: Optional[Union[List[str], Literal["*"]]] = None
direct_access_models: List[str] = []
if user_api_key_dict.user_role == LitellmUserRoles.PROXY_ADMIN:
@ -11059,7 +11069,6 @@ async def get_all_team_and_direct_access_models(
user_db_object=user_object,
llm_router=llm_router,
)
## ADD ACCESS_VIA_TEAM_IDS TO ALL MODELS
if user_teams is not None:
team_models = await get_all_team_models(
user_teams=user_teams,
@ -11082,23 +11091,33 @@ async def get_all_team_and_direct_access_models(
model_id, []
)
## ADD DIRECT_ACCESS TO RELEVANT MODELS
direct_access_model_ids = set(direct_access_models)
for _model in all_models:
model_id = _model.get("model_info", {}).get("id", None)
if model_id is not None and model_id in direct_access_models:
_model["model_info"]["direct_access"] = True
if model_id is not None:
_model["model_info"]["direct_access"] = model_id in direct_access_model_ids
## FILTER OUT MODELS THAT ARE NOT IN DIRECT_ACCESS_MODELS OR ACCESS_VIA_TEAM_IDS - only show user models they can call
all_models = [
_model
for _model in all_models
if _model.get("model_info", {}).get("direct_access", False)
or _model.get("model_info", {}).get("access_via_team_ids", [])
]
return all_models
async def get_all_team_and_direct_access_models(
user_api_key_dict: UserAPIKeyAuth,
prisma_client: PrismaClient,
llm_router: Router,
all_models: List[Dict],
) -> List[Dict]:
"""
Get all models across all teams user is in.
"""
all_models = await _populate_team_access_on_models(
user_api_key_dict=user_api_key_dict,
prisma_client=prisma_client,
llm_router=llm_router,
all_models=all_models,
)
return _filter_models_to_user_accessible(all_models)
def _enrich_model_info_with_litellm_data(
model: Dict[str, Any], debug: bool = False, llm_router: Optional[Router] = None
) -> Dict[str, Any]:
@ -12633,6 +12652,14 @@ def _get_proxy_model_info(model: dict) -> dict:
async def model_info_v1( # noqa: PLR0915
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
litellm_model_id: Optional[str] = None,
include_team_models: Optional[bool] = fastapi.Query(
False,
description="When true, filter to deployments the caller can use via direct access or team membership.",
),
teamId: Optional[str] = fastapi.Query(
None,
description="Filter models by team ID. Returns models with direct_access=True or teamId in access_via_team_ids",
),
):
"""
Provides more info about each model in /models, including config.yaml descriptions (except api key and api base)
@ -12642,6 +12669,11 @@ async def model_info_v1( # noqa: PLR0915
- When litellm_model_id is passed, it will return the info for that specific model
- When litellm_model_id is not passed, it will return the info for all models
- include_team_models: When true, filter to deployments the caller can use (same as /v2/model/info).
- teamId: Filter to models accessible by the given team.
Each model in the list response includes `model_info.access_via_team_ids` and
`model_info.direct_access` when the proxy database is connected.
Returns:
Returns a dictionary containing information about each model.
@ -12668,6 +12700,12 @@ async def model_info_v1( # noqa: PLR0915
"""
global llm_model_list, general_settings, user_config_file_path, proxy_config, llm_router, user_model
# Unit tests call this handler directly; FastAPI normally resolves Query defaults.
if not isinstance(include_team_models, bool):
include_team_models = False
if not isinstance(teamId, str):
teamId = None
if user_model is not None:
# user is trying to get specific model from litellm router
try:
@ -12704,6 +12742,14 @@ async def model_info_v1( # noqa: PLR0915
},
)
if prisma_client is None and (
include_team_models or (teamId is not None and teamId.strip())
):
raise HTTPException(
status_code=500,
detail={"error": CommonProxyErrors.db_not_connected_error.value},
)
if litellm_model_id is not None:
# user is trying to get specific model from litellm router
deployment_info = llm_router.get_deployment(model_id=litellm_model_id)
@ -12717,7 +12763,25 @@ async def model_info_v1( # noqa: PLR0915
_deployment_info_dict = _get_proxy_model_info(
model=deployment_info.model_dump(exclude_none=True)
)
return {"data": [_deployment_info_dict]}
single_model_list: List[dict] = [_deployment_info_dict]
if prisma_client is not None:
single_model_list = await _populate_team_access_on_models(
user_api_key_dict=user_api_key_dict,
prisma_client=prisma_client,
llm_router=llm_router,
all_models=single_model_list,
)
if include_team_models:
single_model_list = _filter_models_to_user_accessible(single_model_list)
if teamId is not None and teamId.strip():
single_model_list = await _filter_models_by_team_id(
all_models=single_model_list,
team_id=teamId.strip(),
prisma_client=prisma_client,
llm_router=llm_router,
user_api_key_dict=user_api_key_dict,
)
return {"data": single_model_list}
# Return router deployments (same source as /v2/model/info), not wildcard-
# expanded model names from get_complete_model_list(). Team-scoped rows
@ -12749,6 +12813,17 @@ async def model_info_v1( # noqa: PLR0915
)
]
if prisma_client is not None:
all_models = await _populate_team_access_on_models(
user_api_key_dict=user_api_key_dict,
prisma_client=prisma_client,
llm_router=llm_router,
all_models=all_models,
)
if include_team_models:
all_models = _filter_models_to_user_accessible(all_models)
all_models = [
_translate_model_name_for_response(
_enrich_model_info_with_litellm_data(model=model, llm_router=llm_router)
@ -12756,6 +12831,15 @@ async def model_info_v1( # noqa: PLR0915
for model in all_models
]
if teamId is not None and teamId.strip():
all_models = await _filter_models_by_team_id(
all_models=all_models,
team_id=teamId.strip(),
prisma_client=cast(PrismaClient, prisma_client),
llm_router=llm_router,
user_api_key_dict=user_api_key_dict,
)
verbose_proxy_logger.debug("all_models: %s", all_models)
return {"data": all_models}

250
tests/test_litellm/proxy/proxy_server/test_team_model_name_translation.py
View file

@ -279,6 +279,11 @@ async def test_model_info_v1_unrestricted_key_hides_other_team_byok(monkeypatch)
prisma_client = MagicMock()
caller_user_row = MagicMock()
caller_user_row.teams = ["team-abc-123"]
caller_user_row.model_dump.return_value = {
"user_id": "user-1",
"teams": ["team-abc-123"],
"models": [],
}
prisma_client.db.litellm_usertable.find_unique = AsyncMock(
return_value=caller_user_row
)
@ -287,6 +292,7 @@ async def test_model_info_v1_unrestricted_key_hides_other_team_byok(monkeypatch)
monkeypatch.setattr(ps, "llm_model_list", router.model_list)
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", prisma_client)
monkeypatch.setattr(ps, "get_all_team_models", AsyncMock(return_value={}))
monkeypatch.setattr(
ps, "_enrich_model_info_with_litellm_data", lambda model, **kw: model
)
@ -343,3 +349,247 @@ async def test_model_info_v1_service_key_hides_all_team_byok(monkeypatch):
resp = await ps.model_info_v1(user_api_key_dict=caller, litellm_model_id=None)
assert [m["model_info"]["id"] for m in resp["data"]] == ["global-id-1"]
@pytest.mark.asyncio
async def test_model_info_v1_populates_access_via_team_ids(monkeypatch):
"""`/v1/model/info` must populate access_via_team_ids when the DB is connected."""
team_id = "team-abc-123"
team_row = _team_row()
global_row = {
"model_name": "gpt-4o",
"litellm_params": {"model": "gpt-4o"},
"model_info": {"id": "global-id-1", "db_model": False},
}
router = MagicMock()
router.model_list = [team_row, global_row]
router.get_model_names.return_value = ["gpt-4o", "team-claude-sonnet"]
router.get_model_access_groups.return_value = {}
router.get_model_ids.return_value = ["global-id-1"]
prisma_client = MagicMock()
async def _fake_populate(**kwargs):
for model in kwargs["all_models"]:
model_id = model["model_info"]["id"]
if model_id == "byok-id-1":
model["model_info"]["access_via_team_ids"] = [team_id]
model["model_info"]["direct_access"] = False
elif model_id == "global-id-1":
model["model_info"]["direct_access"] = True
return kwargs["all_models"]
monkeypatch.setattr(ps, "user_model", None)
monkeypatch.setattr(ps, "llm_model_list", router.model_list)
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", prisma_client)
monkeypatch.setattr(ps, "_populate_team_access_on_models", _fake_populate)
monkeypatch.setattr(
ps, "_enrich_model_info_with_litellm_data", lambda model, **kw: model
)
admin = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.PROXY_ADMIN, team_models=[]
)
resp = await ps.model_info_v1(user_api_key_dict=admin, litellm_model_id=None)
by_id = {m["model_info"]["id"]: m for m in resp["data"]}
assert by_id["byok-id-1"]["model_info"]["access_via_team_ids"] == [team_id]
assert by_id["byok-id-1"]["model_info"]["direct_access"] is False
assert by_id["global-id-1"]["model_info"]["direct_access"] is True
@pytest.mark.asyncio
async def test_populate_team_access_sets_direct_access_false_by_default(monkeypatch):
"""Team-accessible models without direct access must return direct_access=false."""
team_row = _team_row()
global_row = {
"model_name": "gpt-4o",
"litellm_params": {"model": "gpt-4o"},
"model_info": {"id": "global-id-1", "db_model": False},
}
router = MagicMock()
router.get_model_ids.return_value = ["global-id-1"]
monkeypatch.setattr(
ps,
"get_all_team_models",
AsyncMock(return_value={"byok-id-1": ["team-abc-123"]}),
)
admin = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.PROXY_ADMIN, team_models=[]
)
result = await ps._populate_team_access_on_models(
user_api_key_dict=admin,
prisma_client=MagicMock(),
llm_router=router,
all_models=[team_row, global_row],
)
by_id = {m["model_info"]["id"]: m for m in result}
assert by_id["byok-id-1"]["model_info"]["direct_access"] is False
assert by_id["global-id-1"]["model_info"]["direct_access"] is True
@pytest.mark.asyncio
async def test_model_info_v1_team_id_without_db_fails_fast(monkeypatch):
"""`teamId` without a connected DB raises 500 before any enrichment work runs."""
router = MagicMock()
router.model_list = [_team_row()]
enrich_spy = MagicMock(side_effect=lambda model, **kw: model)
monkeypatch.setattr(ps, "user_model", None)
monkeypatch.setattr(ps, "llm_model_list", router.model_list)
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", None)
monkeypatch.setattr(ps, "_enrich_model_info_with_litellm_data", enrich_spy)
admin = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.PROXY_ADMIN, team_models=[]
)
with pytest.raises(ps.HTTPException) as exc_info:
await ps.model_info_v1(
user_api_key_dict=admin, litellm_model_id=None, teamId="team-abc-123"
)
assert exc_info.value.status_code == 500
assert "DB not connected" in exc_info.value.detail["error"]
enrich_spy.assert_not_called()
@pytest.mark.asyncio
async def test_model_info_v1_include_team_models_without_db_fails_fast(monkeypatch):
"""`include_team_models` without a connected DB raises 500 instead of silently
returning an empty list (the access fields can only be populated from the DB)."""
router = MagicMock()
router.model_list = [_team_row()]
enrich_spy = MagicMock(side_effect=lambda model, **kw: model)
monkeypatch.setattr(ps, "user_model", None)
monkeypatch.setattr(ps, "llm_model_list", router.model_list)
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", None)
monkeypatch.setattr(ps, "_enrich_model_info_with_litellm_data", enrich_spy)
admin = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.PROXY_ADMIN, team_models=[]
)
with pytest.raises(ps.HTTPException) as exc_info:
await ps.model_info_v1(
user_api_key_dict=admin, litellm_model_id=None, include_team_models=True
)
assert exc_info.value.status_code == 500
assert "DB not connected" in exc_info.value.detail["error"]
enrich_spy.assert_not_called()
@pytest.mark.asyncio
async def test_model_info_v1_litellm_model_id_team_id_without_db_fails_fast(
monkeypatch,
):
"""`litellm_model_id` + `teamId` without a connected DB must raise 500 too, not
return 200 with a model dict missing direct_access/access_via_team_ids."""
router = MagicMock()
router.model_list = [_team_row()]
monkeypatch.setattr(ps, "user_model", None)
monkeypatch.setattr(ps, "llm_model_list", router.model_list)
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", None)
admin = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.PROXY_ADMIN, team_models=[]
)
with pytest.raises(ps.HTTPException) as exc_info:
await ps.model_info_v1(
user_api_key_dict=admin,
litellm_model_id="byok-id-1",
teamId="team-abc-123",
)
assert exc_info.value.status_code == 500
assert "DB not connected" in exc_info.value.detail["error"]
router.get_deployment.assert_not_called()
@pytest.mark.asyncio
async def test_model_info_v1_litellm_model_id_include_team_models_filters_inaccessible(
monkeypatch,
):
"""`litellm_model_id` + `include_team_models` must drop a model the caller cannot
use instead of returning it unconditionally from the single-model lookup."""
team_row = _team_row()
router = MagicMock()
deployment = MagicMock()
deployment.model_dump.return_value = team_row
router.get_deployment.return_value = deployment
async def _fake_populate(**kwargs):
for model in kwargs["all_models"]:
model["model_info"]["direct_access"] = False
model["model_info"]["access_via_team_ids"] = []
return kwargs["all_models"]
monkeypatch.setattr(ps, "user_model", None)
monkeypatch.setattr(ps, "llm_model_list", [team_row])
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", MagicMock())
monkeypatch.setattr(ps, "_get_proxy_model_info", lambda model: team_row)
monkeypatch.setattr(ps, "_populate_team_access_on_models", _fake_populate)
caller = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.INTERNAL_USER, team_models=[]
)
resp = await ps.model_info_v1(
user_api_key_dict=caller,
litellm_model_id="byok-id-1",
include_team_models=True,
)
assert resp["data"] == []
@pytest.mark.asyncio
async def test_model_info_v1_litellm_model_id_team_id_applies_team_filter(monkeypatch):
"""`litellm_model_id` + `teamId` must run the teamId filter on the single model
rather than returning it regardless of the team's access."""
team_row = _team_row()
router = MagicMock()
deployment = MagicMock()
deployment.model_dump.return_value = team_row
router.get_deployment.return_value = deployment
async def _fake_populate(**kwargs):
return kwargs["all_models"]
team_filter = AsyncMock(return_value=[])
monkeypatch.setattr(ps, "user_model", None)
monkeypatch.setattr(ps, "llm_model_list", [team_row])
monkeypatch.setattr(ps, "llm_router", router)
monkeypatch.setattr(ps, "prisma_client", MagicMock())
monkeypatch.setattr(ps, "_get_proxy_model_info", lambda model: team_row)
monkeypatch.setattr(ps, "_populate_team_access_on_models", _fake_populate)
monkeypatch.setattr(ps, "_filter_models_by_team_id", team_filter)
admin = UserAPIKeyAuth(
user_id="u", user_role=LitellmUserRoles.PROXY_ADMIN, team_models=[]
)
resp = await ps.model_info_v1(
user_api_key_dict=admin,
litellm_model_id="byok-id-1",
teamId="other-team",
)
assert resp["data"] == []
team_filter.assert_awaited_once()
assert team_filter.await_args.kwargs["team_id"] == "other-team"
assert team_filter.await_args.kwargs["all_models"] == [team_row]

18
ui/litellm-dashboard/src/lib/http/schema.d.ts generated vendored
View file

@ -7338,6 +7338,11 @@ export interface paths {
*
* - When litellm_model_id is passed, it will return the info for that specific model
* - When litellm_model_id is not passed, it will return the info for all models
* - include_team_models: When true, filter to deployments the caller can use (same as /v2/model/info).
* - teamId: Filter to models accessible by the given team.
*
* Each model in the list response includes `model_info.access_via_team_ids` and
* `model_info.direct_access` when the proxy database is connected.
*
* Returns:
* Returns a dictionary containing information about each model.
@ -16565,6 +16570,11 @@ export interface paths {
*
* - When litellm_model_id is passed, it will return the info for that specific model
* - When litellm_model_id is not passed, it will return the info for all models
* - include_team_models: When true, filter to deployments the caller can use (same as /v2/model/info).
* - teamId: Filter to models accessible by the given team.
*
* Each model in the list response includes `model_info.access_via_team_ids` and
* `model_info.direct_access` when the proxy database is connected.
*
* Returns:
* Returns a dictionary containing information about each model.
@ -42440,6 +42450,10 @@ export interface operations {
parameters: {
query?: {
litellm_model_id?: string | null;
/** @description When true, filter to deployments the caller can use via direct access or team membership. */
include_team_models?: boolean | null;
/** @description Filter models by team ID. Returns models with direct_access=True or teamId in access_via_team_ids */
teamId?: string | null;
};
header?: never;
path?: never;
@ -53705,6 +53719,10 @@ export interface operations {
parameters: {
query?: {
litellm_model_id?: string | null;
/** @description When true, filter to deployments the caller can use via direct access or team membership. */
include_team_models?: boolean | null;
/** @description Filter models by team ID. Returns models with direct_access=True or teamId in access_via_team_ids */
teamId?: string | null;
};
header?: never;
path?: never;