feat(proxy): auth_v2 slice 8 - domain-scoped roles + docs

Adds tenancy via domain-scoped role assignment (casbin g3): a role can be
granted globally (g) or only within a domain (g3), e.g. admin within team:eng
but nowhere else. The matcher accepts either, so existing global assignments are
unchanged. The policy store buckets g/g2/g3 separately and the assignment
endpoint takes an optional domain.

Adds litellm/proxy/auth/v2/README.md documenting configuration, the policy
admin API, and a live verification runbook.

Tests cover domain-scoped vs global role resolution, coexistence of both, the
g3 split in the policy store, and g3 assignment rule construction.

litellm/proxy/auth/v2/README.md

@@ -0,0 +1,108 @@
+# auth_v2
+
+A clean-slate authentication and authorization path for the proxy, built on
+industry-standard libraries (authlib for JWT/OAuth2, casbin for RBAC/ABAC). It
+runs in parallel with the existing auth behind a flag; when on, the existing
+auth is bypassed entirely.
+
+## Enabling
+
+```yaml
+general_settings:
+  auth_version: v2
+```
+
+With the flag off, nothing changes. With it on, every request flows through the
+auth_v2 entry point. Routes auth_v2 does not yet govern are left open and log a
+warning ("not yet protected"); this is a work-in-progress path and must not be
+enabled in production until coverage is complete.
+
+## How it works
+
+Authentication is a chain dispatched by credential shape:
+
+- master key (exact, constant-time compare) -> proxy admin
+- virtual key (`sk-...`) -> resolved via the existing key store
+- JWT (3-part bearer) -> authlib: JWKS fetch/cache, signature + exp/iss/aud
+- opaque token -> authlib RFC 7662 introspection (when configured)
+
+Authorization is a single casbin engine governing both planes:
+
+- control plane (management routes): RBAC policy rows over resources
+  `model`, `team`, `key`, `user`, `organization`, `policy`, with actions
+  `read` / `write` / `delete` / `manage`. Supports per-resource ids, resource
+  groups (casbin `g2`, mapping to access groups), and roles that are either
+  global or scoped to a domain (casbin `g3`, e.g. "admin within team:eng").
+- data plane (inference: chat/completions, completions, embeddings, responses):
+  casbin ABAC over the principal's allowed-model attribute, read from the
+  already-loaded key. Empty list / `*` / `all-proxy-models` are unrestricted,
+  matching existing key semantics.
+
+Policies live in `LiteLLM_CasbinRule`, loaded on cold routes with a short TTL
+cache; the inference path reads no policy store. A bootstrap policy keeps
+`proxy_admin` fully authorized.
+
+## Configuration
+
+JWT (`general_settings.auth_v2_jwt` or env):
+
+```yaml
+auth_v2_jwt:
+  jwks_uri: https://idp.example/.well-known/jwks.json   # AUTH_V2_JWKS_URI
+  issuer: https://idp.example                            # AUTH_V2_JWT_ISSUER
+  audience: litellm                                      # AUTH_V2_JWT_AUDIENCE
+  user_id_claim: sub
+  team_claim: team_id
+  role_claim: groups
+  role_map: { litellm-admins: proxy_admin }
+```
+
+OAuth2 introspection (`general_settings.auth_v2_oauth2` or env):
+
+```yaml
+auth_v2_oauth2:
+  introspection_endpoint: https://idp.example/introspect  # AUTH_V2_OAUTH2_INTROSPECTION_ENDPOINT
+  client_id: litellm                                      # AUTH_V2_OAUTH2_CLIENT_ID
+  client_secret: ...                                      # AUTH_V2_OAUTH2_CLIENT_SECRET
+  scope_claim: scope
+  role_map: { "litellm:admin": proxy_admin }
+```
+
+## Policy administration
+
+Admin-only endpoints; the policy surface governs itself.
+
+```bash
+# grant a role read access to all models
+curl -sX POST localhost:4000/auth/v2/policy/permission/add \
+  -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
+  -d '{"role":"model_reader","resource":"model","action":"read"}'
+
+# assign the role to a user (globally, or within a domain)
+curl -sX POST localhost:4000/auth/v2/policy/assignment/add \
+  -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
+  -d '{"subject_type":"user","subject_id":"u_123","role":"model_reader"}'
+
+curl -sX POST localhost:4000/auth/v2/policy/assignment/add \
+  -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
+  -d '{"subject_type":"user","subject_id":"u_123","role":"team_admin","domain":"team:eng"}'
+
+# list all rules
+curl -s localhost:4000/auth/v2/policy/list -H "Authorization: Bearer $MASTER_KEY"
+```
+
+## Live verification runbook
+
+Prereq: generate a migration for `LiteLLM_CasbinRule`, run `make format`,
+`make lint`, and `make test-unit tests/test_litellm/proxy/auth/v2/`. Start the
+proxy with `auth_version: v2`.
+
+1. master key works (bootstrap admin):
+   `curl -s localhost:4000/model/info -H "Authorization: Bearer $MASTER_KEY"` -> 200
+2. mint a virtual key for user `u_123` with no policy; `GET /model/info` with it
+   -> 403 (governed, no grant).
+3. grant `model_reader` read on models and assign to `u_123` (above); repeat
+   step 2 -> 200. `POST /model/new` with that key -> 403 (read-only).
+4. inference: a chat completion to a model in the key's `models` -> 200; to a
+   model outside it -> 403; a key with empty `models` -> any model 200.
+5. any ungoverned route still works and logs "not yet protected" in litellm.log.

litellm/proxy/auth/v2/enforcer.py

@@ -22,6 +22,7 @@ class CasbinEnforcer:
         policies: List[Rule],
         groupings: List[Rule],
         resource_groupings: Optional[List[Rule]] = None,
+        domain_groupings: Optional[List[Rule]] = None,
     ):
         self._enforcer = casbin.Enforcer(_MODEL_PATH)
         self._enforcer.enable_auto_save(False)
@@ -31,6 +32,8 @@ class CasbinEnforcer:
             self._enforcer.add_named_grouping_policy("g", *rule)
         for rule in resource_groupings or []:
             self._enforcer.add_named_grouping_policy("g2", *rule)
+        for rule in domain_groupings or []:
+            self._enforcer.add_named_grouping_policy("g3", *rule)
 
     def enforce(self, subject: str, domain: str, obj: str, action: str) -> bool:
         return self._enforcer.enforce(subject, domain, obj, action)

litellm/proxy/auth/v2/entry.py

@@ -58,11 +58,17 @@ async def user_api_key_auth_v2(
         request_data = await _read_request_body(request=request)
         principal = build_principal(identity)
 
-        policies, groupings, resource_groupings = await load_policy_snapshot(
-            prisma_client
-        )
+        (
+            policies,
+            groupings,
+            resource_groupings,
+            domain_groupings,
+        ) = await load_policy_snapshot(prisma_client)
         enforcer = CasbinEnforcer(
-            policies, groupings + principal.groupings, resource_groupings
+            policies,
+            groupings + principal.groupings,
+            resource_groupings,
+            domain_groupings,
         )
 
         try:

litellm/proxy/auth/v2/management_endpoints.py

@@ -29,6 +29,8 @@ class AssignmentRequest(BaseModel):
     subject_type: str
     subject_id: str
     role: str
+    # When set (e.g. "team:eng"), the role applies only within that domain.
+    domain: Optional[str] = None
 
 
 def rule_to_row_data(rule: List[str]) -> Dict[str, str]:
@@ -129,7 +131,9 @@ async def add_assignment(
 ):
     _require_admin(user_api_key_dict)
     try:
-        rule = make_assignment_rule(body.subject_type, body.subject_id, body.role)
+        rule = make_assignment_rule(
+            body.subject_type, body.subject_id, body.role, body.domain
+        )
     except PolicyValidationError as e:
         raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e))
     await _add_rule(rule)
@@ -143,7 +147,9 @@ async def remove_assignment(
 ):
     _require_admin(user_api_key_dict)
     try:
-        rule = make_assignment_rule(body.subject_type, body.subject_id, body.role)
+        rule = make_assignment_rule(
+            body.subject_type, body.subject_id, body.role, body.domain
+        )
     except PolicyValidationError as e:
         raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e))
     removed = await _remove_rule(rule)

litellm/proxy/auth/v2/model.conf

@@ -7,9 +7,10 @@ p = sub, dom, obj, act, eft
 [role_definition]
 g = _, _
 g2 = _, _
+g3 = _, _, _
 
 [policy_effect]
 e = some(where (p.eft == allow)) && !some(where (p.eft == deny))
 
 [matchers]
-m = g(r.sub, p.sub) && (p.dom == "*" || r.dom == p.dom) && (g2(r.obj, p.obj) || keyMatch(r.obj, p.obj)) && (p.act == "*" || r.act == p.act)
+m = (g(r.sub, p.sub) || g3(r.sub, p.sub, r.dom)) && (p.dom == "*" || r.dom == p.dom) && (g2(r.obj, p.obj) || keyMatch(r.obj, p.obj)) && (p.act == "*" || r.act == p.act)

litellm/proxy/auth/v2/policy_admin.py

@@ -42,8 +42,17 @@ def make_permission_rule(
     return ["p", _role_token(role), domain or "*", obj, action, effect]
 
 
-def make_assignment_rule(subject_type: str, subject_id: str, role: str) -> List[str]:
-    """Build a casbin ``g`` rule row binding a user/team subject to a role."""
+def make_assignment_rule(
+    subject_type: str,
+    subject_id: str,
+    role: str,
+    domain: Optional[str] = None,
+) -> List[str]:
+    """Build a casbin role-assignment rule binding a subject to a role.
+
+    A global assignment is a ``g`` row; a domain-scoped one (the role only
+    applies within ``domain``, e.g. ``team:eng``) is a ``g3`` row.
+    """
     if subject_type not in VALID_SUBJECT_TYPES:
         raise PolicyValidationError(
             f"subject_type must be one of {sorted(VALID_SUBJECT_TYPES)}, "
@@ -51,4 +60,9 @@ def make_assignment_rule(subject_type: str, subject_id: str, role: str) -> List[
         )
     if not subject_id or not subject_id.strip():
         raise PolicyValidationError("subject_id is required")
-    return ["g", f"{subject_type}:{subject_id}", _role_token(role)]
+
+    subject = f"{subject_type}:{subject_id}"
+    role_token = _role_token(role)
+    if domain and domain != "*":
+        return ["g3", subject, role_token, domain]
+    return ["g", subject, role_token]

litellm/proxy/auth/v2/policy_store.py

@@ -13,7 +13,13 @@ DEFAULT_POLICIES: List[List[str]] = [
 _CACHE_TTL_SECONDS = 5.0
 
 _cache: Optional[
-    Tuple[float, List[List[str]], List[List[str]], List[List[str]]]
+    Tuple[
+        float,
+        List[List[str]],
+        List[List[str]],
+        List[List[str]],
+        List[List[str]],
+    ]
 ] = None
 
 
@@ -31,10 +37,11 @@ def _row_values(row: Any) -> List[str]:
 
 def _split_rules(
     rows: List[Any],
-) -> Tuple[List[List[str]], List[List[str]], List[List[str]]]:
+) -> Tuple[List[List[str]], List[List[str]], List[List[str]], List[List[str]]]:
     policies: List[List[str]] = [list(p) for p in DEFAULT_POLICIES]
     groupings: List[List[str]] = []
     resource_groupings: List[List[str]] = []
+    domain_groupings: List[List[str]] = []
     for row in rows:
         ptype = getattr(row, "ptype", None)
         values = _row_values(row)
@@ -42,9 +49,11 @@ def _split_rules(
             policies.append(values)
         elif ptype == "g2":
             resource_groupings.append(values)
+        elif ptype == "g3":
+            domain_groupings.append(values)
         elif ptype is not None and ptype.startswith("g"):
             groupings.append(values)
-    return policies, groupings, resource_groupings
+    return policies, groupings, resource_groupings, domain_groupings
 
 
 def reset_cache() -> None:
@@ -54,9 +63,9 @@ def reset_cache() -> None:
 
 async def load_policy_snapshot(
     prisma_client: Any,
-) -> Tuple[List[List[str]], List[List[str]], List[List[str]]]:
-    """Load (policies, groupings, resource_groupings) from LiteLLM_CasbinRule,
-    with a short TTL cache.
+) -> Tuple[List[List[str]], List[List[str]], List[List[str]], List[List[str]]]:
+    """Load (policies, groupings, resource_groupings, domain_groupings) from
+    LiteLLM_CasbinRule, with a short TTL cache.
 
     Returns only the bootstrap defaults when no DB is connected, so the engine is
     always constructible.
@@ -64,12 +73,12 @@ async def load_policy_snapshot(
     global _cache
     now = time.monotonic()
     if _cache is not None and (now - _cache[0]) < _CACHE_TTL_SECONDS:
-        return _cache[1], _cache[2], _cache[3]
+        return _cache[1], _cache[2], _cache[3], _cache[4]
 
     rows: List[Any] = []
     if prisma_client is not None:
         rows = await prisma_client.db.litellm_casbinrule.find_many()
 
-    policies, groupings, resource_groupings = _split_rules(rows)
-    _cache = (now, policies, groupings, resource_groupings)
-    return policies, groupings, resource_groupings
+    policies, groupings, resource_groupings, domain_groupings = _split_rules(rows)
+    _cache = (now, policies, groupings, resource_groupings, domain_groupings)
+    return policies, groupings, resource_groupings, domain_groupings

tests/test_litellm/proxy/auth/v2/test_enforcer.py

@@ -81,3 +81,31 @@ def test_direct_object_matching_still_works_with_grouping_enabled():
     e = _enforcer([READER_POLICY], [["user:alice", "role:model_reader"]])
     assert e.enforce("user:alice", "*", "model:gpt4", "read") is True
     assert e.enforce("user:alice", "*", "model:gpt4", "write") is False
+
+
+def test_domain_scoped_role_applies_only_in_its_domain():
+    e = CasbinEnforcer(
+        policies=[["role:team_admin", "*", "team:*", "manage", "allow"]],
+        groupings=[],
+        resource_groupings=None,
+        domain_groupings=[["user:dana", "role:team_admin", "team:eng"]],
+    )
+    assert e.enforce("user:dana", "team:eng", "team:eng", "manage") is True
+    # Same user+role, different domain -> denied.
+    assert e.enforce("user:dana", "team:sales", "team:sales", "manage") is False
+
+
+def test_global_and_domain_roles_coexist():
+    e = CasbinEnforcer(
+        policies=[
+            ["role:reader", "*", "model:*", "read", "allow"],
+            ["role:team_admin", "*", "team:*", "manage", "allow"],
+        ],
+        groupings=[["user:gina", "role:reader"]],
+        domain_groupings=[["user:gina", "role:team_admin", "team:eng"]],
+    )
+    # Global reader role works anywhere.
+    assert e.enforce("user:gina", "team:sales", "model:x", "read") is True
+    # Domain-scoped admin role only in team:eng.
+    assert e.enforce("user:gina", "team:eng", "team:eng", "manage") is True
+    assert e.enforce("user:gina", "team:sales", "team:sales", "manage") is False

tests/test_litellm/proxy/auth/v2/test_policy_admin.py

@@ -58,6 +58,16 @@ def test_assignment_rule_for_user_and_team():
     assert make_assignment_rule("team", "eng", "role:x") == ["g", "team:eng", "role:x"]
 
 
+def test_global_assignment_when_domain_is_wildcard_or_absent():
+    assert make_assignment_rule("user", "u1", "admin", domain=None)[0] == "g"
+    assert make_assignment_rule("user", "u1", "admin", domain="*")[0] == "g"
+
+
+def test_domain_scoped_assignment_is_a_g3_rule():
+    rule = make_assignment_rule("user", "u1", "team_admin", domain="team:eng")
+    assert rule == ["g3", "user:u1", "role:team_admin", "team:eng"]
+
+
 def test_assignment_rejects_unknown_subject_type():
     with pytest.raises(PolicyValidationError):
         make_assignment_rule("org", "o1", "admin")

tests/test_litellm/proxy/auth/v2/test_policy_store.py

@@ -37,7 +37,7 @@ def _clear_cache():
 
 
 def test_bootstrap_admin_policy_always_present():
-    policies, _, _ = policy_store._split_rules([])
+    policies, _, _, _ = policy_store._split_rules([])
     assert ["role:proxy_admin", "*", "*", "*", "allow"] in policies
     assert DEFAULT_POLICIES[0] == ["role:proxy_admin", "*", "*", "*", "allow"]
 
@@ -47,17 +47,22 @@ def test_rows_split_by_ptype():
         _Row("p", "role:model_reader", "*", "model:*", "read", "allow"),
         _Row("g", "user:alice", "role:model_reader"),
         _Row("g2", "model:gpt-4o", "group:prod"),
+        _Row("g3", "user:bob", "role:team_admin", "team:eng"),
     ]
-    policies, groupings, resource_groupings = policy_store._split_rules(rows)
+    policies, groupings, resource_groupings, domain_groupings = (
+        policy_store._split_rules(rows)
+    )
     assert ["role:model_reader", "*", "model:*", "read", "allow"] in policies
     assert ["user:alice", "role:model_reader"] in groupings
-    # g2 must NOT leak into the g groupings.
+    # Each grouping kind lands in its own bucket; none leak into g.
     assert ["model:gpt-4o", "group:prod"] in resource_groupings
+    assert ["user:bob", "role:team_admin", "team:eng"] in domain_groupings
     assert ["model:gpt-4o", "group:prod"] not in groupings
+    assert ["user:bob", "role:team_admin", "team:eng"] not in groupings
 
 
 def test_empty_trailing_columns_are_trimmed():
-    policies, _, _ = policy_store._split_rules(
+    policies, _, _, _ = policy_store._split_rules(
         [_Row("p", "role:x", "*", "model:*", "read", "allow")]
     )
     assert policies[-1] == ["role:x", "*", "model:*", "read", "allow"]
@@ -65,12 +70,13 @@ def test_empty_trailing_columns_are_trimmed():
 
 @pytest.mark.asyncio
 async def test_load_snapshot_without_db_returns_only_bootstrap():
-    policies, groupings, resource_groupings = await load_policy_snapshot(
-        prisma_client=None
+    policies, groupings, resource_groupings, domain_groupings = (
+        await load_policy_snapshot(prisma_client=None)
     )
     assert policies == [list(p) for p in DEFAULT_POLICIES]
     assert groupings == []
     assert resource_groupings == []
+    assert domain_groupings == []
 
 
 @pytest.mark.asyncio
@@ -80,11 +86,13 @@ async def test_load_snapshot_reads_db_rows():
             _Row("p", "role:model_reader", "*", "model:*", "read", "allow"),
             _Row("g", "user:alice", "role:model_reader"),
             _Row("g2", "model:gpt-4o", "group:prod"),
+            _Row("g3", "user:bob", "role:team_admin", "team:eng"),
         ]
     )
-    policies, groupings, resource_groupings = await load_policy_snapshot(
-        prisma_client=prisma
+    policies, groupings, resource_groupings, domain_groupings = (
+        await load_policy_snapshot(prisma_client=prisma)
     )
     assert ["role:model_reader", "*", "model:*", "read", "allow"] in policies
     assert ["user:alice", "role:model_reader"] in groupings
     assert ["model:gpt-4o", "group:prod"] in resource_groupings
+    assert ["user:bob", "role:team_admin", "team:eng"] in domain_groupings