scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-08 22:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78

feat(router): tag_regex routing — route by User-Agent regex without per-developer tag config (#23594)

Browse source 
* feat(router): add tag_regex support for header-based routing

Adds a new `tag_regex` field to litellm_params that lets operators route
requests based on regex patterns matched against request headers — primarily
User-Agent — without requiring per-developer tag configuration.

Use case: route all Claude Code traffic (User-Agent: claude-code/x.y.z) to
a dedicated deployment by setting:

  tag_regex:
    - "^User-Agent: claude-code\\/"

in the deployment's litellm_params. Works alongside existing `tags` routing;
exact tag match takes precedence over regex match. Unmatched requests fall
through to deployments tagged `default`.

The matched deployment, pattern, and user_agent are recorded in
`metadata["tag_routing"]` so they flow through to SpendLogs automatically.

* fix(tag_regex): address backwards-compat, metadata overwrite, and warning noise

Three issues from code review:

1. Backwards-compat: `has_tag_filter` was widened to activate on any non-empty
   User-Agent, which would raise ValueError for existing deployments using plain
   tags without a `default` fallback. Fix: only activate header-based regex
   filtering when at least one candidate deployment has `tag_regex` configured.

2. Metadata overwrite: `metadata["tag_routing"]` was overwritten for every
   matching deployment in the loop, leaving inaccurate provenance when multiple
   deployments match. Fix: write only for the first match.

3. Warning noise: an invalid regex pattern logged one warning per header string
   rather than once per pattern. Fix: compile first (catching re.error once),
   then iterate over header strings.

Also adds two new tests covering these cases, and adds docs page for
tag_regex routing with a Claude Code walk-through.

* refactor(tag_regex): remove unnecessary _healthy_list copy

* docs: merge tag_regex section into tag_routing.md, remove standalone page

- Add ## Regex-based tag routing (tag_regex) section to existing
  tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
  page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
  routing fires under strict mode (regex always uses OR semantics)

* fix(tag_regex): address greptile review - security docs, strict-mode enforcement, validation order

- Strengthen security note in tag_routing.md: explicitly state User-Agent
  is client-supplied and can be set to any value; frame tag_regex as a
  traffic classification hint, not an access-control mechanism
- Move tag_regex startup validation before _add_deployment() so an invalid
  pattern never leaves partial router state
- Enforce match_any=False strict-tag policy: when a deployment has both
  tags and tag_regex and the strict tag check fails, skip the regex fallback
  rather than silently bypassing the operator's intent
- Extract per-deployment match logic into _match_deployment() helper to
  keep get_deployments_for_tag() readable
- Add two new tests: strict-mode blocks regex fallback, regex-only
  deployment still matches under match_any=False
This commit is contained in:
Ishaan Jaff 2026-03-13 21:13:27 -07:00 • committed by GitHub
parent 64f9e9df9a
commit bd17d123c2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 627 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

100
docs/my-website/docs/proxy/tag_routing.md
View file

@ -209,6 +209,106 @@ Expect to see the following response header when this works
x-litellm-model-id: default-model
```
## Regex-based tag routing (`tag_regex`)
Use `tag_regex` to route requests based on regex patterns matched against request headers, without requiring clients to pass a tag explicitly. This is useful when clients already send a recognisable header, such as `User-Agent`.
**Use case: route all Claude Code traffic to dedicated AWS accounts**
Claude Code always sends `User-Agent: claude-code/<version>`. With `tag_regex` you can route that traffic to a dedicated deployment automatically — no per-developer configuration needed.
### 1. Config
```yaml
model_list:
# Claude Code traffic → dedicated deployment, matched by User-Agent
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_region_name: us-east-1
aws_role_name: arn:aws:iam::111122223333:role/LiteLLMClaudeCode
tag_regex:
- "^User-Agent: claude-code\\/" # matches claude-code/1.x, 2.x, etc.
model_info:
id: claude-code-deployment
# All other traffic falls back to the default deployment
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_region_name: us-east-1
aws_role_name: arn:aws:iam::444455556666:role/LiteLLMDefault
tags:
- default
model_info:
id: regular-deployment
router_settings:
enable_tag_filtering: true
tag_filtering_match_any: true
general_settings:
master_key: sk-1234
```
### 2. Verify routing
Claude Code sets `User-Agent: claude-code/<version>` automatically — no client config needed:
```shell
# Claude Code request (User-Agent set automatically by Claude Code)
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-1234" \
-H "User-Agent: claude-code/1.2.3" \
-d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hi"}]}'
# → x-litellm-model-id: claude-code-deployment
# Any other client (no matching User-Agent) → default deployment
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-1234" \
-d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hi"}]}'
# → x-litellm-model-id: regular-deployment
```
### How matching works
| Priority | Condition | Result |
|----------|-----------|--------|
| 1 | Request has `tags` AND deployment has `tags` | Exact tag match (respects `match_any` setting) |
| 2 | Deployment has `tag_regex` AND request has a `User-Agent` | Regex match (always OR logic — any pattern match suffices) |
| 3 | Deployment has `tags: [default]` | Default fallback |
| 4 | No default set | All healthy deployments returned |
`tag_regex` always uses OR semantics — `tag_filtering_match_any=False` applies only to exact tag matching, not to regex patterns.
### Observability
When a regex matches, `tag_routing` is written into request metadata and flows to SpendLogs:
```json
{
"tag_routing": {
"matched_via": "tag_regex",
"matched_value": "^User-Agent: claude-code\\/",
"user_agent": "claude-code/1.2.3",
"request_tags": []
}
}
```
### Security note
:::caution
**`User-Agent` is a client-supplied header and can be set to any value.** Any API consumer can send `User-Agent: claude-code/1.0` regardless of whether they are actually using Claude Code.
Do not rely on `tag_regex` routing to enforce access controls or spend limits — use [team/key-based routing](./users) for that. `tag_regex` is a **traffic classification hint** (useful for billing visibility, capacity planning, and routing convenience), not a security boundary.
:::
---
## ✨ Team based tag routing (Enterprise)
LiteLLM Proxy supports team-based tag routing, allowing you to associate specific tags with teams and route requests accordingly. Example **Team A can access gpt-4 deployment A, Team B can access gpt-4 deployment B** (LLM Access Control For Teams)

17
litellm/router.py
View file

@ -14,6 +14,7 @@ import hashlib
import inspect
import json
import logging
import re
import threading
import time
import traceback
@ -1551,7 +1552,9 @@ class Router:
# Drain any fire-and-forget tasks (e.g. alerting hooks)
# scheduled via asyncio.create_task during acompletion.
pending = asyncio.all_tasks()
pending.discard(asyncio.current_task())
current = asyncio.current_task()
if current is not None:
pending.discard(current)
if pending:
await asyncio.gather(*pending, return_exceptions=True)
@ -6542,6 +6545,18 @@ class Router:
)
return None
# Validate tag_regex patterns BEFORE adding the deployment so we never
# have partially-initialised router state if a pattern is invalid.
_tag_regex = deployment.litellm_params.get("tag_regex") or []
for pattern in _tag_regex:
try:
re.compile(pattern)
except re.error as exc:
raise ValueError(
f"Invalid regex in tag_regex for model '{deployment.model_name}': "
f"{pattern!r} — {exc}"
) from exc
deployment = self._add_deployment(deployment=deployment)
model = deployment.to_json(exclude_none=True)

148
litellm/router_strategy/tag_based_routing.py
View file

@ -6,6 +6,7 @@ Use this to route requests between Teams
- If no default_deployments are set, return all deployments
"""
import re
from typing import TYPE_CHECKING, Any, Dict, List, Literal, Optional, Union
from litellm._logging import verbose_logger
@ -19,6 +20,31 @@ else:
LitellmRouter = Any
def _is_valid_deployment_tag_regex(
tag_regexes: List[str],
header_strings: List[str],
) -> Optional[str]:
"""
Test compiled regex patterns against "Header-Name: value" strings.
Returns the first matching pattern string, or None if nothing matches.
Compiles each pattern once (re's LRU cache) and logs invalid patterns once
per pattern, not once per header string.
"""
for pattern in tag_regexes:
try:
compiled = re.compile(pattern)
except re.error:
verbose_logger.warning(
"tag_regex: invalid pattern %r — skipping", pattern
)
continue
for header_str in header_strings:
if compiled.search(header_str):
return pattern
return None
def is_valid_deployment_tag(
deployment_tags: List[str], request_tags: List[str], match_any: bool = True
) -> bool:
@ -47,6 +73,54 @@ def is_valid_deployment_tag(
return False
def _match_deployment(
deployment: Any,
request_tags: Optional[List[str]],
header_strings: List[str],
match_any: bool,
) -> Optional[Dict[str, str]]:
"""
Determine whether *deployment* matches the current request.
Returns {"matched_via": ..., "matched_value": ...} if the deployment
should be included, or None if it should be excluded.
Priority:
1. Exact tag match (respects match_any semantics).
2. Regex match — skipped when match_any=False and the tag check already
ran and failed, so the regex cannot override strict-tag policy.
"""
litellm_params = deployment.get("litellm_params", {})
deployment_tags: Optional[List[str]] = litellm_params.get("tags")
deployment_tag_regex: Optional[List[str]] = litellm_params.get("tag_regex")
# 1. Exact tag match (existing behaviour).
if deployment_tags and request_tags:
if is_valid_deployment_tag(deployment_tags, request_tags, match_any):
matched_value = next(
(t for t in deployment_tags if t in set(request_tags)),
deployment_tags[0],
)
return {"matched_via": "tags", "matched_value": matched_value}
# 2. Regex match against request headers.
# When match_any=False and the deployment has both plain tags and tag_regex,
# the strict tag check has already failed (step 1 returned None). Allow
# the regex to fire only when the deployment has NO plain tags, so we never
# use regex as a backdoor around the operator's strict-tag policy.
strict_tag_check_failed = (
not match_any
and bool(deployment_tags)
and bool(request_tags)
)
if deployment_tag_regex and header_strings and not strict_tag_check_failed:
regex_match = _is_valid_deployment_tag_regex(deployment_tag_regex, header_strings)
if regex_match is not None:
return {"matched_via": "tag_regex", "matched_value": regex_match}
return None
async def get_deployments_for_tag(
llm_router_instance: LitellmRouter,
model: str, # used to raise the correct error
@ -83,30 +157,66 @@ async def get_deployments_for_tag(
request_tags = metadata.get("tags")
match_any = llm_router_instance.tag_filtering_match_any
new_healthy_deployments = []
default_deployments = []
if request_tags:
verbose_logger.debug(
"get_deployments_for_tag routing: router_keys: %s", request_tags
)
# example this can be router_keys=["free", "custom"]
for deployment in healthy_deployments:
deployment_litellm_params = deployment.get("litellm_params")
deployment_tags = deployment_litellm_params.get("tags")
# Build header strings for regex matching from what the proxy already stores.
# Currently we match against User-Agent; format matches "^User-Agent: claude-code/..."
user_agent = metadata.get("user_agent", "")
header_strings: List[str] = (
[f"User-Agent: {user_agent}"] if user_agent else []
)
verbose_logger.debug(
"deployment: %s, deployment_router_keys: %s",
deployment,
deployment_tags,
new_healthy_deployments: List[Any] = []
default_deployments: List[Any] = []
# Only activate header-based regex filtering when at least one deployment in
# the candidate set has tag_regex configured. This preserves existing
# behaviour for operators who use plain tags: a request that carries a
# User-Agent (all proxy requests do) but targets deployments with no
# tag_regex will continue to use the original tag-only code path.
has_regex_deployments = any(
d.get("litellm_params", {}).get("tag_regex")
for d in healthy_deployments
)
has_tag_filter = bool(request_tags) or (
bool(header_strings) and has_regex_deployments
)
if has_tag_filter:
verbose_logger.debug(
"get_deployments_for_tag routing: request_tags=%s user_agent=%s",
request_tags,
user_agent,
)
for deployment in healthy_deployments:
deployment_tags = deployment.get("litellm_params", {}).get("tags")
match_result = _match_deployment(
deployment=deployment,
request_tags=request_tags,
header_strings=header_strings,
match_any=match_any,
)
if deployment_tags is None:
continue
if is_valid_deployment_tag(deployment_tags, request_tags, match_any):
if match_result is not None:
verbose_logger.debug(
"tag routing match: deployment=%s matched_via=%s matched_value=%s",
deployment.get("model_name"),
match_result["matched_via"],
match_result["matched_value"],
)
# Record provenance in metadata so it flows to SpendLogs.
# Written only for the first match — load balancer selects one
# deployment from new_healthy_deployments, so overwriting on
# subsequent matches would produce misleading observability data.
if "tag_routing" not in metadata:
metadata["tag_routing"] = {
"matched_deployment": deployment.get("model_name"),
"matched_via": match_result["matched_via"],
"matched_value": match_result["matched_value"],
"request_tags": request_tags or [],
"user_agent": user_agent,
}
new_healthy_deployments.append(deployment)
if "default" in deployment_tags:
if deployment_tags and "default" in deployment_tags:
default_deployments.append(deployment)
if len(new_healthy_deployments) == 0 and len(default_deployments) == 0:

7
litellm/types/router.py
View file

@ -198,6 +198,11 @@ class GenericLiteLLMParams(CredentialLiteLLMParams, CustomPricingLiteLLMParams):
model_info: Optional[Dict] = None
mock_response: Optional[Union[str, ModelResponse, Exception, Any]] = None
# tag-based routing
tags: Optional[List[str]] = None
# regex patterns matched against request headers for tag routing
tag_regex: Optional[List[str]] = None
# auto-router params
auto_router_config_path: Optional[str] = None
auto_router_config: Optional[str] = None
@ -334,6 +339,8 @@ class LiteLLMParamsTypedDict(TypedDict, total=False):
# routing params
# use this for tag-based routing
tags: Optional[List[str]]
# regex patterns matched against request headers (e.g. "^User-Agent:\\s*claude-code\\/")
tag_regex: Optional[List[str]]
# deployment budgets
max_budget: Optional[float]

375
tests/test_litellm/router_strategy/test_router_tag_regex_routing.py Normal file
View file

@ -0,0 +1,375 @@
"""
Unit tests for tag_regex routing.
Tests _is_valid_deployment_tag_regex() and get_deployments_for_tag() with tag_regex
patterns, verifying that regex-based header matching works correctly alongside
existing tag-based routing.
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.abspath("../.."))
from unittest.mock import MagicMock
from litellm.router_strategy import tag_based_routing
from litellm.router_strategy.tag_based_routing import get_deployments_for_tag
_is_valid_deployment_tag_regex = tag_based_routing._is_valid_deployment_tag_regex
# ---------------------------------------------------------------------------
# _is_valid_deployment_tag_regex unit tests
# ---------------------------------------------------------------------------
def test_regex_matches_claude_code_user_agent():
"""^User-Agent: claude-code/ matches a claude-code UA string."""
result = _is_valid_deployment_tag_regex(
tag_regexes=[r"^User-Agent: claude-code\/"],
header_strings=["User-Agent: claude-code/1.2.3"],
)
assert result == r"^User-Agent: claude-code\/"
def test_regex_no_match_for_other_ua():
"""Pattern does not match a non-claude-code User-Agent."""
result = _is_valid_deployment_tag_regex(
tag_regexes=[r"^User-Agent: claude-code\/"],
header_strings=["User-Agent: Mozilla/5.0 (browser)"],
)
assert result is None
def test_regex_returns_first_matching_pattern():
"""When multiple patterns are provided, returns the first match."""
result = _is_valid_deployment_tag_regex(
tag_regexes=[r"^User-Agent: cursor\/", r"^User-Agent: claude-code\/"],
header_strings=["User-Agent: claude-code/2.0.0"],
)
assert result == r"^User-Agent: claude-code\/"
def test_regex_empty_inputs_return_none():
"""Empty lists return None without errors."""
assert _is_valid_deployment_tag_regex([], ["User-Agent: claude-code/1.0"]) is None
assert _is_valid_deployment_tag_regex([r"^User-Agent: claude-code\/"], []) is None
def test_invalid_regex_skipped_does_not_raise():
"""An invalid regex pattern is skipped (warning logged) — no exception raised."""
result = _is_valid_deployment_tag_regex(
tag_regexes=["[invalid(regex"],
header_strings=["User-Agent: claude-code/1.0"],
)
assert result is None
def test_regex_matches_version_range():
"""Semver-aware pattern matches multiple versions."""
pattern = r"^User-Agent: claude-code\/\d"
for ua in ["claude-code/1.0", "claude-code/2.0.0-beta.1", "claude-code/99.0"]:
result = _is_valid_deployment_tag_regex(
tag_regexes=[pattern],
header_strings=[f"User-Agent: {ua}"],
)
assert result == pattern, f"Expected match for UA: {ua}"
# ---------------------------------------------------------------------------
# get_deployments_for_tag integration tests
# ---------------------------------------------------------------------------
CLAUDE_CODE_DEPLOYMENT = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/claude-code-deployment",
"api_key": "fake",
"mock_response": "cc",
"tag_regex": [r"^User-Agent: claude-code\/"],
},
"model_info": {"id": "claude-code-deployment"},
}
REGULAR_DEPLOYMENT = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/regular-deployment",
"api_key": "fake",
"mock_response": "regular",
"tags": ["default"],
},
"model_info": {"id": "regular-deployment"},
}
ALL_DEPLOYMENTS = [CLAUDE_CODE_DEPLOYMENT, REGULAR_DEPLOYMENT]
def _make_router_mock(enable_tag_filtering=True, match_any=True):
mock = MagicMock()
mock.enable_tag_filtering = enable_tag_filtering
mock.tag_filtering_match_any = match_any
return mock
@pytest.mark.asyncio
async def test_claude_code_ua_routes_to_cc_deployment():
"""claude-code/x.y.z UA → claude-code-deployment via tag_regex."""
router = _make_router_mock()
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=ALL_DEPLOYMENTS,
request_kwargs={"metadata": {"user_agent": "claude-code/1.2.3"}},
)
assert len(result) == 1
assert result[0]["model_info"]["id"] == "claude-code-deployment"
@pytest.mark.asyncio
async def test_regular_ua_routes_to_default_deployment():
"""Mozilla UA → regular-deployment via default tag fallback."""
router = _make_router_mock()
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=ALL_DEPLOYMENTS,
request_kwargs={"metadata": {"user_agent": "Mozilla/5.0 (browser)"}},
)
assert len(result) == 1
assert result[0]["model_info"]["id"] == "regular-deployment"
@pytest.mark.asyncio
async def test_no_ua_routes_to_default_deployment():
"""No User-Agent → default deployment."""
router = _make_router_mock()
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=ALL_DEPLOYMENTS,
request_kwargs={"metadata": {}},
)
assert len(result) == 1
assert result[0]["model_info"]["id"] == "regular-deployment"
@pytest.mark.asyncio
async def test_tag_routing_metadata_written_for_regex_match():
"""tag_routing metadata block is populated when regex matches."""
router = _make_router_mock()
metadata: dict = {"user_agent": "claude-code/2.0.0-beta.1"}
await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=ALL_DEPLOYMENTS,
request_kwargs={"metadata": metadata},
)
assert "tag_routing" in metadata
tr = metadata["tag_routing"]
assert tr["matched_via"] == "tag_regex"
assert tr["matched_value"] == r"^User-Agent: claude-code\/"
assert tr["user_agent"] == "claude-code/2.0.0-beta.1"
@pytest.mark.asyncio
async def test_tag_filtering_disabled_returns_all_deployments():
"""When enable_tag_filtering is False, all deployments returned regardless of UA."""
router = _make_router_mock(enable_tag_filtering=False)
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=ALL_DEPLOYMENTS,
request_kwargs={"metadata": {"user_agent": "claude-code/1.0"}},
)
assert result == ALL_DEPLOYMENTS
@pytest.mark.asyncio
async def test_explicit_tag_match_takes_precedence_over_regex():
"""A deployment with both tags and tag_regex: exact tag match fires first."""
deployment_with_both = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/both-deployment",
"api_key": "fake",
"tags": ["premium"],
"tag_regex": [r"^User-Agent: claude-code\/"],
},
"model_info": {"id": "both-deployment"},
}
router = _make_router_mock()
metadata: dict = {
"tags": ["premium"],
"user_agent": "claude-code/1.0",
}
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=[deployment_with_both],
request_kwargs={"metadata": metadata},
)
assert len(result) == 1
tr = metadata.get("tag_routing", {})
assert tr.get("matched_via") == "tags"
@pytest.mark.asyncio
async def test_user_agent_present_no_tag_regex_deployments_does_not_raise():
"""
Backwards-compat: a request that carries a User-Agent but targets plain-tag
deployments (no tag_regex) must NOT raise ValueError — it should fall
through to the default/all-deployments path just like before.
"""
plain_tag_only_deployments = [
{
"model_name": "gpt-4",
"litellm_params": {
"model": "openai/premium-deployment",
"api_key": "fake",
"tags": ["premium"],
},
"model_info": {"id": "premium-deployment"},
},
{
"model_name": "gpt-4",
"litellm_params": {
"model": "openai/free-deployment",
"api_key": "fake",
"tags": ["free"],
},
"model_info": {"id": "free-deployment"},
},
]
router = _make_router_mock()
# The request has a User-Agent (as all proxy requests do) but NO tags and
# neither deployment has tag_regex — must not raise, must return all.
result = await get_deployments_for_tag(
llm_router_instance=router,
model="gpt-4",
healthy_deployments=plain_tag_only_deployments,
request_kwargs={"metadata": {"user_agent": "Mozilla/5.0 (any-client)"}},
)
# Falls through to "return healthy_deployments" path unchanged
assert result == plain_tag_only_deployments
@pytest.mark.asyncio
async def test_tag_routing_metadata_not_overwritten_for_multiple_matches():
"""
When multiple deployments match, tag_routing records only the first match
so the provenance reflects what the load balancer likely selected.
"""
deployment_a = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/cc-deployment-a",
"api_key": "fake",
"tag_regex": [r"^User-Agent: claude-code\/"],
},
"model_info": {"id": "cc-deployment-a"},
}
deployment_b = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/cc-deployment-b",
"api_key": "fake",
"tag_regex": [r"^User-Agent: claude-code\/"],
},
"model_info": {"id": "cc-deployment-b"},
}
router = _make_router_mock()
metadata: dict = {"user_agent": "claude-code/1.0"}
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=[deployment_a, deployment_b],
request_kwargs={"metadata": metadata},
)
assert len(result) == 2
# tag_routing recorded once and reflects the first match
tr = metadata.get("tag_routing", {})
assert tr.get("matched_deployment") == "claude-sonnet"
assert tr.get("matched_via") == "tag_regex"
@pytest.mark.asyncio
async def test_match_any_false_strict_tag_check_blocks_regex_fallback():
"""
When match_any=False and a deployment has both tags and tag_regex:
if the strict tag check fails (request has a tag NOT present on the
deployment, so req_set is NOT a subset of dep_set), the regex fallback
must NOT fire — that would violate the operator's strict-filtering intent.
Semantics of match_any=False: req_set.issubset(dep_set), i.e. every
request tag must appear on the deployment. A request with tags ["vip"]
against a deployment with tags ["premium"] fails because "vip" ∉ dep_set.
"""
deployment_strict = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/strict-deployment",
"api_key": "fake",
"tags": ["premium"],
"tag_regex": [r"^User-Agent: claude-code\/"],
},
"model_info": {"id": "strict-deployment"},
}
default_deployment = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/default-deployment",
"api_key": "fake",
"tags": ["default"],
},
"model_info": {"id": "default-deployment"},
}
# match_any=False: req_set must be a subset of dep_set.
# Request has "vip" which is NOT in ["premium"], so tag check fails.
# Even though UA matches tag_regex, the deployment must NOT be selected.
router = _make_router_mock(enable_tag_filtering=True, match_any=False)
metadata: dict = {
"tags": ["vip"], # "vip" not in deployment tags → strict check fails
"user_agent": "claude-code/1.0",
}
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=[deployment_strict, default_deployment],
request_kwargs={"metadata": metadata},
)
ids = [d["model_info"]["id"] for d in result]
assert "strict-deployment" not in ids, (
"strict-deployment should not be selected: strict tag check failed "
"and regex must not override the strict policy"
)
@pytest.mark.asyncio
async def test_match_any_false_regex_only_deployment_still_matches():
"""
When match_any=False and a deployment has ONLY tag_regex (no plain tags),
there is no strict tag policy to violate, so the regex check must still fire.
"""
regex_only_deployment = {
"model_name": "claude-sonnet",
"litellm_params": {
"model": "openai/regex-only-deployment",
"api_key": "fake",
"tag_regex": [r"^User-Agent: claude-code\/"],
# no "tags" key at all
},
"model_info": {"id": "regex-only-deployment"},
}
router = _make_router_mock(enable_tag_filtering=True, match_any=False)
result = await get_deployments_for_tag(
llm_router_instance=router,
model="claude-sonnet",
healthy_deployments=[regex_only_deployment],
request_kwargs={"metadata": {"user_agent": "claude-code/1.0"}},
)
assert len(result) == 1
assert result[0]["model_info"]["id"] == "regex-only-deployment"