scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88

docs: merge tag_regex section into tag_routing.md, remove standalone page

Browse source 
- Add ## Regex-based tag routing (tag_regex) section to existing
  tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
  page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
  routing fires under strict mode (regex always uses OR semantics)
This commit is contained in:
Ishaan Jaffer 2026-03-13 20:13:32 -07:00
parent 259240fb46
commit 1b36694749
4 changed files with 112 additions and 189 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

188
docs/my-website/docs/proxy/tag_regex_routing.md
View file

@ -1,188 +0,0 @@
# Tag Regex Routing
Route requests to specific deployments based on regex patterns matched against request headers — without requiring per-user tag configuration.
## Overview
With standard [tag-based routing](tag_routing), each request must carry a matching tag (e.g. `tags: ["vibe-coding"]`). This works well when you control the client, but becomes impractical at scale.
**Tag regex routing** lets you match on headers the client already sends automatically — like `User-Agent` — so requests are routed correctly with zero client-side configuration.
### Use case: route all Claude Code traffic to dedicated AWS accounts
> "We need to route Claude Code traffic to a dedicated set of AWS accounts. We want to roll out the Claude Code → LiteLLM integration to 5,000 employees — it's not practical to ask every developer to configure a tag. Claude Code always sends a `User-Agent` header that starts with `claude-code/`, so we'd like LiteLLM to use that automatically."
This is exactly what `tag_regex` is for.
---
## Quick Start
### 1. Configure `tag_regex` on the target deployment
Add a `tag_regex` list to `litellm_params`. Each entry is a regex pattern matched against `"Header-Name: value"` strings built from the request metadata.
```yaml
model_list:
# Claude Code traffic → dedicated Bedrock account, matched by User-Agent
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_region_name: us-east-1
aws_role_name: arn:aws:iam::111122223333:role/LiteLLMRole
tag_regex:
- "^User-Agent: claude-code\\/" # matches claude-code/1.x, claude-code/2.x, …
model_info:
id: claude-code-deployment
# All other traffic → standard deployment (catch-all default)
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_region_name: us-east-1
aws_role_name: arn:aws:iam::444455556666:role/LiteLLMRole
tags:
- default
model_info:
id: regular-deployment
router_settings:
enable_tag_filtering: true
tag_filtering_match_any: true
general_settings:
master_key: sk-1234
```
### 2. Start the proxy
```shell
litellm --config config.yaml
```
### 3. Send a request from Claude Code
Claude Code automatically sets `User-Agent: claude-code/<version>`. No extra configuration needed on the client side.
```shell
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-1234" \
-H "User-Agent: claude-code/1.2.3" \
-d '{
"model": "claude-sonnet",
"messages": [{"role": "user", "content": "hello"}]
}'
```
Check the response header to confirm routing:
```
x-litellm-model-id: claude-code-deployment
```
### 4. Send a request from any other client
No `User-Agent: claude-code/` header → falls through to the `default` deployment.
```shell
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-1234" \
-d '{
"model": "claude-sonnet",
"messages": [{"role": "user", "content": "hello"}]
}'
```
```
x-litellm-model-id: regular-deployment
```
---
## How it works
When `enable_tag_filtering: true` is set and a deployment has `tag_regex` configured, LiteLLM builds a `"Header-Name: value"` string from the request's `User-Agent` and tests each regex pattern against it using `re.search`.
**Matching priority (in order):**
1. **Exact tag match** — if the request includes `tags: ["vibe-coding"]` and a deployment has `tags: ["vibe-coding"]`, that match fires first.
2. **Regex match** — if no exact tag match, `tag_regex` patterns are tested against request headers.
3. **Default fallback** — if nothing matches, deployments tagged `default` are used.
4. **All deployments** — if no `default` tag exists, all healthy deployments are returned (existing behaviour unchanged).
**Backwards compatibility:** deployments that use only plain `tags` (no `tag_regex`) are unaffected, even when requests carry a `User-Agent` header.
---
## Combining `tags` and `tag_regex`
You can mix both on the same deployment — a request matches if either the tag or the regex matches:
```yaml
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_role_name: arn:aws:iam::111122223333:role/LiteLLMRole
tags:
- vibe-coding # explicit tag still works for teams that set it
tag_regex:
- "^User-Agent: claude-code\\/" # automatic match for everyone else
```
---
## Observability: `tag_routing` in SpendLogs
When a regex matches, LiteLLM writes a `tag_routing` block into the request metadata. This flows automatically into SpendLogs so you can see how each request was routed:
```json
{
"tag_routing": {
"matched_deployment": "claude-sonnet",
"matched_via": "tag_regex",
"matched_value": "^User-Agent: claude-code\\/",
"user_agent": "claude-code/1.2.3",
"request_tags": []
}
}
```
| Field | Description |
|-------|-------------|
| `matched_via` | `"tag_regex"` or `"tags"` |
| `matched_value` | The specific pattern or tag that matched |
| `user_agent` | The `User-Agent` value from the request |
| `request_tags` | Explicit tags on the request (if any) |
---
## Reference
### `tag_regex` field
| | |
|-|-|
| **Location** | `litellm_params` in `config.yaml` |
| **Type** | `list[str]` |
| **Matching** | `re.search(pattern, "User-Agent: <value>")` |
| **Error handling** | Invalid regex patterns are skipped with a warning at startup and at match time |
### Supported header sources
Currently `User-Agent` is the only header source. The pattern format is `"Header-Name: value"`, so for a request with `User-Agent: claude-code/1.2.3` the string tested is `"User-Agent: claude-code/1.2.3"`.
### Pattern tips
| Goal | Pattern |
|------|---------|
| Match any Claude Code version | `^User-Agent: claude-code\/` |
| Match specific major version | `^User-Agent: claude-code\/1\.` |
| Match any semver | `^User-Agent: claude-code\/\d+\.\d+` |
---
## Related
- [Tag Based Routing](tag_routing) — explicit per-request tags
- [Team Based Routing](team_based_routing) — route by team membership
- [Request Tags](request_tags) — how tags flow through requests

99
docs/my-website/docs/proxy/tag_routing.md
View file

@ -209,6 +209,105 @@ Expect to see the following response header when this works
x-litellm-model-id: default-model
```
## Regex-based tag routing (`tag_regex`)
Use `tag_regex` to route requests based on regex patterns matched against request headers, without requiring clients to pass a tag explicitly. This is useful when clients already send a recognisable header, such as `User-Agent`.
**Use case: route all Claude Code traffic to dedicated AWS accounts**
Claude Code always sends `User-Agent: claude-code/<version>`. With `tag_regex` you can route that traffic to a dedicated deployment automatically — no per-developer configuration needed.
### 1. Config
```yaml
model_list:
# Claude Code traffic → dedicated deployment, matched by User-Agent
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_region_name: us-east-1
aws_role_name: arn:aws:iam::111122223333:role/LiteLLMClaudeCode
tag_regex:
- "^User-Agent: claude-code\\/" # matches claude-code/1.x, 2.x, etc.
model_info:
id: claude-code-deployment
# All other traffic falls back to the default deployment
- model_name: claude-sonnet
litellm_params:
model: bedrock/converse/anthropic-claude-sonnet-4-6
aws_region_name: us-east-1
aws_role_name: arn:aws:iam::444455556666:role/LiteLLMDefault
tags:
- default
model_info:
id: regular-deployment
router_settings:
enable_tag_filtering: true
tag_filtering_match_any: true
general_settings:
master_key: sk-1234
```
### 2. Verify routing
Claude Code sets `User-Agent: claude-code/<version>` automatically — no client config needed:
```shell
# Claude Code request (User-Agent set automatically by Claude Code)
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-1234" \
-H "User-Agent: claude-code/1.2.3" \
-d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hi"}]}'
# → x-litellm-model-id: claude-code-deployment
# Any other client (no matching User-Agent) → default deployment
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-1234" \
-d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "hi"}]}'
# → x-litellm-model-id: regular-deployment
```
### How matching works
| Priority | Condition | Result |
|----------|-----------|--------|
| 1 | Request has `tags` AND deployment has `tags` | Exact tag match (respects `match_any` setting) |
| 2 | Deployment has `tag_regex` AND request has a `User-Agent` | Regex match (always OR logic — any pattern match suffices) |
| 3 | Deployment has `tags: [default]` | Default fallback |
| 4 | No default set | All healthy deployments returned |
`tag_regex` always uses OR semantics — `tag_filtering_match_any=False` applies only to exact tag matching, not to regex patterns.
### Observability
When a regex matches, `tag_routing` is written into request metadata and flows to SpendLogs:
```json
{
"tag_routing": {
"matched_via": "tag_regex",
"matched_value": "^User-Agent: claude-code\\/",
"user_agent": "claude-code/1.2.3",
"request_tags": []
}
}
```
### Security note
:::caution
`User-Agent` is set by the client and **can be spoofed**. `tag_regex` is designed for **routing convenience** — directing traffic from a known tool to the right backend — not for security isolation or access control.
If you need to restrict which users or teams can reach a deployment, use [API key / team scoping](./users) rather than (or in addition to) regex routing.
:::
---
## ✨ Team based tag routing (Enterprise)
LiteLLM Proxy supports team-based tag routing, allowing you to associate specific tags with teams and route requests accordingly. Example **Team A can access gpt-4 deployment A, Team B can access gpt-4 deployment B** (LLM Access Control For Teams)

1
docs/my-website/sidebars.js
View file

@ -1018,7 +1018,6 @@ const sidebars = {
"proxy/reliability",
"proxy/fallback_management",
"proxy/tag_routing",
"proxy/tag_regex_routing",
"proxy/timeout",
"wildcard_routing"
],

13
litellm/router_strategy/tag_based_routing.py
View file

@ -162,11 +162,24 @@ async def get_deployments_for_tag(
)
# 2. Regex match against request headers (new)
# NOTE: tag_regex always uses OR semantics (any pattern match suffices).
# match_any=False applies only to exact tag matching above; it has no
# "all patterns must match" equivalent for regex and is intentionally
# ignored here. Operators who need strict isolation should enforce
# that via API key / team scoping rather than routing patterns alone,
# since User-Agent is fully client-controlled and can be spoofed.
if matched_via is None and deployment_tag_regex and header_strings:
regex_match = _is_valid_deployment_tag_regex(
deployment_tag_regex, header_strings
)
if regex_match is not None:
if not match_any:
verbose_logger.debug(
"tag_regex match fired on deployment=%s while "
"tag_filtering_match_any=False; regex routing always "
"uses OR semantics — match_any is ignored for tag_regex",
deployment.get("model_name"),
)
matched_via = "tag_regex"
matched_value = regex_match