* fix(bedrock): respect s3_region_name for batch file uploads (#23569)
* fix(bedrock): respect s3_region_name for batch file uploads (GovCloud fix)
* fix: s3_region_name always wins over aws_region_name for S3 signing (Greptile feedback)
* fix: _filter_headers_for_aws_signature - Bedrock KB (#23571)
* fix: _filter_headers_for_aws_signature
* fix: filter None header values in all post-signing re-merge paths
Addresses Greptile feedback: None-valued headers were being filtered
during SigV4 signing but re-merged back into the final headers dict
afterward, which would cause downstream HTTP client failures.
Made-with: Cursor
* feat(router): tag_regex routing — route by User-Agent regex without per-developer tag config (#23594)
* feat(router): add tag_regex support for header-based routing
Adds a new `tag_regex` field to litellm_params that lets operators route
requests based on regex patterns matched against request headers — primarily
User-Agent — without requiring per-developer tag configuration.
Use case: route all Claude Code traffic (User-Agent: claude-code/x.y.z) to
a dedicated deployment by setting:
tag_regex:
- "^User-Agent: claude-code\\/"
in the deployment's litellm_params. Works alongside existing `tags` routing;
exact tag match takes precedence over regex match. Unmatched requests fall
through to deployments tagged `default`.
The matched deployment, pattern, and user_agent are recorded in
`metadata["tag_routing"]` so they flow through to SpendLogs automatically.
* fix(tag_regex): address backwards-compat, metadata overwrite, and warning noise
Three issues from code review:
1. Backwards-compat: `has_tag_filter` was widened to activate on any non-empty
User-Agent, which would raise ValueError for existing deployments using plain
tags without a `default` fallback. Fix: only activate header-based regex
filtering when at least one candidate deployment has `tag_regex` configured.
2. Metadata overwrite: `metadata["tag_routing"]` was overwritten for every
matching deployment in the loop, leaving inaccurate provenance when multiple
deployments match. Fix: write only for the first match.
3. Warning noise: an invalid regex pattern logged one warning per header string
rather than once per pattern. Fix: compile first (catching re.error once),
then iterate over header strings.
Also adds two new tests covering these cases, and adds docs page for
tag_regex routing with a Claude Code walk-through.
* refactor(tag_regex): remove unnecessary _healthy_list copy
* docs: merge tag_regex section into tag_routing.md, remove standalone page
- Add ## Regex-based tag routing (tag_regex) section to existing
tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
routing fires under strict mode (regex always uses OR semantics)
* fix(tag_regex): address greptile review - security docs, strict-mode enforcement, validation order
- Strengthen security note in tag_routing.md: explicitly state User-Agent
is client-supplied and can be set to any value; frame tag_regex as a
traffic classification hint, not an access-control mechanism
- Move tag_regex startup validation before _add_deployment() so an invalid
pattern never leaves partial router state
- Enforce match_any=False strict-tag policy: when a deployment has both
tags and tag_regex and the strict tag check fails, skip the regex fallback
rather than silently bypassing the operator's intent
- Extract per-deployment match logic into _match_deployment() helper to
keep get_deployments_for_tag() readable
- Add two new tests: strict-mode blocks regex fallback, regex-only
deployment still matches under match_any=False
* fix(ci): apply Black formatting to 14 files and stabilize flaky caplog tests
- Run Black formatter on 14 files that were failing the lint check
- Replace caplog-based assertions in TestAliasConflicts with
unittest.mock.patch on verbose_logger.warning for xdist compatibility
- The caplog fixture can produce empty text in pytest-xdist workers
in certain CI environments, causing flaky test failures
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: complexity_router fails on list-format message content (OpenAI multi-part messages)
When a client sends messages with list-format content
(e.g. [{"type": "text", "text": "..."}] as used by the OpenAI JS SDK
and other clients), the complexity_router's async_pre_routing_hook
skipped those messages because it only handled str content. This caused
user_message to be None, the hook returned None, and the router fell
through to selecting the complexity_router deployment itself
(model="auto_router/complexity_router") which litellm cannot dispatch,
resulting in LiteLLMUnknownProvider.
Fixes:
- Extract text from list-format content parts (type=text) before
classifying
- Return default_model instead of None when no user message can be
extracted, preventing the crash fallthrough
- Loosen PreRoutingHookResponse.messages type from Dict[str, str] to
Dict[str, Any] to accommodate list-format content values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update messages type annotation in async_pre_routing_hook to Dict[str, Any]
Consistent with PreRoutingHookResponse.messages type change and the
list-format content support added in the previous commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: normalize None content to empty string in complexity_router message parsing
msg.get("content", "") returns None when the key exists with value None
(e.g. assistant messages with tool calls). Use `or ""` to normalize
None to an empty string explicitly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: strip whitespace from joined list content parts in complexity_router
Prevents leading/trailing spaces when some content parts have empty
text values (e.g. " ".join(["", "hello"]) → " hello").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: avoid NoneType AttributeError when extracting tags
I've been running into this error:
```
21:47:08 - LiteLLM:ERROR: litellm_logging.py:2396 - LiteLLM.LoggingError: [Non-Blocking] Exception occurred while success logging Traceback (most recent call last):
File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/litellm_logging.py", line 2312, in async_success_handler
await callback.async_log_success_event(
...<6 lines>...
)
File "/usr/lib/python3.13/site-packages/litellm/router_strategy/budget_limiter.py", line 396, in async_log_success_event
request_tags = _get_tags_from_request_kwargs(kwargs)
File "/usr/lib/python3.13/site-packages/litellm/router_strategy/tag_based_routing.py", line 144, in _get_tags_from_request_kwargs
return _metadata.get("tags", [])
^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
```
This makes the function more resilient without resorting to try catch.
* add tests
Signed-off-by: Max Falk <gmdfalk@gmail.com>
---------
Signed-off-by: Max Falk <gmdfalk@gmail.com>
* refactor: comment out circuit breaker
causes incorrect rate limiting in high traffic
* fix(base_routing_strategy.py): don't reset value if redis val is lower than current in-memory value
Fixes issue where redis might be trailing in-memory value
* fix(parallel_request_limiter_v2.py): if in-memory higher than redis, don't reset value; add previous slot keys to redis increment to correctly 'get' them
* fix(parallel_request_limiter_v3.py): v3 implementation of parallel request limiter
does not use background redis syncing - increments redis in call
simplify rate limiting logic, to improve accuracy
* fix: fix ruff errors
* fix(parallel_request_limiter_v3.py): don't decrement limit on post call success - causes double decrements
* fix(parallel_request_limiter_v3.py): working accurate multi-instance logic
ensured just 100 requests allowed on 100 users, 10 ramp up, 100 rpm limit key, 2 instances
* fix(parallel_request_limiter_v3.py): working accurate rate limiting with time window resets
allows rate limiting to work across multiple windows
* test: add unit tests for v3 rate limiter
* fix(parallel_request_limiter_v3.py): return window value into in-memory cache
allows in-memory cache checks to be used correctly
* refactor(parallel_request_limiter_v3.py): refactor rate limiting to work for multiple window/counter key pairs
enables using for user/team/model rate limiting
* feat(parallel_request_limiter_v3.py): working rate limiting, across key/user/team/end-user
* fix(parallel_request_limiter_v3.py): add model specific rate limiting
* fix(parallel_request_limiter_v3.py): ignore if no rate limits set
skip unecessary rate limit checks - if no limits set
* fix(parallel_request_limiter_v3.py): initial commit bringing token rate limits back
* fix(parallel_request_limiter_v3.py): increment by value in list + update assertions to handle tokens + max parallel requests
* test(parallel_request_limiter_v3.py): more testing
* fix(parallel_request_limiter.py): working in-memory cache limiter
* fix(redis_cache.py): ignore linting error - use safe hasattr
* fix(parallel_request_limiter_v3.py): fix linting error
* refactor: remove redundant parallel_Request_limiter_v2.py
old / inaccurate implementation
* test: update tests
* style: cleanup
* test: update test
* docs(config_settings.md): document new env var
* test(test_base_routing_strategy.py): update test
* fix(base_routing_strategy.py): compress increments to redis - reduces write ops
* fix(base_routing_strategy.py): make get and reset in memory keys atomic
* fix(base_routing_strategy.py): don't reset keys - causes discrepency on subsequent requests to instance
* fix(parallel_request_limiter.py): retrieve values of previous slots from cache
more accurate rate limiting with sliding window
* fix: fix test
* fix: fix linting error
* feat(parallel_request_limiter_v2.py): add sliding window logic
allows rate limiting to work across minutes
* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error
* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis
* fix(base_routing_strategy.py): catch the delta change during redis sync
ensures values are kept in sync
* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic
* fix: fix linting error
* test: update testing
* test: update tests
* test: skip on rate limit or internal server errors
* test: use pytest fixture instead
* test: bump mistral model
