Commit graph

6 commits

Author SHA1 Message Date
Mateo Wang
dfac770e2b
feat(proxy): map baggage values to budget tags (#47358)
* feat(proxy): map baggage values to budget tags

* fix(proxy): preserve mapped tags in the parsed request cache

* fix(proxy): preserve mapper accounting and tag policy

* fix(proxy): preserve baggage tags across native transports

* fix(proxy): keep mapper rejection and passthrough logging safe

* fix(proxy): scope baggage tag mappings to billed inference and keep them out of tag routing

Mapped tags now apply only to billed LLM API routes, skipping token counting,
model listing, MCP and A2A. They classify spend and budgets but never select
deployments unless a key, team or client set the same tag. WebSocket inference
returns 403 only while a required rule exists. A tag created with /tag/new resets
its spend counter so its budget counts from creation. The General settings editor
keeps a new mapping unsaved until every required field is filled in.

* fix(proxy): keep client-tag rejection identical with baggage, map A2A calls, skip request previews

* fix(proxy): keep mapped tags out of requester metadata, answer 503 on tag lookup errors, map custom pass-through routes, check the key before the WebSocket 403

* fix(proxy): keep caller-sent tags routable, keep mapped tags out of policy and guardrail matching, honor allow_requests_on_db_unavailable, reject control characters in baggage values

* test(proxy): cover baggage tag mappings end to end

Adds integration cells for request_tag_mappings across chat, messages,
responses, A2A and custom pass-through spend, required and budgeted
rules, the WebSocket refusal, unbilled routes, reject_clientside_metadata_tags,
tag routing and policy isolation, a dropped tag read with and without
allow_requests_on_db_unavailable, and the /tag/new spend counter reset

* fix(proxy): keep base tag merges for requests that map nothing, accept JSON-string metadata tags, count litellm_metadata caller tags

Requests with no mapped baggage now reuse the cached body exactly as before, so header and key tag merges still normalize metadata for routing. The client body copy is made only when a tag maps, and the base merges are mirrored onto it. A request that maps nothing no longer gets a 400 for non-object metadata. caller_tags now counts tags sent in both metadata and litellm_metadata. Optional mappings no longer reject JSON-string metadata tags under reject_clientside_metadata_tags.

* fix(router): keep baggage-mapped tags from selecting tag-scoped pre-routing strategies

Auto, complexity, adaptive and quality router selection, the consumed-tags stamp and compression policy lookup now read request tags without baggage-mapped tags, unless the caller sent the tag or the key, team or project carries it, matching deployment tag filtering.

* feat(proxy): parse baggage headers into request tag mappings

* fix(proxy): match plus-sign baggage keys and keep deployment tags in guardrail modes

* fix(proxy): keep baggage-mapped tags behind key, team and deployment tags and serve non-object or invalid DB mapping settings like unset

Mapped tags are added after the key, team, deployment and caller tags, so a tag-mode guardrail picks the same mode with or without baggage. A non-object metadata field maps like empty metadata, and an invalid request_tag_mappings value loaded from the DB is ignored with an ERROR log instead of failing every request

* fix(router): merge deployment tags in linear time and keep malformed client tags as sent under mapped baggage

Merge serving deployment tags into the request tags in one pass, so a large client tag list no longer blocks the event loop. Leave a non-list metadata.tags untouched when baggage maps a tag, so the request gets the same answer as without baggage. Add the C17 integration cell for the team tag's guardrail mode under a mapped deployment tag.

* test(spend): cover string metadata.tags under mapped baggage

* fix(proxy): check mapped tag budgets when metadata.tags is a non-list or holds non-strings

Mapped baggage tags are applied again after the header and key tag merges, so
a header or key tag that replaces a string metadata.tags no longer hides the
mapped tag from the tag budget check. An empty or false metadata.tags is read
like a missing one, and a tags list with non-string items is kept as sent with
the mapped tags added after it, instead of failing auth with a 401
2026-10-20 09:09:42 +00:00
devin-ai-integration[bot]
c3a23fe499
refactor: expose core private helpers under public names (#44871)
* refactor: expose core private symbols with compatibility aliases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* types: narrow core migration diagnostics

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve runtime behavior in core symbol migration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve core private usage migration behavior

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve private value rebinding compatibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: restore optional imports and cover public helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: exempt router property from call coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: align recursive detector ignore names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 00:05:27 +00:00
Harshit28j
f18f4e3bbd feat: allow multiple calls from tags 2026-03-07 11:24:18 +05:30
Harshit28j
d661419109 fix: support list of modes in Mode.default for tag-based guardrails 2026-03-04 01:50:28 +05:30
Krish Dholakia
1543891763
Responses API - support tags in metadata
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors

* fix(vector_store_pre_call_hook.py): Fix https://github.com/BerriAI/litellm/issues/15724

* fix(responses/main.py): have 'tags' work across metadata + litellm_metadata

* fix: add unit testing
2025-10-25 12:07:54 -07:00
Krish Dholakia
569ddde9fe
[Enterprise] Support tag based mode for guardrails (#12508)
* feat(custom_guardrail.py): allow admin to run guardrails on specific modes based on request tags

Enables pre-call checks for openwebui but logging for claude code guardrails

* refactor(custom_guardrail.py): move tag based mode to enterprise

* test(test_custom_guardrail.py): update test

* feat(quick_start.md): add doc on tag-based guardrail mode

* docs(quick_start.md): cleanup doc

* docs: cleanup

* docs(openweb_ui.md): add tutorial to doc

* fix(custom_guardrail.py): ensure user is premium user

* test: update test
2025-07-10 22:27:30 -07:00