The Rust messages bridge logs a parsed Anthropic response without an
httpx_response, so the fallback transform dropped the request speed and
billed fast-mode calls at the standard rate. Thread optional_params
speed into transform_parsed_response and add a regression test for the
parsed-response branch.
A stream can terminate with a response.failed event that still reports
consumed tokens; those were rebuilt as None and logged at zero spend.
Parse response.failed alongside completed and incomplete, matching the
buffered path, which prices any terminal response that reports usage.
The Anthropic batch create never persisted the creating key's hashed token or its
request tags on the managed object, so when CheckBatchCost billed the batch hours
later there was nothing to attribute it to. Key spend, key budgets and tag spend
never moved for batch usage.
Persist both from the create, the way the Vertex passthrough already does, and
register the batch only from the collection route. An id-scoped route cannot
rebuild the unified object id, because it embeds the model and the model comes
from the create's request body, so it could only claim a row it did not create or
fail the model_object_id unique constraint.
The shared metadata helpers, the route predicate and the registration-result
logging now live in batch_attribution instead of being copied per provider. The
Anthropic write previously logged success unconditionally, before the
fire-and-forget task had run.
Resolves LIT-5288
LiteLLMCompletionStreamingIterator overrides __init__ without calling
super().__init__(), so completed_response was only set once the stream
reached RESPONSE_COMPLETED. On a mid-stream provider error the router's
_extract_partial_responses_usage read source_iterator.completed_response
during fallback recovery and raised AttributeError, masking the real
provider error (e.g. Anthropic 529) and bypassing configured retries and
fallbacks. Initialize the attribute to None so recovery degrades to no
partial usage instead of crashing.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Streams that terminate with response.incomplete (e.g. max_output_tokens
reached) carry real usage in the terminal event but were rebuilt as None
and logged at zero spend, letting callers bypass budget enforcement.
Parse response.incomplete alongside response.completed when
reconstructing the streamed response.
Addresses review feedback: the ResponseCompletedEvent SSE parsing now lives
in OpenAIResponsesAPIConfig next to the other Responses stream event handling,
and the proxy logging handler calls it. Adds coverage for streams that end
without a response.completed event.
AWS no longer serves `anthropic.claude-3-sonnet-20240229-v1:0`. The streaming
path returns a plain 404, "Model with the provided id
anthropic.claude-3-sonnet-20240229-v1:0 is not found", and the non-streaming
path answers 500 for the same reason. Our own cost map has carried a
2026-07-30 deprecation date for it since #36538
That accounts for 20 failures across local_testing_part1, local_testing_part2
and llm_translation_testing. litellm maps both statuses correctly, so the
tests are what went stale, not the client
Replacement is `us.anthropic.claude-sonnet-4-5-20250929-v1:0`: a like-for-like
Sonnet, and the newest Bedrock Sonnet this repo exercises against the real API
in tests/e2e. Newer ids exist in the cost map, but nothing in the repo calls
them live, so picking one would be an unverified guess about model access on
the CI account
Scope is limited to the tests that actually issue a request. The occurrences
that assert on the model string itself, or that feed mocked transformations,
keep the old id so their assertions stay meaningful
* feat(router): make routing groups callable as virtual models and list them in /v1/models
* fix(router): traffic-scoped cooldown exemption, live model_names on delete, group-info cache invalidation
* fix(router): share one recognized-model predicate across proxy gates, resolve aliases in group cooldown, read metadata via the dual-bucket owner
* fix(router): close the gate and cache families for callable groups, strip member access_groups from group rows, prove cooldown wiring end to end
* refactor(router): cache materialized group rows under the model-group cache owner and drop the redundant wiring test
* fix(router): warn-and-shadow on group name collisions, name-level test coverage for group helpers, faithful router doubles in a2a and cursor tests
* test(router): pin group cooldown metadata across the retry path
Gemini image usage carries prompt_tokens and friends as extra fields on
ResponseAPIUsage, which collided with the bridge's explicit kwargs and
raised TypeError. Exclude keys the bridge already sets explicitly.
`supports_native_structured_output` was set only on the bare `deepseek.v3.2`
and `zai.glm-5` entries, so the cross-region inference profiles and the
region-pinned ids resolved to None. The flag gates the native
`outputConfig.textFormat` branch in BedrockConverseConfig, so callers
addressing the same model as `us.deepseek.v3.2` or
`bedrock/us-west-2/deepseek.v3.2` silently fell back to synthetic tool
injection. `us.` is the form Bedrock steers callers toward, so the most
common way to reach these models was the one missing the capability.
Adds the flag to the 12 affected ids and keeps the packaged backup in sync.
test_get_model_info_bedrock_models already caught the region-pinned ids, but
it filters on `litellm_provider == "bedrock"` and the cross-region profiles
carry `bedrock_converse`, so reverting just `us.deepseek.v3.2` and
`eu.deepseek.v3.2` left it green. The new parity test covers the prefixed
profiles and fails on exactly that mutation.
* test(e2e): cover google-native generateContent framing and prometheus queue time
Adds live coverage for three shipped regressions that had none, all reached
through surfaces a customer drives from Google SDKs and operator dashboards.
The managed google-native route (`/v1beta/models/{model}:generateContent`) had
no harness support at all, so EndpointsClient gains generate_content and
stream_generate_content plus the request body models, and a new suite asserts
the two contracts that broke there: the response carries
x-litellm-response-cost so SDK traffic reconciles against spend (LIT-4076), and
the stream relays single-prefixed SSE frames with no OpenAI [DONE] terminator.
A doubled `data:` prefix, a leaked bytes literal, or the [DONE] sentinel each
fail the stream test; [DONE] absence is only asserted once real content has
arrived, because a first-chunk upstream error legitimately falls back to the
OpenAI error shape and does emit it.
The prometheus test pins litellm_request_queue_time_seconds to an actual
observation on our own key's series rather than to the family merely existing,
which is the distinction the original regression turned on: the histogram stayed
registered while nothing was ever written to it (LIT-2034).
Each assertion was mutation-checked against the live proxy; inverting the
[DONE] expectation, the cost-header expectation, or the metric name fails the
corresponding test.
* refactor(e2e): simplify google native coverage
* test(e2e): add failing reproducers for two open gateway bugs
Both tests assert the behavior a customer expects and both are red today. They
are reproducers, not regressions: the product is wrong, not the tests.
Native passthrough returns almost none of the operational headers the managed
route does. A /gemini/ generateContent call comes back with three x-litellm-*
headers and no x-ratelimit-* at all, against sixteen and four on
/v1beta/models/{m}:generateContent for the same prompt, and critically it omits
x-litellm-response-cost. Customers front provider-native traffic through this
route and read those headers to reconcile spend and pace themselves, so native
traffic is currently invisible to the tooling that covers every other route.
/budget/update rejects any model_max_budget with a 500. The reported symptom was
model ids containing dots, and that reproduces (prisma raises "Unexpected
`-5.2[FloatValue]` Expected `:`" because the key is interpolated into a GraphQL
query unquoted, so glm-5.2 lexes as an identifier followed by a float), but the
plain name gpt4o fails too, on a separate "model_max_budget should be of any of
the following types: Json" type mismatch at budget_management_endpoints.py:173.
Omitting the field returns 200. The test drives both names so the failure says
whether per-model budgets are broken outright or only for punctuated ids; today
it stops on the plain name, which is the wider bug.
* test(e2e): add reproducer for unenforced end-user per-model rate limits
model_max_budget accepts an rpm_limit alongside the spend cap, and /budget/new
stores it: the create response echoes {"gemini-2.5-flash": {"rpm_limit": 1,
"max_budget": 100.0, "budget_duration": "1d"}}. Attach that budget to an end
user, drive three calls as that user, and all three return 200. The limit is
accepted, persisted, and then ignored.
The same shape already works when the budget hangs off a key, which is what
makes this quietly dangerous: the API gives every indication the cap is in
force. A customer using it to hold one end user to a slow rate on a shared key
gets no throttling at all.
Harness additions this needs: ModelBudgetEntry carries the rpm_limit/tpm_limit
the route already accepts, BudgetNewBody and create_budget carry
model_max_budget, and create_customer can attach an existing budget_id rather
than only an inline max_budget.
Red today, for the reason in the assertion message.
* test(e2e): tighten model_max_budget reproducers and drop in-loop closure
Trim the reproducer docstrings to the contract they assert, keeping the
failure messages that document each red-by-design bug. Replace the nested
per-model closure in the /budget/update test with a module-level predicate
and a per-model helper so nothing closes over a loop variable, and fix the
import order the merge left unsorted.
* test(e2e): skip the three reproducers while their gateway bugs stay open
The passthrough header contract, /budget/update model_max_budget, and
end-user per-model rpm enforcement reproducers all still fail against
staging by design. Skip each with the product gap named so the combined
suite can gate merges on green while the collector keeps reporting the
cells as uncovered.
* test(e2e): validate model budget response contracts
* refactor(e2e): unify model budget schema
* refactor(e2e): reuse shared model budget type
* test(e2e): cover vendor strategy gaps for chat contract, image edits, auth, team activity
Resolves the first slice of LIT-4778 (vendor API testing strategy): image edits happy path, chat multi-turn + validation + sanitization, LLM-route auth header matrix, and /team/daily/activity structure
* test(e2e): expand vendor API strategy coverage across endpoints
Adds validation cases on existing endpoint suites, plus vector stores, search,
bedrock native, realtime HTTP secrets/calls, responses retrieve, files/batches
contract, and chat stream SSE. Registers coverage cells for LIT-4778
* test(e2e): finish vendor strategy open items
Audio transcription negatives, vector-store file attach/poll/search,
OpenAI moderation category matrix across chat/messages/responses, and
smoke model matrix for chat (LIT-4778)
* test(e2e): harden vendor strategy suite against live env edges
Fix stream [DONE] tracking, XSS no-crash contract, realtime model routing,
vector store list/search models, responses validation, and provider-denied
Bedrock paths so the suite is stable against a live proxy
* test(e2e): rename suites, drop vendor_contract, fix greptile gaps
Move shared status helpers into e2e_http, rename chat auth headers and
chat security suites, remove vendor_contract and dev_config files_settings,
and tighten transcription validation plus vector-store search assertions
* test(e2e): route bedrock stream disconnects through e2e_http
Catch mid-stream RequestException in the shared harness so bedrock native
tests do not import requests directly
The otel trace tests asserted that a streamed call produces exactly one gen-AI
span. The proxy opens one gen-AI span per upstream attempt, so a call the
router retried carries an error span for every failed attempt beside the one
that answered, and the assertion fails on a request that succeeded.
Select the served attempt instead: drop spans whose otel.status_code is ERROR,
require exactly one survivor, and run the TTFT and streaming-flag assertions
against it. That keeps what these assertions exist for, a split trace or a
stream logged as two served spans, while tolerating a retry.
Only the failed attempt lacks TTFT, so the old code also had a second failure
mode: when the first span happened to be the error one, the test reported the
attribute as missing rather than as belonging to a different attempt.
test_span_selection.py covers the selection itself against Jaeger-shaped
payloads and carries no e2e marker, since reproducing a first-attempt failure
live is not something a test can arrange.
Drop the transform overrides that swapped response.usage to the chat
shape, which broke the /v1/responses client contract. Provider extras
like server_side_tool_usage_details already survive validation via
ResponseAPIUsage extra fields, so the shared usage bridge now carries
them onto the bridged chat Usage generically. The web_search_call
output gate also reads dict output items, since items that fail SDK
validation stay plain dicts, and the chat path gains billing tests.
* fix(deps): ship boto3 with the base SDK so bedrock works out of the box
* keep boto3 listed in the proxy extra as well
* scrub ambient AWS env vars in the base SDK bedrock smoke check
* fix(alerting): dedupe scheduled Slack spend reports across pods
Every pod ran its own weekly/monthly spend report jobs, prometheus
fallback stats cron, and daily report loop, so deployments with
multiple replicas or uvicorn workers received one copy per pod.
Gate each scheduled send behind the shared PodLockManager redis lock.
The lock is never released: its TTL (the full reporting window for the
weekly interval job, whose per-pod anchors drift by boot time and
jitter) doubles as a sent-this-window marker. acquire_lock returning
None (no redis wired) proceeds, preserving single-pod behavior.
Also generalize the pod lock could-not-acquire log line, which claimed
to be about spend tracking for every consumer.
Fixes#14809
* fix(alerting): harden spend report locks after adversarial review
Weekly lock TTL gets an hour haircut: with ttl equal to the interval,
the winner re-fires just before its own key expires, reacquires without
a TTL refresh, and the key then lapses in time for a trailing pod to
re-send. Job/lock ids move to litellm/constants.py per convention, and
spend_report_frequency now rejects non-positive day counts, which
previously coerced to an every-second schedule and would now compute a
negative lock TTL that silently never sends.
Adds the missing test coverage the review flagged: startup_event's
pod_lock_manager wiring (identity-asserted), the prometheus closure's
positive path, and the ungated immediate prometheus send pinned to
exactly one await.
* test(alerting): consolidate spend_report_frequency validator coverage
Drops a duplicate non-positive-days test and parametrizes the survivor
over the suffix half of the validator too
* fix(alerting): route the startup prometheus fallback send through the pod lock
Greptile caught that the boot-time send still ran once per pod when
PROMETHEUS_URL is set, the same duplication class this PR removes
* fix(alerting): make report lock acquisition non-reentrant
Greptile caught that a pod booting within an hour of the fallback stats
cron sent twice: the startup send takes the lock, then the cron fire
hits acquire_lock's reacquire branch, which returns True for the
holder. Window-marker gates now pass allow_reentrant=False so a live
lock blocks everyone including its holder; leader-election consumers
keep the reentrant default
* test(proxy): give spec'd ProxyLogging mocks a db_spend_update_writer
_initialize_slack_alerting_jobs now reads it for the pod lock manager,
and spec=ProxyLogging blocks instance-only attributes
255d65192e added useCan to UsageTab, whose useIsOrgAdmin leg calls
useOrganizations (react-query), so every UsageTab test died with 'No
QueryClient set'. Stub the org-admin leg; role gating still flows through
the real hasCapability with the varied userRole.
* feat(proxy): per-key prompt caching auto-injection via enable_prompt_caching
Adds a key-level enable_prompt_caching toggle that auto-injects Anthropic
cache_control breakpoints on requests made with that key, without requiring
the gateway-wide enable_anthropic_prompt_caching flag. The flag lives in key
metadata, is stamped onto the request root by add_key_level_controls, rides
kwargs into both the /chat/completions seeding path and the native
/v1/messages path, and reuses every existing gate (anthropic/bedrock only,
supports_prompt_caching, client markers win). Client-supplied body values are
stripped as an untrusted root control field. Includes the Admin UI switch on
key create and key edit plus a read-only settings row, and dedupes the key
edit view's drifted initial-values objects.
* fix(proxy): drop section comment and suppress LIT011 on key-level prompt caching stamp
* feat(router): add required-AND (&) tag prefix and allow_fail_open flag
Tag routing supported inclusion-OR and independent "!" negation, but had no way
to express a hard "must match all of these" constraint per request, and no way
for a model group to opt into degrading gracefully instead of raising when a
constraint eliminates every deployment.
Adds a "&tag" prefix for required-AND inclusion, composing with existing plain
(OR) and "!" (negate) tags: negation still applies first, then required tags
narrow the survivors, then plain tags apply today's OR/AND preference logic
unchanged. Adds model_info.allow_fail_open (default false) so a chain can opt
into falling back to the default-tagged pool instead of raising
no_deployments_with_tag_routing when "!" or "&" empties the candidate set;
existing chains without the flag keep today's fail-closed behavior exactly.
* fix(router): gate mixed negation on allow_fail_open and stop diluting required-only requests
Two gaps in the initial required-AND/allow_fail_open change: a "!" exclusion
combined with a plain positive tag that emptied the candidate set raised
unconditionally, bypassing allow_fail_open entirely, since the fail-open check
only looked at required-AND exhaustion. And a request using only "&" tags
could get narrowed down to just the deployment matching an incidental
tag_regex/User-Agent preference, silently dropping other deployments that
satisfied the required tags but had no tag_regex at all.
Fixes both: the fail-open check now fires whenever either "!" or "&" leaves
the candidate set empty, not just "&". And regex/header preference no longer
counts as a positive filter when a required-AND ask is present, so a
required-only request returns every deployment satisfying the required tags
regardless of regex/header matching.
Also regenerates ui/litellm-dashboard/src/lib/http/schema.d.ts for the new
model_info.allow_fail_open field, and removes source comments explaining
the router logic per repository convention.
* fix(router): let allow_fail_open cover a non-empty !/& survivor set that fails the plain-tag preference
The unconditional raise inside the has_positive_filter loop was the one
remaining path a chain could hit despite setting allow_fail_open: when "!"
or "&" leaves a non-empty candidate set but none of the survivors match the
request's plain preference tag or carry "default", the request still raised
instead of degrading. Routes that raise through the same allow_fail_open
check used everywhere else, so it now falls back to the default-tagged pool
for opted-in chains and keeps raising unconditionally for everyone else.
This also let the now-redundant pre-loop empty-candidates shortcut be
removed, since the loop reaches the same outcome on its own.
* fix(router): deny allow_fail_open when an unrecognized required tag is masking a satisfiable answer
A caller could add a single "&" tag no deployment in the group has ever
carried to force an empty required-AND set on demand. On a chain with
allow_fail_open, that emptied set fell back to the default-tagged pool
unconditionally, discarding every other constraint merged into the same
request, including ones inherited from key/team policy, even when the rest
of those constraints were still individually satisfiable.
Before falling back, drop any required tag not carried by any deployment in
the group and recompute: if a specific, non-empty answer exists using only
the recognized tags, the unrecognized tag was the actual cause of the
exhaustion, and fail-open must not paper over it. If every required tag is
already recognized, or none are, there's nothing hidden behind an invented
tag, and fail-open proceeds exactly as before; this keeps a single opted-in
deployment's legitimate catch-all behavior working when a caller's tag
simply doesn't exist anywhere in that group.
Ratchets ANN401 and LIT001 budgets down to reflect fixes already earned in
this branch.
* test(router): cover required-AND, allow_fail_open, and unknown-tag denial across fallback chains and model groups
Extends coverage beyond single-hop scenarios: & exhausting a primary group
falls through to a fallback group exactly like ! already does; !, &, and
allow_fail_open composed together across three chained model groups each
raise or fall back independently per-hop; and the unknown-tag denial from
the previous commit is evaluated fresh per hop rather than leaking state
across groups in a fallback chain.
* feat(router): add model_info.enable_tag_filtering per-model-group override
enable_tag_filtering was router-wide only: an operator turning it on for one
model group that needs tag-driven routing exposed every other model group on
the same proxy to the same tag evaluation, even ones that never use tags.
Adds model_info.enable_tag_filtering, checked against any deployment sharing
a model_name, so a chain can flip the router-wide default in either
direction for itself alone: opt a specific group into filtering while the
rest of the proxy stays off, or opt a group out (e.g. an incident-response
catch-all) while the rest of the proxy enforces it.
Precedence, low to high: router-wide default, then the chain override if
set, then the existing request-level escalation (from key/team settings),
which still only ever turns filtering on, never off, over whatever the
router and chain already decided.
Also regenerates ui/litellm-dashboard/src/lib/http/schema.d.ts for the new
field.
* fix(router): gate plain-tag exhaustion on allow_fail_open when the tag is known to the group
A model group where every deployment is tagged "default" (a legitimate
cross-cutting safety-net pattern) never has an empty default_deployments
list, so the existing exhaustion check (len(new)==0 and len(default)==0)
never fired for a plain positive tag that matched nothing among the
currently healthy candidates. The request silently fell through to whatever
"default"-tagged deployment happened to survive, even when allow_fail_open
was never set and the caller's intent (e.g. quality:high) was never honored.
Adds a check for whether the requested tag is part of the group's real
vocabulary at all: if some deployment configured under this model_name
(regardless of current health) genuinely carries the tag, and nothing
healthy currently matches it, the request now raises by default or falls
back per allow_fail_open, through the same _resolve_or_fail_open gate every
other exhaustion path already uses. A tag that's foreign to the group
entirely (e.g. one meant for an unrelated mechanism sharing the same
request-tags list) keeps falling back to the default pool unconditionally,
unchanged, since there's nothing this group's own routing intent could be
violating.
* fix(router): preserve inherited tag constraints when allow_fail_open discards a caller-caused exhaustion
Adds metadata.caller_tags in litellm_pre_call_utils.py, populated only from
what the request itself supplied (header, body tags, body metadata.tags),
never from key/team metadata merged into the same metadata.tags list.
get_deployments_for_tag now uses it to compute a trusted-only pool before
falling open: a required/excluded tag attributable to the caller can be
discarded on fail-open, one inherited from key/team policy cannot. If the
trusted-only pool is itself empty, allow_fail_open raises instead of
silently routing around an unsatisfiable inherited constraint. When
caller_tags carries no information at all (direct SDK Router usage,
bypassing the proxy layer), behavior is unchanged: unconditional fall-open
to the default pool, exactly as before this fix.
* feat(router): add opt-in tag_routing_prefix for collision-proof tag disambiguation
router_settings.tag_routing_prefix lets a caller explicitly mark which
x-litellm-tags/metadata.tags values are routing directives, exempting
them from the known-tag-vocabulary heuristic used to guard fail-open
against caller-invented "&"/"!" tags. Unprefixed tags keep going
through today's existing handling unchanged (hybrid, no migration
required); default "" is a full no-op.
Fixes a bug caught during live-proxy verification: the prefix-stripped
"confirmed" set kept the "&"/"!" marker character, so it never matched
required_set/excluded_set (which _split_tags always strips bare) -- the
entire trusted-required/excluded-tag mechanism silently no-opped for
its primary use case. Adds regression tests for the bare-value mismatch
and updates existing _chain_allows_fail_open/_tag_known_to_group/
_caller_constraint_sets call sites for the new routing_confirmed/
routing_prefix parameters.
* fix(router): resolve model_info.enable_tag_filtering override from the full model group, not just healthy deployments
Cooldown filtering runs before get_deployments_for_tag, so
_chain_tag_filtering_override only saw the survivors of that filter.
A model group whose only enable_tag_filtering-carrying deployment goes
into cooldown lost the override entirely, silently falling back to
the router-wide default and letting any !/&/tag constraint on that
chain be bypassed by driving the one overriding deployment into
cooldown. Resolve the override from every deployment configured for
the model instead, mirroring _tag_known_to_group's existing pattern.
Verified live: with a bad-key deployment carrying the override forced
into real cooldown via allowed_fails=1, an explicit "!provider:openai"
ban on the remaining deployment reproducibly returned 200 via OpenAI
before this fix and 401 (tag filtering still enforced) after it.
* fix(router): avoid Final-reassignment lint error and a MagicMock router fixture gap from tag_routing_prefix
_chain_tag_filtering_override's try/except reassigned a Final-annotated
name across branches, which basedpyright flags as illegal; extracted
the lookup-with-fallback into its own helper so the binding is assigned
once. Also sets tag_routing_prefix on the bare MagicMock router used by
test_router_tag_regex_routing.py's fixture, which otherwise returns an
auto-generated MagicMock (truthy, non-string) for the new attribute and
crashes _strip_routing_prefix's removeprefix() call.
* fix(router): key inherited-tag protection off provenance, not value subtraction
allow_fail_open's trusted-only pool computed "not caller-attributable"
as required_set - caller_required_set. A caller who resubmits the
exact value of an inherited "&"/"!" tag (e.g. an inherited "®ion:eu"
alongside a caller-supplied "®ion:eu" plus a conflicting
"!region:eu") collapses both origins to the same set value, so the
subtraction zeroes out the inherited requirement's protection too,
letting fail-open route outside a key/team-enforced constraint.
Adds metadata.inherited_tags in litellm_pre_call_utils.py: a snapshot
of "tags" taken after key/team/project policy is merged in but before
this request's own caller-supplied tags are merged on top. A required
or excluded tag is now protected from fail-open discard if it has ANY
inherited backing (set intersection with inherited_tags), regardless
of whether the caller also happens to submit the identical value --
this is what set membership alone could never tell apart under the
old subtraction-based approach. caller_tags is kept (documented as the
complementary record) but no longer consulted for this decision.
Verified live: a virtual key with metadata.tags=["®ion:eu"] hit
with header x-litellm-tags: ®ion:eu,!region:eu (the exact
value-collision attack) reproducibly routed to the OpenAI/us
deployment before this fix and stayed on the Anthropic/eu deployment
after it.
* fix(lint): re-ratchet budget ceilings after rebasing onto litellm_internal_staging
Regenerated ruff-strict-budget.json and type-discipline-budget.json
via make lint-ruff-budget-update / lint-type-discipline-budget-update
against the post-rebase merge-base.
* fix(proxy): compute inherited_tags from key/team/project sources directly, not a tags-list snapshot
apply_client_tag_policy_pre_auth (run from user_api_key_auth, for
_tag_max_budget_check) merges the caller's x-litellm-tags header into
the same metadata.tags list before add_litellm_data_to_request ever
runs. The previous inherited_tags snapshot ("whatever's in tags before
this function's own caller-tag merge") therefore misattributed that
caller-controlled value as policy-backed whenever a request arrived
with the header set -- Greptile flagged this as a P1 security finding.
inherited_tags is now built directly from key_metadata/team_metadata/
project_metadata's own "tags" fields, independent of the shared,
pipeline-position-dependent "tags" list's mutation history. Verified
with a direct reproduction mirroring the real pipeline (calling
apply_client_tag_policy_pre_auth on the same data dict before
add_litellm_data_to_request, as user_api_key_auth actually does): the
caller's header tag no longer appears in inherited_tags. Added a
regression test exercising that same call order; confirmed it fails
against the pre-fix snapshot approach and passes against this fix.
* fix(router): make tag_routing_prefix configurable through update_settings/get_settings and UpdateRouterConfig
router_settings.tag_routing_prefix was only ever applied via the
Router() constructor. Router.update_settings's _allowed_settings
(used directly by proxy_server.py's _add_router_settings_from_db_config
for the DB-backed router_settings path) and get_settings's
vars_to_include both omitted it, so an operator relying on that path
had the value silently ignored -- flagged by veria-ai. Also adds it to
UpdateRouterConfig (the pydantic schema behind POST /config/update),
the same bug shape LIT-3152 previously fixed for retry_policy: a field
missing from that schema gets silently dropped by
model_dump(exclude_none=True) before update_settings is ever called.
* chore(ui): regenerate schema.d.ts for UpdateRouterConfig.tag_routing_prefix
Adding tag_routing_prefix to UpdateRouterConfig changed the proxy's
OpenAPI spec; regenerate the dashboard's generated API types to match.
* fix(lint): re-ratchet budget ceilings after rebasing onto litellm_internal_staging
Regenerated ruff-strict-budget.json and type-discipline-budget.json
against the post-rebase merge-base. LIT002/LIT011 ceilings reflect
this branch's true current counts (confirmed unchanged across the
rebase by diffing against the pre-rebase commit); the base's own
counts moved independently.
* fix(lint): replace mutable-collection fallbacks with immutable ones in inherited_tags computation
key_metadata/team_metadata/project_metadata's "tags" fallbacks used
`or {}` / `or []` literals, each a LIT002 mutable-collection-construction
violation that pushed the branch 4 over its ratchet ceiling relative to
a moved base. Swapped to MappingProxyType({}) / () to match the
immutable idiom the rest of tag_based_routing.py already uses; no
behavior change, since both are falsy and only ever read via .get()/
unpacking. Tightens type-discipline-budget.json's LIT002 ceiling back
down to match, fully closing that gap (LIT011 keeps a genuine 1-count
gap from pre-existing, untouched lines in this file, non-gating).
* fix(lint): suppress LIT011 on the two new data[...] mutation sites
Both new lines follow this file's established data[...] mutation
idiom for add_litellm_data_to_request, matching the existing
suppression already on the inherited_tags line.
* test(router): lock in fallback + tag-filtering interaction
Cover the router-level fallbacks mechanism composing with tag-based
routing: a plain negation exhausting a group correctly advances to
the fallback group, the same exclusion tag exhausting every hop
correctly raises, and allow_fail_open resolving locally must not
spuriously trigger an unrelated external fallback.
* chore: retrigger CI now that litellm-docs#814 is merged
---------
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
* fix(proxy): add config_updated_at audit timestamp for virtual keys
updated_at carries Prisma's @updatedAt, so every batched spend flush
rewrites it and it cannot distinguish config changes from usage. Add an
additive config_updated_at column stamped only by key management writes
(update, bulk update, regenerate, block, unblock) via a shared helper,
expose it on key responses, and switch the key page's Last Updated to it
with a created_at fallback.
* test(proxy): assert config_updated_at survives key archival
* refactor(proxy): rename config_updated_at to settings_updated_at
