LITELLM_ENABLE_PTU_COST_ATTRIBUTION, read through get_secret_bool and defaulting to
false, makes the whole PTU flat-cost feature inert unless an operator opts in. The
daily rollup cron is not registered at all, so no sentinel row is ever written;
/model/new and /model/{id}/update reject a request that carries any PTU model_info
field with a 400 naming the fields and the env var rather than dropping them; the
daily activity read path reports zero flat cost; and the model add and edit forms
hide the four PTU inputs.
The read gate lives where flat cost enters SpendMetrics rather than in the aggregated
SQL select. /team/daily/activity, the endpoint the Usage page reads, is served by the
paginated find_many path and never runs that query, so forcing the select to a
constant zero would have left the reporting surface that matters still showing flat
cost.
Sentinel row filtering is deliberately not gated. An operator can enable the flag,
accrue rows under the __ptu_flat_cost__ api_key, then disable it, and those rows stay
in LiteLLM_DailyTeamSpend; gating the filter too would surface the sentinel as a bogus
api_key and mint a provider bucket for its empty provider. Response fields keep their
shape and report 0.0, so typed clients are unaffected, and the migration and the
ModelInfo field declarations are untouched.
The write gate reads the incoming request rather than the merged deployment, so a
model configured during an earlier opt-in stays editable, and the edit form drops the
PTU keys from the payload instead of sending nulls that would clear stored config.
The dashboard reads the flag from a read-only enable_ptu_cost_attribution key on
/get/ui_settings, computed from the environment on every read. It is deliberately not
an allowlisted persisted setting, and PATCH /update/ui_settings rejects it with a 400,
so an admin cannot flip an env-gated feature from the UI.
Two review findings on the gate itself. The PTU clear loop now runs only when the
feature is enabled: the write gate rejects a value but lets an explicit null through,
and a client round-tripping a model_info blob sends the PTU keys as nulls, so a
disabled proxy would have quietly erased a billing configuration set up during an
earlier opt-in. Disabling pauses PTU rather than discarding its setup. And the
dashboard flag is re-read every thirty seconds instead of the hour the other UI settings
use, since those are persisted records while this one tracks the proxy process; a
restart that flips the variable would otherwise leave the model form offering inputs
the backend now rejects. The flag is polled rather than only marked stale, since a form
that stays mounted and focused never refetches on its own.
The read gate checks the row before the flag. It runs once per metric accumulation and a
record fans out across roughly a dozen breakdowns, while the flag reads through the secret
manager uncached, so consulting it for every accumulation put thousands of lookups on a
shared endpoint that made none before. Only a row actually carrying flat cost reaches it.
Add PTU count, cost per PTU per hour, and effective-from/to date-time pickers to
the model add and edit forms; the create submit and the edit save map the picker
values to model_info as ISO strings
On the Team Usage Cost tab the money tile becomes Total Cost once a team has
accrued flat cost, and expands to a Request Cost and Flat Cost breakdown, so the
summary row stays at five tiles and the cards keep their width. Each of the three
carries a tooltip, including that flat cost is reported rather than charged
against budgets. The Daily Spend chart stacks Flat cost on Request cost, with a
tooltip that splits the two and shows the total
A team that has accrued no flat cost renders exactly as it did before, and other
entity views are unchanged. CSV export gains Flat Cost and Total Cost columns
when a team accrued non-zero flat cost; the existing Spend header is left alone
so downstream parsers keep working
Both forms validate the PTU pair through one shared module. The count rule rejects
a fractional, zero or negative value, and a rate rule rejects a negative one, each
mirroring a contract the backend enforces. Keeping the rate rule shared rather than
on a single form is deliberate: the edit form previously validated only the count,
so a negative rate typed past the input's min reached the backend and failed the
save with a 400 the operator had no way to anticipate
Both forms require PTU Effective From once PTU Count is set, matching the backend, which
rejects PTU config without a start because flat cost accrues from that instant and an
inferred one would bill days a deployment did not exist. The rule lives beside the count
and rate rules in the shared module, so the add and edit paths cannot drift.
Aggregate the ptu_flat_cost written by the rollup into SpendMetrics.flat_cost and
DailySpendMetadata.total_flat_cost, so /team/daily/activity returns flat cost
alongside per-request spend. The aggregated SQL path selects ptu_flat_cost only
for LiteLLM_DailyTeamSpend and a constant zero for the other daily tables, keeping
the response shape uniform.
Rows written under the PTU sentinel api_key add their flat cost to every parent
bucket (per-model, per-day, per-team totals) but never appear as an api_key row in
any breakdown, and are excluded from the per-request provider breakdown; the
sentinel string is not a real key alias. Both flat_cost and total_flat_cost default
to zero, so a read of any entity without PTU config is unchanged.
The sentinel row now keys on the deployment id, so the per-model breakdown keys it on
model_group instead. That breakdown key is rendered directly as a label by the Usage page
and the daily_with_models export, and a deployment id there would read as a UUID. Two
deployments sharing a public name merge under it, which is the collapse the write path
used to do by summing them into one row. Request rows are untouched and still key on
model, since their model_group is a routing concept rather than a display name.
Add ptu_count, cost_per_ptu_per_hour, ptu_effective_from and ptu_effective_to to
ModelInfo so a model deployment can carry the inputs for provisioned-throughput
flat-cost attribution. ModelInfo validates per-field bounds (positive count,
non-negative rate, effective_to after effective_from); model/new and
model/{id}/update enforce the cross-field invariant (count and rate set together,
team_id required) on the effective model_info so partial updates validate the
merged result, and v1/model/info returns the fields.
LiteLLM_DailyTeamSpend gains ptu_flat_cost and ptu_source_model_id columns plus a
sentinel api_key constant; the daily rollup that writes them lands in a follow-up
PR. Adding the optional model_info fields is backward compatible; models without
them are unaffected.
ptu_effective_from is required alongside the count and rate rather than optional. Flat
cost accrues from that instant, so an absent start has to be inferred, and inferring it
let a deployment configured today be billed for days it did not exist. Both PTU validators
also run over the merged view before any write on the update path, beside the premium check the create path
already runs there: the team ACL update below autocommits, so a validator raising further
down left the team mutated and the deployment row never written.
The update path validates the model_info a patch would store rather than the patch
alone. An invariant holds over the deployment as it will exist, not over whichever
subset of fields a caller sent, and validating the patch rejected raising the rate on
an already configured model because that patch carries no start of its own.
Both restated what the test name and the surrounding setup already say, so
they were maintenance cost without explanatory value. The reasoning they
carried lives in the commit that added the gates.
/policies/list and /prompts/list are default-deny for internal_user, but the
Virtual Keys create/edit flow, the Teams forms and the Playground called them
on mount, so every internal user landing on the dashboard fired two requests
that 401. Add viewPolicies and viewPrompts to the capability map and use them
to gate the nav entry, the form field and the fetch together, following the
pattern from the Tool Policies migration. Non-admins now see no policy or
prompt selector at all rather than an empty dropdown.
The Usage page admits internal users because their own usage view works,
but the entity breakdown selector inside it also offered Organization
Usage, so picking it fired /organization/daily/activity and collected a
401. Neither that route nor /agent/daily/activity appears in any non-admin
route list, so both are default-deny. The team breakdown leaked the second
one too: it fetches agent activity unconditionally to fill its Top Agents
card, which 401s for the same roles.
Adds viewOrganizationUsage and viewAgentUsage to the existing capability
map and points the selector option, the page section, and the fetch's
enabled flag at the same capability, so a role that cannot call the
endpoint never sees the breakdown and never issues the request. The team
and tag breakdowns, which internal users can read, are untouched, and the
default Usage view was already one of those.
The Logs nav entry is open to internal users so they can read their own
request logs, but the page rendered all four tabs unconditionally. Audit
Logs calls GET /audit and Deleted Teams calls GET /v2/team/list?status=deleted,
neither of which an internal user is permitted to call, so the page fired
requests that came back 401.
Gate both tabs on new viewAuditLogs / viewDeletedTeams capabilities, using
the same CAPABILITY_ROLES map and useCan hook introduced for Tool Policies.
Hiding a tab drops its panel from the tree entirely, so the request is never
issued rather than issued and rejected.
Selecting a tab also mapped index 0 to "request logs" and every other index
to "audit logs", which activated the audit panel whenever a user opened
Deleted Keys or Deleted Teams. Derive the active tab from the visible tab
list instead, so the mapping survives tabs being filtered out.
* fix(ui): show team BYOK models in team fallback settings
Team router settings loaded fallback options from /model_group/info, which resolves models without a team, so a team's own BYOK deployments were never selectable in its own fallback config. Load the team-scoped listing when a team id is present.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): ignore stale team model responses in router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): use react-query for fallback model listing in router settings accordion
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Normalize MCP description nullability for MultiSelect, preserve model
selection via functional setState, and update endpoint/vector-store
selector tests for the shadcn combobox API
* feat(proxy): return per-group model provenance on /team/info
/team/info now carries access_group_details, one entry per resolved access
group with its id, name, and model list, so the UI can attribute each
inherited model to the group granting it. The batch resolver returns the
access group rows keyed by id instead of a stringly dict of lists, and the
team member budget helper returns a copy instead of mutating its parameter.
Type discipline and basedpyright budgets ratchet down accordingly.
* feat(ui): allow group-only teams and show model provenance on hover
Team create and edit no longer require a model selection: an empty
selection is saved as the no-default-models sentinel, never as a bare
empty list, since an empty team model list means unrestricted access.
The team info Models card now renders every badge with a hover tooltip
naming how the team got that model: directly, via named access groups,
or both, and group-granted badges stay visible when the direct list is
empty or a sentinel.
* refactor(proxy): dedupe access group ids and return copies instead of mutating
Duplicate access_group_ids no longer amplify the /team/info response: ids
collapse order-preserving before provenance is built, pinned by a regression
test. The resolver returns a model_copy rather than mutating its parameter,
and the team create call sends a new object instead of reassigning
formValues.models. Budgets ratchet down further with the mutation removal.
* feat(ui): resolve user email/alias in usage export instead of raw user id
* test(ui): cover email/alias resolution in usage export data builders
* chore(ui): drop explanatory comment per repo comment policy
GHSA-2v37-7h3g-55p8 (CVE-2026-67213) rates 8.2 against nanoid 3.3.16 and reds osv-scan on every PR into staging. Custom generators can loop indefinitely when size is zero, so a caller that passes through a zero size hangs the process.
nanoid is transitive through the dashboard's toolchain and 3.3.17 is a patch release that published 08-03, so it already clears the .npmrc min-release-age=3 guard. The lock diff is the version, resolved url, and integrity hash for that one package.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): add apply_user_budget_to_team_keys opt-in
PR #32005 made a user's personal max_budget apply to their team-scoped keys
too, and PR #35271 reverted the whole thing (behavior plus the
skip_user_budget_on_team_key opt-out) because that flipped the default for
everyone. This brings the behavior back the other way round: default is
unchanged, and general_settings.apply_user_budget_to_team_keys opts a
deployment into charging the key owner's personal budget on team keys.
The flag reaches all three personal-budget gates so an opted-in deployment
enforces consistently: the read-time check in common_checks, the optimistic
reservation counter in _get_budget_counters, and the _PROXY_MaxBudgetLimiter
pre-call hook. It is also in the /config/list allowed args and, unlike the
reverted flag, in the _update_general_settings propagation allowlist, so the
Admin UI General Settings toggle actually takes effect at runtime; an explicit
YAML value still wins over the DB value on reload.
get_config_list's allowed_args moves to a module-level frozen mapping of
field name to type string, dropping 18 LIT002 violations and rebuilding one
less dict per request.
* style(proxy): drop explanatory comments from the budget flag paths
Replaces the five launch models with the two that SCX.ai now leads on.
Both are live on api.scx.ai and both were verified against it for tool
calling, json_object and json_schema output, reasoning, prompt caching,
and, for Qwen3.8 Max, image input
Pricing follows SCX's published USD rates. GLM-5.2 lands at $0.55/M input
and $1.9255/M output, tracking the recent GLM-5.2 market repricing;
Qwen3.8 Max at $1.815/M and $5.4461/M sits under the only other seller of
that model, and is the first Qwen3.8 Max entry in the catalog
Also corrects a metadata bug the removed entries carried: they set
max_tokens equal to max_input_tokens, conflating the context window with
the output cap. Both new entries declare a max_output_tokens of 131072,
which is what the endpoint's own validator enforces
The Add Model placeholder moves to scx-ai/GLM-5.2 now that MiniMax-M2.7
is no longer in the catalog
Closes GHSA-5p4m-2wfm-xmqj (CVSS 7.5), flagged by osv-scan against
ui/litellm-dashboard/package-lock.json. js-yaml is pinned by an exact
npm override, so the override and the lock move together.
Dev-only dependency: js-yaml reaches the tree through eslintrc, knip
and @redocly/openapi-core, none of which ship in the built dashboard.
4.3.1 published 2026-07-31, clear of the 3-day min-release-age cooldown.
Cancel in-flight model fetches when the key or source changes so an older
response cannot overwrite modelInfo. Drop the inverted endpoint-filter
assertion; filtering coverage lands in the next stack PR
Replace ChatUI cards, inputs, dialogs, popovers, MCP selects, uploads,
tooltips, and icons with shadcn/Base UI and Lucide. Update ChatUI tests
to drive searchable combobox controls instead of Ant Design selectors
Continue the Playground Chat Ant Design/Tremor migration: shared MultiSelect,
upload validation with semantic file inputs, collapsible message widgets, and
AdditionalModelSettings on Base UI controls
* fix(ui): make the expired-miss stat row a focusable tooltip trigger
* fix: auto-router expired-miss percentage and cost-optimization tab labels
- change expired-miss percentage denominator from return-to-tier misses to
all measured turns (same_model + first_visit + return_to_tier). when
auto-routers flip tiers rapidly within TTL, return-to-tier turns become
hits and disappear from the miss count; the old metric reported only the
rare failure population. the new metric contextualizes that population as
a share of overall coverage
- rename usage tab from 'Usage' to 'Overall'
- rename auto-router-usage tab from 'Auto-Router Usage' to 'Auto-Router'
- update component and unit tests to match new semantics
* fix(proxy): include today's UTC bucket when a daily activity range ends at the caller's current day
* fix(proxy): gate the current-UTC-day extension behind an opt-in param sent by the cost optimization dashboard
* fix(ui): label cost optimization savings dates as UTC days
get_deployment_credentials_with_provider dropped s3_region_name,
s3_encryption_key_id, and aws_batch_role_arn because
CredentialLiteLLMParams never declared them, and it never returned the
deployment's model, so proxy batch creation against Bedrock failed with
"LiteLLM doesn't support custom_llm_provider=bedrock for 'create_batch'"
or "AWS IAM role ARN is required" (#25104)
Provider-only file and batch calls keep their no-model contract:
get_team_provider_credentials strips the model key so a provider-scoped
request is not pinned to an arbitrary matching deployment
* fix(auto-router): accept every reminder marker pair a harness emits
reminder_markers held one (open, close) pair, so a harness that wraps
injected context differently per agent type only got the slice of traffic
using the configured envelope stripped. Every other agent type kept hitting
the original bug: its reminder-only turn never stripped to empty, won
"newest human ask", and the harness blob got classified in place of the
real question, choosing the tier and therefore the spend.
The field now takes a list of ReminderMarkerPair, following the
KeywordTierRule pattern already in this file so each pair validates itself
and errors point at reminder_markers.N.close rather than a bare index.
Blocks from different pairs can nest, which the gap construction could not
handle: resuming the kept text at an inner block's end walks back inside
the enclosing block and leaks its remainder. Running the block ends through
a maximum collapses nested and overlapping spans without a separate merge
pass, and stays linear in block count, which a fold over a growing tuple
of merged spans would not.
A single pair's ends already increase, so the maximum is the identity and
the default path is byte-identical: verified against the shipped function
over 200k generated inputs, and every existing reminder test passes
unchanged. The prior single-pair config shape is rejected loudly at
startup and at /model/new rather than silently stripping nothing.
* docs(auto-router): document reminder_markers in the complexity router README
* chore(ui): regenerate dashboard API types for the reminder_markers shape
---------
Co-authored-by: Abhimanyu Kapur <38531241+akapur99@users.noreply.github.com>
