* feat(proxy): split Azure PTU deployments across teams by share
A PTU deployment's model_info takes ptu_shares (team_id -> whole PTUs adding up to
ptu_count) in place of a single team_id. With LITELLM_ENABLE_PTU_COST_ATTRIBUTION on,
routing hides the deployment from teams holding no share, the v3 limiter enforces each
share as a per-team TPM ceiling from Azure's per-model TPM-per-PTU sizing, the daily
flat-cost rollup splits the hourly cost by share, and /team/daily/activity plus the Team
Usage page report PTU-hours next to tokens.
* fix(proxy): weight cached input in PTU ceilings and size team-scoped deployments by public name
The PTU flag is read from the process environment on every check instead of
through the secret-manager path, so a proxy with a hosted secret manager does
not pay a round trip per request. A team's ceiling now carries the model's
cached-input ratio and settlement charges cache reads at that ratio, matching
how Azure sizes a PTU. A deployment registered for one team is found by its
team_public_model_name for PTU-hours and the ceiling, since its model_name is
rewritten to a unique internal name. ptu_shares are StrictInt on ModelInfo and
parsed_ptu_shares refuses booleans, strings, and non-string team ids, a
fractional ptu_count reserves nothing, and models_processed counts distinct
deployments rather than holdings.
* fix(proxy): reserve PTU ceilings in normalized tokens, keep share routing flag-independent, and warn only shared or Azure reservations
* fix(proxy): size the uncapped output floor against the PTU share in raw output tokens and escape the exported team id
* fix(ptu): refuse ptu_shares naming a team that does not exist
* refactor(ptu): keep Azure detection under llms/azure and check share teams with one query
* fix(proxy): drop the reverted TeamIdSearchFilter import after merging main
Main reverted the team key search change that added TeamIdSearchFilter, so the PTU share team check types its where clause locally
* fix(proxy): chunk the ptu_shares team lookup under the IN-list bound
* fix(proxy): count a request naming a shared PTU deployment directly against its group's ceiling
* fix(proxy): resolve every name a shared PTU deployment is served under to its group's ceiling
* fix(ptu): refuse ptu_shares declared without ptu_count and cost_per_ptu_per_hour
* fix(ui): seed total_ptu_hours in the empty daily activity metadata
* fix(ptu): size usage rows by the name the request used
* fix(ptu): size a team's usage rows by the deployment it is served from
* fix(ptu): never size a team's usage row by another team's reservation
* fix(ptu): reserve an uncapped request against the cap the proxy writes, not its prompt length
* fix(ptu): read a share map only through the terms registration requires
A ptu_shares map stored without ptu_count, cost_per_ptu_per_hour and
ptu_effective_from, or one that does not add up to ptu_count, is refused at
registration but was enforced once loaded: every other team got the 400 and
the share holder a ceiling while no flat cost accrued. _deployment_shares now
reads the shares through ptu_terms, so such a row reserves nothing, the same
as before the map was read.
* fix(ptu): keep a declared split as the access rule and check a config split at boot with the flag off
6fb24d1d read ptu_shares through the pricing terms so a termless map reserved
nothing, but with the flag off the router skips ptu_config_error for config.yaml
entries, so such an entry loaded and served every team, against the rule that a
declared split restricts access whether or not cost attribution is on.
_deployment_shares reads the declared map again, and the router now raises
ptu_config_error for a config-sourced entry declaring ptu_shares regardless of
the flag, so a malformed split is refused at boot instead of being honoured or
ignored silently.
* fix(ptu): charge wildcard-served names, pass-through usage, and team-less settlements against the team's PTU ceiling
A name served by a wildcard route now resolves to the shared deployment behind it, so the
limiter's ceiling and the team daily activity sizing both find it. Pass-through responses
settle the PTU counter from combined_usage_object in Azure-normalized tokens, and a
reservation taken at admission is settled even when the success metadata lost the team.
* fix(ptu): resolve a wildcard-served name to the one pattern the router picks, and a deployment id before any wildcard
A team's wildcard reservation behind an open proxy-wide * route was charged and sized for traffic the * route served, and naming a shared deployment by id next to a catch-all wildcard escaped its per-team ceiling. Router.wildcard_route_deployments now owns the most-specific-proxy-wide-else-team precedence, and both the limiter and the consumption page resolve through it after listed rows and deployment ids.
* test(router): pin the wildcard route precedence that PTU sizing and the router share
* fix(proxy): accept whole-number PTU shares written as floats, as ptu_count already does
A share of 2.0 now reads as 2 on POST /model/new and in config.yaml, where it was
refused with a 422 and dropped at load. Fractional, infinite, boolean and string
shares stay refused. Adds unit tests for the malformed-rate, provider-over-prefix,
zero-token Responses usage, fractional-count and sizing-warning paths.
* fix(proxy): settle team PTU reservations in their own minute and keep billed PTU days when shares move
* fix(proxy): write each PTU deployment day's team charges in one transaction
The backfill counts a deployment day as priced once any team's row exists for it,
so a rollup that wrote team A's share and lost team C's left team C unbilled for
good. Each deployment day's team rows now go through one Prisma batch with the
retry around the whole batch, so a day is fully written or not written at all and
the next backfill prices it for every team.
* fix(proxy): read the PTU flag from a secret manager, serve share holders from their share, and refuse unfittable calls and share-holding team deletes
* test(router): list the cached PTU ceiling helpers in the router coverage check
* fix(proxy): read the PTU flag only for share holders, guard org and SCIM team deletes, keep share holders on their shared deployment through retries and the scheduler, and date PTU-hours by the terms in force
* fix(proxy): bill a rerun PTU day to the current share holders alone
* fix(proxy): replace a PTU day's rows by day and deployment, not a negated team list
* fix(proxy): read the PTU flag from the secret manager first, like get_secret
* fix(router): scope retry health lookups to the caller only for PTU share holders
* fix(router): move the share-holder retry check into ptu_shares
Adds ptu_hours and total_ptu_hours to the daily activity integration goldens and fixtures, since every route now reports them
* test(integration): cover PTU shares across routing, delete guards, cost attribution and config regex timing
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
e344dbd8db