mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
c2b3c4b1e4
* feat(ptu): accrue flat cost for PTU deployments declared in config.yaml The flat-cost rollup reads deployments from LiteLLM_ProxyModelTable, and config.yaml models never reach that table by design, so a PTU deployment declared there accrued no flat cost at all while still billing its traffic per token. The provider bills the reservation whichever file declared it. The rollup now also reads the deployments the router holds that no database row owns, identified by db_model, skipping the per-request credential clones that carry original_model_id and reuse their source's PTU config under a fresh id. Registering such a deployment zeroes its pricing, since reserved capacity already pays for the traffic it serves, and leaving a rate unset falls back to the public cost map, which makes the double charge the default rather than an opt-in. The rules both halves apply now live in one module. The rollup's test for what it will charge and the router's test for what to zero have to agree, or a deployment one accepts and the other declines serves its traffic for free. That module also owns the fields the write endpoints already zero, so the two paths cannot drift: tiered_pricing is emptied rather than zeroed because its tiers outrank the rates beside them, the search context table is written zeroed because an absent one means the provider default, and any further rate the deployment itself declares is zeroed alongside the standing set. The prune is bounded to the deployments a run scanned, but only for a run that priced a config-declared deployment. Deciding a row is garbage on staleness alone stays correct while every run derives its charges from the same table, so a database-only run sweeps exactly as it did before; once one host's charges come from a file the others cannot read, a row it never considered is not evidence of anything. Behaviour change worth calling out: a zeroed deployment sorts ahead of an unpriced sibling in QualityRouter's cost tiebreak, where an unset rate previously sorted last. Reserved capacity really is the cheaper choice, but the ordering moves. * refactor(ptu): drop a Final rebind and two redundant isinstance guards The basedpyright budget rejected reassigning a Final in the datetime coercion and two isinstance calls the router entry's own type already guarantees. Filtering the built records rather than the raw entries removes both guards and leaves _router_deployment as the single validator. |
||
|---|---|---|
| .. | ||
| test_budget_reservation_redis_failure.py | ||
| test_cloudzero_endpoints.py | ||
| test_compression_savings.py | ||
| test_ptu_feature_flag.py | ||
| test_ptu_flat_cost_rollup.py | ||
| test_savings.py | ||
| test_spend_log_error_logger.py | ||
| test_spend_management_endpoints.py | ||
| test_spend_query_optimization.py | ||
| test_spend_tracking_utils.py | ||