mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
22f68c0c6b
The auto-router savings driver recomputes what the served request cost, but that request is not a counterfactual: it ran, and the cost calculator already billed it and wrote the number down. Recomputing means restating every pricing dimension the biller applied, and the two this missed were enough to halve it. A request billed at a priority tier is recomputed at standard rates, and a regional host's uplift is dropped entirely, so the driver writes a savings figure into the same rollup row as the `spend` it disagrees with. On `gpt-5.4-mini` at priority the row is billed 0.024 and the driver prices the same usage at 0.012. Neither omission cancels between the two arms, because both are per-model. The uplift is a multiplier read off each model's own entry, so 1.1*A - 1.1*B is 1.1*(A-B) and a model without one does not move at all. Tier coverage is sparser and asymmetric: `gpt-5.6` has priority rates and `gpt-5.4-nano` has none. `cost_breakdown` already carries the answer and already reaches the call site. The cost calculator records it, it rides the standard logging payload into the spend log's metadata, and OTEL, the log drawer and the response headers all read it rather than re-deriving; this driver was the only downstream consumer in the tree still pricing a completed request from its tokens. `input_cost` and `output_cost` sum to exactly what the pricer returns, so the served arm reads them. Tool spend, discount and margin stay out, since the counterfactual cannot be priced with them and charging them to one arm alone would read as the router losing money on every tool call. The baseline never ran, so it is still priced through the cost engine, now on the basis the biller used. `CostBreakdown` carries that basis because it cannot be recovered afterwards: the tier the biller used comes from `optional_params`, which no log record keeps, and the served tier that does survive on the usage object is a different fact with the opposite precedence. Rows written before this shipped carry no basis and price at standard rates, exactly as they do today; there is no backfill. Two smaller things in the same path. The router is passed as a provider rather than a router, so a spend write that was never auto-routed no longer fetches and discards one, and the complexity router resolves its messages once per hook instead of once per consumer. |
||
|---|---|---|
| .. | ||
| test_budget_reservation_redis_failure.py | ||
| test_cloudzero_endpoints.py | ||
| test_compression_savings.py | ||
| test_savings.py | ||
| test_spend_log_error_logger.py | ||
| test_spend_management_endpoints.py | ||
| test_spend_query_optimization.py | ||
| test_spend_tracking_utils.py | ||