scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 83

Cast deployment costs to float in cost-based routing

Browse source 
PyYAML only resolves scientific notation to a float when a decimal point
is present, so input_cost_per_token: 1e-06 in a proxy config parses as a
string while 1.0e-06 parses as a float. lowest_cost.py summed those
values without casting.

With one string and one float cost the router raised TypeError while
sorting candidate deployments, and since async_get_available_deployments
does not catch it and the router re-raises, the caller's completion
request failed. When every cost parsed as a string the sum silently
concatenated instead, so deployments were ordered by string comparison
rather than cost.

Cast both values with a guarded float() and fall back to the existing
model_cost defaults, matching quality_router and complexity_router.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yash Raj Pandey 2026-07-20 10:51:08 -04:00
parent cd63c7e5a7
commit 56d2fdf278
1 changed files with 12 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
litellm/router_strategy/lowest_cost.py
View file

@ -251,10 +251,20 @@ class LowestCostLoggingHandler(CustomLogger):
item_input_cost = None
item_output_cost = None
if _deployment.get("litellm_params", {}).get("input_cost_per_token", None):
item_input_cost = _deployment.get("litellm_params", {}).get("input_cost_per_token")
try:
item_input_cost = float(
_deployment.get("litellm_params", {}).get("input_cost_per_token")
)
except (ValueError, TypeError):
item_input_cost = None
if _deployment.get("litellm_params", {}).get("output_cost_per_token", None):
item_output_cost = _deployment.get("litellm_params", {}).get("output_cost_per_token")
try:
item_output_cost = float(
_deployment.get("litellm_params", {}).get("output_cost_per_token")
)
except (ValueError, TypeError):
item_output_cost = None
if item_input_cost is None:
item_input_cost = item_litellm_model_cost_map.get("input_cost_per_token", 5.0)