scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

fix(batches): resolve the deployment model from model_call_details

Browse source 
On a batch retrieve both self.model and litellm_params[model] come back None, so
the cost model fell through to the provider's own response model (an Anthropic id
like claude-opus-4-5-20251101) which does not resolve under a bedrock provider,
leaving bedrock batches at $0 with correct usage.

model_call_details carries the deployment's provider-qualified model
(bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0), confirmed by
instrumenting a live retrieve, so it is preferred with the previous two sources
kept as fallbacks.
This commit is contained in:
Marty Sullivan 2026-08-16 17:15:37 -04:00 • committed by mateo-berri
parent 0797e266cd
commit 964c9a3ca2
1 changed files with 10 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
litellm/litellm_core_utils/litellm_logging.py
View file

@ -583,13 +583,17 @@ class Logging(LiteLLMLoggingBaseClass):
def get_deployment_model_for_cost(self) -> str | None:
"""The provider-qualified model to price against.
self.model can be the router's model_group alias, which no cost map
resolves, so the deployment's own litellm_params model wins when present.
On a batch retrieve both self.model and litellm_params["model"] can be
unset, and self.model can otherwise carry the router's model_group alias,
which no cost map resolves. model_call_details holds the deployment's own
provider-qualified model, so it is preferred.
"""
deployment_model: Final = self.litellm_params.get("model") if hasattr(self, "litellm_params") else None
if isinstance(deployment_model, str) and deployment_model:
return deployment_model
return self.model
candidates: Final = (
(self.model_call_details or {}).get("model") if hasattr(self, "model_call_details") else None,
self.litellm_params.get("model") if hasattr(self, "litellm_params") else None,
self.model,
)
return next((candidate for candidate in candidates if isinstance(candidate, str) and candidate), None)
def get_router_deployment_model_info(self) -> ModelInfo | None:
"""Pricing the router registered under this deployment's model_info.id.