mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-19 04:54:37 +00:00
docs(cost): rewrite stale completion_cost docstring (#47022)
The old docstring called completion_response required, said total_time was in seconds (it is milliseconds), and only covered a handful of the parameters. Replace it with a short accurate summary and fix the "calculat" typo in the unmapped-model error. Co-authored-by: Kerry Lu <kerry@berri.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
aa32e4cc88
commit
6b3a5dc3ce
1 changed files with 8 additions and 25 deletions
|
|
@ -1438,32 +1438,15 @@ def completion_cost(
|
|||
vertex_location: str | None = None, # for Vertex AI regional-endpoint uplift (e.g. "us-east5", "global")
|
||||
) -> float:
|
||||
"""
|
||||
Calculate the cost of a given completion call fot GPT-3.5-turbo, llama2, any litellm supported llm.
|
||||
Return the USD cost of a LiteLLM call.
|
||||
|
||||
Parameters:
|
||||
completion_response (litellm.ModelResponses): [Required] The response received from a LiteLLM completion request.
|
||||
Pass `completion_response` to price a response LiteLLM returned. Without one, pass `model` plus
|
||||
`messages` / `prompt` and `completion`, and token counts are computed from the text. Prices come from
|
||||
`litellm.model_cost` unless `custom_cost_per_token` or `custom_cost_per_second` is given. `total_time`
|
||||
is in milliseconds and only matters for per-second pricing such as Replicate and SageMaker.
|
||||
|
||||
[OPTIONAL PARAMS]
|
||||
model (str): Optional. The name of the language model used in the completion calls
|
||||
prompt (str): Optional. The input prompt passed to the llm
|
||||
completion (str): Optional. The output completion text from the llm
|
||||
total_time (float, int): Optional. (Only used for Replicate LLMs) The total time used for the request in seconds
|
||||
custom_cost_per_token: Optional[CostPerToken]: the cost per input + output token for the llm api call.
|
||||
custom_cost_per_second: Optional[float]: the cost per second for the llm api call.
|
||||
|
||||
Returns:
|
||||
float: The cost in USD dollars for the completion based on the provided parameters.
|
||||
|
||||
Exceptions:
|
||||
Raises exception if model not in the litellm model cost map. Register model, via custom pricing or PR - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json
|
||||
|
||||
|
||||
Note:
|
||||
- If completion_response is provided, the function extracts token information and the model name from it.
|
||||
- If completion_response is not provided, the function calculates token counts based on the model and input text.
|
||||
- The cost is calculated based on the model, prompt tokens, and completion tokens.
|
||||
- For certain models containing "togethercomputer" in the name, prices are based on the model size.
|
||||
- For un-mapped Replicate models, the cost is calculated based on the total time used for the request.
|
||||
Raises if no candidate model name resolves to pricing. Register the model via `litellm.register_model`
|
||||
or add it to model_prices_and_context_window.json.
|
||||
"""
|
||||
try:
|
||||
call_type = infer_call_type(call_type, completion_response) or "completion"
|
||||
|
|
@ -2066,7 +2049,7 @@ def completion_cost(
|
|||
)
|
||||
if idx == len(potential_model_names) - 1:
|
||||
raise e
|
||||
raise Exception(f"Unable to calculat cost for received potential model names - {potential_model_names}")
|
||||
raise Exception(f"Unable to calculate cost for received potential model names - {potential_model_names}")
|
||||
except Exception as e:
|
||||
raise e
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue