From d59491f704357dd0657aa640b1185b1db942b7f2 Mon Sep 17 00:00:00 2001 From: ishaan-jaff Date: Wed, 18 Oct 2023 17:46:37 -0700 Subject: [PATCH] (docs) calculate cost for streaming responses --- docs/my-website/docs/index.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/my-website/docs/index.md b/docs/my-website/docs/index.md index d7d2bc59bc0..f34f8e9777e 100644 --- a/docs/my-website/docs/index.md +++ b/docs/my-website/docs/index.md @@ -350,7 +350,10 @@ Cost for completion call with gpt-3.5-turbo: $0.0000775000 ``` ### Track Costs, Usage, Latency for streaming -Use a callback function for this - more info on custom callbacks: https://docs.litellm.ai/docs/observability/custom_callback +We use a custom callback function for this - more info on custom callbacks: https://docs.litellm.ai/docs/observability/custom_callback +- We define a callback function to calculate cost `def track_cost_callback()` +- In `def track_cost_callback()` we check if the stream is complete - `if "complete_streaming_response" in kwargs` +- Use `litellm.completion_cost()` to calculate cost, once the stream is complete ```python import litellm