From 005f2fa1aa3991b088e3614548dc7c2309efaf95 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Wed, 25 Dec 2024 20:13:26 -0800 Subject: [PATCH] docs - batches cost tracking (#7422) --- docs/my-website/docs/batches.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/my-website/docs/batches.md b/docs/my-website/docs/batches.md index 2cd4a7c3ba5..4ac9fa61e3b 100644 --- a/docs/my-website/docs/batches.md +++ b/docs/my-website/docs/batches.md @@ -150,4 +150,23 @@ print("list_batches_response=", list_batches_response) ### [Vertex AI](./providers/vertex#batch-apis) +## How Cost Tracking for Batches API Works + +LiteLLM tracks batch processing costs by logging two key events: + +| Event Type | Description | When it's Logged | +|------------|-------------|------------------| +| `acreate_batch` | Initial batch creation | When batch request is submitted | +| `batch_success` | Final usage and cost | When batch processing completes | + +Cost calculation: + +- LiteLLM polls the batch status until completion +- Upon completion, it aggregates usage and costs from all responses in the output file +- Total `token` and `response_cost` reflect the combined metrics across all batch responses + + + + + ## [Swagger API Reference](https://litellm-api.up.railway.app/#/batch)