scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 79

docs - batches cost tracking (#7422)

Browse source
This commit is contained in:
Ishaan Jaff 2024-12-25 20:13:26 -08:00 • committed by GitHub
parent 4f49f836aa
commit 005f2fa1aa
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 19 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
docs/my-website/docs/batches.md
View file

@ -150,4 +150,23 @@ print("list_batches_response=", list_batches_response)
### [Vertex AI](./providers/vertex#batch-apis)
## How Cost Tracking for Batches API Works
LiteLLM tracks batch processing costs by logging two key events:
| Event Type | Description | When it's Logged |
|------------|-------------|------------------|
| `acreate_batch` | Initial batch creation | When batch request is submitted |
| `batch_success` | Final usage and cost | When batch processing completes |
Cost calculation:
- LiteLLM polls the batch status until completion
- Upon completion, it aggregates usage and costs from all responses in the output file
- Total `token` and `response_cost` reflect the combined metrics across all batch responses
## [Swagger API Reference](https://litellm-api.up.railway.app/#/batch)