mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
9076c33347
* fix(batches): price anthropic passthrough message batches correctly in batch cost job
Anthropic message batches created via the /anthropic passthrough were never
cost tracked. The CheckBatchCost job fetched batch results from the Files API
(POST /v1/files/msgbatch_.../content), which Anthropic rejects with "File id
must have file_ prefix"; the error response was silently wrapped as file
content, parsed as zero successful rows, logged as a $0 aretrieve_batch spend
row, and the job was marked batch_processed=true so the $0 was permanent.
Route msgbatch_ file ids to GET /v1/messages/batches/{id}/results in the
anthropic files transformation, raise on HTTP error status in
retrieve_file_content instead of returning the error body as content, parse
Anthropic's results JSONL shape (result.type == "succeeded",
result.message.usage with cache creation/read tokens) in batch_utils, price
cache creation tokens at cache_creation_input_token_cost in the batch cost
fallback (50% batch discount preserved for base input, cache reads, cache
writes, and output), and leave the managed object row unprocessed when cost
tracking fails so a later poll retries instead of permanently recording $0.
* fix(batches): carry cache token details into aggregated anthropic batch usage
|
||
|---|---|---|
| .. | ||
| auth | ||
| common_utils | ||
| hooks | ||
| management_endpoints | ||
| ui_crud_endpoints | ||
| vector_stores | ||
| __init__.py | ||
| audit_logging_endpoints.py | ||
| enterprise_routes.py | ||
| proxy_server.py | ||
| readme.md | ||
| utils.py | ||
LiteLLM Proxy Enterprise Features - Readme
Overview
This directory contains enterprise features used on the LiteLLM proxy.
Format
Create a file for every group of endpoints (e.g. key_management_endpoints.py, user_management_endpoints.py, etc.)
If there is a broader semantic group of endpoints, create a folder for that group (e.g. management_endpoints, auth_endpoints, etc.)