Coverage jobs collect tests/test_litellm/batches. Move the mixed Responses and chat regression into that suite so CI exercises the normalization branch.
Completed-batch cost tracking parsed the whole output file into a list of
dicts, pretty-printed it into debug strings even with debug logging off, and
walked the list three times (cost, usage, models), so a large batch output
could pin a worker's memory. The output is now folded line by line into small
per-line stats records via _aggregate_batch_cost_usage_models, the eager
json.dumps debug calls are gone, and the raw-vertex path computes cost and
usage in one call instead of two. _get_batch_output_file_content_as_dictionary
becomes _fetch_batch_output_file_content (returns bytes); the superseded
three-pass helpers are deleted and their tests migrated
* feat(batches): track cost for unmanaged Bedrock batches, generalize the flag
CheckBatchCost skipped Bedrock batches whose unified_object_id is a raw
model-invocation-job ARN, the same root cause previously fixed for
unmanaged Vertex batches. Bedrock batches embed the model name in their
s3:// input file name instead (litellm-bedrock-files-{model}-{uuid}.jsonl),
so the same routing mechanism now derives the model from that layout and
matches it to a configured bedrock deployment.
track_unmanaged_vertex_batch_cost is renamed to track_unmanaged_batch_cost
since two providers now share this mechanism.
* fix(batches): parse Bedrock batch output and price with deployment model name
Bedrock model-invocation-job results use modelOutput/error rows and short
internal model ids that are not in the cost map, so unmanaged batch cost
tracking logged tokens but $0 spend. Use deployment model name for pricing
and add regression tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(batches): price anthropic passthrough message batches correctly in batch cost job
Anthropic message batches created via the /anthropic passthrough were never
cost tracked. The CheckBatchCost job fetched batch results from the Files API
(POST /v1/files/msgbatch_.../content), which Anthropic rejects with "File id
must have file_ prefix"; the error response was silently wrapped as file
content, parsed as zero successful rows, logged as a $0 aretrieve_batch spend
row, and the job was marked batch_processed=true so the $0 was permanent.
Route msgbatch_ file ids to GET /v1/messages/batches/{id}/results in the
anthropic files transformation, raise on HTTP error status in
retrieve_file_content instead of returning the error body as content, parse
Anthropic's results JSONL shape (result.type == "succeeded",
result.message.usage with cache creation/read tokens) in batch_utils, price
cache creation tokens at cache_creation_input_token_cost in the batch cost
fallback (50% batch discount preserved for base input, cache reads, cache
writes, and output), and leave the managed object row unprocessed when cost
tracking fails so a later poll retries instead of permanently recording $0.
* fix(batches): carry cache token details into aggregated anthropic batch usage
* test(batches): add 1:1 test file scaffold for batches component paths
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add harness test for create batch endpoint
* Add retrieve endpoint harness tests
* Add list endpoint harness tests
* Add cancel endpoint harness tests
* Add cancel endpoint harness tests
* Add test for litellm/batches/main.py
* Add test for litellm/tests/test_litellm/batches/test_batch_utils.py
* Add handler and transformation tests for all providers
* Fix: run batches tests in cicd
* fix(tests): remove azure/__init__.py that shadowed azure namespace package
Adding __init__.py to tests/test_litellm/llms/azure/ caused pytest to
insert tests/test_litellm/llms/ into sys.path[0], making our empty
azure/ dir shadow the real azure-identity namespace package. Any test
that patched azure.identity.* would then fail with AttributeError.
* style(tests): apply ruff format to test_batch_utils.py
Base migrated the formatter from black to ruff format (#31317); reformat the
batches scaffold test file to match.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
