litellm/tests/batches_tests
Sameer Kankute a16d9c6f9e
test(e2e): add live batches suite across providers and routing scenarios (#30958)
* tests: add e2e tests for spend, budgets and llms

* style: make chained comparison of status_code clearer

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* remove e2e_tests folder

* test: add spend tracking tests

* fix: p0 issues, added types and shared functions for each test suite

* style: carry clearer status_code comparison into renamed e2e dir

* refactor: migrate to gateway client

* fix: add new tests, split gateway

* test(e2e): add live batches suite across providers and routing scenarios

* test(batches): cover real cost tracking on completed batch retrieve

* test(e2e): assert managed vs raw file and batch id shapes per routing scenario

* test(e2e): assert full response shape of each batches and files endpoint

* test(e2e): only accept transitional statuses for a freshly created batch

* test(prompt-factory): make test_convert_url deterministic with a data URL

picsum.photos is down (HTTP 522), so test_convert_url failed on every
run. Swap the live external image for an inline data: URL and assert the
round-trip through convert_url_to_base64 genuinely.

A data URL is already inline base64 image data, so convert_url_to_base64
now short-circuits it instead of attempting an impossible HTTP fetch;
add a regression for that branch in the mapped image_handling test

* fix: pass through async image data urls

* fix(image-handling): short-circuit data URLs in async path too

Bugbot flagged that convert_url_to_base64 returns data: base64 URLs
unchanged but async_convert_url_to_base64 still tried to fetch them,
so async OCR flows (Bedrock, Azure) would reject inline images the sync
path accepts. Add the same guard to the async function and a regression
test that asserts the async path returns the data URL without touching
the HTTP client

* Fix: openai batches lifecycle

* Fix: add e2e azure openai tests

* Fix e2e for vertex ai

* Add all models for testing

* test(managed-files): assert idempotent upsert in store_unified_file_id

store_unified_file_id switched from create to upsert to avoid
UniqueViolationError when re-storing the same unified_file_id (e.g.
batch output files stored before metadata is available). Update the
unit test to assert the upsert call and its create payload instead of
the removed create call.

* test(batches): reconcile vertex_ai native batch-id comment with fallback guard

* fix(test-config): keep rust-ocr models in model_list by moving files_settings after it

* fix(test-config): move batch models after OCR block to keep merge with internal_staging clean

* fix(batches): use '24hrs' completion window and allow managed-files listing with provider filter

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: ruff format transformation.py and endpoints.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(e2e/batches): set Azure raw_model to gpt-4.1-mini-batch to match deployed model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vertex-ai/batches): correct completion_window to 24h per Literal type definition

* test(vertex-ai/batches): align completion_window assertion to 24h

* fix: update managed file metadata on upsert

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-02 08:05:23 -07:00
..
batch_job_results_furniture.jsonl (feat) Add cost tracking for /batches requests OpenAI (#7384) 2024-12-23 17:47:26 -08:00
bedrock_batch_completions.jsonl test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
conftest.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
openai_batch_completions.jsonl (feat) Add cost tracking for /batches requests OpenAI (#7384) 2024-12-23 17:47:26 -08:00
pathrise-convert-1606954137718.json use pathrise-convert-1606954137718 2025-01-05 14:14:43 -08:00
test_batch_custom_pricing.py fix(model_prices): correct regional processing uplift to gpt-5.4/5.5 series only (#31136) 2026-06-23 15:57:26 -07:00
test_batch_rate_limits.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_batches_logging_unit_tests.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
test_bedrock_files_and_batches.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_fine_tuning_api.py [Fix] Fine-Tuning Test: Bump Off Deprecated gpt-3.5-turbo-0125 2026-05-07 14:41:58 -07:00
test_hosted_vllm_batches_and_files.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_manus_files_all_methods.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_batches_and_files.py fix(vertex_ai/files): single media upload for batch files to fix 499s on large uploads (#31653) 2026-06-29 17:31:32 -07:00
vertex_batch_completions.jsonl (feat) Add cost tracking for /batches requests OpenAI (#7384) 2024-12-23 17:47:26 -08:00