* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
HTTPHandler.post and AsyncHTTPHandler.post call raise_for_status before returning, so the status_code != 200 branches after the create and cancel POSTs could never run. Non-2xx already surfaces as httpx.HTTPStatusError from inside the client. The checks after GETs stay: the get helpers return without raising. Tests that faked a non-raising POST response are replaced by HTTPStatusError propagation coverage.
* fix: handle explicit outputInfo: null in Vertex AI batch response
Vertex AI can return HTTP 200 for a create_batch/get_batch call with an
explicit "outputInfo": null body (the output directory is assigned
asynchronously and may not be populated yet at response time).
_get_output_file_id_from_vertex_ai_batch_response did:
response.get("outputInfo", OutputInfo()).get("gcsOutputDirectory", "")
dict.get(key, default) only substitutes default when the key is absent,
not when it is present but explicitly None, so this crashed with:
AttributeError: 'NoneType' object has no attribute 'get'
surfaced to callers as an opaque openai.InternalServerError 500 from
litellm.create_batch()/retrieve_batch() for any Vertex AI batch job,
regardless of whether the job ultimately succeeds.
Fixed by guarding with `response.get("outputInfo") or OutputInfo()`,
matching the existing null-safe pattern already used by the sibling
_get_input_file_id_from_vertex_ai_batch_response for inputConfig. The
existing outputConfig fallback branch (a few lines below) already
handles this case correctly once it's reachable - it just never was.
Added 2 regression tests covering outputInfo: null with and without an
outputConfig fallback available.
* test: drop explanatory comment from regression test
---------
Co-authored-by: htourinho-clgx <htourinho@cotality.com>
* tests: add e2e tests for spend, budgets and llms
* style: make chained comparison of status_code clearer
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* remove e2e_tests folder
* test: add spend tracking tests
* fix: p0 issues, added types and shared functions for each test suite
* style: carry clearer status_code comparison into renamed e2e dir
* refactor: migrate to gateway client
* fix: add new tests, split gateway
* test(e2e): add live batches suite across providers and routing scenarios
* test(batches): cover real cost tracking on completed batch retrieve
* test(e2e): assert managed vs raw file and batch id shapes per routing scenario
* test(e2e): assert full response shape of each batches and files endpoint
* test(e2e): only accept transitional statuses for a freshly created batch
* test(prompt-factory): make test_convert_url deterministic with a data URL
picsum.photos is down (HTTP 522), so test_convert_url failed on every
run. Swap the live external image for an inline data: URL and assert the
round-trip through convert_url_to_base64 genuinely.
A data URL is already inline base64 image data, so convert_url_to_base64
now short-circuits it instead of attempting an impossible HTTP fetch;
add a regression for that branch in the mapped image_handling test
* fix: pass through async image data urls
* fix(image-handling): short-circuit data URLs in async path too
Bugbot flagged that convert_url_to_base64 returns data: base64 URLs
unchanged but async_convert_url_to_base64 still tried to fetch them,
so async OCR flows (Bedrock, Azure) would reject inline images the sync
path accepts. Add the same guard to the async function and a regression
test that asserts the async path returns the data URL without touching
the HTTP client
* Fix: openai batches lifecycle
* Fix: add e2e azure openai tests
* Fix e2e for vertex ai
* Add all models for testing
* test(managed-files): assert idempotent upsert in store_unified_file_id
store_unified_file_id switched from create to upsert to avoid
UniqueViolationError when re-storing the same unified_file_id (e.g.
batch output files stored before metadata is available). Update the
unit test to assert the upsert call and its create payload instead of
the removed create call.
* test(batches): reconcile vertex_ai native batch-id comment with fallback guard
* fix(test-config): keep rust-ocr models in model_list by moving files_settings after it
* fix(test-config): move batch models after OCR block to keep merge with internal_staging clean
* fix(batches): use '24hrs' completion window and allow managed-files listing with provider filter
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: ruff format transformation.py and endpoints.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(e2e/batches): set Azure raw_model to gpt-4.1-mini-batch to match deployed model
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(vertex-ai/batches): correct completion_window to 24h per Literal type definition
* test(vertex-ai/batches): align completion_window assertion to 24h
* fix: update managed file metadata on upsert
---------
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(batches): add 1:1 test file scaffold for batches component paths
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add harness test for create batch endpoint
* Add retrieve endpoint harness tests
* Add list endpoint harness tests
* Add cancel endpoint harness tests
* Add cancel endpoint harness tests
* Add test for litellm/batches/main.py
* Add test for litellm/tests/test_litellm/batches/test_batch_utils.py
* Add handler and transformation tests for all providers
* Fix: run batches tests in cicd
* fix(tests): remove azure/__init__.py that shadowed azure namespace package
Adding __init__.py to tests/test_litellm/llms/azure/ caused pytest to
insert tests/test_litellm/llms/ into sys.path[0], making our empty
azure/ dir shadow the real azure-identity namespace package. Any test
that patched azure.identity.* would then fail with AttributeError.
* style(tests): apply ruff format to test_batch_utils.py
Base migrated the formatter from black to ruff format (#31317); reformat the
batches scaffold test file to match.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
