Commit graph

9 commits

Author SHA1 Message Date
yuneng-jiang
6a0d03914c
test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)
* test: drop the cwd-relative sys.path.insert calls from the test suite

TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.

Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.

Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.

* test: drop the duplicate imports the sys.path sweep exposed to F811

* test(pre-call-utils): restore the os import the new bedrock tests need
2026-08-22 09:25:58 -07:00
yuneng-jiang
153b205d3e
test: build redaction and batch limiter fixtures the way production does (#37416)
Two suites broke because they stood in for production objects with stand-ins
that no longer answer the same way.

The redaction test faked a ResponsesAPIResponse and then reassigned
builtins.isinstance so the fake would pass the type check. Redaction now gates
on a tuple of accepted types, and the patched isinstance only recognised the
bare class, so the fake fell through to the generic branch and the assertions
ran against a plain dict. Building a real ResponsesAPIResponse drops the
builtins patch entirely and exercises the same type gate production takes.

The batch rate limiter tests constructed _PROXY_BatchRateLimiter with
parallel_request_limiter=None even though the parameter is not optional. That
stayed harmless until the output-token estimate started reading the limiter,
which turned it into an AttributeError. Inject the limiter the proxy injects,
sharing one InternalUsageCache the way _add_proxy_hooks does.
2026-08-18 19:37:50 -07:00
mateo-berri
0b09588685 refactor(batches): aggregate batch output cost, usage, and models in a single pass
Completed-batch cost tracking parsed the whole output file into a list of
dicts, pretty-printed it into debug strings even with debug logging off, and
walked the list three times (cost, usage, models), so a large batch output
could pin a worker's memory. The output is now folded line by line into small
per-line stats records via _aggregate_batch_cost_usage_models, the eager
json.dumps debug calls are gone, and the raw-vertex path computes cost and
usage in one call instead of two. _get_batch_output_file_content_as_dictionary
becomes _fetch_batch_output_file_content (returns bytes); the superseded
three-pass helpers are deleted and their tests migrated
2026-07-29 22:00:01 -07:00
Mateo Wang
6a9f542f81
test: stabilize batch VCR coverage and stop live upload/network leaks (#29477)
* test: stabilize batch VCR coverage

* test: replay bedrock batch s3 uploads

* test: stop batch tests leaking live uploads

* test: keep bedrock batch workflow off live s3

* test: mock bedrock batch workflow network

* test: accept realtime guardrail refusal wording

* test: update gemini thought signature model

* test: quiet logging worker atexit flush

* test: address Greptile review on batch VCR fixes

Handle content= bodies in the bedrock batch post stub so payload
extraction does not raise a TypeError when a request omits json and
data. Restore litellm list state faithfully by preserving None instead
of coercing it to an empty list, so callbacks that start as None are not
turned into [] after a test. Set logging.raiseExceptions inside the try
block in the atexit flush so the finally always restores the previous
value.

* test: scope atexit logging suppression to the drain loop

Wrap only the queue drain loop in LoggingWorker._flush_on_exit with the
logging.raiseExceptions toggle so the process-wide global is suppressed for
the smallest possible window, keeping other threads' logging error reporting
intact outside the loop.

* test: cover atexit flush error-swallow branch in LoggingWorker

The _flush_on_exit drain loop was wrapped in a try/finally to scope the
logging.raiseExceptions toggle, which reindented the existing edge-case
branches into the diff and dropped patch coverage below target. Add a
regression test that enqueues a coroutine which raises during the atexit
flush and asserts the failure is swallowed while later queued events are
still drained, exercising the silent-failure path directly.
2026-06-02 16:11:52 -07:00
Ishaan Jaffer
e8461b5b97
style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
Sameer Kankute
e0c98d62d4 Fix: LoggingWorker Missing Azure Credentials When Fetching 2026-02-13 13:05:49 +05:30
Sameer Kankute
9a9315043f Fix: Batch Rate Limiter Cannot Access User Files 2026-02-13 12:23:37 +05:30
Sameer Kankute
45c8edbe25 Fix: Bug: Batch Rate Limiter Cannot Access User Files 2026-02-10 12:25:26 +05:30
Ishaan Jaff
aea78b8d1a
[Feat] Add support for Batch API Rate limiting - PR1 adds support for input based rate limits (#16075)
* add count_input_file_usage

* add count_input_file_usage

* fix count_input_file_usage

* _get_batch_job_input_file_usage

* fixes imports

* use _get_batch_job_input_file_usage

* test_batch_rate_limits

* add _check_and_increment_batch_counters

* add get_rate_limiter_for_call_type

* test_batch_rate_limit_multiple_requests

* fixes for batch limits

* fix linting

* fix MYPY linting
2025-10-29 18:28:52 -07:00