litellm/tests/test_litellm/proxy/openai_files_endpoint
yassin e2e7f5f879 feat(vertex_ai): stream GCS batch output files from /v1/files/{id}/content
Vertex AI file content retrieval downloaded the whole GCS object into memory
before responding, which made large batch output files (hundreds of MB, image
generation JSONL past 4 GiB) impractical to fetch through the proxy.

Add BaseLLMHTTPHandler.async_retrieve_file_content_streaming, an httpx
stream=True path that hands the byte iterator to the provider config through
the new BaseFilesConfig.transform_file_content_stream hook and closes the
response on completion, early close, and HTTP error. VertexAIFilesConfig peeks
at the first JSONL row: Generate Content batch output is converted to OpenAI
batch format one row at a time (content-length dropped since it changes),
embeddings output stays buffered so fanned-out rows can be regrouped, and
anything else passes through with the upstream content-type and content-length.

vertex_ai joins FILE_CONTENT_STREAMING_PROVIDERS, so the proxy returns a
StreamingResponse for it while OpenAI-compatible providers and the buffered
Vertex path are unchanged.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:03:22 +00:00
..
test_batch_guardrails.py fix(proxy): scan batch records with the content hooks that are not guardrails (#37786) 2026-08-21 11:20:23 -07:00
test_files_batch_file_validation.py fix(files): treat nonpositive max_batch_file_size_mb as no cap 2026-08-19 15:05:17 -07:00
test_files_common_utils.py fix(bedrock): map real batch record counts and guard zero-count retire 2026-08-29 01:05:36 -07:00
test_files_endpoint.py feat(vertex_ai): stream GCS batch output files from /v1/files/{id}/content 2026-09-16 23:03:22 +00:00
test_general_upload_validation.py refactor(proxy): drop redundant docstrings from upload allowlist helpers and tests 2026-09-14 19:00:49 +00:00
test_storage_backend_service.py fix(proxy): skip prisma-dependent hooks when no database is attached 2026-08-08 01:44:56 -07:00