mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-01 07:03:02 +00:00
a7030a5a2c
* fix(vertex_ai): batch Claude models in Anthropic's native row shape
Claude deployments on Vertex were filed under publishers/google/models/<claude>
when the batch input was uploaded, so create_batch asked Vertex for a Google
model that doesn't exist and got a bare 404 on batchPredictionJobs. The rows
were also written as Gemini GenerateContentRequests, which Vertex rejects for
Claude even with the right model path
Claude uploads now go under publishers/anthropic/models/<model> and each line
becomes {"custom_id", "request": <Anthropic Messages body with
anthropic_version vertex-2023-10-16>} using the existing Vertex Claude
transform. Output rows in the Anthropic message shape are converted back to
OpenAI batch rows (custom_id from the row, errors as failed rows), native rows
are costed from Anthropic usage, and create_batch fails fast with a 400 when a
Claude job targets the global location, which Vertex doesn't serve for
partner model batches
Fixes #45671
* test(vertex_ai): cover malformed Claude output rows and rows without custom_id
* fix(vertex_ai): build Claude batch rows with chat's param mapping and surface Vertex batch create errors
Claude batch rows now go through get_optional_params like Vertex Claude chat, so unsupported params, top_k, response_format and extra_body follow drop_params. Tool names Claude would rename get a 400 at upload. A refused batch create returns Vertex's own status and message.
* fix(vertex_ai): price Claude batch output rows without a request echo, keep @ version ids, refuse a json_tool_call user tool, keep Vertex error headers
* fix(vertex_ai): keep httpx.HTTPStatusError on Vertex batch create errors and apply deployment chat params to Claude batch rows
* fix(vertex_ai): keep Vertex's status code on batch create errors through the proxy
* fix(vertex_ai): keep Vertex's error body on batch create errors and require custom_id on Claude lines
* fix(vertex_ai): keep the old batch create error text and Google object names
A failed Vertex batch create keeps httpx's error line first and adds Vertex's body after it, and logs the status and body again. Only Claude object names keep the @ version, so Google models are stored under the same names as before
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_batch_line_items.py | ||
| test_batch_utils.py | ||
| test_main.py | ||
| test_responses_batch_cost.py | ||