fix(vertex_ai/agent_engine): never surface finish_reason on function-call chunks
Some checks are pending
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run

Setting finish_reason='tool_calls' on intermediate function-call chunks
also closes the downstream stream wrapper (handle_openai_chat_completion_chunk
treats any non-None finish_reason as is_finished=True), dropping later
chunks including the final text reply. Only surface finish_reason when
the chunk has user-facing text. This also keeps the priority order between
finish_reason and delta_kwargs consistent (both gate on text).
This commit is contained in:
Cursor Agent 2026-05-11 20:00:28 +00:00
parent f1e7ee2bc1
commit bb8c02d467
No known key found for this signature in database
2 changed files with 10 additions and 12 deletions

View file

@ -90,21 +90,19 @@ class VertexAgentEngineResponseIterator(BaseModelResponseIterator):
end-of-response signal.
We therefore only surface ``finish_reason`` when the chunk has
user-facing content (text or tool_calls). Otherwise downstream stream
handling closes the stream after the first inner action and the actual
response is dropped (see issue #19121).
user-facing text content. Function-call and thought-only chunks must
keep ``finish_reason=None`` so the downstream stream wrapper does not
close the stream after the first inner action and drop the actual
response (see issue #19121).
"""
text, tool_calls = self._extract_parts_from_chunk(chunk)
finish_reason: Optional[str] = None
raw_finish_reason = chunk.get("finish_reason")
if raw_finish_reason:
if tool_calls:
finish_reason = "tool_calls"
elif text is not None:
finish_reason = (
"stop" if raw_finish_reason == "STOP" else raw_finish_reason.lower()
)
if raw_finish_reason and text is not None:
finish_reason = (
"stop" if raw_finish_reason == "STOP" else raw_finish_reason.lower()
)
usage = None
usage_metadata = chunk.get("usage_metadata") or {}

View file

@ -155,7 +155,7 @@ class TestVertexAgentEngineChunkParser:
result = self._iterator().chunk_parser(chunk)
assert result.choices[0].finish_reason == "tool_calls"
assert result.choices[0].finish_reason is None
tool_calls = result.choices[0].delta.tool_calls
assert tool_calls is not None and len(tool_calls) == 1
assert tool_calls[0]["function"]["name"] == "transfer_to_agent"
@ -207,7 +207,7 @@ class TestVertexAgentEngineChunkParser:
result = self._iterator().chunk_parser(chunk)
assert result.choices[0].finish_reason == "tool_calls"
assert result.choices[0].finish_reason is None
tool_calls = result.choices[0].delta.tool_calls
assert tool_calls is not None and len(tool_calls) == 1
assert tool_calls[0]["function"]["name"] == "list_cases"