mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
fix(vertex_ai/agent_engine): never surface finish_reason on function-call chunks
Some checks are pending
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
Some checks are pending
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
Setting finish_reason='tool_calls' on intermediate function-call chunks also closes the downstream stream wrapper (handle_openai_chat_completion_chunk treats any non-None finish_reason as is_finished=True), dropping later chunks including the final text reply. Only surface finish_reason when the chunk has user-facing text. This also keeps the priority order between finish_reason and delta_kwargs consistent (both gate on text).
This commit is contained in:
parent
f1e7ee2bc1
commit
bb8c02d467
2 changed files with 10 additions and 12 deletions
|
|
@ -90,21 +90,19 @@ class VertexAgentEngineResponseIterator(BaseModelResponseIterator):
|
|||
end-of-response signal.
|
||||
|
||||
We therefore only surface ``finish_reason`` when the chunk has
|
||||
user-facing content (text or tool_calls). Otherwise downstream stream
|
||||
handling closes the stream after the first inner action and the actual
|
||||
response is dropped (see issue #19121).
|
||||
user-facing text content. Function-call and thought-only chunks must
|
||||
keep ``finish_reason=None`` so the downstream stream wrapper does not
|
||||
close the stream after the first inner action and drop the actual
|
||||
response (see issue #19121).
|
||||
"""
|
||||
text, tool_calls = self._extract_parts_from_chunk(chunk)
|
||||
|
||||
finish_reason: Optional[str] = None
|
||||
raw_finish_reason = chunk.get("finish_reason")
|
||||
if raw_finish_reason:
|
||||
if tool_calls:
|
||||
finish_reason = "tool_calls"
|
||||
elif text is not None:
|
||||
finish_reason = (
|
||||
"stop" if raw_finish_reason == "STOP" else raw_finish_reason.lower()
|
||||
)
|
||||
if raw_finish_reason and text is not None:
|
||||
finish_reason = (
|
||||
"stop" if raw_finish_reason == "STOP" else raw_finish_reason.lower()
|
||||
)
|
||||
|
||||
usage = None
|
||||
usage_metadata = chunk.get("usage_metadata") or {}
|
||||
|
|
|
|||
|
|
@ -155,7 +155,7 @@ class TestVertexAgentEngineChunkParser:
|
|||
|
||||
result = self._iterator().chunk_parser(chunk)
|
||||
|
||||
assert result.choices[0].finish_reason == "tool_calls"
|
||||
assert result.choices[0].finish_reason is None
|
||||
tool_calls = result.choices[0].delta.tool_calls
|
||||
assert tool_calls is not None and len(tool_calls) == 1
|
||||
assert tool_calls[0]["function"]["name"] == "transfer_to_agent"
|
||||
|
|
@ -207,7 +207,7 @@ class TestVertexAgentEngineChunkParser:
|
|||
|
||||
result = self._iterator().chunk_parser(chunk)
|
||||
|
||||
assert result.choices[0].finish_reason == "tool_calls"
|
||||
assert result.choices[0].finish_reason is None
|
||||
tool_calls = result.choices[0].delta.tool_calls
|
||||
assert tool_calls is not None and len(tool_calls) == 1
|
||||
assert tool_calls[0]["function"]["name"] == "list_cases"
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue