mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-11 22:51:28 +00:00
test(responses): cancel the streaming response while it is still in flight
test_cancel_streaming_response drained the whole stream and only then called cancel, by which point the response had finished and the cancel was expected to fail. The test passed only because the surrounding except matched the literal string "Cannot cancel a completed response", which is upstream OpenAI's wording, not ours: it appears nowhere in this repo. Any change to that text, a different status code, or the background job still running flipped the result. The test also never exercised cancellation, and its only positive assertion was hasattr(cancel_response, "id"). Break out of the stream at the first chunk carrying a response id and cancel there, with a prompt long enough that the response cannot have completed in the meantime. Both proxy cancel paths, the polling handler and the provider passthrough, settle on status "cancelled", so assert that and the returned id rather than a provider error string.
This commit is contained in:
parent
fb4b1e728a
commit
8ac704a855
1 changed files with 24 additions and 30 deletions
|
|
@ -154,42 +154,36 @@ def test_cancel_response():
|
|||
|
||||
|
||||
def test_cancel_streaming_response():
|
||||
try:
|
||||
client = get_test_client()
|
||||
from litellm.types.llms.openai import ResponsesAPIResponse
|
||||
"""Cancel a background streaming response while it is still generating.
|
||||
|
||||
stream = client.responses.create(
|
||||
model="gpt-5.5",
|
||||
input="just respond with the word 'ping'",
|
||||
stream=True,
|
||||
background=True,
|
||||
)
|
||||
The prompt is deliberately long-running and the stream is abandoned at the first
|
||||
chunk carrying a response id, so the response is provably still in flight when the
|
||||
cancel lands. Draining the stream first would finish the response, making the cancel
|
||||
fail and leaving nothing but the provider's error wording to assert on.
|
||||
"""
|
||||
client = get_test_client()
|
||||
|
||||
collected_chunks = []
|
||||
with client.responses.create(
|
||||
model="gpt-5.5",
|
||||
input="write a 2000 word essay on the history of the printing press",
|
||||
stream=True,
|
||||
background=True,
|
||||
) as stream:
|
||||
chunk_count = 0
|
||||
response_id = None
|
||||
for chunk in stream:
|
||||
print("stream chunk=", chunk)
|
||||
collected_chunks.append(chunk)
|
||||
# Extract response ID from the first chunk that has it
|
||||
if (
|
||||
response_id is None
|
||||
and hasattr(chunk, "response")
|
||||
and hasattr(chunk.response, "id")
|
||||
):
|
||||
response_id = chunk.response.id
|
||||
chunk_count += 1
|
||||
response_id = getattr(getattr(chunk, "response", None), "id", None)
|
||||
if response_id is not None:
|
||||
break
|
||||
|
||||
assert len(collected_chunks) > 0
|
||||
assert chunk_count > 0, "stream produced no chunks"
|
||||
assert response_id is not None, "no streamed chunk carried a response id to cancel"
|
||||
|
||||
# cancel the response if we got a response ID
|
||||
if response_id:
|
||||
cancel_response = client.responses.cancel(response_id)
|
||||
print("CANCEL streaming response=", cancel_response)
|
||||
assert hasattr(cancel_response, "id")
|
||||
except Exception as e:
|
||||
if "Cannot cancel a completed response" in str(e):
|
||||
pass
|
||||
else:
|
||||
raise e
|
||||
cancel_response = client.responses.cancel(response_id)
|
||||
print("CANCEL streaming response=", cancel_response)
|
||||
assert cancel_response.id == response_id
|
||||
assert cancel_response.status == "cancelled"
|
||||
|
||||
|
||||
def test_cancel_invalid_response_id():
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue