refactor(router): validate stream_chunk_builder result instead of casting

The merge with litellm_internal_staging added the type-discipline (LIT) gate,
which flagged the cast(ModelResponse, result) in _collect_stream_with_ttft_timeout
(LIT006). stream_chunk_builder returns Optional[Union[ModelResponse,
TextCompletionResponse]], so replace the None-check + cast with an isinstance
check that narrows to ModelResponse and raises on the empty/non-chat cases. No
behavior change for the chat path; the ttft tests still pass.
This commit is contained in:
Nathan Price 2026-07-08 09:50:40 -05:00
parent d461f1a24b
commit acbf15c8d7

View file

@ -3048,14 +3048,14 @@ class Router:
result = await asyncio.to_thread(
stream_chunk_builder, chunks, messages=messages
)
if result is None:
if not isinstance(result, ModelResponse):
raise litellm.APIError(
status_code=500,
message="stream_chunk_builder returned None: provider returned an empty stream",
message="stream_chunk_builder did not return a ModelResponse: provider returned an empty or non-chat stream",
llm_provider="",
model="",
)
return cast(ModelResponse, result)
return result
async def _collect_until_first_token(
self,