scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-11 22:51:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 124

fix: increase chunk_size to 8 KB for optimal latency

Browse source
This commit is contained in:
AlexsanderHamir 2025-11-17 12:55:58 -08:00
parent 697cb09060
commit f1895265e6
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
litellm/proxy/proxy_server.py
View file

@ -5236,7 +5236,7 @@ async def moderations(
async def _audio_speech_chunk_generator(
_response: HttpxBinaryResponseContent,
) -> AsyncGenerator[bytes, None]:
_generator = await _response.aiter_bytes(chunk_size=4096)
_generator = await _response.aiter_bytes(chunk_size=8192)
async for chunk in _generator:
yield chunk