litellm/tests/unit/proxy/response_api_endpoints
devin-ai-integration[bot] 4f49f88172
fix(proxy): keep idle responses websockets open until a configurable session limit (#44433)
* add config-driven bound to /responses websocket connection

* test(proxy): cover responses websocket session limit and idle first frame

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ci): sync API schema and Ruff formatting

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): close responses websocket client before upstream cleanup at session limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): assert recorded websocket outcomes and simplify session reaping

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): move responses websocket session limit timing coverage to integration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): run provider-close checks off the event loop

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover handshake auth, subprotocol, DB override and worker kill for the responses websocket session limit

Adds the audit cells for the configurable session limit: a rejected bearer fails the handshake with 403, a requested Sec-WebSocket-Protocol is echoed back, a DB override through /config/field/update caps new sessions on every worker without a restart while sessions accepted before it keep their limit, deleting the override restores the default, out-of-range and wrong-type updates answer 400 and leave the stored row alone, and SIGKILLing the worker holding idle sockets drops only its sockets while the proxy keeps serving

* test(integration): open override sockets until both proxy workers hold one

uvicorn workers share one listening socket and a worker that wakes first accepts a whole simultaneous burst, so eight sockets opened at once can all land on one worker. The DB override cell now opens its sockets one at a time, alternating the two paths, until at least eight are open and every worker holds one, bounded at forty

---------

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-07 19:26:30 -07:00
..
__init__.py test(proxy): move utils, agent_endpoints and endpoint tests into tests/unit/proxy (#44006) 2026-10-01 11:06:42 -07:00
test_endpoints.py fix(proxy): keep idle responses websockets open until a configurable session limit (#44433) 2026-10-07 19:26:30 -07:00