mirror of
https://github.com/open-webui/open-webui.git
synced 2026-11-10 08:47:14 +00:00
a5fdd329db
With WEBSOCKET_MANAGER=redis every emit is published on one shared channel and every instance decodes every message: a 16 instance fleet decodes each streamed token delta 16 times and 15 discard it, and streams whose user closed the tab keep publishing every delta until they end. py-spy across a loaded fleet (16 instances, ~4000 users) puts ~31% of all active CPU samples in the pubsub listener parse chain, the largest bucket, growing with instance count. WEBSOCKET_REDIS_ROUTING=true routes instead: instances register room membership in Redis and emits go only to instances holding members. Rooms empty fleet-wide are dropped before serialization, local-only delivery skips Redis and sio.call acks return on the caller's channel. Lookups use a short-TTL cache invalidated on membership changes; any registry uncertainty (wipe, failover, prune, failing writes, suspend) falls back to stock broadcast, so degraded Redis never loses messages hub mode would deliver. Composes with the new delta streaming: deltas cut bytes per message, routing cuts decoders per message. Default off, byte-identical to today; the fleet must switch together with a full restart. Verified end to end with two instances: cross-instance delta streams deliver with zero broadcast-channel traffic. Ref #28173. |
||
|---|---|---|
| .. | ||
| main.py | ||
| redis_router.py | ||
| utils.py | ||