fix(stream): refresh seq TTL with stream TTL, tighten hot-path timeout

- seq key now gets EXPIRE on the same periodic cadence as the stream
  key, not just on the first INCR. Prevents INCR restarting at 1 on
  responses longer than RESUME_STREAM_TTL_SEC (1h), which would have
  corrupted replay ordering.
- Hot-path Redis timeout trimmed 200ms → 100ms. Halves worst-case
  stall under degraded Redis (now ~200ms max per frame if both INCR
  and append time out); healthy Redis (<1ms) is unaffected.

Not addressed: the fundamental 2-RTT-per-frame cost of separate INCR
and XADD. Collapsing them would require either a Lua script (cluster
hash-tag concerns) or switching to Redis stream IDs as the seq cursor
(invasive protocol change). Leaving the structure intact — if
profiling shows real stalls in prod, stream-ID-as-seq is the cleaner
follow-up.
This commit is contained in:
Claude 2026-04-14 22:44:59 +00:00
parent 8c377fd774
commit 7b30714941
No known key found for this signature in database

View file

@ -181,7 +181,7 @@ RESUME_STREAM_DONE_TTL_SEC = 30
RESUME_STREAM_TTL_REFRESH_EVERY = 64
# Tight upper bound on any Redis call in the streaming hot path so a
# slow Redis degrades resume but can't stall live token delivery.
RESUME_STREAM_REDIS_TIMEOUT_SEC = 0.2
RESUME_STREAM_REDIS_TIMEOUT_SEC = 0.1
def _stream_key(user_id: str, message_id: str) -> str:
@ -233,6 +233,7 @@ async def _stream_log_append(user_id: str, message_id: str, envelope: dict, seq:
try:
refresh_ttl = (seq == 1) or (seq % RESUME_STREAM_TTL_REFRESH_EVERY == 0)
key = _stream_key(user_id, message_id)
seq_key = _stream_seq_key(user_id, message_id)
pipe = REDIS.pipeline(transaction=False)
pipe.xadd(
key,
@ -241,7 +242,11 @@ async def _stream_log_append(user_id: str, message_id: str, envelope: dict, seq:
approximate=True,
)
if refresh_ttl:
# Refresh both keys together — without this the seq counter
# can expire mid-stream on responses longer than the TTL and
# INCR restarts at 1, corrupting replay ordering.
pipe.expire(key, RESUME_STREAM_TTL_SEC)
pipe.expire(seq_key, RESUME_STREAM_TTL_SEC)
await asyncio.wait_for(
pipe.execute(), timeout=RESUME_STREAM_REDIS_TIMEOUT_SEC
)