mirror of
https://github.com/supermemoryai/supermemory.git
synced 2026-10-20 05:06:31 +00:00
d84f0bf0dd
Fixes SUPERMEMORY-BACKEND-JMN (17,259 events, 598 users, 111 days). ## Root cause When Cloudflare resets a Durable Object isolate during an active ConnectorsWorkflow sync run, the DO RPC returns an error with `durableObjectReset: true` and/or the message "Aborting engine: Grace period complete". The previous `isRetryableDurableObjectError` check did not match these signals, so the workflow treated the reset as a non-retryable defect and permanently marked the sync as failed. ## Changes ### apps/api/src/lib/do-retry.ts — expanded matcher (core fix) - Added `durableObjectReset: true` check on the error and its cause chain - Added message checks for "Grace period complete", "destroyed", and "Network connection lost" - Exported `DurableObjectResetError` typed class and `liftDurableObjectResetDefect` helper for use in the retry wrapper ### apps/api/src/workflows/connectors/steps/do-retry-wrapper.ts — new - `withDORetry` wraps any Effect with 4 retries + exponential backoff (100 ms base, factor 2, jitter) for DO lifecycle reset errors - Converts DO reset defects into typed `DurableObjectResetError` failures so the retry predicate can match them - Mirrors the retry count/backoff of the existing mcpMountHandler pattern (out of scope for this fix) ### apps/api/src/workflows/connectors/plan-entitlement.ts — new - `coerceConnectorWorkflowDefect` classifies DO reset errors as kind "do_lifecycle_reset" rather than "unhandled_defect" so Sentry and dashboards can distinguish exhausted retries from genuine programming errors ### apps/api/src/workflows/connectors/workflow.ts — new - `runConnectorsWorkflow` wraps every DO-touching step with `withDORetry` so lifecycle resets are transparently retried before reaching the permanent-failure path ### Tests - 27 unit tests for `isRetryableDurableObjectError`, `liftDurableObjectResetDefect`, and `DurableObjectResetError` - 6 unit tests for `coerceConnectorWorkflowDefect` - All 33 tests pass; `tsc --noEmit` is clean Co-authored-by: Dhravya Shah <dhravya@supermemory.com> |
||
|---|---|---|
| .. | ||
| src | ||
| package.json | ||
| pnpm-lock.yaml | ||
| tsconfig.json | ||
| vitest.config.ts | ||