mirror of
https://github.com/supermemoryai/supermemory.git
synced 2026-09-05 08:06:19 +00:00
## Stack Context
This stack moves memory deduplication **out of the playground UI and into the SDKs themselves**, so every integration injects a single, deduplicated, self-replacing memory block. Three PRs:
1. **`sdk-dedup/tools-ts`** (this PR) — TypeScript SDK core + integrations
2. `sdk-dedup/python` — Python SDKs
3. `sdk-dedup/playground` — playground debug view reflects the SDK-owned block
## What?
Move profile deduplication into the SDK middleware for the TypeScript tools package.
- Facts are normalized (strip leading `[YYYY-MM-DD]`, trim, collapse whitespace, casefold) and deduplicated in **`static > dynamic > search`** priority within a single request.
- The result is injected as one **owned `<supermemory>` block** that *replaces* the previous block instead of accumulating a new one each turn.
- Dedup is **mode-aware**: in query mode, search results are not dropped against a profile that isn't being injected.
- Deduplication is **request-local** — no global/browser `Set`. Safe for multiple users, concurrent requests, and Cloudflare Worker isolates.
Covers AI SDK, OpenAI (Chat + Responses), Mastra, and VoltAgent. New `shared/memory-context.ts` owns the block-replacement logic.
## Why?
The earlier "conversation-scoped deduplication" was only a playground browser `Set` — a UI debug affordance that did not change what the SDK sent to the model, and would have been unsafe as server-side global state. Real cross-source dedup belongs in the SDK, applied fresh per stateless model request.
## Testing
- `bun run test` in `packages/tools`: 145 passed (the one failing suite, `claude-memory.test.ts`, is a pre-existing broken import unrelated to this change).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Changes how system prompts and instructions are built across all TypeScript integrations; behavior is well-covered by unit tests but incorrect strip/replace logic could drop or duplicate context in production prompts.
>
> **Overview**
> Moves **cross-source memory deduplication** and **owned prompt injection** into `@supermemory/tools` so every integration sends one deduplicated memory block per request instead of growing context each turn.
>
> **Deduplication:** Facts are normalized via `normalizeMemoryFact` (strip `[YYYY-MM-DD]`, trim, collapse whitespace, lowercase) and deduplicated with **static → dynamic → search** priority. `deduplicateMemoriesForMode` keeps search hits in **query** mode when the profile is not injected.
>
> **Owned `<supermemory>` block:** New `shared/memory-context.ts` wraps memories in `<supermemory context="user-memories" readonly>`, strips stale blocks, and **replaces** prior SDK context while preserving caller system instructions. Applied in AI SDK (`injectMemoriesIntoParams`), OpenAI Chat/Responses middleware, Mastra input processor (`wrapMemoryContext`), and VoltAgent hooks.
>
> **Tests:** Unit coverage for block replacement (with-supermemory, OpenAI, VoltAgent), Mastra wrapper tag assertion, normalized dedup variants, and concurrent `containerTag` isolation.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 2fa2e0d85c. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
132 lines
4.2 KiB
TypeScript
132 lines
4.2 KiB
TypeScript
import { describe, expect, it } from "vitest"
|
|
import {
|
|
DEFAULT_VALUES,
|
|
SEARCH_LIMIT_BOUNDS,
|
|
clampSearchLimit,
|
|
deduplicateMemoriesForMode,
|
|
getContainerTags,
|
|
} from "./tools-shared"
|
|
|
|
describe("clampSearchLimit", () => {
|
|
it("keeps in-range integers", () => {
|
|
expect(clampSearchLimit(7)).toBe(7)
|
|
})
|
|
|
|
it("clamps into SEARCH_LIMIT_BOUNDS", () => {
|
|
expect(clampSearchLimit(0)).toBe(SEARCH_LIMIT_BOUNDS.min)
|
|
expect(clampSearchLimit(-5)).toBe(SEARCH_LIMIT_BOUNDS.min)
|
|
expect(clampSearchLimit(999)).toBe(SEARCH_LIMIT_BOUNDS.max)
|
|
})
|
|
|
|
it("floors fractional values and coerces numeric strings", () => {
|
|
expect(clampSearchLimit(7.9)).toBe(7)
|
|
expect(clampSearchLimit("12")).toBe(12)
|
|
})
|
|
|
|
it("falls back to the default for non-numeric input", () => {
|
|
expect(clampSearchLimit("lots")).toBe(DEFAULT_VALUES.limit)
|
|
expect(clampSearchLimit(undefined)).toBe(DEFAULT_VALUES.limit)
|
|
expect(clampSearchLimit(Number.POSITIVE_INFINITY)).toBe(
|
|
DEFAULT_VALUES.limit,
|
|
)
|
|
})
|
|
})
|
|
|
|
describe("getContainerTags", () => {
|
|
it("uses the default project when no config is provided", () => {
|
|
expect(getContainerTags()).toEqual(["sm_project_default"])
|
|
})
|
|
|
|
it("converts projectId into a project container tag", () => {
|
|
expect(getContainerTags({ projectId: "abc" })).toEqual(["sm_project_abc"])
|
|
})
|
|
|
|
it("uses explicit container tags", () => {
|
|
expect(getContainerTags({ containerTags: ["tag-a", "tag-b"] })).toEqual([
|
|
"tag-a",
|
|
"tag-b",
|
|
])
|
|
})
|
|
|
|
it("rejects config with both projectId and containerTags", () => {
|
|
expect(() =>
|
|
getContainerTags({
|
|
projectId: "abc",
|
|
containerTags: ["tag-a"],
|
|
}),
|
|
).toThrow("either projectId or containerTags")
|
|
})
|
|
})
|
|
|
|
describe("deduplicateMemoriesForMode", () => {
|
|
// The profile is not injected in "query" mode, so a memory that is both a
|
|
// profile fact and a search hit must survive in the search results —
|
|
// otherwise it is dropped from the prompt entirely.
|
|
it("keeps a search result that duplicates a profile memory in query mode", () => {
|
|
const deduplicated = deduplicateMemoriesForMode("query", {
|
|
static: [{ memory: "User is allergic to peanuts" }],
|
|
dynamic: [],
|
|
searchResults: [{ memory: "User is allergic to peanuts" }],
|
|
})
|
|
|
|
expect(deduplicated.searchResults).toEqual(["User is allergic to peanuts"])
|
|
expect(deduplicated.static).toEqual([])
|
|
expect(deduplicated.dynamic).toEqual([])
|
|
})
|
|
|
|
it("still deduplicates within the search results in query mode", () => {
|
|
const deduplicated = deduplicateMemoriesForMode("query", {
|
|
static: [],
|
|
dynamic: [],
|
|
searchResults: [
|
|
{ memory: "User likes TypeScript" },
|
|
"User likes TypeScript",
|
|
],
|
|
})
|
|
|
|
expect(deduplicated.searchResults).toEqual(["User likes TypeScript"])
|
|
})
|
|
|
|
it("deduplicates normalized fact variants within and across sources", () => {
|
|
const deduplicated = deduplicateMemoriesForMode("full", {
|
|
static: [
|
|
{ memory: "User likes TypeScript" },
|
|
{ memory: " user likes typescript " },
|
|
],
|
|
dynamic: [{ memory: "[2026-08-10] USER LIKES TYPESCRIPT" }],
|
|
searchResults: [{ memory: "User prefers async/await" }],
|
|
})
|
|
|
|
expect(deduplicated).toEqual({
|
|
static: ["User likes TypeScript"],
|
|
dynamic: [],
|
|
searchResults: ["User prefers async/await"],
|
|
})
|
|
})
|
|
|
|
it("deduplicates search results against the profile in full mode", () => {
|
|
const deduplicated = deduplicateMemoriesForMode("full", {
|
|
static: [{ memory: "User is allergic to peanuts" }],
|
|
dynamic: [{ memory: "User is shipping a release today" }],
|
|
searchResults: [
|
|
{ memory: "User is allergic to peanuts" },
|
|
{ memory: "User prefers async/await" },
|
|
],
|
|
})
|
|
|
|
expect(deduplicated.static).toEqual(["User is allergic to peanuts"])
|
|
expect(deduplicated.dynamic).toEqual(["User is shipping a release today"])
|
|
expect(deduplicated.searchResults).toEqual(["User prefers async/await"])
|
|
})
|
|
|
|
it("deduplicates search results against the profile in profile mode", () => {
|
|
const deduplicated = deduplicateMemoriesForMode("profile", {
|
|
static: [{ memory: "User is allergic to peanuts" }],
|
|
dynamic: [],
|
|
searchResults: [{ memory: "User is allergic to peanuts" }],
|
|
})
|
|
|
|
expect(deduplicated.static).toEqual(["User is allergic to peanuts"])
|
|
expect(deduplicated.searchResults).toEqual([])
|
|
})
|
|
})
|