supermemory/packages/tools/src/tools-shared.test.ts
Dhravya d0f53b0d64
feat(tools): SDK-level cross-source memory deduplication (#1531)
## Stack Context

This stack moves memory deduplication **out of the playground UI and into the SDKs themselves**, so every integration injects a single, deduplicated, self-replacing memory block. Three PRs:

1. **`sdk-dedup/tools-ts`** (this PR) — TypeScript SDK core + integrations
2. `sdk-dedup/python` — Python SDKs
3. `sdk-dedup/playground` — playground debug view reflects the SDK-owned block

## What?

Move profile deduplication into the SDK middleware for the TypeScript tools package.

- Facts are normalized (strip leading `[YYYY-MM-DD]`, trim, collapse whitespace, casefold) and deduplicated in **`static > dynamic > search`** priority within a single request.
- The result is injected as one **owned `<supermemory>` block** that *replaces* the previous block instead of accumulating a new one each turn.
- Dedup is **mode-aware**: in query mode, search results are not dropped against a profile that isn't being injected.
- Deduplication is **request-local** — no global/browser `Set`. Safe for multiple users, concurrent requests, and Cloudflare Worker isolates.

Covers AI SDK, OpenAI (Chat + Responses), Mastra, and VoltAgent. New `shared/memory-context.ts` owns the block-replacement logic.

## Why?

The earlier "conversation-scoped deduplication" was only a playground browser `Set` — a UI debug affordance that did not change what the SDK sent to the model, and would have been unsafe as server-side global state. Real cross-source dedup belongs in the SDK, applied fresh per stateless model request.

## Testing

- `bun run test` in `packages/tools`: 145 passed (the one failing suite, `claude-memory.test.ts`, is a pre-existing broken import unrelated to this change).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes how system prompts and instructions are built across all TypeScript integrations; behavior is well-covered by unit tests but incorrect strip/replace logic could drop or duplicate context in production prompts.
>
> **Overview**
> Moves **cross-source memory deduplication** and **owned prompt injection** into `@supermemory/tools` so every integration sends one deduplicated memory block per request instead of growing context each turn.
>
> **Deduplication:** Facts are normalized via `normalizeMemoryFact` (strip `[YYYY-MM-DD]`, trim, collapse whitespace, lowercase) and deduplicated with **static → dynamic → search** priority. `deduplicateMemoriesForMode` keeps search hits in **query** mode when the profile is not injected.
>
> **Owned `<supermemory>` block:** New `shared/memory-context.ts` wraps memories in `<supermemory context="user-memories" readonly>`, strips stale blocks, and **replaces** prior SDK context while preserving caller system instructions. Applied in AI SDK (`injectMemoriesIntoParams`), OpenAI Chat/Responses middleware, Mastra input processor (`wrapMemoryContext`), and VoltAgent hooks.
>
> **Tests:** Unit coverage for block replacement (with-supermemory, OpenAI, VoltAgent), Mastra wrapper tag assertion, normalized dedup variants, and concurrent `containerTag` isolation.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 2fa2e0d85c. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-09-01 06:10:36 +00:00

132 lines
4.2 KiB
TypeScript

import { describe, expect, it } from "vitest"
import {
DEFAULT_VALUES,
SEARCH_LIMIT_BOUNDS,
clampSearchLimit,
deduplicateMemoriesForMode,
getContainerTags,
} from "./tools-shared"
describe("clampSearchLimit", () => {
it("keeps in-range integers", () => {
expect(clampSearchLimit(7)).toBe(7)
})
it("clamps into SEARCH_LIMIT_BOUNDS", () => {
expect(clampSearchLimit(0)).toBe(SEARCH_LIMIT_BOUNDS.min)
expect(clampSearchLimit(-5)).toBe(SEARCH_LIMIT_BOUNDS.min)
expect(clampSearchLimit(999)).toBe(SEARCH_LIMIT_BOUNDS.max)
})
it("floors fractional values and coerces numeric strings", () => {
expect(clampSearchLimit(7.9)).toBe(7)
expect(clampSearchLimit("12")).toBe(12)
})
it("falls back to the default for non-numeric input", () => {
expect(clampSearchLimit("lots")).toBe(DEFAULT_VALUES.limit)
expect(clampSearchLimit(undefined)).toBe(DEFAULT_VALUES.limit)
expect(clampSearchLimit(Number.POSITIVE_INFINITY)).toBe(
DEFAULT_VALUES.limit,
)
})
})
describe("getContainerTags", () => {
it("uses the default project when no config is provided", () => {
expect(getContainerTags()).toEqual(["sm_project_default"])
})
it("converts projectId into a project container tag", () => {
expect(getContainerTags({ projectId: "abc" })).toEqual(["sm_project_abc"])
})
it("uses explicit container tags", () => {
expect(getContainerTags({ containerTags: ["tag-a", "tag-b"] })).toEqual([
"tag-a",
"tag-b",
])
})
it("rejects config with both projectId and containerTags", () => {
expect(() =>
getContainerTags({
projectId: "abc",
containerTags: ["tag-a"],
}),
).toThrow("either projectId or containerTags")
})
})
describe("deduplicateMemoriesForMode", () => {
// The profile is not injected in "query" mode, so a memory that is both a
// profile fact and a search hit must survive in the search results —
// otherwise it is dropped from the prompt entirely.
it("keeps a search result that duplicates a profile memory in query mode", () => {
const deduplicated = deduplicateMemoriesForMode("query", {
static: [{ memory: "User is allergic to peanuts" }],
dynamic: [],
searchResults: [{ memory: "User is allergic to peanuts" }],
})
expect(deduplicated.searchResults).toEqual(["User is allergic to peanuts"])
expect(deduplicated.static).toEqual([])
expect(deduplicated.dynamic).toEqual([])
})
it("still deduplicates within the search results in query mode", () => {
const deduplicated = deduplicateMemoriesForMode("query", {
static: [],
dynamic: [],
searchResults: [
{ memory: "User likes TypeScript" },
"User likes TypeScript",
],
})
expect(deduplicated.searchResults).toEqual(["User likes TypeScript"])
})
it("deduplicates normalized fact variants within and across sources", () => {
const deduplicated = deduplicateMemoriesForMode("full", {
static: [
{ memory: "User likes TypeScript" },
{ memory: " user likes typescript " },
],
dynamic: [{ memory: "[2026-08-10] USER LIKES TYPESCRIPT" }],
searchResults: [{ memory: "User prefers async/await" }],
})
expect(deduplicated).toEqual({
static: ["User likes TypeScript"],
dynamic: [],
searchResults: ["User prefers async/await"],
})
})
it("deduplicates search results against the profile in full mode", () => {
const deduplicated = deduplicateMemoriesForMode("full", {
static: [{ memory: "User is allergic to peanuts" }],
dynamic: [{ memory: "User is shipping a release today" }],
searchResults: [
{ memory: "User is allergic to peanuts" },
{ memory: "User prefers async/await" },
],
})
expect(deduplicated.static).toEqual(["User is allergic to peanuts"])
expect(deduplicated.dynamic).toEqual(["User is shipping a release today"])
expect(deduplicated.searchResults).toEqual(["User prefers async/await"])
})
it("deduplicates search results against the profile in profile mode", () => {
const deduplicated = deduplicateMemoriesForMode("profile", {
static: [{ memory: "User is allergic to peanuts" }],
dynamic: [],
searchResults: [{ memory: "User is allergic to peanuts" }],
})
expect(deduplicated.static).toEqual(["User is allergic to peanuts"])
expect(deduplicated.searchResults).toEqual([])
})
})