From c6fd7289812fdac1f770ea55c2db13fa647ad885 Mon Sep 17 00:00:00 2001 From: Aswin-Ram-K Date: Wed, 23 Sep 2026 03:18:51 -0500 Subject: [PATCH] docs(api): add V3/V4 to V5 migration guide ## What and why Add an agent-oriented V3/V4 to V5 migration guide covering document ingestion and updates, content management, search, profiles, memory forgetting, namespaces, organization settings, typed filters, and rollout verification. Shared snippets keep the comprehensive guide and the focused topic pages consistent, following the existing `/snippets/*.mdx` import convention used elsewhere in the docs. ```mermaid flowchart LR Legacy[Legacy integration inventory] --> Mapping[Domain migration guidance] Mapping --> V5[V5 requests and response readers] V5 --> Verify[Side-by-side verification and rollout] ``` ## Grounding Every old-vs-new claim traces to code in this repository: - `packages/validation/api.ts` - `SearchRequestSchema`, `Searchv4RequestSchema`, `ListMemoriesQuerySchema`, `MemoryUpdateSchema`, `BulkDeleteMemoriesSchema`, `ContainerTagListTypeSchema`, `SearchFiltersSchema`. - `packages/validation/schemas.ts` - `MemoryEntrySchema`, `OrganizationSettingsSchema`, `MemoryRelationEnum`. - `packages/tools/src/shared/memory-client.ts` and `packages/tools/src/shared/types.ts` - the `/v4/profile` request and `profile.static` / `profile.dynamic` / `profile.buckets` reader. - `apps/mcp/src/server/client/index.ts` - live `/v3/container-tags/list`, `/v4/memories/list`, and `/v3/documents/file` call sites. ## Notable findings documented - `SearchFiltersSchema` is `z.array(z.unknown())` behind a `// TODO: Improve filter schema` comment, so legacy conditions were never validated at the edge. The typed-filter page documents the mechanical conversion and calls out the numeric-string-to-JSON-number trap that a straight rename would miss. - `OrganizationSettingsSchema` carries connector credentials that are absent from the public V5 `/organization` response; the page warns against reading them from `GET /v3/settings`. - Operations with no V5 replacement are listed explicitly rather than given invented substitutes. ## Validation - `docs.json` parses as JSON; all 11 new nav entries resolve to authored pages, and every pre-existing nav entry still resolves. - All 12 snippet imports and every relative/absolute link across the 23 new files resolve to real targets. - Fences, braces, and JSX component tags balance across all new files; frontmatter parses as YAML. - `git diff --check` passes. ## Impact Documentation only. No runtime, schema, or SDK behavior changes. --- apps/docs/docs.json | 17 +++ apps/docs/migration/api-v5-document-reads.mdx | 11 ++ .../migration/api-v5-document-updates.mdx | 11 ++ .../docs/migration/api-v5-document-writes.mdx | 11 ++ apps/docs/migration/api-v5-filters.mdx | 11 ++ .../migration/api-v5-memory-forgetting.mdx | 11 ++ apps/docs/migration/api-v5-organization.mdx | 11 ++ apps/docs/migration/api-v5-profiles.mdx | 11 ++ apps/docs/migration/api-v5-recall.mdx | 11 ++ apps/docs/migration/api-v5-rollout.mdx | 11 ++ apps/docs/migration/api-v5-settings.mdx | 11 ++ apps/docs/migration/api-v5.mdx | 63 ++++++++++ apps/docs/snippets/api-v5-completion.mdx | 29 +++++ .../snippets/api-v5-content-management.mdx | 105 +++++++++++++++++ .../snippets/api-v5-document-ingestion.mdx | 91 +++++++++++++++ .../docs/snippets/api-v5-document-updates.mdx | 95 +++++++++++++++ apps/docs/snippets/api-v5-filters.mdx | 109 ++++++++++++++++++ .../snippets/api-v5-memory-forgetting.mdx | 93 +++++++++++++++ apps/docs/snippets/api-v5-namespaces.mdx | 86 ++++++++++++++ apps/docs/snippets/api-v5-organization.mdx | 76 ++++++++++++ apps/docs/snippets/api-v5-overview.mdx | 90 +++++++++++++++ apps/docs/snippets/api-v5-profiles.mdx | 95 +++++++++++++++ apps/docs/snippets/api-v5-rollout.mdx | 77 +++++++++++++ apps/docs/snippets/api-v5-search.mdx | 98 ++++++++++++++++ 24 files changed, 1234 insertions(+) create mode 100644 apps/docs/migration/api-v5-document-reads.mdx create mode 100644 apps/docs/migration/api-v5-document-updates.mdx create mode 100644 apps/docs/migration/api-v5-document-writes.mdx create mode 100644 apps/docs/migration/api-v5-filters.mdx create mode 100644 apps/docs/migration/api-v5-memory-forgetting.mdx create mode 100644 apps/docs/migration/api-v5-organization.mdx create mode 100644 apps/docs/migration/api-v5-profiles.mdx create mode 100644 apps/docs/migration/api-v5-recall.mdx create mode 100644 apps/docs/migration/api-v5-rollout.mdx create mode 100644 apps/docs/migration/api-v5-settings.mdx create mode 100644 apps/docs/migration/api-v5.mdx create mode 100644 apps/docs/snippets/api-v5-completion.mdx create mode 100644 apps/docs/snippets/api-v5-content-management.mdx create mode 100644 apps/docs/snippets/api-v5-document-ingestion.mdx create mode 100644 apps/docs/snippets/api-v5-document-updates.mdx create mode 100644 apps/docs/snippets/api-v5-filters.mdx create mode 100644 apps/docs/snippets/api-v5-memory-forgetting.mdx create mode 100644 apps/docs/snippets/api-v5-namespaces.mdx create mode 100644 apps/docs/snippets/api-v5-organization.mdx create mode 100644 apps/docs/snippets/api-v5-overview.mdx create mode 100644 apps/docs/snippets/api-v5-profiles.mdx create mode 100644 apps/docs/snippets/api-v5-rollout.mdx create mode 100644 apps/docs/snippets/api-v5-search.mdx diff --git a/apps/docs/docs.json b/apps/docs/docs.json index 0d41141c..802a8e4e 100644 --- a/apps/docs/docs.json +++ b/apps/docs/docs.json @@ -218,6 +218,23 @@ { "group": "Migration Guides", "pages": [ + { + "group": "V3/V4 to V5", + "icon": "arrow-up-right", + "pages": [ + "migration/api-v5", + "migration/api-v5-document-writes", + "migration/api-v5-document-updates", + "migration/api-v5-document-reads", + "migration/api-v5-recall", + "migration/api-v5-profiles", + "migration/api-v5-memory-forgetting", + "migration/api-v5-settings", + "migration/api-v5-organization", + "migration/api-v5-filters", + "migration/api-v5-rollout" + ] + }, { "group": "From another provider", "icon": "truck", diff --git a/apps/docs/migration/api-v5-document-reads.mdx b/apps/docs/migration/api-v5-document-reads.mdx new file mode 100644 index 00000000..90fca05a --- /dev/null +++ b/apps/docs/migration/api-v5-document-reads.mdx @@ -0,0 +1,11 @@ +--- +title: "Reading and deleting content on V5" +description: "Fetching documents, listing resources, and deleting documents or memories" +sidebarTitle: "Reading content" +--- + +import ContentManagement from "/snippets/api-v5-content-management.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-document-updates.mdx b/apps/docs/migration/api-v5-document-updates.mdx new file mode 100644 index 00000000..c80bf773 --- /dev/null +++ b/apps/docs/migration/api-v5-document-updates.mdx @@ -0,0 +1,11 @@ +--- +title: "Updating documents and files on V5" +description: "Pick deliberately between appending, replacing, and touching only metadata" +sidebarTitle: "Updating documents" +--- + +import DocumentUpdates from "/snippets/api-v5-document-updates.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-document-writes.mdx b/apps/docs/migration/api-v5-document-writes.mdx new file mode 100644 index 00000000..f6a59240 --- /dev/null +++ b/apps/docs/migration/api-v5-document-writes.mdx @@ -0,0 +1,11 @@ +--- +title: "Writing documents on V5" +description: "Single, batch, and file ingestion that keeps legacy append semantics" +sidebarTitle: "Ingesting documents" +--- + +import DocumentIngestion from "/snippets/api-v5-document-ingestion.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-filters.mdx b/apps/docs/migration/api-v5-filters.mdx new file mode 100644 index 00000000..d04858a6 --- /dev/null +++ b/apps/docs/migration/api-v5-filters.mdx @@ -0,0 +1,11 @@ +--- +title: "Metadata filters on V5" +description: "Turning loose legacy filter objects into expressions the server validates" +sidebarTitle: "Filter expressions" +--- + +import Filters from "/snippets/api-v5-filters.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-memory-forgetting.mdx b/apps/docs/migration/api-v5-memory-forgetting.mdx new file mode 100644 index 00000000..684314b5 --- /dev/null +++ b/apps/docs/migration/api-v5-memory-forgetting.mdx @@ -0,0 +1,11 @@ +--- +title: "Forgetting memories on V5" +description: "Exact and semantic deletion now share a single response contract" +sidebarTitle: "Forgetting memories" +--- + +import MemoryForgetting from "/snippets/api-v5-memory-forgetting.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-organization.mdx b/apps/docs/migration/api-v5-organization.mdx new file mode 100644 index 00000000..7c2cb9b8 --- /dev/null +++ b/apps/docs/migration/api-v5-organization.mdx @@ -0,0 +1,11 @@ +--- +title: "Organization settings on V5" +description: "What remains is the shared context string and a read-only namespace count" +sidebarTitle: "Organization context" +--- + +import Organization from "/snippets/api-v5-organization.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-profiles.mdx b/apps/docs/migration/api-v5-profiles.mdx new file mode 100644 index 00000000..dbb27513 --- /dev/null +++ b/apps/docs/migration/api-v5-profiles.mdx @@ -0,0 +1,11 @@ +--- +title: "Profiles and buckets on V5" +description: "Profiles no longer take search parameters; buckets get their own resource" +sidebarTitle: "Profiles & buckets" +--- + +import Profiles from "/snippets/api-v5-profiles.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-recall.mdx b/apps/docs/migration/api-v5-recall.mdx new file mode 100644 index 00000000..a2d46703 --- /dev/null +++ b/apps/docs/migration/api-v5-recall.mdx @@ -0,0 +1,11 @@ +--- +title: "Searching on V5" +description: "Modes, filters, attachments, changed defaults, and response readers" +sidebarTitle: "Searching" +--- + +import Search from "/snippets/api-v5-search.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-rollout.mdx b/apps/docs/migration/api-v5-rollout.mdx new file mode 100644 index 00000000..3e5c04f7 --- /dev/null +++ b/apps/docs/migration/api-v5-rollout.mdx @@ -0,0 +1,11 @@ +--- +title: "Checking parity, then rolling out V5" +description: "Establish behavioral parity, account for intended differences, and cut over safely" +sidebarTitle: "Parity and rollout" +--- + +import Rollout from "/snippets/api-v5-rollout.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5-settings.mdx b/apps/docs/migration/api-v5-settings.mdx new file mode 100644 index 00000000..9f88f6d9 --- /dev/null +++ b/apps/docs/migration/api-v5-settings.mdx @@ -0,0 +1,11 @@ +--- +title: "Container tags become namespaces" +description: "Listing namespaces, editing their settings, deleting them, and moving their content" +sidebarTitle: "Namespace lifecycle" +--- + +import Namespaces from "/snippets/api-v5-namespaces.mdx"; + +## Migration details + + diff --git a/apps/docs/migration/api-v5.mdx b/apps/docs/migration/api-v5.mdx new file mode 100644 index 00000000..67152a98 --- /dev/null +++ b/apps/docs/migration/api-v5.mdx @@ -0,0 +1,63 @@ +--- +title: "Moving a V3/V4 integration onto V5" +description: "Move existing API integrations onto the namespace-scoped V5 surface" +sidebarTitle: "V3/V4 to V5" +icon: "arrow-up-right" +--- + +import Overview from "/snippets/api-v5-overview.mdx"; +import DocumentIngestion from "/snippets/api-v5-document-ingestion.mdx"; +import DocumentUpdates from "/snippets/api-v5-document-updates.mdx"; +import ContentManagement from "/snippets/api-v5-content-management.mdx"; +import Search from "/snippets/api-v5-search.mdx"; +import Profiles from "/snippets/api-v5-profiles.mdx"; +import MemoryForgetting from "/snippets/api-v5-memory-forgetting.mdx"; +import Namespaces from "/snippets/api-v5-namespaces.mdx"; +import Organization from "/snippets/api-v5-organization.mdx"; +import Filters from "/snippets/api-v5-filters.mdx"; +import Rollout from "/snippets/api-v5-rollout.mdx"; +import Completion from "/snippets/api-v5-completion.mdx"; + + + +## Document ingestion + + + +## Document updates + + + +## Content management + + + +## Search + + + +## Profiles and buckets + + + +## Memory forgetting + + + +## Namespaces + + + +## Organization settings + + + +## Typed filters + + + +## Verification and rollout + + + + diff --git a/apps/docs/snippets/api-v5-completion.mdx b/apps/docs/snippets/api-v5-completion.mdx new file mode 100644 index 00000000..bfcb2907 --- /dev/null +++ b/apps/docs/snippets/api-v5-completion.mdx @@ -0,0 +1,29 @@ +## Agent migration prompt + +```text +Move this repository off the legacy Supermemory V3/V4 API and onto V5. Begin by +taking inventory: list every call site still targeting a legacy route. Then work +through the per-domain guides linked from this page. Address exactly one +namespace in each request. Where a default differs between the two versions, +state it in the request rather than relying on whatever the server supplies. +Bring response readers up to date as well. Back the change with contract tests +that exercise old and new side by side. Leave application routes unprefixed, and +leave connector routes alone. When an operation has been dropped, name it as +dropped rather than manufacturing a stand-in. +``` + +## Operations without a direct replacement + +- **Direct memory creation and version updates** (`POST /v4/memories`, + `PATCH /v4/memories`): ingest or replace a source document instead. +- **Organization bucket suggestion** (`POST /v3/settings/suggest-buckets`) and + **organization reset** (`POST /v3/settings/reset`): not part of the V5 public + surface. +- **Connector routes** that still use `/v3`: unchanged; do not rewrite them. + +## Where to go next + +- [Authentication](/authentication) — unchanged bearer keys +- [Search guide](/recall/search) +- [Memory operations](/recall/memory-operations) +- [API reference](/api-reference/overview) — legacy V3/V4 reference diff --git a/apps/docs/snippets/api-v5-content-management.mdx b/apps/docs/snippets/api-v5-content-management.mdx new file mode 100644 index 00000000..d523b31d --- /dev/null +++ b/apps/docs/snippets/api-v5-content-management.mdx @@ -0,0 +1,105 @@ +In V5, retrieving, listing, and deleting all happen inside one namespace you name +explicitly. + +### Retrieve a document and its derived context + + +```bash Legacy +GET /v3/documents/{id} +GET /v3/documents/{id}/chunks +``` + +```bash V5 +GET /ns/{namespace}/document/{id}?attach=chunks&attach=memories +``` + + +Supply `attach` more than once to pull in chunks, memories, or both. An +attachment you never asked for is simply **missing** from the payload, whereas one +you did ask for that found nothing comes back as an **empty array**. The +difference is real: `result.chunks === undefined` and `result.chunks.length === 0` +do not mean the same thing. + +The lifecycle fields have moved beneath `system`: + +```json +{"system":{"status":"done","createdAt":"...","updatedAt":"..."}} +``` + +Legacy `GET /v3/documents/{id}` returned `status`, `createdAt`, and `updatedAt` +as top-level fields, and needed a separate `GET /v3/documents/{id}/chunks` call +for chunk data. V5 folds both into one request. + +### List documents, chunks, or memories + + +```bash Legacy +POST /v3/documents/list +POST /v4/memories/list +``` + +```bash V5 +POST /ns/{namespace}/list/{type}?page=1&limit=100&sort=createdAt&order=desc +{} +``` + + +Give `type` one of `documents`, `chunks`, or `memories`. Paging and ordering move +into the **query string**; the body carries nothing beyond an optional `filter`. + +Whichever type you ask for, the response carries `documents`, `chunks`, +`memories`, and `pagination` — but only the array you selected has entries in it. +Code that used to pull `memories` out of a document list has to read `documents` +now. + + + This is the one place where the legacy and V5 shapes diverge most sharply. + `ListMemoriesQuerySchema` in this repository takes `page`, `limit`, `sort`, + `order`, and a **stringified** `filters` value in the query string; V5 keeps + the paging controls in the query string but moves filtering into a typed body + field. + + +### Delete documents + + +```bash Legacy +DELETE /v3/documents/{id} +DELETE /v3/documents/bulk +``` + +```bash V5 +DELETE /ns/{namespace}/document +{"ids":["doc_1","external_id_2"]} +``` + + +The V5 array accepts **1–100** Supermemory or caller-defined IDs. Inspect both +`deletedCount` **and** per-ID `errors`: HTTP success can still include partial +failures. The legacy bulk schema capped `ids` at 100 as well, so the limit is +unchanged — but the legacy single-delete route is gone, and one call now covers +both cases. + +### Forget memories + +| Legacy intent | V5 operation | +| --- | --- | +| Forget exact IDs | `DELETE /ns/{namespace}/memories` with `{"ids":[...]}` | +| Find memories by meaning | `DELETE /ns/{namespace}/memories/semantic` with `{"query":"...","dryRun":true}` | + +Both answer with `{ count, matches, errors }`. To delete semantically without +drift, preview under `dryRun: true`, inspect the returned IDs, and send that exact +list to the exact-ID endpoint. + +The full dry-run, approval, response, and audit flow lives in +[memory forgetting](./api-v5-memory-forgetting). + +### Verification + +- Assert that requested-but-empty attachments are `[]` while omitted attachments + are absent. +- Paginate each resource type until `currentPage >= totalPages`; confirm + unselected arrays stay empty. +- Verify chunk rows carry their parent `documentId`. +- Try a document delete that only partly succeeds, and a semantic dry run. +- Verify an ID cannot read, list, or delete content outside its namespace. diff --git a/apps/docs/snippets/api-v5-document-ingestion.mdx b/apps/docs/snippets/api-v5-document-ingestion.mdx new file mode 100644 index 00000000..67fbaf0e --- /dev/null +++ b/apps/docs/snippets/api-v5-document-ingestion.mdx @@ -0,0 +1,91 @@ +V5 moves document scope into the URL and keeps a repeated caller-supplied ID attached to one evolving document instead of creating a new one each time. + +### Rename common fields + +| Legacy | V5 | Location | +| --- | --- | --- | +| `containerTag` | `{namespace}` | Path | +| `customId` | `id` | JSON body | +| `entityContext` | `supportingContext` | JSON / form body | +| `filterByMetadata` | `group` | JSON / form body | +| `documentDate` | `date` | JSON / form body | +| `taskType`, `dreaming` | unchanged | Query string | + +In the legacy schema these live in `MemoryUpdateSchema` (`customId`, +`entityContext`, `metadata`, `containerTags`) and are sent in the request body. +In V5 the scope leaves the body entirely and becomes a path segment, so a single +request can only ever address one namespace. + +### Add or append one document + + +```bash Legacy +POST /v3/documents +{"content":"new turn","customId":"conv_1","containerTag":"user_1"} +``` + +```bash V5 +POST /ns/user_1/document?dreaming=dynamic +{"content":"new turn","id":"conv_1"} +``` + + +The response is still an acceptance result carrying `id` and `status`. Repeating +the V5 request with the same `id: "conv_1"` adds or diffs the new content into +that document — it does **not** silently replace the canonical source. + +### Batch ingestion + + +```json Legacy +{"documents":["first","second"],"containerTag":"user_1"} +``` + +```json V5 +{"documents":[{"content":"first","id":"doc_1"},{"content":"second","id":"doc_2"}]} +``` + + +Post that V5 body to `POST /ns/user_1/document/batch`. The array holds anything +from **1** to **600** document objects. Namespace, `taskType`, and processing mode +apply once for the whole request, while content, ID, context, metadata, grouping, +and date are set per item. + +Results come back in submission order. Read `success`, `failed`, and every entry +of `results` — a single batch may mix accepted and rejected documents, so a `200` +on its own proves nothing. + +### File ingestion + +`POST /v3/documents/file` becomes `POST /ns/{namespace}/document/file`. Keep +sending `multipart/form-data`: + +| Part | Encoding | +| --- | --- | +| `file` | Binary file | +| `supportingContext`, `date` | Plain strings | +| `metadata`, `group` | JSON-encoded strings | +| `fileType`, `mimeType` | Query parameters, when inference is not enough | + +The API acknowledges the file once it is durably accepted. Extraction, indexing, +and memory formation continue asynchronously — poll the document rather than +assuming the first response means processing is complete. + +### Processing choices + +- `taskType=memory` extracts long-term memories; `taskType=superrag` indexes + source context without generating memories. +- `dreaming=dynamic` (the default) groups related documents so memories form + from coherent units rather than one isolated entry at a time. +- `dreaming=instant` handles documents one at a time and adds one billable + operation for each of them. + +### Verification + +- Ingest text, a public URL, and a file, then wait for each document to reach a + terminal processing state. +- Repeat a caller-defined ID and confirm append/diff behavior rather than + replacement. +- Post a batch in which some documents succeed and some fail; check that the + result order holds and that per-item errors are reported. +- Confirm metadata and grouping are still filterable after processing completes. diff --git a/apps/docs/snippets/api-v5-document-updates.mdx b/apps/docs/snippets/api-v5-document-updates.mdx new file mode 100644 index 00000000..bfb1818c --- /dev/null +++ b/apps/docs/snippets/api-v5-document-updates.mdx @@ -0,0 +1,95 @@ +V5 makes the difference between *adding* new information and *replacing* the canonical source explicit. Legacy `PATCH /v3/documents/{id}` overloaded both meanings, so pick the write deliberately. + +### Choose the correct write + +| Intent | Operation | Content behavior | +| --- | --- | --- | +| Add information to a stable caller ID | `POST /ns/{namespace}/document` | Append / diff | +| Replace text or URL content | `PATCH /ns/{namespace}/document/{id}` | Replace and reprocess | +| Update only supporting fields | Same `PATCH`, without `content` | Canonical content unchanged | +| Replace a source file and its user metadata | `POST /ns/{namespace}/document/file/{id}` | Full replacement and reprocess | +| Partially update a file or its supporting fields | `PATCH /ns/{namespace}/document/file/{id}` | Omitted fields unchanged | + +### Update text or URL content + + +```bash Legacy +PATCH /v3/documents/doc_1 +{"content":"corrected source","metadata":{"revision":2}} +``` + +```bash V5 +PATCH /ns/user_1/document/doc_1?dreaming=dynamic +{"content":"corrected source","metadata":{"revision":2}} +``` + + +You may send any non-empty selection from `content`, `supportingContext`, +`metadata`, `group`, and `date`. Once `content` is present it becomes the new +canonical source — which means facts that only the previous source supported can +vanish when the document is reprocessed. + + + Replacing `content` is not a metadata edit. If you only want to re-label a + document, send `metadata` / `group` / `date` and omit `content` entirely. + + +### Replace a file-backed document + +```bash +POST /ns/user_1/document/file/doc_1?dreaming=dynamic +Content-Type: multipart/form-data + +file=@corrected.pdf +metadata={"revision":2} +``` + +`file` is mandatory on `POST`, and the call swaps out the canonical source along +with the user-controlled metadata, grouping, context, and date. Any supporting +field you leave out is **cleared**. Reach for this when the request you are +sending is the whole new state of the file-backed document. + +### Partially update a file-backed document + +```bash +PATCH /ns/user_1/document/file/doc_1?dreaming=dynamic +Content-Type: multipart/form-data + +metadata={"reviewed":true} +``` + +`PATCH` touches nothing beyond the fields you actually send. Send `file` to swap +the source while the supporting fields you omit survive, or drop `file` entirely +for a change confined to metadata, group, context, or date. Note the encoding +split: `metadata` and `group` travel as JSON-encoded strings, whereas +`supportingContext` and `date` are plain strings. + + + V5 exposes no public `PUT /ns/{namespace}/document/file/{id}`. For a full + replacement use `POST`; for a partial one use `PATCH`. + + +### IDs and scope + +The path `id` may be the Supermemory document ID **or** your caller-defined ID. +It is resolved only inside `{namespace}`. An ID from another namespace is not a +cross-namespace update mechanism — the request is rejected or not found rather +than reaching across the boundary. + +### Processing and conflicts + +Content or file replacement is accepted before downstream processing finishes. A +document that is still processing, a namespace conflict, or a conflicting +internal file path can return `409`. Retry only after the conflicting operation +reaches a terminal state. + +### Verification + +- Patch metadata alone and confirm the document content and its derived facts are + unchanged. +- Patch content, then check that the replacement source is canonical once + processing finishes. +- Replace a file with `POST` and confirm omitted user metadata is cleared. +- Patch a file-backed document and confirm omitted fields survive. +- Try the same ID against a different namespace; the update should be rejected or + come back not found. diff --git a/apps/docs/snippets/api-v5-filters.mdx b/apps/docs/snippets/api-v5-filters.mdx new file mode 100644 index 00000000..72dc3d9e --- /dev/null +++ b/apps/docs/snippets/api-v5-filters.mdx @@ -0,0 +1,109 @@ +V5 uses one optional **singular** `filter` field for search, profiles, and list +operations, replacing the loosely-typed `filters` object. + +### Why the legacy shape was hard to migrate + +In this repository the legacy filter type is deliberately permissive: + +```ts +export const SearchFiltersSchema = z + .object({ + AND: z.array(z.unknown()).optional(), + OR: z.array(z.unknown()).optional(), + }) + .or(z.record(z.unknown())) +``` + +`z.array(z.unknown())` means the individual conditions are **unvalidated**. The +only place their intended shape is written down is an OpenAPI example and a +`// TODO: Improve filter schema` comment above `ListMemoriesQuerySchema`. Any +condition that "looked right" was accepted at the edge and interpreted +downstream. V5 replaces that with a discriminated union the server can reject. + +### Shape + +```ts +type Filter = + | { field: string; operator: "eq" | "neq"; value: string; caseSensitive?: boolean } + | { field: string; operator: "eq" | "neq"; value: number | boolean } + | { field: string; operator: "gt" | "gte" | "lt" | "lte"; value: number } + | { field: string; operator: "contains" | "notContains"; value: string; caseSensitive?: boolean } + | { field: string; operator: "arrayContains" | "arrayNotContains"; value: string } + | { operator: "and" | "or"; operands: Filter[] }; +``` + +A field name may be built from letters, numbers, `_`, `.`, and `-`. An expression +may nest at most **five levels** deep and carry at most **200 operands** in a +single logical group. + +### Operator mapping + +| Legacy condition | V5 predicate | +| --- | --- | +| `{ key, value }` | `{ field: key, operator: "eq", value }` | +| `negate: true` equality | `operator: "neq"` | +| `filterType: "string_contains"` | `operator: "contains"` | +| contains + `negate: true` | `operator: "notContains"` | +| `filterType: "array_contains"` | `operator: "arrayContains"` | +| array contains + `negate: true` | `operator: "arrayNotContains"` | +| numeric `=` / numeric `=` + `negate: true` | `eq` / `neq` with a JSON number | +| numeric `>`, `>=`, `<`, `<=` | `gt`, `gte`, `lt`, `lte` | +| `AND` / `OR` arrays | lowercase `and` / `or` with `operands` | +| `ignoreCase: true` | `caseSensitive: false` | + +### Before and after + + +```json Legacy +{ + "AND": [ + { "key": "category", "value": "research" }, + { "key": "score", "value": 0.8, "filterType": "numeric", "numericOperator": ">=" } + ] +} +``` + +```json V5 +{ + "operator": "and", + "operands": [ + { "field": "category", "operator": "eq", "value": "research" }, + { "field": "score", "operator": "gte", "value": 0.8 } + ] +} +``` + + +### Deterministic conversion + +Apply these steps in order. They are mechanical on purpose — a codemod should be +able to do them without judgement calls. + +1. Rename the outer `filters` field to `filter`. +2. Recursively replace `AND` / `OR` objects with `{ operator, operands }`. +3. The legacy `key` becomes `field`. +4. Convert legacy flags (`negate`, `numericOperator`, `filterType`) into one + explicit operator. +5. Keep numeric values as JSON **numbers**, not numeric strings. +6. Delete the legacy `filterType`, `negate`, `numericOperator`, and `ignoreCase` + keys. + + + Step 5 is the step most likely to be missed. Three schemas in this repository + carry the same trap — `ListMemoriesQuerySchema`, `SearchRequestSchema` + (`packages/validation/api.ts:414`), and `Searchv4RequestSchema` + (`packages/validation/api.ts:508`) each ship an example that passes + `"value": "1742745777"`, a numeric comparison written as a **string**. Every + `gt`/`gte`/`lt`/`lte` operator in V5's union demands a JSON number, so a + straight rename will fail validation. + + +### Verification + +- Compare result IDs for equality, inequality, contains, numeric, array, nested + `AND`, and nested `OR` fixtures. +- Add negative tests: legacy shapes, empty `operands`, incompatible value types, + and unknown keys must all return `400`. +- Confirm an omitted `filter` preserves unfiltered behavior. +- Test one fixture at the nesting limit and one beyond it, to pin the five-level + bound. diff --git a/apps/docs/snippets/api-v5-memory-forgetting.mdx b/apps/docs/snippets/api-v5-memory-forgetting.mdx new file mode 100644 index 00000000..0e20c365 --- /dev/null +++ b/apps/docs/snippets/api-v5-memory-forgetting.mdx @@ -0,0 +1,93 @@ +Forgetting in V5 acts on a single namespace, and an exact-ID request and a +semantic request come back in the same result envelope. + +### Choose exact or semantic forgetting + +| Intent | V5 operation | +| --- | --- | +| Forget reviewed memory IDs | `DELETE /ns/{namespace}/memories` | +| Find memories by meaning | `DELETE /ns/{namespace}/memories/semantic` | + +The legacy API split these across two differently shaped endpoints: +`DELETE /v4/memories` (identify by `id` **or** exact `content`, plus a `reason`) +and `POST /v4/memories/forget-matching` (semantic `query` or explicit `ids`, with +`dryRun`, `threshold`, and `maxForget`). V5 normalizes both onto one response +contract. + +### Forget exact IDs + +```bash +DELETE /ns/user_1/memories +Content-Type: application/json + +{"ids":["mem_1","mem_2"]} +``` + +The count runs from **1** to **500** IDs. Your successes arrive under `matches`; +anything missing or ineligible lands in `errors`. A 2xx status therefore does not +prove that every ID you named was actually forgotten. + +### Preview a semantic request + +```bash +DELETE /ns/user_1/memories/semantic +Content-Type: application/json + +{"query":"outdated home address","dryRun":true} +``` + +With `dryRun: true` the request only selects — memory state is left **untouched**. +With `dryRun: false` it removes whatever the selection resolves to at the moment +the request runs. + +### Avoid selection drift + +```text +semantic request with dryRun: true + | + v +review matches[].id + | + v +exact DELETE with reviewed IDs +``` + +Use this workflow whenever a human or a policy must approve the exact set. +Re-running the semantic request with `dryRun: false` can select a **different** +set if memories changed after the preview. + + + Do not assume the semantic endpoint is idempotent with respect to your earlier + preview. The legacy `forget-matching` route had the same property, but the + legacy docs described the two-step preview flow as optional; in V5 it is the + recommended path for any deletion a reviewer has to sign off on. + + +### Read the normalized response + +```json +{ + "count": 1, + "matches": [{ "id": "mem_1", "memory": "Old address" }], + "errors": [{ "id": "mem_2", "error": "Memory not found" }] +} +``` + +The value of `count` is by definition the length of `matches`. Dry-run and +applied semantic requests share this shape, so the response body on its own will +not reveal which mode produced it — record the mode next to your audit entry. + +### Removed memory-write routes + +V5 offers nothing in place of direct V4 memory creation or version updates. Feed +source material in through the document routes and rewrite the canonical document +whenever the underlying facts change. + +### Verification + +- Try ID sets that are all-success, partially successful, duplicated, unknown, + and drawn from across namespaces. +- A dry run should leave every matched memory recallable. +- Forget the reviewed IDs exactly, then check that ordinary recall no longer + surfaces them. +- Log the request mode alongside the audit trail for semantic operations. diff --git a/apps/docs/snippets/api-v5-namespaces.mdx b/apps/docs/snippets/api-v5-namespaces.mdx new file mode 100644 index 00000000..4df57764 --- /dev/null +++ b/apps/docs/snippets/api-v5-namespaces.mdx @@ -0,0 +1,86 @@ +V5 renames the public isolation boundary from *container tag* to *namespace*. +Existing values remain valid identifiers — no stored data rename is required. + +### Endpoint mapping + +| Legacy | V5 | +| --- | --- | +| `GET /v3/container-tags/list` | `GET /namespaces` | +| `GET /v3/container-tags/{tag}` | `GET /ns/{namespace}` | +| `PATCH /v3/container-tags/{tag}` | `PATCH /ns/{namespace}` | +| `DELETE /v3/container-tags/{tag}` | `DELETE /ns/{namespace}` | +| Merge container tags | `DELETE /ns/{source}` with `{"moveTo":"target"}` | + +`GET /ns` also works and answers the same way as `GET /namespaces`; new code +should standardize on the longer form. + +### List namespaces + +Each entry now exposes `id`, `namespace`, `documentCount`, `memoryCount`, +nullable `description`, and `system.createdAt` / `system.updatedAt`. If a reader +of yours still looks for `containerTag`, top-level timestamps, or the internal +settings blobs, it needs to change. + + + The legacy list response in this repository is a flat array whose entries carry + `id`, `name`, `containerTag`, `createdAt`, `updatedAt`, `isExperimental`, + `emoji`, `isNova`, and `visibility`. V5 replaces `containerTag` with + `namespace`, nests the timestamps under `system`, and drops the + project/consumer distinction fields from the public shape. + + +### Read and update settings + + +```bash Legacy +PATCH /v3/container-tags/project_alpha +{"entityContext":"Research project for distributed systems"} +``` + +```bash V5 +PATCH /ns/project_alpha +{"supportingContext":"Research project for distributed systems"} +``` + + +The public `GET` and `PATCH` shapes contain only `namespace`, `supportingContext`, +and lifecycle timestamps. Profile buckets have their own `/profile/buckets` +resource and are **not** namespace settings. + +Passing `supportingContext: null` wipes whatever context is stored. You cannot +simply leave the field out — `PATCH` demands at least one supported setting. + +### Permanently delete a namespace + +```bash +DELETE /ns/project_alpha +{} +``` + +Omit `moveTo` and the delete runs synchronously: you get `200` back along with +`deletedDocumentsCount` and `deletedMemoriesCount`. Namespace and content are +both gone. + +### Move then remove a namespace + +```bash +DELETE /ns/project_alpha +{"moveTo":"project_archive"} +``` + +You get `202` back with `status: "queued"` and an `operationId`. Source and +destination must not be the same namespace. A `202` here means the move was +queued, not that it finished. + +Where legacy merge took several sources at once, V5 takes **one source per +request**. Chain the moves and log each one separately. + +### Verification + +- Check that existing container-tag values still resolve when used as namespace + paths. +- Compare namespace counts against namespace-scoped document and memory lists. +- A permanent delete should return final counts; a move should return `202`. +- Confirm a restricted caller can neither read nor change namespaces beyond its + own scope. +- Confirm only organization-authorized callers can change settings or lifecycle. diff --git a/apps/docs/snippets/api-v5-organization.mdx b/apps/docs/snippets/api-v5-organization.mdx new file mode 100644 index 00000000..5038dd85 --- /dev/null +++ b/apps/docs/snippets/api-v5-organization.mdx @@ -0,0 +1,76 @@ +What survives into V5 is the single organization-wide context string that steers +memory formation. The administrative knobs and the profile-bucket mutation +endpoints are gone from this resource. + +### Endpoint mapping + +| Legacy | V5 | +| --- | --- | +| `GET /v3/settings` | `GET /organization` | +| `PATCH /v3/settings` | `PATCH /organization` | +| `filterPrompt` | `organizationalContext` | + +### Read organization settings + +```bash +GET /organization +``` + +```json +{ + "organizationalContext": "Acme builds security tools for enterprises", + "namespaceCount": 42 +} +``` + +Any reader you have for a legacy setting that is absent from this allowlisted +payload should be deleted. `namespaceCount` is read-only information; `PATCH` +will not move it. + +### Update organization context + + +```bash Legacy +PATCH /v3/settings +{"filterPrompt":"Acme builds security tools for enterprises"} +``` + +```bash V5 +PATCH /organization +{"organizationalContext":"Acme builds security tools for enterprises"} +``` + + +The write is exhaustive — whatever you supply becomes the context, fully +**replacing** what was there. Passing `null` clears it; an empty string is +refused. + +An organization administrator is required for these writes. A `403` means the +caller is not one — it is not an invitation to fall back to namespace context +quietly. + +### Removed public operations + +- There is no organization-settings route for profile-bucket mutation. +- Bucket suggestion (`POST /v3/settings/suggest-buckets`) is not part of V5. +- Organization data reset (`POST /v3/settings/reset`) is not part of V5. +- Buckets belonging to a namespace live under + `/ns/{namespace}/profile/buckets`. + + + The legacy `OrganizationSettingsSchema` in this repository carries connection + credentials (`googleDriveClientId`, `notionClientId`, `onedriveClientId`, and + their secrets and enablement flags) alongside the filtering fields. Those are + **not** part of the public V5 `/organization` response. If your code read them + from `GET /v3/settings`, move that configuration to your own secret store or to + the connector setup flow. + + +### Verification + +- Diff the V5 context against the legacy `filterPrompt` before you cut over. +- Write the context, rewrite it, then clear it. +- Check that `namespaceCount` matches what `GET /namespaces` reports for the same + credentials. +- Verify non-admin callers receive `403` on `PATCH`. +- Check that nothing downstream still depends on the fields V5 dropped. diff --git a/apps/docs/snippets/api-v5-overview.mdx b/apps/docs/snippets/api-v5-overview.mdx new file mode 100644 index 00000000..5e8d3b71 --- /dev/null +++ b/apps/docs/snippets/api-v5-overview.mdx @@ -0,0 +1,90 @@ +V5 keeps the same ingestion and recall workflows as V3/V4, but makes scope explicit in the URL, consolidates routes that used to overlap, and tightens request and response types. + + + This is a **breaking API migration**. Changing only the URL is not enough: fields moved between path, query, and body, search defaults changed, response envelopes changed, and a few legacy operations have no V5 replacement at all. + + +The API base URL and your bearer keys do not change. V5 application routes are **unversioned** — `/v5/reference` identifies the documentation version, not an API path prefix. Do not put `/v5` in application request paths. + +## Recommended migration process + + + + Grep your integration for `/v3/`, `/v4/`, `containerTag`, `containerTags`, + `customId`, `entityContext`, `filterByMetadata`, `filters`, and any legacy + SDK method wrappers. Record each one with its caller and its purpose. + + + Move the legacy `containerTag` value into the `/ns/{namespace}` path + segment. Never infer scope from a document or memory ID, and never send + more than one namespace in a single V5 request. + + + Apply [document writes](./api-v5-document-writes), + [document updates](./api-v5-document-updates), + [content management](./api-v5-document-reads), + [recall/search](./api-v5-recall), + [profiles](./api-v5-profiles), + [forgetting](./api-v5-memory-forgetting), + [namespaces](./api-v5-settings), + [organization](./api-v5-organization), and + [typed filters](./api-v5-filters) independently. + + + Migrate envelopes, attachments, pagination, profile buckets, `system` + lifecycle fields, and partial-error handling. A translated request with an + untranslated reader still fails. + + + Work through the [verification and rollout guide](./api-v5-rollout). Check + identity and behavior — never raw JSON ordering — and pass changed defaults + explicitly so a default change cannot masquerade as a bug. + + + Switch traffic, monitor failures and semantic drift, then delete that + domain's legacy compatibility code only after it passes verification. + + + +## Endpoint map + +| Legacy | V5 | +| --- | --- | +| `POST /v3/documents` | `POST /ns/{namespace}/document` | +| `POST /v3/documents/batch` | `POST /ns/{namespace}/document/batch` | +| `POST /v3/documents/file` | `POST /ns/{namespace}/document/file` | +| `GET/PATCH /v3/documents/{id}` | `GET/PATCH /ns/{namespace}/document/{id}` | +| `DELETE /v3/documents/{id}`, `DELETE /v3/documents/bulk` | `DELETE /ns/{namespace}/document` | +| `POST /v3/documents/list`, `POST /v4/memories/list` | `POST /ns/{namespace}/list/{type}` | +| `POST /v3/search`, `POST /v4/search` | `POST /ns/{namespace}/search` | +| `POST /v4/profile` | `POST /ns/{namespace}/profile` | +| `POST /v4/profile/buckets` | `GET/PUT/DELETE /ns/{namespace}/profile/buckets` | +| `DELETE /v4/memories`, `POST /v4/memories/forget-matching` | `DELETE /ns/{namespace}/memories`, `DELETE /ns/{namespace}/memories/semantic` | +| `GET/PATCH/DELETE /v3/container-tags/{tag}`, `POST /v3/container-tags/merge` | `GET /namespaces`, `GET/PATCH/DELETE /ns/{namespace}` | +| `GET/PATCH /v3/settings` | `GET/PATCH /organization` | + +## What has no V5 replacement + +These legacy operations were deliberately dropped. Do not invent a substitute — +rework the caller instead. + +- **Direct memory creation and version updates.** `POST /v4/memories` and + `PATCH /v4/memories` have no V5 equivalent. Ingest or replace a source + document and let memories form from it. +- **Organization bucket suggestion.** `POST /v3/settings/suggest-buckets` is not + part of the V5 public surface. +- **Organization data reset.** `POST /v3/settings/reset` is not part of the V5 + public surface. +- **Connector routes.** Connections still use their existing `/v3/connections/*` + paths and are **unchanged** by this migration. Do not rewrite them. + +## Grounding note + +This guide tracks the contract described by the V5 OpenAPI document that +accompanies the V5 deployment. (For the legacy surfaces, the live +`/v4/openapi` and `/v3/openapi` documents remain the references.) The legacy +field names and defaults contrasted here are the ones still present in this +repository's shared validation package — notably `SearchRequestSchema`, +`Searchv4RequestSchema`, `ListMemoriesQuerySchema`, `MemoryUpdateSchema`, and the +`SearchFiltersSchema` union. Where a behavior is not described by either source, +this guide says so rather than guessing. diff --git a/apps/docs/snippets/api-v5-profiles.mdx b/apps/docs/snippets/api-v5-profiles.mdx new file mode 100644 index 00000000..a4660b5b --- /dev/null +++ b/apps/docs/snippets/api-v5-profiles.mdx @@ -0,0 +1,95 @@ +V5 returns a maintained profile directly, and gives profile bucket definitions +their own namespace-scoped resource. + +### Remove search behavior from profile calls + + +```bash Legacy +POST /v4/profile +{"containerTag":"user_1","q":"work preferences","threshold":0.6,"include":{}} +``` + +```bash V5 +POST /ns/user_1/profile +{"filter":{"field":"region","operator":"eq","value":"us-west"},"buckets":["work"]} +``` + + +Remove the legacy `q`, `threshold`, and `include`. If the caller needs +query-ranked results, issue a **separate** V5 search request. Move `containerTag` +to the path and rename `filters` to the singular `filter`. + + + The legacy shape is visible in this repository: `POST /v4/profile` is called + with a body of `{ q, containerTag, include: ["static","dynamic"] }` and the + response is read as `profile.static`, `profile.dynamic`, and + `profile.buckets`. V5 keeps the `profile.static` / `profile.dynamic` / + `profile.buckets` reader intact — what changes is that query parameters no + longer belong on the profile call. + + +### Read the V5 profile shape + +```json +{ + "profile": { + "static": ["The user works in design"], + "dynamic": ["The user is preparing a launch"], + "buckets": { "work": ["Prefers concise project updates"] } + } +} +``` + +There is no way to switch off `static` or `dynamic`; both come back on every +call. Leave `buckets` out of the request and you receive every effective custom +bucket — or name up to 50 of them to restrict just that section. + +### Read bucket definitions + + +```bash Legacy +POST /v4/profile/buckets +{"containerTag":"user_1"} +``` + +```bash V5 +GET /ns/user_1/profile/buckets +``` + + +The response changes from key/description objects to a plain map: + +```json +{"buckets":{"work":"Professional preferences and ongoing work"}} +``` + +### Add or edit namespace buckets + +```bash +PUT /ns/user_1/profile/buckets +{"buckets":{"work":"Professional preferences and ongoing work"}} +``` + +Supply anywhere between one and 50 name-to-description entries. Names already in +the namespace get rewritten, new ones get created, and any namespace bucket you +leave out stays exactly as it was. + +### Delete namespace buckets + +```bash +DELETE /ns/user_1/profile/buckets +{"buckets":["work"]} +``` + +Every name has to be distinct. Buckets owned by the organization may show up in +the effective `GET` response, but a namespace-level `PUT` or `DELETE` will not +alter or drop them. + +### Verification + +- Check that `static`, `dynamic`, and `buckets` are present in every profile + response. +- Compare an omitted `buckets` request against one-name and multi-name narrowing. +- Add, edit, and delete a namespace bucket without disturbing omitted buckets. +- Attempt to mutate an inherited organization bucket and expect the documented + error rather than a silent no-op. diff --git a/apps/docs/snippets/api-v5-rollout.mdx b/apps/docs/snippets/api-v5-rollout.mdx new file mode 100644 index 00000000..f5ace46b --- /dev/null +++ b/apps/docs/snippets/api-v5-rollout.mdx @@ -0,0 +1,77 @@ +Treat migration as a **behavioral comparison**, not a raw response-snapshot +update. A diff of two JSON blobs will be dominated by generated IDs and +timestamps and will tell you nothing about whether recall still works. + +### Build deterministic fixtures + +Use an isolated namespace, give every fixture a stable ID, and hold the source +content fixed. The fixture set should span plaintext, URL, file, and batch +ingestion, plus metadata, grouping, profile facts, related memories, forgotten +memories, and the case where nothing matches. + +For each case, write down the legacy request, the V5 request, the semantic +outcome you expect, and any delta you are knowingly accepting. Whatever you do, +keep generated IDs, signed URLs, timings, and JSON key ordering out of the +comparison unless the contract actually promises them. + +### Compare writes + +- Repeated `POST` appends or diffs; `PATCH` replaces canonical content. +- A batch keeps its results in the order you submitted them, and reports + per-item failures rather than hiding them. +- Touching only metadata leaves the source content as it was. +- An accepted write is not a finished one: poll until processing settles into a + terminal state. + +### Compare reads and recall + +- Attachments you did not request are missing from the payload; attachments you + did request but that matched nothing come back as empty arrays. +- A list call fills in the array for the resource type you asked for and leaves + the others untouched. +- Before trying V5's own search defaults, establish parity with the V4 mode and + threshold stated explicitly. +- Profiles always contain `static`, `dynamic`, and `buckets`. + +### Exercise boundaries + +| Boundary | Cases | +| --- | --- | +| Namespace | Correct, missing, unauthorized, cross-namespace ID | +| Pagination | First, middle, final, empty, maximum limit | +| Filters | Every operator, nested AND/OR, invalid type, excessive depth | +| Deletion | All success, partial success, unknown IDs, semantic dry run | +| Settings | Admin, non-admin, null removal, invalid empty value | + +### Classify differences + +```text +same request intent + | + +-- same semantic result ------> parity + +-- documented V5 difference --> update assertion + +-- undocumented difference ----> block cutover +``` + +An undocumented difference is a finding, not noise to be smoothed over. Save the +request pair, the namespace, the IDs, the response status, and the smallest slice +of the response that reproduces it. + +### Cut over by domain + +1. Build V5 requests behind a switch you can flip per domain. +2. Where side effects permit, read both paths or shadow-call the new one. +3. Cut over ingestion, content management, search, profiles, and settings one at + a time, in that order. +4. Watch for validation and authorization failures, latency shifts, a changing + empty-result rate, and processing errors. +5. Keep the legacy path in place until the observation window closes. + +### Completion checklist + +- No application call ends up pointing at a `/v5` prefix by mistake. +- Connector routes, which this migration leaves alone, still carry their + documented `/v3` paths. +- Nothing still reads a legacy field name or legacy response shape. +- Every default that changed is either deliberately accepted or sent explicitly. +- Rolling back returns the previous caller to service without a data repair. diff --git a/apps/docs/snippets/api-v5-search.mdx b/apps/docs/snippets/api-v5-search.mdx new file mode 100644 index 00000000..610078e2 --- /dev/null +++ b/apps/docs/snippets/api-v5-search.mdx @@ -0,0 +1,98 @@ +A V5 search runs against a single namespace, falls back to hybrid recall when you +say nothing, and takes its ranking controls from a typed request body. + +### Request mapping + +| Legacy | V5 | +| --- | --- | +| `containerTag` | `/ns/{namespace}` | +| body `q` | body `query` | +| body `limit` | query `limit` | +| `searchMode: "documents"` | `searchMode=chunks` | +| omitted search mode | `searchMode=hybrid` | +| `filters` | singular `filter` | +| `include.documents` or `include.summaries` | `attach.documents` | +| `include.relatedMemories` | `attach.related` | +| `include.forgottenMemories` | `attach.forgotten` | +| `rerank: true` / `aggregate: true` | `rerank: "order"` / `"aggregate"` | + + +```bash Legacy +POST /v4/search +{"q":"What did the user decide?","containerTag":"user_1","limit":10,"searchMode":"memories"} +``` + +```bash V5 +POST /ns/user_1/search?limit=10&searchMode=memories +{"query":"What did the user decide?","threshold":0.6,"rewriteQuery":false} +``` + + +### Changed defaults + +| Setting | V4 | V5 | +| --- | --- | --- | +| Search mode | `memories` | `hybrid` | +| Similarity threshold | `0.6` | `0.3` | +| Reranking | Disabled | `rerank: "none"` | +| Query rewriting | Disabled | `false` | + +These are the values actually declared in `Searchv4RequestSchema` in this +repository: `threshold` defaults to `0.6`, `limit` to `10`, and `rerank` and +`rewriteQuery` both default to `false`. V5 widens recall by default, so set mode +and threshold **explicitly** while you compare versions. Once parity testing +passes, drop them only if you actually want the broader V5 hybrid defaults. + + + A silently widened threshold is the single most common false "regression" + during this migration. It returns *more* results, not wrong ones — pin + `threshold` and `searchMode` until your comparison is green. + + +### Search modes + +| Mode | Returns | +| --- | --- | +| `memories` | Formed memories only | +| `chunks` | Source chunks only | +| `hybrid` | Both result types in one ranked list | + +Nothing in V5 corresponds to the legacy `include.chunks` flag. Pick `chunks` or +`hybrid` instead. + +### Attachments and ranking + +- Asking for `attach.documents` pulls the single most relevant source document + into each result. +- `attach.related` brings in parent, child, and sibling memories. The legacy + `Searchv4RequestSchema` exposed this as `include.relatedMemories`, and memory + results still carry a `context` object with `parents` and `children` arrays + whose entries use the `updates` / `extends` / `derives` relation enum. +- `attach.forgotten` lets forgotten memories appear in related context; it does + not promote them to primary results. +- The `rerank` setting takes `none`, `order`, or `aggregate`, while + `rewriteQuery` governs query rewriting aimed at retrieval. + +### Response mapping + +| Legacy reader | V5 reader | +| --- | --- | +| Result array | `results` | +| Timing | `searchTime` | +| Source expansion | `result.included.document` | +| Related context | `result.included.related.{parents,children,siblings}` | +| Lifecycle fields | `result.system` | + +Each primary result contains either `memory`, `chunk`, or both only where the +contract allows it. Branch on field presence rather than assuming one result +shape — in hybrid mode a single result array mixes both kinds. + +### Verification + +- Pin the V4 defaults explicitly and check result IDs against them first; treat + V5 hybrid behavior as a separate test. +- Cover all three modes, thresholds at `0` and `1`, each rerank option, and query + rewriting on and off. +- Try each attachment on its own and in combination, including the empty case. +- Verify filters, namespace isolation, result limits, and rejection of body + parameters placed in the query string (or vice versa).