From 5567e82377495ffbc60ae830c1b848d5a3fa3699 Mon Sep 17 00:00:00 2001 From: Dhravya Shah Date: Fri, 10 Jul 2026 19:15:46 -0700 Subject: [PATCH] docs(self-hosting): configurable embeddings for Supermemory local (#1210) Co-authored-by: Cursor Agent --- README.md | 3 +- apps/docs/docs.json | 1 + apps/docs/self-hosting/configuration.mdx | 28 ++++- apps/docs/self-hosting/embeddings.mdx | 135 +++++++++++++++++++++++ apps/docs/self-hosting/overview.mdx | 9 +- apps/docs/self-hosting/quickstart.mdx | 11 +- 6 files changed, 176 insertions(+), 11 deletions(-) create mode 100644 apps/docs/self-hosting/embeddings.mdx diff --git a/README.md b/README.md index 4a17c593..33a98c5f 100644 --- a/README.md +++ b/README.md @@ -341,11 +341,12 @@ const client = new Supermemory({ ``` - **Bring any model** — OpenAI, Anthropic, Gemini, Groq, or any OpenAI-compatible endpoint. An interactive wizard walks you through it on first boot. +- **Embeddings** — local `Xenova/bge-base-en-v1.5` by default (no API key); optionally OpenAI, Gemini, or Ollama. Same provider stack as cloud. - **Fully offline if you want** — point it at Ollama (`gpt-oss:20b` works great) and nothing leaves your machine. - **Your data, one directory** — everything lives in `./.supermemory`, easy to back up or move. - **Same API as the platform** — prototype locally, ship on the hosted platform by changing `baseURL`. -Read the [self-hosting docs](https://supermemory.ai/docs/self-hosting/overview) — quickstart, configuration, and [local vs. Enterprise](https://supermemory.ai/docs/self-hosting/local-vs-enterprise). +Read the [self-hosting docs](https://supermemory.ai/docs/self-hosting/overview) — quickstart, [configuration](https://supermemory.ai/docs/self-hosting/configuration), [embeddings](https://supermemory.ai/docs/self-hosting/embeddings), and [local vs. Enterprise](https://supermemory.ai/docs/self-hosting/local-vs-enterprise). --- diff --git a/apps/docs/docs.json b/apps/docs/docs.json index 71ff23c0..d3a62eff 100644 --- a/apps/docs/docs.json +++ b/apps/docs/docs.json @@ -77,6 +77,7 @@ "self-hosting/overview", "self-hosting/quickstart", "self-hosting/configuration", + "self-hosting/embeddings", "self-hosting/local-vs-enterprise" ] }, diff --git a/apps/docs/self-hosting/configuration.mdx b/apps/docs/self-hosting/configuration.mdx index 49580e6d..9714653f 100644 --- a/apps/docs/self-hosting/configuration.mdx +++ b/apps/docs/self-hosting/configuration.mdx @@ -5,7 +5,7 @@ description: "Every environment variable the self-hosted server understands." icon: "settings" --- -The self-hosted server aims for **zero configuration** — the only thing it needs is one model provider key, which the first-boot wizard collects interactively (or set it via env var for non-interactive deployments). Everything else below is opt-in, layered on top as you need it. +The self-hosted server aims for **zero configuration** — the only required input is one LLM provider key, which the first-boot wizard collects interactively (or set via env var for non-interactive deployments). Embeddings default to local English; you can pick another provider in the optional wizard step or via env. Everything else below is opt-in. The installer writes API keys to `~/.supermemory/env`, which is loaded on every launch. You can also set variables in your shell or a process manager. @@ -18,7 +18,7 @@ The installer writes API keys to `~/.supermemory/env`, which is loaded on every ## LLM providers -In production, Supermemory uses its own proprietary models tuned for long-horizon data understanding. Self-hosted, you bring your own: embeddings are computed locally, and a model of your choice powers the intelligent steps — summaries, contextual chunking, and memory extraction. Configure **at least one**: +In production, Supermemory uses its own proprietary models tuned for long-horizon data understanding. Self-hosted, you bring your own LLM for the intelligent steps — summaries, contextual chunking, and memory extraction. Embeddings default to a local model (no API key) and can optionally use OpenAI, Gemini, or Ollama — see [Embeddings](/self-hosting/embeddings). Configure **at least one** LLM provider: | Variable | Provider | |---|---| @@ -61,7 +61,20 @@ OPENAI_MODEL=gpt-oss:20b Nothing to configure. Uploaded files (PDFs, images) are stored on local disk inside `$SUPERMEMORY_DATA_DIR` and served by the server at `/files/:key`. -## Embedding performance +## Embeddings + +By default, vectors are computed locally with `Xenova/bge-base-en-v1.5` (768d) — no embedding API key. On interactive first boot you can pick a different provider after the LLM key step; for Docker/CI set env vars instead. + +Full provider table, multilingual guidance, remote examples (OpenAI / Gemini / Ollama), and the re-ingestion / dimension-lock warning: **[Embeddings (self-hosted)](/self-hosting/embeddings)**. + +| Variable | Purpose | Default | +|---|---|---| +| `SUPERMEMORY_EMBEDDING_PROVIDER` | `local`, `openai`, `gemini`, or OpenAI-compatible remote | `local` | +| `SUPERMEMORY_EMBEDDING_MODEL` | Model id for the chosen provider | `Xenova/bge-base-en-v1.5` | +| `SUPERMEMORY_EMBEDDING_DIMENSIONS` | Vector size; must match model and stored data | `768` | +| `SUPERMEMORY_EMBEDDING_BASE_URL` | Base URL for OpenAI-compatible embedding APIs | unset | + +### Embedding performance Local embeddings are prewarmed at startup with conservative defaults — one worker, minimal CPU footprint. Turn these up if you're ingesting heavily and prefer throughput over headroom: @@ -128,8 +141,13 @@ Any other environment variables you may find referenced in the codebase are plat # Persistent data location SUPERMEMORY_DATA_DIR=/var/lib/supermemory -# One LLM provider +# One LLM provider (required for extraction) OPENAI_API_KEY=sk-... + +# Optional — omit to keep local Xenova/bge-base-en-v1.5 (768d) +# SUPERMEMORY_EMBEDDING_PROVIDER=openai +# SUPERMEMORY_EMBEDDING_MODEL=text-embedding-3-small +# SUPERMEMORY_EMBEDDING_DIMENSIONS=1536 ``` -That's enough for full ingestion, memory extraction, and hybrid search. +That's enough for full ingestion, memory extraction, and hybrid search with the default local embeddings. diff --git a/apps/docs/self-hosting/embeddings.mdx b/apps/docs/self-hosting/embeddings.mdx new file mode 100644 index 00000000..95385ff0 --- /dev/null +++ b/apps/docs/self-hosting/embeddings.mdx @@ -0,0 +1,135 @@ +--- +title: "Embeddings (self-hosted)" +sidebarTitle: "Embeddings" +description: "Local and remote embedding providers for Supermemory local — defaults, env vars, multilingual options, and dimension lock." +icon: "waypoints" +--- + +Self-hosted Supermemory uses the **same embedding provider stack** as the hosted platform: local ONNX models, OpenAI, Gemini, or any OpenAI-compatible embeddings endpoint (including Ollama). LLM keys power extraction and summarization; embeddings are configured separately. + +## Defaults + +| | | +|---|---| +| Provider | `local` | +| Model | `Xenova/bge-base-en-v1.5` | +| Dimensions | `768` | +| API key | None — runs on your machine | + +Press Enter at the optional first-boot picker to keep this default. Nothing is sent off-box to embed. + + +The default local model is **English-only**. Non-English content can ingest successfully while dense semantic recall stays weak. See [Multilingual](#multilingual). + + +## First-time setup (interactive) + +On first boot with a TTY, Supermemory asks for an LLM API key (required), then optionally which embedding model to use. + +1. Choose or paste an LLM provider key (OpenAI, Anthropic, Gemini, Groq, or OpenAI-compatible). +2. Optionally pick an embedding provider/model. **Press Enter to keep the local English model.** +3. Choices are saved encrypted under your data directory (`$SUPERMEMORY_DATA_DIR`, typically `./.supermemory` / `~/.supermemory`). + +Boot order is intentional: LLM keys load first so remote embedding options can reuse them (for example OpenAI or Gemini embeddings with the same key). + + +**First boot (terminal):** Supermemory asks for an LLM API key (required), then optionally which embedding model to use. Press Enter to keep the local English model. Choices are saved encrypted under your data directory. + + +## Configuration (env) + +For Docker, CI, or any non-interactive deploy, set env vars — there is **no interactive prompt without a TTY**. + +| Variable | Purpose | Default | +|---|---|---| +| `SUPERMEMORY_EMBEDDING_PROVIDER` | Embedding backend: `local`, `openai`, `gemini`, or an OpenAI-compatible remote (`ollama` / custom base URL) | `local` | +| `SUPERMEMORY_EMBEDDING_MODEL` | Model id for the chosen provider | `Xenova/bge-base-en-v1.5` (local) | +| `SUPERMEMORY_EMBEDDING_DIMENSIONS` | Vector size; must match the model and any already-stored data | `768` (local default) | +| `SUPERMEMORY_EMBEDDING_BASE_URL` | Base URL for OpenAI-compatible embedding APIs (Ollama, vLLM, etc.) | unset | +| `OPENAI_API_KEY` | Used when provider is `openai` (or compatible) if not otherwise supplied | unset | +| `GEMINI_API_KEY` | Used when provider is `gemini` | unset | + +Local worker tuning (throughput only — does not change model or dimensions): + +| Variable | Purpose | Default | +|---|---|---| +| `SUPERMEMORY_LOCAL_EMBEDDING_POOL_SIZE` | Number of embedding workers | `1` | +| `SUPERMEMORY_LOCAL_EMBEDDING_WASM_THREADS` | Compute threads per worker | `1` | +| `SUPERMEMORY_LOCAL_EMBEDDING_BATCH_SIZE` | Texts per worker dispatch | `8` | +| `SUPERMEMORY_LOCAL_EMBEDDING_IDLE_TIMEOUT_MS` | Idle time before workers shut down | `120000` | +| `SUPERMEMORY_SKIP_EMBEDDING_PREWARM` | Skip startup prewarm, load on first use | unset | + +Ingestion memory headroom is controlled by `SUPERMEMORY_EMBEDDING_RAM_LIMIT` — see [Memory limits & ingestion queue](/self-hosting/configuration#memory-limits--ingestion-queue). + + +**Docker / production:** Set at least one LLM key and, if you don’t want local embeddings, set `SUPERMEMORY_EMBEDDING_PROVIDER` / `SUPERMEMORY_EMBEDDING_MODEL` / `SUPERMEMORY_EMBEDDING_DIMENSIONS` (and base URL or API key as needed). There is no interactive prompt without a TTY. + + +## Multilingual + +The default `Xenova/bge-base-en-v1.5` model is trained for English. For German, Dutch, and other non-English corpora, dense recall can fail even when hybrid keyword search still finds rare tokens. + +For multilingual or non-English deployments, switch **before** large backfills: + +```bash +# Example: local multilingual (set dimensions to match the model) +SUPERMEMORY_EMBEDDING_PROVIDER=local +SUPERMEMORY_EMBEDDING_MODEL=Xenova/bge-m3 +SUPERMEMORY_EMBEDDING_DIMENSIONS=1024 +``` + +Or use a remote multilingual embedding API (OpenAI, Gemini, or Ollama with a multilingual embed model). Set provider, model, and dimensions together. Changing them later requires a fresh data directory or full re-ingestion — see below. + +## Remote providers + +### Local (default) + +```bash +# Explicit local default — no embedding API key +SUPERMEMORY_EMBEDDING_PROVIDER=local +SUPERMEMORY_EMBEDDING_MODEL=Xenova/bge-base-en-v1.5 +SUPERMEMORY_EMBEDDING_DIMENSIONS=768 +``` + +### OpenAI + +```bash +OPENAI_API_KEY=sk-... +SUPERMEMORY_EMBEDDING_PROVIDER=openai +SUPERMEMORY_EMBEDDING_MODEL=text-embedding-3-small +SUPERMEMORY_EMBEDDING_DIMENSIONS=1536 +``` + +### Gemini + +```bash +GEMINI_API_KEY=... +SUPERMEMORY_EMBEDDING_PROVIDER=gemini +SUPERMEMORY_EMBEDDING_MODEL=text-embedding-004 +SUPERMEMORY_EMBEDDING_DIMENSIONS=768 +``` + +### Ollama (OpenAI-compatible) + +```bash +SUPERMEMORY_EMBEDDING_PROVIDER=openai +SUPERMEMORY_EMBEDDING_BASE_URL=http://localhost:11434/v1 +OPENAI_API_KEY=ollama +SUPERMEMORY_EMBEDDING_MODEL=nomic-embed-text +SUPERMEMORY_EMBEDDING_DIMENSIONS=768 +``` + +Use the dimension published for your chosen model. A mismatch with vectors already in the store fails boot. + +## Changing models later + + +**Not supported in place.** Embeddings from different models (or different dimensions) are not comparable. Start from a fresh data directory or re-ingest all content so vectors stay in one space. If configured dimensions disagree with stored data, the server **refuses to boot**. + + +**Changing embeddings later:** Not supported in place. Start from a fresh data directory or re-ingest all content so vectors stay comparable. + +## Related + +- [Configuration](/self-hosting/configuration) — LLM providers, storage, ingestion limits +- [Quickstart](/self-hosting/quickstart) — install and first memory diff --git a/apps/docs/self-hosting/overview.mdx b/apps/docs/self-hosting/overview.mdx index b9db33e4..c1ef34a1 100644 --- a/apps/docs/self-hosting/overview.mdx +++ b/apps/docs/self-hosting/overview.mdx @@ -24,7 +24,7 @@ No Docker. No database to provision. No config files. It boots in seconds with e Run the binary with nothing set and you get a complete memory system: - **The Supermemory graph engine, embedded** — created automatically on first boot. No database to stand up, no connection strings. -- **Built-in local embeddings** — vectors are computed on your machine. Nothing is sent anywhere to be embedded. +- **Built-in local embeddings** — default `Xenova/bge-base-en-v1.5` (768d) on your machine, no API key. Same provider stack as cloud if you opt into OpenAI, Gemini, or Ollama — see [Embeddings](/self-hosting/embeddings). - **An API key, generated for you** — printed on first boot, ready to paste into any SDK. - **The full Memory API** — `/v3/documents`, `/v4/search`, `/v4/profile`, spaces, the works. @@ -64,7 +64,7 @@ Self-hosted is free, open source, and great for local development, air-gapped en |---|---|---| | Full Memory API | ✅ | ✅ | | Hybrid semantic search | ✅ | ✅ | -| Local embeddings | ✅ | Managed | +| Embeddings | Local default (or OpenAI / Gemini / Ollama) | Same provider stack, managed | | File ingestion (PDFs, images) | ✅ | ✅ | | [Connectors](/connectors/overview) (Google Drive, Notion, Gmail, OneDrive) | — | ✅ | | [Supermemory MCP](/supermemory-mcp/mcp) | — | ✅ | @@ -75,11 +75,14 @@ If you outgrow a single machine — or want connectors, MCP, and the best-tuned ## Next steps - + Install, run, and store your first memory in under two minutes Every environment variable: LLM providers, storage, auth, tuning + + Local default, remote providers, multilingual, dimension lock + diff --git a/apps/docs/self-hosting/quickstart.mdx b/apps/docs/self-hosting/quickstart.mdx index 713ea98b..2e4dfe60 100644 --- a/apps/docs/self-hosting/quickstart.mdx +++ b/apps/docs/self-hosting/quickstart.mdx @@ -47,9 +47,13 @@ First boot sets everything up — the embedded Supermemory graph engine, local e Save that API key — it's your bearer token for every request. -In production, Supermemory runs proprietary models tuned for long-horizon data understanding. Self-hosted, you bring any model: if no provider key is set, first boot launches an interactive setup wizard — pick a provider (OpenAI, Anthropic, Gemini, Groq, or any OpenAI-compatible endpoint like Ollama), paste your key, and it's saved encrypted for every future launch. See [all providers](/self-hosting/configuration#llm-providers), including [fully-offline local models](/self-hosting/configuration#fully-offline-with-local-models). +In production, Supermemory runs proprietary models tuned for long-horizon data understanding. Self-hosted, you bring any model: if no provider key is set, first boot launches an interactive setup wizard — pick a provider (OpenAI, Anthropic, Gemini, Groq, or any OpenAI-compatible endpoint like Ollama), paste your key, and it's saved encrypted for every future launch. After the LLM key, you can optionally pick an embedding model (press Enter to keep local `Xenova/bge-base-en-v1.5`). See [all providers](/self-hosting/configuration#llm-providers), [embeddings](/self-hosting/embeddings), and [fully-offline local models](/self-hosting/configuration#fully-offline-with-local-models). + +**Docker / non-interactive:** set an LLM key via env and, if you don’t want local embeddings, set `SUPERMEMORY_EMBEDDING_PROVIDER` / `MODEL` / `DIMENSIONS`. There is no wizard without a TTY. + + ## Add your first memory @@ -141,10 +145,13 @@ By default, all state lives in a single directory you can back up or move: ## Next steps - + LLM providers, local models, performance tuning + + Local default, OpenAI / Gemini / Ollama, multilingual + The full API — it all works against your local server