From ce7f45e18d8dceedbcecffad83e5ae23ca105149 Mon Sep 17 00:00:00 2001 From: TuZaaaaa <70996583+TuZaaaaa@users.noreply.github.com> Date: Mon, 1 Jun 2026 17:32:33 +0800 Subject: [PATCH] docs: document embeddings node limit options (#1961) Co-authored-by: wujuntong_a --- README.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a5e04e728..e6f50950a 100644 --- a/README.md +++ b/README.md @@ -219,7 +219,7 @@ gitnexus analyze --skills # Generate repo-specific skill files from detec gitnexus analyze --skip-embeddings # Skip embedding generation (faster) gitnexus analyze --skip-agents-md # Preserve custom AGENTS.md/CLAUDE.md gitnexus section edits gitnexus analyze --skip-git # Index folders that are not Git repositories -gitnexus analyze --embeddings # Enable embedding generation (slower, better search) +gitnexus analyze --embeddings [limit] # Enable embedding generation (slower, better search) gitnexus analyze --verbose # Log skipped files when parsers are unavailable gitnexus analyze --worker-timeout 60 # Increase worker idle timeout for slow parses gitnexus analyze --wal-checkpoint-threshold 67108864 # 64 MiB. Control LadybugDB WAL auto-checkpoint threshold (default: 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB) @@ -248,6 +248,23 @@ gitnexus group status # Check staleness of repos in a group If `analyze` reports a worker parse timeout on a large or unusual repository, it keeps running and falls back safely. To give slow worker jobs more time, use `gitnexus analyze --worker-timeout 60` or set `GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=60000`. For very large files, `GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES` controls the worker job byte budget. +#### Embeddings node limit + +`gitnexus analyze --embeddings` generates semantic search vectors with a default 50,000-node safety cap to protect memory on large repositories. Override the cap when you know the host has enough memory for a larger graph, or disable it entirely for a one-off full embeddings run. + +```bash +# Generate embeddings with the default 50,000 node safety cap +gitnexus analyze --embeddings + +# Disable the safety cap entirely +gitnexus analyze --embeddings 0 + +# Use a custom cap +gitnexus analyze --embeddings 100000 +``` + +If embeddings are skipped on a large repository, the indexed graph likely exceeds the default safety cap. Re-run with `gitnexus analyze --embeddings 0` to remove the cap, or `gitnexus analyze --embeddings ` to choose a higher limit while still keeping memory bounded. + #### Environment variables Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max-file-size`, `--verbose`). Use the env-var form when you'd otherwise repeat the same flag every run, or when invoking GitNexus from a long-running host (MCP server, eval-server, CI shell) that already manages its own environment. CLI flags take precedence over env vars; env vars take precedence over built-in defaults.