scriptoriumadmin/GitNexus
1
0
Fork 0
mirror of https://github.com/abhigyanpatwari/GitNexus.git synced 2026-11-15 09:11:33 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 7

Merge branch 'main' into perf/lexical-lookup-pr

Browse source
This commit is contained in:
Gergő Magyar 2026-11-04 08:05:53 +00:00 • committed by GitHub
commit cd879f43f2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 843 additions and 135 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/cli-reference.md
View file

@ -233,6 +233,13 @@ Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max
| `GITNEXUS_QUERY_MAX_RSS_MB` | max(8192 MiB, 25% of available RAM), capped at 75% of available RAM | Process-memory ceiling while pooled reads run; available RAM is the smaller of physical RAM and a reported runtime/container constraint. Positive MiB values override it; `0` disables the guard. Invalid, overflowing, or sub-byte values use the default. A breach exits the process with code 70. | Set in the CLI/MCP/serve environment when a legitimate read workload needs a different budget. It does not resize the database buffer pool. |
| `GITNEXUS_QUERY_TRACE` | unset | `1` writes query duration, row count, and sampled memory to stderr. Bound parameters are omitted; query text is included. | Diagnose an expensive read or a memory-guard exit. Inspect trace output before sharing it. |
For local model setup and switching instructions, see [Local Embedding Model](../gitnexus/README.md#local-embedding-model).
| Local embedding variable | Default | Effect | When to tune |
| ----------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `GITNEXUS_EMBEDDING_MODEL_ID` | `Snowflake/snowflake-arctic-embed-xs` | Selects the local transformers.js model. Trims whitespace; unset or blank uses the default. HTTP model selection uses `GITNEXUS_EMBEDDING_MODEL` instead. | Use another ONNX-compatible Hugging Face model. Set the same model in analyze and query/server environments, restart servers, and run `analyze --force --embeddings` when switching. |
| `GITNEXUS_EMBEDDING_DIMS` | `384` | Width of the stored vector column; also the expected output width for a custom local model. | Match the selected model's output width. Unset when returning to the default model; use the same width for indexing and queries. |
| Analyze configuration | Default | Effect | When to tune |
| --- | --- | --- | --- |
| `GITNEXUS_RC` | unset | Absolute path to a JSON file with the same keys as `.gitnexusrc`. Its keys override repository configuration; CLI flags still win. Missing, invalid, or oversized files fail analysis. | Keep local configuration outside the checkout. Re-indexing hooks and watch processes must inherit it too. Changing or removing router aliases or handler wrappers rebuilds route evidence. |

14
gitnexus/README.md
View file

@ -432,6 +432,20 @@ Like Actuator snapshots, documents are external to git freshness — replacing o
> **`gitnexus uninstall`** removes the named GitNexus MCP entries, hooks, and skill directories from detected editors. This includes a URL-based MCP entry you configured yourself, even if setup preserved it. Skill directories are identified **by bundled gitnexus skill name** (e.g. `gitnexus-cli/`), so if you customized files inside an installed skill directory, back them up first. It is a dry-run preview by default and prints the exact paths it would remove; pass `--force` to apply. Per-repo indexes (`gitnexus clean --all`) and the global npm package (`npm uninstall -g gitnexus`) are left for you to remove.
## Local Embedding Model
Local embeddings use `Snowflake/snowflake-arctic-embed-xs` (384 dimensions). To use another Hugging Face model with ONNX weights for transformers.js, set its id and its vector size:
```bash
export GITNEXUS_EMBEDDING_MODEL_ID=Xenova/bge-base-en-v1.5
export GITNEXUS_EMBEDDING_DIMS=768 # required unless the model returns 384-dimensional vectors
gitnexus analyze . --force --embeddings
```
Set the same two variables wherever the index is queried (MCP server, `serve`, `query`), and restart running servers after changing them. A server using another model of the same width cannot detect the mismatch and will return unreliable semantic results. A model that returns a different vector size than `GITNEXUS_EMBEDDING_DIMS` stops the run with the size to set. Local models use mean pooling and normalization; model-specific query prefixes and pooling settings are not configurable.
When changing models, run `gitnexus analyze . --force --embeddings`. A plain `analyze`, even with `--embeddings`, can return "Already up to date" on unchanged source; without `--embeddings`, changed-source runs normally only preserve existing vectors. The forced embedding pass replaces vectors whose model changed, including switches between models of the same width. To return to the default model, unset both `GITNEXUS_EMBEDDING_MODEL_ID` and `GITNEXUS_EMBEDDING_DIMS`, then run the same command. For repositories above the default 50,000-node embedding cap, use `--embeddings 0` (no cap) or an explicit higher limit.
## Remote Embeddings
Set these env vars to use a remote OpenAI-compatible `/v1/embeddings` endpoint instead of the local model:

21
gitnexus/src/cli/analyze.ts
View file

@ -74,6 +74,7 @@ import {
import { EMBEDDING_DIMS_ERROR, normalizeEmbeddingDims } from './embedding-dims.js';
import { formatElapsed } from './format-elapsed.js';
import { isHfDownloadFailure } from '../core/embeddings/hf-env.js';
import { LocalEmbeddingDimensionError } from '../core/embeddings/config.js';
import {
isHttpEmbeddingDimsError,
isHttpEmbeddingError,
@ -1899,11 +1900,12 @@ const analyzeCommandImpl = async (
}
// An extracted edge whose FROM→TO label pair is missing from GitNexus's own
// relation DDL (#2789). `assertDeclaredPair` aborts the run rather than let
// the bulk COPY fail late and silently drop the edge, so the user sees a
// mid-run crash inside GitNexus internals with nothing to act on. Name the
// pair, the relationship and the file that produced it, and say plainly that
// a re-run cannot help — this is deterministic for the same input.
// relation DDL (#2789). `checkDeclaredPair` fails the run, with every such
// pair (#2797), rather than let the bulk COPY fail late and silently drop the
// edge, so the user sees a failure inside GitNexus internals with nothing to
// act on. Name the pair, the relationship and the file that produced it, and
// say plainly that a re-run cannot help — this is deterministic for the same
// input.
// Checked by TYPE (repo norm, #2385) BEFORE the message-text heuristics
// below, and through the `cause` chain because the ingestion phase runner
// rewraps every phase failure as `Phase 'X' failed: …`.
@ -1921,6 +1923,7 @@ const analyzeCommandImpl = async (
cliError(` ${undeclaredPair.message.replace(/\n/g, '\n ')}\n`, {
recoveryHint: 'undeclared-relation-pair',
labelPair: undeclaredPair.pairKey,
labelPairs: undeclaredPair.pairs.map((pair) => pair.pairKey),
relationType: undeclaredPair.relationType,
sourceFile: undeclaredPair.sourceFile,
});
@ -2070,6 +2073,14 @@ const analyzeCommandImpl = async (
return;
}
// A custom local model returned another width than GITNEXUS_EMBEDDING_DIMS:
// the message names the value to set, so no stack dump.
if (err instanceof LocalEmbeddingDimensionError) {
cliError(` ${msg}\n`, { recoveryHint: 'embedding-dims-invalid' });
process.exitCode = 1;
return;
}
// Custom HTTP embedding endpoint failure (#2385). When a `--embedding-base-url`
// is configured, HTTP mode never downloads a model — so a failure talking to
// that endpoint must NOT show the huggingface-download guidance. Keyed on the

2
gitnexus/src/cli/i18n/en.ts
View file

@ -434,5 +434,5 @@ export const en = {
'help.identityCache.environment':
'\nAnalyzer identity cache:\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir\n Operator-trusted persistent cache for warm cross-process status. The directory must pre-exist, be outside the GitNexus package/build roots, and contain no symlink or junction components. Defaults remain fail-closed on platforms without POSIX ownership APIs.',
'help.analyze.environment':
'\nEnvironment variables:\n GITNEXUS_NO_GITIGNORE=1 Skip .gitignore parsing (still reads .gitnexusignore)\n GITNEXUS_MAX_FILE_SIZE=N Override large-file skip threshold (KB). Default 512, max 32768.\n GITNEXUS_STORAGE_PATH=/absolute/index Complete external index directory. Preserves the existing configuration semantics and overrides GITNEXUS_STORAGE_ROOT when both are set.\n GITNEXUS_STORAGE_ROOT=/absolute/root External index root; each repository uses an isolated <repo-basename>-<canonical-path-hash>/ slot.\n GITNEXUS_CONTENT_RETENTION=full Source-text retention profile: full, symbol, or none. Default full.\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir Operator-trusted persistent analyzer identity cache; must pre-exist, be outside package/build roots, and contain no symlink/junction components.\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker idle timeout in milliseconds. Default 30000.\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL auto-checkpoint threshold in bytes (default 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB).\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker job byte budget. Default 8388608.\n GITNEXUS_WORKER_POOL_SIZE=N Parse worker count override. Default cores-1 capped at 16.\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N Concurrent in-flight parse chunks. Default 2.\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N Max replacement spawns per slot before drop. Default 3.\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N Total retry wall-time per job. Default 5x sub-batch timeout.\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N Per-slot deaths to trip circuit breaker. Default max(3, poolSize).\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N Max wait at pool shutdown for a retired worker still inside native code (terminated at its next safe point instead of aborting the process). Default 30000.\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N Per-file wall-clock budget for C++ capture extraction; on breach the file keeps partial captures with a warning. Default 20000.\n GITNEXUS_EMBEDDING_THREADS=N Limit local ONNX CPU threads for --embeddings.\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 Retry per-attempt HTTP embedding timeouts through GITNEXUS_EMBEDDING_MAX_ATTEMPTS (default off; timeouts stay terminal).\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N Max embedding chunks for exact-scan fallback. Default 10000.\n GITNEXUS_VECTOR_MAX_DISTANCE=N Max accepted semantic/vector cosine distance (0 < N <= 2; higher values clamp to 2). Default 0.6 for CLI/MCP query, 0.5 for standalone semantic search. Tune for your embedding model: higher values can improve recall but may reduce relevance. Set in the query/server environment; no reindex needed.\n GITNEXUS_MAX_PROCESSES=N Process-detection process cap (positive integer). Replaces the dynamic max(20, round(symbols/10)) formula. Distinct from query-time IMPACT_MAX_CHUNKS.\n GITNEXUS_MAX_PROCESS_BRANCHING=N Process-detection per-node branching cap. Default 4.\n GITNEXUS_MAX_PROCESS_TRACE_DEPTH=N Process-detection DFS depth cap. Default 10.\n GITNEXUS_MAX_ENTRY_POINT_CANDIDATES=N Ranked entry-point candidate pool. Default 200. Raise when the warning names this knob; doubling is the usual first raise.\n\nCLI flags take precedence over `.gitnexusrc`, which takes precedence over env vars, which take precedence over built-in defaults.\n\nTip: `.gitnexusignore` supports `.gitignore`-style negation. Add e.g.\n `!__tests__/` to index a directory that is auto-filtered by default (#771).',
'\nEnvironment variables:\n GITNEXUS_NO_GITIGNORE=1 Skip .gitignore parsing (still reads .gitnexusignore)\n GITNEXUS_MAX_FILE_SIZE=N Override large-file skip threshold (KB). Default 512, max 32768.\n GITNEXUS_STORAGE_PATH=/absolute/index Complete external index directory. Preserves the existing configuration semantics and overrides GITNEXUS_STORAGE_ROOT when both are set.\n GITNEXUS_STORAGE_ROOT=/absolute/root External index root; each repository uses an isolated <repo-basename>-<canonical-path-hash>/ slot.\n GITNEXUS_CONTENT_RETENTION=full Source-text retention profile: full, symbol, or none. Default full.\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir Operator-trusted persistent analyzer identity cache; must pre-exist, be outside package/build roots, and contain no symlink/junction components.\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker idle timeout in milliseconds. Default 30000.\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL auto-checkpoint threshold in bytes (default 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB).\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker job byte budget. Default 8388608.\n GITNEXUS_WORKER_POOL_SIZE=N Parse worker count override. Default cores-1 capped at 16.\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N Concurrent in-flight parse chunks. Default 2.\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N Max replacement spawns per slot before drop. Default 3.\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N Total retry wall-time per job. Default 5x sub-batch timeout.\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N Per-slot deaths to trip circuit breaker. Default max(3, poolSize).\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N Max wait at pool shutdown for a retired worker still inside native code (terminated at its next safe point instead of aborting the process). Default 30000.\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N Per-file wall-clock budget for C++ capture extraction; on breach the file keeps partial captures with a warning. Default 20000.\n GITNEXUS_EMBEDDING_THREADS=N Limit local ONNX CPU threads for --embeddings.\n GITNEXUS_EMBEDDING_MODEL_ID=id Local Hugging Face embedding model. Default Snowflake/snowflake-arctic-embed-xs; set GITNEXUS_EMBEDDING_DIMS to its vector size.\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 Retry per-attempt HTTP embedding timeouts through GITNEXUS_EMBEDDING_MAX_ATTEMPTS (default off; timeouts stay terminal).\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N Max embedding chunks for exact-scan fallback. Default 10000.\n GITNEXUS_VECTOR_MAX_DISTANCE=N Max accepted semantic/vector cosine distance (0 < N <= 2; higher values clamp to 2). Default 0.6 for CLI/MCP query, 0.5 for standalone semantic search. Tune for your embedding model: higher values can improve recall but may reduce relevance. Set in the query/server environment; no reindex needed.\n GITNEXUS_MAX_PROCESSES=N Process-detection process cap (positive integer). Replaces the dynamic max(20, round(symbols/10)) formula. Distinct from query-time IMPACT_MAX_CHUNKS.\n GITNEXUS_MAX_PROCESS_BRANCHING=N Process-detection per-node branching cap. Default 4.\n GITNEXUS_MAX_PROCESS_TRACE_DEPTH=N Process-detection DFS depth cap. Default 10.\n GITNEXUS_MAX_ENTRY_POINT_CANDIDATES=N Ranked entry-point candidate pool. Default 200. Raise when the warning names this knob; doubling is the usual first raise.\n\nCLI flags take precedence over `.gitnexusrc`, which takes precedence over env vars, which take precedence over built-in defaults.\n\nTip: `.gitnexusignore` supports `.gitignore`-style negation. Add e.g.\n `!__tests__/` to index a directory that is auto-filtered by default (#771).',
} as const;

2
gitnexus/src/cli/i18n/zh-CN.ts
View file

@ -395,5 +395,5 @@ export const zhCN = {
'help.identityCache.environment':
'\n分析器身份缓存：\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir\n 由操作员明确信任的持久缓存，用于跨进程快速查询状态。目录必须预先存在、位于 GitNexus 包/构建根目录之外，且路径中不得包含符号链接或 junction。缺少 POSIX 所有权 API 的平台默认保持故障关闭。',
'help.analyze.environment':
'\n环境变量：\n GITNEXUS_NO_GITIGNORE=1 跳过 .gitignore 解析（仍读取 .gitnexusignore）\n GITNEXUS_MAX_FILE_SIZE=N 覆盖大文件跳过阈值（KB）。默认 512，最大 32768。\n GITNEXUS_STORAGE_PATH=/absolute/index 完整外部索引目录。保留既有配置语义；与 GITNEXUS_STORAGE_ROOT 同时设置时优先使用。\n GITNEXUS_STORAGE_ROOT=/absolute/root 外部索引根目录；每个仓库使用独立的 <仓库名>-<规范路径哈希>/ 子目录。\n GITNEXUS_CONTENT_RETENTION=full 源码文本保留策略：full、symbol 或 none。默认 full。\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir 由操作员明确信任的持久分析器身份缓存；目录必须预先存在、位于包/构建根目录之外，且路径中不得包含符号链接或 junction。\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker 空闲超时（毫秒）。默认 30000。\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL 自动 checkpoint 阈值（字节，默认 67108864 = 64 MiB；-1 保持 Ladybug 默认约 16 MiB）。\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker 作业字节预算。默认 8388608。\n GITNEXUS_WORKER_POOL_SIZE=N 解析 worker 数量覆盖值。默认 cores-1，最多 16。\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N 并发进行中的解析分块数。默认 2。\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N 每个 slot 丢弃前允许的最大替换进程数。默认 3。\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N 每个作业的总重试墙钟时间。默认 5 倍子批次超时。\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N 每个 slot 触发熔断的死亡次数。默认 max(3, poolSize)。\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N 线程池关闭时等待仍在原生代码中的已退役 worker 的最长时间（到达安全点后再终止，避免进程级 abort）。默认 30000。\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N C++ 捕获提取的每文件墙钟预算；超出后该文件保留部分捕获并输出警告。默认 20000。\n GITNEXUS_EMBEDDING_THREADS=N 限制 --embeddings 的本地 ONNX CPU 线程数。\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 将单次 HTTP 嵌入超时纳入 GITNEXUS_EMBEDDING_MAX_ATTEMPTS 重试（默认关闭，超时仍为终止错误）。\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N exact-scan 回退的最大嵌入分块数。默认 10000。\n GITNEXUS_VECTOR_MAX_DISTANCE=N 语义/向量搜索接受的最大余弦距离（0 < N <= 2；超出则钳制为 2）。CLI/MCP query 默认 0.6，独立语义搜索默认 0.5。应根据嵌入模型调节：提高阈值可增加召回，但可能降低相关性。在查询/服务器进程的环境中设置，无需重新索引。\n GITNEXUS_MAX_PROCESSES=N 流程检测的流程数量上限（正整数）。覆盖动态的 max(20, round(symbols/10)) 公式。与查询时的 IMPACT_MAX_CHUNKS 无关。\n GITNEXUS_MAX_PROCESS_BRANCHING=N 流程检测的单节点分支上限。默认 4。\n GITNEXUS_MAX_PROCESS_TRACE_DEPTH=N 流程检测的 DFS 深度上限。默认 10。\n GITNEXUS_MAX_ENTRY_POINT_CANDIDATES=N 排序后的入口点候选池。默认 200。仅在警告点名该上限时提高；那时通常先翻倍。\n\nCLI 参数优先于 `.gitnexusrc`，后者优先于环境变量，环境变量优先于内置默认值。\n\n提示：`.gitnexusignore` 支持 `.gitignore` 风格的取反。比如添加\n `!__tests__/` 可以索引默认自动过滤的目录（#771）。',
'\n环境变量：\n GITNEXUS_NO_GITIGNORE=1 跳过 .gitignore 解析（仍读取 .gitnexusignore）\n GITNEXUS_MAX_FILE_SIZE=N 覆盖大文件跳过阈值（KB）。默认 512，最大 32768。\n GITNEXUS_STORAGE_PATH=/absolute/index 完整外部索引目录。保留既有配置语义；与 GITNEXUS_STORAGE_ROOT 同时设置时优先使用。\n GITNEXUS_STORAGE_ROOT=/absolute/root 外部索引根目录；每个仓库使用独立的 <仓库名>-<规范路径哈希>/ 子目录。\n GITNEXUS_CONTENT_RETENTION=full 源码文本保留策略：full、symbol 或 none。默认 full。\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir 由操作员明确信任的持久分析器身份缓存；目录必须预先存在、位于包/构建根目录之外，且路径中不得包含符号链接或 junction。\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker 空闲超时（毫秒）。默认 30000。\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL 自动 checkpoint 阈值（字节，默认 67108864 = 64 MiB；-1 保持 Ladybug 默认约 16 MiB）。\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker 作业字节预算。默认 8388608。\n GITNEXUS_WORKER_POOL_SIZE=N 解析 worker 数量覆盖值。默认 cores-1，最多 16。\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N 并发进行中的解析分块数。默认 2。\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N 每个 slot 丢弃前允许的最大替换进程数。默认 3。\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N 每个作业的总重试墙钟时间。默认 5 倍子批次超时。\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N 每个 slot 触发熔断的死亡次数。默认 max(3, poolSize)。\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N 线程池关闭时等待仍在原生代码中的已退役 worker 的最长时间（到达安全点后再终止，避免进程级 abort）。默认 30000。\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N C++ 捕获提取的每文件墙钟预算；超出后该文件保留部分捕获并输出警告。默认 20000。\n GITNEXUS_EMBEDDING_THREADS=N 限制 --embeddings 的本地 ONNX CPU 线程数。\n GITNEXUS_EMBEDDING_MODEL_ID=id 本地 Hugging Face 嵌入模型。默认 Snowflake/snowflake-arctic-embed-xs；请将 GITNEXUS_EMBEDDING_DIMS 设为其向量维度。\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 将单次 HTTP 嵌入超时纳入 GITNEXUS_EMBEDDING_MAX_ATTEMPTS 重试（默认关闭，超时仍为终止错误）。\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N exact-scan 回退的最大嵌入分块数。默认 10000。\n GITNEXUS_VECTOR_MAX_DISTANCE=N 语义/向量搜索接受的最大余弦距离（0 < N <= 2；超出则钳制为 2）。CLI/MCP query 默认 0.6，独立语义搜索默认 0.5。应根据嵌入模型调节：提高阈值可增加召回，但可能降低相关性。在查询/服务器进程的环境中设置，无需重新索引。\n GITNEXUS_MAX_PROCESSES=N 流程检测的流程数量上限（正整数）。覆盖动态的 max(20, round(symbols/10)) 公式。与查询时的 IMPACT_MAX_CHUNKS 无关。\n GITNEXUS_MAX_PROCESS_BRANCHING=N 流程检测的单节点分支上限。默认 4。\n GITNEXUS_MAX_PROCESS_TRACE_DEPTH=N 流程检测的 DFS 深度上限。默认 10。\n GITNEXUS_MAX_ENTRY_POINT_CANDIDATES=N 排序后的入口点候选池。默认 200。仅在警告点名该上限时提高；那时通常先翻倍。\n\nCLI 参数优先于 `.gitnexusrc`，后者优先于环境变量，环境变量优先于内置默认值。\n\n提示：`.gitnexusignore` 支持 `.gitignore` 风格的取反。比如添加\n `!__tests__/` 可以索引默认自动过滤的目录（#771）。',
} satisfies EnglishMessages;

46
gitnexus/src/core/embeddings/config.ts
View file

@ -71,6 +71,48 @@ const parseDevice = (value: string | undefined): EmbeddingConfig['device'] | und
throw new Error(`embedding device must be one of auto, dml, cuda, cpu, wasm; got "${value}"`);
};
/**
* Resolve the local (transformers.js) model id. Reads `GITNEXUS_EMBEDDING_MODEL_ID`;
* unset or blank falls back to the explicit config, then the built-in default.
* HTTP mode names its model with `GITNEXUS_EMBEDDING_MODEL` instead.
*/
const resolveLocalModelId = (value: string | undefined, fallback: string): string => {
const trimmed = value?.trim();
return trimmed ? trimmed : fallback;
};
/** The local model id from the environment alone, without resolving the rest of the config. */
export const resolveLocalEmbeddingModelId = (): string =>
resolveLocalModelId(process.env.GITNEXUS_EMBEDDING_MODEL_ID, DEFAULT_EMBEDDING_CONFIG.modelId);
/** A local model returned vectors of another width than the configured one. */
export class LocalEmbeddingDimensionError extends Error {
constructor(modelId: string, actual: number, expected: number) {
super(
`Embedding dimension mismatch: local model ${modelId} returned ${actual}d vectors, ` +
`but expected ${expected}d. Set GITNEXUS_EMBEDDING_DIMS=${actual} to match your model output.`,
);
this.name = 'LocalEmbeddingDimensionError';
}
}
/**
* Vector width of the local model. The built-in model is always
* {@link DEFAULT_EMBEDDING_CONFIG.dimensions}; a custom model takes
* `GITNEXUS_EMBEDDING_DIMS`, the same value the vector column is created with.
*/
export const resolveLocalEmbeddingDimensions = (): number => {
if (resolveLocalEmbeddingModelId() === DEFAULT_EMBEDDING_CONFIG.modelId) {
return DEFAULT_EMBEDDING_CONFIG.dimensions;
}
const raw = process.env.GITNEXUS_EMBEDDING_DIMS;
if (raw === undefined) return DEFAULT_EMBEDDING_CONFIG.dimensions;
if (!/^\d+$/.test(raw) || parseInt(raw, 10) <= 0) {
throw new Error(`GITNEXUS_EMBEDDING_DIMS must be a positive integer, got "${raw}"`);
}
return parseInt(raw, 10);
};
export const resolveEmbeddingConfig = (
overrides: Partial<EmbeddingConfig> = {},
): EmbeddingConfig => {
@ -86,6 +128,10 @@ export const resolveEmbeddingConfig = (
return {
...DEFAULT_EMBEDDING_CONFIG,
...overrides,
modelId: resolveLocalModelId(
env.GITNEXUS_EMBEDDING_MODEL_ID,
overrides.modelId ?? DEFAULT_EMBEDDING_CONFIG.modelId,
),
maxInputChars,
batchSize: parsePositiveInt(
'GITNEXUS_EMBEDDING_BATCH_SIZE',

23
gitnexus/src/core/embeddings/embedder.ts
View file

@ -19,6 +19,11 @@ import {
type EmbeddingRequestOptions,
} from './http-client.js';
import { assessLocalEmbeddingRuntime, getLocalEmbeddingRuntimeBlocker } from './runtime-support.js';
import {
LocalEmbeddingDimensionError,
resolveLocalEmbeddingDimensions,
resolveLocalEmbeddingModelId,
} from './config.js';
import {
ensureEmbeddingSidecar,
getSidecarDevice,
@ -83,7 +88,19 @@ export const getEmbeddingDimensions = (): number => {
if (isHttpMode()) {
return getHttpDimensions() ?? DEFAULT_EMBEDDING_CONFIG.dimensions;
}
return DEFAULT_EMBEDDING_CONFIG.dimensions;
return resolveLocalEmbeddingDimensions();
};
// A custom model fails before a wrong-width vector reaches the FLOAT[N] column, like the HTTP client.
const assertLocalDimensions = (vectors: Float32Array[]): Float32Array[] => {
const modelId = resolveLocalEmbeddingModelId();
if (modelId === DEFAULT_EMBEDDING_CONFIG.modelId) return vectors;
const expected = resolveLocalEmbeddingDimensions();
const wrong = vectors.find((vec) => vec.length !== expected);
if (wrong) {
throw new LocalEmbeddingDimensionError(modelId, wrong.length, expected);
}
return vectors;
};
export const embedText = async (
@ -101,7 +118,7 @@ export const embedText = async (
throw new Error(runtimeBlocker);
}
const [vec] = await sidecarEmbedBatch([text], options);
const [vec] = assertLocalDimensions(await sidecarEmbedBatch([text], options));
return vec;
};
@ -123,7 +140,7 @@ export const embedBatch = async (
throw new Error(runtimeBlocker);
}
return sidecarEmbedBatch(texts, options);
return assertLocalDimensions(await sidecarEmbedBatch(texts, options));
};
export const embeddingToArray = (embedding: Float32Array): number[] => {

17
gitnexus/src/core/embeddings/embedding-pipeline.ts
View file

@ -38,6 +38,7 @@ import {
import {
DEFAULT_VECTOR_MAX_DISTANCE,
getVectorMaxDistance,
LocalEmbeddingDimensionError,
resolveEmbeddingConfig,
} from './config.js';
import { rankExactEmbeddingRows, type ExactEmbeddingRow } from './exact-search.js';
@ -106,11 +107,14 @@ export const contentHashForNode = (
node.content,
config,
);
return createHash('sha1')
const hash = createHash('sha1')
.update(EMBEDDING_TEXT_VERSION)
.update(`\n${config.maxInputChars ?? DEFAULT_EMBEDDING_CONFIG.maxInputChars}\n`)
.update(text)
.digest('hex');
.update(`\n${config.maxInputChars ?? DEFAULT_EMBEDDING_CONFIG.maxInputChars}\n`);
// Only a non-default model joins the hash, so default-model hashes stay unchanged.
if (config.modelId !== undefined && config.modelId !== DEFAULT_EMBEDDING_CONFIG.modelId) {
hash.update(`model:${config.modelId}\n`);
}
return hash.update(text).digest('hex');
};
const isInputRejection = (error: unknown): boolean =>
@ -913,6 +917,11 @@ export const runEmbeddingPipeline = async (
// keep going; the cleanup DELETE after this loop makes the drop safe.
const droppedNodeIds = new Set(subUpdates.map((u) => u.nodeId));
for (const nodeId of droppedNodeIds) batchFailedNodeIds.add(nodeId);
// A wrong vector width is configuration: every later sub-batch would fail the same way.
if (embedErr instanceof LocalEmbeddingDimensionError) {
abortError = { err: embedErr };
break;
}
if (isInputRejection(embedErr)) {
// Bad inputs are pending work, not evidence of an endpoint outage.
// Do not let many rejected chunks starve later healthy nodes.

2
gitnexus/src/core/ingestion/scope-resolution/graph-bridge/node-lookup.ts
View file

@ -272,7 +272,7 @@ export function buildGraphNodeLookup(
*
* That makes it load-bearing for the LadybugDB relation DDL: a label added here
* without the matching `FROM x TO y` pairs in `RELATION_SCHEMA` crashes
* `analyze` at `assertDeclaredPair` on whichever codebase first emits the pair
* `analyze` at `checkDeclaredPair` on whichever codebase first emits the pair
* (#2792). `test/unit/schema-pair-coverage.test.ts` derives the required pairs
* from this set and fails in CI instead.
*/

26
gitnexus/src/core/lbug/graph-emit-sink.ts
View file

@ -99,7 +99,8 @@ import type { KnowledgeGraph } from '../graph/types.js';
import { DECLARED_RELATION_PAIRS, REL_CSV_HEADER, buildRelRow } from './csv-generator.js';
import {
VALID_NODE_TABLES,
assertDeclaredPair,
checkDeclaredPair,
UndeclaredPairCollector,
relPairKeyFor,
splitRelPairKey,
} from './rel-pair-routing.js';
@ -331,6 +332,8 @@ export class GraphEmitSink implements KnowledgeGraph, GraphEmitControl {
* swallowed by a caller's try/catch and silently drop the rest of the rows.
*/
private openFailure: unknown | undefined = undefined;
/** Undeclared pairs met while streaming; {@link finalize} fails with all of them. */
private readonly undeclared = new UndeclaredPairCollector();
constructor(
private readonly real: KnowledgeGraph,
@ -447,13 +450,18 @@ export class GraphEmitSink implements KnowledgeGraph, GraphEmitControl {
const pairKey = relPairKeyFor(relationship.sourceId, relationship.targetId, VALID_NODE_TABLES);
if (pairKey === undefined) return;
assertDeclaredPair(
pairKey,
DECLARED_RELATION_PAIRS,
relationship.type,
relationship.sourceId,
relationship.targetId,
);
if (
!checkDeclaredPair(
pairKey,
DECLARED_RELATION_PAIRS,
this.undeclared,
relationship.type,
relationship.sourceId,
relationship.targetId,
)
) {
return;
}
let writer = this.relWriters.get(pairKey);
if (writer === undefined) {
// Cold: once per pair, so decoding the key back into its labels is free.
@ -530,6 +538,8 @@ export class GraphEmitSink implements KnowledgeGraph, GraphEmitControl {
);
}
this.undeclared.throwIfAny();
return { relsByPair, totalRows, structuralRows: this.structuralRows };
}

26
gitnexus/src/core/lbug/pdg-emit-sink.ts
View file

@ -56,7 +56,8 @@ import {
} from './csv-generator.js';
import {
VALID_NODE_TABLES,
assertDeclaredPair,
checkDeclaredPair,
UndeclaredPairCollector,
relPairKeyFor,
splitRelPairKey,
} from './rel-pair-routing.js';
@ -118,6 +119,8 @@ export class PdgEmitSink implements KnowledgeGraph {
* of that file's rows (#2202 review #4/#6).
*/
private openFailure: unknown | undefined = undefined;
/** Undeclared pairs met while streaming; {@link finalize} fails with all of them. */
private readonly undeclared = new UndeclaredPairCollector();
// NOTE on dedup: the same file can be PDG-emitted in more than one language
// pass (e.g. a `.ts` module imported by a `.vue` SFC is emitted in both the
// TypeScript pass and the Vue context pass over the same worker-built
@ -173,13 +176,18 @@ export class PdgEmitSink implements KnowledgeGraph {
VALID_NODE_TABLES,
);
if (pairKey === undefined) return;
assertDeclaredPair(
pairKey,
DECLARED_RELATION_PAIRS,
relationship.type,
relationship.sourceId,
relationship.targetId,
);
if (
!checkDeclaredPair(
pairKey,
DECLARED_RELATION_PAIRS,
this.undeclared,
relationship.type,
relationship.sourceId,
relationship.targetId,
)
) {
return;
}
let writer = this.relWriters.get(pairKey);
if (writer === undefined) {
// Cold: once per pair, so decoding the key back into labels is free.
@ -245,6 +253,8 @@ export class PdgEmitSink implements KnowledgeGraph {
);
}
this.undeclared.throwIfAny();
return { nodeFiles, relsByPair };
}

211
gitnexus/src/core/lbug/rel-pair-routing.ts
View file

@ -17,9 +17,9 @@
* regex over the ESCAPED row — so for an id containing a `"` the router is the
* more-correct path (it routes the edge to the right pair; the oracle's regex
* mis-buckets or drops it). Unlike the legacy oracle, the router also rejects a
* valid node-label pair absent from the relationship DDL. That distinction is
* load-bearing: silently writing such a CSV lets COPY fail late and can drop an
* otherwise valid edge during fallback. `splitRelCsvByLabelPair` is retained as
* valid node-label pair absent from the relationship DDL: it skips the edge,
* keeps routing, and fails at {@link RelPairRouter.close} with every such pair
* (see {@link checkDeclaredPair} for why the run fails rather than degrades). `splitRelCsvByLabelPair` is retained as
* the differential oracle (the quote-in-id divergence is asserted explicitly).
*
* Backpressure: at most one stream is awaited at a time (the caller routes
@ -111,7 +111,7 @@ export const relPairKeyFor = (
* built once per edge inside {@link relPairKeyFor} (~1M edges on a large
* repo), where a function call is a real regression risk; every decode site is
* cold by construction (once per pair when its CSV is opened, or on the
* throw path of {@link assertDeclaredPair}).
* failure path of {@link checkDeclaredPair}).
*/
export const splitRelPairKey = (key: string): readonly [from: string, to: string] => {
const sep = key.indexOf('|');
@ -179,26 +179,77 @@ const deriveNodeFilePath = (nodeId: string): string | undefined => {
const UNDECLARED_PAIR_ISSUE_URL = 'https://github.com/abhigyanpatwari/GitNexus/issues/new';
/**
* An edge whose endpoint-label pair is absent from the relationship DDL.
* One undeclared label pair seen during an emit: the first edge that hit it,
* kept as the example a bug report needs, and how many edges hit it in all.
*/
export interface UndeclaredPairOccurrence {
readonly pairKey: string;
readonly relationType: string;
readonly fromId: string;
readonly toId: string;
readonly edges: number;
}
const describePair = (
occurrence: Omit<UndeclaredPairOccurrence, 'edges'>,
indent: string,
): string => {
const sourceFile = deriveNodeFilePath(occurrence.fromId) ?? deriveNodeFilePath(occurrence.toId);
return (
`${indent}relationship type: ${occurrence.relationType}\n` +
`${indent}from node: ${occurrence.fromId}\n` +
`${indent}to node: ${occurrence.toId}\n` +
`${indent}source file: ${sourceFile ?? '(none — synthetic node id)'}\n`
);
};
const describeOnePair = (occurrence: Omit<UndeclaredPairOccurrence, 'edges'>): string => {
const [fromLabel, toLabel] = splitRelPairKey(occurrence.pairKey);
return (
`GitNexus extracted a relationship its own database schema cannot store.\n` +
`Relationship label pair ${fromLabel} → ${toLabel} is not declared in the ` +
`LadybugDB relation schema.\n` +
describePair(occurrence, ' ')
);
};
const describeSeveralPairs = (pairs: readonly UndeclaredPairOccurrence[]): string =>
`GitNexus extracted relationships its own database schema cannot store.\n` +
`${pairs.length} relationship label pairs are not declared in the LadybugDB ` +
`relation schema:\n` +
pairs
.map((pair) => {
const [fromLabel, toLabel] = splitRelPairKey(pair.pairKey);
const edges = `${pair.edges} edge${pair.edges === 1 ? '' : 's'}`;
return ` ${fromLabel} → ${toLabel} (${edges}, first one below)\n${describePair(pair, ' ')}`;
})
.join('');
/**
* Edges whose endpoint-label pair is absent from the relationship DDL.
*
* Carries the context the emit call site already has — relationship type, both
* node ids, and the source file derived from them — so a user whose `analyze`
* just died mid-run can see WHICH of their files produced the edge and file a
* bug report that names the missing pair. The abstract label pair alone is
* just failed can see WHICH of their files produced the edge and file a bug
* report that names the missing pair. The abstract label pair alone is
* unactionable outside GitNexus's own source (#2789).
*
* The emit paths collect every undeclared pair of a run through
* {@link UndeclaredPairCollector} and throw ONCE, so {@link pairs} lists all of
* them (#2797). The top-level fields describe the first one.
*
* Classify by TYPE (`err instanceof UndeclaredRelationPairError`, or
* {@link findUndeclaredRelationPairError} when the error may be wrapped in a
* phase `cause` chain) — the repo norm from #2385 — never by message text.
*
* THE MESSAGE IS THE ONLY RENDERING. It carries the five context fields AND
* the two actionable next steps (report the pair; `.gitnexusignore` the file to
* finish the rest of the index), because `gitnexus serve` forwards nothing but
* THE MESSAGE IS THE ONLY RENDERING. It carries the context fields AND the two
* actionable next steps (report the pair; `.gitnexusignore` the file to finish
* the rest of the index), because `gitnexus serve` forwards nothing but
* `err.message` over worker IPC — anything a consumer re-renders from the
* structured fields instead is invisible to a serve-hosted user. The CLI
* branch in `cli/analyze.ts` therefore prints this message indented and adds
* only the machine-readable `cliError` fields, the same idiom `LbugWipeError`
* uses there. It used to re-render the five fields with its own wording; the
* uses there. It used to re-render the context fields with its own wording; the
* two copies had already drifted on the pair separator, the no-file text and
* the closing sentence within a single PR, and each had its own pinning test.
*/
@ -211,26 +262,42 @@ export class UndeclaredRelationPairError extends Error {
readonly toId: string;
/** Source file derived from the node ids; `undefined` for synthetic ids. */
readonly sourceFile: string | undefined;
/** Every undeclared pair of the emit, the first one included. */
readonly pairs: readonly UndeclaredPairOccurrence[];
constructor(pairKey: string, relationType: string, fromId: string, toId: string) {
constructor(
pairKey: string,
relationType: string,
fromId: string,
toId: string,
pairs: readonly UndeclaredPairOccurrence[] = [
{ pairKey, relationType, fromId, toId, edges: 1 },
],
) {
const sourceFile = deriveNodeFilePath(fromId) ?? deriveNodeFilePath(toId);
const [fromLabel, toLabel] = splitRelPairKey(pairKey);
const several = pairs.length > 1;
const header = several
? describeSeveralPairs(pairs)
: describeOnePair({ pairKey, relationType, fromId, toId });
super(
`GitNexus extracted a relationship its own database schema cannot store.\n` +
`Relationship label pair ${fromLabel} → ${toLabel} is not declared in the ` +
`LadybugDB relation schema.\n` +
` relationship type: ${relationType}\n` +
` from node: ${fromId}\n` +
` to node: ${toId}\n` +
` source file: ${sourceFile ?? '(none — synthetic node id)'}\n` +
header +
`This is a gap in GitNexus's own relation schema, not a problem with the ` +
`analyzed code, and re-running the analysis will fail in exactly the same place.\n` +
`Suggestions:\n` +
` 1. Report the missing pair so it can be declared:\n` +
` ${UNDECLARED_PAIR_ISSUE_URL}\n` +
` Include the label pair, the relationship type, and the source file above.\n` +
` 2. To finish indexing the rest meanwhile, add that file (or its directory)\n` +
` to .gitnexusignore and re-run.`,
(several
? ` 1. Report the missing pairs so they can be declared:\n` +
` ${UNDECLARED_PAIR_ISSUE_URL}\n` +
` Include the label pairs, the relationship types, and the source files above.\n`
: ` 1. Report the missing pair so it can be declared:\n` +
` ${UNDECLARED_PAIR_ISSUE_URL}\n` +
` Include the label pair, the relationship type, and the source file above.\n`) +
(pairs.some((pair) => pair.edges > 1)
? ` 2. To work around this, start by adding the example files (or their directories)\n` +
` to .gitnexusignore and re-run. Only the first edge of each pair is shown;\n` +
` other files may contain the same pairs and need to be excluded too.`
: ` 2. To finish indexing the rest meanwhile, add ${
several ? 'those files (or their directories)' : 'that file (or its directory)'
}\n` + ` to .gitnexusignore and re-run.`),
);
this.name = 'UndeclaredRelationPairError';
this.pairKey = pairKey;
@ -238,6 +305,48 @@ export class UndeclaredRelationPairError extends Error {
this.fromId = fromId;
this.toId = toId;
this.sourceFile = sourceFile;
this.pairs = pairs;
}
}
/**
* Collects the undeclared pairs one emit path meets, so the run fails once with
* all of them instead of on the first (#2797). Each pair keeps its first edge as
* the example and counts the rest. Nothing is recorded, and nothing allocated
* per edge, for a declared pair: {@link checkDeclaredPair} only calls
* {@link UndeclaredPairCollector.record} once its `has` check has failed.
*/
export class UndeclaredPairCollector {
private readonly byPair = new Map<
string,
{ relationType: string; fromId: string; toId: string; edges: number }
>();
get size(): number {
return this.byPair.size;
}
record(pairKey: string, relationType: string, fromId: string, toId: string): void {
const seen = this.byPair.get(pairKey);
if (seen !== undefined) {
seen.edges++;
return;
}
this.byPair.set(pairKey, { relationType, fromId, toId, edges: 1 });
}
/** Throw one {@link UndeclaredRelationPairError} naming every recorded pair. */
throwIfAny(): void {
if (this.byPair.size === 0) return;
const pairs = Array.from(this.byPair, ([pairKey, seen]) => ({ pairKey, ...seen }));
const [first] = pairs;
throw new UndeclaredRelationPairError(
first.pairKey,
first.relationType,
first.fromId,
first.toId,
pairs,
);
}
}
@ -263,35 +372,41 @@ export const findUndeclaredRelationPairError = (
);
/**
* Fail fast on an endpoint-label pair absent from the relationship DDL, the
* same guard `RelPairRouter.route` applies to the whole-graph emit. Exported
* so the streamed sinks (`GraphEmitSink`, `PdgEmitSink`) can apply it too —
* without this, an undeclared pair on a streaming run reaches `COPY`, fails
* the bulk insert, and is silently dropped by the per-edge fallback instead
* of failing loudly like the non-streaming path does.
* Check an edge's endpoint-label pair against the relationship DDL, the same
* guard `RelPairRouter.route` applies to the whole-graph emit. Exported so the
* streamed sinks (`GraphEmitSink`, `PdgEmitSink`) apply it too — without it, an
* undeclared pair on a streaming run reaches `COPY`, fails the bulk insert, and
* is silently dropped by the per-edge fallback.
*
* An undeclared pair is recorded in `undeclared` and the caller skips the edge;
* the emit path then fails at its close/finalize with every pair it met, not
* just the first (#2797). Failing at all, rather than skipping and warning, is
* about graph completeness: a dropped `CALLS` edge makes `impact` answer wrong,
* which is worse than no index. That argument is relationship-type-scoped — a
* missing evidence edge such as `CONDITIONAL_ON` corrupts no query — so if the
* policy is ever relaxed, the split belongs by relation type, not globally.
*
* Takes the already-built `From|To` pairKey (from {@link relPairKeyFor})
* rather than the two labels — every caller needs that same key immediately
* after for its own Map/stream lookup, and this is on the per-edge hot path,
* so building it twice would be a needless allocation per edge. The error
* splits it back apart with {@link splitRelPairKey}, which only the throw path
* reaches. The edge context is passed POSITIONALLY for the same reason: a
* `{ relationType, fromId, toId }` context object would allocate on every
* edge, including the ~1M that never fail.
* so building it twice would be a needless allocation per edge. The edge
* context is passed POSITIONALLY for the same reason: a context object would
* allocate on every edge, including the ~1M that pass.
*
* The success path must stay allocation-free: no object literal, no template
* string, no closure, no `Error` constructed before the failure branch.
* The declared path must stay allocation-free: no object literal, no template
* string, no closure before the `has` check fails.
*/
export const assertDeclaredPair = (
export const checkDeclaredPair = (
pairKey: string,
declaredPairs: ReadonlySet<string>,
undeclared: UndeclaredPairCollector,
relationType: string,
fromId: string,
toId: string,
): void => {
if (!declaredPairs.has(pairKey)) {
throw new UndeclaredRelationPairError(pairKey, relationType, fromId, toId);
}
): boolean => {
if (declaredPairs.has(pairKey)) return true;
undeclared.record(pairKey, relationType, fromId, toId);
return false;
};
/**
@ -305,6 +420,8 @@ export class RelPairRouter {
private readonly streams = new Map<string, WriteStream>();
skipped = 0;
total = 0;
/** Undeclared pairs met so far; {@link close} fails with all of them. */
readonly undeclared = new UndeclaredPairCollector();
private streamError: Error | null = null;
private readonly abort = new AbortController();
@ -351,7 +468,9 @@ export class RelPairRouter {
return;
}
assertDeclaredPair(pairKey, this.declaredPairs, relType, fromId, toId);
if (!checkDeclaredPair(pairKey, this.declaredPairs, this.undeclared, relType, fromId, toId)) {
return;
}
const ws = this.streams.get(pairKey);
if (ws === undefined) {
// First edge for this pair: open the stream, write header + row.
@ -382,7 +501,8 @@ export class RelPairRouter {
}
}
/** Flush + close every pair stream. Rejects if any stream errored. */
/** Flush + close every pair stream. Rejects if any stream errored, then if
* any edge had an undeclared pair (all of them in one error). */
async close(): Promise<void> {
if (this.streamError) {
this.destroy();
@ -395,6 +515,7 @@ export class RelPairRouter {
}),
);
if (this.streamError) throw this.streamError;
this.undeclared.throwIfAny();
}
/** Tear down all streams (no flush) — used on the error path. */

2
gitnexus/src/core/lbug/schema.ts
View file

@ -647,7 +647,7 @@ export const STRUCTURAL_PAIR_DDL = ` FROM File TO Folder,
*
* Generated rather than hand-listed because in both families the endpoint
* labels are LOOKUP RESULTS, not literals at the emit site — so any pair drawn
* from the sets can reach `assertDeclaredPair`, and an undeclared one aborts
* from the sets can reach `checkDeclaredPair`, and an undeclared one fails
* `analyze` outright on whichever codebase happens to produce it. Every
* hand-listed fix so far declared only the pair in the stack trace and left the
* rest of its family missing: `Const→Method` (#2781), `Class→Variable` (#2792),

3
gitnexus/src/mcp/core/embedder.ts
View file

@ -28,7 +28,8 @@ export const embedQuery = async (query: string): Promise<number[]> => {
/**
* Query-vector width for CAST. HTTP uses GITNEXUS_EMBEDDING_DIMS when set;
* local is the model default. Do not bind this to schema EMBEDDING_DIMS.
* local uses 384 for the built-in model and GITNEXUS_EMBEDDING_DIMS for a custom
* GITNEXUS_EMBEDDING_MODEL_ID. Do not bind this to schema EMBEDDING_DIMS.
*/
export const getEmbeddingDims = (): number => getEmbeddingDimensions();

82
gitnexus/test/integration/embedding-identity-integrity.test.ts
View file

@ -14,6 +14,9 @@ import * as adapter from '../../src/core/lbug/lbug-adapter.js';
import { runFullAnalysis } from '../../src/core/run-analyze.js';
import { getStoragePaths, loadMeta } from '../../src/storage/repo-manager.js';
import { seedEmbeddingsForFiles, stampEmbeddingCount } from '../helpers/embedding-seed.js';
import * as sidecar from '../../src/core/embeddings/embedding-sidecar-client.js';
import * as runtimeSupport from '../../src/core/embeddings/runtime-support.js';
import { DEFAULT_EMBEDDING_CONFIG } from '../../src/core/embeddings/types.js';
describe('embedding identity integrity (#3569)', () => {
let tmp: TestDBHandle | undefined;
@ -21,9 +24,88 @@ describe('embedding identity integrity (#3569)', () => {
await adapter.closeLbug();
await tmp?.cleanup();
tmp = undefined;
vi.restoreAllMocks();
vi.unstubAllEnvs();
});
it('replaces same-width model vectors on forced analyze and reuses only matching hashes', async () => {
tmp = await createTempDir('gitnexus-test-embedding-identity-');
const repo = path.join(tmp.dbPath, 'repo');
await fs.mkdir(repo);
vi.stubEnv('GITNEXUS_HOME', path.join(tmp.dbPath, 'home'));
vi.stubEnv('GITNEXUS_SHARED_STORE', 'off');
vi.stubEnv('GITNEXUS_LBUG_EXTENSION_INSTALL', 'never');
vi.stubEnv('GITNEXUS_EMBEDDING_URL', '');
vi.stubEnv('GITNEXUS_EMBEDDING_MODEL', '');
vi.stubEnv('GITNEXUS_EMBEDDING_DIMS', String(EMBEDDING_DIMS));
vi.spyOn(runtimeSupport, 'getLocalEmbeddingRuntimeBlocker').mockReturnValue(null);
const init = vi.spyOn(sidecar, 'ensureEmbeddingSidecar').mockResolvedValue({ device: 'cpu' });
vi.spyOn(sidecar, 'reapEmbeddingSidecarAndWait').mockResolvedValue(undefined);
let marker = 0;
// Only inference is faked: analyze, cache restore, hashes and DB writes are real.
const embed = vi.spyOn(sidecar, 'sidecarEmbedBatch').mockImplementation(async (texts) =>
texts.map(() => {
const vector = new Float32Array(EMBEDDING_DIMS);
vector[marker] = 1;
return vector;
}),
);
await fs.writeFile(path.join(repo, 'main.ts'), 'export function main() { return 1; }\n');
await fs.writeFile(path.join(repo, 'util.ts'), 'export function util() { return 2; }\n');
const git = (...args: string[]) => execFileSync('git', args, { cwd: repo, stdio: 'pipe' });
git('init', '-q');
git('add', '.');
git('-c', 'user.name=test', '-c', 'user.email=test@test', 'commit', '-qm', 'fixture');
const options = {
force: true,
embeddings: true,
skipFts: true,
skipAgentsMd: true,
skipSkills: true,
};
const readRows = async () => {
await adapter.initLbug(getStoragePaths(repo).lbugPath, { skipFts: true });
try {
return await adapter.executeQuery(
'MATCH (e:CodeEmbedding) RETURN e.nodeId AS nodeId, e.contentHash AS hash, e.embedding AS vector ORDER BY nodeId',
);
} finally {
await adapter.closeLbug();
}
};
let previousHashes: string[] = [];
let defaultHashes: string[] = [];
let nodeIds: string[] = [];
for (const modelId of ['', 'fixture/model-a', 'fixture/model-b', '']) {
vi.stubEnv('GITNEXUS_EMBEDDING_MODEL_ID', modelId);
embed.mockClear();
await runFullAnalysis(repo, options, { onProgress: () => {} });
expect(init.mock.lastCall?.[0]?.embeddingConfig?.modelId).toBe(
modelId || DEFAULT_EMBEDDING_CONFIG.modelId,
);
const rows = await readRows();
expect(rows).toHaveLength(2);
expect(embed.mock.calls.flatMap(([texts]) => texts)).toHaveLength(2);
expect(rows.map((row) => row.vector[marker])).toEqual([1, 1]);
const hashes = rows.map((row) => row.hash);
if (marker === 0) {
defaultHashes = hashes;
nodeIds = rows.map((row) => row.nodeId);
} else {
expect(rows.map((row) => row.nodeId)).toEqual(nodeIds);
hashes.forEach((hash, index) => expect(hash).not.toBe(previousHashes[index]));
if (modelId === '') expect(hashes).toEqual(defaultHashes);
}
previousHashes = hashes;
embed.mockClear();
await runFullAnalysis(repo, options, { onProgress: () => {} });
expect(embed).not.toHaveBeenCalled();
expect(await readRows()).toEqual(rows);
marker += 1;
}
}, 180_000);
it.each([
'',
' ',

4
gitnexus/test/integration/structural-pair-coverage.test.ts
View file

@ -199,7 +199,7 @@ const OPTIONAL_GRAMMAR_CORPUS = [
// MEMBER_OWNER_NODE_TYPES, so the definition phase emits HAS_PROPERTY /
// HAS_METHOD FROM a `Union` node. Shipped once with `Union` off the
// generated grid — every resolver test passed on the in-memory graph while
// a real `analyze` of THIS fixture aborted at `assertDeclaredPair`
// a real `analyze` of THIS fixture aborted at `checkDeclaredPair`
// (`Union|Property`). `Union` is now in `LINKABLE_LABELS`; take it out and
// both sentinels vanish from the DDL and this fails.
fixture: 'zig-basic',
@ -273,7 +273,7 @@ afterAll(() => vi.unstubAllEnvs());
*
* Classifies through `relPairKeyFor` — the same call the emit path routes with
* — so this guard and the router cannot disagree about which edges are skipped.
* It keys off the node IDS, which is what `assertDeclaredPair` actually sees,
* It keys off the node IDS, which is what `checkDeclaredPair` actually sees,
* not the node's `label` field.
*/
const pairsEmittedBy = async (fixture: string): Promise<Set<string>> => {

23
gitnexus/test/unit/analyze-local-embedding-error.test.ts
View file

@ -163,6 +163,29 @@ describe('analyzeCommand local-embedding-runtime error handling', () => {
});
});
it('prints a local model width mismatch as one config message, without a stack dump', async () => {
const { LocalEmbeddingDimensionError } = await import('../../src/core/embeddings/config.js');
const error = new LocalEmbeddingDimensionError('Xenova/bge-base-en-v1.5', 768, 384);
error.stack = 'local-dimension-error-stack-marker';
runFullAnalysisMock.mockRejectedValue(error);
await withCapturedLogger(async (cap) => {
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, { embeddings: true });
expect(process.exitCode).toBe(1);
const records = cap.records();
const dimsRecords = records.filter((r) => r.recoveryHint === 'embedding-dims-invalid');
expect(dimsRecords).toHaveLength(1);
expect(dimsRecords[0]?.msg).toContain('Set GITNEXUS_EMBEDDING_DIMS=768');
expect(JSON.stringify(records)).not.toContain(error.stack);
expect(
records.some((r) => typeof r.msg === 'string' && r.msg.includes('Analysis failed')),
).toBe(false);
});
});
it('does NOT route unrelated errors through the local-embedding branch', async () => {
runFullAnalysisMock.mockRejectedValue(
new Error('Some unexpected failure unrelated to embeddings'),

48
gitnexus/test/unit/analyze-undeclared-pair-error.test.ts
View file

@ -2,7 +2,7 @@
* Tests for the undeclared FROM→TO label-pair failure path in the
* `analyzeCommand` CLI (#2789).
*
* `assertDeclaredPair` aborts the run when an extracted edge's endpoint-label
* `checkDeclaredPair` fails the run when an extracted edge's endpoint-label
* pair is missing from GitNexus's own relation DDL — deliberately, because the
* alternative is a late `COPY` failure that silently drops edges. Before this
* branch existed the user got `Analysis failed` plus a stack trace through
@ -152,6 +152,52 @@ describe('analyzeCommand undeclared relation-pair handling (#2789)', () => {
expect(records.some((r) => r.recoveryHint === 'module-not-found')).toBe(false);
});
it('lists every undeclared pair and reports them all in labelPairs (#2797)', async () => {
const { UndeclaredPairCollector } = await import('../../src/core/lbug/rel-pair-routing.js');
const undeclared = new UndeclaredPairCollector();
undeclared.record(...PAIR_ERROR_ARGS);
undeclared.record(
'Method|File',
'CALLS',
'Method:src/App.vue:App.onSave#3',
'File:src/Child.vue',
);
let thrown: unknown;
try {
undeclared.throwIfAny();
} catch (err) {
thrown = err;
}
runFullAnalysisMock.mockRejectedValue(
new Error(`Phase 'graph-emit' failed`, { cause: thrown }),
);
const { _captureLogger } = await import('../../src/core/logger.js');
const cap = _captureLogger();
const { analyzeCommand } = await import('../../src/cli/analyze.js');
await analyzeCommand(undefined, {});
expect(process.exitCode).toBe(1);
const record = cap.records().find((r) => r.recoveryHint === 'undeclared-relation-pair');
cap.restore();
expect(record).toMatchObject({
labelPair: 'Method|Annotation',
labelPairs: ['Method|Annotation', 'Method|File'],
});
const text = typeof record?.msg === 'string' ? record.msg : '';
expect(
[
'2 relationship label pairs are not declared',
'Method → Annotation',
'Method → File',
'src/App.vue',
'src/main/java/app/BeanConfig.java',
].filter((needle) => !text.includes(needle)),
).toEqual([]);
});
it('does not claim an unrelated failure', async () => {
runFullAnalysisMock.mockRejectedValue(new Error('LadybugDB write failed'));

21
gitnexus/test/unit/embedding-config.test.ts
View file

@ -1,8 +1,13 @@
import { afterEach, describe, expect, it } from 'vitest';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { resolveEmbeddingConfig } from '../../src/core/embeddings/config.js';
import { DEFAULT_EMBEDDING_CONFIG } from '../../src/core/embeddings/types.js';
const OLD_ENV = { ...process.env };
beforeEach(() => {
delete process.env.GITNEXUS_EMBEDDING_MODEL_ID;
});
afterEach(() => {
process.env = { ...OLD_ENV };
});
@ -22,6 +27,20 @@ describe('resolveEmbeddingConfig', () => {
expect(config.device).toBe('cpu');
});
it('reads the local model id from GITNEXUS_EMBEDDING_MODEL_ID over explicit config', () => {
process.env.GITNEXUS_EMBEDDING_MODEL_ID = ' Xenova/bge-base-en-v1.5 ';
expect(resolveEmbeddingConfig({ modelId: 'other/model' }).modelId).toBe(
'Xenova/bge-base-en-v1.5',
);
});
it('keeps the explicit or default model id when GITNEXUS_EMBEDDING_MODEL_ID is unset or blank', () => {
expect(resolveEmbeddingConfig().modelId).toBe(DEFAULT_EMBEDDING_CONFIG.modelId);
process.env.GITNEXUS_EMBEDDING_MODEL_ID = ' ';
expect(resolveEmbeddingConfig().modelId).toBe(DEFAULT_EMBEDDING_CONFIG.modelId);
expect(resolveEmbeddingConfig({ modelId: 'other/model' }).modelId).toBe('other/model');
});
it('rejects invalid numeric env values', () => {
process.env.GITNEXUS_EMBEDDING_THREADS = '0';

34
gitnexus/test/unit/embedding-pipeline.test.ts
View file

@ -1514,6 +1514,40 @@ describe('runEmbeddingPipeline incremental filter', () => {
expect(createdRows()).toEqual([]);
});
it('aborts on the first local vector-width mismatch instead of waiting for the ceiling', async () => {
const { LocalEmbeddingDimensionError } = await import('../../src/core/embeddings/config.js');
const subBatchTexts: number[] = [];
mockEmbedderWith(
failingEmbedBatch(
[1, 2, 3, 4, 5],
() => new LocalEmbeddingDimensionError('Xenova/bge-base-en-v1.5', 768, 384),
subBatchTexts,
),
);
const nodes = ['n1', 'n2', 'n3', 'n4', 'n5'].map((n) => makeEnumNode(n, `enum ${n} {}`));
const executeQuery = mockExecuteQuery(nodes);
const executeWithReusedStatement = mockExecuteWithReusedStatement();
const { runEmbeddingPipeline } =
await import('../../src/core/embeddings/embedding-pipeline.js');
await expect(
runEmbeddingPipeline(
executeQuery,
executeWithReusedStatement,
onProgress,
{ chunkSize: 100, overlap: 0, subBatchSize: 1 },
undefined,
new Map(),
),
).rejects.toThrow('Set GITNEXUS_EMBEDDING_DIMS=768');
expect(subBatchTexts).toHaveLength(1);
expect(nodeIdDeletes().flatMap((c) => c.params.map((p) => p.nodeId))).toEqual([nodes[0].id]);
expect(createdRows()).toEqual([]);
});
it('rethrows the FIRST error of the streak, not the generic one that tripped the ceiling', async () => {
// Verified trace for a permanently misconfigured endpoint (#2790): the HTTP
// client's shared circuit breaker opens after 3 rejections, so sub-batches 4

39
gitnexus/test/unit/lbug/graph-emit-sink.test.ts
View file

@ -24,6 +24,7 @@ import {
RETAINED_REL_TYPES,
StreamedRelationshipRemovalError,
} from '../../../src/core/lbug/graph-emit-sink.js';
import { UndeclaredRelationPairError } from '../../../src/core/lbug/rel-pair-routing.js';
import type { GraphRelationship } from 'gitnexus-shared';
const fnId = (name: string): string => `Function:src/a.ts:${name}`;
@ -128,27 +129,45 @@ describe('GraphEmitSink routing', () => {
expect(real.relationshipCount).toBe(0);
});
it('throws rather than silently dropping an undeclared endpoint-label pair (#2769)', () => {
it('fails at finalize, with every undeclared endpoint-label pair, rather than dropping them (#2769, #2797)', async () => {
// Before #2769's fix, this pair reached COPY, failed the bulk insert, and
// the per-edge fallback swallowed the failure into `catch {}` — the run
// still exited 0 with the edge silently missing. Both endpoints are valid
// node tables (so the validTables gate above does not catch it); the pair
// itself is simply absent from RELATION_SCHEMA.
// itself is simply absent from RELATION_SCHEMA. The sink now records it,
// keeps streaming, and fails once at finalize naming all such pairs.
const real = createKnowledgeGraph();
const sink = new GraphEmitSink(real, csvDir);
sink.beginStreaming();
const undeclared: GraphRelationship = {
id: 'CALLS:Static:a->Static:b',
sourceId: 'Static:src/a.ts:A',
targetId: 'Static:src/a.ts:B',
const undeclared = (from: string, to: string): GraphRelationship => ({
id: `CALLS:${from}->${to}`,
sourceId: from,
targetId: to,
type: 'CALLS',
confidence: 1,
reason: 'direct',
};
expect(() => sink.addRelationship(undeclared)).toThrow(
/Relationship label pair Static → Static is not declared/,
);
});
expect(() =>
sink.addRelationship(undeclared('Static:src/a.ts:A', 'Static:src/a.ts:B')),
).not.toThrow();
sink.addRelationship(rel('CALLS', 'f', 'g'));
sink.addRelationship(undeclared('Static:src/b.ts:C', 'Folder:src/b'));
let thrown: unknown;
try {
sink.finalize();
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(UndeclaredRelationPairError);
expect((thrown as UndeclaredRelationPairError).pairs.map((pair) => pair.pairKey)).toEqual([
'Static|Static',
'Static|Folder',
]);
// The declared edge was still streamed; the undeclared ones were not.
expect(await dataRows(path.join(csvDir, 'rel_Function_Function.csv'))).toHaveLength(1);
expect(fs.existsSync(path.join(csvDir, 'rel_Static_Static.csv'))).toBe(false);
});
});

12
gitnexus/test/unit/lbug/pdg-emit-sink.test.ts
View file

@ -78,11 +78,14 @@ describe('PdgEmitSink — routing', () => {
sink.finalize();
});
it('throws rather than silently dropping an undeclared endpoint-label pair (#2769)', () => {
it('fails at finalize instead of throwing from addRelationship on an undeclared pair (#2769, #2797)', () => {
// PDG edges are always BasicBlock|BasicBlock in production, but the sink
// applies no declared-pair gate of its own — this pins that the shared
// `assertDeclaredPair` guard (also wired into GraphEmitSink) still fires
// here rather than reaching COPY and silently dropping the edge.
// `checkDeclaredPair` guard (also wired into GraphEmitSink) still fails the
// run rather than reaching COPY and silently dropping the edge. It must not
// throw from addRelationship: the PDG emit loop in scope-resolution wraps
// each file in a try/catch that only warns, so a throw there used to cost
// that file's CFG and let the run succeed.
const real = createKnowledgeGraph();
const sink = new PdgEmitSink(real, path.join(tmpRoot, 'pdg-csv'));
@ -94,7 +97,8 @@ describe('PdgEmitSink — routing', () => {
confidence: 1,
reason: 'seq',
};
expect(() => sink.addRelationship(undeclared)).toThrow(
expect(() => sink.addRelationship(undeclared)).not.toThrow();
expect(() => sink.finalize()).toThrow(
/Relationship label pair Static → Static is not declared/,
);
});

113
gitnexus/test/unit/local-embedding-model.test.ts Normal file
View file

@ -0,0 +1,113 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { contentHashForNode } from '../../src/core/embeddings/embedding-pipeline.js';
import { DEFAULT_EMBEDDING_CONFIG, type EmbeddableNode } from '../../src/core/embeddings/types.js';
const sidecarEmbedBatchMock = vi.hoisted(() => vi.fn());
vi.mock('../../src/core/embeddings/embedding-sidecar-client.js', () => ({
ensureEmbeddingSidecar: vi.fn(),
getSidecarDevice: () => 'cpu',
reapEmbeddingSidecarAndWait: vi.fn(),
sidecarEmbedBatch: (...args: unknown[]) => sidecarEmbedBatchMock(...args),
}));
vi.mock('../../src/core/embeddings/runtime-support.js', () => ({
assessLocalEmbeddingRuntime: () => ({ status: 'ready' }),
getLocalEmbeddingRuntimeBlocker: () => null,
}));
const { embedBatch, embedText, getEmbeddingDimensions } =
await import('../../src/core/embeddings/embedder.js');
const { resolveEmbeddingIdentity } =
await import('../../src/core/embeddings/embedding-identity.js');
const ENV_KEYS = [
'GITNEXUS_EMBEDDING_MODEL_ID',
'GITNEXUS_EMBEDDING_DIMS',
'GITNEXUS_EMBEDDING_URL',
'GITNEXUS_EMBEDDING_MODEL',
] as const;
const savedEnv: Record<string, string | undefined> = {};
const vectors = (count: number, width: number): Float32Array[] =>
Array.from({ length: count }, () => new Float32Array(width));
beforeEach(() => {
for (const key of ENV_KEYS) {
savedEnv[key] = process.env[key];
delete process.env[key];
}
sidecarEmbedBatchMock.mockReset();
});
afterEach(() => {
for (const key of ENV_KEYS) {
if (savedEnv[key] === undefined) delete process.env[key];
else process.env[key] = savedEnv[key];
}
});
describe('local embedding model selection', () => {
it('keeps the default model at 384 dimensions even when GITNEXUS_EMBEDDING_DIMS is set', () => {
process.env.GITNEXUS_EMBEDDING_DIMS = '1024';
expect(getEmbeddingDimensions()).toBe(DEFAULT_EMBEDDING_CONFIG.dimensions);
});
it('takes the vector width of a custom model from GITNEXUS_EMBEDDING_DIMS', () => {
process.env.GITNEXUS_EMBEDDING_MODEL_ID = 'Xenova/bge-base-en-v1.5';
expect(getEmbeddingDimensions()).toBe(384);
process.env.GITNEXUS_EMBEDDING_DIMS = '768';
expect(getEmbeddingDimensions()).toBe(768);
process.env.GITNEXUS_EMBEDDING_DIMS = '768x';
expect(() => getEmbeddingDimensions()).toThrow(
'GITNEXUS_EMBEDDING_DIMS must be a positive integer',
);
});
it('records the custom model and its width in the embedding identity', () => {
process.env.GITNEXUS_EMBEDDING_MODEL_ID = 'Xenova/bge-base-en-v1.5';
process.env.GITNEXUS_EMBEDDING_DIMS = '768';
expect(resolveEmbeddingIdentity()).toEqual({
model: 'Xenova/bge-base-en-v1.5',
dimensions: 768,
provider: 'local',
});
});
it('rejects local vectors whose width differs from the configured dimensions', async () => {
process.env.GITNEXUS_EMBEDDING_MODEL_ID = 'Xenova/bge-base-en-v1.5';
sidecarEmbedBatchMock.mockResolvedValue(vectors(2, 768));
await expect(embedBatch(['a', 'b'])).rejects.toThrow(
'local model Xenova/bge-base-en-v1.5 returned 768d vectors, but expected 384d. ' +
'Set GITNEXUS_EMBEDDING_DIMS=768',
);
sidecarEmbedBatchMock.mockResolvedValue(vectors(1, 768));
await expect(embedText('a')).rejects.toThrow('Set GITNEXUS_EMBEDDING_DIMS=768');
});
it('returns local vectors that match the configured dimensions', async () => {
process.env.GITNEXUS_EMBEDDING_MODEL_ID = 'Xenova/bge-base-en-v1.5';
process.env.GITNEXUS_EMBEDDING_DIMS = '768';
sidecarEmbedBatchMock.mockResolvedValue(vectors(2, 768));
const result = await embedBatch(['a', 'b']);
expect(result.map((vec) => vec.length)).toEqual([768, 768]);
});
it('changes the content hash for a custom model only', () => {
const node: EmbeddableNode = {
id: 'Function:foo:src/main.ts',
name: 'foo',
label: 'Function',
filePath: 'src/main.ts',
content: 'function foo() { return 1; }',
};
const base = contentHashForNode(node);
expect(contentHashForNode(node, { modelId: DEFAULT_EMBEDDING_CONFIG.modelId })).toBe(base);
const custom = contentHashForNode(node, { modelId: 'Xenova/bge-base-en-v1.5' });
expect(custom).not.toBe(base);
expect(contentHashForNode(node, { modelId: 'Xenova/bge-small-en-v1.5' })).not.toBe(custom);
});
});

198
gitnexus/test/unit/rel-pair-routing.test.ts
View file

@ -5,8 +5,9 @@ import path from 'path';
import os from 'os';
import {
RelPairRouter,
UndeclaredPairCollector,
UndeclaredRelationPairError,
assertDeclaredPair,
checkDeclaredPair,
createRelationPairMatcher,
findUndeclaredRelationPairError,
getNodeLabel,
@ -215,45 +216,61 @@ describe('createRelationPairMatcher', () => {
});
});
describe('assertDeclaredPair', () => {
describe('checkDeclaredPair', () => {
const DECLARED_ONE = new Set<string>(['Function|Function']);
it('passes a declared pair through and throws a typed error for an undeclared one', () => {
/** Run one undeclared edge through the guard and return what the collector throws. */
const failOnce = (
pairKey: string,
relationType: string,
fromId: string,
toId: string,
): unknown => {
const undeclared = new UndeclaredPairCollector();
checkDeclaredPair(pairKey, DECLARED_ONE, undeclared, relationType, fromId, toId);
try {
undeclared.throwIfAny();
return undefined;
} catch (err) {
return err;
}
};
it('passes a declared pair and records an undeclared one instead of throwing', () => {
const undeclared = new UndeclaredPairCollector();
expect(
assertDeclaredPair(
checkDeclaredPair(
'Function|Function',
DECLARED_ONE,
undeclared,
'CALLS',
'Function:src/a.ts:f:1',
'Function:src/a.ts:g:2',
),
).toBeUndefined();
expect(() =>
assertDeclaredPair(
).toBe(true);
expect(() => undeclared.throwIfAny()).not.toThrow();
expect(
checkDeclaredPair(
'Method|Annotation',
DECLARED_ONE,
undeclared,
'ANNOTATED_BY',
'Method:src/app/Config.java:Config.dataSource#12',
'Annotation:src/app/Config.java:ConditionalOnMissingBean',
),
).toThrow(UndeclaredRelationPairError);
).toBe(false);
expect(undeclared.size).toBe(1);
expect(() => undeclared.throwIfAny()).toThrow(UndeclaredRelationPairError);
});
it('carries the pair, relationship type, both node ids and the source file (#2789)', () => {
const thrown = (() => {
try {
assertDeclaredPair(
'Method|Annotation',
DECLARED_ONE,
'ANNOTATED_BY',
'Method:src/app/Config.java:Config.dataSource#12',
'Annotation:src/app/Config.java:ConditionalOnMissingBean',
);
return undefined;
} catch (err) {
return err;
}
})();
const thrown = failOnce(
'Method|Annotation',
'ANNOTATED_BY',
'Method:src/app/Config.java:Config.dataSource#12',
'Annotation:src/app/Config.java:ConditionalOnMissingBean',
);
expect(thrown).toBeInstanceOf(UndeclaredRelationPairError);
expect(thrown).toMatchObject({
@ -285,6 +302,100 @@ describe('assertDeclaredPair', () => {
're-running the analysis will fail in exactly the same place',
];
expect(required.filter((needle) => !message.includes(needle))).toEqual([]);
expect(message).not.toContain('other files may contain the same pairs');
});
it('warns about unlisted files when a single pair occurs more than once', () => {
const undeclared = new UndeclaredPairCollector();
for (const file of ['Config.java', 'Other.kt']) {
undeclared.record(
'Method|Annotation',
'ANNOTATED_BY',
`Method:src/app/${file}:configure#12`,
`Annotation:src/app/${file}:Bean`,
);
}
let thrown: unknown;
try {
undeclared.throwIfAny();
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(UndeclaredRelationPairError);
const err = thrown as UndeclaredRelationPairError;
expect(err.pairs).toHaveLength(1);
expect(err.pairs[0]).toMatchObject({ pairKey: 'Method|Annotation', edges: 2 });
expect(err.message).toContain('Only the first edge of each pair is shown');
expect(err.message).toContain(
'other files may contain the same pairs and need to be excluded too',
);
expect(err.message).not.toContain('To finish indexing the rest');
expect(err.message).toContain('src/app/Config.java');
expect(err.message).not.toContain('Other.kt');
});
it('reports every undeclared pair of the emit in one error (#2797)', () => {
const undeclared = new UndeclaredPairCollector();
const edges: Array<[string, string, string, string]> = [
[
'Method|Annotation',
'ANNOTATED_BY',
'Method:src/app/Config.java:Config.dataSource#12',
'Annotation:src/app/Config.java:ConditionalOnMissingBean',
],
['Method|File', 'CALLS', 'Method:src/App.vue:App.onSave#3', 'File:src/Child.vue'],
[
'Method|Annotation',
'ANNOTATED_BY',
'Method:src/app/Other.kt:Other.cache#7',
'Annotation:src/app/Other.kt:Bean',
],
['Class|Tool', 'DEFINES', 'Class:tools/search.py:Search', 'Tool:tools/search.py:search'],
];
for (const [pairKey, type, from, to] of edges) {
expect(checkDeclaredPair(pairKey, DECLARED_ONE, undeclared, type, from, to)).toBe(false);
}
let thrown: unknown;
try {
undeclared.throwIfAny();
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(UndeclaredRelationPairError);
const err = thrown as UndeclaredRelationPairError;
// The top-level fields stay those of the first pair, so existing consumers
// (the CLI's machine-readable fields) keep their meaning.
expect(err).toMatchObject({
pairKey: 'Method|Annotation',
relationType: 'ANNOTATED_BY',
sourceFile: 'src/app/Config.java',
});
expect(err.pairs.map((pair) => [pair.pairKey, pair.edges])).toEqual([
['Method|Annotation', 2],
['Method|File', 1],
['Class|Tool', 1],
]);
const required = [
'3 relationship label pairs are not declared',
'Method → Annotation (2 edges, first one below)',
'Method → File (1 edge, first one below)',
'Class → Tool (1 edge, first one below)',
'Method:src/app/Config.java:Config.dataSource#12',
'src/App.vue',
'tools/search.py',
'Report the missing pairs',
'adding the example files (or their directories)',
'Only the first edge of each pair is shown',
'other files may contain the same pairs and need to be excluded too',
'https://github.com/abhigyanpatwari/GitNexus/issues/new',
're-running the analysis will fail in exactly the same place',
];
expect(required.filter((needle) => !err.message.includes(needle))).toEqual([]);
expect(err.message).not.toContain('To finish indexing the rest');
// The second Method→Annotation edge is counted, not listed.
expect(err.message).not.toContain('Other.kt');
});
it('reports no source file for synthetic community/process ids instead of guessing', () => {
@ -341,27 +452,38 @@ describe('RelPairRouter', () => {
expect(streams.every((s) => s.ended)).toBe(true);
});
it('rejects a valid-label pair that is absent from the relation schema', () => {
it('skips every valid-label pair absent from the relation schema and rejects at close with all of them', async () => {
const streams: MockWriteStream[] = [];
const router = new RelPairRouter(tmpDir, HEADER, VALID, DECLARED, mockFactory(streams));
const route = () =>
router.route(
'File:src/a.ts',
'Community:1',
row('File:src/a.ts', 'Community:1', 'DEFINES'),
'DEFINES',
);
expect(route).toThrow('File → Community is not declared in the LadybugDB relation schema');
const route = async (from: string, to: string, type: string) => {
const p = router.route(from, to, row(from, to, type), type);
if (p) await p;
};
// Routing does not throw on an undeclared pair: it keeps going so the
// later ones are found in the same run.
await route('File:src/a.ts', 'Community:1', 'DEFINES');
await route('File:src/a.ts', 'Function:src/a.ts:f:1', 'DEFINES');
await route('Function:src/b.ts:g:2', 'Process:3', 'STEP_IN_PROCESS');
await route('File:src/c.ts', 'Community:4', 'DEFINES');
const closing = router.close();
await expect(closing).rejects.toThrow(UndeclaredRelationPairError);
const err = (await closing.catch((e: unknown) => e)) as UndeclaredRelationPairError;
// The row is already CSV-escaped here, so the router must forward the edge
// context itself — otherwise the crash names only the abstract label pair.
expect(route).toThrow(UndeclaredRelationPairError);
expect(route).toThrow(/DEFINES/);
expect(route).toThrow(/File:src\/a\.ts/);
expect(route).toThrow(/Community:1/);
expect(streams).toHaveLength(0);
// context itself — otherwise the error names only the abstract label pair.
expect(err.pairs.map((pair) => [pair.pairKey, pair.relationType, pair.edges])).toEqual([
['File|Community', 'DEFINES', 2],
['Function|Process', 'STEP_IN_PROCESS', 1],
]);
expect(err.message).toContain('File → Community (2 edges, first one below)');
expect(err.message).toContain('Function:src/b.ts:g:2');
// Only the declared edge reached a pair file; undeclared ones are not
// counted as skipped (that counter is for invalid node labels).
expect([...router.byPair.keys()]).toEqual(['File|Function']);
expect(router.total).toBe(1);
expect(router.skipped).toBe(0);
expect(router.total).toBe(0);
expect(streams.every((s) => s.ended)).toBe(true);
});
it('returns a drain promise under backpressure and completes once unblocked', async () => {

2
gitnexus/test/unit/schema-pair-coverage.test.ts
View file

@ -13,7 +13,7 @@ import { CALL_TARGET_TYPES } from '../../src/core/ingestion/model/symbol-table.j
/**
* `RELATION_SCHEMA` is generated from two cross products plus one hand-written
* block. This file guards the generated half; a pair drawn from either rule and
* absent from the DDL does not degrade — `assertDeclaredPair` throws and
* absent from the DDL does not degrade — `checkDeclaredPair` fails the run and
* `analyze` dies mid-phase on whichever codebase first produces it. Failing
* here means fixing the rule, not the assertion.
*