mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-03 02:21:44 +00:00
feat(analyze): expose process-detection budget overrides (#3324)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
* feat(analyze): expose process-detection budget overrides (#3313) Operators can raise or lower process count, branching, trace depth, and the entry-point candidate pool via CLI, .gitnexusrc, or GITNEXUS_* without changing shipped defaults. A budget-only change re-detects flows on the next analyze without --force. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): say invalid budget flags still honor env A rejected --max-processes value was described as falling back to the built-in default even when GITNEXUS_MAX_* still won the next precedence tier. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(analyze): share process-detection defaults and skip unused walks Keep DEFAULT_CONFIG aligned with the budget resolver and count symbols only when maxProcesses is still dynamic. Co-authored-by: Cursor <cursoragent@cursor.com> * style(analyze): wrap process-detection budget files for prettier Co-authored-by: Cursor <cursoragent@cursor.com> * docs(analyze): name the real process-detection default formula Co-authored-by: Cursor <cursoragent@cursor.com> * docs(analyze): stop calling maxProcesses*2 a hard trace quota Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): say invalid env budget tokens fall back to defaults Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): recertify process-detection after in-place FTS abort (#3324) Persist processDetection.uncertified on the in-place FTS dirty stamp when the budget mismatched so a flagless retry cannot keep rewritten flows. Qualify .gitnexusrc fail-fast copy and tighten related tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): skip live dirty stamp on atomic incremental (#3324) POSIX atomic incremental mutates a staging copy, so stamping live incrementalInProgress before swap made a crash force-rebuild a healthy index. Align analyze --help with CLI > .gitnexusrc > env > default. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(changelog): drop the atomic-incremental dirty-stamp note The code fix stays; Unreleased no longer lists that recovery change. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): survive FTS SIGSEGV in --limit e2e CREATE_FTS_INDEX can kill the setup analyze on some WSL hosts (status null). Rebuild with --skip-fts and skip BM25-only query --limit cases unless GITNEXUS_REQUIRE_FTS=1. Refs #3324 Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): mark update-check child at import Writing refresh-started from fetch() raced a 30s poll against cold tsx boot on a loaded default-project worker. Refs #3324 Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3324) Isolate default-budget FTS crash-marker tests from GITNEXUS_MAX_* env, assert uncertify-before-FTS order and deferred flow detection on park recovery, drop the dangling "then" from entry-point help, and correct stale streamGraphEmit docs without skipping the process-detection stamp. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(changelog): drop Unreleased process-detection notes Keep the #3313 / #3322 code; Unreleased changelog matches main until release. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Gergő Magyar
GitHub
parent
9d95af9fc3
commit
ba39d5c009
27 changed files with 1515 additions and 80 deletions
15
README.md
15
README.md
|
|
@ -407,7 +407,8 @@ backoff. Invalid `.gitnexusrc` or ignore-file reloads pause ordinary refreshes
|
|||
until the control file is fixed. Stop the watcher with Ctrl+C.
|
||||
|
||||
Watch mode accepts `--debounce`, `--workers`, `--worker-timeout`,
|
||||
`--max-file-size`, `--branch`, `--pdg`, `--skip-fts`, `--name`, `--allow-duplicate-name`, and
|
||||
`--max-file-size`, `--max-processes`, `--max-process-branching`,
|
||||
`--max-process-trace-depth`, `--max-entry-point-candidates`, `--branch`, `--pdg`, `--skip-fts`, `--name`, `--allow-duplicate-name`, and
|
||||
`--verbose`. Explicit one-shot options such as `--force`, `--repair-fts`,
|
||||
embedding flags, `--skills`, `--self-commit`, `--index-only`, and `--skip-git`
|
||||
are rejected. Unsupported defaults from `.gitnexusrc` are ignored with a
|
||||
|
|
@ -453,6 +454,8 @@ gitnexus analyze --verbose # Log skipped files when parsers are unavailabl
|
|||
gitnexus analyze --worker-timeout 60 # Increase worker idle timeout for slow parses
|
||||
gitnexus analyze --workers <n> # Parse worker pool size (>=1; default: cores-1, capped at 16,
|
||||
# auto-sized to the repo). 0 is rejected — there is no sequential mode.
|
||||
gitnexus analyze --max-processes <n> # Process-detection process cap (replaces dynamic max(20, round(symbols/10)))
|
||||
gitnexus analyze --max-entry-point-candidates <n> # Ranked entry-point pool (default 200; raise when the warning names it)
|
||||
gitnexus analyze --spring-actuator ./actuator # Enrich with local Spring Boot Actuator JSON snapshots
|
||||
gitnexus analyze --asyncapi-spec ./docs/asyncapi # Resolve broker addresses from AsyncAPI 3.x documents
|
||||
gitnexus analyze --wal-checkpoint-threshold 67108864 # LadybugDB WAL auto-checkpoint threshold in bytes
|
||||
|
|
@ -576,15 +579,15 @@ Notes:
|
|||
|
||||
- The default branch is resolved as: `--default-branch` > `.gitnexusrc` `defaultBranch`/`branch` > auto-detected `origin/HEAD` > `main`.
|
||||
- `skipContextFiles` / `skipAiContext` are aliases for `skipAgentsMd` — they skip the `AGENTS.md` / `CLAUDE.md` block only. They do **not** imply `skipSkills`. `indexOnly` is the stronger option that skips all file injection.
|
||||
- Supported keys: `defaultBranch` (`branch`), `skipAgentsMd` (`skipContextFiles`, `skipAiContext`), `skipSkills`, `indexOnly`, `stats`/`noStats`, `embeddings`, `dropEmbeddings`, `name`, `allowDuplicateName`, `maxFileSize`, `workerTimeout`, `walCheckpointThreshold`, `workers`, `springActuator`, `embeddingThreads`, `embeddingBatchSize`, `embeddingSubBatchSize`, `embeddingDevice`.
|
||||
- The file is JSON only. Unknown keys and invalid values fail fast with an actionable error before analysis starts.
|
||||
- Supported keys: `defaultBranch` (`branch`), `skipAgentsMd` (`skipContextFiles`, `skipAiContext`), `skipSkills`, `indexOnly`, `stats`/`noStats`, `embeddings`, `dropEmbeddings`, `name`, `allowDuplicateName`, `maxFileSize`, `workerTimeout`, `walCheckpointThreshold`, `workers`, `maxProcesses`, `maxProcessBranching`, `maxProcessTraceDepth`, `maxEntryPointCandidates`, `springActuator`, `embeddingThreads`, `embeddingBatchSize`, `embeddingSubBatchSize`, `embeddingDevice`.
|
||||
- The file is JSON only. Unknown keys and wrong JSON types fail fast with an actionable error before analysis starts. Process-detection knobs (`maxProcesses`, `maxProcessBranching`, `maxProcessTraceDepth`, `maxEntryPointCandidates`) that are not a positive integer warn and fall through to env, then the built-in default.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Environment variables</strong></summary>
|
||||
|
||||
Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max-file-size`, `--verbose`). Use the env-var form when you'd otherwise repeat the same flag every run, or when invoking GitNexus from a long-running host (MCP server, eval-server, CI shell) that already manages its own environment. CLI flags take precedence over env vars; env vars take precedence over built-in defaults.
|
||||
Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max-file-size`, `--verbose`). Use the env-var form when you'd otherwise repeat the same flag every run, or when invoking GitNexus from a long-running host (MCP server, eval-server, CI shell) that already manages its own environment. CLI flags take precedence over `.gitnexusrc`, which takes precedence over env vars, which take precedence over built-in defaults.
|
||||
|
||||
| Variable | Default | Effect | Tune when… |
|
||||
| ----------------------------------------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
|
@ -601,6 +604,10 @@ Most `analyze` knobs are also CLI flags (`--workers`, `--worker-timeout`, `--max
|
|||
| `GITNEXUS_PROFILE_DEFERRED_SLOW_MS` | `3000` (verbose) / `5000` | Per-file threshold in ms above which `processCallsFromExtracted` emits a `slow file …` log line. Parsed via `Number()`: accepts integers (`5000`), scientific notation (`2.5e3`), decimals (`.5`), and hex (`0x10`). Non-finite or non-positive values fall back to the default. | Hunting a few outlier files dominating the deferred call-resolution stage; lower to surface more, raise to focus only on the worst. |
|
||||
| `PROF_LBUG_LOAD` | unset | When `1`, emits one `[lbug-load prof]` summary line per `loadGraphToLbug` call breaking the graph-DB persistence wall into stages (`csv-emit` / `copy-nodes` / `copy-rels` / `fallback` / `total`) plus node & edge counts. Zero-cost when unset. | Attributing large-repo analyze wall time across CSV generation vs. LadybugDB `COPY` (issue #2203) — the analyze "emit" timing is the scope-resolution bucket, not this DB-write path. |
|
||||
| `GITNEXUS_MAX_FILE_SIZE` | `512` (KB) | Walker skip threshold in KB. Hard cap is `32768` (tree-sitter buffer ceiling). Equivalent to `--max-file-size <kb>`. | Indexing repos with intentionally-large source files (generated parsers, vendored bundles) that should still be parsed. |
|
||||
| `GITNEXUS_MAX_PROCESSES` | dynamic (`max(20, round(symbols/10))`) | Analyze-time process-detection process cap. Equivalent to `--max-processes <n>` / `.gitnexusrc` `maxProcesses`. Explicit values replace the dynamic formula (not a multiplier). `0` is invalid, not unlimited. Changing this re-detects flows on the next analyze without `--force`. Distinct from query-time `IMPACT_MAX_CHUNKS`. | `[processes] … whole flows are MISSING` names `--max-processes` after entry points were never traced or flows were dropped. Tracing does not start the next entry once collected traces already reach `maxProcesses * 2`; a started entry can still emit every trace that entry produces. |
|
||||
| `GITNEXUS_MAX_PROCESS_BRANCHING` | `4` | Analyze-time per-node branching cap during flow tracing. Equivalent to `--max-process-branching <n>`. Shape-only: raising it shortens fewer traces; it does not restore whole missing flows. | A flow is present but `calleesDropped` is high at debug. |
|
||||
| `GITNEXUS_MAX_PROCESS_TRACE_DEPTH` | `10` | Analyze-time DFS depth cap during flow tracing. Equivalent to `--max-process-trace-depth <n>`. Shape-only. | A reported flow is shorter than the code path (`tracesDepthCapped` at debug). |
|
||||
| `GITNEXUS_MAX_ENTRY_POINT_CANDIDATES` | `200` | Ranked entry-point candidate pool. Equivalent to `--max-entry-point-candidates <n>`. Raising `--max-processes` alone does not clear `entryPointCandidatesDropped`. Doubling the current cap is the usual first raise; setting it to the full remaining candidate count can exhaust CPU and memory. | The `[processes]` warning reports candidate entry points that never ranked in. |
|
||||
| `GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS` | `30000` | Worker idle timeout in milliseconds before retry/fallback. Equivalent to `--worker-timeout <seconds>` × 1000. | Slow-parsing files (large minified JS, deeply-nested TS types) that legitimately need more than 30s. |
|
||||
| `GITNEXUS_WORKER_READY_TIMEOUT_MS` | `5000` | Startup budget in milliseconds for a parse worker to load its grammar bindings and report `{type:'ready'}`. Slots that miss it are treated as startup crashes. | Slow or heavily loaded hosts where a full pool cold-starting concurrently needs more than 5s, and analyze aborts with "did not report ready within 5000ms". |
|
||||
| `GITNEXUS_FTS_STEMMER` | `porter` | Stemmer used when rebuilding BM25/FTS indexes. Use `none` for CJK-heavy repositories, or a language stemmer such as `german`, `french`, or `spanish` for matching repository comments. Re-run `gitnexus analyze --repair-fts` after changing it. | Keyword search quality is poor for non-English comments or identifiers under English stemming. |
|
||||
|
|
|
|||
|
|
@ -245,6 +245,8 @@ gitnexus analyze --skip-agents-md # Preserve custom AGENTS.md/CLAUDE.md gitnexu
|
|||
gitnexus analyze --skip-skills # Skip installing standard .claude/skills/gitnexus-* skill files
|
||||
gitnexus analyze --skip-git # Index folders that are not Git repositories
|
||||
gitnexus analyze --workers <n> # Parse worker pool size (>=1; default: cores-1, capped at 16)
|
||||
gitnexus analyze --max-processes <n> # Process-detection process cap (replaces dynamic max(20, round(symbols/10)))
|
||||
gitnexus analyze --max-entry-point-candidates <n> # Ranked entry-point pool (default 200; raise when the warning names it)
|
||||
gitnexus analyze --spring-actuator ./actuator # Enrich with local Spring Boot Actuator JSON snapshots
|
||||
gitnexus analyze --verbose # Log skipped files when parsers are unavailable
|
||||
gitnexus analyze --max-file-size 1024 # Skip files larger than N KB (default: 512, cap: 32768)
|
||||
|
|
@ -298,7 +300,8 @@ installation. Run a one-shot `gitnexus analyze` when those generated files need
|
|||
updating. Stop watch mode with Ctrl+C.
|
||||
|
||||
Watch mode accepts `--debounce`, `--workers`, `--worker-timeout`,
|
||||
`--max-file-size`, `--branch`, `--pdg`, `--name`, `--allow-duplicate-name`, and
|
||||
`--max-file-size`, `--max-processes`, `--max-process-branching`,
|
||||
`--max-process-trace-depth`, `--max-entry-point-candidates`, `--branch`, `--pdg`, `--name`, `--allow-duplicate-name`, and
|
||||
`--verbose`. Explicit one-shot options such as `--force`, `--repair-fts`,
|
||||
embedding flags, `--skills`, `--default-branch`, `--skip-agents-md`,
|
||||
`--skip-skills`, `--no-stats`, `--self-commit`, `--index-only`, and `--skip-git`
|
||||
|
|
@ -764,6 +767,22 @@ npx gitnexus analyze
|
|||
|
||||
Values above **32768 KB (32 MB)** are clamped to the tree-sitter parser ceiling; invalid values fall back to the 512 KB default with a one-time warning. When an override is active, `analyze` prints the effective threshold in its startup banner (e.g. `GITNEXUS_MAX_FILE_SIZE: effective threshold 2048KB (default 512KB)`).
|
||||
|
||||
### Process detection reports missing flows
|
||||
|
||||
On a large repository, `analyze` may warn that `[processes] … whole flows are MISSING`. That means ranked entry points or completed flows were sampled away by the analyze-time detection budget — not that the code path is absent, and not the query-time `IMPACT_MAX_CHUNKS` cap.
|
||||
|
||||
Defaults stay in place when nothing is set: dynamic `maxProcesses = max(20, round(non-File symbols / 10))`, branching `4`, trace depth `10`, entry-point candidate pool `200`. Raise a knob only when the warning names it:
|
||||
|
||||
```bash
|
||||
# Usual first move when entryPointCandidatesDropped is the loud counter
|
||||
npx gitnexus analyze --max-entry-point-candidates 400
|
||||
|
||||
# When ranked entry points were never traced, or flows were dropped at maxProcesses
|
||||
npx gitnexus analyze --max-processes 80
|
||||
```
|
||||
|
||||
Equivalent `.gitnexusrc` keys: `maxProcesses`, `maxProcessBranching`, `maxProcessTraceDepth`, `maxEntryPointCandidates`. Equivalent env vars: `GITNEXUS_MAX_PROCESSES`, `GITNEXUS_MAX_PROCESS_BRANCHING`, `GITNEXUS_MAX_PROCESS_TRACE_DEPTH`, `GITNEXUS_MAX_ENTRY_POINT_CANDIDATES`. Precedence is CLI > `.gitnexusrc` > env > default. `0` is invalid, not unlimited. Changing these knobs re-runs process detection on the next `analyze` without `--force`. Raising them increases CPU and memory; this is not a heap-OOM fix.
|
||||
|
||||
### Analyze reports a worker timeout
|
||||
|
||||
Worker parse timeouts are recoverable. GitNexus retries stalled worker jobs with backoff, splits large jobs to isolate slow files, and quarantines a file that repeatedly crashes its worker (respawning the slot so the pool keeps going). If a large repository needs more time per worker job, use either:
|
||||
|
|
|
|||
|
|
@ -102,6 +102,10 @@ const KEY_SPECS: Record<string, KeySpec> = {
|
|||
workerTimeout: { target: 'workerTimeout', kind: 'numeric-string' },
|
||||
walCheckpointThreshold: { target: 'walCheckpointThreshold', kind: 'numeric-string' },
|
||||
workers: { target: 'workers', kind: 'numeric-string' },
|
||||
maxProcesses: { target: 'maxProcesses', kind: 'numeric-string' },
|
||||
maxProcessBranching: { target: 'maxProcessBranching', kind: 'numeric-string' },
|
||||
maxProcessTraceDepth: { target: 'maxProcessTraceDepth', kind: 'numeric-string' },
|
||||
maxEntryPointCandidates: { target: 'maxEntryPointCandidates', kind: 'numeric-string' },
|
||||
embeddingThreads: { target: 'embeddingThreads', kind: 'numeric-string' },
|
||||
embeddingBatchSize: { target: 'embeddingBatchSize', kind: 'numeric-string' },
|
||||
embeddingSubBatchSize: { target: 'embeddingSubBatchSize', kind: 'numeric-string' },
|
||||
|
|
|
|||
|
|
@ -115,6 +115,14 @@ export interface AnalyzeOptions {
|
|||
walCheckpointThreshold?: string;
|
||||
/** Parse worker pool size (>=1); 0 is rejected (no sequential mode). */
|
||||
workers?: string;
|
||||
/** Process-detection process cap. Positive integer string; `0` is invalid. */
|
||||
maxProcesses?: string;
|
||||
/** Process-detection per-node branching cap. Positive integer string. */
|
||||
maxProcessBranching?: string;
|
||||
/** Process-detection DFS depth cap. Positive integer string. */
|
||||
maxProcessTraceDepth?: string;
|
||||
/** Ranked entry-point candidate pool. Positive integer string. */
|
||||
maxEntryPointCandidates?: string;
|
||||
embeddingThreads?: string;
|
||||
embeddingBatchSize?: string;
|
||||
embeddingSubBatchSize?: string;
|
||||
|
|
|
|||
|
|
@ -22,6 +22,10 @@ import {
|
|||
import type { AnalyzeOptions } from './analyze-options.js';
|
||||
import { ensureHeap } from './analyze.js';
|
||||
import { cliError, cliInfo, cliWarn } from './cli-message.js';
|
||||
import {
|
||||
formatInvalidProcessDetectionOverride,
|
||||
parseProcessDetectionBudgetStrings,
|
||||
} from '../core/ingestion/process-detection-budget.js';
|
||||
import {
|
||||
WATCH_FULL_REFRESH_PATH,
|
||||
WatchRefreshQueue,
|
||||
|
|
@ -164,6 +168,17 @@ export async function resolveWatchOptions(
|
|||
const workerPoolSize = positiveInteger(merged.workers, '--workers');
|
||||
const workerTimeoutSeconds = positiveInteger(merged.workerTimeout, 'workerTimeout');
|
||||
const maxFileSize = positiveInteger(merged.maxFileSize, 'maxFileSize', MAX_FILE_SIZE_KB);
|
||||
const processDetection = parseProcessDetectionBudgetStrings(
|
||||
{
|
||||
maxProcesses: merged.maxProcesses,
|
||||
maxProcessBranching: merged.maxProcessBranching,
|
||||
maxProcessTraceDepth: merged.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: merged.maxEntryPointCandidates,
|
||||
},
|
||||
(flag, raw) => {
|
||||
cliWarn(formatInvalidProcessDetectionOverride(flag, raw));
|
||||
},
|
||||
);
|
||||
|
||||
setEnvironment(
|
||||
'GITNEXUS_MAX_FILE_SIZE',
|
||||
|
|
@ -183,6 +198,10 @@ export async function resolveWatchOptions(
|
|||
registryName: merged.name,
|
||||
allowDuplicateName: merged.allowDuplicateName,
|
||||
workerPoolSize,
|
||||
maxProcesses: processDetection.maxProcesses,
|
||||
maxProcessBranching: processDetection.maxProcessBranching,
|
||||
maxProcessTraceDepth: processDetection.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: processDetection.maxEntryPointCandidates,
|
||||
fetchWrappers: merged.fetchWrappers,
|
||||
skipAgentsMd: true,
|
||||
skipSkills: true,
|
||||
|
|
|
|||
|
|
@ -54,6 +54,12 @@ import type { AnalyzeOptions } from './analyze-options.js';
|
|||
import { runFullAnalysis } from '../core/run-analyze.js';
|
||||
import { getRuntimeFingerprint } from '../core/platform/capabilities.js';
|
||||
import { getMaxFileSizeBannerMessage } from '../core/ingestion/utils/max-file-size.js';
|
||||
import {
|
||||
formatInvalidProcessDetectionOverride,
|
||||
formatProcessDetectionBudgetBanner,
|
||||
parseProcessDetectionBudgetStrings,
|
||||
resolveProcessDetectionBudget,
|
||||
} from '../core/ingestion/process-detection-budget.js';
|
||||
import { warnMissingOptionalGrammars, getOptionalGrammarExtensions } from './optional-grammars.js';
|
||||
import { glob } from 'glob';
|
||||
import fs from 'fs/promises';
|
||||
|
|
@ -947,6 +953,18 @@ const analyzeCommandImpl = async (
|
|||
workerPoolSize = parsedWorkers;
|
||||
}
|
||||
|
||||
const processDetectionFromFlags = parseProcessDetectionBudgetStrings(
|
||||
{
|
||||
maxProcesses: options.maxProcesses,
|
||||
maxProcessBranching: options.maxProcessBranching,
|
||||
maxProcessTraceDepth: options.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: options.maxEntryPointCandidates,
|
||||
},
|
||||
(flag, raw) => {
|
||||
cliWarn(` ${formatInvalidProcessDetectionOverride(flag, raw)}\n`);
|
||||
},
|
||||
);
|
||||
|
||||
// Parse `--embeddings [limit]`: `true` → default cap, string → numeric cap
|
||||
// (0 disables the cap entirely). Validated up here so failures match the
|
||||
// sibling-validation pattern (exit before bar.start() — otherwise
|
||||
|
|
@ -1236,6 +1254,12 @@ const analyzeCommandImpl = async (
|
|||
if (maxFileSizeBanner) {
|
||||
console.log(`${maxFileSizeBanner}\n`);
|
||||
}
|
||||
const processDetectionBanner = formatProcessDetectionBudgetBanner(
|
||||
resolveProcessDetectionBudget(processDetectionFromFlags),
|
||||
);
|
||||
if (processDetectionBanner) {
|
||||
console.log(`${processDetectionBanner}\n`);
|
||||
}
|
||||
|
||||
// ── CLI progress bar setup ─────────────────────────────────────────
|
||||
const barOptions: cliProgress.Options & { terminal?: CliProgressTerminal } = {
|
||||
|
|
@ -1376,6 +1400,10 @@ const analyzeCommandImpl = async (
|
|||
// GITNEXUS_WORKER_POOL_SIZE env mutation. `undefined` defers to the
|
||||
// env / auto-formula fallback inside the pipeline.
|
||||
workerPoolSize,
|
||||
maxProcesses: processDetectionFromFlags.maxProcesses,
|
||||
maxProcessBranching: processDetectionFromFlags.maxProcessBranching,
|
||||
maxProcessTraceDepth: processDetectionFromFlags.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: processDetectionFromFlags.maxEntryPointCandidates,
|
||||
// Extra fetch-wrapper names from `.gitnexusrc` (#1589/#1852 residual);
|
||||
// forwarded to the routes phase consumer scan.
|
||||
fetchWrappers: options.fetchWrappers,
|
||||
|
|
|
|||
|
|
@ -72,6 +72,10 @@ const OPTION_DESCRIPTION_KEYS = {
|
|||
'analyze|--worker-timeout <seconds>': 'help.option.analyze.workerTimeout',
|
||||
'analyze|--wal-checkpoint-threshold <bytes>': 'help.option.analyze.walCheckpointThreshold',
|
||||
'analyze|--workers <n>': 'help.option.analyze.workers',
|
||||
'analyze|--max-processes <n>': 'help.option.analyze.maxProcesses',
|
||||
'analyze|--max-process-branching <n>': 'help.option.analyze.maxProcessBranching',
|
||||
'analyze|--max-process-trace-depth <n>': 'help.option.analyze.maxProcessTraceDepth',
|
||||
'analyze|--max-entry-point-candidates <n>': 'help.option.analyze.maxEntryPointCandidates',
|
||||
'analyze|--embedding-threads <n>': 'help.option.analyze.embeddingThreads',
|
||||
'analyze|--embedding-batch-size <n>': 'help.option.analyze.embeddingBatchSize',
|
||||
'analyze|--embedding-sub-batch-size <n>': 'help.option.analyze.embeddingSubBatchSize',
|
||||
|
|
|
|||
|
|
@ -254,6 +254,14 @@ export const en = {
|
|||
'LadybugDB WAL auto-checkpoint threshold in bytes during analyze (integer >= -1; default: 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB).',
|
||||
'help.option.analyze.workers':
|
||||
'Parse worker pool size (>=1). Default: cores-1 capped at 16, auto-sized to the repo.',
|
||||
'help.option.analyze.maxProcesses':
|
||||
'Process-detection process cap (positive integer). Replaces the dynamic max(20, round(symbols/10)) formula. Default: dynamic.',
|
||||
'help.option.analyze.maxProcessBranching':
|
||||
'Process-detection per-node branching cap (positive integer). Default: 4.',
|
||||
'help.option.analyze.maxProcessTraceDepth':
|
||||
'Process-detection DFS depth cap (positive integer). Default: 10.',
|
||||
'help.option.analyze.maxEntryPointCandidates':
|
||||
'Ranked entry-point candidate pool (positive integer). Default: 200. Raise when the warning names this knob; doubling is the usual first raise.',
|
||||
'help.option.analyze.embeddingThreads': 'Limit local ONNX embedding CPU threads',
|
||||
'help.option.analyze.embeddingBatchSize': 'Number of nodes per embedding batch',
|
||||
'help.option.analyze.embeddingSubBatchSize': 'Number of chunks per embedding model call',
|
||||
|
|
@ -358,5 +366,5 @@ export const en = {
|
|||
'help.identityCache.environment':
|
||||
'\nAnalyzer identity cache:\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir\n Operator-trusted persistent cache for warm cross-process status. The directory must pre-exist, be outside the GitNexus package/build roots, and contain no symlink or junction components. Defaults remain fail-closed on platforms without POSIX ownership APIs.',
|
||||
'help.analyze.environment':
|
||||
'\nEnvironment variables:\n GITNEXUS_NO_GITIGNORE=1 Skip .gitignore parsing (still reads .gitnexusignore)\n GITNEXUS_MAX_FILE_SIZE=N Override large-file skip threshold (KB). Default 512, max 32768.\n GITNEXUS_STORAGE_PATH=/absolute/index Complete external index directory. Preserves the existing configuration semantics and overrides GITNEXUS_STORAGE_ROOT when both are set.\n GITNEXUS_STORAGE_ROOT=/absolute/root External index root; each repository uses an isolated <repo-basename>-<canonical-path-hash>/ slot.\n GITNEXUS_CONTENT_RETENTION=full Source-text retention profile: full, symbol, or none. Default full.\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir Operator-trusted persistent analyzer identity cache; must pre-exist, be outside package/build roots, and contain no symlink/junction components.\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker idle timeout in milliseconds. Default 30000.\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL auto-checkpoint threshold in bytes (default 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB).\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker job byte budget. Default 8388608.\n GITNEXUS_WORKER_POOL_SIZE=N Parse worker count override. Default cores-1 capped at 16.\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N Concurrent in-flight parse chunks. Default 2.\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N Max replacement spawns per slot before drop. Default 3.\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N Total retry wall-time per job. Default 5x sub-batch timeout.\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N Per-slot deaths to trip circuit breaker. Default max(3, poolSize).\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N Max wait at pool shutdown for a retired worker still inside native code (terminated at its next safe point instead of aborting the process). Default 30000.\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N Per-file wall-clock budget for C++ capture extraction; on breach the file keeps partial captures with a warning. Default 20000.\n GITNEXUS_EMBEDDING_THREADS=N Limit local ONNX CPU threads for --embeddings.\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 Retry per-attempt HTTP embedding timeouts through GITNEXUS_EMBEDDING_MAX_ATTEMPTS (default off; timeouts stay terminal).\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N Max embedding chunks for exact-scan fallback. Default 10000.\n GITNEXUS_VECTOR_MAX_DISTANCE=N Max accepted semantic/vector cosine distance (0 < N <= 2; higher values clamp to 2). Default 0.6 for MCP, 0.5 elsewhere.\n\nFlags override the corresponding env vars when both are provided.\n\nTip: `.gitnexusignore` supports `.gitignore`-style negation. Add e.g.\n `!__tests__/` to index a directory that is auto-filtered by default (#771).',
|
||||
'\nEnvironment variables:\n GITNEXUS_NO_GITIGNORE=1 Skip .gitignore parsing (still reads .gitnexusignore)\n GITNEXUS_MAX_FILE_SIZE=N Override large-file skip threshold (KB). Default 512, max 32768.\n GITNEXUS_STORAGE_PATH=/absolute/index Complete external index directory. Preserves the existing configuration semantics and overrides GITNEXUS_STORAGE_ROOT when both are set.\n GITNEXUS_STORAGE_ROOT=/absolute/root External index root; each repository uses an isolated <repo-basename>-<canonical-path-hash>/ slot.\n GITNEXUS_CONTENT_RETENTION=full Source-text retention profile: full, symbol, or none. Default full.\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir Operator-trusted persistent analyzer identity cache; must pre-exist, be outside package/build roots, and contain no symlink/junction components.\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker idle timeout in milliseconds. Default 30000.\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL auto-checkpoint threshold in bytes (default 67108864 = 64 MiB; -1 keeps Ladybug stock ~16 MiB).\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker job byte budget. Default 8388608.\n GITNEXUS_WORKER_POOL_SIZE=N Parse worker count override. Default cores-1 capped at 16.\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N Concurrent in-flight parse chunks. Default 2.\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N Max replacement spawns per slot before drop. Default 3.\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N Total retry wall-time per job. Default 5x sub-batch timeout.\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N Per-slot deaths to trip circuit breaker. Default max(3, poolSize).\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N Max wait at pool shutdown for a retired worker still inside native code (terminated at its next safe point instead of aborting the process). Default 30000.\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N Per-file wall-clock budget for C++ capture extraction; on breach the file keeps partial captures with a warning. Default 20000.\n GITNEXUS_EMBEDDING_THREADS=N Limit local ONNX CPU threads for --embeddings.\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 Retry per-attempt HTTP embedding timeouts through GITNEXUS_EMBEDDING_MAX_ATTEMPTS (default off; timeouts stay terminal).\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N Max embedding chunks for exact-scan fallback. Default 10000.\n GITNEXUS_VECTOR_MAX_DISTANCE=N Max accepted semantic/vector cosine distance (0 < N <= 2; higher values clamp to 2). Default 0.6 for MCP, 0.5 elsewhere.\n GITNEXUS_MAX_PROCESSES=N Process-detection process cap (positive integer). Replaces the dynamic max(20, round(symbols/10)) formula. Distinct from query-time IMPACT_MAX_CHUNKS.\n GITNEXUS_MAX_PROCESS_BRANCHING=N Process-detection per-node branching cap. Default 4.\n GITNEXUS_MAX_PROCESS_TRACE_DEPTH=N Process-detection DFS depth cap. Default 10.\n GITNEXUS_MAX_ENTRY_POINT_CANDIDATES=N Ranked entry-point candidate pool. Default 200. Raise when the warning names this knob; doubling is the usual first raise.\n\nCLI flags take precedence over `.gitnexusrc`, which takes precedence over env vars, which take precedence over built-in defaults.\n\nTip: `.gitnexusignore` supports `.gitignore`-style negation. Add e.g.\n `!__tests__/` to index a directory that is auto-filtered by default (#771).',
|
||||
} as const;
|
||||
|
|
|
|||
|
|
@ -235,6 +235,12 @@ export const zhCN = {
|
|||
'analyze 期间 LadybugDB WAL 自动 checkpoint 阈值（字节，整数 >= -1；默认：67108864 = 64 MiB；-1 保持 Ladybug 默认约 16 MiB）。',
|
||||
'help.option.analyze.workers':
|
||||
'解析 worker 池大小（>=1）。默认：cores-1，最多 16，按仓库规模自适应。',
|
||||
'help.option.analyze.maxProcesses':
|
||||
'流程检测的流程数量上限（正整数）。覆盖动态的 max(20, round(symbols/10)) 公式。默认：动态。',
|
||||
'help.option.analyze.maxProcessBranching': '流程检测的单节点分支上限（正整数）。默认：4。',
|
||||
'help.option.analyze.maxProcessTraceDepth': '流程检测的 DFS 深度上限（正整数）。默认：10。',
|
||||
'help.option.analyze.maxEntryPointCandidates':
|
||||
'排序后的入口点候选池（正整数）。默认：200。仅在警告点名该上限时提高；那时通常先翻倍。',
|
||||
'help.option.analyze.embeddingThreads': '限制本地 ONNX 嵌入 CPU 线程数',
|
||||
'help.option.analyze.embeddingBatchSize': '每个嵌入批次的节点数',
|
||||
'help.option.analyze.embeddingSubBatchSize': '每次嵌入模型调用的分块数',
|
||||
|
|
@ -331,5 +337,5 @@ export const zhCN = {
|
|||
'help.identityCache.environment':
|
||||
'\n分析器身份缓存：\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir\n 由操作员明确信任的持久缓存，用于跨进程快速查询状态。目录必须预先存在、位于 GitNexus 包/构建根目录之外，且路径中不得包含符号链接或 junction。缺少 POSIX 所有权 API 的平台默认保持故障关闭。',
|
||||
'help.analyze.environment':
|
||||
'\n环境变量：\n GITNEXUS_NO_GITIGNORE=1 跳过 .gitignore 解析（仍读取 .gitnexusignore）\n GITNEXUS_MAX_FILE_SIZE=N 覆盖大文件跳过阈值（KB）。默认 512，最大 32768。\n GITNEXUS_STORAGE_PATH=/absolute/index 完整外部索引目录。保留既有配置语义；与 GITNEXUS_STORAGE_ROOT 同时设置时优先使用。\n GITNEXUS_STORAGE_ROOT=/absolute/root 外部索引根目录；每个仓库使用独立的 <仓库名>-<规范路径哈希>/ 子目录。\n GITNEXUS_CONTENT_RETENTION=full 源码文本保留策略：full、symbol 或 none。默认 full。\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir 由操作员明确信任的持久分析器身份缓存；目录必须预先存在、位于包/构建根目录之外，且路径中不得包含符号链接或 junction。\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker 空闲超时（毫秒）。默认 30000。\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL 自动 checkpoint 阈值（字节，默认 67108864 = 64 MiB；-1 保持 Ladybug 默认约 16 MiB）。\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker 作业字节预算。默认 8388608。\n GITNEXUS_WORKER_POOL_SIZE=N 解析 worker 数量覆盖值。默认 cores-1，最多 16。\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N 并发进行中的解析分块数。默认 2。\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N 每个 slot 丢弃前允许的最大替换进程数。默认 3。\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N 每个作业的总重试墙钟时间。默认 5 倍子批次超时。\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N 每个 slot 触发熔断的死亡次数。默认 max(3, poolSize)。\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N 线程池关闭时等待仍在原生代码中的已退役 worker 的最长时间（到达安全点后再终止，避免进程级 abort）。默认 30000。\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N C++ 捕获提取的每文件墙钟预算；超出后该文件保留部分捕获并输出警告。默认 20000。\n GITNEXUS_EMBEDDING_THREADS=N 限制 --embeddings 的本地 ONNX CPU 线程数。\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 将单次 HTTP 嵌入超时纳入 GITNEXUS_EMBEDDING_MAX_ATTEMPTS 重试（默认关闭，超时仍为终止错误）。\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N exact-scan 回退的最大嵌入分块数。默认 10000。\n GITNEXUS_VECTOR_MAX_DISTANCE=N 语义/向量搜索接受的最大余弦距离（0 < N <= 2；超出则钳制为 2）。MCP 默认 0.6，其他路径默认 0.5。\n\n当参数和对应环境变量同时提供时，参数优先。\n\n提示：`.gitnexusignore` 支持 `.gitignore` 风格的取反。比如添加\n `!__tests__/` 可以索引默认自动过滤的目录（#771）。',
|
||||
'\n环境变量：\n GITNEXUS_NO_GITIGNORE=1 跳过 .gitignore 解析（仍读取 .gitnexusignore）\n GITNEXUS_MAX_FILE_SIZE=N 覆盖大文件跳过阈值（KB）。默认 512，最大 32768。\n GITNEXUS_STORAGE_PATH=/absolute/index 完整外部索引目录。保留既有配置语义；与 GITNEXUS_STORAGE_ROOT 同时设置时优先使用。\n GITNEXUS_STORAGE_ROOT=/absolute/root 外部索引根目录；每个仓库使用独立的 <仓库名>-<规范路径哈希>/ 子目录。\n GITNEXUS_CONTENT_RETENTION=full 源码文本保留策略：full、symbol 或 none。默认 full。\n GITNEXUS_ANALYZER_IDENTITY_CACHE_DIR=/absolute/protected/dir 由操作员明确信任的持久分析器身份缓存；目录必须预先存在、位于包/构建根目录之外，且路径中不得包含符号链接或 junction。\n GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS=N Worker 空闲超时（毫秒）。默认 30000。\n GITNEXUS_WAL_CHECKPOINT_THRESHOLD=N LadybugDB WAL 自动 checkpoint 阈值（字节，默认 67108864 = 64 MiB；-1 保持 Ladybug 默认约 16 MiB）。\n GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES=N Worker 作业字节预算。默认 8388608。\n GITNEXUS_WORKER_POOL_SIZE=N 解析 worker 数量覆盖值。默认 cores-1，最多 16。\n GITNEXUS_PARSE_CHUNK_CONCURRENCY=N 并发进行中的解析分块数。默认 2。\n GITNEXUS_WORKER_MAX_RESPAWNS_PER_SLOT=N 每个 slot 丢弃前允许的最大替换进程数。默认 3。\n GITNEXUS_WORKER_MAX_CUMULATIVE_TIMEOUT_MS=N 每个作业的总重试墙钟时间。默认 5 倍子批次超时。\n GITNEXUS_WORKER_CONSECUTIVE_FAILURE_THRESHOLD=N 每个 slot 触发熔断的死亡次数。默认 max(3, poolSize)。\n GITNEXUS_WORKER_SHUTDOWN_DRAIN_MS=N 线程池关闭时等待仍在原生代码中的已退役 worker 的最长时间（到达安全点后再终止，避免进程级 abort）。默认 30000。\n GITNEXUS_CPP_CAPTURE_BUDGET_MS=N C++ 捕获提取的每文件墙钟预算；超出后该文件保留部分捕获并输出警告。默认 20000。\n GITNEXUS_EMBEDDING_THREADS=N 限制 --embeddings 的本地 ONNX CPU 线程数。\n GITNEXUS_EMBEDDING_RETRY_TIMEOUTS=1 将单次 HTTP 嵌入超时纳入 GITNEXUS_EMBEDDING_MAX_ATTEMPTS 重试（默认关闭，超时仍为终止错误）。\n GITNEXUS_SEMANTIC_EXACT_SCAN_LIMIT=N exact-scan 回退的最大嵌入分块数。默认 10000。\n GITNEXUS_VECTOR_MAX_DISTANCE=N 语义/向量搜索接受的最大余弦距离（0 < N <= 2；超出则钳制为 2）。MCP 默认 0.6，其他路径默认 0.5。\n GITNEXUS_MAX_PROCESSES=N 流程检测的流程数量上限（正整数）。覆盖动态的 max(20, round(symbols/10)) 公式。与查询时的 IMPACT_MAX_CHUNKS 无关。\n GITNEXUS_MAX_PROCESS_BRANCHING=N 流程检测的单节点分支上限。默认 4。\n GITNEXUS_MAX_PROCESS_TRACE_DEPTH=N 流程检测的 DFS 深度上限。默认 10。\n GITNEXUS_MAX_ENTRY_POINT_CANDIDATES=N 排序后的入口点候选池。默认 200。仅在警告点名该上限时提高；那时通常先翻倍。\n\nCLI 参数优先于 `.gitnexusrc`，后者优先于环境变量，环境变量优先于内置默认值。\n\n提示：`.gitnexusignore` 支持 `.gitignore` 风格的取反。比如添加\n `!__tests__/` 可以索引默认自动过滤的目录（#771）。',
|
||||
} satisfies EnglishMessages;
|
||||
|
|
|
|||
|
|
@ -163,6 +163,22 @@ program
|
|||
'--workers <n>',
|
||||
'Parse worker pool size (>=1). Default: cores-1 capped at 16, auto-sized to the repo.',
|
||||
)
|
||||
.option(
|
||||
'--max-processes <n>',
|
||||
'Process-detection process cap (positive integer). Replaces the dynamic max(20, round(symbols/10)) formula. Default: dynamic.',
|
||||
)
|
||||
.option(
|
||||
'--max-process-branching <n>',
|
||||
'Process-detection per-node branching cap (positive integer). Default: 4.',
|
||||
)
|
||||
.option(
|
||||
'--max-process-trace-depth <n>',
|
||||
'Process-detection DFS depth cap (positive integer). Default: 10.',
|
||||
)
|
||||
.option(
|
||||
'--max-entry-point-candidates <n>',
|
||||
'Ranked entry-point candidate pool (positive integer). Default: 200. Raise when the warning names this knob; doubling is the usual first raise.',
|
||||
)
|
||||
.option(
|
||||
'--spring-actuator <path>',
|
||||
'Import local Spring Boot Actuator JSON snapshots (mappings, beans, conditions, ' +
|
||||
|
|
|
|||
|
|
@ -19,6 +19,12 @@ import type { ToolsOutput } from './tools.js';
|
|||
import type { StructureOutput } from './structure.js';
|
||||
import type { ParseOutput } from './parse.js';
|
||||
import { processProcesses, type ProcessDetectionResult } from '../process-processor.js';
|
||||
import {
|
||||
buildProcessDetectionPhaseConfig,
|
||||
formatWholeFlowsMissingRemedies,
|
||||
processDetectionEffectiveLimits,
|
||||
resolveProcessDetectionBudget,
|
||||
} from '../process-detection-budget.js';
|
||||
import { generateId } from '../../../lib/utils.js';
|
||||
import { routeNodeKey } from '../route-extractors/route-path.js';
|
||||
import { isDev } from '../utils/env.js';
|
||||
|
|
@ -79,11 +85,29 @@ export const processesPhase: PipelinePhase<ProcessesOutput> = {
|
|||
stats: { filesProcessed: totalFiles, totalFiles, nodesCreated: ctx.graph.nodeCount },
|
||||
});
|
||||
|
||||
const resolvedBudget = resolveProcessDetectionBudget(
|
||||
{
|
||||
maxProcesses: ctx.options?.maxProcesses,
|
||||
maxProcessBranching: ctx.options?.maxProcessBranching,
|
||||
maxProcessTraceDepth: ctx.options?.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: ctx.options?.maxEntryPointCandidates,
|
||||
},
|
||||
// Env is resolved in `runFullAnalysis` and threaded on PipelineOptions.
|
||||
// The phase reads only those fields so unit tests stay isolated from
|
||||
// the host environment.
|
||||
{},
|
||||
);
|
||||
let symbolCount = 0;
|
||||
ctx.graph.forEachNode((n) => {
|
||||
if (n.label !== 'File') symbolCount++;
|
||||
});
|
||||
const dynamicMaxProcesses = computeDynamicMaxProcesses(symbolCount);
|
||||
if (resolvedBudget.maxProcesses === undefined) {
|
||||
ctx.graph.forEachNode((n) => {
|
||||
if (n.label !== 'File') symbolCount++;
|
||||
});
|
||||
}
|
||||
const detectionConfig = buildProcessDetectionPhaseConfig(
|
||||
resolvedBudget,
|
||||
symbolCount,
|
||||
computeDynamicMaxProcesses,
|
||||
);
|
||||
|
||||
// R3-6: where the program reaches outward. Already collected by the parse
|
||||
// phase for FILE-level FETCHES/QUERIES edges; reused here at function
|
||||
|
|
@ -123,7 +147,7 @@ export const processesPhase: PipelinePhase<ProcessesOutput> = {
|
|||
stats: { filesProcessed: totalFiles, totalFiles, nodesCreated: ctx.graph.nodeCount },
|
||||
});
|
||||
},
|
||||
{ maxProcesses: dynamicMaxProcesses, minSteps: 3 },
|
||||
detectionConfig,
|
||||
outwardActionSites,
|
||||
);
|
||||
|
||||
|
|
@ -143,8 +167,8 @@ export const processesPhase: PipelinePhase<ProcessesOutput> = {
|
|||
// "unexplored entry points mean whole flows are missing, while a
|
||||
// depth-capped trace means a flow is present but shorter than it really is"
|
||||
// — and it is what keeps the line worth reading. Warning on every counter
|
||||
// meant warning on every run: this phase overrides only `maxProcesses`, so
|
||||
// at the shipped defaults (`maxBranching: 4`, `maxTraceDepth: 10`,
|
||||
// meant warning on every run: at the shipped defaults (`maxBranching: 4`,
|
||||
// `maxTraceDepth: 10`,
|
||||
// per-entry trace budget 12) `calleesDropped` fires for any function with
|
||||
// five callees, `tracesDepthCapped` for any chain deeper than ten, and
|
||||
// `walksCutByBudget` for any entry point with twelve paths under it. All
|
||||
|
|
@ -171,6 +195,15 @@ export const processesPhase: PipelinePhase<ProcessesOutput> = {
|
|||
truncation.entryPointCandidatesDropped > 0 ||
|
||||
truncation.entryPointsUnexplored > 0 ||
|
||||
truncation.processesDropped > 0;
|
||||
const effectiveLimits = processDetectionEffectiveLimits(
|
||||
detectionConfig.maxProcesses,
|
||||
resolvedBudget,
|
||||
);
|
||||
const remedies = formatWholeFlowsMissingRemedies(
|
||||
truncation,
|
||||
effectiveLimits,
|
||||
entryPointCandidates,
|
||||
);
|
||||
const shape =
|
||||
`${truncation.entryPointCandidatesDropped} of ${entryPointCandidates} candidate entry point(s) never ranked in, ` +
|
||||
`${truncation.entryPointsUnexplored} ranked entry point(s) never traced, ` +
|
||||
|
|
@ -180,13 +213,13 @@ export const processesPhase: PipelinePhase<ProcessesOutput> = {
|
|||
`${truncation.walksCutByBudget} walk(s) cut by the per-entry trace budget.`;
|
||||
if (flowsMissing) {
|
||||
logger.warn(
|
||||
{ truncation },
|
||||
{ truncation, effectiveLimits },
|
||||
`[processes] ${processResult.stats.totalProcesses} flows reported, but whole flows are MISSING: ` +
|
||||
`${shape} An absent flow does NOT mean the code path does not exist.`,
|
||||
`${shape}${remedies} An absent flow does NOT mean the code path does not exist.`,
|
||||
);
|
||||
} else if (truncation.truncated) {
|
||||
logger.debug(
|
||||
{ truncation },
|
||||
{ truncation, effectiveLimits },
|
||||
`[processes] ${processResult.stats.totalProcesses} flows reported; every flow found is present, ` +
|
||||
`but some are shorter than the code path they describe: ${shape}`,
|
||||
);
|
||||
|
|
|
|||
|
|
@ -250,6 +250,15 @@ export interface PipelineOptions {
|
|||
* analyze invocations.
|
||||
*/
|
||||
workerPoolSize?: number;
|
||||
/**
|
||||
* Process-detection budget (#3313). Explicit `maxProcesses` replaces the
|
||||
* dynamic `symbols / 10` formula; the other three replace compiled defaults.
|
||||
* Unset fields keep shipped behavior. `0` is rejected upstream — not unlimited.
|
||||
*/
|
||||
maxProcesses?: number;
|
||||
maxProcessBranching?: number;
|
||||
maxProcessTraceDepth?: number;
|
||||
maxEntryPointCandidates?: number;
|
||||
/**
|
||||
* Number of chunks whose file contents may be read into memory in
|
||||
* parallel while the worker pool is busy dispatching the current
|
||||
|
|
@ -443,8 +452,8 @@ export const runPipelineFromRepo = async (
|
|||
const propertyInference = scopeResolutionOutput.propertyInference;
|
||||
|
||||
// Presence check, not `!skipGraphPhases`: phases can now be filtered out by
|
||||
// any `enabledWhen` predicate (streamGraphEmit disables communities/processes
|
||||
// too), and `getPhaseOutput` THROWS on a phase that was never resolved. Keying
|
||||
// any `enabledWhen` predicate (`skipGraphPhases` drops communities/processes),
|
||||
// and `getPhaseOutput` THROWS on a phase that was never resolved. Keying
|
||||
// off the options flag alone made every filtered-out combination crash here
|
||||
// rather than return undefined results.
|
||||
if (results.has('communities') && results.has('processes')) {
|
||||
|
|
|
|||
304
gitnexus/src/core/ingestion/process-detection-budget.ts
Normal file
304
gitnexus/src/core/ingestion/process-detection-budget.ts
Normal file
|
|
@ -0,0 +1,304 @@
|
|||
/**
|
||||
* Analyze-time process-detection budget (#3313).
|
||||
*
|
||||
* Four knobs control how many execution flows `processProcesses` keeps:
|
||||
* process count, per-node branching, trace depth, and the ranked entry-point
|
||||
* pool. Precedence is CLI / explicit `AnalyzeOptions` > `.gitnexusrc` (already
|
||||
* merged into those fields) > `GITNEXUS_*` env > built-in defaults. Unset
|
||||
* `maxProcesses` keeps the dynamic `symbols / 10` formula. `0` is invalid, not
|
||||
* unlimited.
|
||||
*/
|
||||
|
||||
import type { ProcessDetectionConfig, ProcessTruncationStats } from './process-processor.js';
|
||||
import { parsePositiveIntEnv } from './utils/env.js';
|
||||
|
||||
export const PROCESS_DETECTION_BUDGET_DEFAULTS = {
|
||||
maxProcessBranching: 4,
|
||||
maxProcessTraceDepth: 10,
|
||||
maxEntryPointCandidates: 200,
|
||||
minSteps: 3,
|
||||
} as const;
|
||||
|
||||
export const PROCESS_DETECTION_ENV = {
|
||||
maxProcesses: 'GITNEXUS_MAX_PROCESSES',
|
||||
maxProcessBranching: 'GITNEXUS_MAX_PROCESS_BRANCHING',
|
||||
maxProcessTraceDepth: 'GITNEXUS_MAX_PROCESS_TRACE_DEPTH',
|
||||
maxEntryPointCandidates: 'GITNEXUS_MAX_ENTRY_POINT_CANDIDATES',
|
||||
} as const;
|
||||
|
||||
export const PROCESS_DETECTION_CLI_FLAGS = {
|
||||
maxProcesses: '--max-processes',
|
||||
maxProcessBranching: '--max-process-branching',
|
||||
maxProcessTraceDepth: '--max-process-trace-depth',
|
||||
maxEntryPointCandidates: '--max-entry-point-candidates',
|
||||
} as const;
|
||||
|
||||
const BUDGET_KEYS = [
|
||||
'maxProcesses',
|
||||
'maxProcessBranching',
|
||||
'maxProcessTraceDepth',
|
||||
'maxEntryPointCandidates',
|
||||
] as const;
|
||||
|
||||
export type ProcessDetectionBudgetKey = (typeof BUDGET_KEYS)[number];
|
||||
|
||||
export type ProcessDetectionBudgetFields = {
|
||||
maxProcesses?: number;
|
||||
maxProcessBranching?: number;
|
||||
maxProcessTraceDepth?: number;
|
||||
maxEntryPointCandidates?: number;
|
||||
};
|
||||
|
||||
export type ProcessDetectionBudgetStrings = {
|
||||
maxProcesses?: string;
|
||||
maxProcessBranching?: string;
|
||||
maxProcessTraceDepth?: string;
|
||||
maxEntryPointCandidates?: string;
|
||||
};
|
||||
|
||||
export type ProcessDetectionStamp = {
|
||||
/** Explicit override, or `null` when this run used the dynamic formula. */
|
||||
maxProcesses: number | null;
|
||||
maxProcessBranching: number;
|
||||
maxProcessTraceDepth: number;
|
||||
maxEntryPointCandidates: number;
|
||||
/**
|
||||
* In-place FTS park after a derived-layer rewrite (#3322). Missing stamp +
|
||||
* defaults is a match, so recovery must persist a complete stamp that still
|
||||
* mismatches until a successful analyze certifies the live Community/Process
|
||||
* rows. Success writes omit this flag.
|
||||
*/
|
||||
uncertified?: true;
|
||||
};
|
||||
|
||||
export type ResolvedProcessDetectionBudget = {
|
||||
/** Set only when an explicit override won. */
|
||||
maxProcesses?: number;
|
||||
maxProcessBranching: number;
|
||||
maxProcessTraceDepth: number;
|
||||
maxEntryPointCandidates: number;
|
||||
overridden: {
|
||||
maxProcesses: boolean;
|
||||
maxProcessBranching: boolean;
|
||||
maxProcessTraceDepth: boolean;
|
||||
maxEntryPointCandidates: boolean;
|
||||
};
|
||||
};
|
||||
|
||||
export type ProcessDetectionEffectiveLimits = {
|
||||
maxProcesses: number;
|
||||
maxProcessBranching: number;
|
||||
maxProcessTraceDepth: number;
|
||||
maxEntryPointCandidates: number;
|
||||
/**
|
||||
* Pre-entry gate: `processProcesses` does not start the next entry once
|
||||
* collected traces already reach `maxProcesses * 2`. One started entry can
|
||||
* still append every trace `traceFromEntryPoint` returns.
|
||||
*/
|
||||
maxProcessTraces: number;
|
||||
};
|
||||
|
||||
export type InvalidBudgetHandler = (knob: string, raw: string) => void;
|
||||
|
||||
/** Operator copy when a CLI/rc/env token is rejected. Next precedence still applies. */
|
||||
export const formatInvalidProcessDetectionOverride = (knob: string, raw: string): string => {
|
||||
const next = knob.startsWith('GITNEXUS_')
|
||||
? 'the built-in default'
|
||||
: 'the next source (env, then the built-in default)';
|
||||
return `${knob} must be a positive integer (got ${JSON.stringify(raw)}); ignoring it so ${next} applies.`;
|
||||
};
|
||||
|
||||
export const parsePositiveIntegerOverride = (
|
||||
raw: string | number | undefined | null,
|
||||
onInvalid?: (raw: string) => void,
|
||||
): number | undefined => {
|
||||
if (raw === undefined || raw === null) return undefined;
|
||||
const text = typeof raw === 'number' ? String(raw) : raw;
|
||||
const parsed = parsePositiveIntEnv(text);
|
||||
if (parsed === undefined) onInvalid?.(typeof raw === 'number' ? text : text.trim());
|
||||
return parsed;
|
||||
};
|
||||
|
||||
export const parseProcessDetectionBudgetStrings = (
|
||||
raw: ProcessDetectionBudgetStrings,
|
||||
onInvalid?: InvalidBudgetHandler,
|
||||
): ProcessDetectionBudgetFields => {
|
||||
const out: ProcessDetectionBudgetFields = {};
|
||||
for (const key of BUDGET_KEYS) {
|
||||
if (raw[key] === undefined) continue;
|
||||
const parsed = parsePositiveIntegerOverride(raw[key], (invalid) =>
|
||||
onInvalid?.(PROCESS_DETECTION_CLI_FLAGS[key], invalid),
|
||||
);
|
||||
if (parsed !== undefined) out[key] = parsed;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
const readOverride = (
|
||||
options: ProcessDetectionBudgetFields,
|
||||
env: NodeJS.ProcessEnv,
|
||||
key: ProcessDetectionBudgetKey,
|
||||
onInvalid?: InvalidBudgetHandler,
|
||||
): number | undefined => {
|
||||
const fromOptions = options[key];
|
||||
if (fromOptions !== undefined) {
|
||||
const parsed = parsePositiveIntegerOverride(fromOptions, (invalid) =>
|
||||
onInvalid?.(PROCESS_DETECTION_CLI_FLAGS[key], invalid),
|
||||
);
|
||||
if (parsed !== undefined) return parsed;
|
||||
}
|
||||
const envName = PROCESS_DETECTION_ENV[key];
|
||||
const fromEnv = env[envName];
|
||||
if (fromEnv === undefined) return undefined;
|
||||
return parsePositiveIntegerOverride(fromEnv, (invalid) => onInvalid?.(envName, invalid));
|
||||
};
|
||||
|
||||
export const resolveProcessDetectionBudget = (
|
||||
options: ProcessDetectionBudgetFields = {},
|
||||
env: NodeJS.ProcessEnv = process.env,
|
||||
onInvalid?: InvalidBudgetHandler,
|
||||
): ResolvedProcessDetectionBudget => {
|
||||
const maxProcesses = readOverride(options, env, 'maxProcesses', onInvalid);
|
||||
const branching = readOverride(options, env, 'maxProcessBranching', onInvalid);
|
||||
const depth = readOverride(options, env, 'maxProcessTraceDepth', onInvalid);
|
||||
const entryPoints = readOverride(options, env, 'maxEntryPointCandidates', onInvalid);
|
||||
return {
|
||||
...(maxProcesses === undefined ? {} : { maxProcesses }),
|
||||
maxProcessBranching: branching ?? PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessBranching,
|
||||
maxProcessTraceDepth: depth ?? PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates:
|
||||
entryPoints ?? PROCESS_DETECTION_BUDGET_DEFAULTS.maxEntryPointCandidates,
|
||||
overridden: {
|
||||
maxProcesses: maxProcesses !== undefined,
|
||||
maxProcessBranching: branching !== undefined,
|
||||
maxProcessTraceDepth: depth !== undefined,
|
||||
maxEntryPointCandidates: entryPoints !== undefined,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const hasProcessDetectionOverride = (resolved: ResolvedProcessDetectionBudget): boolean =>
|
||||
resolved.overridden.maxProcesses ||
|
||||
resolved.overridden.maxProcessBranching ||
|
||||
resolved.overridden.maxProcessTraceDepth ||
|
||||
resolved.overridden.maxEntryPointCandidates;
|
||||
|
||||
export const toProcessDetectionStamp = (
|
||||
resolved: ResolvedProcessDetectionBudget,
|
||||
): ProcessDetectionStamp => ({
|
||||
maxProcesses: resolved.maxProcesses ?? null,
|
||||
maxProcessBranching: resolved.maxProcessBranching,
|
||||
maxProcessTraceDepth: resolved.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: resolved.maxEntryPointCandidates,
|
||||
});
|
||||
|
||||
/** Complete stamp that always mismatches until the next successful analyze. */
|
||||
export const uncertifyProcessDetectionStamp = (
|
||||
recorded: ProcessDetectionStamp | undefined,
|
||||
): ProcessDetectionStamp => ({
|
||||
maxProcesses: recorded?.maxProcesses ?? null,
|
||||
maxProcessBranching:
|
||||
recorded?.maxProcessBranching ?? PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessBranching,
|
||||
maxProcessTraceDepth:
|
||||
recorded?.maxProcessTraceDepth ?? PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates:
|
||||
recorded?.maxEntryPointCandidates ?? PROCESS_DETECTION_BUDGET_DEFAULTS.maxEntryPointCandidates,
|
||||
uncertified: true,
|
||||
});
|
||||
|
||||
const isCompleteStamp = (
|
||||
recorded: ProcessDetectionStamp | undefined,
|
||||
): recorded is ProcessDetectionStamp =>
|
||||
recorded !== undefined &&
|
||||
(recorded.maxProcesses === null ||
|
||||
(typeof recorded.maxProcesses === 'number' && Number.isInteger(recorded.maxProcesses))) &&
|
||||
Number.isInteger(recorded.maxProcessBranching) &&
|
||||
Number.isInteger(recorded.maxProcessTraceDepth) &&
|
||||
Number.isInteger(recorded.maxEntryPointCandidates);
|
||||
|
||||
export const processDetectionBudgetMismatch = (
|
||||
recorded: ProcessDetectionStamp | undefined,
|
||||
resolved: ResolvedProcessDetectionBudget,
|
||||
): boolean => {
|
||||
if (recorded?.uncertified === true) return true;
|
||||
if (!isCompleteStamp(recorded)) {
|
||||
// Legacy meta: same defaults as today's shipped behavior stay a match so
|
||||
// an upgrade backfills the stamp instead of re-detecting. Any explicit
|
||||
// override is a mismatch — otherwise a budget-only raise on a pre-#3313
|
||||
// index would preserve the sampled Community/Process layer.
|
||||
return hasProcessDetectionOverride(resolved);
|
||||
}
|
||||
const stamp = toProcessDetectionStamp(resolved);
|
||||
return (
|
||||
recorded.maxProcesses !== stamp.maxProcesses ||
|
||||
recorded.maxProcessBranching !== stamp.maxProcessBranching ||
|
||||
recorded.maxProcessTraceDepth !== stamp.maxProcessTraceDepth ||
|
||||
recorded.maxEntryPointCandidates !== stamp.maxEntryPointCandidates
|
||||
);
|
||||
};
|
||||
|
||||
export const buildProcessDetectionPhaseConfig = (
|
||||
resolved: ResolvedProcessDetectionBudget,
|
||||
symbolCount: number,
|
||||
computeDynamicMaxProcesses: (n: number) => number,
|
||||
): Pick<
|
||||
ProcessDetectionConfig,
|
||||
'maxProcesses' | 'maxBranching' | 'maxTraceDepth' | 'maxEntryPointCandidates' | 'minSteps'
|
||||
> => ({
|
||||
maxProcesses: resolved.maxProcesses ?? computeDynamicMaxProcesses(symbolCount),
|
||||
maxBranching: resolved.maxProcessBranching,
|
||||
maxTraceDepth: resolved.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: resolved.maxEntryPointCandidates,
|
||||
minSteps: PROCESS_DETECTION_BUDGET_DEFAULTS.minSteps,
|
||||
});
|
||||
|
||||
export const processDetectionEffectiveLimits = (
|
||||
maxProcesses: number,
|
||||
resolved: ResolvedProcessDetectionBudget,
|
||||
): ProcessDetectionEffectiveLimits => ({
|
||||
maxProcesses,
|
||||
maxProcessBranching: resolved.maxProcessBranching,
|
||||
maxProcessTraceDepth: resolved.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: resolved.maxEntryPointCandidates,
|
||||
maxProcessTraces: maxProcesses * 2,
|
||||
});
|
||||
|
||||
export const formatWholeFlowsMissingRemedies = (
|
||||
truncation: Pick<
|
||||
ProcessTruncationStats,
|
||||
'entryPointCandidatesDropped' | 'entryPointsUnexplored' | 'processesDropped'
|
||||
>,
|
||||
limits: ProcessDetectionEffectiveLimits,
|
||||
observedEntryPointCandidates: number,
|
||||
): string => {
|
||||
const parts: string[] = [];
|
||||
if (truncation.entryPointCandidatesDropped > 0) {
|
||||
parts.push(
|
||||
`${PROCESS_DETECTION_CLI_FLAGS.maxEntryPointCandidates} ` +
|
||||
`(this run ranked ${observedEntryPointCandidates} candidate(s))`,
|
||||
);
|
||||
}
|
||||
if (truncation.entryPointsUnexplored > 0 || truncation.processesDropped > 0) {
|
||||
parts.push(
|
||||
`${PROCESS_DETECTION_CLI_FLAGS.maxProcesses} ` +
|
||||
`(this run used ${limits.maxProcesses}; next entry is skipped once ` +
|
||||
`collected traces reach ${limits.maxProcessTraces})`,
|
||||
);
|
||||
}
|
||||
return parts.length === 0 ? '' : ` Raise ${parts.join('; ')}.`;
|
||||
};
|
||||
|
||||
export const formatProcessDetectionBudgetBanner = (
|
||||
resolved: ResolvedProcessDetectionBudget,
|
||||
): string | null => {
|
||||
if (!hasProcessDetectionOverride(resolved)) return null;
|
||||
const maxProcesses = resolved.overridden.maxProcesses
|
||||
? String(resolved.maxProcesses)
|
||||
: 'dynamic (max(20, round(symbols/10)))';
|
||||
return (
|
||||
` Process-detection budget: maxProcesses=${maxProcesses}, ` +
|
||||
`maxProcessBranching=${resolved.maxProcessBranching}, ` +
|
||||
`maxProcessTraceDepth=${resolved.maxProcessTraceDepth}, ` +
|
||||
`maxEntryPointCandidates=${resolved.maxEntryPointCandidates}`
|
||||
);
|
||||
};
|
||||
|
|
@ -17,6 +17,7 @@ import { CommunityMembership } from './community-processor.js';
|
|||
import { calculateEntryPointScore, isTestFile } from './entry-point-scoring.js';
|
||||
import { SupportedLanguages } from 'gitnexus-shared';
|
||||
import { isDev } from './utils/env.js';
|
||||
import { PROCESS_DETECTION_BUDGET_DEFAULTS } from './process-detection-budget.js';
|
||||
|
||||
import { logger } from '../logger.js';
|
||||
// ============================================================================
|
||||
|
|
@ -25,16 +26,18 @@ import { logger } from '../logger.js';
|
|||
|
||||
export interface ProcessDetectionConfig {
|
||||
maxTraceDepth: number; // Maximum steps to trace (default: 10)
|
||||
maxBranching: number; // Max branches to follow per node (default: 3)
|
||||
maxProcesses: number; // Maximum processes to detect (default: 50)
|
||||
minSteps: number; // Minimum steps for a valid process (default: 2)
|
||||
maxBranching: number; // Max branches to follow per node (default: 4)
|
||||
maxProcesses: number; // Maximum processes to detect (default: 75)
|
||||
minSteps: number; // Minimum steps for a valid process (default: 3)
|
||||
maxEntryPointCandidates: number; // Ranked entry-point pool (default: 200)
|
||||
}
|
||||
|
||||
const DEFAULT_CONFIG: ProcessDetectionConfig = {
|
||||
maxTraceDepth: 10,
|
||||
maxBranching: 4,
|
||||
export const DEFAULT_CONFIG: ProcessDetectionConfig = {
|
||||
maxTraceDepth: PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessTraceDepth,
|
||||
maxBranching: PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessBranching,
|
||||
maxProcesses: 75,
|
||||
minSteps: 3, // 3+ steps = genuine multi-hop flow (2-step is just "A calls B")
|
||||
minSteps: PROCESS_DETECTION_BUDGET_DEFAULTS.minSteps, // 3+ steps = genuine multi-hop flow (2-step is just "A calls B")
|
||||
maxEntryPointCandidates: PROCESS_DETECTION_BUDGET_DEFAULTS.maxEntryPointCandidates,
|
||||
};
|
||||
|
||||
// ============================================================================
|
||||
|
|
@ -89,7 +92,7 @@ export interface ProcessTruncationStats {
|
|||
truncated: boolean;
|
||||
/**
|
||||
* Scoring candidates that never reached the trace loop because
|
||||
* `findEntryPoints` keeps only the top `ENTRY_POINT_CANDIDATE_LIMIT`.
|
||||
* `findEntryPoints` keeps only the top `maxEntryPointCandidates`.
|
||||
* Counted BEFORE the slice, so it sees what `entryPointsFound` cannot.
|
||||
*/
|
||||
entryPointCandidatesDropped: number;
|
||||
|
|
@ -166,11 +169,17 @@ export const processProcesses = async (
|
|||
for (const n of knowledgeGraph.iterNodes()) nodeMap.set(n.id, n);
|
||||
|
||||
// Declared before Step 1 because `findEntryPoints` has a ceiling of its own
|
||||
// (see `ENTRY_POINT_CANDIDATE_LIMIT`) and reports it through the same record.
|
||||
// (`cfg.maxEntryPointCandidates`, default 200) and reports it through the same record.
|
||||
const truncation = emptyTruncation();
|
||||
|
||||
// Step 1: Find entry points (functions that call others but have few callers)
|
||||
const entryPoints = findEntryPoints(knowledgeGraph, reverseCallsEdges, callsEdges, truncation);
|
||||
const entryPoints = findEntryPoints(
|
||||
knowledgeGraph,
|
||||
reverseCallsEdges,
|
||||
callsEdges,
|
||||
truncation,
|
||||
cfg.maxEntryPointCandidates,
|
||||
);
|
||||
|
||||
onProgress?.(`Found ${entryPoints.length} entry points, tracing flows...`, 20);
|
||||
|
||||
|
|
@ -199,7 +208,7 @@ export const processProcesses = async (
|
|||
// the remainder are not "no flows found" — they were never looked at.
|
||||
//
|
||||
// Counted over the list `findEntryPoints` RETURNS, which is already capped at
|
||||
// `ENTRY_POINT_CANDIDATE_LIMIT`; candidates beyond that cap are invisible here
|
||||
// `maxEntryPointCandidates`; candidates beyond that cap are invisible here
|
||||
// by construction and are reported separately as
|
||||
// `entryPointCandidatesDropped`.
|
||||
truncation.entryPointsUnexplored = entryPoints.length - tracedEntryPoints;
|
||||
|
|
@ -480,13 +489,6 @@ const buildReverseCallsGraph = (graph: KnowledgeGraph): AdjacencyList => {
|
|||
return adj;
|
||||
};
|
||||
|
||||
/**
|
||||
* How many ranked candidates survive to be traced. Everything below this line
|
||||
* is discarded — see `ProcessTruncationStats.entryPointCandidatesDropped`, the
|
||||
* counter that exists because this cap spent a release being invisible.
|
||||
*/
|
||||
const ENTRY_POINT_CANDIDATE_LIMIT = 200;
|
||||
|
||||
/**
|
||||
* Find functions/methods that are good entry points for tracing.
|
||||
*
|
||||
|
|
@ -495,7 +497,9 @@ const ENTRY_POINT_CANDIDATE_LIMIT = 200;
|
|||
* 2. Export status (exported/public functions rank higher)
|
||||
* 3. Name patterns (handle*, on*, *Controller, etc.)
|
||||
*
|
||||
* Test files are excluded entirely.
|
||||
* Test files are excluded entirely. How many ranked candidates survive to be
|
||||
* traced is `maxEntryPointCandidates` (default 200) — see
|
||||
* `ProcessTruncationStats.entryPointCandidatesDropped`.
|
||||
*/
|
||||
const findEntryPoints = (
|
||||
graph: KnowledgeGraph,
|
||||
|
|
@ -509,6 +513,7 @@ const findEntryPoints = (
|
|||
* to unwrap a counter to ask for entry points.
|
||||
*/
|
||||
truncation?: ProcessTruncationStats,
|
||||
maxEntryPointCandidates: number = DEFAULT_CONFIG.maxEntryPointCandidates,
|
||||
): string[] => {
|
||||
const symbolTypes = new Set<NodeLabel>(['Function', 'Method']);
|
||||
const entryPointCandidates: {
|
||||
|
|
@ -576,15 +581,15 @@ const findEntryPoints = (
|
|||
|
||||
// Limit to prevent explosion — and SAY SO. This is the ceiling that decides
|
||||
// how much of a repository process detection ever looks at: on anything with
|
||||
// more than 200 scoring candidates the reported flows are a sample of the
|
||||
// top-ranked ones, and every downstream count (`entryPointsFound`,
|
||||
// `entryPointsUnexplored`) is computed over the survivors, so none of them can
|
||||
// see what was cut here.
|
||||
if (truncation !== undefined && sorted.length > ENTRY_POINT_CANDIDATE_LIMIT) {
|
||||
truncation.entryPointCandidatesDropped = sorted.length - ENTRY_POINT_CANDIDATE_LIMIT;
|
||||
// more scoring candidates than `maxEntryPointCandidates` the reported flows
|
||||
// are a sample of the top-ranked ones, and every downstream count
|
||||
// (`entryPointsFound`, `entryPointsUnexplored`) is computed over the
|
||||
// survivors, so none of them can see what was cut here.
|
||||
if (truncation !== undefined && sorted.length > maxEntryPointCandidates) {
|
||||
truncation.entryPointCandidatesDropped = sorted.length - maxEntryPointCandidates;
|
||||
}
|
||||
|
||||
return sorted.slice(0, ENTRY_POINT_CANDIDATE_LIMIT).map((c) => c.id);
|
||||
return sorted.slice(0, maxEntryPointCandidates).map((c) => c.id);
|
||||
};
|
||||
|
||||
// ============================================================================
|
||||
|
|
@ -822,7 +827,7 @@ const compareOrderKeys = (a: string, b: string): number => (a < b ? -1 : a > b ?
|
|||
* joined strings, so an O(n log n) sort performs O(n log n) joins of
|
||||
* O(depth x id-length) characters each.
|
||||
*
|
||||
* `n` is bounded here (`ENTRY_POINT_CANDIDATE_LIMIT` entry points x the
|
||||
* `n` is bounded here (`maxEntryPointCandidates` entry points x the
|
||||
* per-entry trace budget), so the cost is small and once-per-analyze: measured
|
||||
* at the ceiling, 23,851 comparisons performed 70,524 joins, and end-to-end
|
||||
* `processProcesses` at 80,000 functions / 640k CALLS went 456 -> 555 ms. It is
|
||||
|
|
|
|||
|
|
@ -198,6 +198,13 @@ import {
|
|||
nodeTablesForIncrementalDelete,
|
||||
shouldPreservePersistedDerivedGraph,
|
||||
} from './incremental/derived-writeback.js';
|
||||
import {
|
||||
formatInvalidProcessDetectionOverride,
|
||||
processDetectionBudgetMismatch,
|
||||
resolveProcessDetectionBudget,
|
||||
toProcessDetectionStamp,
|
||||
uncertifyProcessDetectionStamp,
|
||||
} from './ingestion/process-detection-budget.js';
|
||||
import { NODE_TABLES } from './lbug/schema.js';
|
||||
import {
|
||||
loadParseCache,
|
||||
|
|
@ -485,8 +492,9 @@ export interface AnalyzeOptions {
|
|||
pdgEmitChunkSize?: number;
|
||||
/** Streamed structural graph emit (#2680). Honored only on a full rebuild
|
||||
* (`force === true`). May also be enabled via `GITNEXUS_STREAM_GRAPH_EMIT`.
|
||||
* Trades community detection, process extraction and PDG taint summaries for
|
||||
* a ~2.9x reduction of in-memory graph heap. */
|
||||
* The sink answers a complete relationship read, so community detection,
|
||||
* process extraction, and PDG taint summaries still run; streaming reduces
|
||||
* in-memory graph heap (~2.9x) by keeping those edges on disk. */
|
||||
streamGraphEmit?: boolean;
|
||||
/**
|
||||
* Default branch threaded into generated AGENTS.md / CLAUDE.md so the
|
||||
|
|
@ -528,6 +536,16 @@ export interface AnalyzeOptions {
|
|||
* removed); `undefined` defers to the env / auto-formula fallback.
|
||||
*/
|
||||
workerPoolSize?: number;
|
||||
/**
|
||||
* Process-detection budget overrides (#3313). Threaded to
|
||||
* `PipelineOptions` without mutating `process.env`. Unset fields fall
|
||||
* back to `GITNEXUS_*` env, then shipped defaults / the dynamic
|
||||
* `maxProcesses` formula.
|
||||
*/
|
||||
maxProcesses?: number;
|
||||
maxProcessBranching?: number;
|
||||
maxProcessTraceDepth?: number;
|
||||
maxEntryPointCandidates?: number;
|
||||
/**
|
||||
* Extra fetch-wrapper function names to treat as HTTP consumers, forwarded to
|
||||
* `PipelineOptions.fetchWrappers` (#1589/#1852 residual). Sourced from the CLI
|
||||
|
|
@ -2066,13 +2084,37 @@ async function runFullAnalysisInner(
|
|||
options = { ...options, force: true };
|
||||
}
|
||||
|
||||
// Process-detection budget (#3313). Resolve CLI/options then env here so
|
||||
// MCP/server jobs honor GITNEXUS_* without a CLI merge. Compare against
|
||||
// the persisted stamp BEFORE the already-up-to-date fast path: a clean
|
||||
// same-commit raise must re-detect flows rather than return the sampled
|
||||
// index. Does NOT set force — incremental empty-diff + skip derived
|
||||
// preserve is enough.
|
||||
const processDetectionBudget = resolveProcessDetectionBudget(
|
||||
{
|
||||
maxProcesses: options.maxProcesses,
|
||||
maxProcessBranching: options.maxProcessBranching,
|
||||
maxProcessTraceDepth: options.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: options.maxEntryPointCandidates,
|
||||
},
|
||||
process.env,
|
||||
(knob, raw) => {
|
||||
log(formatInvalidProcessDetectionOverride(knob, raw));
|
||||
},
|
||||
);
|
||||
const processDetectionMismatch = processDetectionBudgetMismatch(
|
||||
existingMeta?.processDetection,
|
||||
processDetectionBudget,
|
||||
);
|
||||
|
||||
// ── Early-return: already up to date ──────────────────────────────
|
||||
if (
|
||||
existingMeta &&
|
||||
!existingMeta.embeddingCheckpoint &&
|
||||
!options.force &&
|
||||
existingMeta.lastCommit === currentCommit &&
|
||||
!ftsModeChanged
|
||||
!ftsModeChanged &&
|
||||
!processDetectionMismatch
|
||||
) {
|
||||
// Non-git folders have currentCommit = '' — always rebuild since we can't detect changes
|
||||
if (currentCommit !== '') {
|
||||
|
|
@ -2119,6 +2161,8 @@ async function runFullAnalysisInner(
|
|||
// later read on a host where it loads — which is a legitimate, common
|
||||
// state, and the invariant `analyzer-identity-cli.test.ts` pins.
|
||||
if (!dirty && !healUnregistered) {
|
||||
const processDetectionStamp =
|
||||
existingMeta.processDetection ?? toProcessDetectionStamp(processDetectionBudget);
|
||||
if (options.registryName) {
|
||||
await registerRepo(repoPath, existingMeta, {
|
||||
name: options.registryName,
|
||||
|
|
@ -2170,7 +2214,11 @@ async function runFullAnalysisInner(
|
|||
// documented Docker :ro workflow (#1549) — degrades to a warning.
|
||||
try {
|
||||
await adoptFlatBranchLabel(repoPath, branchLabel, storagePath);
|
||||
await saveMeta(metaDir, { ...existingMeta, branch: branchLabel });
|
||||
await saveMeta(metaDir, {
|
||||
...existingMeta,
|
||||
branch: branchLabel,
|
||||
processDetection: processDetectionStamp,
|
||||
});
|
||||
} catch (err) {
|
||||
// EACCES/EPERM also arise from ownership problems and transient
|
||||
// Windows locks, so keep the real error visible alongside the
|
||||
|
|
@ -2183,12 +2231,26 @@ async function runFullAnalysisInner(
|
|||
// Discriminator-only restamp (flag↔env). `existingMeta` already
|
||||
// carries the folded skipReason; persist it without a write plan.
|
||||
try {
|
||||
await saveMeta(metaDir, existingMeta);
|
||||
await saveMeta(metaDir, {
|
||||
...existingMeta,
|
||||
processDetection: processDetectionStamp,
|
||||
});
|
||||
} catch (err) {
|
||||
log(
|
||||
`Warning: could not restamp the FTS skip reason (${formatMetaWriteFailureReason(err)}); will retry on the next run.`,
|
||||
);
|
||||
}
|
||||
} else if (!existingMeta.processDetection) {
|
||||
try {
|
||||
await saveMeta(metaDir, {
|
||||
...existingMeta,
|
||||
processDetection: processDetectionStamp,
|
||||
});
|
||||
} catch (err) {
|
||||
log(
|
||||
`Warning: could not backfill the process-detection stamp (${formatMetaWriteFailureReason(err)}); will retry on the next run.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
await ensureGitNexusIgnored(repoPath, storagePath);
|
||||
return {
|
||||
|
|
@ -2375,6 +2437,16 @@ async function runFullAnalysisInner(
|
|||
{
|
||||
parseCache,
|
||||
workerPoolSize: options.workerPoolSize,
|
||||
maxProcesses: processDetectionBudget.maxProcesses,
|
||||
maxProcessBranching: processDetectionBudget.overridden.maxProcessBranching
|
||||
? processDetectionBudget.maxProcessBranching
|
||||
: undefined,
|
||||
maxProcessTraceDepth: processDetectionBudget.overridden.maxProcessTraceDepth
|
||||
? processDetectionBudget.maxProcessTraceDepth
|
||||
: undefined,
|
||||
maxEntryPointCandidates: processDetectionBudget.overridden.maxEntryPointCandidates
|
||||
? processDetectionBudget.maxEntryPointCandidates
|
||||
: undefined,
|
||||
// CFG/PDG opt-in (#2081 M1). PipelineOptions.pdg fans out to the worker
|
||||
// build gate (workerData.pdg) and the scope-resolution emit gate.
|
||||
pdg: options.pdg === true,
|
||||
|
|
@ -2506,7 +2578,8 @@ async function runFullAnalysisInner(
|
|||
skipDerivedGraphPhases &&
|
||||
isIncremental &&
|
||||
!!hashDiff &&
|
||||
shouldPreservePersistedDerivedGraph(hashDiff);
|
||||
shouldPreservePersistedDerivedGraph(hashDiff) &&
|
||||
!processDetectionMismatch;
|
||||
if (skipDerivedGraphPhases && !preserveDerivedLayer) {
|
||||
progress('communities', 58, 'Detecting code communities and flows...');
|
||||
await pipelineResult.runDeferredDerivedPhases?.();
|
||||
|
|
@ -2589,17 +2662,21 @@ async function runFullAnalysisInner(
|
|||
);
|
||||
// Set the dirty flag BEFORE any destructive DB mutation. Cleared on
|
||||
// success at the meta-save step. Scoped to this branch's meta.json.
|
||||
const now = Date.now();
|
||||
await saveMeta(metaDir, {
|
||||
...existingMeta!,
|
||||
incrementalInProgress: {
|
||||
startedAt: now,
|
||||
updatedAt: now,
|
||||
phase: 'pre-write',
|
||||
toWriteCount: hashDiff.toWrite.length,
|
||||
directWriteCount: hashDiff.toWrite.length,
|
||||
},
|
||||
});
|
||||
// POSIX atomic incremental mutates the copy, so a live dirty stamp would
|
||||
// force-rebuild a healthy index after a crash before swap.
|
||||
if (!atomicIncremental) {
|
||||
const now = Date.now();
|
||||
await saveMeta(metaDir, {
|
||||
...existingMeta!,
|
||||
incrementalInProgress: {
|
||||
startedAt: now,
|
||||
updatedAt: now,
|
||||
phase: 'pre-write',
|
||||
toWriteCount: hashDiff.toWrite.length,
|
||||
directWriteCount: hashDiff.toWrite.length,
|
||||
},
|
||||
});
|
||||
}
|
||||
if (atomicIncremental) {
|
||||
// Stage the live index into the temp so the in-place delete/writeback
|
||||
// below mutates the COPY, and the end-of-run swap publishes it atomically.
|
||||
|
|
@ -3516,6 +3593,11 @@ async function runFullAnalysisInner(
|
|||
lastCommit: '',
|
||||
indexedAt: new Date().toISOString(),
|
||||
};
|
||||
// #3322: persist uncertified *before* CREATE_FTS_INDEX. Park keeps this
|
||||
// stamp; it must not invent one on every FTS-only crash. Missing stamp +
|
||||
// shipped defaults is a match, so a budget-mismatch derived rewrite that
|
||||
// dies in FTS would otherwise recertify the rewritten Community/Process
|
||||
// rows on a flagless retry.
|
||||
await saveMeta(metaDir, {
|
||||
...base,
|
||||
incrementalInProgress: buildFtsDirtyStamp({
|
||||
|
|
@ -3523,6 +3605,11 @@ async function runFullAnalysisInner(
|
|||
writePlan: 'in-place',
|
||||
checkpointSucceeded: boundaryCheckpointSucceeded,
|
||||
}),
|
||||
...(processDetectionMismatch
|
||||
? {
|
||||
processDetection: uncertifyProcessDetectionStamp(base.processDetection),
|
||||
}
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
|
||||
|
|
@ -4529,6 +4616,7 @@ async function runFullAnalysisInner(
|
|||
// stamp after an on→off flip; the next pdgModeMismatch then compares
|
||||
// off==off and incremental eligibility is restored.
|
||||
pdg: resolvePdgConfig(options),
|
||||
processDetection: toProcessDetectionStamp(processDetectionBudget),
|
||||
};
|
||||
// Re-resolve at the commit boundary. Long analyses can overlap an npm
|
||||
// upgrade, rebuilt dist tree, or native dependency replacement; stamping
|
||||
|
|
|
|||
|
|
@ -617,6 +617,22 @@ export interface RepoMeta {
|
|||
*/
|
||||
hasCallSummary?: boolean;
|
||||
};
|
||||
/**
|
||||
* The process-detection budget this index's Community/Process rows were
|
||||
* built under (#3313). Compared on the next analyze so a budget-only
|
||||
* config change re-detects flows without `--force`. `maxProcesses: null`
|
||||
* means the dynamic `symbols / 10` formula. Absent on pre-#3313 metas:
|
||||
* that absence matches default/dynamic knobs (backfill) and mismatches
|
||||
* any explicit override.
|
||||
*/
|
||||
processDetection?: {
|
||||
maxProcesses: number | null;
|
||||
maxProcessBranching: number;
|
||||
maxProcessTraceDepth: number;
|
||||
maxEntryPointCandidates: number;
|
||||
/** Live Community/Process rows are not certified under this stamp (#3322). */
|
||||
uncertified?: true;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -32,12 +32,20 @@ const FIXTURE_SRC = path.resolve(testDir, '..', 'fixtures', 'mini-repo');
|
|||
let MINI_REPO: string;
|
||||
let tmpParent: string;
|
||||
let suiteGitnexusHome: string;
|
||||
/** False when setup analyze fell back to `--skip-fts` after CREATE_FTS_INDEX native-aborted. */
|
||||
let ftsIndexed = false;
|
||||
|
||||
function cliEnv(extraEnv: Record<string, string> = {}) {
|
||||
return {
|
||||
...process.env,
|
||||
GITNEXUS_HOME: suiteGitnexusHome,
|
||||
NODE_OPTIONS: `${process.env.NODE_OPTIONS || ''} --max-old-space-size=8192`.trim(),
|
||||
// Cold parse-worker loads every tree-sitter grammar before the ready
|
||||
// handshake. The default 5s budget classifies that as a deterministic
|
||||
// crash-loop on a loaded WSL/CI host (status 1) or the 60s spawnSync
|
||||
// timeout kills the child first (status null). Sibling integration
|
||||
// suites pin 60s.
|
||||
GITNEXUS_WORKER_READY_TIMEOUT_MS: process.env.GITNEXUS_WORKER_READY_TIMEOUT_MS || '60000',
|
||||
...extraEnv,
|
||||
};
|
||||
}
|
||||
|
|
@ -52,6 +60,22 @@ function runCliRaw(extraArgs: string[], cwd: string, timeoutMs = 30000) {
|
|||
});
|
||||
}
|
||||
|
||||
function isNativeAbort(result: ReturnType<typeof runCliRaw>): boolean {
|
||||
return (
|
||||
result.signal === 'SIGSEGV' ||
|
||||
result.signal === 'SIGABRT' ||
|
||||
result.signal === 'SIGBUS' ||
|
||||
result.status === 139
|
||||
);
|
||||
}
|
||||
|
||||
function isFatalAnalyzeHarness(result: ReturnType<typeof runCliRaw>): boolean {
|
||||
return (
|
||||
result.stderr?.includes('Worker script not found') === true ||
|
||||
result.stderr?.includes('deterministic crash-loop') === true
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse stdout as JSON, returning null on failure (e.g., text output).
|
||||
*/
|
||||
|
|
@ -116,14 +140,39 @@ beforeAll(() => {
|
|||
},
|
||||
});
|
||||
|
||||
// Run analyze to populate .gitnexus/ index (required for all tool commands)
|
||||
const analyzeResult = runCliRaw(['analyze', '--force'], MINI_REPO, 60000);
|
||||
if (analyzeResult.status !== 0) {
|
||||
// Index once so every --limit command has a registered repo. Match cli-e2e:
|
||||
// a tiny fixture analyzes in seconds on a quiet machine, but spawnSync
|
||||
// status null is SIGTERM from the timeout under load (not an analyze
|
||||
// exit). Retry timeouts; alreadyUpToDate makes a repeat cheap.
|
||||
//
|
||||
// CREATE_FTS_INDEX can SIGSEGV the analyze process on some WSL/native
|
||||
// hosts (status null, signal SIGSEGV) even when `doctor` reports FTS
|
||||
// LOAD-able. Do not retry that path — rebuild with --skip-fts so graph
|
||||
// tools still run. query --limit needs BM25 and is skipped in that case.
|
||||
let analyzeResult: ReturnType<typeof runCliRaw> | undefined;
|
||||
for (let attempt = 0; attempt < 3; attempt++) {
|
||||
analyzeResult = runCliRaw(['analyze', '--force'], MINI_REPO, 90_000);
|
||||
if (analyzeResult.status === 0) {
|
||||
ftsIndexed = true;
|
||||
break;
|
||||
}
|
||||
if (isFatalAnalyzeHarness(analyzeResult) || isNativeAbort(analyzeResult)) break;
|
||||
}
|
||||
if (
|
||||
analyzeResult &&
|
||||
!ftsIndexed &&
|
||||
isNativeAbort(analyzeResult) &&
|
||||
!isFatalAnalyzeHarness(analyzeResult)
|
||||
) {
|
||||
analyzeResult = runCliRaw(['analyze', '--force', '--skip-fts'], MINI_REPO, 90_000);
|
||||
}
|
||||
if (!analyzeResult || analyzeResult.status !== 0) {
|
||||
const err = analyzeResult?.error;
|
||||
throw new Error(
|
||||
`Analyze failed (status ${analyzeResult.status}):\nstdout: ${analyzeResult.stdout}\nstderr: ${analyzeResult.stderr}`,
|
||||
`Analyze failed (status ${analyzeResult?.status}, signal ${analyzeResult?.signal}, error ${err?.message ?? 'none'}):\nstdout: ${analyzeResult?.stdout}\nstderr: ${analyzeResult?.stderr}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
}, 300_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (tmpParent) cleanupTempDirSync(tmpParent);
|
||||
|
|
@ -357,6 +406,18 @@ describe('CLI --limit flag E2E', () => {
|
|||
// ─── query ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('query --limit', () => {
|
||||
beforeEach((ctx) => {
|
||||
if (ftsIndexed) return;
|
||||
if (process.env.GITNEXUS_REQUIRE_FTS === '1') {
|
||||
throw new Error(
|
||||
'GITNEXUS_REQUIRE_FTS=1 but setup analyze native-aborted during CREATE_FTS_INDEX; ' +
|
||||
'query --limit cannot be verified without BM25.',
|
||||
);
|
||||
}
|
||||
ctx.skip(
|
||||
'query --limit needs BM25; CREATE_FTS_INDEX native-aborted and analyze fell back to --skip-fts',
|
||||
);
|
||||
});
|
||||
it('truncates processes to --limit 1', () => {
|
||||
// "message" matches logMessage / createLogEntry / formatLogEntry → 4 processes
|
||||
const limited = runJson<QueryResult>([
|
||||
|
|
|
|||
|
|
@ -139,11 +139,16 @@ describe('CLI update notice subprocess behavior', () => {
|
|||
preload,
|
||||
`import fs from 'node:fs';
|
||||
Object.defineProperty(process.stderr, 'isTTY', { value: true, configurable: true });
|
||||
globalThis.fetch = async () => {
|
||||
// Mark the detached child at --import time, before tsx compiles the CLI.
|
||||
// Writing this from fetch() raced a 30s poll against cold boot + lock
|
||||
// acquisition on a loaded default-project worker (status: poll timeout).
|
||||
if (process.argv.includes('__update-check')) {
|
||||
fs.writeFileSync(${JSON.stringify(started)}, '');
|
||||
}
|
||||
globalThis.fetch = async () => {
|
||||
// Bounded so an abandoned child (test failed before releasing, temp home
|
||||
// already deleted) still exits instead of spinning forever.
|
||||
const deadline = Date.now() + 60_000;
|
||||
const deadline = Date.now() + 90_000;
|
||||
while (!fs.existsSync(${JSON.stringify(release)}) && Date.now() < deadline) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 25));
|
||||
}
|
||||
|
|
@ -176,19 +181,21 @@ globalThis.fetch = async () => {
|
|||
});
|
||||
});
|
||||
|
||||
// The parent already exited above, so reaching a still-parked child proves
|
||||
// the refresh outlived it and was never awaited.
|
||||
// The parent already exited above. `started` is written by the child's
|
||||
// --import hook, so this wait is "did the detached process actually
|
||||
// start?" not "did tsx finish compiling the CLI?" The fetch mock still
|
||||
// parks until `release` so the cache cannot appear before we unblock it.
|
||||
const cache = path.join(home, 'update-check.json');
|
||||
await expect.poll(() => fs.existsSync(started), { timeout: 30_000, interval: 50 }).toBe(true);
|
||||
expect(fs.existsSync(cache)).toBe(false);
|
||||
|
||||
fs.writeFileSync(release, '');
|
||||
await expect.poll(() => fs.existsSync(cache), { timeout: 30_000, interval: 50 }).toBe(true);
|
||||
await expect.poll(() => fs.existsSync(cache), { timeout: 60_000, interval: 50 }).toBe(true);
|
||||
expect(JSON.parse(fs.readFileSync(cache, 'utf8'))).toMatchObject({
|
||||
latestVersion: '99.0.0',
|
||||
registry: 'https://registry.npmjs.org',
|
||||
});
|
||||
}, 90_000);
|
||||
}, 120_000);
|
||||
|
||||
it('prints the localized notice on a forced-TTY stderr and keeps stdout clean', () => {
|
||||
const home = tempHome();
|
||||
|
|
|
|||
|
|
@ -133,6 +133,33 @@ describe('analyze-config (.gitnexusrc support, #243)', () => {
|
|||
expect(() => loadAnalyzeConfig(dir)).toThrow(/Unknown key "defalutBranch"/);
|
||||
});
|
||||
|
||||
it('parses process-detection budget keys as numeric strings (#3313)', async () => {
|
||||
await writeRc(
|
||||
JSON.stringify({
|
||||
maxProcesses: 40,
|
||||
maxProcessBranching: '2',
|
||||
maxProcessTraceDepth: 8,
|
||||
maxEntryPointCandidates: 400,
|
||||
}),
|
||||
);
|
||||
expect(loadAnalyzeConfig(dir)).toEqual({
|
||||
maxProcesses: '40',
|
||||
maxProcessBranching: '2',
|
||||
maxProcessTraceDepth: '8',
|
||||
maxEntryPointCandidates: '400',
|
||||
});
|
||||
});
|
||||
|
||||
it('lets a nested analyze block override flat process-detection keys (#3313)', async () => {
|
||||
await writeRc(
|
||||
JSON.stringify({
|
||||
maxProcesses: 80,
|
||||
analyze: { maxProcesses: 25 },
|
||||
}),
|
||||
);
|
||||
expect(loadAnalyzeConfig(dir)).toEqual({ maxProcesses: '25' });
|
||||
});
|
||||
|
||||
it('accepts embeddingBaseUrl / embeddingModel but rejects embeddingDims (CLI/env-only)', async () => {
|
||||
// URL + MODEL are read lazily at runtime, so they are valid config keys.
|
||||
await writeRc(JSON.stringify({ embeddingBaseUrl: 'http://h/v1', embeddingModel: 'm' }));
|
||||
|
|
|
|||
|
|
@ -237,4 +237,18 @@ describe('analyzeCommand .gitnexusrc wiring (#243)', () => {
|
|||
expect(refreshBaseRefLineMock).toHaveBeenCalledTimes(1);
|
||||
expect(refreshBaseRefLineMock).toHaveBeenCalledWith(dir, 'develop', expect.any(Object));
|
||||
});
|
||||
|
||||
it('threads .gitnexusrc process-detection knobs and lets CLI win (AE3)', async () => {
|
||||
await writeRc({ maxProcesses: '40', maxEntryPointCandidates: 300 });
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
|
||||
await analyzeCommand(dir, { maxProcesses: '25' });
|
||||
|
||||
expect(runFullAnalysisMock.mock.calls[0][1]).toEqual(
|
||||
expect.objectContaining({
|
||||
maxProcesses: 25,
|
||||
maxEntryPointCandidates: 300,
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
133
gitnexus/test/unit/analyze-process-budget.test.ts
Normal file
133
gitnexus/test/unit/analyze-process-budget.test.ts
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const runFullAnalysisMock = vi.fn();
|
||||
|
||||
vi.mock('../../src/core/run-analyze.js', () => ({
|
||||
runFullAnalysis: runFullAnalysisMock,
|
||||
}));
|
||||
|
||||
vi.mock('../../src/core/lbug/lbug-adapter.js', () => ({
|
||||
closeLbug: vi.fn(async () => undefined),
|
||||
closeLbugBeforeExit: vi.fn(async () => undefined),
|
||||
isLbugReady: vi.fn(() => false),
|
||||
}));
|
||||
|
||||
vi.mock('../../src/storage/repo-manager.js', () => ({
|
||||
getStoragePaths: vi.fn(() => ({ storagePath: '.gitnexus', lbugPath: '.gitnexus/lbug' })),
|
||||
getGlobalRegistryPath: vi.fn(() => 'registry.json'),
|
||||
RegistryNameCollisionError: class RegistryNameCollisionError extends Error {},
|
||||
AnalysisNotFinalizedError: class AnalysisNotFinalizedError extends Error {},
|
||||
assertAnalysisFinalized: vi.fn(async () => undefined),
|
||||
}));
|
||||
|
||||
vi.mock('../../src/storage/git.js', () => ({
|
||||
getGitRoot: vi.fn(() => '/repo'),
|
||||
hasGitDir: vi.fn(() => true),
|
||||
}));
|
||||
|
||||
vi.mock('../../src/core/ingestion/utils/max-file-size.js', () => ({
|
||||
getMaxFileSizeBannerMessage: vi.fn(() => null),
|
||||
}));
|
||||
|
||||
describe('analyzeCommand process-detection budget (#3313)', () => {
|
||||
const ORIGINAL_NODE_OPTIONS = process.env.NODE_OPTIONS;
|
||||
|
||||
beforeEach(() => {
|
||||
vi.resetModules();
|
||||
runFullAnalysisMock.mockReset();
|
||||
process.exitCode = undefined;
|
||||
process.env.NODE_OPTIONS = `${process.env.NODE_OPTIONS ?? ''} --max-old-space-size=8192`.trim();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (ORIGINAL_NODE_OPTIONS === undefined) {
|
||||
delete process.env.NODE_OPTIONS;
|
||||
} else {
|
||||
process.env.NODE_OPTIONS = ORIGINAL_NODE_OPTIONS;
|
||||
}
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
|
||||
const upToDate = {
|
||||
repoName: 'repo',
|
||||
repoPath: '/repo',
|
||||
stats: {},
|
||||
alreadyUpToDate: true,
|
||||
};
|
||||
|
||||
it('threads the four CLI flags through runFullAnalysis without env mutation', async () => {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
runFullAnalysisMock.mockResolvedValue(upToDate);
|
||||
const before = {
|
||||
processes: process.env.GITNEXUS_MAX_PROCESSES,
|
||||
branching: process.env.GITNEXUS_MAX_PROCESS_BRANCHING,
|
||||
depth: process.env.GITNEXUS_MAX_PROCESS_TRACE_DEPTH,
|
||||
entries: process.env.GITNEXUS_MAX_ENTRY_POINT_CANDIDATES,
|
||||
};
|
||||
|
||||
await analyzeCommand(undefined, {
|
||||
maxProcesses: '25',
|
||||
maxProcessBranching: '2',
|
||||
maxProcessTraceDepth: '8',
|
||||
maxEntryPointCandidates: '400',
|
||||
});
|
||||
|
||||
expect(runFullAnalysisMock).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
expect.objectContaining({
|
||||
maxProcesses: 25,
|
||||
maxProcessBranching: 2,
|
||||
maxProcessTraceDepth: 8,
|
||||
maxEntryPointCandidates: 400,
|
||||
}),
|
||||
expect.any(Object),
|
||||
);
|
||||
expect(process.env.GITNEXUS_MAX_PROCESSES).toBe(before.processes);
|
||||
expect(process.env.GITNEXUS_MAX_PROCESS_BRANCHING).toBe(before.branching);
|
||||
expect(process.env.GITNEXUS_MAX_PROCESS_TRACE_DEPTH).toBe(before.depth);
|
||||
expect(process.env.GITNEXUS_MAX_ENTRY_POINT_CANDIDATES).toBe(before.entries);
|
||||
});
|
||||
|
||||
it.each(['0', 'abc', '-4'])(
|
||||
'warns and continues when --max-processes is %s (AE5)',
|
||||
async (value) => {
|
||||
const { _captureLogger } = await import('../../src/core/logger.js');
|
||||
const cap = _captureLogger();
|
||||
try {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
runFullAnalysisMock.mockResolvedValue(upToDate);
|
||||
|
||||
await analyzeCommand(undefined, { maxProcesses: value });
|
||||
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
expect(runFullAnalysisMock).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
expect.not.objectContaining({ maxProcesses: expect.any(Number) }),
|
||||
expect.any(Object),
|
||||
);
|
||||
expect(
|
||||
cap.records().some((r) => {
|
||||
const msg = String(r.msg ?? '');
|
||||
return (
|
||||
msg.includes('--max-processes must be a positive integer') &&
|
||||
msg.includes('next source (env, then the built-in default)')
|
||||
);
|
||||
}),
|
||||
).toBe(true);
|
||||
} finally {
|
||||
cap.restore();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('leaves option fields unset so runFullAnalysis can honor env-only overrides', async () => {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
runFullAnalysisMock.mockResolvedValue(upToDate);
|
||||
vi.stubEnv('GITNEXUS_MAX_PROCESSES', '80');
|
||||
|
||||
await analyzeCommand(undefined, {});
|
||||
|
||||
const opts = runFullAnalysisMock.mock.calls[0][1] as { maxProcesses?: number };
|
||||
expect(opts.maxProcesses).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
|
@ -196,10 +196,13 @@ describe('CLI help surface', () => {
|
|||
expect(result.stdout).toContain('外部索引根目录');
|
||||
expect(result.stdout).toContain('GITNEXUS_CONTENT_RETENTION=full');
|
||||
expect(result.stdout).toContain('源码文本保留策略');
|
||||
expect(result.stdout).toContain('当参数和对应环境变量同时提供时，参数优先。');
|
||||
expect(result.stdout).toContain(
|
||||
'CLI 参数优先于 `.gitnexusrc`，后者优先于环境变量，环境变量优先于内置默认值。',
|
||||
);
|
||||
expect(result.stdout).toContain('提示：`.gitnexusignore` 支持 `.gitignore` 风格的取反。');
|
||||
expect(result.stdout).not.toContain('Environment variables:');
|
||||
expect(result.stdout).not.toContain('Flags override the corresponding env vars');
|
||||
expect(result.stdout).not.toContain('当参数和对应环境变量同时提供时，参数优先。');
|
||||
});
|
||||
|
||||
it('analyze help documents the external storage root layout', () => {
|
||||
|
|
@ -213,6 +216,10 @@ describe('CLI help surface', () => {
|
|||
expect(result.stdout).toContain('GITNEXUS_CONTENT_RETENTION=full');
|
||||
expect(result.stdout).toContain('Source-text retention profile');
|
||||
expect(result.stdout).toContain('<repo-basename>-<canonical-path-hash>/');
|
||||
expect(result.stdout).toContain(
|
||||
'CLI flags take precedence over `.gitnexusrc`, which takes precedence over env vars, which take precedence over built-in defaults.',
|
||||
);
|
||||
expect(result.stdout).not.toContain('Flags override the corresponding env vars');
|
||||
});
|
||||
|
||||
it('query help keeps advanced search options without importing analyze deps', () => {
|
||||
|
|
|
|||
235
gitnexus/test/unit/process-detection-budget.test.ts
Normal file
235
gitnexus/test/unit/process-detection-budget.test.ts
Normal file
|
|
@ -0,0 +1,235 @@
|
|||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
buildProcessDetectionPhaseConfig,
|
||||
formatInvalidProcessDetectionOverride,
|
||||
formatProcessDetectionBudgetBanner,
|
||||
formatWholeFlowsMissingRemedies,
|
||||
parsePositiveIntegerOverride,
|
||||
parseProcessDetectionBudgetStrings,
|
||||
processDetectionBudgetMismatch,
|
||||
processDetectionEffectiveLimits,
|
||||
resolveProcessDetectionBudget,
|
||||
toProcessDetectionStamp,
|
||||
uncertifyProcessDetectionStamp,
|
||||
} from '../../src/core/ingestion/process-detection-budget.js';
|
||||
|
||||
describe('parsePositiveIntegerOverride', () => {
|
||||
it('accepts positive integers and rejects 0 / non-integers', () => {
|
||||
const invalid: string[] = [];
|
||||
expect(parsePositiveIntegerOverride('25')).toBe(25);
|
||||
expect(parsePositiveIntegerOverride(40)).toBe(40);
|
||||
expect(parsePositiveIntegerOverride('0', (raw) => invalid.push(raw))).toBeUndefined();
|
||||
expect(parsePositiveIntegerOverride('abc', (raw) => invalid.push(raw))).toBeUndefined();
|
||||
expect(parsePositiveIntegerOverride('-3', (raw) => invalid.push(raw))).toBeUndefined();
|
||||
expect(parsePositiveIntegerOverride('', (raw) => invalid.push(raw))).toBeUndefined();
|
||||
expect(invalid).toEqual(['0', 'abc', '-3', '']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveProcessDetectionBudget (#3313)', () => {
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
|
||||
it('keeps shipped defaults when nothing is set', () => {
|
||||
const resolved = resolveProcessDetectionBudget({}, {});
|
||||
expect(resolved.maxProcesses).toBeUndefined();
|
||||
expect(resolved.maxProcessBranching).toBe(4);
|
||||
expect(resolved.maxProcessTraceDepth).toBe(10);
|
||||
expect(resolved.maxEntryPointCandidates).toBe(200);
|
||||
expect(resolved.overridden).toEqual({
|
||||
maxProcesses: false,
|
||||
maxProcessBranching: false,
|
||||
maxProcessTraceDepth: false,
|
||||
maxEntryPointCandidates: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('lets an invalid option fall through to env instead of claiming a hard default', () => {
|
||||
const invalid: Array<[string, string]> = [];
|
||||
const resolved = resolveProcessDetectionBudget(
|
||||
{ maxProcesses: 0 },
|
||||
{ GITNEXUS_MAX_PROCESSES: '80' },
|
||||
(knob, raw) => invalid.push([knob, raw]),
|
||||
);
|
||||
expect(resolved.maxProcesses).toBe(80);
|
||||
expect(invalid).toEqual([['--max-processes', '0']]);
|
||||
expect(formatInvalidProcessDetectionOverride('--max-processes', '0')).toContain(
|
||||
'next source (env, then the built-in default)',
|
||||
);
|
||||
expect(formatInvalidProcessDetectionOverride('GITNEXUS_MAX_PROCESSES', '0')).toContain(
|
||||
'the built-in default',
|
||||
);
|
||||
expect(formatInvalidProcessDetectionOverride('GITNEXUS_MAX_PROCESSES', '0')).not.toContain(
|
||||
'next source (env, then the built-in default)',
|
||||
);
|
||||
});
|
||||
|
||||
it('lets explicit options beat env (AE3 remainder)', () => {
|
||||
const resolved = resolveProcessDetectionBudget(
|
||||
{ maxProcesses: 25 },
|
||||
{ GITNEXUS_MAX_PROCESSES: '80' },
|
||||
);
|
||||
expect(resolved.maxProcesses).toBe(25);
|
||||
expect(resolved.overridden.maxProcesses).toBe(true);
|
||||
});
|
||||
|
||||
it('reads env when option fields are unset', () => {
|
||||
const resolved = resolveProcessDetectionBudget(
|
||||
{},
|
||||
{
|
||||
GITNEXUS_MAX_PROCESSES: '40',
|
||||
GITNEXUS_MAX_PROCESS_BRANCHING: '2',
|
||||
GITNEXUS_MAX_PROCESS_TRACE_DEPTH: '8',
|
||||
GITNEXUS_MAX_ENTRY_POINT_CANDIDATES: '400',
|
||||
},
|
||||
);
|
||||
expect(resolved.maxProcesses).toBe(40);
|
||||
expect(resolved.maxProcessBranching).toBe(2);
|
||||
expect(resolved.maxProcessTraceDepth).toBe(8);
|
||||
expect(resolved.maxEntryPointCandidates).toBe(400);
|
||||
});
|
||||
|
||||
it('warns and falls back on invalid env (AE5)', () => {
|
||||
const invalid: Array<[string, string]> = [];
|
||||
const zero = resolveProcessDetectionBudget({}, { GITNEXUS_MAX_PROCESSES: '0' }, (knob, raw) =>
|
||||
invalid.push([knob, raw]),
|
||||
);
|
||||
const garbage = resolveProcessDetectionBudget(
|
||||
{},
|
||||
{ GITNEXUS_MAX_PROCESSES: 'abc' },
|
||||
(knob, raw) => invalid.push([knob, raw]),
|
||||
);
|
||||
expect(zero.maxProcesses).toBeUndefined();
|
||||
expect(garbage.maxProcesses).toBeUndefined();
|
||||
expect(invalid).toEqual([
|
||||
['GITNEXUS_MAX_PROCESSES', '0'],
|
||||
['GITNEXUS_MAX_PROCESSES', 'abc'],
|
||||
]);
|
||||
});
|
||||
|
||||
it('replaces the dynamic formula only when maxProcesses is explicit', () => {
|
||||
const unset = buildProcessDetectionPhaseConfig(
|
||||
resolveProcessDetectionBudget({}, {}),
|
||||
1000,
|
||||
(n) => Math.max(20, Math.round(n / 10)),
|
||||
);
|
||||
const explicit = buildProcessDetectionPhaseConfig(
|
||||
resolveProcessDetectionBudget({ maxProcesses: 5 }, {}),
|
||||
1000,
|
||||
(n) => Math.max(20, Math.round(n / 10)),
|
||||
);
|
||||
expect(unset.maxProcesses).toBe(100);
|
||||
expect(explicit.maxProcesses).toBe(5);
|
||||
expect(explicit.minSteps).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe('processDetectionBudgetMismatch (KTD4)', () => {
|
||||
const defaults = resolveProcessDetectionBudget({}, {});
|
||||
|
||||
it('matches a missing stamp against default/dynamic knobs', () => {
|
||||
expect(processDetectionBudgetMismatch(undefined, defaults)).toBe(false);
|
||||
});
|
||||
|
||||
it('mismatches a missing stamp when any override is set', () => {
|
||||
expect(
|
||||
processDetectionBudgetMismatch(
|
||||
undefined,
|
||||
resolveProcessDetectionBudget({ maxEntryPointCandidates: 400 }, {}),
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('mismatches when a present stamp differs', () => {
|
||||
const recorded = toProcessDetectionStamp(defaults);
|
||||
expect(
|
||||
processDetectionBudgetMismatch({ ...recorded, maxEntryPointCandidates: 400 }, defaults),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('mismatches explicit maxProcesses vs later dynamic even when the integer equals the formula', () => {
|
||||
const explicit = resolveProcessDetectionBudget({ maxProcesses: 100 }, {});
|
||||
expect(processDetectionBudgetMismatch(toProcessDetectionStamp(explicit), defaults)).toBe(true);
|
||||
expect(toProcessDetectionStamp(explicit).maxProcesses).toBe(100);
|
||||
expect(toProcessDetectionStamp(defaults).maxProcesses).toBe(null);
|
||||
});
|
||||
|
||||
it('matches an identical present stamp', () => {
|
||||
const resolved = resolveProcessDetectionBudget(
|
||||
{ maxProcesses: 25, maxEntryPointCandidates: 400 },
|
||||
{},
|
||||
);
|
||||
expect(processDetectionBudgetMismatch(toProcessDetectionStamp(resolved), resolved)).toBe(false);
|
||||
});
|
||||
|
||||
it('mismatches an uncertified stamp even when the numeric fields match defaults', () => {
|
||||
const recorded = uncertifyProcessDetectionStamp(toProcessDetectionStamp(defaults));
|
||||
expect(recorded.uncertified).toBe(true);
|
||||
expect(recorded.maxProcesses).toBe(null);
|
||||
expect(processDetectionBudgetMismatch(recorded, defaults)).toBe(true);
|
||||
expect(processDetectionBudgetMismatch(toProcessDetectionStamp(defaults), defaults)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('warning copy and banner', () => {
|
||||
it('maps loud counters to the matching knobs and names the pre-entry trace gate', () => {
|
||||
const limits = processDetectionEffectiveLimits(20, resolveProcessDetectionBudget({}, {}));
|
||||
expect(limits.maxProcessTraces).toBe(40);
|
||||
expect(
|
||||
formatWholeFlowsMissingRemedies(
|
||||
{ entryPointCandidatesDropped: 210, entryPointsUnexplored: 0, processesDropped: 0 },
|
||||
limits,
|
||||
410,
|
||||
),
|
||||
).toContain('--max-entry-point-candidates');
|
||||
expect(
|
||||
formatWholeFlowsMissingRemedies(
|
||||
{ entryPointCandidatesDropped: 210, entryPointsUnexplored: 0, processesDropped: 0 },
|
||||
limits,
|
||||
410,
|
||||
),
|
||||
).not.toContain('--max-process-branching');
|
||||
expect(
|
||||
formatWholeFlowsMissingRemedies(
|
||||
{ entryPointCandidatesDropped: 0, entryPointsUnexplored: 3, processesDropped: 1 },
|
||||
limits,
|
||||
10,
|
||||
),
|
||||
).toContain('--max-processes');
|
||||
expect(
|
||||
formatWholeFlowsMissingRemedies(
|
||||
{ entryPointCandidatesDropped: 0, entryPointsUnexplored: 3, processesDropped: 1 },
|
||||
limits,
|
||||
10,
|
||||
),
|
||||
).toContain('40');
|
||||
expect(
|
||||
formatWholeFlowsMissingRemedies(
|
||||
{ entryPointCandidatesDropped: 0, entryPointsUnexplored: 3, processesDropped: 1 },
|
||||
limits,
|
||||
10,
|
||||
),
|
||||
).toContain('next entry is skipped');
|
||||
});
|
||||
|
||||
it('prints a banner only when an override is active', () => {
|
||||
expect(formatProcessDetectionBudgetBanner(resolveProcessDetectionBudget({}, {}))).toBeNull();
|
||||
expect(
|
||||
formatProcessDetectionBudgetBanner(resolveProcessDetectionBudget({ maxProcesses: 25 }, {})),
|
||||
).toContain('maxProcesses=25');
|
||||
expect(
|
||||
formatProcessDetectionBudgetBanner(
|
||||
resolveProcessDetectionBudget({ maxProcessBranching: 6 }, {}),
|
||||
),
|
||||
).toContain('maxProcesses=dynamic (max(20, round(symbols/10)))');
|
||||
});
|
||||
|
||||
it('parses CLI/rc numeric strings without treating 0 as unlimited', () => {
|
||||
const parsed = parseProcessDetectionBudgetStrings({
|
||||
maxProcesses: '25',
|
||||
maxProcessBranching: '0',
|
||||
});
|
||||
expect(parsed).toEqual({ maxProcesses: 25 });
|
||||
});
|
||||
});
|
||||
|
|
@ -585,7 +585,13 @@ describe('process depth (D1/D2)', () => {
|
|||
// `findEntryPoints` returns several starting points, so the deep chain is
|
||||
// traced from inside it whatever the traversal order does — a test there
|
||||
// passes under BOTH traversals and guards nothing.
|
||||
const cfg = { maxTraceDepth: 10, maxBranching: 4, maxProcesses: 75, minSteps: 3 };
|
||||
const cfg = {
|
||||
maxTraceDepth: 10,
|
||||
maxBranching: 4,
|
||||
maxProcesses: 75,
|
||||
minSteps: 3,
|
||||
maxEntryPointCandidates: 200,
|
||||
};
|
||||
|
||||
const deepAndShallow = (order: readonly string[]): Map<string, string[]> => {
|
||||
// Fan-out is capped at maxBranching (4), so the budget is exhausted BELOW
|
||||
|
|
@ -1214,6 +1220,26 @@ describe('the entry-point candidate cap is disclosed too', () => {
|
|||
expect(result.stats.truncation.entryPointCandidatesDropped).toBe(0);
|
||||
expect(result.stats.truncation.truncated).toBe(false);
|
||||
});
|
||||
|
||||
it('honors maxEntryPointCandidates instead of the compiled 200 (#3313)', async () => {
|
||||
const result = await processProcesses(manyCandidates(), [], undefined, {
|
||||
maxProcesses: 1000,
|
||||
maxEntryPointCandidates: 410,
|
||||
});
|
||||
|
||||
expect(result.stats.entryPointsFound).toBe(410);
|
||||
expect(result.stats.truncation.entryPointCandidatesDropped).toBe(0);
|
||||
});
|
||||
|
||||
it('drops one candidate when the override is one below the list length (#3313)', async () => {
|
||||
const result = await processProcesses(manyCandidates(), [], undefined, {
|
||||
maxProcesses: 1000,
|
||||
maxEntryPointCandidates: 409,
|
||||
});
|
||||
|
||||
expect(result.stats.entryPointsFound).toBe(409);
|
||||
expect(result.stats.truncation.entryPointCandidatesDropped).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// The three trace sorts in this file each joined the path inside the COMPARATOR
|
||||
|
|
|
|||
|
|
@ -24,11 +24,12 @@ import type {
|
|||
PhaseResult,
|
||||
PipelineContext,
|
||||
} from '../../src/core/ingestion/pipeline-phases/types.js';
|
||||
import type { PipelineOptions } from '../../src/core/ingestion/pipeline.js';
|
||||
import type { KnowledgeGraph } from '../../src/core/graph/types.js';
|
||||
import type { GraphNode, NodeLabel } from 'gitnexus-shared';
|
||||
|
||||
function makeCtx(graph: KnowledgeGraph): PipelineContext {
|
||||
return { repoPath: '/tmp/repo', graph, onProgress: () => {}, pipelineStart: 0 };
|
||||
function makeCtx(graph: KnowledgeGraph, options?: PipelineOptions): PipelineContext {
|
||||
return { repoPath: '/tmp/repo', graph, onProgress: () => {}, pipelineStart: 0, options };
|
||||
}
|
||||
|
||||
function phaseResult<T>(phaseName: string, output: T): PhaseResult<T> {
|
||||
|
|
@ -219,12 +220,16 @@ describe('processes phase — truncation is disclosed proportionately (#2899)',
|
|||
|
||||
const runCaptured = async (
|
||||
graph: KnowledgeGraph,
|
||||
options?: PipelineOptions,
|
||||
): Promise<{ output: ProcessesOutput; records: ReturnType<LoggerCapture['records']> }> => {
|
||||
// Captured at `debug` so an ABSENT warn can be distinguished from a silent
|
||||
// phase: the debug line has to be there instead.
|
||||
const capture = _captureLogger('debug');
|
||||
try {
|
||||
const output = (await processesPhase.execute(makeCtx(graph), baseDeps())) as ProcessesOutput;
|
||||
const output = (await processesPhase.execute(
|
||||
makeCtx(graph, options),
|
||||
baseDeps(),
|
||||
)) as ProcessesOutput;
|
||||
return { output, records: capture.records() };
|
||||
} finally {
|
||||
capture.restore();
|
||||
|
|
@ -297,5 +302,77 @@ describe('processes phase — truncation is disclosed proportionately (#2899)',
|
|||
const lines = records.filter((r) => PROCESS_LINES.test(String(r.msg)));
|
||||
expect(lines.map((r) => r.level)).toEqual([40]);
|
||||
expect(String(lines[0]?.msg)).toContain('210 of 410 candidate entry point(s) never ranked in');
|
||||
expect(String(lines[0]?.msg)).toContain('--max-entry-point-candidates');
|
||||
expect(String(lines[0]?.msg)).not.toContain('--max-process-branching');
|
||||
expect(lines[0]?.effectiveLimits).toEqual(
|
||||
expect.objectContaining({
|
||||
maxEntryPointCandidates: 200,
|
||||
maxProcessTraces: expect.any(Number),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('honors an explicit maxProcesses instead of the dynamic formula (#3313)', async () => {
|
||||
const { output } = await runCaptured(flowsMissing(), { maxProcesses: 40 });
|
||||
expect(output.processResult.stats.truncation.processesDropped).toBe(0);
|
||||
expect(output.processResult.stats.truncation.entryPointsUnexplored).toBe(0);
|
||||
});
|
||||
|
||||
it('clears the entry-point drop when the candidate cap is raised (#3313 AE2)', async () => {
|
||||
const graph = createKnowledgeGraph();
|
||||
for (let c = 0; c < 205; c++) {
|
||||
for (let i = 0; i < 3; i++) addFn(graph, `func:r${c}_${i}`);
|
||||
for (let i = 0; i < 2; i++) addCallEdge(graph, `func:r${c}_${i}`, `func:r${c}_${i + 1}`);
|
||||
}
|
||||
const { output } = await runCaptured(graph, { maxEntryPointCandidates: 410 });
|
||||
expect(output.processResult.stats.truncation.entryPointCandidatesDropped).toBe(0);
|
||||
});
|
||||
|
||||
it('still warns when only maxProcesses is raised on a 410-candidate fixture (#3313 AE2)', async () => {
|
||||
const graph = createKnowledgeGraph();
|
||||
for (let c = 0; c < 205; c++) {
|
||||
for (let i = 0; i < 3; i++) addFn(graph, `func:s${c}_${i}`);
|
||||
for (let i = 0; i < 2; i++) addCallEdge(graph, `func:s${c}_${i}`, `func:s${c}_${i + 1}`);
|
||||
}
|
||||
const { output, records } = await runCaptured(graph, { maxProcesses: 1000 });
|
||||
expect(output.processResult.stats.truncation.entryPointCandidatesDropped).toBe(210);
|
||||
const lines = records.filter((r) => PROCESS_LINES.test(String(r.msg)));
|
||||
expect(lines.map((r) => r.level)).toEqual([40]);
|
||||
});
|
||||
|
||||
it('increases process count when both binders are raised (#3313 AE6)', async () => {
|
||||
const graph = createKnowledgeGraph();
|
||||
for (let c = 0; c < 210; c++) {
|
||||
for (let i = 0; i < 3; i++) addFn(graph, `func:ae6_${c}_${i}`);
|
||||
for (let i = 0; i < 2; i++)
|
||||
addCallEdge(graph, `func:ae6_${c}_${i}`, `func:ae6_${c}_${i + 1}`);
|
||||
}
|
||||
// 210 chains × 3 = 630 symbols. Dynamic maxProcesses needs ≥ 210, so pad.
|
||||
for (let i = 0; i < 1470; i++) addFn(graph, `func:pad_${i}`);
|
||||
|
||||
const baseline = await runCaptured(graph);
|
||||
const raised = await runCaptured(graph, {
|
||||
maxEntryPointCandidates: 420,
|
||||
maxProcesses: 300,
|
||||
});
|
||||
expect(
|
||||
baseline.output.processResult.stats.truncation.entryPointCandidatesDropped,
|
||||
).toBeGreaterThan(0);
|
||||
expect(raised.output.processResult.stats.truncation.entryPointCandidatesDropped).toBe(0);
|
||||
expect(raised.output.processResult.processes.length).toBeGreaterThan(
|
||||
baseline.output.processResult.processes.length,
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps shape-only caps at debug when branching or depth is raised (#3313 AE7)', async () => {
|
||||
const { output, records } = await runCaptured(shortenedOnly(), {
|
||||
maxProcessBranching: 8,
|
||||
maxProcessTraceDepth: 20,
|
||||
});
|
||||
expect(output.processResult.stats.truncation.calleesDropped).toBe(0);
|
||||
expect(output.processResult.stats.truncation.tracesDepthCapped).toBe(0);
|
||||
const lines = records.filter((r) => PROCESS_LINES.test(String(r.msg)));
|
||||
expect(lines.some((r) => r.level === 40)).toBe(false);
|
||||
expect(lines.some((r) => String(r.msg).includes('whole flows are MISSING'))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ import { readFileSync } from 'node:fs';
|
|||
import fs from 'fs/promises';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
getStoragePaths,
|
||||
loadMeta,
|
||||
|
|
@ -17,11 +17,16 @@ import {
|
|||
type RepoMeta,
|
||||
} from '../../src/storage/repo-manager.js';
|
||||
import { createTempDir } from '../helpers/test-db.js';
|
||||
import { computeFileHash } from '../../src/storage/file-hash.js';
|
||||
import { ANALYSIS_FEATURES } from '../../src/core/analysis-feature-registry.js';
|
||||
import { resolveAnalysisFeatureVersions } from '../../src/core/analysis-features.js';
|
||||
import { createKnowledgeGraph } from '../../src/core/graph/graph.js';
|
||||
import { resolveAnalyzerRunnerIdentity } from '../../src/core/analyzer-identity.js';
|
||||
import { EMBEDDING_DIMS, SCHEMA_FINGERPRINT } from '../../src/core/lbug/schema.js';
|
||||
import {
|
||||
PROCESS_DETECTION_BUDGET_DEFAULTS,
|
||||
PROCESS_DETECTION_ENV,
|
||||
} from '../../src/core/ingestion/process-detection-budget.js';
|
||||
import { getSearchFTSCjkSegmentation } from '../../src/core/search/cjk-segmentation.js';
|
||||
import {
|
||||
FTS_DIRTY_PHASE,
|
||||
|
|
@ -301,6 +306,11 @@ describe('FTS crash-marker policy (characterization)', () => {
|
|||
});
|
||||
|
||||
describe('runFullAnalysis FTS crash marker', () => {
|
||||
beforeEach(() => {
|
||||
for (const key of Object.values(PROCESS_DETECTION_ENV)) {
|
||||
vi.stubEnv(key, undefined);
|
||||
}
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.doUnmock('../../src/core/lbug/lbug-adapter.js');
|
||||
vi.doUnmock('../../src/core/search/fts-indexes.js');
|
||||
|
|
@ -376,6 +386,7 @@ describe('runFullAnalysis FTS crash marker', () => {
|
|||
writePlan: 'in-place',
|
||||
checkpointSucceeded: true,
|
||||
});
|
||||
expect(midBuild?.processDetection?.uncertified).toBeUndefined();
|
||||
expect(sequence.indexOf('checkpoint')).toBeLessThan(sequence.indexOf('stamp-fts'));
|
||||
expect(sequence.indexOf('stamp-fts')).toBeLessThan(sequence.indexOf('build'));
|
||||
expect(checkpointOnce).toHaveBeenCalled();
|
||||
|
|
@ -387,6 +398,73 @@ describe('runFullAnalysis FTS crash marker', () => {
|
|||
}
|
||||
});
|
||||
|
||||
it('stamps processDetection.uncertified before in-place FTS when the budget mismatched', async () => {
|
||||
const sequence: string[] = [];
|
||||
let midBuild: RepoMeta | null = null;
|
||||
vi.doMock('../../src/core/lbug/wal-checkpoint-driver.js', async (importActual) => ({
|
||||
...(await importActual<typeof import('../../src/core/lbug/wal-checkpoint-driver.js')>()),
|
||||
checkpointOnce: vi.fn(async () => true),
|
||||
}));
|
||||
vi.doMock('../../src/core/lbug/lbug-adapter.js', mockLbugAdapter);
|
||||
vi.doMock('../../src/core/search/fts-indexes.js', async (importActual) => ({
|
||||
...(await importActual<typeof import('../../src/core/search/fts-indexes.js')>()),
|
||||
initialiseSearchFTSStemmer: vi.fn(() => 'porter'),
|
||||
missingSearchFTSIndexTables: vi.fn(async () => []),
|
||||
dropSearchFTSIndexes: vi.fn(async () => undefined),
|
||||
buildSearchIndexesOrDegrade: vi.fn(async () => {
|
||||
sequence.push('build');
|
||||
return { ok: true };
|
||||
}),
|
||||
}));
|
||||
vi.doMock('../../src/core/ingestion/pipeline.js', () => ({
|
||||
runPipelineFromRepo: vi.fn(async (repoPath: string) => ({
|
||||
repoPath,
|
||||
graph: fileGraph(),
|
||||
})),
|
||||
}));
|
||||
vi.doMock('../../src/storage/repo-manager.js', async (importActual) => {
|
||||
const actual = await importActual<typeof import('../../src/storage/repo-manager.js')>();
|
||||
return {
|
||||
...actual,
|
||||
saveMeta: async (...args: Parameters<typeof actual.saveMeta>) => {
|
||||
if (args[1].incrementalInProgress?.phase === FTS_DIRTY_PHASE) {
|
||||
sequence.push('stamp-fts');
|
||||
midBuild = args[1];
|
||||
}
|
||||
return actual.saveMeta(...args);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
const tmpRepo = await createTempDir('gitnexus-fts-crash-uncertify-');
|
||||
try {
|
||||
await seedGitFile(tmpRepo.dbPath);
|
||||
const { storagePath } = getStoragePaths(tmpRepo.dbPath);
|
||||
await fs.mkdir(storagePath, { recursive: true });
|
||||
await saveMeta(storagePath, incrementalMeta(tmpRepo.dbPath));
|
||||
|
||||
const { runFullAnalysis } = await import('../../src/core/run-analyze.js');
|
||||
await runFullAnalysis(
|
||||
tmpRepo.dbPath,
|
||||
{ maxProcesses: 25, skipAgentsMd: true, skipSkills: true },
|
||||
{ onProgress: () => {}, onLog: () => {} },
|
||||
);
|
||||
|
||||
expect(midBuild?.processDetection).toMatchObject({
|
||||
uncertified: true,
|
||||
maxProcesses: null,
|
||||
});
|
||||
expect(sequence.indexOf('stamp-fts')).toBeGreaterThan(-1);
|
||||
expect(sequence.indexOf('build')).toBeGreaterThan(-1);
|
||||
expect(sequence.indexOf('stamp-fts')).toBeLessThan(sequence.indexOf('build'));
|
||||
const finalMeta = await loadMeta(storagePath);
|
||||
expect(finalMeta?.processDetection?.uncertified).toBeUndefined();
|
||||
expect(finalMeta?.processDetection?.maxProcesses).toBe(25);
|
||||
} finally {
|
||||
await tmpRepo.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === 'win32')(
|
||||
'does not stamp an FTS phase on a staging plan',
|
||||
async () => {
|
||||
|
|
@ -440,6 +518,123 @@ describe('runFullAnalysis FTS crash marker', () => {
|
|||
},
|
||||
);
|
||||
|
||||
it.skipIf(process.platform === 'win32')(
|
||||
'does not persist live incrementalInProgress when atomic incremental dies during staging copy',
|
||||
async () => {
|
||||
vi.doMock('../../src/core/lbug/lbug-adapter.js', mockLbugAdapter);
|
||||
vi.doMock('../../src/core/search/fts-indexes.js', async (importActual) => ({
|
||||
...(await importActual<typeof import('../../src/core/search/fts-indexes.js')>()),
|
||||
initialiseSearchFTSStemmer: vi.fn(() => 'porter'),
|
||||
missingSearchFTSIndexTables: vi.fn(async () => []),
|
||||
dropSearchFTSIndexes: vi.fn(async () => undefined),
|
||||
buildSearchIndexesOrDegrade: vi.fn(async () => ({ ok: true })),
|
||||
}));
|
||||
vi.doMock('../../src/core/ingestion/pipeline.js', () => ({
|
||||
runPipelineFromRepo: vi.fn(async (repoPath: string) => ({
|
||||
repoPath,
|
||||
graph: fileGraph(),
|
||||
})),
|
||||
}));
|
||||
|
||||
const tmpRepo = await createTempDir('gitnexus-atomic-incr-copy-crash-');
|
||||
try {
|
||||
await seedGitFile(tmpRepo.dbPath);
|
||||
const { storagePath, lbugPath } = getStoragePaths(tmpRepo.dbPath);
|
||||
await fs.mkdir(storagePath, { recursive: true });
|
||||
const fileHash = await computeFileHash(path.join(tmpRepo.dbPath, REL_FILE));
|
||||
await saveMeta(storagePath, {
|
||||
...incrementalMeta(tmpRepo.dbPath),
|
||||
lastCommit: headCommit(tmpRepo.dbPath),
|
||||
fileHashes: { [REL_FILE]: fileHash! },
|
||||
processDetection: {
|
||||
maxProcesses: 80,
|
||||
maxProcessBranching: 4,
|
||||
maxProcessTraceDepth: 10,
|
||||
maxEntryPointCandidates: 200,
|
||||
},
|
||||
});
|
||||
await createPlaceholderGraphStore(lbugPath);
|
||||
|
||||
const originalCopyFile: typeof fs.copyFile = fs.copyFile.bind(fs);
|
||||
const copyFile = vi.spyOn(fs, 'copyFile').mockImplementation(async (src, dest, mode) => {
|
||||
if (String(dest).includes('.staging.')) {
|
||||
throw new Error('simulated staging copy crash');
|
||||
}
|
||||
return originalCopyFile(src, dest, mode);
|
||||
});
|
||||
|
||||
const { runFullAnalysis } = await import('../../src/core/run-analyze.js');
|
||||
await expect(
|
||||
runFullAnalysis(
|
||||
tmpRepo.dbPath,
|
||||
{ atomicIncremental: true, maxProcesses: 25, skipAgentsMd: true, skipSkills: true },
|
||||
{ onProgress: () => {}, onLog: () => {} },
|
||||
),
|
||||
).rejects.toThrow('simulated staging copy crash');
|
||||
expect(copyFile).toHaveBeenCalled();
|
||||
|
||||
const liveMeta = await loadMeta(storagePath);
|
||||
expect(liveMeta?.incrementalInProgress).toBeUndefined();
|
||||
expect(liveMeta?.processDetection?.maxProcesses).toBe(80);
|
||||
expect(liveMeta?.processDetection?.uncertified).toBeUndefined();
|
||||
} finally {
|
||||
await tmpRepo.cleanup();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('stamps phase pre-write on live meta before in-place incremental writeback', async () => {
|
||||
vi.doMock('../../src/core/lbug/lbug-adapter.js', mockLbugAdapter);
|
||||
vi.doMock('../../src/core/search/fts-indexes.js', async (importActual) => ({
|
||||
...(await importActual<typeof import('../../src/core/search/fts-indexes.js')>()),
|
||||
initialiseSearchFTSStemmer: vi.fn(() => 'porter'),
|
||||
missingSearchFTSIndexTables: vi.fn(async () => []),
|
||||
dropSearchFTSIndexes: vi.fn(async () => undefined),
|
||||
buildSearchIndexesOrDegrade: vi.fn(async () => ({ ok: true })),
|
||||
}));
|
||||
vi.doMock('../../src/core/ingestion/pipeline.js', () => ({
|
||||
runPipelineFromRepo: vi.fn(async (repoPath: string) => ({
|
||||
repoPath,
|
||||
graph: fileGraph(),
|
||||
})),
|
||||
}));
|
||||
vi.doMock('../../src/storage/repo-manager.js', async (importActual) => {
|
||||
const actual = await importActual<typeof import('../../src/storage/repo-manager.js')>();
|
||||
return {
|
||||
...actual,
|
||||
saveMeta: async (...args: Parameters<typeof actual.saveMeta>) => {
|
||||
const result = await actual.saveMeta(...args);
|
||||
if (args[1].incrementalInProgress?.phase === 'pre-write') {
|
||||
throw new Error('stop after in-place dirty stamp');
|
||||
}
|
||||
return result;
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
const tmpRepo = await createTempDir('gitnexus-inplace-pre-write-stamp-');
|
||||
try {
|
||||
await seedGitFile(tmpRepo.dbPath);
|
||||
const { storagePath } = getStoragePaths(tmpRepo.dbPath);
|
||||
await fs.mkdir(storagePath, { recursive: true });
|
||||
await saveMeta(storagePath, incrementalMeta(tmpRepo.dbPath));
|
||||
|
||||
const { runFullAnalysis } = await import('../../src/core/run-analyze.js');
|
||||
await expect(
|
||||
runFullAnalysis(
|
||||
tmpRepo.dbPath,
|
||||
{ skipAgentsMd: true, skipSkills: true },
|
||||
{ onProgress: () => {}, onLog: () => {} },
|
||||
),
|
||||
).rejects.toThrow('stop after in-place dirty stamp');
|
||||
|
||||
const liveMeta = await loadMeta(storagePath);
|
||||
expect(liveMeta?.incrementalInProgress).toMatchObject({ phase: 'pre-write' });
|
||||
} finally {
|
||||
await tmpRepo.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
it('clears the FTS phase on the degrade path as well as on success', async () => {
|
||||
vi.doMock('../../src/core/lbug/lbug-adapter.js', mockLbugAdapter);
|
||||
vi.doMock('../../src/core/search/fts-indexes.js', async (importActual) => ({
|
||||
|
|
@ -906,6 +1101,64 @@ describe('runFullAnalysis FTS crash marker', () => {
|
|||
}
|
||||
});
|
||||
|
||||
it('re-detects flows after FTS park when processDetection is uncertified', async () => {
|
||||
const wipeLbugDbFiles = vi.fn(async () => undefined);
|
||||
const runDeferredDerivedPhases = vi.fn(async () => undefined);
|
||||
const runPipelineFromRepo = vi.fn(async (repoPath: string) => ({
|
||||
repoPath,
|
||||
graph: fileGraph(),
|
||||
runDeferredDerivedPhases,
|
||||
}));
|
||||
vi.doMock('../../src/core/lbug/lbug-adapter.js', async () => ({
|
||||
...(await mockLbugAdapter()),
|
||||
wipeLbugDbFiles,
|
||||
}));
|
||||
vi.doMock('../../src/core/search/fts-indexes.js', async (importActual) => ({
|
||||
...(await importActual<typeof import('../../src/core/search/fts-indexes.js')>()),
|
||||
initialiseSearchFTSStemmer: vi.fn(() => 'porter'),
|
||||
missingSearchFTSIndexTables: vi.fn(async () => []),
|
||||
dropSearchFTSIndexes: vi.fn(async () => undefined),
|
||||
buildSearchIndexesOrDegrade: vi.fn(async () => ({ ok: true })),
|
||||
}));
|
||||
vi.doMock('../../src/core/ingestion/pipeline.js', () => ({ runPipelineFromRepo }));
|
||||
|
||||
const tmpRepo = await createTempDir('gitnexus-fts-crash-uncertified-park-');
|
||||
try {
|
||||
await seedGitFile(tmpRepo.dbPath);
|
||||
const { storagePath, lbugPath } = getStoragePaths(tmpRepo.dbPath);
|
||||
await fs.mkdir(storagePath, { recursive: true });
|
||||
await saveMeta(storagePath, {
|
||||
...incrementalMeta(tmpRepo.dbPath),
|
||||
lastCommit: headCommit(tmpRepo.dbPath),
|
||||
processDetection: {
|
||||
maxProcesses: null,
|
||||
maxProcessBranching: PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessBranching,
|
||||
maxProcessTraceDepth: PROCESS_DETECTION_BUDGET_DEFAULTS.maxProcessTraceDepth,
|
||||
maxEntryPointCandidates: PROCESS_DETECTION_BUDGET_DEFAULTS.maxEntryPointCandidates,
|
||||
uncertified: true,
|
||||
},
|
||||
incrementalInProgress: ftsInPlaceDirty,
|
||||
});
|
||||
await fs.writeFile(lbugPath, GRAPH_BYTES);
|
||||
await fs.writeFile(`${lbugPath}.wal`, WAL_PATTERN);
|
||||
|
||||
const { runFullAnalysis } = await import('../../src/core/run-analyze.js');
|
||||
const result = await runFullAnalysis(
|
||||
tmpRepo.dbPath,
|
||||
{ skipAgentsMd: true, skipSkills: true },
|
||||
{ onProgress: () => {}, onLog: () => {} },
|
||||
);
|
||||
|
||||
expect(result.alreadyUpToDate).not.toBe(true);
|
||||
expect(runPipelineFromRepo).toHaveBeenCalled();
|
||||
expect(runDeferredDerivedPhases).toHaveBeenCalled();
|
||||
const finalMeta = await loadMeta(storagePath);
|
||||
expect(finalMeta?.processDetection?.uncertified).toBeUndefined();
|
||||
} finally {
|
||||
await tmpRepo.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
it('does not keep the graph for a non-FTS dirty flag', async () => {
|
||||
const wipeLbugDbFiles = vi.fn(async () => undefined);
|
||||
vi.doMock('../../src/core/lbug/lbug-adapter.js', async () => ({
|
||||
|
|
|
|||
|
|
@ -171,6 +171,27 @@ describe('watch path selection', () => {
|
|||
}
|
||||
});
|
||||
|
||||
it('applies process-detection budget keys from rc and CLI without throwing (#3313)', async () => {
|
||||
await fs.writeFile(
|
||||
path.join(repoPath, '.gitnexusrc'),
|
||||
JSON.stringify({ maxProcesses: '40', maxEntryPointCandidates: 400 }),
|
||||
);
|
||||
const baseline = { maxFileSize: undefined, workerTimeout: undefined, verbose: undefined };
|
||||
await expect(resolveWatchOptions(repoPath, {}, baseline)).resolves.toMatchObject({
|
||||
maxProcesses: 40,
|
||||
maxEntryPointCandidates: 400,
|
||||
});
|
||||
await expect(
|
||||
resolveWatchOptions(repoPath, { maxProcesses: '25' }, baseline),
|
||||
).resolves.toMatchObject({
|
||||
maxProcesses: 25,
|
||||
maxEntryPointCandidates: 400,
|
||||
});
|
||||
const zeroBudget = await resolveWatchOptions(repoPath, { maxProcesses: '0' }, baseline);
|
||||
expect(zeroBudget).toMatchObject({ maxEntryPointCandidates: 400 });
|
||||
expect(zeroBudget.maxProcesses).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects a watch file-size threshold above the parser ceiling', async () => {
|
||||
await expect(
|
||||
resolveWatchOptions(
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue