perf(go): kill O(n²) scope-capture re-walks (resolves #1848 quarantine) (#1915)

* test(go): add #1848 Go pipeline + worker-pool benchmark

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* optimize(go-scope-capture): thread captured nodes to kill O(n^2) findNodeAtRange re-walks

emitGoScopeCaptures re-derived each match's AST node via findNodeAtRange from
the tree root on every query match, giving O(matches x rootChildren) ~ O(n^2)
behaviour (the #1848 root cause: a 250-struct generated DAO took ~10.8s, 800
structs ~100s+ — long enough to trip the worker sub-batch idle timeout and get
quarantined). Thread the query-captured SyntaxNode (c.node) through a parallel
tag->node map and use it directly (or via a bounded local parent walk for the
import_declaration ancestor case) instead of re-walking from root.

Output is byte-identical (capture fingerprint over the DAO file + all 89 go-*
fixtures unchanged; capture_groups=13501). 250 entities: 10835ms -> 114ms (95x).
800 entities: ~100s -> 384ms. Go resolver + scope-resolution suites: 165/165 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(go-scope-capture): address code-review findings

Self-review (ce-code-review) polish on the #1848 fix + benchmark:

- benchmark: tighten the scaling guard from timeRatio/fileRatio < 3 to < 1.5.
  At the 2.5x/2x scale steps, a quadratic regression yields ratio == fileRatio
  (2.5, 2.0), which < 3 waved through — the guard could not detect the O(n^2)
  it exists for. Measured O(n) ratios are 0.45/0.59, so < 1.5 has headroom.
- benchmark: add a non-gated O(n^2) regression tripwire that calls
  emitGoScopeCaptures on a 400-struct source directly (no worker, no
  GITNEXUS_BENCH gate) so the regression is actually guarded in CI.
- benchmark: clearTimeout the Promise.race timer in finally (no lingering
  rejection); set the worker-suite env vars inside the try so finally always
  restores them.
- captures.ts: clarify the isRawMultiAssignTypeBinding comment to name both
  var-form cases (assertion + call-return). Comment-only.

Left as-is: resolveImportNode's defensive range-equality branch — deleting it
as dead code would remove the self-documentation of the grammar invariant the
threaded-node logic depends on (reviewer tension; a wash).

Verified: tsc clean; 165/165 Go resolver + scope tests; new tripwire passes
(237ms); scaling suite passes at <1.5; #1848 worker suite still green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(go): golden capture-parity guard for emitGoScopeCaptures (#1848 U1)

Pins emitGoScopeCaptures output across all 89 go-* fixtures + a synthetic DAO
shape as a committed golden (test/fixtures/go-captures-golden/expected-captures.json),
so future drift in the Go scope-capture path fails CI instead of only the coarse
perf tripwire. Match-grouped, order-independent sha256 canonicalization; regenerate
intentionally with UPDATE_GOLDEN=1. Mirrors test/integration/pipeline-graph-golden.test.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(go): cover func_literal, var-form bindings, single import, generics (#1848 U2)

Adds smoke cases for the Go shapes the #1915 captured-node refactor reasons
about but no lang-resolution fixture exercised: func_literal under @scope.function
(no receiver synthesized), var-form @type-binding.assertion and .call-return (not
dropped by isRawMultiAssignTypeBinding), a single unparenthesized import through
resolveImportNode, and a generic function declaration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(go): tighten O(n^2) tripwire budget 10s -> 5s (#1848 U3)

The fixed path is ~250ms; a quadratic regression at 400 structs is ~25s. 5s keeps
~20x headroom over the fixed path while tripping a ~20x regression (vs the prior
~40x). Correctness is guarded separately by the U1 golden test, so this stays a
pure perf tripwire.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(autofix): apply prettier + eslint fixes via /autofix command

* test(go): fail on a missing golden in CI via a pure resolveGoldenAction helper (#1848 U1)

Extracts the golden test's missing-file gate into a pure
resolveGoldenAction({update,exists,isCI}) -> regenerate|compare|fail helper, so
a missing golden no longer self-heals + passes in CI (Codex F2). The rule is
unit-tested directly across all combos with no filesystem mutation (can't corrupt
the committed golden). CI detection uses a truthy check (!!process.env.CI) so it
fires on any runner. Locally a missing golden still regenerates as first-run convenience.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(go): make the golden digest order-sensitive (#1848 U2)

Drops the cross-match .sort() in digestCaptures so the digest reflects emission
order — a true byte-identical guard that catches a reordering refactor (Codex F1),
not just a set-equality check. Safe because emitGoScopeCaptures output is
deterministic. Within-match key order stays normalized (a CaptureMatch is a Record).
Replaces the order-independence test with an order-sensitivity assertion and
regenerates expected-captures.json under the new scheme (all 90 digests).
Trade-off: a tree-sitter-go grammar bump that reorders matches now requires a
deliberate UPDATE_GOLDEN=1 regen — intentional (a tree-shape change deserves a look).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(go): strengthen func_literal smoke case to a positive receiver assertion (#1848 U3)

The old case used a closure-only source and only asserted ABSENCE of
@type-binding.self, so it would pass even if the method_declaration receiver
branch regressed (Codex F3). The fixture now has both a method and a closure, and
positively asserts exactly one @type-binding.self from the method (name=u,
type=User — the type also confirms *User pointer-stripping) and none from the closure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(test): remove TOCTOU file-system race in golden test + format

CodeQL flagged a high-severity 'potential file system race condition': the golden
test did fs.existsSync(GOLDEN_FILE) then later writeFileSync/readFileSync on it.
Replace the existsSync-then-use with a single race-free read (ENOENT => missing),
reusing the read content for the compare path. Behaviour is unchanged (the pure
resolveGoldenAction helper still decides regenerate/compare/fail). Also applies
prettier formatting to the file (fixes the quality/format check).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Gergő Magyar 2026-05-30 13:05:08 +01:00 committed by GitHub
parent a93ecee068
commit f5915ca9ab
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 1243 additions and 36 deletions

View file

@ -1,10 +1,5 @@
import type { Capture, CaptureMatch } from 'gitnexus-shared';
import {
findNodeAtRange,
nodeToCapture,
syntheticCapture,
type SyntaxNode,
} from '../../utils/ast-helpers.js';
import { nodeToCapture, syntheticCapture, type SyntaxNode } from '../../utils/ast-helpers.js';
import { getGoParser, getGoScopeQuery } from './query.js';
import { recordGoCacheHit, recordGoCacheMiss } from './cache-stats.js';
import { computeGoCallArity, computeGoDeclarationArity } from './arity-metadata.js';
@ -34,18 +29,29 @@ export function emitGoScopeCaptures(
for (const m of rawMatches) {
const grouped: Record<string, Capture> = {};
// Parallel tag -> captured SyntaxNode map. The tree-sitter query already
// hands us the matched node as `c.node`; keeping it here lets us derive the
// anchor/relative node by walking LOCALLY (parent chain / own subtree)
// instead of re-walking from tree.rootNode (the O(matches x rootChildren)
// hotpath that made #1848's 250-struct DAO file take ~10s). The captured
// node either IS the node the old findNodeAtRange re-derived, or is a close
// relative reachable by a bounded local walk.
const nodeMap: Record<string, SyntaxNode> = {};
for (const c of m.captures) {
const tag = '@' + c.name;
if (tag.startsWith('@_')) continue; // skip anonymous captures
grouped[tag] = nodeToCapture(tag, c.node);
nodeMap[tag] = c.node;
}
if (Object.keys(grouped).length === 0) continue;
if (grouped['@import.statement'] !== undefined) {
const anchor = grouped['@import.statement']!;
const importNode =
findNodeAtRange(tree.rootNode, anchor.range, 'import_declaration') ??
findNodeAtRange(tree.rootNode, anchor.range, 'import_spec');
// The captured node is the `import_spec`; the original code preferred its
// enclosing `import_declaration` ONLY when that ancestor shares the exact
// same range (which never happens — the declaration always includes the
// `import` keyword prefix — so it falls back to the import_spec itself).
// Replicate that exactly via a local ancestor walk, never from root.
const importNode = resolveImportNode(nodeMap['@import.statement']!);
if (importNode !== null) {
out.push(...splitGoImportStatement(importNode));
continue;
@ -53,23 +59,33 @@ export function emitGoScopeCaptures(
}
if (grouped['@scope.function'] !== undefined) {
const scopeCap = grouped['@scope.function']!;
// @scope.function captures function_declaration | method_declaration |
// func_literal. The original looked for a function_declaration or
// method_declaration at the captured range; the captured node IS that
// node for the first two, and a func_literal never coincides in range
// with either, so the lookup yields null for func_literal.
const scopeNode = nodeMap['@scope.function']!;
const fnNode =
findNodeAtRange(tree.rootNode, scopeCap.range, 'function_declaration') ??
findNodeAtRange(tree.rootNode, scopeCap.range, 'method_declaration');
scopeNode.type === 'function_declaration' || scopeNode.type === 'method_declaration'
? scopeNode
: null;
if (fnNode !== null) {
const receiver = synthesizeGoReceiverBinding(fnNode);
if (receiver !== null) out.push(receiver);
}
}
if (isRawMultiAssignTypeBinding(tree.rootNode, grouped)) continue;
if (isRawMultiAssignTypeBinding(nodeMap)) continue;
const declAnchor = grouped['@declaration.function'] ?? grouped['@declaration.method'];
if (declAnchor !== undefined) {
const declAnchorNode = nodeMap['@declaration.function'] ?? nodeMap['@declaration.method'];
if (declAnchorNode !== undefined) {
// @declaration.function / @declaration.method are captured directly on
// the function_declaration / method_declaration node.
const fnNode =
findNodeAtRange(tree.rootNode, declAnchor.range, 'function_declaration') ??
findNodeAtRange(tree.rootNode, declAnchor.range, 'method_declaration');
declAnchorNode.type === 'function_declaration' ||
declAnchorNode.type === 'method_declaration'
? declAnchorNode
: null;
if (fnNode !== null) {
const arity = computeGoDeclarationArity(fnNode);
if (arity.parameterCount !== undefined) {
@ -98,15 +114,15 @@ export function emitGoScopeCaptures(
continue;
}
const callAnchor =
grouped['@reference.call.free'] ??
grouped['@reference.call.member'] ??
grouped['@reference.call.constructor'];
if (callAnchor !== undefined && grouped['@reference.arity'] === undefined) {
const callNode =
findNodeAtRange(tree.rootNode, callAnchor.range, 'call_expression') ??
findNodeAtRange(tree.rootNode, callAnchor.range, 'composite_literal');
if (callNode !== null) {
// @reference.call.free / .member are captured on the call_expression;
// @reference.call.constructor on the composite_literal. The captured node
// IS the node the old findNodeAtRange re-derived for each, so use it.
const callNode =
nodeMap['@reference.call.free'] ??
nodeMap['@reference.call.member'] ??
nodeMap['@reference.call.constructor'];
if (callNode !== undefined && grouped['@reference.arity'] === undefined) {
if (callNode.type === 'call_expression' || callNode.type === 'composite_literal') {
grouped['@reference.arity'] = syntheticCapture(
'@reference.arity',
callNode,
@ -146,18 +162,56 @@ export function emitGoScopeCaptures(
return out;
}
function isRawMultiAssignTypeBinding(
rootNode: SyntaxNode,
grouped: Record<string, Capture>,
): boolean {
/**
* Resolve the node passed to `splitGoImportStatement` for an @import.statement
* match. The capture is on the `import_spec`; the original preferred an
* `import_declaration` at the SAME range, else the import_spec. An
* import_declaration always includes the `import` keyword and so never shares
* the spec's exact range the only candidate is an ancestor, and it can only
* match when ranges coincide. Walk the parent chain (bounded, local) for an
* import_declaration whose range equals the spec's; otherwise return the spec.
*/
function resolveImportNode(importSpec: SyntaxNode): SyntaxNode {
let current: SyntaxNode | null = importSpec.parent;
while (current !== null) {
if (current.type === 'import_declaration') {
if (nodeRangeEquals(current, importSpec)) return current;
break;
}
// import_spec is nested at most under import_declaration ->
// import_spec_list -> import_spec; stop once we leave the import subtree.
if (current.type !== 'import_spec_list') break;
current = current.parent;
}
return importSpec;
}
/** True iff two nodes occupy the exact same source range. */
function nodeRangeEquals(a: SyntaxNode, b: SyntaxNode): boolean {
return (
a.startPosition.row === b.startPosition.row &&
a.startPosition.column === b.startPosition.column &&
a.endPosition.row === b.endPosition.row &&
a.endPosition.column === b.endPosition.column
);
}
function isRawMultiAssignTypeBinding(nodeMap: Record<string, SyntaxNode>): boolean {
const anchor =
grouped['@type-binding.constructor'] ??
grouped['@type-binding.call-return'] ??
grouped['@type-binding.assertion'];
nodeMap['@type-binding.constructor'] ??
nodeMap['@type-binding.call-return'] ??
nodeMap['@type-binding.assertion'];
if (anchor === undefined) return false;
const node = findNodeAtRange(rootNode, anchor.range, 'short_var_declaration');
if (node === null) return false;
// These tags are captured directly ON the short_var_declaration, so the
// captured node IS what the original findNodeAtRange(root, range,
// 'short_var_declaration') re-derived. The var_declaration (var-form)
// variants — @type-binding.assertion (`var x = e.(T)`) and
// @type-binding.call-return (`var x = Func()`) — anchor on a var_declaration
// instead; the old range+type lookup found no short_var_declaration at that
// range and returned null -> false, which this type guard reproduces exactly.
if (anchor.type !== 'short_var_declaration') return false;
const node = anchor;
const lhs = node.childForFieldName('left');
const rhs = node.childForFieldName('right');
if (lhs === null) return false;

View file

@ -0,0 +1,362 @@
{
"go-aliased-package-import/internal/util/log.go": {
"captureGroups": 4,
"digest": "77be0bd9a9df464f42a6cefd0c16064d4ef97e79f5f61faa8df6069989cec9f6"
},
"go-aliased-package-import/main.go": {
"captureGroups": 7,
"digest": "3eb2e6d441dadede554b271b7a87b7b9ca557ab418bfb9e8524e6ace4c8fc547"
},
"go-ambiguous/internal/models/handler.go": {
"captureGroups": 9,
"digest": "619c516a5791095bc6380de62fa861364b3f9480f2ec87d4499c2c998f228713"
},
"go-ambiguous/internal/other/handler.go": {
"captureGroups": 9,
"digest": "08a61721581c4f17741ef0c4ee1c8945235ee7f2aca7d88d2fe122071cd62e6f"
},
"go-ambiguous/internal/services/user.go": {
"captureGroups": 8,
"digest": "802b81a07c64c01f2381cf33d41cdb1a58f535c0007b5ae151c38cda87f28321"
},
"go-assignment-chain/cmd/main.go": {
"captureGroups": 50,
"digest": "47ba5fd2ea96ee202b3a3de5c0db75ea18d0889064ed2138d62c67594f7d22e9"
},
"go-assignment-chain/models/repo.go": {
"captureGroups": 8,
"digest": "1b3acbf48751105d056253488f34159266bdb0b229248c85065e927b72bf4801"
},
"go-assignment-chain/models/user.go": {
"captureGroups": 8,
"digest": "457c76ebf1c86efac7a7d9f384a8e49acd5649c006e6cc8aa29a59c36f0b102a"
},
"go-call-result-binding/cmd/main.go": {
"captureGroups": 16,
"digest": "83d611dcee826ec848a0b3fd5a18a98103896d17c353d8d63cd15551039818e9"
},
"go-call-result-binding/models/user.go": {
"captureGroups": 10,
"digest": "5c31199e148340ee32dd48e32a00003e094be963b1287b9b40c6b4effca12175"
},
"go-calls/cmd/main.go": {
"captureGroups": 6,
"digest": "deaea55087c2652aa2d39fefa30048d5393cec40f7af1ab13921331b035030d6"
},
"go-calls/internal/onearg/log.go": {
"captureGroups": 6,
"digest": "a101eafe9f08396bb176cf3cadd3960d752802048e47c0ac9e9ace07244a46fb"
},
"go-calls/internal/zeroarg/log.go": {
"captureGroups": 5,
"digest": "7b322767a38298de8c6ce99fa6b1d69dda8bbb79704053644bfaa7d2e4473fdb"
},
"go-chain-call/cmd/main.go": {
"captureGroups": 20,
"digest": "5a8d7de8ae87887902d16f28cb70d31014c96ab5d2eb63cbd6c8be27650fa9a1"
},
"go-chain-call/models/repo.go": {
"captureGroups": 10,
"digest": "ac4799aae638d528c5c7c01c8b9734fc61e995596a12b12e790dcffab97b4029"
},
"go-chain-call/models/user.go": {
"captureGroups": 10,
"digest": "5c31199e148340ee32dd48e32a00003e094be963b1287b9b40c6b4effca12175"
},
"go-child-extends-parent/models/child.go": {
"captureGroups": 3,
"digest": "6fd9fe7b82066f82a93bf5e5024ddf89382091ec04e55648845c8295f13bd412"
},
"go-child-extends-parent/models/parent.go": {
"captureGroups": 8,
"digest": "454a724f571a5aede89d7657f76ed8e3c9b19bfc18d524141d1b646005f98e49"
},
"go-child-extends-parent/services/app.go": {
"captureGroups": 10,
"digest": "05f5df0369c90bc6e0da5a8a76e0661be36cddbe913ab2f83da2ee3733a46ec2"
},
"go-cmd-helper/cmd/server/internal/config/config.go": {
"captureGroups": 5,
"digest": "b10874198d380b0a186fb1e1ee8cedac644eb3b6b624398110a660183e59a0b5"
},
"go-cmd-helper/cmd/server/main.go": {
"captureGroups": 7,
"digest": "a1f9453bd71926d60e3f148f43b9af813cbd1cccc11b323896a55bdb443f8931"
},
"go-constructor-type-inference/cmd/main.go": {
"captureGroups": 15,
"digest": "4c496bf5ebaebae8c7480b1826b3285752a79ce50562756ded3b7fe0c6d6b325"
},
"go-constructor-type-inference/models/repo.go": {
"captureGroups": 8,
"digest": "1b3acbf48751105d056253488f34159266bdb0b229248c85065e927b72bf4801"
},
"go-constructor-type-inference/models/user.go": {
"captureGroups": 8,
"digest": "457c76ebf1c86efac7a7d9f384a8e49acd5649c006e6cc8aa29a59c36f0b102a"
},
"go-deep-field-chain/cmd/main.go": {
"captureGroups": 13,
"digest": "ca9cc7ae0f75928b1ea338f42e58cf02502e0c93ce4ea8867258c90543f79d16"
},
"go-deep-field-chain/models/models.go": {
"captureGroups": 33,
"digest": "0ad1df946f58e446a8dbd8ef13041b6e0177f53293946b955acf3c46684e4295"
},
"go-field-types/cmd/main.go": {
"captureGroups": 9,
"digest": "3bc98cf5640568cdff3202f2594fb39d563ee595560cb28d1adb31f264670ceb"
},
"go-field-types/models/models.go": {
"captureGroups": 22,
"digest": "fbdbf74d927c0ed07f4820c190fc6f2039b74ddead8abb45fc238812dfa4d4ef"
},
"go-for-call-expr/cmd/main.go": {
"captureGroups": 27,
"digest": "95217f85260d85baeb57638fff470c0a358be92cc49a5f918b084d809f47ba2a"
},
"go-for-call-expr/models/repo.go": {
"captureGroups": 14,
"digest": "312e59c1402cb83a4ce51c27866fde6c597f7121098b57ff1d3bd4dc6363834a"
},
"go-for-call-expr/models/user.go": {
"captureGroups": 14,
"digest": "c442a26c4051c2b7426850a381137507a147c21d631d6535d71ef1035b1506fb"
},
"go-inc-dec-write-access/main.go": {
"captureGroups": 21,
"digest": "0414398239624e44b1589f6a68f2c19636bf4b3ce74990cee1bd7cbeeb0591e6"
},
"go-local-shadow/cmd/main.go": {
"captureGroups": 12,
"digest": "1cda9982d9b4e4878208894523f0a9c7a806c6f08a5582d4771205716a1b2733"
},
"go-local-shadow/internal/utils/utils.go": {
"captureGroups": 6,
"digest": "51e941fa4c7765efc6e472d15a9d7ea31f59b67a6b606be868f4af47e5f54cb1"
},
"go-make-builtin/main.go": {
"captureGroups": 18,
"digest": "7c56328d8416338ae0075ae7dd9669b16f2035aa353fac7ee1e46109a58e80a6"
},
"go-make-builtin/models.go": {
"captureGroups": 15,
"digest": "4d5628f66471f1ad1180a47b797195506d190ddc84f61f6c57f2e22afd754c1e"
},
"go-map-range/main.go": {
"captureGroups": 10,
"digest": "7f3580a0e7858e6eb3176e0b5e1bec4867a2d5d07f2b216a04a3de79ff96170b"
},
"go-map-range/models/repo.go": {
"captureGroups": 9,
"digest": "6cbc4422fb287007735b4c58b5e9c84bbd6b8f6082e3b4d5fe48c306e6acc75b"
},
"go-map-range/models/user.go": {
"captureGroups": 9,
"digest": "7e4dbc05ad1de859cd3103c27cddb60566d95a8a8354c752e8c75f84d7ca055c"
},
"go-member-calls/cmd/main.go": {
"captureGroups": 11,
"digest": "e46f6d1dff39943f8f89c05e0d28f61f8471cdc729c91f01ca693a38a244b8ba"
},
"go-member-calls/models/user.go": {
"captureGroups": 8,
"digest": "457c76ebf1c86efac7a7d9f384a8e49acd5649c006e6cc8aa29a59c36f0b102a"
},
"go-method-chain-binding/cmd/main.go": {
"captureGroups": 21,
"digest": "0fff0df5dd77e13e0b9e62dd7f38efc038d33ffd5d891f684289eccbc46ff583"
},
"go-method-chain-binding/models/user.go": {
"captureGroups": 24,
"digest": "504ea598176dd8e01d759cc54e012735c746f14ec3a660af2c362fc356326f65"
},
"go-method-enrichment/animal.go": {
"captureGroups": 15,
"digest": "ac0933f59d4a88a25629d02f308c66847fd34ceba5831660fc8bff07109b7708"
},
"go-method-enrichment/app.go": {
"captureGroups": 15,
"digest": "ac0bdc2e6daf7d4e28fd255fe2edd143e8fab7ff316f4e32e2ca04a3b33f71c2"
},
"go-mixed-chain/cmd/main.go": {
"captureGroups": 20,
"digest": "3662b803da4f4fc1ac0552a45d3d262bd0db57f97b056614caacea8ad856250e"
},
"go-mixed-chain/models/models.go": {
"captureGroups": 41,
"digest": "dc2cedaefcd73faf13a9f704a4be8f0bd4140c608cd86480e1f4400fec49dbd1"
},
"go-multi-assign/app.go": {
"captureGroups": 16,
"digest": "66958a3e86caa54aedd795227dac274cfb3f4b39ab98964e8a7bcbdfc8a08ca6"
},
"go-multi-assign/models.go": {
"captureGroups": 19,
"digest": "a4d43cf2cd2f7bdbc750a7ee1f5e9c7611e5d1e25f1a46386d0eeb9f73bb7846"
},
"go-multi-return-inference/cmd/main.go": {
"captureGroups": 36,
"digest": "6229b69cfd15bf1465d770d97486522faac5a91cf8828c424c8c8c989b310224"
},
"go-multi-return-inference/models/repo.go": {
"captureGroups": 10,
"digest": "ac4799aae638d528c5c7c01c8b9734fc61e995596a12b12e790dcffab97b4029"
},
"go-multi-return-inference/models/user.go": {
"captureGroups": 10,
"digest": "5c31199e148340ee32dd48e32a00003e094be963b1287b9b40c6b4effca12175"
},
"go-new-builtin/main.go": {
"captureGroups": 12,
"digest": "1177b99217a42a28b0d768d29e1df4198f0d5ca5f1c2b584d528f5f02354b38f"
},
"go-new-builtin/models.go": {
"captureGroups": 17,
"digest": "40c9f89942406caf2610f0d1954b66e905c39dd5d617d7359e572445d586c15d"
},
"go-nullable-receiver/cmd/main.go": {
"captureGroups": 27,
"digest": "7a93b339f8c68ed3882802da21343c1d6b5231ae5cd8dd13eb4c05e2e7716320"
},
"go-nullable-receiver/models/repo.go": {
"captureGroups": 8,
"digest": "1b3acbf48751105d056253488f34159266bdb0b229248c85065e927b72bf4801"
},
"go-nullable-receiver/models/user.go": {
"captureGroups": 8,
"digest": "457c76ebf1c86efac7a7d9f384a8e49acd5649c006e6cc8aa29a59c36f0b102a"
},
"go-parent-resolution/models/base.go": {
"captureGroups": 8,
"digest": "2afaeb50d544a55fe437ef20e2c0de92152d2ba62f2693c329255787bb3d0a02"
},
"go-parent-resolution/models/user.go": {
"captureGroups": 8,
"digest": "c76ba16343dd94024fdaac10fa7640536966e530d675c0084434f42edb5b5f15"
},
"go-pkg/cmd/main.go": {
"captureGroups": 14,
"digest": "a7781d23802e876b9ca2951b7ebb317c5df5adf390ae9a57e8e2906af34f3255"
},
"go-pkg/internal/auth/service.go": {
"captureGroups": 17,
"digest": "2204643b50f486423ee7a5877b2bab7d6334cbe62b14b4435fe4ba8a6465ce92"
},
"go-pkg/internal/models/admin.go": {
"captureGroups": 13,
"digest": "1a5ec9fd5e752adcfec91cd03b3c2a67c124852228a527002237ec51cd4b39b7"
},
"go-pkg/internal/models/repository.go": {
"captureGroups": 3,
"digest": "7de6e11a3cf9c89afa9d89fe37dab85b208699f20105fbade223e4f78a63b15c"
},
"go-pkg/internal/models/user.go": {
"captureGroups": 13,
"digest": "e56fcea1c473866ed06fc0262702e54c72016a9556cc6337330c03cc1f638fe1"
},
"go-pointer-constructor-inference/cmd/main.go": {
"captureGroups": 15,
"digest": "39f9030e909a37f0e13724f50673dd4a72ba240fb6ac263192ee4feadbc88adf"
},
"go-pointer-constructor-inference/models/repo.go": {
"captureGroups": 10,
"digest": "ac4799aae638d528c5c7c01c8b9734fc61e995596a12b12e790dcffab97b4029"
},
"go-pointer-constructor-inference/models/user.go": {
"captureGroups": 10,
"digest": "5c31199e148340ee32dd48e32a00003e094be963b1287b9b40c6b4effca12175"
},
"go-receiver-method-free-call/example.go": {
"captureGroups": 8,
"digest": "2a3c26672d3b997bdc39644361c550f8cf0749489f0945d210e2fb7f3bca9383"
},
"go-receiver-method-free-call/util.go": {
"captureGroups": 4,
"digest": "0ac9740c13c851ca101e074bd16422f9da45b8c27db4befbc0c5e7479c2ccdda"
},
"go-receiver-resolution/cmd/main.go": {
"captureGroups": 13,
"digest": "e92c59312a46972a6083ba489888b550cdd024b809acbbf367ad025efb844a5c"
},
"go-receiver-resolution/models/repo.go": {
"captureGroups": 8,
"digest": "1b3acbf48751105d056253488f34159266bdb0b229248c85065e927b72bf4801"
},
"go-receiver-resolution/models/user.go": {
"captureGroups": 8,
"digest": "457c76ebf1c86efac7a7d9f384a8e49acd5649c006e6cc8aa29a59c36f0b102a"
},
"go-return-type-inference/cmd/main.go": {
"captureGroups": 39,
"digest": "ac8ca3dc1fb7fb4d7a1f947a77e93db890f04d328135dc514f36ba5cd04bc3e1"
},
"go-return-type-inference/models/repo.go": {
"captureGroups": 16,
"digest": "8c500db82093b4622acbca734f3040e6176000a9d9a4843e2a81e3f2b3daee7c"
},
"go-return-type-inference/models/user.go": {
"captureGroups": 16,
"digest": "a70b19b9c46a02003f26d0b70fa5368983a791583ec3776e983c733a9283eefa"
},
"go-same-package-factory/main.go": {
"captureGroups": 14,
"digest": "505d4d279615f3c99457d8cb0f29fdf946b547c6b1fda6d43aa6d4302dd61809"
},
"go-same-package-factory/repo.go": {
"captureGroups": 8,
"digest": "f8bb213588f517e166b421f19d72cde981153464cc3f3168499769e778c4a22e"
},
"go-same-package-factory/user.go": {
"captureGroups": 8,
"digest": "4daaad60f35d4519a24cd067baf4cd4bdc421e3dd80b9110a8dfaa8a9e75d9eb"
},
"go-split-method-owner/main.go": {
"captureGroups": 9,
"digest": "e9c105208ad6ef2f087f758972475aa403294e886751d75113b4ca45fa4e0b8a"
},
"go-split-method-owner/repo.go": {
"captureGroups": 8,
"digest": "f8bb213588f517e166b421f19d72cde981153464cc3f3168499769e778c4a22e"
},
"go-split-method-owner/save.go": {
"captureGroups": 6,
"digest": "6e0e1b5521a2fad1cd00252e771926d5caa7930440278968f7f8607821ad339c"
},
"go-split-method-owner/user.go": {
"captureGroups": 3,
"digest": "827dc0208b47776976313a4560fbd250ab7fa521ae46581213a79fc15fd52ce8"
},
"go-struct-literals/app.go": {
"captureGroups": 10,
"digest": "97ec29ec0e0dc1f23a804ff6808023a32eafdeb00f194d3bea9b50ed54c7f258"
},
"go-struct-literals/user.go": {
"captureGroups": 10,
"digest": "89b791a9d150fe341924d54d3173d9f35f72be899d0e7ac03a35d32119b94f8f"
},
"go-type-assertion/main.go": {
"captureGroups": 11,
"digest": "b8bd327d3965531802a93bc01e3c24969a027540397a1635119ef0b7da46c348"
},
"go-type-assertion/models.go": {
"captureGroups": 17,
"digest": "3780f8f7c145a15f849ae6958e492db0044e825b81aa8dfd21b9873759fad65e"
},
"go-variadic-resolution/cmd/main.go": {
"captureGroups": 6,
"digest": "443f9736b9c67df30fe97d8353ddc2584de4c88ba698e62878683d5d77362d89"
},
"go-variadic-resolution/internal/logger/logger.go": {
"captureGroups": 4,
"digest": "83b987f527f95e360966793148d07e569fa02fdc5e4febc09daa51cfce16b114"
},
"go-write-access/main.go": {
"captureGroups": 20,
"digest": "92fdad46b6933fd78dcbb19677a720f037306a4f03fedcca9156b54c1c9bf994"
},
"synthetic:dao-20": {
"captureGroups": 481,
"digest": "1698b5dd78c8094f251b10ab8cacebfbf453f38eddf7233fbc7964e32f04ceeb"
}
}

View file

@ -0,0 +1,456 @@
/**
* Go ingestion pipeline benchmark.
*
* Generates synthetic Go codebases at increasing scales and measures
* wall-clock time and peak heap through the full pipeline parsing,
* Go scope capture (emitGoScopeCaptures), package/import resolution, and
* call resolution.
*
* Run: GITNEXUS_BENCH=1 npx vitest run test/integration/go-pipeline-benchmark.test.ts
*
* The first suite ("scales with file count") generates many small files
* each a struct plus getter/setter/compute methods, the DAO-style shape that
* stresses the Go scope-capture path. Run under vitest it falls back to the
* sequential path (no compiled worker), so it measures parse + scope-capture
* scaling without the worker pool.
*
* The second suite ("worker pool — issue #1848") reproduces the actual bug:
* it points the pool at the COMPILED `dist/.../parse-worker.js` (real
* worker_threads), generates one ~400 KiB generated-DAO file plus padding so
* the 15-file worker threshold trips, and runs with a short sub-batch idle
* timeout. Under the buggy code the worker looks idle while inside
* emitGoScopeCaptures and the file is quarantined; the fix emits progress so
* the file survives. Requires a build first:
*
* (cd gitnexus && npm run build)
* GITNEXUS_BENCH=1 npx vitest run test/integration/go-pipeline-benchmark.test.ts
*
* The worker suite auto-skips if the compiled worker is absent.
*/
import { describe, it, expect } from 'vitest';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { runPipelineFromRepo } from '../../src/core/ingestion/pipeline.js';
import { emitGoScopeCaptures } from '../../src/core/ingestion/languages/go/index.js';
const BENCH_ENABLED = process.env.GITNEXUS_BENCH === '1';
const MODULE_PATH = 'example.com/go-bench';
/**
* The compiled worker the pool spawns. Under vitest, `import.meta.url`
* resolves to `src/`, where no `.js` exists so we point straight at the
* `dist/` build, the same fallback parse-impl uses. `null` when unbuilt.
*/
const DIST_WORKER_URL = new URL(
'../../dist/core/ingestion/workers/parse-worker.js',
import.meta.url,
);
const DIST_WORKER_AVAILABLE = fs.existsSync(fileURLToPath(DIST_WORKER_URL));
interface BenchResult {
fileCount: number;
structCount: number;
packageCount: number;
elapsedMs: number;
peakHeapMB: number;
nodeCount: number;
edgeCount: number;
}
function generateGoFixture(
fileCount: number,
packageCount: number,
): { dir: string; structCount: number; packageCount: number } {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), `go-bench-${fileCount}-`));
fs.writeFileSync(path.join(dir, 'go.mod'), `module ${MODULE_PATH}\n\ngo 1.22\n`);
const packages: string[] = [];
for (let i = 0; i < packageCount; i++) {
packages.push(`pkg${i}`);
}
const structCount = fileCount;
const createdPackages = new Set<string>();
for (let f = 0; f < fileCount; f++) {
const pkg = packages[f % packages.length];
const pkgDir = path.join(dir, pkg);
if (!createdPackages.has(pkg)) {
fs.mkdirSync(pkgDir, { recursive: true });
createdPackages.add(pkg);
}
const structName = `Item${f}`;
const siblingIdx = (f + 1) % fileCount;
const siblingStruct = `Item${siblingIdx}`;
const siblingPkg = packages[siblingIdx % packages.length];
const crossIdx = (f + Math.floor(fileCount / 3)) % fileCount;
const crossStruct = `Item${crossIdx}`;
const crossPkg = packages[crossIdx % packages.length];
// Only import packages we actually reference, and never our own.
const imports = new Set<string>();
if (siblingPkg !== pkg) imports.add(siblingPkg);
if (crossPkg !== pkg) imports.add(crossPkg);
const importBlock =
imports.size > 0
? ['import (', ...[...imports].map((p) => `\t"${MODULE_PATH}/${p}"`), ')', '']
: [];
const qualify = (otherPkg: string, name: string) =>
otherPkg === pkg ? name : `${otherPkg}.${name}`;
const siblingRef = qualify(siblingPkg, siblingStruct);
const siblingCtor = qualify(siblingPkg, `New${siblingStruct}`);
const crossRef = qualify(crossPkg, crossStruct);
const crossCtor = qualify(crossPkg, `New${crossStruct}`);
const content = [
`package ${pkg}`,
'',
...importBlock,
`type ${structName} struct {`,
`\tID int64`,
`\tName string`,
`\tEmail string`,
`\tValue float64`,
`}`,
'',
`func New${structName}(id int64, name string) *${structName} {`,
`\treturn &${structName}{ID: id, Name: name}`,
`}`,
'',
`func (i *${structName}) GetID() int64 {`,
`\treturn i.ID`,
`}`,
'',
`func (i *${structName}) SetID(id int64) {`,
`\ti.ID = id`,
`}`,
'',
`func (i *${structName}) GetName() string {`,
`\treturn i.Name`,
`}`,
'',
`func (i *${structName}) SetValue(v float64) {`,
`\ti.Value = v`,
`}`,
'',
`func (i *${structName}) Compute() float64 {`,
`\treturn i.Value * float64(i.ID)`,
`}`,
'',
`func (i *${structName}) Process() *${siblingRef} {`,
`\tsibling := ${siblingCtor}(i.ID, i.Name)`,
`\tsibling.SetValue(i.Compute())`,
`\treturn sibling`,
`}`,
'',
`func (i *${structName}) CrossCall() *${crossRef} {`,
`\tcross := ${crossCtor}(i.ID, i.Name)`,
`\t_ = cross.GetID()`,
`\treturn cross`,
`}`,
'',
].join('\n');
fs.writeFileSync(path.join(pkgDir, `item${f}.go`), content);
}
return { dir, structCount, packageCount: createdPackages.size };
}
async function runBenchmark(
fileCount: number,
packageCount: number,
budgetMs: number,
): Promise<BenchResult> {
const {
dir,
structCount,
packageCount: actualPackages,
} = generateGoFixture(fileCount, packageCount);
let peakHeapMB = 0;
const heapSampler = setInterval(() => {
const heap = process.memoryUsage().heapUsed / 1024 / 1024;
if (heap > peakHeapMB) peakHeapMB = heap;
}, 50);
let timeoutHandle: ReturnType<typeof setTimeout> | undefined;
try {
const start = Date.now();
const result = await Promise.race([
runPipelineFromRepo(dir, () => {}, { skipGraphPhases: true }),
new Promise<never>((_, reject) => {
// Hold the handle so the winning (pipeline) path can cancel it in the
// finally — otherwise the timer lingers for up to budgetMs and its
// late rejection surfaces as an unhandled promise rejection.
timeoutHandle = setTimeout(
() => reject(new Error(`Pipeline exceeded ${budgetMs}ms at ${fileCount} files`)),
budgetMs,
);
}),
]);
const elapsedMs = Date.now() - start;
return {
fileCount,
structCount,
packageCount: actualPackages,
elapsedMs,
peakHeapMB: Math.round(peakHeapMB),
nodeCount: result.graph.nodeCount,
edgeCount: result.graph.relationshipCount,
};
} finally {
if (timeoutHandle !== undefined) clearTimeout(timeoutHandle);
clearInterval(heapSampler);
fs.rmSync(dir, { recursive: true, force: true });
}
}
function printResults(label: string, results: BenchResult[]) {
console.log(`\n${label}`);
console.log('┌──────────┬─────────┬──────────┬───────────┬──────────┬───────┬───────┐');
console.log('│ Files │ Structs │ Packages │ Time (ms) │ Heap MB │ Nodes │ Edges │');
console.log('├──────────┼─────────┼──────────┼───────────┼──────────┼───────┼───────┤');
for (const r of results) {
console.log(
`${String(r.fileCount).padStart(8)}${String(r.structCount).padStart(7)}${String(r.packageCount).padStart(8)}${String(r.elapsedMs).padStart(9)}${String(r.peakHeapMB).padStart(8)}${String(r.nodeCount).padStart(5)}${String(r.edgeCount).padStart(5)}`,
);
}
console.log('└──────────┴─────────┴──────────┴───────────┴──────────┴───────┴───────┘');
if (results.length >= 2) {
console.log('\nScaling ratios (time_ratio / file_ratio):');
for (let i = 1; i < results.length; i++) {
const fileRatio = results[i].fileCount / results[i - 1].fileCount;
const timeRatio = results[i].elapsedMs / results[i - 1].elapsedMs;
const scaling = timeRatio / fileRatio;
console.log(
` ${results[i - 1].fileCount}${results[i].fileCount}: ${scaling.toFixed(2)}x (${scaling < 1.5 ? 'linear' : scaling < 3 ? 'superlinear' : 'WARNING: quadratic'})`,
);
}
}
}
/**
* Mirrors the issue #1848 reproduction fixture: one large generated-DAO Go
* file (`package generated`, struct + 7 methods per entity) plus `padCount`
* trivial files so parse-impl crosses the 15-file worker threshold and the
* real worker pool engages.
*/
function generateGoQuarantineFixture(
entityCount: number,
padCount: number,
): { dir: string; bigFileBytes: number; fileCount: number } {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), `go-bench-1848-${entityCount}-`));
const lines = [
'package generated',
'',
'// Code generated for GitNexus issue #1848 repro. DO NOT EDIT.',
'',
];
for (let i = 0; i < entityCount; i++) {
const n = String(i).padStart(4, '0');
lines.push(`type DefUserDao${n} struct {`);
lines.push('\tid int64');
lines.push('\tname string');
lines.push('\temail string');
lines.push('\tcreatedAt int64');
lines.push('}', '');
lines.push(`func (d *DefUserDao${n}) GetID() int64 { return d.id }`);
lines.push(`func (d *DefUserDao${n}) SetID(id int64) { d.id = id }`);
lines.push(`func (d *DefUserDao${n}) GetName() string { return d.name }`);
lines.push(`func (d *DefUserDao${n}) SetName(name string) { d.name = name }`);
lines.push(`func (d *DefUserDao${n}) GetEmail() string { return d.email }`);
lines.push(`func (d *DefUserDao${n}) SetEmail(email string) { d.email = email }`);
lines.push(`func (d *DefUserDao${n}) Validate() error { return nil }`);
lines.push('');
}
const bigContent = lines.join('\n');
fs.writeFileSync(path.join(dir, 'zz_generated.def_userdao.go'), bigContent);
for (let i = 0; i < padCount; i++) {
const idx = String(i).padStart(2, '0');
fs.writeFileSync(
path.join(dir, `pad_${idx}.go`),
`package pad${i}\n\nfunc Ping${i}() int { return ${i} }\n`,
);
}
return {
dir,
bigFileBytes: Buffer.byteLength(bigContent),
fileCount: 1 + padCount,
};
}
describe.skipIf(!BENCH_ENABLED)('Go pipeline benchmark', () => {
it('scales with file count (workers enabled)', async () => {
const scales = [100, 250, 500];
const results: BenchResult[] = [];
for (const fileCount of scales) {
const packageCount = Math.max(4, Math.ceil(Math.sqrt(fileCount)));
const result = await runBenchmark(fileCount, packageCount, 180_000);
results.push(result);
console.log(
` ${fileCount} files: ${result.elapsedMs}ms, ${result.peakHeapMB}MB heap, ${result.nodeCount} nodes, ${result.edgeCount} edges`,
);
}
printResults('Go Pipeline — Workers Enabled', results);
for (let i = 1; i < results.length; i++) {
const fileRatio = results[i].fileCount / results[i - 1].fileCount;
const timeRatio = results[i].elapsedMs / results[i - 1].elapsedMs;
// The scale steps are 2.5x (100->250) and 2x (250->500). A quadratic
// regression makes timeRatio ~= fileRatio^2, i.e. timeRatio/fileRatio ~=
// fileRatio (2.5 and 2.0) — which a `< 3` bound would wave through. The
// O(n) path keeps this ratio ~1 (measured 0.44 and 0.75 post-fix), so a
// `< 1.5` bound (the printResults "linear" boundary) actually fails on a
// re-regression to O(n^2) while leaving comfortable headroom for linear.
expect(timeRatio / fileRatio).toBeLessThan(1.5);
}
}, 300_000);
});
describe.skipIf(!BENCH_ENABLED || !DIST_WORKER_AVAILABLE)(
'Go pipeline benchmark — worker pool (issue #1848)',
() => {
if (BENCH_ENABLED && !DIST_WORKER_AVAILABLE) {
// Surfaced once when the bench is requested but the worker is unbuilt.
console.warn(
`\n[go-bench] Skipping worker-pool suite: compiled worker not found at\n ${fileURLToPath(DIST_WORKER_URL)}\n Build first: (cd gitnexus && npm run build)\n`,
);
}
// Tunables mirror run-analyze-repro.sh. 800 entities ≈ 406 KiB — under the
// 512 KiB GITNEXUS_MAX_FILE_SIZE ceiling so the file is parsed, not skipped.
const entityCount = Number(process.env.REPRO_GO_ENTITIES ?? 800);
// 30 s reproduces on the report author's machine; raising it (e.g. 120000)
// is the documented workaround. Kept overridable so the same test can both
// reproduce the bug and confirm the fix.
const subBatchTimeoutMs = Number(process.env.GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS ?? 30_000);
it('does not quarantine the large generated Go file on sub-batch idle timeout', async () => {
const { dir, bigFileBytes, fileCount } = generateGoQuarantineFixture(entityCount, 14);
// Sub-batch knobs that force fine chunking onto the worker (env-only —
// there is no PipelineOptions field for these two). Capture the prior
// values before the try; set them as the first statements INSIDE it so
// the finally's restore covers every path that mutated the process env.
const prevMaxBytes = process.env.GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES;
const prevTimeout = process.env.GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS;
let peakHeapMB = 0;
const heapSampler = setInterval(() => {
const heap = process.memoryUsage().heapUsed / 1024 / 1024;
if (heap > peakHeapMB) peakHeapMB = heap;
}, 50);
try {
process.env.GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES = '262144';
process.env.GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS = String(subBatchTimeoutMs);
const start = Date.now();
const result = await runPipelineFromRepo(dir, () => {}, {
skipGraphPhases: true,
// Real worker_threads against the compiled worker — the surface the
// bug actually lives on.
workerUrlForTest: DIST_WORKER_URL,
// Match the repro's chunking so the byte budget mirrors the issue.
chunkByteBudget: 262144,
parseChunkConcurrency: 1,
});
const elapsedMs = Date.now() - start;
// The big file alone emits ≥ entityCount*5 nodes (1 struct + 7 methods
// each). If the worker is quarantined on the idle timeout, those
// vanish — so this threshold is the regression guard.
const survivalFloor = entityCount * 5;
const survived = result.graph.nodeCount >= survivalFloor;
console.log(
`\nGo Pipeline — Worker Pool (issue #1848)` +
`\n files: ${fileCount} (1 big @ ${Math.round(bigFileBytes / 1024)} KiB + 14 pad)` +
`\n entities: ${entityCount}, sub-batch idle timeout: ${subBatchTimeoutMs}ms` +
`\n elapsed: ${elapsedMs}ms, peak heap: ${Math.round(peakHeapMB)}MB` +
`\n usedWorkerPool: ${result.usedWorkerPool}` +
`\n nodes: ${result.graph.nodeCount} (survival floor ${survivalFloor}) → ${survived ? 'SURVIVED' : 'QUARANTINED (bug reproduced)'}`,
);
// Sanity: the worker path must have actually engaged, else the test
// proves nothing about the worker bug.
expect(result.usedWorkerPool).toBe(true);
// The fix: the generated file is fully parsed despite the idle timeout.
expect(result.graph.nodeCount).toBeGreaterThanOrEqual(survivalFloor);
} finally {
clearInterval(heapSampler);
if (prevMaxBytes === undefined) delete process.env.GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES;
else process.env.GITNEXUS_WORKER_SUB_BATCH_MAX_BYTES = prevMaxBytes;
if (prevTimeout === undefined) delete process.env.GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS;
else process.env.GITNEXUS_WORKER_SUB_BATCH_TIMEOUT_MS = prevTimeout;
fs.rmSync(dir, { recursive: true, force: true });
}
}, 360_000);
},
);
/**
* Unlike the two suites above, this one is NOT gated behind GITNEXUS_BENCH and
* needs no compiled worker so it runs in normal CI and is the actual guard
* against an O(n^2) re-regression of emitGoScopeCaptures (issue #1848). It calls
* the hotpath directly on a ~400-struct generated source. The O(n) path does
* this in a few hundred ms; the old findNodeAtRange-from-root behaviour took
* ~25s+ at this size. The budget is a coarse tripwire (huge margin over the
* fixed path, far below a quadratic regression), not a microbenchmark keep it
* generous so it never flakes on a loaded CI runner.
*/
describe('Go scope-capture O(n^2) regression tripwire', () => {
function generateGoStructSource(structCount: number): string {
const lines = ['package generated', ''];
for (let i = 0; i < structCount; i++) {
const n = String(i).padStart(4, '0');
lines.push(
`type Item${n} struct {`,
'\tid int64',
'\tname string',
'}',
'',
`func (d *Item${n}) GetID() int64 { return d.id }`,
`func (d *Item${n}) SetID(id int64) { d.id = id }`,
`func (d *Item${n}) GetName() string { return d.name }`,
`func (d *Item${n}) Validate() error { return nil }`,
'',
);
}
return lines.join('\n');
}
it('parses a 400-struct file in well under the O(n^2) tripwire budget', () => {
const STRUCT_COUNT = 400;
const BUDGET_MS = 5_000; // coarse: ~20x the fixed path (~250ms), trips a ~20x regression; a quadratic regression at 400 structs is ~25s
const src = generateGoStructSource(STRUCT_COUNT);
emitGoScopeCaptures(src, 'tripwire-warmup.go'); // warm up the parser/query JIT
const start = Date.now();
const matches = emitGoScopeCaptures(src, 'tripwire.go');
const elapsedMs = Date.now() - start;
// Sanity: the captures are actually produced (each struct + 4 methods emits
// far more than 10 capture groups), so a fast-but-empty result can't pass.
expect(matches.length).toBeGreaterThan(STRUCT_COUNT * 10);
// The actual regression guard: a re-regression to O(n^2) blows this budget.
expect(elapsedMs).toBeLessThan(BUDGET_MS);
}, 30_000);
});

View file

@ -0,0 +1,235 @@
/**
* Golden capture-parity test for `emitGoScopeCaptures` (issue #1848 follow-up).
*
* Pins the exact capture output of `emitGoScopeCaptures` across the whole
* `test/fixtures/lang-resolution/go-*` corpus plus a synthetic generated-DAO
* source, so any future drift in the Go scope-capture path fails CI rather than
* only being caught by a coarse perf tripwire or pipeline-level resolver tests.
*
* This is a FORWARD-DRIFT guard: it locks in the current (post-#1915, verified)
* output as the baseline. It does not independently re-prove the original
* pre-fix parity that was established during PR #1915.
*
* Regenerate the golden intentionally with `UPDATE_GOLDEN=1` in the environment.
*
* Per fixture the snapshot stores `{ captureGroups, digest }`:
* - captureGroups: number of capture matches (makes a count change legible)
* - digest: sha256 of a match-grouped, order-sensitive (emission-order)
* canonicalization (see canonicalize* below). Order-sensitivity is safe
* because emitGoScopeCaptures output is deterministic, and it makes the
* digest a true byte-identical guard (a reordering refactor is real drift).
* Nothing path/time/id-dependent leaks in.
*
* Pattern: mirrors test/integration/pipeline-graph-golden.test.ts.
*/
import { describe, it, expect } from 'vitest';
import path from 'path';
import fs from 'fs';
import crypto from 'crypto';
import { emitGoScopeCaptures } from '../../../../src/core/ingestion/languages/go/index.js';
import type { CaptureMatch } from 'gitnexus-shared';
// This test lives at test/unit/scope-resolution/go/, so fixtures are THREE
// levels up (unlike pipeline-graph-golden.test.ts at test/integration/).
const FIXTURE_ROOT = path.resolve(__dirname, '..', '..', '..', 'fixtures', 'lang-resolution');
const GOLDEN_DIR = path.resolve(__dirname, '..', '..', '..', 'fixtures', 'go-captures-golden');
const GOLDEN_FILE = path.join(GOLDEN_DIR, 'expected-captures.json');
const UPDATE = process.env.UPDATE_GOLDEN === '1';
interface FixtureSnapshot {
captureGroups: number;
digest: string;
}
type Snapshot = Record<string, FixtureSnapshot>;
/**
* Canonicalize ONE match. A CaptureMatch is a Record<tag, Capture> (multiple
* captures per match), so we group by match to preserve match identity:
* build one `tag|text|startLine:startCol-endLine:endCol` string per capture,
* sort them within the match, and join. We deliberately do NOT flatten every
* capture into one global list that would lose match boundaries.
*/
function canonicalizeMatch(match: CaptureMatch): string {
const parts: string[] = [];
for (const tag of Object.keys(match)) {
const cap = match[tag]!;
const r = cap.range;
parts.push(`${tag}|${cap.text}|${r.startLine}:${r.startCol}-${r.endLine}:${r.endCol}`);
}
parts.sort();
return parts.join(';');
}
/** Order-sensitive (emission-order) digest of a full capture result (match-grouped). */
function digestCaptures(matches: readonly CaptureMatch[]): string {
// No cross-match sort: the digest reflects emission order so a reordering
// refactor surfaces as drift. Within-match key order IS normalized
// (canonicalizeMatch sorts), since a CaptureMatch is an unordered Record.
const matchStrings = matches.map(canonicalizeMatch);
return crypto.createHash('sha256').update(matchStrings.join('\n')).digest('hex');
}
function snapshotOf(src: string, filePath: string): FixtureSnapshot {
const matches = emitGoScopeCaptures(src, filePath);
return { captureGroups: matches.length, digest: digestCaptures(matches) };
}
/** All `.go` files under `lang-resolution/go-*`, as sorted repo-relative-ish keys. */
function collectGoFixtures(): { key: string; absPath: string }[] {
const out: { key: string; absPath: string }[] = [];
for (const entry of fs.readdirSync(FIXTURE_ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || !entry.name.startsWith('go-')) continue;
const stack = [path.join(FIXTURE_ROOT, entry.name)];
while (stack.length) {
const dir = stack.pop()!;
for (const c of fs.readdirSync(dir, { withFileTypes: true })) {
const p = path.join(dir, c.name);
if (c.isDirectory()) stack.push(p);
else if (c.name.endsWith('.go')) {
out.push({ key: path.relative(FIXTURE_ROOT, p).split(path.sep).join('/'), absPath: p });
}
}
}
}
out.sort((a, b) => a.key.localeCompare(b.key));
return out;
}
/** Small deterministic generated-DAO source — the #1848 shape at correctness scale. */
function generateDao(entityCount: number): string {
const lines = ['package generated', ''];
for (let i = 0; i < entityCount; i++) {
const n = String(i).padStart(4, '0');
lines.push(
`type DefUserDao${n} struct {`,
'\tid int64',
'\tname string',
'}',
'',
`func (d *DefUserDao${n}) GetID() int64 { return d.id }`,
`func (d *DefUserDao${n}) SetName(name string) { d.name = name }`,
`func (d *DefUserDao${n}) Validate() error { return nil }`,
'',
);
}
return lines.join('\n');
}
function buildSnapshot(): Snapshot {
const snap: Snapshot = {};
for (const { key, absPath } of collectGoFixtures()) {
snap[key] = snapshotOf(fs.readFileSync(absPath, 'utf8'), absPath);
}
snap['synthetic:dao-20'] = snapshotOf(generateDao(20), 'zz_generated.def_userdao.go');
// Stable key order for deterministic JSON serialization.
return Object.fromEntries(
Object.keys(snap)
.sort()
.map((k) => [k, snap[k]!]),
);
}
function formatGolden(snap: Snapshot): string {
return JSON.stringify(snap, null, 2) + '\n';
}
/**
* Pure decision for what the golden test should do extracted so the
* fail-on-missing-in-CI rule is unit-testable without touching the filesystem
* (and can never corrupt the committed golden). A missing golden must NOT
* self-heal in CI; locally it regenerates as a first-run convenience.
*/
type GoldenAction = 'regenerate' | 'compare' | 'fail';
function resolveGoldenAction(opts: {
update: boolean;
exists: boolean;
isCI: boolean;
}): GoldenAction {
if (opts.update) return 'regenerate';
if (!opts.exists) return opts.isCI ? 'fail' : 'regenerate';
return 'compare';
}
describe('Go scope captures — golden parity', () => {
it('matches the committed golden snapshot across all go-* fixtures + DAO shape', () => {
const snapshot = buildSnapshot();
// Read the golden once (no existsSync-then-use, which is a TOCTOU race):
// ENOENT means the golden is missing; reuse `existing` for the compare path.
let existing: string | undefined;
try {
existing = fs.readFileSync(GOLDEN_FILE, 'utf8');
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') throw err;
}
const action = resolveGoldenAction({
update: UPDATE,
exists: existing !== undefined,
isCI: !!process.env.CI, // truthy check: fires on any CI runner, not just CI==='true'
});
if (action === 'fail') {
throw new Error(
`[go-captures-golden] golden file missing at ${GOLDEN_FILE} in CI. A missing golden must ` +
`not self-heal in CI — regenerate it locally with UPDATE_GOLDEN=1 and commit it.`,
);
}
if (action === 'regenerate') {
fs.mkdirSync(GOLDEN_DIR, { recursive: true });
fs.writeFileSync(GOLDEN_FILE, formatGolden(snapshot), 'utf8');
console.log(
`[go-captures-golden] ${UPDATE ? 'Regenerated' : 'Created'} golden at ${GOLDEN_FILE}`,
);
return;
}
const expected: Snapshot = JSON.parse(existing!);
expect(
snapshot,
'emitGoScopeCaptures output drifted from the committed golden. If this drift is intentional ' +
'(or the digest scheme changed), regenerate with ' +
'UPDATE_GOLDEN=1 npx vitest run test/unit/scope-resolution/go/go-captures-golden.test.ts',
).toEqual(expected);
});
// The fail-on-missing-in-CI rule, asserted purely (no filesystem mutation).
it.each([
{ update: true, exists: false, isCI: true, expected: 'regenerate' },
{ update: false, exists: false, isCI: true, expected: 'fail' },
{ update: false, exists: false, isCI: false, expected: 'regenerate' },
{ update: false, exists: true, isCI: true, expected: 'compare' },
{ update: false, exists: true, isCI: false, expected: 'compare' },
])(
'resolveGoldenAction($update,$exists,$isCI) -> $expected',
({ update, exists, isCI, expected }) => {
expect(resolveGoldenAction({ update, exists, isCI })).toBe(expected);
},
);
it('produces a deterministic digest across repeated runs', () => {
const src = generateDao(8);
expect(digestCaptures(emitGoScopeCaptures(src, 'a.go'))).toBe(
digestCaptures(emitGoScopeCaptures(src, 'a.go')),
);
});
it('digest is sensitive to capture-match emission order', () => {
const matches = emitGoScopeCaptures(generateDao(6), 'a.go');
expect(matches.length).toBeGreaterThan(1);
const reversed = [...matches].reverse();
// Reordering the emission changes the digest — the true byte-identical guard.
expect(digestCaptures(reversed)).not.toBe(digestCaptures(matches));
});
it('records a capture-group count for every fixture and the DAO shape', () => {
const snapshot = buildSnapshot();
const fixtureKeys = collectGoFixtures().map((f) => f.key);
// Every collected fixture is present in the snapshot.
for (const k of fixtureKeys) expect(snapshot[k]).toBeDefined();
// The DAO shape (which has symbols) yields a non-empty capture set.
expect(snapshot['synthetic:dao-20']!.captureGroups).toBeGreaterThan(0);
});
});

View file

@ -62,4 +62,104 @@ func main() {
expect(tags).toContain('@reference.read');
expect(tags).toContain('@reference.write');
});
// ── Edge shapes the #1915 captured-node refactor reasons about but no
// lang-resolution fixture exercises (issue #1848 follow-up U2). ──
it('synthesizes a receiver for a method but not for a func_literal scope', () => {
// Source has BOTH a real method and a closure. A weak "no @type-binding.self
// anywhere" assertion would pass even if the method_declaration receiver
// branch regressed (a closure-only fixture has none to lose); asserting the
// method's receiver IS present catches that regression.
const src = `
package main
type User struct{ Name string }
func (u *User) Save() { _ = u.Name }
func main() {
f := func() int { return 1 }
_ = f()
}
`;
const matches = emitGoScopeCaptures(src, 'main.go');
// The closure is still captured as a @scope.function...
expect(matches.some((m) => m['@scope.function']?.text.startsWith('func()'))).toBe(true);
// ...and the method's receiver self-binding is synthesized (name + pointer-stripped type)...
const selves = matches.filter((m) => m['@type-binding.self'] !== undefined);
expect(selves).toHaveLength(1); // exactly one — from the method, not the closure
expect(selves[0]!['@type-binding.name']?.text).toBe('u');
expect(selves[0]!['@type-binding.type']?.text).toBe('User');
});
it('does not drop a var-form type assertion binding', () => {
// `var x int = e.(T)` anchors on a var_declaration, not a short_var_declaration,
// so isRawMultiAssignTypeBinding must NOT filter it (old findNodeAtRange path
// returned null -> false; the new anchor.type guard reproduces that).
const src = `
package main
func main() {
var x int = any(1).(int)
_ = x
}
`;
const assertion = emitGoScopeCaptures(src, 'main.go').find(
(m) => m['@type-binding.assertion'] !== undefined,
);
expect(assertion).toBeDefined();
expect(assertion!['@type-binding.name']?.text).toBe('x');
});
it('does not drop a var-form call-return binding', () => {
const src = `
package main
func NewThing() int { return 1 }
func main() {
var y = NewThing()
_ = y
}
`;
const callReturn = emitGoScopeCaptures(src, 'main.go').find(
(m) => m['@type-binding.call-return'] !== undefined,
);
expect(callReturn).toBeDefined();
expect(callReturn!['@type-binding.name']?.text).toBe('y');
});
it('resolves a single unparenthesized import the same as a grouped one', () => {
// Exercises resolveImportNode's no-import_spec_list parent chain. NOTE: not
// redundant with go-imports.test.ts, which calls splitGoImportStatement
// directly and bypasses captures.ts / resolveImportNode.
const single = emitGoScopeCaptures(
`
package main
import "fmt"
func main() { fmt.Println() }
`,
'main.go',
);
const sources = single
.filter((m) => m['@import.source'] !== undefined)
.map((m) => m['@import.source']!.text);
expect(sources).toEqual(['fmt']);
});
it('captures a generic function declaration', () => {
const src = `
package main
func Map[T any](x T) T { return x }
`;
const decl = emitGoScopeCaptures(src, 'main.go').find(
(m) => m['@declaration.function'] !== undefined,
);
expect(decl).toBeDefined();
expect(decl!['@declaration.name']?.text).toBe('Map');
});
});