GitNexus/gitnexus/test/unit/csharp-namespace-extraction.test.ts
Gergő Magyar 4b787be835
fix(csharp): stop spurious IMPORTS edges from ungated using-resolution (#1881) (#1908)
* fix(csharp): eliminate O(S·D) BindingRef OOM in namespace siblings

Types declared in the C# global (default) namespace are visible from
every file, so the previous per-scope augmentation materialized
O(scopes × defs) BindingRefs — on large Unity solutions (tens of
thousands of global types) this caused severe slowness and OOM.

Route global-namespace types through a single workspace-level binding
channel (workspaceFqnBindings, consulted by lookupBindingsAt) for O(D)
memory. Also fix quadratic costs in the non-global path: append defs in
place instead of copying (was O(D²) per bucket), pre-index the first
scope per file (was O(S²·D)), and seed de-dup sets instead of repeated
.some scans.

Add csharp-pipeline-benchmark.test.ts (mirrors the PHP benchmark) with
spread and concentrated-global-namespace scenarios to track elapsedMs,
peakHeapMB, nodeCount, and edgeCount. Post-fix runs show linear scaling
and stable heap.

Co-authored-by: Cursor <cursoragent@cursor.com>

* perf(csharp): scanner fallback for namespace siblings on the worker path

Worker threads can't return tree-sitter Trees across MessageChannels, so
the cross-phase tree cache is empty for worker-parsed files. The C#
same-namespace pass (populateCsharpNamespaceSiblings -> extractFileStructure)
then re-parsed every file with tree-sitter to find namespace / using-static
nodes — effectively parsing a large solution a second time during scope
resolution.

Add a line-scanner fallback (extractCsharpStructureViaScanner) used only
when no cached Tree is available, mirroring PHP's fix for issue #1741. It
extracts the same namespaces / usingStaticPaths the AST walk produces for
the common line-anchored forms (file-scoped + block namespaces, plain /
global / aliased `using static`). The AST walk stays authoritative on the
sequential / warm-cache path.

Micro-benchmark over 3000 synthetic files: scanner is ~188x faster than
parse+walk (0.001 vs 0.251 ms/file) with identical output on the parity
spot-check; real-world files are larger, so the worker-path saving is
bigger. Adds csharp-namespace-extraction.test.ts (12 cases) covering all
declaration forms plus negative cases (using var, plain using, comments).

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore(autofix): apply prettier + eslint fixes via /autofix command

* fix(csharp): cover global-namespace workspaceFqnBindings path + doc + using-static perf

Addresses the production-readiness review of the namespace-siblings OOM fix.

- Add a unit test proving global-(default-)namespace C# types route to
  indexes.workspaceFqnBindings (one entry per simple name) with ZERO
  bindingAugmentations — pinning the O(D) invariant behind the #1871
  Unity-scale OOM fix and guarding against a revert to per-scope
  O(scopes x defs) augmentation. (The csharp-hooks mock now supplies
  workspaceFqnBindings, which the global fast path reads directly.)
- Correct the workspaceFqnBindings doc comment: it is shared by PHP
  (backslash-FQN keys) and C# (global-namespace simple-name keys); the two
  key formats are disjoint.
- Pre-index parsedFiles by path before the `using static` member-injection
  loop, replacing an O(files) find-per-import with an O(1) Map lookup.

Verified: tsc --noEmit clean; csharp-hooks + csharp-namespace-extraction
suites pass (38 tests); prettier clean; eslint 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(csharp): apply PR-review polish to namespace-siblings (tests, types, docs)

Addresses the multi-agent code review of this PR — the concrete, defensible
findings. Two items intentionally deferred (below).

- namespace-siblings.ts: couple the augmentation bucket + its de-dup set into
  one nullable lifecycle, removing the seen!/bucketArr! non-null assertions
  (identical runtime, still lazy).
- validate-bindings-immutability.ts: extend the dev-mode immutability validator
  to the third channel (workspaceFqnBindings) + a test; complete the validator
  test mock with workspaceFqnBindings.
- walkers.ts: document that namesAtScope deliberately excludes the
  scope-independent workspaceFqnBindings channel (enumerating workspace names at
  every scope would flood per-scope callers; lookupBindingsAt still consults it
  when resolving a specific name).
- scope-resolution-indexes.ts: reframe the workspaceFqnBindings doc to describe
  the key-format contract language-neutrally (examples, not language branching).
- csharp-hooks.test.ts: assert workspace entries carry origin:'namespace'; add a
  partial-class test (same simple name, distinct nodeIds across global files →
  both kept); rename the stale "parses" cache-miss test to "scans".
- csharp-pipeline-benchmark.test.ts: clearTimeout the Promise.race budget timer
  (dangling handle when the pipeline won the race).
- csharp.test.ts: correct the #1066 comment — extractFileStructure no longer
  re-parses on cache miss (line scanner); only emitCsharpScopeCaptures re-parses.

Deferred (surfaced, not applied): (1) worker-path scanner mis-reads
namespace/using-static inside block comments and verbatim/raw strings — an
explicitly documented trade-off mirroring the PHP scanner; hardening it to track
comment/string state is a separate decision. (2) workspaceFqnBindings is read
via an `as Map` cast; a type-safe mutable handle from finalize-orchestrator is a
cross-module contract change.

Verified: tsc --noEmit clean; 49 unit tests pass (incl. 3 new); prettier clean;
eslint 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(csharp): harden worker-path scanner + localize workspace-map cast

Addresses the two deferred PR-review findings plus the remaining test gap.

#1 — Worker-path scanner false positives: the line scanner now tracks block-
comment and string state across lines (advanceCsScanState), so a `namespace` /
`using static` keyword at the start of a line inside a block comment, verbatim
string (@"..."), or raw string literal ("""...""") is no longer mistaken for a
declaration on the worker cache-miss path. It matches only at code-state line
starts. 5 new scanner tests cover the block-comment / raw / verbatim cases.

#4 — workspaceFqnBindings type safety: the ReadonlyMap->Map cast is localized
to one documented line, and global-namespace writes go through a new
getWorkspaceBucket helper (mirroring getAugmentationBucket) rather than an
inline `.set()` at the mutation site.

#2 — lookupBindingsAt workspace-channel coverage: walkers-augmentations.test.ts
now exercises the third (workspace) channel: workspace-only, append-after-
finalized/augmented, and dedup-loses-to-finalized/augmented precedence.

#5 — OOM CI guard: the deterministic O(D) invariant (zero per-scope
augmentation for global types) is already asserted by the always-on
csharp-hooks unit tests added earlier; the scale/time benchmark stays
appropriately opt-in (skipIf).

Verified: tsc --noEmit clean; 69 unit tests (4 suites) + 210 C# integration
resolver tests pass; prettier clean; eslint 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(csharp): replace remaining O(A) .some dedup scans with seeded Sets

The using-static member-injection loop and the cross-namespace import loop both
de-duped via `bucketArr.some((b) => b.def.nodeId === ...)` — O(A) per item. Both
now use a per-file `Map<simpleName, Set<nodeId>>`, seeded lazily from the
augmentation bucket (capturing entries from earlier passes), matching the
global and named-namespace paths. Same dedup semantics, O(1) amortized.

Verified: tsc --noEmit clean; csharp-hooks unit (27) + C# integration resolver
(210) tests pass; prettier + eslint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(csharp): gate suffix-fallback import resolution to declared namespaces (#1881)

C# `using` directives were resolving via an ungated suffix match, so a BCL
using like `System.Threading.Tasks` matched a coincidental local `Tasks.cs`
and emitted spurious IMPORTS edges. Add a declared-namespace gate that only
permits suffix-fallback when the import plausibly refers to an in-repo
namespace (exact, immediate-parent-declared, or ancestor-of a declared
namespace anchored at an in-repo root). Both resolution legs — the legacy
DAG and the registry-primary scope resolver — thread the same evidence to
the gate, including the no-csproj path.

Declared namespaces are collected with #1905's comment/string-aware scanner
(extractCsharpStructureViaScanner, lazily imported) instead of a regex, so
`namespace` tokens in comments/strings can't seed phantom namespaces. Scan
truncation or unreadable subtrees fail OPEN (gate disabled) and are logged.

Stacked on #1905 (fix/csharp-namespace-scope-oom).

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(csharp): cap per-file size in namespace scan; fail open on skip (#1881)

scanCSharpProject read every .cs/.csproj in full with no size guard and
issued per-directory reads with no concurrency bound, an OOM/FD-exhaustion
vector on large or generated repos. Add an fs.stat size guard before each
read, reusing getMaxFileSizeBytes() (the same 512KB cap the Phase-1 walker
uses). An oversized or unreadable .cs now signals truncation so the #1881
suffix-fallback gate fails OPEN rather than wrongly suppressing an import
whose declaring namespace lived in the skipped file (previously a silent
return left the scan looking complete). Adds a size-cap scan test.

* fix(csharp): bound per-directory read concurrency in namespace scan (#1881)

The scan issued every .cs/.csproj read in a directory at once via
Promise.all, so in-flight file descriptors scaled with the largest
directory's file count. Issue reads in bounded windows (32, mirroring
the Phase-1 filesystem-walker) via Promise.allSettled; an unexpected
read/scan rejection now trips truncation (fail open) instead of
rejecting the whole scan. Behavior-preserving for namespace collection
(C# scope-resolution parity passes on both legs).

* style(csharp): apply prettier to #1881 files to clear quality/format gate (#1908)

Reflow hand-wrapped lines in scope-resolver.ts and the csharp integration
test that prettier collapses under printWidth 100. Formatting only, no
behavioral change; clears the failing quality/format CI gate.

* fix(csharp): stream namespace scan so large generated files don't disable the #1881 gate (#1908)

Code-review follow-up. The scan read each .cs fully into a string behind a
512KB size cap (the tree-sitter parse budget); a single larger generated file
(*.g.cs, EF/gRPC output) tripped `truncated`, making the #1881 suffix-fallback
gate fail open repo-wide and silently undoing the fix on real repos.

Stream each .cs line-by-line via createReadStream + readline into a new
incremental scanner (createCsharpStructureScanner) instead of buffering the
whole file. Memory is now constant regardless of file size, so the per-file
size cap is dropped for the namespace line-scan and large generated files are
fully collected. extractCsharpStructureViaScanner is reimplemented on the same
incremental scanner (byte-identical; C# parity 2/2). collectDeclaredNamespaces
returns 'ok' | 'truncated' (truncation now only from an unreadable file) and the
truncation warn lists its real causes. csproj reads keep their size guard.

Prior art: ripgrep/ctags/Node readline stream rather than cap for line scans;
GitHub (384KB) and Sourcegraph (1MB) cap only their full-content indexes.

* fix(csharp): cap .csproj read via stream, not stat-then-read, to clear CodeQL TOCTOU (#1908)

CodeQL js/file-system-race flagged the fs.stat + fs.readFile size guard in
readCsprojConfig as a check-then-use filesystem race. Replace it with a
length-capped createReadStream (readFileTextCapped) — same memory bound on
untrusted input, no stat-then-read race, and consistent with the streamed
.cs scan. Behavior is unchanged for real .csproj files (parity 2/2).

* fix(csharp): keep BCL/external roots gated through scan truncation (#1908, Codex F1)

A single scan truncation (unreadable dir/file, depth/dir cap) set one
repo-wide `truncated` flag that made csharpSuffixFallbackAllowed fail
open for EVERY import, silently re-enabling the #1881 BCL->local suffix
matches. Add a CSHARP_EXTERNAL_ROOTS denylist (System/Microsoft/...): an
external-rooted using that does not align with an in-repo declared
namespace stays BLOCKED even under truncation, while genuinely
local-looking usings still fail open. A repo that declares the root is
allowed via the alignment escape hatch. Shared predicate, so both legs
inherit it.

* fix(csharp): gate the registry no-csproj direct-match path (#1908, Codex F2)

In the no-csproj branch of resolveCsharpImportTarget, resolveDirectMatch
ran BEFORE the gate, so a path-aligned Legacy/System/Threading/Tasks.cs
satisfied 'using System.Threading.Tasks;' even though System.* is not a
declared in-repo namespace — while the legacy leg (gate-first) blocked
it, so the legs were not equivalent. Run csharpSuffixFallbackAllowed
first (return null on fail), then direct-match, then progressive
stripping — mirroring the legacy ordering. Adds a no-csproj fixture with
a deep path-aligned Tasks.cs and dual-leg integration describes (registry
+ forced-legacy), plus a path-aligned unit case. Parity 2/2.

* fix(csharp): flag scanner-uncaptured namespaces incomplete; Unicode/@ matchers (#1908, Codex F3)

The line scanner treated its output as complete even when it missed valid
C# namespace forms, so the gate failed CLOSED and over-blocked legit
imports. Make CS_NAMESPACE_RE/CS_USING_STATIC_RE Unicode-aware (\p{L}\p{N}
+ u flag) and strip leading/segment @ so verbatim/Unicode identifiers are
captured to match the AST. For forms the regex still can't capture (split
across lines, not at line start, attributed), set a per-file 'incomplete'
flag; collectDeclaredNamespaces returns 'truncated' for such files so the
#1881 gate fails OPEN instead of dropping the namespace. High-precision
detectors + guard tests keep ordinary forms (incl. // namespace comments)
from tripping incomplete.

* fix(csharp): stream the .csproj RootNamespace read, no byte cap (#1908, Codex F4)

readCsprojConfig read only the first 512KB of a .csproj and, on a
match-miss, couldn't tell 'no RootNamespace' from 'RootNamespace past
the cap' — both synthesized a filename root. A wrong authoritative root
makes imports under the real root resolve to nothing AND suppresses the
fallback. Replace the capped read with a streamed early-stop search
(findCsprojRootNamespace) that reads until the tag or EOF: filename
fallback ONLY on genuine read-to-EOF absence; on a soft-budget cap-hit or
unreadable file, OMIT the config so the no-csproj fallback stays
reachable. Removes the now-unused readFileTextCapped + getMaxFileSizeBytes
cap from the scan. Parity 2/2.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 09:56:26 +01:00

179 lines
7.7 KiB
TypeScript

import { describe, it, expect } from 'vitest';
import { extractCsharpStructureViaScanner } from '../../src/core/ingestion/languages/csharp/namespace-siblings.js';
// Scanner fallback used on the worker path, where native tree-sitter Trees
// can't cross MessageChannels so `treeCache` is empty. It must reproduce
// the AST walk's `namespaces` / `usingStaticPaths` for the common
// line-anchored declaration forms (see namespace-siblings.ts).
describe('extractCsharpStructureViaScanner', () => {
it('extracts a file-scoped namespace declaration', () => {
const src = `namespace App.Models;\n\npublic class User {}`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Models']);
});
it('extracts a block namespace declaration', () => {
const src = `namespace App.Services\n{\n public class Svc {}\n}`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Services']);
});
it('extracts multiple namespaces in source order', () => {
const src = `namespace A.One\n{\n}\nnamespace A.Two\n{\n}`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['A.One', 'A.Two']);
});
it('returns empty namespaces for a global (no-namespace) file', () => {
const src = `public class Global {}\n`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual([]);
});
it('captures a plain `using static` path', () => {
const src = `using static System.Math;\nnamespace App;`;
const out = extractCsharpStructureViaScanner(src);
expect(out.usingStaticPaths).toEqual(['System.Math']);
expect(out.namespaces).toEqual(['App']);
});
it('captures a `global using static` path', () => {
const src = `global using static App.Utils.Logger;\n`;
expect(extractCsharpStructureViaScanner(src).usingStaticPaths).toEqual(['App.Utils.Logger']);
});
it('captures the RHS path of an aliased `using static`', () => {
const src = `using static M = App.Utils.MathUtils;\n`;
expect(extractCsharpStructureViaScanner(src).usingStaticPaths).toEqual(['App.Utils.MathUtils']);
});
it('does not treat a plain `using` directive as using-static', () => {
const src = `using System.Collections.Generic;\nusing App.Models;\n`;
expect(extractCsharpStructureViaScanner(src).usingStaticPaths).toEqual([]);
});
it('does not treat a `using var`/`using (...)` statement as using-static', () => {
const src = `using var stream = File.Open(p);\nusing (var x = Get()) { }\n`;
expect(extractCsharpStructureViaScanner(src).usingStaticPaths).toEqual([]);
});
it('ignores a `// namespace X` line comment', () => {
const src = `// namespace Fake.Comment;\nnamespace App.Real;`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Real']);
});
it('handles indentation before declarations', () => {
const src = `\t\tnamespace App.Indented;\n`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Indented']);
});
it('handles an empty file', () => {
const out = extractCsharpStructureViaScanner('');
expect(out.namespaces).toEqual([]);
expect(out.usingStaticPaths).toEqual([]);
});
// Cross-line comment/string state: a keyword at the start of a line inside
// a block comment or multi-line string must NOT be read as a declaration
// (the worker path would otherwise mis-bucket the file vs the AST).
it('skips a `namespace` line inside a block comment', () => {
const src = `/*\nnamespace Fake.InComment;\n*/\nnamespace App.Real;`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Real']);
});
it('skips a `using static` line inside a block comment', () => {
const src = `/*\nusing static Fake.Helpers;\n*/\nusing static App.Real.Helpers;`;
expect(extractCsharpStructureViaScanner(src).usingStaticPaths).toEqual(['App.Real.Helpers']);
});
it('skips a `namespace` line inside a raw string literal', () => {
const src = `var sql = """\nnamespace Fake.InRaw;\n""";\nnamespace App.Real;`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Real']);
});
it('skips a `namespace` line inside a verbatim string literal', () => {
const src = `var s = @"\nnamespace Fake.InVerbatim;\n";\nnamespace App.Real;`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Real']);
});
it('still reads a real declaration after a closed same-line block comment', () => {
const src = `/* header */ class C {}\nnamespace App.Real;`;
expect(extractCsharpStructureViaScanner(src).namespaces).toEqual(['App.Real']);
});
// --- Unicode / @-verbatim identifiers (Codex F3): these must be CAPTURED, not
// truncated/dropped, so the #1881 gate doesn't over-block legitimate imports.
it('captures a Unicode namespace identifier', () => {
const out = extractCsharpStructureViaScanner('namespace Café.Modèles;');
expect(out.namespaces).toEqual(['Café.Modèles']);
expect(out.incomplete).toBeFalsy();
});
it('captures a non-Latin (Greek) namespace identifier', () => {
expect(extractCsharpStructureViaScanner('namespace Ωμέγα.Models;').namespaces).toEqual([
'Ωμέγα.Models',
]);
});
it('strips a leading @ from a verbatim namespace identifier to match the AST', () => {
expect(extractCsharpStructureViaScanner('namespace @namespace.Models;').namespaces).toEqual([
'namespace.Models',
]);
});
it('strips a mid-path @ from a verbatim namespace segment', () => {
expect(extractCsharpStructureViaScanner('namespace App.@class.Models;').namespaces).toEqual([
'App.class.Models',
]);
});
// --- Forms the line scanner cannot capture must flag `incomplete` so the
// caller fails the #1881 gate OPEN (Codex F3) instead of dropping the namespace.
it('flags `incomplete` for a namespace declaration split across lines', () => {
const out = extractCsharpStructureViaScanner('namespace\n App.Models;');
expect(out.namespaces).toEqual([]);
expect(out.incomplete).toBe(true);
});
it('flags `incomplete` for a namespace keyword not at line start', () => {
const out = extractCsharpStructureViaScanner('class C {} namespace App.Models;');
expect(out.incomplete).toBe(true);
});
it('flags `incomplete` for an attributed same-line namespace', () => {
const out = extractCsharpStructureViaScanner('[Obsolete] namespace App.Legacy;');
expect(out.incomplete).toBe(true);
});
// --- Guards: ordinary / handled forms must NEVER set `incomplete`, or one
// exotic line would wrongly disable the gate repo-wide.
it('does NOT flag `incomplete` for ordinary handled forms', () => {
for (const src of [
'namespace App.Models;',
'namespace App.Services\n{\n}',
'namespace A.One {}\nnamespace A.Two {}',
'using static System.Math;\nnamespace App;',
'global using static App.Utils.Logger;',
'using static M = App.Utils.MathUtils;',
'using System.Collections.Generic;\nusing App.Models;',
'\t\tnamespace App.Indented;',
'public class Global {}',
'',
]) {
expect(extractCsharpStructureViaScanner(src).incomplete).toBeFalsy();
}
});
it('does NOT flag `incomplete` for a `// namespace` line comment or a namespace mentioned after `//`', () => {
expect(
extractCsharpStructureViaScanner('// namespace Fake.Comment;\nnamespace App.Real;')
.incomplete,
).toBeFalsy();
expect(
extractCsharpStructureViaScanner('public class C {} // namespace Foo').incomplete,
).toBeFalsy();
});
it('does NOT flag `incomplete` for an identifier that merely starts with "namespace"', () => {
// `namespaceManager` is an ordinary identifier, not the keyword.
expect(
extractCsharpStructureViaScanner('var namespaceManager = Get();').incomplete,
).toBeFalsy();
});
});