GitNexus/gitnexus/test/unit/ingestion-utils.test.ts
Gergő Magyar 98ee665889
Some checks are pending
CI / quality (push) Waiting to run
CI / tests (push) Waiting to run
CI / e2e (push) Waiting to run
CI / scope-parity (push) Waiting to run
CI / Save PR Metadata (push) Blocked by required conditions
CI / CI Gate (push) Blocked by required conditions
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
Release Candidate / Build & Push RC Docker images (push) Blocked by required conditions
fix(ingestion): two-channel binding lifecycle (closes #1066) + scope-resolution I8 hardening (#1082)
* fix(csharp): adaptive tree-sitter buffer + frozen-bucket clone for cross-namespace siblings (#1066)

Two coupled regressions surfaced when analyzing real-world C# repos with
large source files (issue #1066):

1. Tree-sitter `parser.parse()` is hard-coded to a 32 KB buffer by
   default. Any file exceeding that threshold throws `Invalid argument`
   on the worker re-parse path of `populateCsharpNamespaceSiblings`
   (and the analogous Python / TypeScript captures fallbacks).
2. After the buffer fix unblocks the AST walk, the hook tries to
   `push()` onto the inner `BindingRef[]` array fetched from
   `indexes.bindings` — but `materializeBindings` froze that array via
   `Object.freeze(refs.slice())`. Result: `Cannot add property N,
   object is not extensible`.

Fixes:

- `csharp/captures.ts`, `python/captures.ts`, `typescript/captures.ts`:
  pass `bufferSize: getTreeSitterBufferSize(sourceText.length)` to
  `parser.parse()` on the cache-miss path so multi-MB files parse.
- `csharp/namespace-siblings.ts`: introduce `cloneBindingBucket` to
  copy the frozen array before mutating, then `set()` the new array
  back. This is a working but architecturally compromised workaround
  (#1050 follow-up will replace it with an explicit augmentation
  channel — see docs/plans/2026-04-26-001 plan).

Tests:

- New `csharp-large-cache-miss-resolution` fixture (Models/Services/
  Other layout, ~77 KB padded UserService.cs) drives the buffer-size
  failure end-to-end through worker mode.
- `csharp.test.ts`: 4 new regression assertions covering both the
  parse-time buffer-size failure and the freeze workaround.
- Per-language captures unit tests gain "large cache-miss file uses
  adaptive buffer" coverage (TS, Python, C#).
- `csharp-hooks.test.ts`: in-memory freeze regression test that
  reproduces the `Cannot add property` crash without invoking the C#
  parser at all.

Made-with: Cursor

* refactor(scope-resolution): add bindingAugmentations channel to indexes

Step 1 of the binding-augmentation-channel refactor (issue #1066
follow-up). Pure shape change — no consumers yet.

Adds a new `readonly bindingAugmentations` field to
`ScopeResolutionIndexes` initialized as an empty `Map` by
`finalizeScopeModel`. The new channel is the dedicated post-finalize
write target for hooks like `populateCsharpNamespaceSiblings`, so
`indexes.bindings` can stay frozen and finalize-owned.

Behavior unchanged: nothing reads or writes the new field yet. tsc and
the full unit suite remain green.

Plan: docs/plans/2026-04-26-001-binding-augmentation-channel.md (local
only — `docs/plans/` is gitignored).

Made-with: Cursor

* feat(scope-resolution): add lookupBindingsAt dual-source helper

Step 2 of the binding-augmentation-channel refactor. Introduces a
single primitive every walker uses to read both the finalize-owned
`indexes.bindings` channel and the post-finalize
`indexes.bindingAugmentations` channel.

Contract:
- Finalized refs come first (preserves existing precedence).
- Augmented refs append, deduped by `def.nodeId`.
- Empty input on both channels returns a shared frozen empty array.
- Single-channel hits return the bucket by reference (no allocation).

No consumers are wired yet — Step 3 routes the existing walker
primitives through this helper. Augmentations remain empty for every
language; behavior of the full suite is unchanged.

8 unit tests pin precedence, dedup, identity for single-channel hits,
and the shared-empty-frozen-array sentinel.

Made-with: Cursor

* refactor(scope-resolution): route binding lookups through lookupBindingsAt

Step 3 of the binding-augmentation-channel refactor. Every direct
`indexes.bindings.get(...)` consumer in the post-finalize phase is
now routed through `lookupBindingsAt` (per-name) or `namesAtScope`
+ `lookupBindingsAt` (bulk iteration).

Routed sites:
- `findClassBindingInScope` (walkers.ts) — class-receiver lookups.
- `findCallableBindingInScope` (walkers.ts) — free-call lookups.
- `findExportedDefByName` (walkers.ts) — module-scope-fallback
  callable lookups.
- `propagateImportedReturnTypes` (passes/imported-return-types.ts)
  — bulk iteration over an importer's binding entries; switched to
  `namesAtScope` + per-name `lookupBindingsAt` so post-finalize
  augmentations are visible to import-derived typeBinding mirrors.

Behavior unchanged: augmentations are empty across the suite (Step 4
populates them for C# `populateNamespaceSiblings`). 587
scope-resolution unit tests + 50 integration resolver suites green
(4 pre-existing Swift method-implements failures unrelated to this
work).

Adds `namesAtScope` companion helper for the bulk-iteration callers.

Made-with: Cursor

* refactor(csharp): write namespace siblings to bindingAugmentations channel

Step 4 of the binding-augmentation-channel refactor. The C#
`populateNamespaceSiblings` hook is the only consumer that needed
to inject cross-file bindings post-finalize, and prior to this
change it cloned the (frozen) finalized `BindingRef[]` arrays
through a `cloneBindingBucket` helper, then `set()`-back the new
array — a workaround for the `Object.freeze` applied by
`finalize-algorithm.ts` (issue #1066 root cause).

Architecturally that violated `ScopeResolver` Invariant I8 (which
permits post-finalize modifications but not in-place mutation of
finalized buckets). It also forced read-side consumers to be aware
of the workaround.

This change:
* Switches the three C# write sites to append into
  `indexes.bindingAugmentations` via `getAugmentationBucket`. The
  augmentation channel was added in Step 1 and is mutable by
  contract: inner `BindingRef[]` arrays here are NEVER frozen.
* Deletes `cloneBindingBucket` and `getMutableScopeBindings`
  (workaround helpers no longer needed).
* `lookupBindingsAt` (Step 2) merges the two channels transparently
  for every walker (Step 3), so behavior is unchanged for callers.
* Updates the unit test to assert against both channels: finalized
  bucket stays frozen and untouched, cross-file siblings show up in
  augmentations only. Renamed the test accordingly.

Validation:
* `npx tsc --noEmit` clean.
* csharp hooks unit + walkers-augmentations unit + csharp integration
  resolver suite all green (236/236).
* Wider `test/unit/scope-resolution test/integration/resolvers`
  suite: 2507 pass, only 4 pre-existing Swift METHOD_IMPLEMENTS
  failures remain (unrelated to this work, present on baseline).

Refs: issue #1066, ADR-pending binding-augmentation-channel.
Made-with: Cursor

* feat(scope-resolution): tighten I8 + add validateBindingsImmutability dev guard

Step 5 of the binding-augmentation-channel refactor. Captures the
new two-channel binding lifecycle in the contract docs and adds a
dev-mode runtime validator so a future hook cannot silently drift
back into mutating `indexes.bindings`.

Contract changes:
* `contract/scope-resolver.ts` — rewrote Invariant I8 to describe
  the two channels (`indexes.bindings` is finalize-output and
  immutable post-finalize; `indexes.bindingAugmentations` is the
  append-only post-finalize channel populated by hooks like
  `populateNamespaceSiblings`). Documented `lookupBindingsAt` as
  the read-side merger and pointed at the new validator as the
  enforcement mechanism.
* `gitnexus-shared/src/scope-resolution/types.ts` — extended the
  module-header lifecycle contract to call out
  `bindingAugmentations` alongside `ReferenceIndex` as the two
  structures populated after the freeze.

Validator:
* New `pipeline/validate-bindings-immutability.ts` mirrors the
  shape of `validateOwnershipParity` (#909): runs only when
  `NODE_ENV !== 'production' && VALIDATE_SEMANTIC_MODEL !== '0'`,
  emits via `onWarn`, never throws. Asserts (a) every inner
  `BindingRef[]` in `indexes.bindings` is `Object.isFrozen`, and
  (b) every inner array in `indexes.bindingAugmentations` is NOT
  frozen.
* Wired into `pipeline/run.ts` after both
  `populateNamespaceSiblings` and `propagateImportedReturnTypes`,
  before `resolveReferenceSites`. One sweep covers the full
  post-finalize surface.

Tests:
* `validate-bindings-immutability.test.ts` — 6 cases pinning happy
  path, both drift directions, multi-violation accumulation, and
  both production no-op gates.

All scope-resolution + csharp resolver tests green (242/242 in the
focused run; matches the wider Step 4 baseline).

Made-with: Cursor

* fix(ingestion): size tree-sitter buffers from UTF-8 bytes

Tree-sitter buffer sizing is byte-based, so computing adaptive buffers from JavaScript string length under-sized UTF-8-heavy files. Make getTreeSitterBufferSize accept source text directly and compute Buffer.byteLength internally, then update all parse call sites and max-buffer skip checks to use byte length.

Add multibyte cache-miss and cap regressions for C#, Python, TypeScript, and the C# namespace-sibling fallback parse path.

Made-with: Cursor

* test(scope-resolution): pin augmentation read paths

Add focused unit coverage for augmented-only binding reads across the routed walker helpers and imported-return-type propagation path. Clarify I8 wording around lexical Scope.bindings versus post-finalize index channels, and document the intentional local-only behavior of findExportedDef.

Also switch the immutability validator tests to Vitest env stubs, document one intentional validator blind spot, and split C# namespace-sibling tests so UTF-8 parsing and augmentation-channel behavior are asserted independently.

Made-with: Cursor

* test(scope-resolution): avoid slow parser stress fixtures

Replace high-cardinality large-file capture fixtures with large padding plus a trailing declaration. This still proves adaptive tree-sitter buffers parse beyond large ASCII and UTF-8-heavy input, without making query matching process thousands of declarations and risking timeouts.

Made-with: Cursor

* test(scope-resolution): add python and typescript cache-miss resolver regressions

Add worker-mode resolver integration coverage mirroring the C# #1066 scenario for Python and TypeScript. Each test builds a temp fixture with large ASCII and UTF-8-heavy source padding, then asserts trailing declarations and call edges still resolve after scope-resolution cache-miss reparsing.

Made-with: Cursor

* refactor(scope-resolution): gate I8 validator and fast-path namesAtScope

Addresses SPARC reviewer feedback on the binding-augmentation channel:

- Validator gate is now opt-in outside development. Extract
  isSemanticModelValidatorEnabled() in utils/env.ts as the single
  predicate; both validateBindingsImmutability and phase.ts's warn
  handler share it. Default CLI runs no longer pay the O(binding-buckets)
  scan, and explicit VALIDATE_SEMANTIC_MODEL=1 now emits warnings even
  when NODE_ENV is unset.
- namesAtScope returns Iterable<string> and zero-allocates when at most
  one channel is populated (returns Map.keys() directly), only
  materializing a Set when both channels carry names. The caller-side
  branching and EMPTY_NAMES escape hatch in propagateImportedReturnTypes
  are gone -- both helpers handle the empty-augmentation case internally.
- C# namespace-siblings header/JSDoc, model JSDoc, I8 contract prose, and
  the #1066 integration-test header rewritten to say post-finalize fanout
  appends only to bindingAugmentations; finalized refs come first and win
  duplicate def.nodeId metadata; local lexical Scope.bindings remains the
  first-tier shadowing channel.

Validator unit-test setup deduplicated via beforeEach and extended with
default-CLI no-op + explicit-opt-in cases.

Made-with: Cursor
2026-04-26 12:16:09 +01:00

707 lines
25 KiB
TypeScript

import { describe, it, expect } from 'vitest';
import { getLanguageFromFilename, SupportedLanguages } from 'gitnexus-shared';
import { getProvider } from '../../src/core/ingestion/languages/index.js';
import type { SyntaxNode } from '../../src/core/ingestion/utils/ast-helpers.js';
import type { NodeLabel } from 'gitnexus-shared';
import type { LanguageProvider } from '../../src/core/ingestion/language-provider.js';
import {
getTreeSitterBufferSize,
getTreeSitterContentByteLength,
TREE_SITTER_BUFFER_SIZE,
TREE_SITTER_MAX_BUFFER,
} from '../../src/core/ingestion/constants.js';
import Parser from 'tree-sitter';
import C from 'tree-sitter-c';
import CPP from 'tree-sitter-cpp';
import Python from 'tree-sitter-python';
import TypeScript from 'tree-sitter-typescript';
describe('getLanguageFromFilename', () => {
describe('TypeScript', () => {
it('detects .ts files', () => {
expect(getLanguageFromFilename('index.ts')).toBe(SupportedLanguages.TypeScript);
});
it('detects .tsx files', () => {
expect(getLanguageFromFilename('Component.tsx')).toBe(SupportedLanguages.TypeScript);
});
it('detects .ts files in paths', () => {
expect(getLanguageFromFilename('src/core/utils.ts')).toBe(SupportedLanguages.TypeScript);
});
});
describe('JavaScript', () => {
it('detects .js files', () => {
expect(getLanguageFromFilename('index.js')).toBe(SupportedLanguages.JavaScript);
});
it('detects .jsx files', () => {
expect(getLanguageFromFilename('App.jsx')).toBe(SupportedLanguages.JavaScript);
});
});
describe('Python', () => {
it('detects .py files', () => {
expect(getLanguageFromFilename('main.py')).toBe(SupportedLanguages.Python);
});
});
describe('Java', () => {
it('detects .java files', () => {
expect(getLanguageFromFilename('Main.java')).toBe(SupportedLanguages.Java);
});
});
describe('C', () => {
it('detects .c files', () => {
expect(getLanguageFromFilename('main.c')).toBe(SupportedLanguages.C);
});
});
describe('C++', () => {
it.each(['.cpp', '.cc', '.cxx', '.h', '.hpp', '.hxx', '.hh'])('detects %s files', (ext) => {
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.CPlusPlus);
});
});
describe('C#', () => {
it('detects .cs files', () => {
expect(getLanguageFromFilename('Program.cs')).toBe(SupportedLanguages.CSharp);
});
});
describe('Go', () => {
it('detects .go files', () => {
expect(getLanguageFromFilename('main.go')).toBe(SupportedLanguages.Go);
});
});
describe('Rust', () => {
it('detects .rs files', () => {
expect(getLanguageFromFilename('main.rs')).toBe(SupportedLanguages.Rust);
});
});
describe('PHP', () => {
it.each(['.php', '.phtml', '.php3', '.php4', '.php5', '.php8'])('detects %s files', (ext) => {
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.PHP);
});
});
describe('Swift', () => {
it('detects .swift files', () => {
expect(getLanguageFromFilename('App.swift')).toBe(SupportedLanguages.Swift);
});
});
describe('Ruby', () => {
it.each(['.rb', '.rake', '.gemspec'])('detects %s files', (ext) => {
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.Ruby);
});
it('detects extensionless Rakefile', () => {
expect(getLanguageFromFilename('Rakefile')).toBe(SupportedLanguages.Ruby);
});
it('detects extensionless Gemfile', () => {
expect(getLanguageFromFilename('Gemfile')).toBe(SupportedLanguages.Ruby);
});
});
describe('Kotlin', () => {
it.each(['.kt', '.kts'])('detects %s files', (ext) => {
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.Kotlin);
});
});
describe('unsupported', () => {
it.each(['.scala', '.r', '.lua', '.zig', '.txt', '.md', '.json', '.yaml'])(
'returns null for %s files',
(ext) => {
expect(getLanguageFromFilename(`file${ext}`)).toBeNull();
},
);
it('returns null for files without extension', () => {
expect(getLanguageFromFilename('Makefile')).toBeNull();
});
it('returns null for empty string', () => {
expect(getLanguageFromFilename('')).toBeNull();
});
});
});
describe('isBuiltInOrNoise', () => {
const js = getProvider(SupportedLanguages.JavaScript);
const py = getProvider(SupportedLanguages.Python);
const php = getProvider(SupportedLanguages.PHP);
const c = getProvider(SupportedLanguages.C);
const kt = getProvider(SupportedLanguages.Kotlin);
const swift = getProvider(SupportedLanguages.Swift);
const rust = getProvider(SupportedLanguages.Rust);
const cs = getProvider(SupportedLanguages.CSharp);
describe('JavaScript/TypeScript', () => {
it('filters console methods', () => {
expect(js.isBuiltInName('console')).toBe(true);
expect(js.isBuiltInName('log')).toBe(true);
expect(js.isBuiltInName('warn')).toBe(true);
});
it('filters React hooks', () => {
expect(js.isBuiltInName('useState')).toBe(true);
expect(js.isBuiltInName('useEffect')).toBe(true);
expect(js.isBuiltInName('useCallback')).toBe(true);
});
it('filters array methods', () => {
expect(js.isBuiltInName('map')).toBe(true);
expect(js.isBuiltInName('filter')).toBe(true);
expect(js.isBuiltInName('reduce')).toBe(true);
});
});
describe('Python', () => {
it('filters built-in functions', () => {
expect(py.isBuiltInName('print')).toBe(true);
expect(py.isBuiltInName('len')).toBe(true);
expect(py.isBuiltInName('range')).toBe(true);
});
});
describe('PHP', () => {
it('filters PHP built-in functions', () => {
expect(php.isBuiltInName('echo')).toBe(true);
expect(php.isBuiltInName('isset')).toBe(true);
expect(php.isBuiltInName('date')).toBe(true);
expect(php.isBuiltInName('json_encode')).toBe(true);
expect(php.isBuiltInName('array_map')).toBe(true);
});
it('filters PHP string functions', () => {
expect(php.isBuiltInName('strlen')).toBe(true);
expect(php.isBuiltInName('substr')).toBe(true);
expect(php.isBuiltInName('str_replace')).toBe(true);
});
});
describe('C/C++', () => {
it('filters standard library functions', () => {
expect(c.isBuiltInName('printf')).toBe(true);
expect(c.isBuiltInName('malloc')).toBe(true);
expect(c.isBuiltInName('free')).toBe(true);
});
it('filters Linux kernel macros', () => {
expect(c.isBuiltInName('container_of')).toBe(true);
expect(c.isBuiltInName('ARRAY_SIZE')).toBe(true);
expect(c.isBuiltInName('pr_info')).toBe(true);
});
});
describe('Kotlin', () => {
it('filters stdlib functions', () => {
expect(kt.isBuiltInName('println')).toBe(true);
expect(kt.isBuiltInName('listOf')).toBe(true);
expect(kt.isBuiltInName('TODO')).toBe(true);
});
it('filters coroutine functions', () => {
expect(kt.isBuiltInName('launch')).toBe(true);
expect(kt.isBuiltInName('async')).toBe(true);
});
});
describe('Swift', () => {
it('filters built-in functions', () => {
expect(swift.isBuiltInName('print')).toBe(true);
expect(swift.isBuiltInName('fatalError')).toBe(true);
});
it('filters UIKit methods', () => {
expect(swift.isBuiltInName('addSubview')).toBe(true);
expect(swift.isBuiltInName('reloadData')).toBe(true);
});
});
describe('Rust', () => {
it('filters Result/Option methods', () => {
expect(rust.isBuiltInName('unwrap')).toBe(true);
expect(rust.isBuiltInName('expect')).toBe(true);
expect(rust.isBuiltInName('unwrap_or')).toBe(true);
expect(rust.isBuiltInName('unwrap_or_else')).toBe(true);
expect(rust.isBuiltInName('unwrap_or_default')).toBe(true);
expect(rust.isBuiltInName('ok')).toBe(true);
expect(rust.isBuiltInName('err')).toBe(true);
expect(rust.isBuiltInName('is_ok')).toBe(true);
expect(rust.isBuiltInName('is_err')).toBe(true);
expect(rust.isBuiltInName('map_err')).toBe(true);
expect(rust.isBuiltInName('and_then')).toBe(true);
expect(rust.isBuiltInName('or_else')).toBe(true);
});
it('filters trait conversion methods', () => {
expect(rust.isBuiltInName('clone')).toBe(true);
expect(rust.isBuiltInName('to_string')).toBe(true);
expect(rust.isBuiltInName('to_owned')).toBe(true);
expect(rust.isBuiltInName('into')).toBe(true);
expect(rust.isBuiltInName('from')).toBe(true);
expect(rust.isBuiltInName('as_ref')).toBe(true);
expect(rust.isBuiltInName('as_mut')).toBe(true);
});
it('filters iterator methods', () => {
expect(rust.isBuiltInName('iter')).toBe(true);
expect(rust.isBuiltInName('into_iter')).toBe(true);
expect(rust.isBuiltInName('collect')).toBe(true);
expect(rust.isBuiltInName('fold')).toBe(true);
expect(rust.isBuiltInName('for_each')).toBe(true);
});
it('filters collection methods', () => {
expect(rust.isBuiltInName('len')).toBe(true);
expect(rust.isBuiltInName('is_empty')).toBe(true);
expect(rust.isBuiltInName('push')).toBe(true);
expect(rust.isBuiltInName('pop')).toBe(true);
expect(rust.isBuiltInName('insert')).toBe(true);
expect(rust.isBuiltInName('remove')).toBe(true);
expect(rust.isBuiltInName('contains')).toBe(true);
});
it('filters macro-like and panic functions', () => {
expect(rust.isBuiltInName('format')).toBe(true);
expect(rust.isBuiltInName('panic')).toBe(true);
expect(rust.isBuiltInName('unreachable')).toBe(true);
expect(rust.isBuiltInName('todo')).toBe(true);
expect(rust.isBuiltInName('unimplemented')).toBe(true);
expect(rust.isBuiltInName('vec')).toBe(true);
expect(rust.isBuiltInName('println')).toBe(true);
expect(rust.isBuiltInName('eprintln')).toBe(true);
expect(rust.isBuiltInName('dbg')).toBe(true);
});
it('filters sync primitives', () => {
expect(rust.isBuiltInName('lock')).toBe(true);
expect(rust.isBuiltInName('try_lock')).toBe(true);
expect(rust.isBuiltInName('spawn')).toBe(true);
expect(rust.isBuiltInName('join')).toBe(true);
expect(rust.isBuiltInName('sleep')).toBe(true);
});
it('filters enum constructors', () => {
expect(rust.isBuiltInName('Some')).toBe(true);
expect(rust.isBuiltInName('None')).toBe(true);
expect(rust.isBuiltInName('Ok')).toBe(true);
expect(rust.isBuiltInName('Err')).toBe(true);
});
it('does not filter user-defined Rust functions', () => {
expect(rust.isBuiltInName('process_request')).toBe(false);
expect(rust.isBuiltInName('handle_connection')).toBe(false);
expect(rust.isBuiltInName('build_response')).toBe(false);
});
});
describe('C#/.NET', () => {
it('filters Console I/O', () => {
expect(cs.isBuiltInName('Console')).toBe(true);
expect(cs.isBuiltInName('WriteLine')).toBe(true);
expect(cs.isBuiltInName('ReadLine')).toBe(true);
});
it('filters LINQ methods', () => {
expect(cs.isBuiltInName('Where')).toBe(true);
expect(cs.isBuiltInName('Select')).toBe(true);
expect(cs.isBuiltInName('GroupBy')).toBe(true);
expect(cs.isBuiltInName('OrderBy')).toBe(true);
expect(cs.isBuiltInName('FirstOrDefault')).toBe(true);
expect(cs.isBuiltInName('ToList')).toBe(true);
});
it('filters Task async methods', () => {
expect(cs.isBuiltInName('Task')).toBe(true);
expect(cs.isBuiltInName('Run')).toBe(true);
expect(cs.isBuiltInName('WhenAll')).toBe(true);
expect(cs.isBuiltInName('ConfigureAwait')).toBe(true);
});
it('filters Object base methods', () => {
expect(cs.isBuiltInName('ToString')).toBe(true);
expect(cs.isBuiltInName('GetType')).toBe(true);
expect(cs.isBuiltInName('Equals')).toBe(true);
expect(cs.isBuiltInName('GetHashCode')).toBe(true);
});
});
describe('user-defined functions', () => {
it('does not filter custom function names', () => {
expect(js.isBuiltInName('myCustomFunction')).toBe(false);
expect(py.isBuiltInName('processData')).toBe(false);
expect(rust.isBuiltInName('handleUserRequest')).toBe(false);
});
});
});
describe('extractFunctionName (via methodExtractor)', () => {
const parser = new Parser();
const cProvider = getProvider(SupportedLanguages.C);
const cppProvider = getProvider(SupportedLanguages.CPlusPlus);
const tsProvider = getProvider(SupportedLanguages.TypeScript);
/** Test helper: extracts function name using methodExtractor hook with generic fallback. */
const extractFunctionName = (
node: SyntaxNode | null,
provider?: LanguageProvider,
): { funcName: string | null; label: NodeLabel } => {
if (!node) return { funcName: null, label: 'Function' };
const result = provider?.methodExtractor?.extractFunctionName?.(node);
if (result) return result;
const funcName = node.childForFieldName?.('name')?.text ?? null;
return { funcName, label: 'Function' };
};
describe('C', () => {
it('extracts function name from C function definition', () => {
parser.setLanguage(C);
const code = `int main() { return 0; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cProvider);
expect(result.funcName).toBe('main');
expect(result.label).toBe('Function');
});
it('extracts function name with parameters', () => {
parser.setLanguage(C);
const code = `void helper(int a, char* b) {}`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cProvider);
expect(result.funcName).toBe('helper');
expect(result.label).toBe('Function');
});
});
describe('C++', () => {
it('extracts method name from C++ class method definition', () => {
parser.setLanguage(CPP);
const code = `int MyClass::OnEncryptData() { return 0; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('OnEncryptData');
expect(result.label).toBe('Method');
});
it('extracts method name with namespace', () => {
parser.setLanguage(CPP);
const code = `void HuksListener::OnDataOprEvent(int type, DataInfo& info) {}`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('OnDataOprEvent');
expect(result.label).toBe('Method');
});
it('extracts C function (not method)', () => {
parser.setLanguage(CPP);
const code = `void standalone_function() {}`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('standalone_function');
expect(result.label).toBe('Function');
});
it('extracts method with parenthesized declarator', () => {
parser.setLanguage(CPP);
const code = `void (MyClass::handler)() {}`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('handler');
expect(result.label).toBe('Method');
});
});
describe('C pointer returns', () => {
it('extracts name from function returning pointer', () => {
parser.setLanguage(C);
const code = `int* get_data() { return 0; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cProvider);
expect(result.funcName).toBe('get_data');
expect(result.label).toBe('Function');
});
it('extracts name from function returning double pointer', () => {
parser.setLanguage(C);
const code = `char** get_strings() { return 0; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cProvider);
expect(result.funcName).toBe('get_strings');
expect(result.label).toBe('Function');
});
it('extracts name from struct pointer return', () => {
parser.setLanguage(C);
const code = `struct Node* create_node(int val) { return 0; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cProvider);
expect(result.funcName).toBe('create_node');
expect(result.label).toBe('Function');
});
});
describe('C++ pointer/reference returns', () => {
it('extracts name from method returning pointer', () => {
parser.setLanguage(CPP);
const code = `int* MyClass::getData() { return nullptr; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('getData');
expect(result.label).toBe('Method');
});
it('extracts name from function returning reference', () => {
parser.setLanguage(CPP);
const code = `std::string& get_name() { static std::string s; return s; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('get_name');
expect(result.label).toBe('Function');
});
it('extracts name from method returning reference', () => {
parser.setLanguage(CPP);
const code = `int& Container::at(int i) { return data[i]; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('at');
expect(result.label).toBe('Method');
});
it('extracts name from method returning const reference', () => {
parser.setLanguage(CPP);
const code = `const std::string& Config::getName() const { return name_; }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
expect(result.funcName).toBe('getName');
expect(result.label).toBe('Method');
});
});
describe('C++ destructors', () => {
it('extracts destructor name from out-of-line definition', () => {
parser.setLanguage(CPP);
const code = `MyClass::~MyClass() { cleanup(); }`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode, cppProvider);
// destructor_name includes the ~ prefix
expect(result.funcName).toBe('~MyClass');
expect(result.label).toBe('Method');
});
});
describe('TypeScript', () => {
it('extracts arrow function name from variable declarator', () => {
parser.setLanguage(TypeScript.typescript);
const code = `const myHandler = () => { return 1; }`;
const tree = parser.parse(code);
const program = tree.rootNode;
const varDecl = program.child(0);
const declarator = varDecl!.namedChild(0);
const arrowFunc = declarator!.namedChild(1);
const result = extractFunctionName(arrowFunc, tsProvider);
expect(result.funcName).toBe('myHandler');
expect(result.label).toBe('Function');
});
it('extracts function expression name from variable declarator', () => {
parser.setLanguage(TypeScript.typescript);
const code = `const processItem = function() { }`;
const tree = parser.parse(code);
const program = tree.rootNode;
const varDecl = program.child(0);
const declarator = varDecl!.namedChild(0);
const funcExpr = declarator!.namedChild(1);
const result = extractFunctionName(funcExpr, tsProvider);
expect(result.funcName).toBe('processItem');
expect(result.label).toBe('Function');
});
});
describe('Python', () => {
it('extracts function name from Python function definition', () => {
parser.setLanguage(Python);
const code = `def hello_world():\n pass`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode);
expect(result.funcName).toBe('hello_world');
expect(result.label).toBe('Function');
});
it('extracts function name with parameters', () => {
parser.setLanguage(Python);
const code = `def calculate_sum(a, b):\n return a + b`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode);
expect(result.funcName).toBe('calculate_sum');
expect(result.label).toBe('Function');
});
it('extracts async function name', () => {
parser.setLanguage(Python);
const code = `async def fetch_data():\n pass`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode);
expect(result.funcName).toBe('fetch_data');
expect(result.label).toBe('Function');
});
it('extracts function name with type hints', () => {
parser.setLanguage(Python);
const code = `def process_data(items: list[int]) -> bool:\n return True`;
const tree = parser.parse(code);
const funcNode = tree.rootNode.child(0);
const result = extractFunctionName(funcNode);
expect(result.funcName).toBe('process_data');
expect(result.label).toBe('Function');
});
it('extracts nested function name', () => {
parser.setLanguage(Python);
const code = `def outer():\n def inner():\n pass`;
const tree = parser.parse(code);
const outerFunc = tree.rootNode.child(0);
const block = outerFunc!.child(4);
const innerFunc = block!.namedChild(0);
const result = extractFunctionName(innerFunc);
expect(result.funcName).toBe('inner');
expect(result.label).toBe('Function');
});
});
});
describe('getTreeSitterBufferSize', () => {
const expectedBufferSize = (byteLength: number): number =>
Math.min(Math.max(byteLength * 2, TREE_SITTER_BUFFER_SIZE), TREE_SITTER_MAX_BUFFER);
it('returns minimum 512KB for small files', () => {
expect(getTreeSitterBufferSize('x'.repeat(100))).toBe(TREE_SITTER_BUFFER_SIZE);
expect(getTreeSitterBufferSize('')).toBe(TREE_SITTER_BUFFER_SIZE);
expect(getTreeSitterBufferSize('x'.repeat(1000))).toBe(TREE_SITTER_BUFFER_SIZE);
});
it('returns 2x content length when larger than minimum', () => {
const size = 400 * 1024; // 400 KB, 2x = 800 KB > 512 KB min
expect(getTreeSitterBufferSize('x'.repeat(size))).toBe(size * 2);
});
it('caps at 32MB for very large files', () => {
const huge = 'x'.repeat(20 * 1024 * 1024); // 20 MB, 2x = 40 MB > 32 MB cap
expect(getTreeSitterBufferSize(huge)).toBe(32 * 1024 * 1024);
});
it('returns exactly 512KB at the boundary', () => {
// 256KB * 2 = 512KB = minimum, so should return minimum
expect(getTreeSitterBufferSize('x'.repeat(256 * 1024))).toBe(TREE_SITTER_BUFFER_SIZE);
});
it('scales linearly between min and max', () => {
const small = getTreeSitterBufferSize('x'.repeat(300 * 1024));
const medium = getTreeSitterBufferSize('x'.repeat(1 * 1024 * 1024));
const large = getTreeSitterBufferSize('x'.repeat(5 * 1024 * 1024));
expect(small).toBeLessThan(medium);
expect(medium).toBeLessThan(large);
});
it('sizes from UTF-8 bytes, not UTF-16 code units', () => {
const source = '漢'.repeat(190_000);
const byteLength = getTreeSitterContentByteLength(source);
expect(byteLength).toBe(source.length * 3);
expect(getTreeSitterBufferSize(source)).toBe(expectedBufferSize(byteLength));
});
it('caps UTF-8-heavy sources using byte length', () => {
const source = '漢'.repeat(6_000_000);
expect(getTreeSitterContentByteLength(source)).toBe(source.length * 3);
expect(getTreeSitterBufferSize(source)).toBe(TREE_SITTER_MAX_BUFFER);
});
it('TREE_SITTER_MAX_BUFFER is 32MB', () => {
expect(TREE_SITTER_MAX_BUFFER).toBe(32 * 1024 * 1024);
});
it('returns max buffer at exact boundary (16MB input)', () => {
// 16MB * 2 = 32MB = max
expect(getTreeSitterBufferSize('x'.repeat(16 * 1024 * 1024))).toBe(TREE_SITTER_MAX_BUFFER);
});
it('file just over max returns max buffer', () => {
// 17MB * 2 = 34MB > 32MB cap
expect(getTreeSitterBufferSize('x'.repeat(17 * 1024 * 1024))).toBe(TREE_SITTER_MAX_BUFFER);
});
it('handles files between old 512KB limit and new 32MB limit', () => {
const sizes = [600 * 1024, 1024 * 1024, 5 * 1024 * 1024, 10 * 1024 * 1024];
for (const size of sizes) {
expect(getTreeSitterBufferSize('x'.repeat(size))).toBe(expectedBufferSize(size));
}
});
});