mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-22 00:31:17 +00:00
Some checks are pending
CI / quality (push) Waiting to run
CI / tests (push) Waiting to run
CI / e2e (push) Waiting to run
CI / scope-parity (push) Waiting to run
CI / Save PR Metadata (push) Blocked by required conditions
CI / CI Gate (push) Blocked by required conditions
Release Candidate / Check if release candidate should run (push) Waiting to run
Release Candidate / ci (push) Blocked by required conditions
Release Candidate / Publish release candidate to npm (push) Blocked by required conditions
Release Candidate / Build & Push RC Docker images (push) Blocked by required conditions
* fix(csharp): adaptive tree-sitter buffer + frozen-bucket clone for cross-namespace siblings (#1066) Two coupled regressions surfaced when analyzing real-world C# repos with large source files (issue #1066): 1. Tree-sitter `parser.parse()` is hard-coded to a 32 KB buffer by default. Any file exceeding that threshold throws `Invalid argument` on the worker re-parse path of `populateCsharpNamespaceSiblings` (and the analogous Python / TypeScript captures fallbacks). 2. After the buffer fix unblocks the AST walk, the hook tries to `push()` onto the inner `BindingRef[]` array fetched from `indexes.bindings` — but `materializeBindings` froze that array via `Object.freeze(refs.slice())`. Result: `Cannot add property N, object is not extensible`. Fixes: - `csharp/captures.ts`, `python/captures.ts`, `typescript/captures.ts`: pass `bufferSize: getTreeSitterBufferSize(sourceText.length)` to `parser.parse()` on the cache-miss path so multi-MB files parse. - `csharp/namespace-siblings.ts`: introduce `cloneBindingBucket` to copy the frozen array before mutating, then `set()` the new array back. This is a working but architecturally compromised workaround (#1050 follow-up will replace it with an explicit augmentation channel — see docs/plans/2026-04-26-001 plan). Tests: - New `csharp-large-cache-miss-resolution` fixture (Models/Services/ Other layout, ~77 KB padded UserService.cs) drives the buffer-size failure end-to-end through worker mode. - `csharp.test.ts`: 4 new regression assertions covering both the parse-time buffer-size failure and the freeze workaround. - Per-language captures unit tests gain "large cache-miss file uses adaptive buffer" coverage (TS, Python, C#). - `csharp-hooks.test.ts`: in-memory freeze regression test that reproduces the `Cannot add property` crash without invoking the C# parser at all. Made-with: Cursor * refactor(scope-resolution): add bindingAugmentations channel to indexes Step 1 of the binding-augmentation-channel refactor (issue #1066 follow-up). Pure shape change — no consumers yet. Adds a new `readonly bindingAugmentations` field to `ScopeResolutionIndexes` initialized as an empty `Map` by `finalizeScopeModel`. The new channel is the dedicated post-finalize write target for hooks like `populateCsharpNamespaceSiblings`, so `indexes.bindings` can stay frozen and finalize-owned. Behavior unchanged: nothing reads or writes the new field yet. tsc and the full unit suite remain green. Plan: docs/plans/2026-04-26-001-binding-augmentation-channel.md (local only — `docs/plans/` is gitignored). Made-with: Cursor * feat(scope-resolution): add lookupBindingsAt dual-source helper Step 2 of the binding-augmentation-channel refactor. Introduces a single primitive every walker uses to read both the finalize-owned `indexes.bindings` channel and the post-finalize `indexes.bindingAugmentations` channel. Contract: - Finalized refs come first (preserves existing precedence). - Augmented refs append, deduped by `def.nodeId`. - Empty input on both channels returns a shared frozen empty array. - Single-channel hits return the bucket by reference (no allocation). No consumers are wired yet — Step 3 routes the existing walker primitives through this helper. Augmentations remain empty for every language; behavior of the full suite is unchanged. 8 unit tests pin precedence, dedup, identity for single-channel hits, and the shared-empty-frozen-array sentinel. Made-with: Cursor * refactor(scope-resolution): route binding lookups through lookupBindingsAt Step 3 of the binding-augmentation-channel refactor. Every direct `indexes.bindings.get(...)` consumer in the post-finalize phase is now routed through `lookupBindingsAt` (per-name) or `namesAtScope` + `lookupBindingsAt` (bulk iteration). Routed sites: - `findClassBindingInScope` (walkers.ts) — class-receiver lookups. - `findCallableBindingInScope` (walkers.ts) — free-call lookups. - `findExportedDefByName` (walkers.ts) — module-scope-fallback callable lookups. - `propagateImportedReturnTypes` (passes/imported-return-types.ts) — bulk iteration over an importer's binding entries; switched to `namesAtScope` + per-name `lookupBindingsAt` so post-finalize augmentations are visible to import-derived typeBinding mirrors. Behavior unchanged: augmentations are empty across the suite (Step 4 populates them for C# `populateNamespaceSiblings`). 587 scope-resolution unit tests + 50 integration resolver suites green (4 pre-existing Swift method-implements failures unrelated to this work). Adds `namesAtScope` companion helper for the bulk-iteration callers. Made-with: Cursor * refactor(csharp): write namespace siblings to bindingAugmentations channel Step 4 of the binding-augmentation-channel refactor. The C# `populateNamespaceSiblings` hook is the only consumer that needed to inject cross-file bindings post-finalize, and prior to this change it cloned the (frozen) finalized `BindingRef[]` arrays through a `cloneBindingBucket` helper, then `set()`-back the new array — a workaround for the `Object.freeze` applied by `finalize-algorithm.ts` (issue #1066 root cause). Architecturally that violated `ScopeResolver` Invariant I8 (which permits post-finalize modifications but not in-place mutation of finalized buckets). It also forced read-side consumers to be aware of the workaround. This change: * Switches the three C# write sites to append into `indexes.bindingAugmentations` via `getAugmentationBucket`. The augmentation channel was added in Step 1 and is mutable by contract: inner `BindingRef[]` arrays here are NEVER frozen. * Deletes `cloneBindingBucket` and `getMutableScopeBindings` (workaround helpers no longer needed). * `lookupBindingsAt` (Step 2) merges the two channels transparently for every walker (Step 3), so behavior is unchanged for callers. * Updates the unit test to assert against both channels: finalized bucket stays frozen and untouched, cross-file siblings show up in augmentations only. Renamed the test accordingly. Validation: * `npx tsc --noEmit` clean. * csharp hooks unit + walkers-augmentations unit + csharp integration resolver suite all green (236/236). * Wider `test/unit/scope-resolution test/integration/resolvers` suite: 2507 pass, only 4 pre-existing Swift METHOD_IMPLEMENTS failures remain (unrelated to this work, present on baseline). Refs: issue #1066, ADR-pending binding-augmentation-channel. Made-with: Cursor * feat(scope-resolution): tighten I8 + add validateBindingsImmutability dev guard Step 5 of the binding-augmentation-channel refactor. Captures the new two-channel binding lifecycle in the contract docs and adds a dev-mode runtime validator so a future hook cannot silently drift back into mutating `indexes.bindings`. Contract changes: * `contract/scope-resolver.ts` — rewrote Invariant I8 to describe the two channels (`indexes.bindings` is finalize-output and immutable post-finalize; `indexes.bindingAugmentations` is the append-only post-finalize channel populated by hooks like `populateNamespaceSiblings`). Documented `lookupBindingsAt` as the read-side merger and pointed at the new validator as the enforcement mechanism. * `gitnexus-shared/src/scope-resolution/types.ts` — extended the module-header lifecycle contract to call out `bindingAugmentations` alongside `ReferenceIndex` as the two structures populated after the freeze. Validator: * New `pipeline/validate-bindings-immutability.ts` mirrors the shape of `validateOwnershipParity` (#909): runs only when `NODE_ENV !== 'production' && VALIDATE_SEMANTIC_MODEL !== '0'`, emits via `onWarn`, never throws. Asserts (a) every inner `BindingRef[]` in `indexes.bindings` is `Object.isFrozen`, and (b) every inner array in `indexes.bindingAugmentations` is NOT frozen. * Wired into `pipeline/run.ts` after both `populateNamespaceSiblings` and `propagateImportedReturnTypes`, before `resolveReferenceSites`. One sweep covers the full post-finalize surface. Tests: * `validate-bindings-immutability.test.ts` — 6 cases pinning happy path, both drift directions, multi-violation accumulation, and both production no-op gates. All scope-resolution + csharp resolver tests green (242/242 in the focused run; matches the wider Step 4 baseline). Made-with: Cursor * fix(ingestion): size tree-sitter buffers from UTF-8 bytes Tree-sitter buffer sizing is byte-based, so computing adaptive buffers from JavaScript string length under-sized UTF-8-heavy files. Make getTreeSitterBufferSize accept source text directly and compute Buffer.byteLength internally, then update all parse call sites and max-buffer skip checks to use byte length. Add multibyte cache-miss and cap regressions for C#, Python, TypeScript, and the C# namespace-sibling fallback parse path. Made-with: Cursor * test(scope-resolution): pin augmentation read paths Add focused unit coverage for augmented-only binding reads across the routed walker helpers and imported-return-type propagation path. Clarify I8 wording around lexical Scope.bindings versus post-finalize index channels, and document the intentional local-only behavior of findExportedDef. Also switch the immutability validator tests to Vitest env stubs, document one intentional validator blind spot, and split C# namespace-sibling tests so UTF-8 parsing and augmentation-channel behavior are asserted independently. Made-with: Cursor * test(scope-resolution): avoid slow parser stress fixtures Replace high-cardinality large-file capture fixtures with large padding plus a trailing declaration. This still proves adaptive tree-sitter buffers parse beyond large ASCII and UTF-8-heavy input, without making query matching process thousands of declarations and risking timeouts. Made-with: Cursor * test(scope-resolution): add python and typescript cache-miss resolver regressions Add worker-mode resolver integration coverage mirroring the C# #1066 scenario for Python and TypeScript. Each test builds a temp fixture with large ASCII and UTF-8-heavy source padding, then asserts trailing declarations and call edges still resolve after scope-resolution cache-miss reparsing. Made-with: Cursor * refactor(scope-resolution): gate I8 validator and fast-path namesAtScope Addresses SPARC reviewer feedback on the binding-augmentation channel: - Validator gate is now opt-in outside development. Extract isSemanticModelValidatorEnabled() in utils/env.ts as the single predicate; both validateBindingsImmutability and phase.ts's warn handler share it. Default CLI runs no longer pay the O(binding-buckets) scan, and explicit VALIDATE_SEMANTIC_MODEL=1 now emits warnings even when NODE_ENV is unset. - namesAtScope returns Iterable<string> and zero-allocates when at most one channel is populated (returns Map.keys() directly), only materializing a Set when both channels carry names. The caller-side branching and EMPTY_NAMES escape hatch in propagateImportedReturnTypes are gone -- both helpers handle the empty-augmentation case internally. - C# namespace-siblings header/JSDoc, model JSDoc, I8 contract prose, and the #1066 integration-test header rewritten to say post-finalize fanout appends only to bindingAugmentations; finalized refs come first and win duplicate def.nodeId metadata; local lexical Scope.bindings remains the first-tier shadowing channel. Validator unit-test setup deduplicated via beforeEach and extended with default-CLI no-op + explicit-opt-in cases. Made-with: Cursor
707 lines
25 KiB
TypeScript
707 lines
25 KiB
TypeScript
import { describe, it, expect } from 'vitest';
|
|
import { getLanguageFromFilename, SupportedLanguages } from 'gitnexus-shared';
|
|
import { getProvider } from '../../src/core/ingestion/languages/index.js';
|
|
import type { SyntaxNode } from '../../src/core/ingestion/utils/ast-helpers.js';
|
|
import type { NodeLabel } from 'gitnexus-shared';
|
|
import type { LanguageProvider } from '../../src/core/ingestion/language-provider.js';
|
|
import {
|
|
getTreeSitterBufferSize,
|
|
getTreeSitterContentByteLength,
|
|
TREE_SITTER_BUFFER_SIZE,
|
|
TREE_SITTER_MAX_BUFFER,
|
|
} from '../../src/core/ingestion/constants.js';
|
|
import Parser from 'tree-sitter';
|
|
import C from 'tree-sitter-c';
|
|
import CPP from 'tree-sitter-cpp';
|
|
import Python from 'tree-sitter-python';
|
|
import TypeScript from 'tree-sitter-typescript';
|
|
|
|
describe('getLanguageFromFilename', () => {
|
|
describe('TypeScript', () => {
|
|
it('detects .ts files', () => {
|
|
expect(getLanguageFromFilename('index.ts')).toBe(SupportedLanguages.TypeScript);
|
|
});
|
|
|
|
it('detects .tsx files', () => {
|
|
expect(getLanguageFromFilename('Component.tsx')).toBe(SupportedLanguages.TypeScript);
|
|
});
|
|
|
|
it('detects .ts files in paths', () => {
|
|
expect(getLanguageFromFilename('src/core/utils.ts')).toBe(SupportedLanguages.TypeScript);
|
|
});
|
|
});
|
|
|
|
describe('JavaScript', () => {
|
|
it('detects .js files', () => {
|
|
expect(getLanguageFromFilename('index.js')).toBe(SupportedLanguages.JavaScript);
|
|
});
|
|
|
|
it('detects .jsx files', () => {
|
|
expect(getLanguageFromFilename('App.jsx')).toBe(SupportedLanguages.JavaScript);
|
|
});
|
|
});
|
|
|
|
describe('Python', () => {
|
|
it('detects .py files', () => {
|
|
expect(getLanguageFromFilename('main.py')).toBe(SupportedLanguages.Python);
|
|
});
|
|
});
|
|
|
|
describe('Java', () => {
|
|
it('detects .java files', () => {
|
|
expect(getLanguageFromFilename('Main.java')).toBe(SupportedLanguages.Java);
|
|
});
|
|
});
|
|
|
|
describe('C', () => {
|
|
it('detects .c files', () => {
|
|
expect(getLanguageFromFilename('main.c')).toBe(SupportedLanguages.C);
|
|
});
|
|
});
|
|
|
|
describe('C++', () => {
|
|
it.each(['.cpp', '.cc', '.cxx', '.h', '.hpp', '.hxx', '.hh'])('detects %s files', (ext) => {
|
|
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.CPlusPlus);
|
|
});
|
|
});
|
|
|
|
describe('C#', () => {
|
|
it('detects .cs files', () => {
|
|
expect(getLanguageFromFilename('Program.cs')).toBe(SupportedLanguages.CSharp);
|
|
});
|
|
});
|
|
|
|
describe('Go', () => {
|
|
it('detects .go files', () => {
|
|
expect(getLanguageFromFilename('main.go')).toBe(SupportedLanguages.Go);
|
|
});
|
|
});
|
|
|
|
describe('Rust', () => {
|
|
it('detects .rs files', () => {
|
|
expect(getLanguageFromFilename('main.rs')).toBe(SupportedLanguages.Rust);
|
|
});
|
|
});
|
|
|
|
describe('PHP', () => {
|
|
it.each(['.php', '.phtml', '.php3', '.php4', '.php5', '.php8'])('detects %s files', (ext) => {
|
|
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.PHP);
|
|
});
|
|
});
|
|
|
|
describe('Swift', () => {
|
|
it('detects .swift files', () => {
|
|
expect(getLanguageFromFilename('App.swift')).toBe(SupportedLanguages.Swift);
|
|
});
|
|
});
|
|
|
|
describe('Ruby', () => {
|
|
it.each(['.rb', '.rake', '.gemspec'])('detects %s files', (ext) => {
|
|
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.Ruby);
|
|
});
|
|
|
|
it('detects extensionless Rakefile', () => {
|
|
expect(getLanguageFromFilename('Rakefile')).toBe(SupportedLanguages.Ruby);
|
|
});
|
|
|
|
it('detects extensionless Gemfile', () => {
|
|
expect(getLanguageFromFilename('Gemfile')).toBe(SupportedLanguages.Ruby);
|
|
});
|
|
});
|
|
|
|
describe('Kotlin', () => {
|
|
it.each(['.kt', '.kts'])('detects %s files', (ext) => {
|
|
expect(getLanguageFromFilename(`file${ext}`)).toBe(SupportedLanguages.Kotlin);
|
|
});
|
|
});
|
|
|
|
describe('unsupported', () => {
|
|
it.each(['.scala', '.r', '.lua', '.zig', '.txt', '.md', '.json', '.yaml'])(
|
|
'returns null for %s files',
|
|
(ext) => {
|
|
expect(getLanguageFromFilename(`file${ext}`)).toBeNull();
|
|
},
|
|
);
|
|
|
|
it('returns null for files without extension', () => {
|
|
expect(getLanguageFromFilename('Makefile')).toBeNull();
|
|
});
|
|
|
|
it('returns null for empty string', () => {
|
|
expect(getLanguageFromFilename('')).toBeNull();
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('isBuiltInOrNoise', () => {
|
|
const js = getProvider(SupportedLanguages.JavaScript);
|
|
const py = getProvider(SupportedLanguages.Python);
|
|
const php = getProvider(SupportedLanguages.PHP);
|
|
const c = getProvider(SupportedLanguages.C);
|
|
const kt = getProvider(SupportedLanguages.Kotlin);
|
|
const swift = getProvider(SupportedLanguages.Swift);
|
|
const rust = getProvider(SupportedLanguages.Rust);
|
|
const cs = getProvider(SupportedLanguages.CSharp);
|
|
|
|
describe('JavaScript/TypeScript', () => {
|
|
it('filters console methods', () => {
|
|
expect(js.isBuiltInName('console')).toBe(true);
|
|
expect(js.isBuiltInName('log')).toBe(true);
|
|
expect(js.isBuiltInName('warn')).toBe(true);
|
|
});
|
|
|
|
it('filters React hooks', () => {
|
|
expect(js.isBuiltInName('useState')).toBe(true);
|
|
expect(js.isBuiltInName('useEffect')).toBe(true);
|
|
expect(js.isBuiltInName('useCallback')).toBe(true);
|
|
});
|
|
|
|
it('filters array methods', () => {
|
|
expect(js.isBuiltInName('map')).toBe(true);
|
|
expect(js.isBuiltInName('filter')).toBe(true);
|
|
expect(js.isBuiltInName('reduce')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('Python', () => {
|
|
it('filters built-in functions', () => {
|
|
expect(py.isBuiltInName('print')).toBe(true);
|
|
expect(py.isBuiltInName('len')).toBe(true);
|
|
expect(py.isBuiltInName('range')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('PHP', () => {
|
|
it('filters PHP built-in functions', () => {
|
|
expect(php.isBuiltInName('echo')).toBe(true);
|
|
expect(php.isBuiltInName('isset')).toBe(true);
|
|
expect(php.isBuiltInName('date')).toBe(true);
|
|
expect(php.isBuiltInName('json_encode')).toBe(true);
|
|
expect(php.isBuiltInName('array_map')).toBe(true);
|
|
});
|
|
|
|
it('filters PHP string functions', () => {
|
|
expect(php.isBuiltInName('strlen')).toBe(true);
|
|
expect(php.isBuiltInName('substr')).toBe(true);
|
|
expect(php.isBuiltInName('str_replace')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('C/C++', () => {
|
|
it('filters standard library functions', () => {
|
|
expect(c.isBuiltInName('printf')).toBe(true);
|
|
expect(c.isBuiltInName('malloc')).toBe(true);
|
|
expect(c.isBuiltInName('free')).toBe(true);
|
|
});
|
|
|
|
it('filters Linux kernel macros', () => {
|
|
expect(c.isBuiltInName('container_of')).toBe(true);
|
|
expect(c.isBuiltInName('ARRAY_SIZE')).toBe(true);
|
|
expect(c.isBuiltInName('pr_info')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('Kotlin', () => {
|
|
it('filters stdlib functions', () => {
|
|
expect(kt.isBuiltInName('println')).toBe(true);
|
|
expect(kt.isBuiltInName('listOf')).toBe(true);
|
|
expect(kt.isBuiltInName('TODO')).toBe(true);
|
|
});
|
|
|
|
it('filters coroutine functions', () => {
|
|
expect(kt.isBuiltInName('launch')).toBe(true);
|
|
expect(kt.isBuiltInName('async')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('Swift', () => {
|
|
it('filters built-in functions', () => {
|
|
expect(swift.isBuiltInName('print')).toBe(true);
|
|
expect(swift.isBuiltInName('fatalError')).toBe(true);
|
|
});
|
|
|
|
it('filters UIKit methods', () => {
|
|
expect(swift.isBuiltInName('addSubview')).toBe(true);
|
|
expect(swift.isBuiltInName('reloadData')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('Rust', () => {
|
|
it('filters Result/Option methods', () => {
|
|
expect(rust.isBuiltInName('unwrap')).toBe(true);
|
|
expect(rust.isBuiltInName('expect')).toBe(true);
|
|
expect(rust.isBuiltInName('unwrap_or')).toBe(true);
|
|
expect(rust.isBuiltInName('unwrap_or_else')).toBe(true);
|
|
expect(rust.isBuiltInName('unwrap_or_default')).toBe(true);
|
|
expect(rust.isBuiltInName('ok')).toBe(true);
|
|
expect(rust.isBuiltInName('err')).toBe(true);
|
|
expect(rust.isBuiltInName('is_ok')).toBe(true);
|
|
expect(rust.isBuiltInName('is_err')).toBe(true);
|
|
expect(rust.isBuiltInName('map_err')).toBe(true);
|
|
expect(rust.isBuiltInName('and_then')).toBe(true);
|
|
expect(rust.isBuiltInName('or_else')).toBe(true);
|
|
});
|
|
|
|
it('filters trait conversion methods', () => {
|
|
expect(rust.isBuiltInName('clone')).toBe(true);
|
|
expect(rust.isBuiltInName('to_string')).toBe(true);
|
|
expect(rust.isBuiltInName('to_owned')).toBe(true);
|
|
expect(rust.isBuiltInName('into')).toBe(true);
|
|
expect(rust.isBuiltInName('from')).toBe(true);
|
|
expect(rust.isBuiltInName('as_ref')).toBe(true);
|
|
expect(rust.isBuiltInName('as_mut')).toBe(true);
|
|
});
|
|
|
|
it('filters iterator methods', () => {
|
|
expect(rust.isBuiltInName('iter')).toBe(true);
|
|
expect(rust.isBuiltInName('into_iter')).toBe(true);
|
|
expect(rust.isBuiltInName('collect')).toBe(true);
|
|
expect(rust.isBuiltInName('fold')).toBe(true);
|
|
expect(rust.isBuiltInName('for_each')).toBe(true);
|
|
});
|
|
|
|
it('filters collection methods', () => {
|
|
expect(rust.isBuiltInName('len')).toBe(true);
|
|
expect(rust.isBuiltInName('is_empty')).toBe(true);
|
|
expect(rust.isBuiltInName('push')).toBe(true);
|
|
expect(rust.isBuiltInName('pop')).toBe(true);
|
|
expect(rust.isBuiltInName('insert')).toBe(true);
|
|
expect(rust.isBuiltInName('remove')).toBe(true);
|
|
expect(rust.isBuiltInName('contains')).toBe(true);
|
|
});
|
|
|
|
it('filters macro-like and panic functions', () => {
|
|
expect(rust.isBuiltInName('format')).toBe(true);
|
|
expect(rust.isBuiltInName('panic')).toBe(true);
|
|
expect(rust.isBuiltInName('unreachable')).toBe(true);
|
|
expect(rust.isBuiltInName('todo')).toBe(true);
|
|
expect(rust.isBuiltInName('unimplemented')).toBe(true);
|
|
expect(rust.isBuiltInName('vec')).toBe(true);
|
|
expect(rust.isBuiltInName('println')).toBe(true);
|
|
expect(rust.isBuiltInName('eprintln')).toBe(true);
|
|
expect(rust.isBuiltInName('dbg')).toBe(true);
|
|
});
|
|
|
|
it('filters sync primitives', () => {
|
|
expect(rust.isBuiltInName('lock')).toBe(true);
|
|
expect(rust.isBuiltInName('try_lock')).toBe(true);
|
|
expect(rust.isBuiltInName('spawn')).toBe(true);
|
|
expect(rust.isBuiltInName('join')).toBe(true);
|
|
expect(rust.isBuiltInName('sleep')).toBe(true);
|
|
});
|
|
|
|
it('filters enum constructors', () => {
|
|
expect(rust.isBuiltInName('Some')).toBe(true);
|
|
expect(rust.isBuiltInName('None')).toBe(true);
|
|
expect(rust.isBuiltInName('Ok')).toBe(true);
|
|
expect(rust.isBuiltInName('Err')).toBe(true);
|
|
});
|
|
|
|
it('does not filter user-defined Rust functions', () => {
|
|
expect(rust.isBuiltInName('process_request')).toBe(false);
|
|
expect(rust.isBuiltInName('handle_connection')).toBe(false);
|
|
expect(rust.isBuiltInName('build_response')).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe('C#/.NET', () => {
|
|
it('filters Console I/O', () => {
|
|
expect(cs.isBuiltInName('Console')).toBe(true);
|
|
expect(cs.isBuiltInName('WriteLine')).toBe(true);
|
|
expect(cs.isBuiltInName('ReadLine')).toBe(true);
|
|
});
|
|
|
|
it('filters LINQ methods', () => {
|
|
expect(cs.isBuiltInName('Where')).toBe(true);
|
|
expect(cs.isBuiltInName('Select')).toBe(true);
|
|
expect(cs.isBuiltInName('GroupBy')).toBe(true);
|
|
expect(cs.isBuiltInName('OrderBy')).toBe(true);
|
|
expect(cs.isBuiltInName('FirstOrDefault')).toBe(true);
|
|
expect(cs.isBuiltInName('ToList')).toBe(true);
|
|
});
|
|
|
|
it('filters Task async methods', () => {
|
|
expect(cs.isBuiltInName('Task')).toBe(true);
|
|
expect(cs.isBuiltInName('Run')).toBe(true);
|
|
expect(cs.isBuiltInName('WhenAll')).toBe(true);
|
|
expect(cs.isBuiltInName('ConfigureAwait')).toBe(true);
|
|
});
|
|
|
|
it('filters Object base methods', () => {
|
|
expect(cs.isBuiltInName('ToString')).toBe(true);
|
|
expect(cs.isBuiltInName('GetType')).toBe(true);
|
|
expect(cs.isBuiltInName('Equals')).toBe(true);
|
|
expect(cs.isBuiltInName('GetHashCode')).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('user-defined functions', () => {
|
|
it('does not filter custom function names', () => {
|
|
expect(js.isBuiltInName('myCustomFunction')).toBe(false);
|
|
expect(py.isBuiltInName('processData')).toBe(false);
|
|
expect(rust.isBuiltInName('handleUserRequest')).toBe(false);
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('extractFunctionName (via methodExtractor)', () => {
|
|
const parser = new Parser();
|
|
const cProvider = getProvider(SupportedLanguages.C);
|
|
const cppProvider = getProvider(SupportedLanguages.CPlusPlus);
|
|
const tsProvider = getProvider(SupportedLanguages.TypeScript);
|
|
|
|
/** Test helper: extracts function name using methodExtractor hook with generic fallback. */
|
|
const extractFunctionName = (
|
|
node: SyntaxNode | null,
|
|
provider?: LanguageProvider,
|
|
): { funcName: string | null; label: NodeLabel } => {
|
|
if (!node) return { funcName: null, label: 'Function' };
|
|
const result = provider?.methodExtractor?.extractFunctionName?.(node);
|
|
if (result) return result;
|
|
const funcName = node.childForFieldName?.('name')?.text ?? null;
|
|
return { funcName, label: 'Function' };
|
|
};
|
|
|
|
describe('C', () => {
|
|
it('extracts function name from C function definition', () => {
|
|
parser.setLanguage(C);
|
|
const code = `int main() { return 0; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cProvider);
|
|
|
|
expect(result.funcName).toBe('main');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts function name with parameters', () => {
|
|
parser.setLanguage(C);
|
|
const code = `void helper(int a, char* b) {}`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cProvider);
|
|
|
|
expect(result.funcName).toBe('helper');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
});
|
|
|
|
describe('C++', () => {
|
|
it('extracts method name from C++ class method definition', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `int MyClass::OnEncryptData() { return 0; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('OnEncryptData');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
|
|
it('extracts method name with namespace', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `void HuksListener::OnDataOprEvent(int type, DataInfo& info) {}`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('OnDataOprEvent');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
|
|
it('extracts C function (not method)', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `void standalone_function() {}`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('standalone_function');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts method with parenthesized declarator', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `void (MyClass::handler)() {}`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('handler');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
});
|
|
|
|
describe('C pointer returns', () => {
|
|
it('extracts name from function returning pointer', () => {
|
|
parser.setLanguage(C);
|
|
const code = `int* get_data() { return 0; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cProvider);
|
|
|
|
expect(result.funcName).toBe('get_data');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts name from function returning double pointer', () => {
|
|
parser.setLanguage(C);
|
|
const code = `char** get_strings() { return 0; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cProvider);
|
|
|
|
expect(result.funcName).toBe('get_strings');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts name from struct pointer return', () => {
|
|
parser.setLanguage(C);
|
|
const code = `struct Node* create_node(int val) { return 0; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cProvider);
|
|
|
|
expect(result.funcName).toBe('create_node');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
});
|
|
|
|
describe('C++ pointer/reference returns', () => {
|
|
it('extracts name from method returning pointer', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `int* MyClass::getData() { return nullptr; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('getData');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
|
|
it('extracts name from function returning reference', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `std::string& get_name() { static std::string s; return s; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('get_name');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts name from method returning reference', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `int& Container::at(int i) { return data[i]; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('at');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
|
|
it('extracts name from method returning const reference', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `const std::string& Config::getName() const { return name_; }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
expect(result.funcName).toBe('getName');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
});
|
|
|
|
describe('C++ destructors', () => {
|
|
it('extracts destructor name from out-of-line definition', () => {
|
|
parser.setLanguage(CPP);
|
|
const code = `MyClass::~MyClass() { cleanup(); }`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode, cppProvider);
|
|
|
|
// destructor_name includes the ~ prefix
|
|
expect(result.funcName).toBe('~MyClass');
|
|
expect(result.label).toBe('Method');
|
|
});
|
|
});
|
|
|
|
describe('TypeScript', () => {
|
|
it('extracts arrow function name from variable declarator', () => {
|
|
parser.setLanguage(TypeScript.typescript);
|
|
const code = `const myHandler = () => { return 1; }`;
|
|
const tree = parser.parse(code);
|
|
const program = tree.rootNode;
|
|
const varDecl = program.child(0);
|
|
const declarator = varDecl!.namedChild(0);
|
|
const arrowFunc = declarator!.namedChild(1);
|
|
|
|
const result = extractFunctionName(arrowFunc, tsProvider);
|
|
|
|
expect(result.funcName).toBe('myHandler');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts function expression name from variable declarator', () => {
|
|
parser.setLanguage(TypeScript.typescript);
|
|
const code = `const processItem = function() { }`;
|
|
const tree = parser.parse(code);
|
|
const program = tree.rootNode;
|
|
const varDecl = program.child(0);
|
|
const declarator = varDecl!.namedChild(0);
|
|
const funcExpr = declarator!.namedChild(1);
|
|
|
|
const result = extractFunctionName(funcExpr, tsProvider);
|
|
|
|
expect(result.funcName).toBe('processItem');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
});
|
|
|
|
describe('Python', () => {
|
|
it('extracts function name from Python function definition', () => {
|
|
parser.setLanguage(Python);
|
|
const code = `def hello_world():\n pass`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode);
|
|
|
|
expect(result.funcName).toBe('hello_world');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts function name with parameters', () => {
|
|
parser.setLanguage(Python);
|
|
const code = `def calculate_sum(a, b):\n return a + b`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode);
|
|
|
|
expect(result.funcName).toBe('calculate_sum');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts async function name', () => {
|
|
parser.setLanguage(Python);
|
|
const code = `async def fetch_data():\n pass`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode);
|
|
|
|
expect(result.funcName).toBe('fetch_data');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts function name with type hints', () => {
|
|
parser.setLanguage(Python);
|
|
const code = `def process_data(items: list[int]) -> bool:\n return True`;
|
|
const tree = parser.parse(code);
|
|
const funcNode = tree.rootNode.child(0);
|
|
|
|
const result = extractFunctionName(funcNode);
|
|
|
|
expect(result.funcName).toBe('process_data');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
|
|
it('extracts nested function name', () => {
|
|
parser.setLanguage(Python);
|
|
const code = `def outer():\n def inner():\n pass`;
|
|
const tree = parser.parse(code);
|
|
const outerFunc = tree.rootNode.child(0);
|
|
const block = outerFunc!.child(4);
|
|
const innerFunc = block!.namedChild(0);
|
|
|
|
const result = extractFunctionName(innerFunc);
|
|
|
|
expect(result.funcName).toBe('inner');
|
|
expect(result.label).toBe('Function');
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('getTreeSitterBufferSize', () => {
|
|
const expectedBufferSize = (byteLength: number): number =>
|
|
Math.min(Math.max(byteLength * 2, TREE_SITTER_BUFFER_SIZE), TREE_SITTER_MAX_BUFFER);
|
|
|
|
it('returns minimum 512KB for small files', () => {
|
|
expect(getTreeSitterBufferSize('x'.repeat(100))).toBe(TREE_SITTER_BUFFER_SIZE);
|
|
expect(getTreeSitterBufferSize('')).toBe(TREE_SITTER_BUFFER_SIZE);
|
|
expect(getTreeSitterBufferSize('x'.repeat(1000))).toBe(TREE_SITTER_BUFFER_SIZE);
|
|
});
|
|
|
|
it('returns 2x content length when larger than minimum', () => {
|
|
const size = 400 * 1024; // 400 KB, 2x = 800 KB > 512 KB min
|
|
expect(getTreeSitterBufferSize('x'.repeat(size))).toBe(size * 2);
|
|
});
|
|
|
|
it('caps at 32MB for very large files', () => {
|
|
const huge = 'x'.repeat(20 * 1024 * 1024); // 20 MB, 2x = 40 MB > 32 MB cap
|
|
expect(getTreeSitterBufferSize(huge)).toBe(32 * 1024 * 1024);
|
|
});
|
|
|
|
it('returns exactly 512KB at the boundary', () => {
|
|
// 256KB * 2 = 512KB = minimum, so should return minimum
|
|
expect(getTreeSitterBufferSize('x'.repeat(256 * 1024))).toBe(TREE_SITTER_BUFFER_SIZE);
|
|
});
|
|
|
|
it('scales linearly between min and max', () => {
|
|
const small = getTreeSitterBufferSize('x'.repeat(300 * 1024));
|
|
const medium = getTreeSitterBufferSize('x'.repeat(1 * 1024 * 1024));
|
|
const large = getTreeSitterBufferSize('x'.repeat(5 * 1024 * 1024));
|
|
expect(small).toBeLessThan(medium);
|
|
expect(medium).toBeLessThan(large);
|
|
});
|
|
|
|
it('sizes from UTF-8 bytes, not UTF-16 code units', () => {
|
|
const source = '漢'.repeat(190_000);
|
|
const byteLength = getTreeSitterContentByteLength(source);
|
|
expect(byteLength).toBe(source.length * 3);
|
|
expect(getTreeSitterBufferSize(source)).toBe(expectedBufferSize(byteLength));
|
|
});
|
|
|
|
it('caps UTF-8-heavy sources using byte length', () => {
|
|
const source = '漢'.repeat(6_000_000);
|
|
expect(getTreeSitterContentByteLength(source)).toBe(source.length * 3);
|
|
expect(getTreeSitterBufferSize(source)).toBe(TREE_SITTER_MAX_BUFFER);
|
|
});
|
|
|
|
it('TREE_SITTER_MAX_BUFFER is 32MB', () => {
|
|
expect(TREE_SITTER_MAX_BUFFER).toBe(32 * 1024 * 1024);
|
|
});
|
|
|
|
it('returns max buffer at exact boundary (16MB input)', () => {
|
|
// 16MB * 2 = 32MB = max
|
|
expect(getTreeSitterBufferSize('x'.repeat(16 * 1024 * 1024))).toBe(TREE_SITTER_MAX_BUFFER);
|
|
});
|
|
|
|
it('file just over max returns max buffer', () => {
|
|
// 17MB * 2 = 34MB > 32MB cap
|
|
expect(getTreeSitterBufferSize('x'.repeat(17 * 1024 * 1024))).toBe(TREE_SITTER_MAX_BUFFER);
|
|
});
|
|
|
|
it('handles files between old 512KB limit and new 32MB limit', () => {
|
|
const sizes = [600 * 1024, 1024 * 1024, 5 * 1024 * 1024, 10 * 1024 * 1024];
|
|
for (const size of sizes) {
|
|
expect(getTreeSitterBufferSize('x'.repeat(size))).toBe(expectedBufferSize(size));
|
|
}
|
|
});
|
|
});
|