mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-07 08:26:11 +00:00
* Initial plan * feat(ingestion): add variable extraction types, factory, configs, and wire into language providers - Create variable-types.ts with VariableInfo, VariableExtractionConfig, VariableExtractor interfaces - Create variable-extractors/generic.ts with createVariableExtractor() factory - Add variableExtractor field to LanguageProvider interface - Create per-language variable extraction configs for all 16 languages - Wire variableExtractor into all language providers - Add variable metadata enrichment to parse-worker for Const/Static/Variable labels Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * feat(ingestion): add variable extraction tests and fix Python/TS config issues - Create test/unit/variable-extraction.test.ts with 29 tests covering TypeScript, JavaScript, Python, Go, Rust, C, C++, Ruby, and factory behavior - Fix isConst in generic factory to use config.isConst over node-type membership (TS let/const both use lexical_declaration) - Fix Python type extraction for annotated assignments at module scope - Fix Python dunder name visibility (e.g., __name__ is public, not protected) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address code review feedback — move imports, clarify scope comment, use shared test context Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address review comments, fix prettier formatting and lint errors - Fix prettier formatting in 5 files (c-cpp, jvm, swift configs, test file) - Remove unused SyntaxNode imports in php.ts and ruby.ts (lint errors) - Remove unused constNodeSet/variableNodeSet variables in generic.ts (warnings) - Remove semantically wrong `methodProps.isReadonly = varInfo.isConst` (review) - Remove dead `nodeLabel === 'Variable'` guard in parse-worker (review) - Fix test guard: replace `if (declNode)` with `expect(declNode).toBeDefined()` (review) - Add comment about Python expression_statement broadness (review) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/040edbbf-65b5-40e1-80c8-e98f7c4bb54a * feat(ingestion): add block-scoped variable extraction via tree-sitter queries Add @definition.const and @definition.variable tree-sitter query patterns for TypeScript, JavaScript, Python, Go, Java, C, C++, C#, PHP, Ruby, and Dart. Add parse-worker dedup logic to avoid duplicate nodes when variable captures overlap with existing function/property captures. Add 'Variable' label support in getLabelFromCaptures and DEFINITION_CAPTURE_KEYS. Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test: add block-scoped variable extraction tests and query capture tests Add 6 tests for block-scoped variable extraction (TypeScript, Go, Rust, C, Python). Add 14 tests verifying @definition.const/@definition.variable query patterns exist in all language query strings. Import RUBY_QUERIES in test file. Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test: add Python non-assignment expression statement rejection test Addresses code review feedback: verify that the Python variable extractor returns null for expression_statement nodes that contain function calls rather than assignments (e.g. `print("hello")`). Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: Dart query node type, add Variable schema, update schema counts - Change `top_level_variable_declaration` → `declaration` in DART_QUERIES (the former doesn't exist in tree-sitter-dart grammar, causing all Dart integration tests to fail with TSQueryErrorNodeType) - Add VARIABLE_SCHEMA to schema.ts and register in initLbug() so that Variable-labeled nodes are persisted to LadybugDB (not silently dropped) - Add 'Variable' to MULTI_LANG_TYPES in csv-generator.ts - Update Dart variable config to remove invalid node type - Update schema test counts (30→31 node schemas, 32→33 total) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f79931d1-207f-4fbb-91da-259d44f7fd88 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address code review comment improvements - Clarify processedDefinitionNodes tracks start indices, not nodes - Improve Python variableNodeTypes comment wording Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f79931d1-207f-4fbb-91da-259d44f7fd88 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: add Variable to NODE_TABLES, RELATION_SCHEMA, update golden snapshot - Add 'Variable' to NODE_TABLES in gitnexus-shared so validTables.has('Variable') returns true and Variable graph edges are not silently dropped - Add FROM File TO Variable, FROM Variable TO Community, FROM Variable TO Process to RELATION_SCHEMA so KuzuDB can represent edges connecting Variable nodes - Update schema.test.ts: add Variable to multiLang list, fix count 30→31 - Regenerate pipeline-graph-golden snapshot for mini-repo fixture Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e3aad558-e7bb-40d1-b53f-0a2c0132ca96 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: isolate golden test from cli-e2e fixture pollution The pipeline-graph-golden test was non-deterministic because cli-e2e.test.ts creates AGENTS.md, CLAUDE.md, .claude/skills/, and .gitignore in the shared mini-repo fixture during analyze. These leftover files caused the golden test to find 9 files instead of 7 when tests ran in parallel. Fixes: - Golden test now copies the fixture to a temp dir before running, making it immune to concurrent test pollution - cli-e2e afterAll cleanup now removes ALL generated files (AGENTS.md, CLAUDE.md, .claude/, .gitignore) not just .git/ and .gitnexus/ - Golden snapshot regenerated from clean 7-file fixture Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bd378e73-6f37-49c6-aed6-7fabf4dc6183 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
225 lines
6.8 KiB
TypeScript
225 lines
6.8 KiB
TypeScript
import { describe, it, expect } from 'vitest';
|
|
import {
|
|
NODE_TABLES,
|
|
REL_TABLE_NAME,
|
|
REL_TYPES,
|
|
EMBEDDING_TABLE_NAME,
|
|
NODE_SCHEMA_QUERIES,
|
|
REL_SCHEMA_QUERIES,
|
|
SCHEMA_QUERIES,
|
|
FILE_SCHEMA,
|
|
FOLDER_SCHEMA,
|
|
FUNCTION_SCHEMA,
|
|
CLASS_SCHEMA,
|
|
INTERFACE_SCHEMA,
|
|
METHOD_SCHEMA,
|
|
CODE_ELEMENT_SCHEMA,
|
|
COMMUNITY_SCHEMA,
|
|
PROCESS_SCHEMA,
|
|
RELATION_SCHEMA,
|
|
EMBEDDING_SCHEMA,
|
|
CREATE_VECTOR_INDEX_QUERY,
|
|
} from '../../src/core/lbug/schema.js';
|
|
|
|
describe('LadybugDB Schema', () => {
|
|
describe('NODE_TABLES', () => {
|
|
it('includes all core node types', () => {
|
|
const core = [
|
|
'File',
|
|
'Folder',
|
|
'Function',
|
|
'Class',
|
|
'Interface',
|
|
'Method',
|
|
'CodeElement',
|
|
'Community',
|
|
'Process',
|
|
];
|
|
for (const t of core) {
|
|
expect(NODE_TABLES).toContain(t);
|
|
}
|
|
});
|
|
|
|
it('includes multi-language node types', () => {
|
|
const multiLang = [
|
|
'Struct',
|
|
'Enum',
|
|
'Macro',
|
|
'Typedef',
|
|
'Union',
|
|
'Namespace',
|
|
'Trait',
|
|
'Impl',
|
|
'TypeAlias',
|
|
'Const',
|
|
'Static',
|
|
'Variable',
|
|
'Property',
|
|
'Record',
|
|
'Delegate',
|
|
'Annotation',
|
|
'Constructor',
|
|
'Template',
|
|
'Module',
|
|
];
|
|
for (const t of multiLang) {
|
|
expect(NODE_TABLES).toContain(t);
|
|
}
|
|
});
|
|
|
|
it('has expected total count', () => {
|
|
// 9 core + 19 multi-language + Route + Tool = 31
|
|
expect(NODE_TABLES).toHaveLength(31);
|
|
});
|
|
});
|
|
|
|
describe('REL_TYPES', () => {
|
|
it('includes all expected relationship types', () => {
|
|
const expected = [
|
|
'CONTAINS',
|
|
'DEFINES',
|
|
'IMPORTS',
|
|
'CALLS',
|
|
'EXTENDS',
|
|
'IMPLEMENTS',
|
|
'MEMBER_OF',
|
|
'STEP_IN_PROCESS',
|
|
];
|
|
for (const t of expected) {
|
|
expect(REL_TYPES).toContain(t);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('node schema DDL', () => {
|
|
it.each([
|
|
['FILE_SCHEMA', FILE_SCHEMA, 'File'],
|
|
['FOLDER_SCHEMA', FOLDER_SCHEMA, 'Folder'],
|
|
['FUNCTION_SCHEMA', FUNCTION_SCHEMA, 'Function'],
|
|
['CLASS_SCHEMA', CLASS_SCHEMA, 'Class'],
|
|
['INTERFACE_SCHEMA', INTERFACE_SCHEMA, 'Interface'],
|
|
['METHOD_SCHEMA', METHOD_SCHEMA, 'Method'],
|
|
['CODE_ELEMENT_SCHEMA', CODE_ELEMENT_SCHEMA, 'CodeElement'],
|
|
['COMMUNITY_SCHEMA', COMMUNITY_SCHEMA, 'Community'],
|
|
['PROCESS_SCHEMA', PROCESS_SCHEMA, 'Process'],
|
|
])('%s contains CREATE NODE TABLE for %s', (_, schema, tableName) => {
|
|
expect(schema).toContain('CREATE NODE TABLE');
|
|
expect(schema).toContain(tableName);
|
|
expect(schema).toContain('PRIMARY KEY');
|
|
});
|
|
|
|
it('Function schema has startLine and endLine', () => {
|
|
expect(FUNCTION_SCHEMA).toContain('startLine INT64');
|
|
expect(FUNCTION_SCHEMA).toContain('endLine INT64');
|
|
});
|
|
|
|
it('Function schema has isExported', () => {
|
|
expect(FUNCTION_SCHEMA).toContain('isExported BOOLEAN');
|
|
});
|
|
|
|
it('Community schema has heuristicLabel and cohesion', () => {
|
|
expect(COMMUNITY_SCHEMA).toContain('heuristicLabel STRING');
|
|
expect(COMMUNITY_SCHEMA).toContain('cohesion DOUBLE');
|
|
});
|
|
|
|
it('Process schema has processType and stepCount', () => {
|
|
expect(PROCESS_SCHEMA).toContain('processType STRING');
|
|
expect(PROCESS_SCHEMA).toContain('stepCount INT32');
|
|
});
|
|
});
|
|
|
|
describe('relation schema', () => {
|
|
it('creates a single REL TABLE named CodeRelation', () => {
|
|
expect(RELATION_SCHEMA).toContain(`CREATE REL TABLE ${REL_TABLE_NAME}`);
|
|
});
|
|
|
|
it('has type, confidence, reason, step properties', () => {
|
|
expect(RELATION_SCHEMA).toContain('type STRING');
|
|
expect(RELATION_SCHEMA).toContain('confidence DOUBLE');
|
|
expect(RELATION_SCHEMA).toContain('reason STRING');
|
|
expect(RELATION_SCHEMA).toContain('step INT32');
|
|
});
|
|
|
|
it('connects Function to Function (CALLS)', () => {
|
|
expect(RELATION_SCHEMA).toContain('FROM Function TO Function');
|
|
});
|
|
|
|
it('connects File to Function (CONTAINS/DEFINES)', () => {
|
|
expect(RELATION_SCHEMA).toContain('FROM File TO Function');
|
|
});
|
|
|
|
it('connects symbols to Community (MEMBER_OF)', () => {
|
|
expect(RELATION_SCHEMA).toContain('FROM Function TO Community');
|
|
expect(RELATION_SCHEMA).toContain('FROM Class TO Community');
|
|
});
|
|
|
|
it('connects symbols to Process (STEP_IN_PROCESS)', () => {
|
|
expect(RELATION_SCHEMA).toContain('FROM Function TO Process');
|
|
expect(RELATION_SCHEMA).toContain('FROM Method TO Process');
|
|
});
|
|
|
|
it('has all FROM/TO pairs needed for HAS_METHOD edges', () => {
|
|
// HAS_METHOD sources: Class, Interface, Struct, Trait, Impl, Record
|
|
// HAS_METHOD targets: Method, Constructor (Property is now HAS_PROPERTY)
|
|
const sources = ['Class', 'Interface'];
|
|
const backtickSources = ['Struct', 'Trait', 'Impl', 'Record'];
|
|
const targets = ['Method'];
|
|
const backtickTargets = ['Constructor'];
|
|
|
|
// Non-backtick source → non-backtick target
|
|
for (const src of sources) {
|
|
for (const tgt of targets) {
|
|
expect(RELATION_SCHEMA).toContain(`FROM ${src} TO ${tgt}`);
|
|
}
|
|
for (const tgt of backtickTargets) {
|
|
expect(RELATION_SCHEMA).toContain(`FROM ${src} TO \`${tgt}\``);
|
|
}
|
|
}
|
|
|
|
// Backtick source → all targets
|
|
for (const src of backtickSources) {
|
|
for (const tgt of targets) {
|
|
expect(RELATION_SCHEMA).toContain(`FROM \`${src}\` TO ${tgt}`);
|
|
}
|
|
for (const tgt of backtickTargets) {
|
|
expect(RELATION_SCHEMA).toContain(`FROM \`${src}\` TO \`${tgt}\``);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('embedding schema', () => {
|
|
it('creates CodeEmbedding table', () => {
|
|
expect(EMBEDDING_SCHEMA).toContain(`CREATE NODE TABLE ${EMBEDDING_TABLE_NAME}`);
|
|
expect(EMBEDDING_SCHEMA).toContain('embedding FLOAT[384]');
|
|
});
|
|
|
|
it('has vector index query', () => {
|
|
expect(CREATE_VECTOR_INDEX_QUERY).toContain('CREATE_VECTOR_INDEX');
|
|
expect(CREATE_VECTOR_INDEX_QUERY).toContain('cosine');
|
|
});
|
|
});
|
|
|
|
describe('schema query ordering', () => {
|
|
it('NODE_SCHEMA_QUERIES has correct count', () => {
|
|
expect(NODE_SCHEMA_QUERIES).toHaveLength(31);
|
|
});
|
|
|
|
it('REL_SCHEMA_QUERIES has one relation table', () => {
|
|
expect(REL_SCHEMA_QUERIES).toHaveLength(1);
|
|
});
|
|
|
|
it('SCHEMA_QUERIES includes all node + rel + embedding schemas', () => {
|
|
// 31 node + 1 rel + 1 embedding = 33
|
|
expect(SCHEMA_QUERIES).toHaveLength(33);
|
|
});
|
|
|
|
it('node schemas come before relation schemas in SCHEMA_QUERIES', () => {
|
|
const relIndex = SCHEMA_QUERIES.indexOf(RELATION_SCHEMA);
|
|
const lastNodeIndex = SCHEMA_QUERIES.indexOf(
|
|
NODE_SCHEMA_QUERIES[NODE_SCHEMA_QUERIES.length - 1],
|
|
);
|
|
expect(relIndex).toBeGreaterThan(lastNodeIndex);
|
|
});
|
|
});
|
|
});
|