2 commits

Author	SHA1	Message	Date
Gergő Magyar	df5ce1f49b	fix(ingestion): close remaining open language parsing-layer coverage gaps (#1919) (#2072) ... * fix(c): skip computed #include MACRO instead of emitting a garbage import source (F5) * fix(cpp): emit a Variable per name for structured-binding declarations (F9) * fix(dart): extract static const/final class fields (F26) * fix(dart): capture old-style function typedefs (F28) * fix(dart): read real top-level variable shape instead of a dead type field (F29) * fix(kotlin): capture callable references (F47) * fix(kotlin): anchor infix-call capture to the operator only (F49) * fix(kotlin): extract secondary constructors as members (F48) * fix(kotlin): capture destructuring declarations (F51) * fix(kotlin): index companion-object properties as fields (F52) * test(kotlin): assert callable-reference coverage runs on the worker path (F47) * fix(swift): extract protocol property requirements (F75) * fix(swift): recognize enum_class_body as a method body node (F79) * test(ingestion): rebaseline swift captures-golden + scope-capture fingerprints (#1919) * fix(kotlin): attribute secondary-constructor body calls to the Constructor node (#1919 review CF1) A Kotlin secondary constructor's body executes statements like a method body, but the registry-primary scope-resolution path had no Function scope or Constructor def for it. A call inside the body resolved its caller anchor up to the enclosing Class scope, mis-attributing the CALLS edge to the class rather than the Constructor. Add `(secondary_constructor) @scope.function` to the Kotlin scope query so the body becomes its own scope, and synthesize a `@declaration.constructor` (named `constructor`, qualified `<Class>.constructor`, with parameter metadata) so the scope owns a Constructor def that bridges to the structure-phase Constructor node. Also add an arity-disambiguating lookup key for overloadable callables: two same-name secondary constructors of different arity (e.g. a zero-arg vs a 2-arg) share the qualified key whose first-write-wins assignment is source-order- dependent — so a zero-arg overload could resolve to a sibling. The structure node id encodes `#<arity>`; mirror that in the bridge keyspace and match by the def's parameterCount. Same-arity overloads collapse onto one arity key exactly as before, so no regression there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(kotlin): do not own function-local property bindings under the enclosing class (#1919 review CF3) Kotlin emits destructuring / loop bindings (`val (a,b) = pair`, `for ((k,v) in m)`) as `@definition.property` to dodge the block-scope local-symbol pruner. When such a binding sits inside a method body of a class, the structure-phase owner walk found the enclosing class and emitted a spurious HAS_PROPERTY edge (e.g. `C -> k`), treating a function-local as a class member. Guard the Property owner resolution: if a function-like ancestor is reached before any class container, the property is function-local and gets no owner edge (it falls back to a File DEFINES edge). Language-agnostic — genuine class fields sit directly in the class body with no intervening function, so they keep their HAS_PROPERTY owner edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(kotlin): guard non-companion property isStatic=false (#1919 review CF4) Add a field-extraction case for a plain non-companion class `class C { val x: Int = 1 }` asserting the property `x` has isStatic=false, guarding the `isInsideKotlinCompanion` walk against false-positives. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(kotlin): dedup type_identifier lookup in extractOwnerName (#1919 review CF5) The `node.namedChildren.find(c => c.type === 'type_identifier')?.text` lookup was duplicated across the companion and non-companion branches of the Kotlin field-extractor's extractOwnerName. Hoist it into a single local, preserving the existing behavior (anonymous companion falls back to "Companion"; other nodes prefer the `name` field, else the type_identifier text, else undefined). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dart): capture generic old-style function typedefs (#1919 review CF2) * test(dart): guard multi-name field count and top-level-var labels (#1919 review CF4) * docs(swift): correct isStatic comment re multi-modifier hasKeyword (#1919 review CF5) * test(ingestion): rebaseline dart+kotlin scope-capture fingerprints after review remediation (#1919) * fix(ingestion): correct CF3 owner-strip boundary set for accessor/init bodies and Dart signatures (#1919 review) The CF3 property-ownership guard used FUNCTION_NODE_TYPES, which (a) includes Dart bare signatures (function_signature/method_signature) — over-stripping every Dart class getter/setter's HAS_PROPERTY owner — and (b) omits Kotlin anonymous_initializer/getter/setter and Swift computed accessors — under- stripping destructuring/locals inside init{} and accessor bodies, emitting spurious Class->local HAS_PROPERTY edges. Introduces a guard-specific LOCAL_SCOPE_BODY_NODE_TYPES set (signatures excluded, accessor/init bodies included). Adds Dart accessor-ownership + Kotlin init/accessor destructuring regression fixtures. Both confirmed on the worker pipeline; no cross-language regression (1597 cross-language tests green). --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-08 08:09:43 +01:00
Gergő Magyar	f0132c1077	feat: Phase 6 type resolution — for-loop Tier 1c, pattern matching, container descriptors, 10-language coverage (#318) ... * feat: Phase 6 type resolution — pattern matching, for-loop Tier 1c, coverage completion - Add patternBindingNodeTypes gate to LanguageTypeConfig for 50% perf improvement - Expand ForLoopExtractor signature with optional declarationTypeNodes + scope - Add extractElementTypeFromString shared utility for container type parsing - Python match/case: extractPatternBinding for `case User() as u:` pattern - C# refactor: move is_pattern_expression from extractDeclaration to extractPatternBinding - Ruby: add extractPendingAssignment for assignment chain propagation - TS/JS: add for-loop Tier 1c for `for (const user of users)` with User[] inference - Python: add for-loop Tier 1c for `for user in users:` with type annotation inference - Go: add for-loop Tier 1c for `for _, user := range users` with []User inference - Fix 'Property' as any stale cast in call-processor.ts - Add dual return-type string length cap (2048 pre-cap, 512 post-cap) - Add chain call integration tests for C#, Go, Rust, Python, JS, C++ - Add Python match/case integration test fixtures - 27 new extractElementTypeFromString unit tests - 3 for-loop edge cases skipped (declarationTypeNodes scope key lookup) * fix: address code review findings for Phase 6 - Add missing patternBindingNodeTypes to C# typeConfig (perf gate) - Add 2048-char input length guard to extractElementTypeFromString - Skip Python match/case integration tests (call extraction needs query updates) * reorganise * fix: Phase 1 bug fixes — Go range semantics, typed_parameter, bracket depth - Go single-var range correctly returns early for slices/maps (index, not element) - Go single-var range on channels correctly resolves element type - Added map_type and channel_type to extractGoElementTypeFromTypeNode - Added isChannelType helper for channel detection before skip decision - Added 'typed_parameter' to TYPED_PARAMETER_TYPES for Python annotated params - Fixed bracket depth tracking in extractElementTypeFromString — only match selected closeChar at depth 0, return undefined for mismatched brackets - Un-skipped 3 prematurely skipped tests (TS local const, Python List/Sequence) - Added tests for map range, single-var range semantics, bracket edge cases * refactor: Phase 2 architecture — shared helper, required params, decoupled type nodes - Extract resolveIterableElementType shared helper in shared.ts implementing 3-strategy fallback (declarationTypeNodes → scopeEnv string → AST walk) - Refactor TS, Python, Go extractors to use shared helper (eliminates 3x duplication) - Make ForLoopExtractor params required (aligned with PatternBindingExtractor) - Update Java, Kotlin, C# extractor signatures to accept required params - Decouple declarationTypeNodes from scopeEnv — capture raw type annotation nodes BEFORE extractDeclaration for container types (User[], []User, List[User]) - Hybrid approach: direct name extraction + keysBefore fallback for multi-declarator - Document declarationTypeNodes invariant change (superset of scopeEnv) * feat: Phase 3 partial — Rust for-loop + C# var foreach Tier 1c - Rust: add extractForLoopBinding with for_expression support - Handles &users, &mut users via reference_expression unwrapping - extractRustElementTypeFromTypeNode: generic_type, reference_type, slice/array - findRustParamElementType: AST walk with reference/mut pattern unwrapping - 4 unit tests (Vec<User>, &[User], range expr negative, no-annotation negative) - C#: upgrade foreach to handle var (implicit_type) via Tier 1c - extractCSharpElementTypeFromTypeNode: generic_name, array_type, nullable_type - findCSharpParamElementType: AST walk to method_declaration parameters - 3 unit tests (var foreach, explicit type regression, no-annotation negative) * feat: Phase 3 complete — all language gaps + pattern matching Kotlin Tier 1c: - Unannotated for-loop resolves via shared helper - extractKotlinElementTypeFromTypeNode handles type_projection unwrapping - findKotlinParamElementType walks to function_declaration Java Tier 1c: - var foreach resolves via shared helper - extractJavaElementTypeFromTypeNode handles generic_type, array_type - findJavaParamElementType walks to method_declaration TypeScript: - readonly User[] unwrapped via readonly_type → array_type recursion C# switch patterns: - declaration_pattern added to patternBindingNodeTypes - extractPatternBinding handles standalone declaration_pattern (switch case/expr) Rust match arms: - match_arm added to patternBindingNodeTypes - extractPatternBinding extended with match_arm → match_expression parent traversal Python: - as_pattern tries childForFieldName('alias') before positional fallback Tests: 237 pass (was 224), 13 new tests added * feat: Phase 4 — known limitation tests, match arm fix, final verification - Fix Rust match_arm pattern extraction: unwrap match_pattern to get tuple_struct_pattern inside (tree-sitter-rust wraps in match_pattern node) - Add first-writer-wins regression test for match arm scope leakage - Add 5 documented skip tests for known limitations: - TS destructured for-of (tuple destructuring) - Python tuple unpacking in for-loops - TS instanceof narrowing (block-level scoping) - Rust for with .iter() (method call iterable) - Ruby block parameters (closure param inference) Final: 238 passed, 5 skipped (documented limitations), tsc clean * test: integration tests for all Phase 6 language gaps + fix Rust param pattern field Integration test fixtures and tests (30 new tests, all with exact match + negative): Rust for-loop (5 tests): - for user in &users with Vec<User> → User#save, negative Repo#save - for repo in &repos with Vec<Repo> → Repo#save, negative User#save Rust match arm (5 tests): - match opt { Some(user) => user.save() } → User#save, negative Repo#save - if let Ok(repo) = res → Repo#save, negative User#save C# var foreach (5 tests): - foreach (var user in users) with List<User> → User#Save, negative Repo#Save - foreach (var repo in repos) with List<Repo> → Repo#Save C# switch pattern (4 tests): - is User user → User#Save, case Repo repo → Repo#Save Kotlin unannotated for (4 tests): - for (user in users) with List<User> → user.save, negative repo.save Go map range (3 tests): - for _, user := range userMap with map[string]User → User#Save, negative TypeScript readonly (4 tests): - for (const user of users) with readonly User[] → user.save, negative Bug fix: type-env.ts parameter branch now falls back to childForFieldName('pattern') for Rust parameters (Rust uses 'pattern' not 'name' for parameter names) * test: add assertion bodies to known limitation skip tests Convert empty skip test stubs to proper tests with parse/buildTypeEnv/expect assertions following the codebase convention (e.g., call-processor.test.ts:319). Each skip test now documents the exact expected behavior, so removing .skip will cause a meaningful failure when the limitation is eventually fixed. Also clarify Python integration skip tests as call-extraction issues (not type-env) and Swift integration skips as build-dep issues (self/super resolution code already exists in type-env.ts). * feat: resolve 4 known limitation skip tests + method-aware type arg selection Unskip 4 of 5 type-env known limitations with full integration test coverage: 1. TS destructured for-of: handle array_pattern by binding last named child to element type. Fix Map<K,V> to return last generic arg (value type). 2. Python dict.items() loop: handle `call` iterables + `pattern_list` left side. Fix dict[K,V] extraction via type_parameter with last-arg heuristic. Unwrap `type` wrapper in extractPyElementTypeFromAnnotation. 3. TS instanceof narrowing: add extractPatternBinding for binary_expression with positional child access. First-writer-wins (not block-scoped). 4. Rust .iter() for-loops: handle call_expression in for_expression value node by extracting receiver from field_expression. Method-aware type arg resolution: - Add TypeArgPosition ('first'|'last') to resolveIterableElementType - .keys()/.keySet()/.Keys → first type arg (key); all else → last (value) - Thread position through all 3 strategy callbacks in TS/Rust/Python - Add predefined_type to extractSimpleTypeName for TS primitives (string etc) New fixtures: rust-iter-for-loop, typescript-destructured-for-of, typescript-instanceof-narrowing, python-dict-items-loop. 248 unit tests pass (6 new), 1 skip (Ruby block params). * feat: container descriptor table for generic type arg resolution Replace simple KEY_METHODS heuristic with CONTAINER_DESCRIPTORS table that maps 30+ container types across all languages to their type parameter semantics per access method. Key improvements: - Container-aware resolution: HashMap.iter() correctly yields V (arity 2), while Vec.iter() yields T (arity 1) — same method, different semantics - Cross-language coverage: Map/HashMap/BTreeMap/dict/Dict/Dictionary/ ConcurrentHashMap + List/Vec/Set/HashSet/Queue/Deque/Stack etc. - Method categorization: keyMethods (keys/keySet/Keys) vs valueMethods (values/get/pop/iter/first/last) per container type - Fallback for unknown containers: still uses method name heuristic, so MyCache<K,V>.keys() correctly returns first arg - Exported getContainerDescriptor() for future heritage-chain lookups Each language extractor now passes containerTypeName from scopeEnv to methodToTypeArgPosition for descriptor-aware resolution. 252 unit tests pass (4 new descriptor tests), 1 skip (Ruby). * feat: method-aware for-loop extractors + integration tests for all languages Upgrade 4 existing extractors + create 3 new ones for full cross-language coverage of call_expression iterables and container descriptor resolution: Upgraded (add call expr iterable + methodToTypeArgPosition): - Java: method_invocation (data.keySet(), data.values()) - Kotlin: navigation_expression + call_expression (data.keys, data.values()) - C#: member_access_expression + invocation_expression (data.Keys, data.Values) - Go: TypeArgPosition threading for Go 1.18+ generics New for-loop extractors: - C++: for_range_loop with auto& unwrapping, template_type + qualified_identifier (std::vector<User>) extraction, explicit vs auto type handling - PHP: foreach_statement with simple/key-value/by-reference forms, PHPDoc @param priority over AST array type - Ruby: for-in with YARD @param type resolution via comment parsing Integration test fixtures + tests for all 6 languages: - java-map-keys-values (Map.values() + List iteration) - kotlin-map-keys-values (HashMap.values + List iteration) - csharp-dictionary-keys-values (Dictionary.Values foreach) - cpp-range-for (auto& + const auto& range-based for) - php-foreach-loop (foreach with PHPDoc @param User[]) - ruby-for-in-loop (for-in with YARD @param Array<User>) Bugs fixed during integration testing: - C++: qualified_identifier (std::vector) not unwrapped to template_type - PHP: extractParameter overwrote PHPDoc-derived types with bare 'array' 252 unit tests pass, 201 integration tests pass across 6 languages. * fix: update extractElementTypeFromString tests for last-arg default TypeArgPosition change (default 'last') broke 5 existing tests expecting first arg from multi-arg generics. Updated expectations and added explicit pos='first' tests for key type extraction. * fix: rename C++ fixture files to correct case for case-sensitive CI On case-sensitive filesystems (Linux/macOS CI), git tracked both the old lowercase files (app.cpp, user.h) and the new uppercase files (App.cpp, User.h) as separate files. The pipeline processed both, causing the old app.cpp (with explicit User& type) to interfere with the new auto& test. Removes old lowercase entries and re-adds with uppercase casing to match the #include directives in the fixture. * feat: PR #318 review findings — pattern bindings, member access iterables, structured bindings Address all 7 genuine gaps identified in PR #318 deep code review: - Kotlin: add extractKotlinPatternBinding for when/is (type_test AST node) with allowPatternBindingOverwrite for smart-cast semantics - Java: add type_pattern branch for Java 17+ switch pattern variables - TypeScript: explicit object_pattern skip in for-of (no false bindings) - Cross-language: member access iterables (self.users, this.users, repo.users) across all 10 language extractors - C++: structured_binding_declarator handling in range-for (last-child heuristic) - Rust: closure_parameter added to TYPED_PARAMETER_TYPES - PHP: normalizePhpType handles angle-bracket generics (Collection<User>) Code review fixes applied: - Remove 4 debug console.log statements (c-cpp.ts, call-processor.ts) - Hoist KNOWN_CONTAINER_PROPS to module scope (csharp.ts) - Guard keysBefore allocation behind typeNode check (type-env.ts) - Add depth limits (50) to 7 recursive type extraction functions - Add 2048-char length cap to extractSimpleTypeName - Fix PHP/Ruby missing typeArgPos parameter in resolveIterableElementType Integration test fixtures: kotlin-when-pattern, java-switch-pattern, cpp-structured-binding, typescript-member-access-for-loop, python-member-access-for-loop * fix: position-indexed when/is bindings, Kotlin param extraction, HashMap.values for-loop Three root causes for failing Kotlin integration tests: 1. When/is multi-arm resolution: flat scopeEnv stored only the last arm's type (last-writer-wins). Added PatternOverrides with AST range indexing so each when arm resolves to its narrowed type independently. 2. HashMap.values for-loop: navigation_expression without call_suffix was classified as bare property access (iterableName='values' instead of 'data'). Now tries object-as-iterable + property-as-method first, with fallback to property-as-iterable for this.users patterns. 3. Kotlin parameter extraction: tree-sitter-kotlin parameter nodes use positional children (simple_identifier, user_type) not named fields (name, type). Added fallback to findChildByType in both extractKotlinParameter and extractTypeBinding. Integration tests added for .keys/.values/Set/MutableMap iteration, 3-arm when/is, multi-call within arms, and when+else branch. * feat: enhance PHP type resolution for generics and member access in foreach loops * feat: Phase 6.1 type resolution gap closure — container descriptors, recursive_pattern, class fields Add 13 missing container type descriptors (Collection, MutableMap, Stream, SortedSet, etc.) to CONTAINER_DESCRIPTORS for correct element type extraction across C#, Kotlin, and Java. Extend C# pattern binding to handle recursive_pattern (obj is User { Name: "Alice" } u) in both is-expression and switch expression contexts. Add TypeScript class field declaration support (public_field_definition) so for-loop iteration over this.fieldName resolves element types from class field type annotations. Includes file-scope fallback in resolveIterableElementType and nested member_expression handling for this.field.method() patterns. * docs: add type resolution system documentation with roadmap Covers the full architecture, resolution tiers (0-2), scope model, language feature matrix, container descriptors, pipeline integration, and the Phase 7-9 roadmap for cross-scope propagation, field-type resolution, and return-type-aware binding. * feat: Phase 6.2 review findings — C# nested member foreach, C++ deref range-for, Java field_access Close two gaps found during fourth-pass review of PR #318: - C# foreach (var user in this.data.Values): nested member_access_expression now extracts intermediate property name for scopeEnv lookup - C++ for (auto& user : *ptr): pointer_expression dereference now recognized as range-for iterable Root causes fixed in shared infrastructure: - extractSimpleTypeName: add template_type (C++) and generic_name (C#) - extractGenericTypeArgs: add generic_name for consistency - type-env.ts: unwrap variable_declaration wrapper in field_declaration for declarationTypeNodes capture (zero-allocation manual loop) Additional review findings addressed: - Java: add field_access handler for this.data.values() in method_invocation - C++ pointer_expression: document limitation (*identifier only) - TypeScript: fix stale comment about property_identifier All 525 tests pass (278 unit + 247 integration). * perf: optimize type resolution pipeline — worker threshold, skip graph phases, AST pruning - Skip worker pool creation for small repos (<15 files or <512KB) — saves 100-400ms - Add skipGraphPhases option to runPipelineFromRepo to skip MRO/community/process phases - Add conservative SKIP_SUBTREE_TYPES for leaf-only AST nodes (string, comment, number) - Pre-compute interestingNodeTypes set — single Set.has() replaces 3 checks per node - Add fastStripNullable — skip full stripNullable for simple identifiers (90%+ case) - Replace .children?.find() with manual for loops in extractFunctionName (no array alloc) - Add hookTimeout: 120000 to vitest.config.ts for CI beforeAll hooks * fix: review findings — remove template_string from SKIP_SUBTREE_TYPES, handle bare nullable keywords - Remove template_string and concatenated_string from SKIP_SUBTREE_TYPES (template literals contain interpolated expressions with typed code) - Add FAST_NULLABLE_KEYWORDS check to fastStripNullable for behavioral parity with stripNullable on bare null/undefined/void/None/nil - Add explanatory comment on extractPendingAssignment scopeEnv guard * feat: add type resolution system and roadmap documentation	2026-03-17 17:10:22 +00:00