* test(go): pin calls through an interface-typed struct field (#2813)
A call through an interface-typed struct field never reaches the
implementation: the CALLS edge stops at the interface DECLARATION, so
`impact()` on the implementing method reports 0 callers. This commit adds
the executable statement of that defect; the fixes follow.
Two stacked defects produce it, and either alone is enough to reproduce —
which is why no existing fixture could observe it:
D1 `buildDetectionIndexes` skips every POINTER-receiver method, so a
struct whose methods are all `func (r *T)` has an empty method set,
structurally satisfies nothing, and gets no IMPLEMENTS edge. Go's
rule is that the method set of *T includes pointer-receiver methods,
and idiomatic Go stores *T in an interface-typed field.
D2 Case 0 (compound receiver) emits its primary edge and short-circuits
without the interface-dispatch fan-out Case 4 performs. A struct
field receiver `s.orderRepo` contains a dot and so always takes
Case 0; a local or parameter receiver is a bare name and reaches
Case 4.
Every implementor in both pre-existing structural-dispatch fixtures uses a
VALUE receiver, and the one pointer-receiver type is pinned as a negative
(`not.toContain('PointerOnlyThing -> PointerOnly')`), so the corpus could
not see D1 by construction. The new fixture is pointer-receiver
throughout, cross-package, and carries concrete-field controls in the same
structs.
Failing-first, verified against this tree: 7 of the 11 new assertions fail
and 4 pass. The 4 that pass are exactly the controls that must not
regress — the primary edge to the interface declaration, the concrete-field
call, the absence of fan-out on a concrete field, and the partial-signature
negative — so the suite discriminates rather than merely failing.
Two recorded artifacts move here because the FIXTURE was added, not
because capture output changed:
- test/fixtures/go-captures-golden/expected-captures.json — regenerated
additively (32 insertions, 0 deletions).
- bench/scope-capture/baselines.json — go fingerprint, fixture_count
102 -> 110.
Both are regenerated in this commit rather than deferred to the end of the
series: the fixture is their only cause, no later commit touches capture
emission, so they cannot re-drift and every commit stays green. The check
that this is corpus growth and not a capture regression is that go was the
only one of 15 language fingerprints to move on the same run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(go): count pointer-receiver methods toward structural interface satisfaction (#2813)
D1 of two stacked defects. `buildDetectionIndexes` skipped every method whose
receiver is a pointer, so a struct declaring `func (r *OrderRepo) DeleteItem(...)`
had an EMPTY method set, structurally satisfied nothing, and produced no
IMPLEMENTS edge at all.
Go's method-set rule is per-type, and there are two types involved: the method
set of `T` holds only value-receiver methods, while the method set of `*T` holds
both. #1966 implemented the `T` reading, which is exactly right for `T` — and
leaves `*T` permanently empty. GitNexus models one Struct node per type with no
separate `*T` node, so only one of the two can be represented, and the `T`
reading is the one idiomatic Go almost never uses: methods take pointer
receivers so they can mutate, and `*T` is what gets stored in an interface-typed
field.
The cost was silence rather than caution. With no IMPLEMENTS edge, a call
through an interface-typed field resolved to the interface DECLARATION and
`impact()` on the implementing method returned 0 callers — byte-identical to a
symbol that genuinely has none, which is what made the reporter's blast-radius
check unusable rather than merely incomplete.
This picks the `*T` reading: the graph now answers "which types provide this
interface's behaviour", and no longer proves `var x I = T{}` invalid. The trade
is deliberate and was checked against every consumer of IMPLEMENTS before being
made — MRO/METHOD_IMPLEMENTS derivation, community clustering, the
receiver-dispatch fan-out index, and the epistemic heritage probe. None performs
value-assignability checking.
Two negative pins encoded the #1966 decision and are REVERSED here rather than
deleted, each keeping a comment that explains why the polarity moved:
- go.test.ts: `PointerOnlyThing -> PointerOnly` now expected to be emitted.
- go-hooks.test.ts: the pointer-receiver-only unit case now expects the
implementor instead of `undefined`.
`goReceiverKind` is still stamped in method-owners.ts — it is the hook a future
value/pointer-aware model would read — but is deliberately no longer a filter.
Its now-dead local predicate and type alias are removed so the file no longer
carries a helper asserting the reverted rule.
Measured on the #2813 fixture, this commit alone: the two IMPLEMENTS assertions
flip to passing (6 pass, up from 4) while the five interface-dispatch fan-out
assertions still fail — those are D2, fixed in the next commit. Keeping the two
commits separate is what makes that attribution visible.
Go unit suite: 91 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(resolution): fan out interface dispatch from a compound receiver (#2813)
D2 of two stacked defects, and the one that closes the issue. Case 0
(compound receiver) emitted its primary edge and short-circuited without the
interface-dispatch fan-out that Case 4 performs, so a call whose receiver is a
struct FIELD stopped at the interface's method DECLARATION and never reached
any implementation.
The gap was a property of receiver SYNTAX rather than of types. Case 0 is
selected by `receiverName.includes('.')`, so a field receiver (`s.orderRepo`)
always lands there, while the very same interface reached through a local or a
parameter is a bare name and falls through to Case 4 — which fans out
correctly. Field-held interfaces, i.e. dependency injection, were the half that
silently lost every implementation edge; the pre-existing fixtures exercise the
local and parameter forms only, which is why the suite was green.
The fix is the call Case 4 already makes, placed after Case 0's primary
`tryEmitEdge` and before its `handledSites.add`. It stays language-agnostic
(AGENTS.md section 42): `emitInterfaceDispatchFor` self-gates on
`ownerDef.type !== 'Interface'`, so a receiver that folds to a Struct emits
nothing extra and no language check is needed. Confidence is Case 0's own 0.85
literal, not Case 4's site.kind-dependent value — Case 0 has no read/write arm
to mirror.
The case ladder itself is untouched: invariant I4 in contract/scope-resolver.ts
makes the ordering a contract, so the fan-out is added INSIDE Case 0 rather
than by reordering or merging cases.
Also flips a second, previously unnoticed encoding of the #1966 value-only
reading that the full sweep surfaced: the exact-set assertion at
go.test.ts:361 enumerates every structural IMPLEMENTS edge, and D1 correctly
adds `PointerOnlyThing -> PointerOnly` to it. It is D1 fallout rather than D2's,
but D1 had already landed; recording it here with its reason beats amending a
commit whose separate measurability is the point.
Measured:
- #2813 suite: 11 of 11 pass (was 7 failing after D1 alone, which fixed only
the two IMPLEMENTS rows).
- go.test.ts: 160 passed.
- Full cross-language sweep, test/integration/resolvers: 3027 passed,
1 skipped, across 52 files. The single failure in that run was the
exact-set assertion above, fixed here; no other language regressed.
`detect_changes` rates this HIGH (6 affected flows, all EmitReceiverBoundCalls
at step 1) — inherent to editing a hub symbol in the resolution pipeline. The
sweep above is the empirical answer to that label.
An existing index must be re-analyzed to show the new edges; this changes what
the resolver produces, not how it is stored, so no SCHEMA_BUMP applies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(go): pin the heritage edges that make impact() hedge an interface-bound count (#2813)
The epistemic half of the issue, resolved by MEASUREMENT rather than by new
code, and pinned at its mechanism.
The reporter's disqualifying complaint was that `impact()` reported
`impactedCount 0, epistemic "exact", risk LOW` for a method reachable only
through an interface-typed field — byte-identical to what it reports for a
symbol that genuinely has no callers. A zero therefore could not be used
defensively, which was the entire use case.
That verdict comes from `computeEpistemicBoundary`, which has two producers and
neither fired: the call sites were not DROPPED (they resolved, just to the
interface declaration, so the #2744 receiver-typing producer saw nothing), and
its heritage probe walks IMPLEMENTS/METHOD_IMPLEMENTS edges out of the queried
symbol — of which there were none, because the pointer-receiver exclusion (D1)
meant no such edge was ever emitted.
Restoring those edges fixes the epistemics as a side effect, so the planned
conditional change to local-backend.ts is NOT needed. Measured on this fixture
against the fixed tree:
impact(OrderRepo.DeleteItem, upstream)
before: impactedCount 0, epistemic "exact"
after: impactedCount 3, epistemic "lower-bound", with an interface
boundary note; the three callers are OrderHandlers.Delete,
PickService.StartSession and WaveService.Release — all correct.
impact(CartRepo.Get, upstream) [concrete receiver, no interface]
after: impactedCount 1, epistemic "exact"
The second row is the one that matters for trust: the hedge discriminates
instead of firing on everything, so "exact" still means exact.
This test asserts the METHOD_IMPLEMENTS edges the probe walks. Pinning the
mechanism keeps the resolver suite from reaching into the MCP layer while still
failing loudly if the edges regress; the impact() numbers above are recorded in
the commit message and PR body rather than re-asserted here.
#2813 suite: 12 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(go): model Go method sets exactly so interface satisfaction is decidable (#2813)
Replaces the approximate structural-interface model with the rules the Go spec
actually defines, so the graph answers what the compiler answers instead of a
useful-but-wrong summary of it. Three answers were provably wrong before; all
three are now exact and covered.
Method sets (go.dev/ref/spec#Method_sets):
MS(T) = methods declared with receiver T
MS(*T) = methods declared with receiver *T OR T
Promotion (#Struct_types):
S embeds T -> MS(S) and MS(*S) get promoted methods with receiver T;
MS(*S) ALSO gets those with receiver *T
S embeds *T -> MS(S) AND MS(*S) both get receiver T or *T
Identifier identity (#Uniqueness_of_identifiers): "Two identifiers are different
if they are spelled differently, OR IF THEY APPEAR IN DIFFERENT PACKAGES AND ARE
NOT EXPORTED."
func (b *Base) Ping() // pointer receiver
type ByValue struct{ Base }
type ByPointer struct{ *Base }
type before exact answer
Base IMPLEMENTS only *Base implements
ByValue IMPLEMENTS only *ByValue implements
ByPointer IMPLEMENTS the VALUE type implements
All three were the same edge. Two of the three were wrong, and nothing in the
graph could tell them apart.
Worse, in a different direction:
package sealed; type Sealed interface { seal() }
package foreign; func (t *T) seal() {}
`foreign.T` cannot implement `sealed.Sealed` in Go — `seal` is unexported, so the
two identifiers are DIFFERENT. Matching on the bare name emitted a FALSE
IMPLEMENTS edge, and the interface-dispatch fan-out then turned it into an
impossible CALLS edge. That is the entire basis of the sealed-interface idiom.
- `methodSetKey` qualifies UNEXPORTED method names with their declaring package,
leaving exported names unqualified (which is what makes cross-package
satisfaction work at all). Exactness, not a heuristic: the sealed case now
emits no edge, while the legitimate same-package implementor is retained.
- `collectStructMethodEntries` builds MS(T) and MS(*T) together and applies the
promotion table above. The embed FORM is load-bearing, so it is now captured:
`@reference.embedded-pointer` records `*T` versus `T`, which the parser
previously discarded (the `*` is an unnamed token).
- Detection returns `{ structDefId, receiverForm }`. `receiverForm: 'pointer'`
means the value type does NOT implement and only `*T` does — the fact
`var x I = T{}` turns on.
- The form rides in the edge `reason` (`-structural-implements-pointer`).
Relationships carry no arbitrary properties, so a new field would change the
relation DDL, move SCHEMA_FINGERPRINT and force a rebuild for a fact a string
already expresses. Value-form implementors keep the ORIGINAL unsuffixed
reason, so a consumer matching the old string now sees exactly the assignable
set — which is what that string always claimed to mean.
- `emitInterfaceDispatchFor` walks the SUBTYPE CLOSURE (IMPLEMENTS + EXTENDS) and
skips bodiless declarations, instead of stopping at depth 1. Two reproduced
Java shapes emitted an edge to a second abstract declaration while the only
class with a body got nothing: a sub-interface that re-declares the method, and
an abstract base between interface and implementation. Both now reach the
implementation and neither emits the declaration edge.
- The fan-out is bounded by `MAX_INTERFACE_DISPATCH_FANOUT` (32,
`GITNEXUS_MAX_INTERFACE_DISPATCH_FANOUT`) and reports what it dropped, mirroring
`MAX_PROPERTY_DISPATCH_FANOUT`. A bare cap would silently discard valid dispatch
targets, which is the same false-safe silence this issue is about.
- Corrects a rationale comment that was factually wrong about the code 70 lines
above it (Case 0 DOES branch on `site.kind`, at :713-716; what it lacks is a
read/write branch in its reason/confidence computation).
- Updates both copies of the case-ladder contract, which still described the
fan-out as Case-4-exclusive.
The embed-pointer marker is PARSE-TIME capture emission, so a warm cache would
replay the pre-marker capture set and the distinction would never appear —
silently, the v27/v30 failure mode. 43 and not 40 because origin/main allocated
40, 41 and 42 while this branch was in review, which is exactly the window this
file's history records both prior EXACT clashes landing in. Pin moved with it.
RE-CHECK AGAINST origin/main IMMEDIATELY BEFORE MERGE.
- Go unit: 93 passed, including new rows pinning that `populateGoOwners` stamps
`goReceiverKind` (previously the field had no reader and could rot silently)
and that a pointer-receiver-only type implements in POINTER form only.
- Cross-language sweep, test/integration/resolvers: 3034 passed, 1 skipped,
52 files, zero regressions.
- scope-capture bench: PASS (15 languages). Go is the ONLY fingerprint that
moved, which is the check that this is a Go capture change and not a
cross-language regression; rebaselined with rationale.
- Also closes review gaps in this PR's own tests: the concrete-field control was
vacuous with respect to the type gate (repointed at a struct that IS an
implementor), the two-service-file row could not distinguish the two files it
is named for (both ends now file-qualified), plus new rows for signature
mismatch, emitted confidence, and an exact N-by-M fan-out bound.
An existing index must be re-analyzed; the schema bump forces it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(helpers): add the shared temp-repo lifecycle helper
`createTempDirPool` gives a suite one owner for its temp fixture repos —
create on demand, remove them all in one `afterAll` — instead of a hand-rolled
mkdtemp/rmSync pair per file. The PDG receiver pin added in the next commit
uses it.
Cherry-picked verbatim from ec36c6dda on the #2802 branch, where it was
extracted to collapse five hand-rolled cleanups. Identical content, so if both
branches land the add resolves as a duplicate rather than a divergence.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(typescript): type a class field from its initializer so it can be a receiver
A field whose type had to be inferred from its initializer produced no CALLS
edge at all — not a truncated chain, nothing. `this.p.inner().compute(x)` lost
`Outer.inner` too, an ordinary named-receiver call, because `typeOfMemberOnClass`
found no `typeBindings` entry for `p` and `foldReceiverChain` declines at its
first untypeable step rather than folding on a guessed owner.
The initializer was never invisible: `new Outer()` emitted its own constructor
edge exactly as the annotated twin does. What was missing was the step turning
that initializer into a TYPE BINDING, i.e. capture patterns for the two shapes
the query never covered:
private p = new Outer(); // public_field_definition value:
private p; constructor() { this.p = new … } // this.<field> = new …
Both are `@type-binding.constructor`, so `annotation` still outranks them in
`typeBindingStrength` and an annotated field keeps resolving through its
annotation. The assignment form carries a narrow `@type-binding.this-field`
marker on its `(this)` node — anchorCaptureFor takes the broadest range, so the
statement stays the anchor — which `tsBindingScopeFor` reads to hoist the
binding onto the Class scope, the only place `typeOfMemberOnClass` looks. The
marker must stay specific to that pattern: hoisting every constructor-inferred
binding would move method-local `const o = new Outer()` out of its own scope.
Kotlin and Swift needed no such pattern for the initializer form because one
grammar node (property_declaration) covers both a local and a stored property;
TypeScript splits them, and only the local half was ever covered.
Both self-diffing pins flip and gain rows: a method-assigned field, and a
deliberately mistyped `private p: Mismatch = new Outer()` that asserts the
source-strength tie-break executably. That row also pins a pre-existing
artifact — `Inner.compute` still resolves through the hoisted module-level
return-type binding — verified byte-identical on the pre-fix tree.
Fixes#2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(javascript): type a class field from its initializer so it can be a receiver
JavaScript has no field annotations at all, so a class field's type can only
ever come from its initializer — which made this the strictly worse half of
#2807: `class C { p = new Outer(); }` gave `this.p` no type, and
`this.p.inner()` emitted nothing.
`synthesizeConstructorFieldBindings` in captures.ts already covered the sibling
shape, `this.p = new Outer()`, which is why THAT row resolved — but it only
walks `constructor` bodies, so a field initialized at its declaration matched
no pattern anywhere.
Adds the `field_definition` + `value: (new_expression)` patterns (the JS grammar
names the field `property:`, not `name:`), anchored so the binding lands in the
class body scope where `typeOfMemberOnClass` reads it. No hook change needed:
`jsBindingScopeFor` already delegates to `tsBindingScopeFor`, so it inherits the
`@type-binding.this-field` branch too.
Measured: `InferredField.run` now emits `Outer.inner`, exact parity with both
the local-const control and the constructor-assigned row. The second chain link
(`Inner.compute`) stays absent in ALL THREE rows — that is JavaScript's separate
return-type-inference gap, not this one.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(python): infer an instance field's type from the constructor it calls
`self.outer = Outer()` in `__init__` bound nothing, so `self.outer.inner()`
had no receiver type and the fold declined the whole chain — the Python half
of #2807. An annotated field (`self.outer: Outer = ...`) or one assigned from
an annotated parameter already worked.
`synthesizeConstructorFieldTypeBindings` deliberately refused to infer "from
arbitrary unannotated RHS expressions ... not a name-only guess". A CALL is not
that: Python has no `new`, so a call to a plain (or dotted) name is the only
syntactic construction form there is, and it is the same positive evidence
every other language reads from `= new X()`. A bare name, subscript, await or
comprehension is still refused.
Adds it as a THIRD and weakest tier. The existing explicit/parameter boolean
becomes a rank, so precedence is now explicit annotation > parameter annotation
> construction, and a later same-tier assignment still wins (the last write in
`__init__` is the live one). `interpretPythonTypeBinding` maps the new marker to
`constructor-inferred` (strength 1) — checked before the parameter branch, which
would otherwise have read the absent parameter marker as `annotation` and
promoted a guess to the strongest tier.
The Class-scope hoist needed no change: `@type-binding.instance-field` already
carries it in `pythonBindingScopeFor`.
Measured: `AssignedField.run` now emits `Outer.inner`, exact parity with the
annotated-field and local-const rows.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ruby): infer an instance variable's type from the constructor it calls
`@service = UserService.new` in `initialize` bound nothing, so `@service.inner`
had no receiver type and the fold declined the whole chain — the Ruby half of
#2807. An instance variable is the ONLY way a Ruby object gets a field, and
Ruby has no annotations, so this was the single shape that could have worked
and did not: the existing constructor-inferred patterns bind a local
(`x = Foo.new`) and a constant (`SERVICE = Foo.new`), never an ivar.
Adds the plain and `Foo::Bar` qualified ivar forms. `@type-binding.name` is
captured on the `instance_variable` node so the bound name keeps its `@` sigil
and matches the receiver text at the call site verbatim — the resolver compares
spellings, and `service` would never have matched `@service`.
`rubyBindingScopeFor` gains a Class hoist gated on a narrow
`@type-binding.ivar-field` marker riding the same node: an ivar declares a field
of the enclosing class, so the binding must live on the Class scope or no other
method can see it. Gated on the dedicated marker, never on
`@type-binding.constructor` at large, which also fires for `x = Foo.new` locals
that must stay in their own method.
Measured: `AssignedField.run` now emits BOTH chain links, exact parity with the
local-const control.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(resolvers): pin inference-typed field receivers across eight languages
#2807 was filed against TypeScript, but the defect class is cross-language:
"can a field whose type is inferred act as a call receiver". This measures all
eight languages where the shape exists at all, in one table.
The metric is parity with EACH LANGUAGE'S OWN CONTROL ROW, not "both chain
links present". JavaScript, Python, Dart and PHP lose the second link
(`Inner.compute`) even for a plain local, because nothing annotates `inner()`'s
return type — a separate return-type-inference gap. Scoring against "both
links" would have accused those four of a bug they do not have; scoring against
their own control isolates the field-typing question cleanly.
Recorded state: TypeScript, JavaScript, Python and Ruby now match their
controls. Kotlin and PHP already did before #2807 and are pinned so the shared
fold cannot regress them unnoticed — the languages that got receiver typing for
free are precisely the ones nobody re-checks.
Two rows stay pinned BROKEN, at their exact current value:
Dart — real and narrow: the annotated control resolves, the inferred one
does not. Its bindings are synthesized in dart/captures.ts rather
than by a query, so the fix is its own change.
Swift — blocked by a different defect found while measuring: with several
classes each defining `run`, every `run`'s edges are attributed to
the FIRST-declared one, which collects duplicates while its siblings
— including the ANNOTATED control — collect none. Receiver typing
cannot be measured there until that is fixed, and "fixing" it against
this observable would be fitting to a broken measurement.
Both gap rows carry a `callerExists` probe in the same assertion object, so an
empty list can never read as "resolved fine, wrong node id", plus a whole-matrix
guard that every language keeps a resolving control — that is what makes a gap
row mean "broken" instead of "fixture never worked".
Targets are deduplicated before comparison: Swift emits one edge more than once
per call site, and edge multiplicity is a different question from whether the
receiver typed at all.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(python): a method call on the receiver is not a construction
Review finding on f4e1ead0d. `constructorCallTypeName` accepted ANY call with
an identifier or attribute callee, so `self.p = self.build()` bound `p` to the
non-type `"self.build"` — and because that shares the weakest tier with a real
construction, a later such assignment DISPLACED an earlier `self.p = Outer()`
and left the field untyped again.
Measured before the fix: `self.p = Outer()` followed by `self.p = self.rebuild()`
emitted no CALLS edge at all from a method chaining off `self.p`, and
`self.q = self.make()` bound a type name that resolves to nothing. After:
the real construction survives the reassignment, and a pure method call binds
nothing rather than something wrong.
Rejects a callee rooted at the receiver name. `models.Outer()` still binds —
only `self`-rooted callees are refused, which is exactly the method-call shape.
The matrix gains a `reassigned-from-method-call` row that fails without this
rejection; that discrimination is the only reason the row exists.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(swift): resolve a method def to its own node when the labels disagree
Two classes in one Swift file each declaring `func run` collapsed onto one
node: every call in BOTH bodies was attributed to whichever `run` registered
first, which collected duplicate edges while its twin collected none. Renaming
one method fixed it; moving it to another file fixed it; so the collision was
name-keyed and per-file, not positional.
Root cause is a LABEL split, not a name. Swift's structure phase emits a type's
methods as `Function` nodes, while the scope extractor derives `Method` from the
`@declaration.method` anchor. Every key in `resolveDefGraphId` — qualified,
parameter-types, arity, shape — is label-scoped, so such a pair misses all of
them and lands on the bottom fallback, `simpleKey(filePath, name)`, which is
deliberately label-agnostic and first-write-wins.
Fixed at both ends:
- Swift qualifies a method def as `<Type>.<method>`, matching the qualifier
the structure phase already encoded in the node id. `class`, `struct` and
`extension` all parse to `class_declaration`, so one ancestor walk covers
them; a generic `class Box<T>` and an `extension Foo` wrapping a `user_type`
both reduce to the bare owner name.
- The bridge retries the qualified keys under the sibling callable label.
Gated on the name containing a dot: `A.run` names one construct whatever the
label, while a bare `run` is exactly the top-level-vs-method aliasing the
label was added to prevent, so the original guarantee is untouched.
This also unmasked Swift's #2807 row. `let p = Outer()` had always bound
correctly — its edges were being credited to the wrong caller, so the
inference-typed receiver looked broken when it was not. `InferredField.run` now
emits `Outer.inner`, matching its control.
Verified on the full resolver + CFG suite: 3165 passed, 0 failed, against a
3164-passing baseline.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(dart): declare inference-typed class fields so they can be receivers
`var b = Outer();` produced no `@declaration.property` capture at all — no
Property node, and nothing for the capture layer to hang a type binding on — so
`b.inner()` could not type its receiver while the annotated twin
`Outer b = Outer();` resolved fine (#2807).
The gap was in the query, one layer below where the binding is emitted: both
class-field patterns require a leading `(type_identifier)` or `(nullable_type)`,
i.e. a WRITTEN type. Dart puts the keyword there instead for an inferred field,
and spells it two ways — `inferred_type` for `var`, `final_builtin` for `final`
and `late final`. Covering only `var` would have left the more idiomatic Dart
style broken, so both are matched.
With the field declared, the capture layer types it from the constructor its
initializer calls, as `constructor-inferred` — the weakest source, and the
annotated branch returns before it, so an annotated field is untouched. Only a
direct construction is accepted (a bare identifier followed by a `selector`
carrying an `argument_part`, the same shape `findDirectCallValue` accepts for
locals); a literal, member call or await is left alone rather than guessed at.
Note this is the LOCAL/field split that made the gap invisible: `emitVarTypeBinding`
already handled `initialized_variable_definition`, but a class field is
`declaration(<keyword>, initialized_identifier_list(initialized_identifier))`.
`InferredField.run` now emits `Outer.inner`, matching its control. Dart's
`var r; C() { r = Outer(); }` shape stays pinned as a known gap: Dart writes the
field with no receiver prefix, so binding it means treating assignment to a bare
identifier as a field write, indistinguishable from a constructor-local.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(resolvers): record Swift and Dart reaching parity in the matrix
Both languages' inference-typed field rows move from KNOWN GAP to resolving,
which is the self-diffing signal this file was built to produce: closing either
gap failed it with the newly resolved ids in the diff.
The header table and prose are corrected together with the rows, as the file's
own instructions require — including WHY Swift moved. Its `let p = Outer()`
binding had always been correct; a separate label-split defect attributed the
second same-named method's calls to the first, which masked this row entirely.
Recording that is the point: a future reader comparing the table against the
code needs to know the row was never a receiver-typing failure.
One row stays pinned: Dart's `var r; C() { r = Outer(); }`. Dart writes fields
without a receiver prefix, so binding it means treating assignment to a bare
identifier as a field write — indistinguishable from a constructor-local until
the field set is known. Idiomatic Dart writes `final r = Outer();`, which the
inferred-field row now covers.
Every language keeps its resolving control row, so the remaining gap still means
"broken" rather than "fixture never worked".
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(dart): type a field from a constructor assigned to it
`var r; C() { r = Outer(); }` bound nothing, so `r.inner()` had no receiver
type — the last inference-typed field shape still failing after the initializer
form was fixed (#2807).
Dart is the one language here that writes a field with NO receiver prefix, so
`r = Outer()` inside a constructor is syntactically identical to assigning a
constructor-local. That ambiguity is why this was initially left pinned — but
the field set IS knowable: the class body declares `var r`, which the
initializer fix already turned into a property declaration. So a bare name binds
exactly when Dart itself resolves it to the field: the enclosing class declares
it AND the enclosing body declares no local of that name. A `this.`-prefixed
write is unambiguous and needs neither test.
The shadowing case is asserted, not assumed: with a body-local `var s = Outer()`
in scope, the field stays unbound while the local still resolves on its own.
Binds `constructor-inferred` (weakest source, so an annotation still wins), and
only for a direct construction — an identifier followed by a `selector` carrying
an `argument_part`, the same shape accepted for locals. The narrow
`@type-binding.dart-field` marker drives the Class-scope hoist in
`dartBindingScopeFor`; gating on it rather than on `@type-binding.constructor`
at large is what keeps genuine locals in their own scope.
All three shapes now match their control: bare `r = Outer()`, `this.s = …`, and
a non-constructor `setUp()` assignment.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(swift): type an optional field and read through its force-unwrap
Swift cannot declare a stored property with neither a type nor an initializer,
so its "declare now, assign in init" idiom is an OPTIONAL field read back
through a force-unwrap. That shape resolved nothing, and it was broken in two
independent places — each alone leaves it broken:
1. `var a: Outer?` parses as `type_annotation(optional_type(user_type(…)))`,
but the property-annotation pattern required the `user_type` to be a DIRECT
child, so an optional field was never typed at all. The pattern added here
captures the INNER `type_identifier`, so the binding is `Outer` without
relying on `stripOptional` reducing an `Outer?` spelling.
2. `self.a!` is a `postfix_expression`, which the receiver walk did not peel,
so even a typed field could not be read through the unwrap.
For (2), `postfix_expression` is NOT added to `TRANSPARENT_RECEIVER_WRAPPERS`
outright: unlike TypeScript's `non_null_expression` — which is only ever `!` —
Swift's node also carries user-defined postfix operators, which can return
anything. Peeling those would type the receiver as the operand and mint a
confidently WRONG owner, the failure mode compound-receiver.ts calls strictly
worse than no edge. So the peel is operator-gated: transparent only when the
node's text ends in `!`, which is provably type-preserving.
Verified: force-unwrap `self.a!.inner()`, optional chain `self.b?.inner()`, and
the plain annotated field all resolve; previously only the plain one did.
The gate keeps this off every other language — `postfix_expression` is not a
node type the other grammars produce here — and the full resolver + CFG suite is
green at 3166 passed / 0 failed, against a 3165 baseline.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(resolvers): close the last two matrix gaps
Dart's `assigned-field` and Swift's new `optional-assigned-field` rows now
resolve, leaving no known-gap row in the matrix: every language reaches parity
with its own control on both the initializer and the assigned shape it can
express.
The Swift row is new because the shape it covers did not exist in the fixture:
Swift cannot declare a stored property with neither type nor initializer, so its
assigned form is an optional field written in `init` and read through a
force-unwrap — a shape that needed both an optional-annotation pattern and an
operator-gated receiver peel, which is why the row's comment names both.
The header records how the two hard cases were fixed, including the Dart
shadowing rule the fix depends on: a bare `r = Outer()` binds only when the class
declares that field and the body declares no local of the same name.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(bench): rebaseline the receiver-resolution and scope-capture gates
Both gates are exact-match, so the improvements in this branch fail CI until the
baselines move and the movement is explained. Caught by running the CI gates
locally — the resolver and CFG suites are green throughout and never see these.
receiver-resolution — three shapes moved to RESOLVES, no drop-count changed:
ruby.fieldReceiverCall INVISIBLE-GAP -> RESOLVES (`@ivar = Foo.new`)
swift.decoratedFieldType INVISIBLE-GAP -> RESOLVES (`var a: Outer?`)
kotlin.nonNullAssert VISIBLE-GAP -> RESOLVES (`x!!` receiver)
scope-capture — swift and typescript fingerprints, both ADD captures and remove
none; the per-language `_rebaselined_inferred_field_receiver_2807` notes carry
the detail and the prior digests. The other 13 languages are unchanged, which is
the check that this is the intended emission and not a capture regression.
CORRECTION to d5d878033's message, which claimed the operator-gated
`postfix_expression` peel "keeps this off every other language — postfix_expression
is not a node type the other grammars produce here". That is wrong: Kotlin's
grammar produces it too, and `kotlin.nonNullAssert` moving to RESOLVES is the
proof. The peel is still correct there — Kotlin `!!` is a non-null assertion with
exactly the type-preserving semantics the `!` gate tests for — but it is a
BEHAVIOUR CHANGE IN KOTLIN, not Swift-only as stated. The gate is what surfaced
it; the claim should have been verified rather than asserted.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(cache): bump SCHEMA_BUMP for the six-language capture change, + review fixes
SCHEMA_BUMP 39 -> 40. THIS IS THE MERGE-BLOCKER of the review: every language
change in this PR is PARSE-TIME capture emission, and `analyze` skips tree-sitter
dispatch for byte-unchanged chunks (GUARDRAILS.md:34), so a warm cache replays
the pre-fix capture set verbatim and the new receiver edges never appear —
silently, no error. Exactly the v27/v30 failure mode this file already documents.
The PR description's claim that "no schema or version constant applies" was
wrong on both counts: a bump IS required, and a plain re-analyze does NOT
surface the captures without it. Re-check against origin/main before merging —
main was also at 39 when 40 was allocated, and this file records eight prior
collisions.
Also from the review:
- dart/simple-hooks.ts hand-rolled a 9-line parent walk byte-identical to the
shared `walkToScope(innermost, tree, 'Class')` that TypeScript and Ruby call
in one line in this same PR. Now uses the helper.
- utils/call-analysis.ts: the doc framed the postfix-`!` peel as Swift-only. It
is not — Kotlin `!!` parses as the same node and is peeled too, which the
receiver-resolution bench proved (kotlin.nonNullAssert VISIBLE-GAP ->
RESOLVES). The comment now says so, and names the `!` gate rather than the
language as the bound.
- test/helpers/temp-dir-pool.ts: its doc claimed four consumers; on THIS branch
only `pdg-chained-receiver-callees` uses it (the other three convert on
#2802). Corrected, and the byte-identical-to-#2802 intent recorded.
- inferred-field-receiver-matrix: adds the Dart shadowing assertion the header
comment already CLAIMED to make but never did. First attempt was vacuous —
`var s = Outer()` is a declaration, so it never produced the bare
`assignment_expression` the guard inspects; removing the guard did not fail
the row. Fixture corrected to `var s; s = Outer();`, and mutation-verified:
guard present 35 pass, guard removed the row goes red.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(cache): move the SCHEMA_BUMP pin to 40
The pin at incremental-parse-cache.test.ts asserts the exact value on purpose —
it exists to catch two branches claiming one number, and it has earned that
eight times. Bumping the constant to 40 without moving the pin turned it red.
Found by the Codex (gpt-5.6-sol) review leg, which flagged it as a
deterministic committed-test failure. The Claude lanes could not have caught it:
they were dispatched before the bump landed.
The comment now records the 39 -> 40 movement and its reason, matching the
existing convention in that block.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(dart): treat every binder as a field shadow, not just local declarations
Review P1, reproduced by two independent reviewers. `emitDartFieldAssignmentBindings`
binds a bare `r = Outer()` to the FIELD when the class declares `r` and the body
declares no local `r` — but the shadow set was built by walking for
`initialized_variable_definition` only. That is one binder form out of many, so a
formal PARAMETER named like a field slipped through:
void reset(Alpha r) { r = Alpha(); } // r is the PARAMETER
retyped the FIELD to `Alpha`, fabricating an edge AND destroying the correct
`Beta` binding the constructor had established. The mutation test shows exactly
that: the pre-fix result is not a missing edge but a WRONG one (`Other.inner#0`
instead of `Outer.inner#0`) — the failure mode compound-receiver.ts:519-537 calls
strictly worse than no edge.
The node types were chosen from real grammar output, not assumed. Two facts drove
the design: formal parameters live on the SIBLING `method_signature`, never inside
`function_body`, so no walk of the body could ever have seen them; and
`formal_parameter` carries a `name` field only when typed — untyped, `this.` and
`super.` forms do not. `collectDartBodyShadows` therefore walks the signature AND
the body, collecting formal/closure/local-function/named/optional params,
`this.`/`super.` constructor params, catch bindings, for-in variables, and both
local-declarator forms. A parameter shape whose name cannot be read contributes
nothing — declining to bind is the safe direction.
A 27-case binder sweep passes: 26 shadow shapes bind nothing, the no-binder
control still binds.
Four new matrix rows (param, closure param, catch, loop var) assert a surviving
POSITIVE target rather than an empty list — deliberately, because the pre-fix
value is a different non-empty target, so these rows cannot pass vacuously the way
an empty-assert row can. Mutation-verified: reverting captures.ts turns exactly
those four red and leaves every pre-existing row green.
SCHEMA_BUMP is already at 40 on this branch for the six-language capture change and
has not shipped, so it covers this too; re-check against origin/main before merge.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ruby): don't bind a class-object @ivar as an instance field
Review P1. The `@ivar = Foo.new` patterns added on this branch hoist to the
enclosing Class scope without asking WHOSE `self` owns the ivar. In Ruby an ivar
written in singleton context belongs to the class object, not to instances, so
def self.build; @pool = Alpha.new; end
class << self; def make; @cache = Alpha.new; end; end
bound `@pool`/`@cache` as INSTANCE fields, fabricating edges from instance methods
that read an ivar which is never assigned on an instance.
Three corrections came out of fixing it:
1. The detection premise was wrong. `def self.build` is NOT a `method` node with a
`self` receiver — it is its own node type, `singleton_method`, and
`childForFieldName('receiver')` returns NONE on it. Matching on a receiver field
would have detected nothing, silently. Detection is by node type:
`singleton_method` / `singleton_class`.
2. A THIRD form exists that the review did not name: a class-body-level
`class C; @shared = Outer.new;` is the same defect (self is the class object),
and is likewise new on this branch — before it, `left: (instance_variable)`
matched nothing at all.
3. Dropping only the `@type-binding.ivar-field` marker is NOT sufficient, and the
class-body case is what proves it: with the marker gone the binding falls back
to its innermost scope, which at class-body level ALREADY IS the Class scope, so
it still lands in the wrong place. The whole match is therefore discarded.
The check lives in `languages/ruby/captures.ts` because `Capture` carries only
`{name, range, text}` — no AST node — so `rubyBindingScopeFor` structurally cannot
ask whose `self` owns the ivar. All Ruby logic stays under `languages/ruby/`.
`method` alone is not a sufficient "instance" signal, since a `def` inside
`class << self` is reached through a `method` node first.
Cost relative to main is zero: a class-object ivar goes back to binding nothing,
exactly as before these patterns existed.
The three new rows are structurally two-sided, not just mutation-checked: each
empty row is paired with a non-empty `*-instance-ivar` row on the SAME fixture
class, so breaking the hoist entirely turns the partner red while an unconditional
hoist turns the empty row red. Mutation-verified in both directions.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(typescript,javascript): bind `this.field = new X()` only inside a class method
Review P0, the most serious finding of the tri-review and reproduced by two
independent reviewers. The `this.<field> = new X()` patterns added on this branch
were CONTEXT-FREE: they matched anywhere in the file, and `tsBindingScopeFor`
hoisted to the nearest enclosing Class without asking whose `this` that was. Since
the binding lands on the same Class scope with the same `constructor-inferred`
source as the field-initializer pattern, and pass4CollectTypeBindings prefers the
later match on `>=`, it OVERWROTE the class's real field type. Reproduced from a
non-arrow callback, an object-literal method, a static method, and module level.
Fixed STRUCTURALLY, in the query: both patterns are now nested under
`class_body -> method_definition -> body: (statement_block) -> (expression_statement)`,
which kills the callback, object-literal and top-level triggers with no runtime
code and mirrors JavaScript's `synthesizeConstructorFieldBindings` discipline.
TypeScript still accepts ANY method, not just `constructor`, so the setter case
this branch deliberately supports keeps working.
`static` needed one emit-side guard: it is an ANONYMOUS token on `method_definition`
with no field name, and tree-sitter patterns cannot negate an anonymous token
(checked against node-types.json), so `isStaticMethodThis` drops it in captures.ts.
`simple-hooks.ts` is comment-only — the unconditional Class hoist is now documented
as safe BECAUSE the marker's producers are bounded, with a note that widening them
means re-establishing that.
Also fixes a `.ts`/`.js` disagreement the narrowing itself created: JavaScript's
synthesis matched `method_definition` ANYWHERE, so an object literal containing a
method named `constructor` still typed the enclosing class's field. Measured on
identical source — JS emitted `p -> Alien`, narrowed TS emitted nothing — and
closed with a `node.parent?.type !== 'class_body'` guard in javascript/captures.ts.
The two languages must not disagree about the same source.
Deliberately NOT matched (a missing binding, never a wrong one — JS declines these
too): an assignment in a nested block, or inside an arrow where `this` genuinely IS
the instance.
Evidence the narrowing removed nothing legitimate: `bench/scope-capture --check`
passes with the TypeScript AND JavaScript fingerprints BYTE-IDENTICAL. The five new
matrix rows use an `Alien` class that also declares `inner()`, so a regression SWAPS
the target rather than emptying the set — they cannot pass vacuously. Mutation
test: reverting the source turns exactly those rows red (`+ "Alien.inner#0"`,
`- "Outer.inner#0"`).
SCHEMA_BUMP stays at 40 — this PR's existing bump covers the capture change being
narrowed, and the buggy variant never shipped outside this branch.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(resolution): consult the sibling callable label in the position key too
Review P1. This branch added a sibling Method<->Function retry to the qualified
keys in `resolveDefGraphId`, but not to the #2699 POSITION key or its fail-closed
guard, which both stayed scoped to `def.type`. Since the premise of the whole fix
is that Swift defs are `Method` while nodes are `Function`, the position lookup
missed and the fail-closed guard could NEVER FIRE for exactly the case the retry
serves — so a function-local `func helper` inside `Host.run` was deterministically
aliased onto the class method `Host.helper`, even with differing arity.
Two earlier reviewers REFUTED this by arguing the guard runs before `lookupTagged`.
That is true and irrelevant: the guard is scoped to `def.type`, so in the
label-split case it is unreachable. Recording it because two independent lanes
agreeing on a refutation is not proof.
`siblingCallableLabel(label)` is now the single definition, consulted by all three
key families:
- position key: retried under the sibling label, gated on `posHit === undefined`
so an AMBIGUOUS_POSITION tombstone still falls through to the name keys rather
than being resolved by relabelling. Deliberately NOT dot-gated — a position key
is not a name, so the aliasing risk the dot gate exists for does not apply.
- fail-closed guard: mirrored unconditionally (it only ever returns undefined).
- qualified retry: dot gate untouched.
Measured before -> after on a Swift fixture: `Host.helper#1 -> sink` (the local
body's call credited to the public 1-arg method) becomes
`Host.run.helper@8:8#2 -> sink`, with the local's own node no longer edgeless.
SCOPE CORRECTION to the P1 report: only the first consequence is a bridge defect.
The second — "`run`'s call to the local resolves to the method" — is NOT reachable
from ids.ts. Both defs carry qualifiedName `Host.helper` and label `Method`, and
the binding hands the target side the class-member def, so the scope walk in
free-call-fallback picks the member. No def->node mapping can change that; it is
pinned as an explicitly labelled KNOWN GAP rather than left implied.
Verification, on shared code so the full bar: resolvers+cfg 3170 passed / 1 skipped
/ 0 failed; `bench/receiver-resolution --check` OK; `bench/scope-capture --check`
PASS (15 languages, Swift fingerprint unchanged) — i.e. the bridge change altered
no capture output. The 3170 reconciles against the 3167 pre-existing at a5bf4c2da
plus exactly 3 new tests; 3167 differs from the older 3166 baseline because
0418b0aac added the matrix's only known-gap row, which emits one extra `it`.
Mutation test: with both arms reverted, 3 of the 5 new cases go red, each arm
pinned independently — the guard case registers no position key, the position case
registers no local-name key.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor: apply cleanup-review findings across the receiver-typing change
Four parallel quality lanes (reuse, simplification, efficiency, altitude) over
`origin/main...HEAD`. Eleven fixes; both exact-match bench gates hold with every
capture fingerprint BYTE-IDENTICAL, so none of this changed what the analyser emits.
Reuse — stop re-rolling helpers that already exist:
- `walkToScope` moved out of the TypeScript provider into a language-neutral
`utils/scope-tree-walk.ts`. Ruby and Dart had begun importing it FROM
`languages/typescript/`, which made three unrelated providers depend on the TS
module for a generic `Scope`/`ScopeTree` walk. Python's hand-rolled copy — the
one this PR's new `self.x = Outer()` path routes through — is folded in, so all
six languages now share one traversal.
- Swift stops string-parsing a type name. `swiftEnclosingTypeName` split on `<`
and `.`; `swiftBaseTypeIdentifier` + `swiftQualifiedBaseTail` do it structurally
and correctly skip the sibling `type_arguments` node, which the string form only
guessed at. `findEnclosingTypeDeclaration` replaces the inlined ancestor walk.
- TypeScript uses the canonical `hasKeyword(method, 'static')`. The previous
`child.type === 'static'` is the exact form `isStaticMember` documents as
grammar-version-fragile: "`static` can appear as an unnamed token or as a
keyword node depending on grammar version; check text."
- Both new test suites use `cleanupTempDirSync`, which exists because a pipeline
test's open handle surfaces as EBUSY/EPERM on Windows and `force` does not
suppress it. This repo shards Windows CI.
LATENT DEFECT, found by the reuse lane and fixed: `var a = X(), b = Y();` parses
as ONE `declaration` with two declarators, and the query matches it once per
declarator with the SAME node — so the first-descendant search handed every
declarator the FIRST one's initializer. `b` resolved as `X`. Now reads
`nameNode.nextNamedSibling`, which is both correct and free. Pinned by a
`multi-declarator-inferred-field` row ordered so the declarator under test is the
second; reverting the fix turns exactly that row red with the wrong edge.
Efficiency — measured, not asserted:
- Dart's shadow set was built eagerly for EVERY method body and discarded 87-100%
of the time (a `this.`-prefixed write never reads it). Now lazy and memoised per
body, gated on `fields.has()`. Semantics are unchanged: the set is body-wide, so
deferring construction cannot change its contents.
Worth recording WHY CI could never have caught this: `bench/scope-capture` gates
the SCALING RATIO, and the work is linear — ratio stays 1.0 against a 1.5 budget
while a constant-factor regression passes straight through.
- `isTransparentReceiverWrapper` crossed the `node.type` native getter twice on the
common path. One hoisted read, and — since absent and ungated are distinguishable —
one `get` replaces `has`+`get`.
Simplification:
- One `Map<string, string | null>` replaces the parallel Set + Map that both
expressed "this wrapper is transparent", with `null` meaning unconditional.
- `ids.ts` computed `siblingCallableLabel` twice under two names. The three retry
blocks are deliberately NOT collapsed — they use different key builders and
materially different gates.
- Python's `interpret.ts` nesting was only a consequence of arm ORDER; swapping the
arms is unconditionally equivalent (the two differ only when both markers are
present, and both orders then yield `constructor-inferred`).
- One `isDirectConstruction` predicate replaces the construction-shape test that
had been written four times in dart/captures.ts.
Deliberately NOT done, each needing a fingerprint rebaseline or new node ids:
unifying the six `@type-binding.*-field` markers into one canonical capture (it
would change Python's anchor semantics, which must be verified not assumed); a
Swift `labelOverride` mirroring Kotlin's four-line fix, which is the real cure for
the Method/Function split the bridge currently compensates for; generalising the
Swift optional-annotation pattern to `(type_annotation (_))` so the existing
strippers handle every wrapper; and merging the TS query with the JS walker, which
also carries a JSDoc branch no query can express.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(swift): regenerate the Swift captures golden for the optional-annotation pattern
CI caught what my local runs did not: `swift-captures-golden.test.ts` pins
`emitSwiftScopeCaptures` output across every `swift-*` fixture, and this branch
changes that output. It is a THIRD capture gate, separate from the two
exact-match benches already rebaselined here — `bench/scope-capture` hashes a
different corpus, so its Swift fingerprint moving did not imply this one, and
passing it was not evidence this was clean.
The drift is digest-only: 37 changed lines, 37 in each direction, no capture
entry added or removed. That is the expected shape for
`(type_annotation (optional_type (user_type …)))` making optional properties emit
an annotation binding they previously did not, plus the `@declaration.qualified_name`
now carried on Swift method declarations.
Regenerated with the mechanism the test itself prescribes (`UPDATE_GOLDEN=1`),
not by relaxing the assertion. Verified after: all Swift unit + resolver suites
green (4 files, 124 tests), and `bench/receiver-resolution --check` still exactly
matches its baseline.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: three more wrong-owner defects, found by a second review round
A second tri-review of this PR found THREE new P1 wrong-owner defects — every one
of them in code the FIRST round had already fixed. All three are the same root
shape: an incomplete ENUMERATION of binder or scope forms. That class has now
bitten this branch four times (formal parameters, then these), so two of the
three fixes below deliberately attack the class rather than the instance.
1. DART 3 PATTERN BINDERS (P1, reproduced by two independent lanes).
`addDartBinderName` enumerated five binder node types, and every Dart 3
pattern form parses into node types in NONE of them — so a pattern-bound local
did not count as a shadow and its write retyped the CLASS FIELD:
class Host {
var session = Cache();
void load() { final (session, count) = (Session(), 2); }
void use() { session.ping(); } // resolved Session.ping, not Cache.ping
}
The grammar hides every rule that would carry a binder (`_pattern_field`,
`_list_pattern_element`, `_guarded_pattern`, …), inlining children onto the
enclosing visible pattern node, so binders land as direct `identifier` children
of just two leaf types. Covers all 10 pattern types that can hold one; the
eight container types are defence, since the grammar demonstrably inlines
identifiers onto containers already.
THE COMPOUNDING PART: a grammar-derived coverage guard reads `nodeTypeInfo` and
fails if the grammar declares a `*pattern*` type the fixtures do not exercise.
A grammar bump adding an 11th type now turns the suite red instead of silently
reopening this bug a third time.
2. RUBY BLOCK-RECEIVER `self` REBINDING (P1 here, both Claude lanes + Codex, which
rated it P2 — the engines agreed the defect is real and disagreed on severity).
`isRubyInstanceIvarWrite` enumerated `singleton_method`/`singleton_class` as the
ways `self` gets rebound. A `def` inside a BLOCK attaches to the block's
receiver, so `Struct.new(:x) do def warm; @a = Beta.new; end end`,
`Class.new do … end`, `class_eval`, and `other.instance_eval { @a = … }` all
published onto the nearest LEXICAL class.
Deliberately NOT fixed by listing rebinding call names: that set is OPEN —
`def helper(&blk) = Foo.class_eval(&blk)` rebinds a block it merely receives,
and nothing in the block's own syntax reveals it. An allow-list of "safe"
iterators would be the same defect one level down. The rule is structural:
crossing ANY block boundary makes ownership unprovable, so discard. Complete by
construction rather than by enumeration.
ACCEPTED COST, asserted not hidden: `[1].each { @shared = X.new }` in an
instance method really is the instance's `self`, and this drops it — that block
is syntactically identical to the `instance_eval` one. It has its own row
(`plain-block-self-ivar`) so the loss is visible rather than discovered later.
3. STATIC FIELD INITIALIZERS (P1, found by Codex/gpt-5.6-sol, corroborated).
A `static` field initializer was captured as an ordinary instance binding, and
since both land on one Class scope at the same `constructor-inferred` strength,
the later wins the `>=` tie-break — so a static field retyped the instance
field of the same name (`this.p.hit()` -> `Wrong.hit`). Unguarded in BOTH
`javascript/query.ts` and `typescript/query.ts`; the existing
`isStaticMethodThis` only ever covered the `this.x =` assignment form.
Two things surfaced while fixing it: the TS `annotation` pattern collides
identically and is PRE-EXISTING, not introduced here; and JS `static
constructor(){}` had no guard where TS did — the .ts/.js divergence this PR's
own comment claimed could not happen.
Dart has no same-name twin (the language forbids it), but a static method's
receiver-less write named a library-level variable and DISPLACED the
constructor's binding. Fixed narrowly, with a counterweight row
(`static-field-declaration-still-types-its-receiver`) that goes red if anyone
widens the guard into "drop every static binding" — reading a static by bare
name from an instance method is ordinary Dart and must keep working.
ACCEPTED COST: `typeBindings` has one map per Class scope with no static/
instance split, so a static field is dropped rather than recorded separately,
losing typing on a TS/JS `Host.p.hit()` static receiver chain. Missed edge over
wrong edge, per compound-receiver.ts:519-537.
Every new row asserts a SURVIVING POSITIVE target, never an empty set: the pre-fix
value in each case is a DIFFERENT non-empty target, so none can pass vacuously —
the trap this branch already fell into once. Mutation-verified per fix: reverting
each turns exactly its own rows red (17 Dart, 6 Ruby blocks, 5 static) with every
pre-existing row green.
Matrix 49 -> 80 tests. Siblings 510 passed. tsc clean. scope-capture PASS (15
languages, all ratios within gate).
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(dart): mask a shadowed field on the READ side, not just the write side
The review critic refused to pass this PR while this was open, and it was right
to: this is the same wrong-owner shape as the three defects fixed in the previous
commit, except this one is introduced BY this PR rather than merely missed by it.
THE DEFECT. Typing an unannotated field from a constructor assignment
(`var conn; Host() { conn = Alpha(); }` binds `conn` on the CLASS scope) is this
PR's whole point. `emitDartFieldAssignmentBindings` correctly declines to WRITE
that binding when a member body rebinds the name — but the shadow set gated
writes ONLY. `collectDartBodyShadows` had exactly one call site, inside the
bare-name write branch. Nothing consulted it on the read side, so a bare-name
READ of a shadowing binder the resolver cannot type walked straight past the
local and hit the field binding this feature mints:
class Host {
var conn;
Host() { conn = Alpha(); }
void probe(List<Beta> xs) {
for (final conn in xs) { conn.inner(); } // conn is a Beta element
}
}
// resolved Alpha.inner, not Beta.inner
Reproduced in SEVEN binder shapes, not the one the review reported: for-in
(`final` and `var`), untyped formal parameter, plain local `var`, catch binding,
closure parameter, and record pattern. Delete the constructor and the same read
emits NOTHING — which is what proves this PR introduced it. "No edge" became
"wrong edge", the one failure mode compound-receiver.ts:519-537 exists to prevent.
THE FIX uses `Scope.ownsReceivers` (#2701), the primitive that already exists for
exactly this, rather than inventing a mechanism. `scope/walkers.ts` consults
`typeBindings` FIRST at every scope and only then honours the mask, so a shadow
the resolver CAN type still wins — an annotated `void probe(Beta conn)` keeps
`Beta`, because `synthesizeDartSignatureBindings` anchors parameter bindings on
the same body node and they land on the same Function scope. The mask fires only
where the alternative was a fabricated type.
Plumbing follows TypeScript's `@receiver-owner.this` precedent: the marker rides
the same synthesized match as `@scope.function` and sits outside the `@scope.`
namespace so `anchorCaptureFor` cannot mistake it for the anchor. Dart differs
only in that its function scopes are synthesized in captures.ts rather than
declared in the .scm, so the names travel as capture TEXT — a `CaptureMatch`
carries no AST node, so the reader cannot re-derive them.
SCOPE, and the costs taken knowingly rather than hidden. The mask is
`shadows ∩ fields` and nothing wider. Masking every locally bound name would
also fix a library-level `var logger = Logger();` shadowed by a loop variable,
but it changes resolution for code this PR never touched. Three consequences are
documented on `dartShadowedFieldsCapture`, not buried: the wider case is left
open; an ANNOTATED field shadowed by a binder is masked too (correct Dart, but it
touches resolution predating #2807); and `mixin` bodies are reached, since the
grammar gives them a `class_body`.
PERFORMANCE, measured rather than asserted. The mask is emitted eagerly in Pass A,
where `collectDartBodyShadows` used to be lazy — the replaced comment recorded
87-100% of eagerly built sets being discarded, ~15% of Dart emission. Actual cost
on the scope-capture large corpus, median of 3: 405.6ms with the mask vs 390.0ms
without, ≈ +4%. Fingerprint and capture_groups are byte-identical across both
arms, so no corpus fixture emits a mask at all — that 4% is the cost of the CHECK
alone. Not visible to `bench/scope-capture`, which gates the scaling RATIO and is
blind to a linear constant factor; stated here because the gate cannot state it.
(3 samples per arm, blocked not interleaved — an estimate, not a rigorous number.)
A per-file memo keyed by node span makes both passes share one walk per body, so
the write side no longer pays a second one.
SCHEMA_BUMP 40 -> 41 with its exact-value pin, since capture emission changed.
Re-check against origin/main immediately before merge — main was 39 at commit time.
Mutation-verified both directions, which is the part that matters:
- unwire `scopeOwnsReceivers`, rebuild -> exactly 2 rows red
(`loop-var-read-does-not-see-the-field`, `pattern-read-does-not-see-the-field`),
83/85 green.
- over-widen the mask (drop the `shadows.has` test) -> 28 Dart rows red,
including `unshadowed-read-in-a-shadowing-class-still-resolves`.
The three control rows stay green under the first mutation BY DESIGN — they guard
overreach, not the defect; the second mutation is what proves they are live. Pre/post
on the trigger row: `{Class:Alien, Alien.inner#0, Outer.inner#0}` -> `{Class:Alien,
Alien.inner#0}`, so no row can pass vacuously.
Matrix 80 -> 85 tests. Sweep 3220 passed (was 3215; exactly +5). tsc clean. All four
capture gates green: receiver-resolution OK, scope-capture PASS (15 languages, no
fingerprint moved, nothing rebaselined), callable-value-flow PASS, swift golden 9.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ts,js): let each language name its own class-field node type
CI caught a defect the whole review round missed. `grammar-literal-validation`:
1 dead grammar literal(s) found:
- node-type "field_definition" — languages/typescript/captures.ts:0
— not valid in [typescript]
`isStaticClassFieldBinding` held BOTH spellings in one set —
`public_field_definition` (TypeScript) and `field_definition` (JavaScript) — so
that one predicate could serve both languages. But the predicate lives in
`typescript/captures.ts`, and the gate checks every literal against the grammar
of the FILE it appears in. `field_definition` is not a TypeScript node type.
The literal was NOT dead code: `javascript/captures.ts:42` imports the predicate
and calls it against real JS nodes, so the guard worked. The gate is still right
to fail it, and for exactly the reason this predicate's own docblock gives for
preferring `hasKeyword` over a node-type test — "a node-type test silently stops
firing on a grammar bump and every static field starts retyping its instance
twin again". A literal already dead in its own file is that failure shipped
pre-broken: nothing in the TypeScript file would ever have told us.
Each language now names its own node type and passes it in
(`TS_CLASS_FIELD_DEFINITION_TYPES` / `JS_CLASS_FIELD_DEFINITION_TYPES`), so every
literal is checked against the grammar it belongs to. The `hasKeyword` logic and
the static/instance reasoning stay shared and unchanged — only the node-type set
moves to the caller.
WHY THE LOCAL SWEEP DID NOT CATCH IT: I ran `test/integration/resolvers` and
`test/integration/cfg`. The gate is `test/integration/grammar-literal-validation.
test.ts`, in the parent directory. Scoping a sweep to the subdirectories a change
touches is precisely how a cross-cutting gate gets skipped.
grammar-literal-validation 4 passed. tsc clean. Full `test/integration` +
`test/unit/scope-resolution`: 6305 passed, 14 failed — all 14 in e2e/environment
suites (fts-extension-e2e 9, analyze-heap-oom-e2e, cli-e2e,
analyze-wal-checkpoint-failure, plus interproc-taint and parse-impl-env-reads,
which BOTH pass in isolation and fail only under 28-worker load). CI runs the
same files green.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(dart,ts): pin all seven read-side binder shapes; correct a wrong accepted-cost claim
Two review findings, one of which turned out to be a documentation defect rather
than the design defect it was filed as.
S8 — THE READ-SIDE FIX PINNED 2 OF THE 7 SHAPES IT REPORTED REPRODUCING.
`ab2f48c17` reported the wrong-edge defect reproducing in seven binder shapes and
landed rows for two. The stated mitigation was that all seven route through one
`collectDartBodyShadows` enumeration whose completeness the grammar-derived
coverage guard protects. That mitigation is NARROWER THAN CLAIMED: the guard
filters `nodeTypeInfo` on `type.includes('pattern')`, so it covers the pattern
family and NOT catch bindings, closure parameters, plain locals, or formal
parameters. Narrowing `addDartBinderName`'s catch arm would have turned no row red.
All seven were re-measured by unwiring `dartScopeOwnsReceivers` and rebuilding.
Every one gained the wrong edge `Outer.inner#0` — none had to be dropped as
non-reproducing. Five new rows: formal parameter, plain local `var`, catch
binding, closure parameter, for-in `var`.
Non-vacuity established structurally, not by assertion: the Dart AST was dumped
first to confirm each fixture produces the node `addDartBinderName` actually
inspects (`formal_parameter`, `initialized_variable_definition`,
`catch_parameters`, `for_loop_parts`). The catch row uses a bare `catch (zf)`
rather than `on Err catch` deliberately — an `on` clause names a type, which
would make the row measure type resolution instead of the mask.
S7 — THE ACCEPTED-COST COMMENT WAS WRONG, AND THAT IS THE FINDING.
It claimed dropping a static field's binding trades a wrong edge for a missed one.
Measured on a same-name twin, that is false:
read with the drop without it
this.p (instance twin) Outer correct Alien wrong
Host.p (static twin) Outer WRONG Alien correct
Host.q (static, no twin) none — missed Alien correct
The wrong edge did not disappear. It MOVED to the static read, which now picks up
the instance twin's type. Only the no-twin case is a genuine missed edge. The
trade is still right — `this.p` is far more common than `Host.p` — but it was
documented as safer than it is, and a reader deciding whether to revisit it was
being given the wrong picture.
NAMESPACING WAS EVALUATED AND DELIBERATELY NOT DONE. `Host.p.hit()` resolves
through `foldReceiverChain` in shared `compound-receiver.ts`, which explicitly
discards whether a chain's base was a class reference or a value (:519-527). The
class-constant bit exists only on the text-cascade path (`currentIsClassConstant`)
and is consumed solely by `isConstructionSelectorHop`; TS/JS take the fold, not
the cascade. `Scope.typeBindings` is `ReadonlyMap<string, TypeRef>` with no static
field. `ownsReceivers` cannot help — it is a suppressor that can only REMOVE a
binding, never route to a second one. A real fix needs `FoldState` to carry the
bit plus a key convention in shared code (an AGENTS.md:42 hook if not
language-neutral), it crosses the worker boundary so it needs a SCHEMA_BUMP, and
`compound-receiver.ts:826` iterates every binding for `fieldFallback` so a
namespaced key would leak straight back in as an ordinary field. Not a cheap or
safe change — and it would have been made with ZERO existing tests pinning
static-read behaviour.
So: smallest safe step instead. Two rows pin the measured behaviour
(`static-read-of-a-same-name-twin-picks-up-the-instance-type` asserts the positive
wrong target, not an empty set; `static-read-without-a-twin-loses-its-type` is a
known-gap), and the comment now says what actually happens. Anyone who revisits
this starts from measurements rather than from a claim.
No SCHEMA_BUMP: the `captures.ts` change is comment-only — verified, the diff has
no non-comment added lines.
Mutation red-rows 2/85 -> 7/90; each new row fails with a strictly larger set
(`+Outer.inner#0`), so none can pass vacuously. Overreach control still live:
dropping `shadows.has` turns 28 rows red including
`unshadowed-read-in-a-shadowing-class-still-resolves`.
Matrix 85 -> 92 tests. Sweep 3231 passed, 0 failed. tsc clean. All four gates
green, no fingerprint moved, nothing rebaselined.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(python): stop a dotted callee from fabricating a constructor type
S4 and S5 from the review round. They are ONE defect, not two, and the real one
is wider than the review described. Both live in a 32-line block THIS PR adds
(`@@ -116,0 +117,32 @@` — a pure addition), so neither is pre-existing.
THE DEFECT. `constructorCallTypeName` rejected only a callee rooted at the
receiver and returned every other dotted callee whole to `resolveTypeRef`, which
resolves dotted names through `QualifiedNameIndex` — and that index matches the
TRAILING SEGMENT against a class of that name even when the callee is a method on
an unrelated object:
class Alpha:
def ping(self): return 1
class Factory:
def Alpha(self): return "not an Alpha" # a METHOD
class Host:
def __init__(self, f): self.svc = f.Alpha() # svc is a str
def run(self): return self.svc.ping()
# measured: Host.run -> Alpha.ping, fabricated
WIDER THAN FILED: the review framed the trigger as a callee rooted at an
`__init__` PARAMETER. Measured, the root's binding form is irrelevant — a
module-level variable (`shared_factory.Alpha()`) fabricates identically. Any rule
written against what the root binds to would have fixed half the defect and left
the other half looking fixed. Both fabrications now have rows.
S5 IS A SYMPTOM, NOT A SECOND DEFECT. `self.conn = Outer()` then
`self.conn = Registry.get()` typed the field as `"Registry.get"` (resolving to
nothing, so the edge vanished) only because the dotted arm accepted
`Registry.get` as a constructor in the first place. Once dotted callees yield no
candidate, there is nothing weak left to displace with and `Outer` survives. So
`>=` is untouched and NO second mechanism was added: between two REAL
constructions last-write-wins is correct, and the existing `ReassignedField`
matrix row depends on it. Tightening the tie-break would have been the wrong fix
to a symptom.
THE FIX: accept a bare `identifier` callee only. Refusing ambiguous evidence at
CAPTURE time rather than resolving-then-rejecting is deliberate — the target-kind
route is not reachable from this file (`resolveTypeRef` already filters
`TYPE_KINDS`; the fabrication comes from a trailing-segment match in
`scope/walkers.ts`), and the root-alias route would collide with PR #2828, which
is rewriting exactly how an unaliased dotted namespace import resolves. This
change is orthogonal to #2828 by construction: it changes what is CAPTURED, never
how a name is looked up, and touches none of its files.
WHAT THE DOTTED ARM WAS ACTUALLY BUYING: nothing. The review (and this PR's own
docblock) justified it with `self.u = models.User()`. Measured, that shape emits
NO edge before or after this change — an instance field's binding lands in CLASS
scope, which never reaches the namespace split. The shape that really resolves is
the module-level local `u = models.User()`, which comes from `query.ts` and is
untouched here. The arm's entire measured contribution was fabrications, which is
what made the fix cheap.
#2828 COMPATIBILITY, checked not assumed: `import pkg.user` -> `self.u =
pkg.user.User()` resolves to nothing both before and after, so this cannot stop it
resolving. No test row pins that shape ON PURPOSE — asserting its current empty
state would plant a tripwire that goes red the moment #2828 lands. If #2828 also
teaches the FIELD path the namespace split, re-enabling dotted field callees
becomes a live option; the docblock says so, and says why redoing it capture-side
would re-open the fabrication.
SCHEMA_BUMP 41 -> 42 with its pin. This is parse-time capture emission: after the
fix `self.svc = f.Alpha()` emits no `@type-binding.constructor` capture at all, so
a v41 warm cache replays the pre-fix capture set for byte-unchanged files and
keeps serving the fabricated edge (GUARDRAILS.md:34). A within-PR re-bump, not a
collision fix — 40/41/42 are all this unmerged branch's, and `origin/main` is at
39. Re-check against origin/main immediately before merging.
Mutation-verified in BOTH directions, which is what shows the fix is placed at the
right width rather than merely working:
- revert the fix -> exactly 3 red: both S4 fabrication rows + the S5
displacement row (8 green)
- reject EVERY callee -> exactly 3 red: the three positive-typing rows (8 green);
the S4 rows correctly stay green
The two mutations hit DISJOINT row sets — too loose and too tight each break a
different half.
No row asserts an empty set: the three "must not type" rows call `Alien.ping()` as
a witness so a regression SWAPS a target in rather than emptying. Non-vacuity is
asserted in the test itself — one guard checks every caller node is live, another
asserts the `Alpha` class / `Factory.Alpha` method name collision the fabrication
NEEDS is actually present, so the rows cannot rot into passing for the wrong reason.
Sweep 3268 passed, 0 failed. Python unit + python.test.ts 342 passed. tsc clean.
All four gates green — no bench cell moved, nothing rebaselined.
Refs #2807
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(swift): preprocess indented conditional directives so class bodies survive parsing
* fix(swift): make conditional-directive blanking comment-, string- and brace-aware (#2771)
Addresses the review findings on PR #2771. The transform fired
unconditionally, which turned valid Swift into parse errors while missing the
most common shape it was written for.
- The blank/keep decision now consults `blockCommentDepth`, so ` #endif */` —
the result of commenting out a conditional block — keeps its comment
terminator. Previously `hasError` went raw=false -> preprocessed=true and the
rest of the file was swallowed.
- The decision keys on the scanner's brace depth instead of indentation. A
column-0 `#if` inside a class body is blanked (6 of 7 body shapes previously
still lost the enclosing declaration) and an indented file-scope directive is
not — matching what the doc comment already claimed. Bare-CR line endings,
NBSP/ideographic indentation and a leading BOM are recognized too.
- A group is blanked only when every branch is brace-balanced. An `#if`/`#else`
that splits a declaration header leaves one unmatched `{` once both branches
survive, which collapsed five top-level nodes into one and gave unrelated
types fabricated `NetworkClient.` qualified names. Such a group now degrades
to the pre-fix behavior.
- Multiline strings honour `\"""` escapes, and a plain `"""` closes even when a
`#` follows it, so the scanner no longer wedges in string state and silently
stops blanking for the rest of the file.
- The pound run is counted once per position and skipped. It was quadratic:
10.6s for one 64k-`#` line, well inside the 512 KB walker limit.
- Extended regex literals (`#/.../#`) no longer open a phantom block comment.
- Directive-free files return early, matching `stripUeMacros`.
Worker parity: `emitSwiftScopeCaptures` and `emitCppScopeCaptures` re-apply
their provider's `preprocessSource` on the parse-cache-miss path — Dart already
did this — and the embedding parse in `ensureAndParse` applies the hook as
well. Before this the worker and the scope-capture/embedding halves analyzed
different programs, turning a consistent degradation into cold-run/warm-run
non-determinism. A new parity test pins the equivalence for every provider that
defines the hook.
SCHEMA_BUMP 37 -> 38: this changes parse semantics, the chunk key hashes raw
on-disk bytes, and `preprocessSource` runs after the key is computed — so a
same-package-version warm cache would replay pre-fix Swift results verbatim,
including across `--force`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(ingestion): apply preprocessSource once in the scope bridge (#2771)
Follow-up cleanup on the review fixes. The previous commit re-applied each
provider's `preprocessSource` inside `emitSwiftScopeCaptures` and
`emitCppScopeCaptures`, mirroring what Dart already did — three copies of the
same rule, and a contract that asked every future emitter to remember it.
`extractParsedFile` is the single funnel every `emitScopeCaptures` caller
passes through (parse worker, scope-resolution run, Vue script extraction), and
it already receives the provider. Applying the hook there on the cache-miss
path covers all three languages and every future one, names no language in
shared code, and drops Dart's unconditional transform on the cache-hit path.
Verified the three emitters use `sourceText` for nothing but the parse, so the
substitution is output-identical — which the parity test asserts directly.
Also from the cleanup pass:
- the parity test derives its language list from the provider registry, so a
new provider adopting the hook fails until it adds a fixture
- `ensureAndParse` resolves the provider from the language it already computed,
instead of a second extension table (`getProviderForFile`)
- the preprocessor returns `sourceText` unchanged when no group was blanked,
which is the common case for files whose only directives are top-level
- `split(/(\r\n|\n|\r)/)` replaces the hand-rolled line splitter, and the
per-group brace bookkeeping is two scalars instead of an array
- the hint regex is derived from the line regex so the two cannot drift
- unit assertions compare the WHOLE preprocessed file against the expected
blanking, replacing per-line spot checks; the pipeline tests share one
`runFixture` helper and `getNodesForFile` in the resolver test helpers
- `LanguageProvider.preprocessSource` documents the real call sites and says
plainly that the set is not closed — `populateRangeBindings` still hands
language helpers raw text
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(autofix): apply prettier + eslint fixes via /autofix command
---------
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ingestion): join multi-line closure bindings on initializer startLine
Graph-node captures sit on the outer binding wrapper while scope-resolution
anchors on the inner callable; the line-only position join missed when those
split across lines and fail-closed dropped the real CALLS edge (#2735).
* fix(ingestion): unwrap Ruby call+block for multi-line lambda joins
Cover Kotlin/Ruby/Dart multi-line closure CALLS in integration tests, and
dig Ruby's call/block field so do-end bindings join on the block start line.
* style(ingestion): format closure join changes
* fix(ingestion): make closure position join language agnostic
* fix(rust): let the qualified-call filter see inline modules
The negative filter added in #2741 builds its set of known module names from
FILE PATHS, so an inline `mod x { … }` — which appears in no path — was absent
from it. Every module-qualified call into an inline module was therefore
rejected before any candidate channel ran, which is a hole in that optimisation
rather than in the resolution logic it guards.
The per-pass index now unions the file-derived names with inline module names
taken from the scope model: a `mod` declaration binds a `Namespace` def locally
in the declaring scope, and that binding is the only place an inline module's
name exists. Collected in the same walk that already builds the module → scope
map, so it costs no extra pass.
Found while fixing #2742, where a correctly resolved call into `mod inner { … }`
still could not reach its target.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV
* fix(scope-resolution): try the namespace-prefixed node key before the bare one
`resolveDefGraphId` looked up the plain `qualifiedName` key first and only
retried with the `namespacePrefix`-qualified key afterwards. For the defs that
carry a prefix the qualified name is a bare TAIL, so the plain key happily
matched a same-named item at a different namespace depth in the same file and
returned it before the more specific retry was ever reached.
The namespace-prefixed key is strictly the more specific of the two, so it is
now tried first. Where no such node exists the lookup falls through to exactly
the previous order, which keeps the #1982 behaviour this retry was added for.
Without this, a call into `mod inner { fn dispatch }` resolved to the correct
definition and then mapped it onto the crate-root `fn dispatch` node — the
self-loop #2742 describes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV
* fix(rust): qualify items by their enclosing mod chain so same-named ones stay distinct (#2742)
Node identity is `<label>:<file>:<qualifiedName>` and carried no module path, so
an inline `mod inner { fn dispatch }` and a crate-root `fn dispatch` in the same
file collapsed onto `Function:<file>:dispatch`, first-wins. Resolution already
picked the right definition — the target simply was not representable, so a
correct resolution still rendered as a self-loop and `impact` reported the real
callee as unreached.
The mechanism already existed: `qualifyRustImplTargetByModScope` has walked
`mod_item` ancestors for impl targets since #1982. Generalised to
`qualifyByEnclosingModScope` and applied to free items, so
`mod inner { fn dispatch }` becomes `Function:<file>:inner.dispatch`. Keyed
purely on the `mod_item` node type, exactly as the impl qualifier already was,
so it is a no-op for every language whose grammar has no such node.
Two constraints found by tests rather than by reading, both now encoded:
- The helper normalised `::` to `.` unconditionally. With no enclosing `mod`
that rewrote a top-level `impl a::Inner` from `a::Inner` to `a.Inner` and
moved its node id away from the one the HAS_METHOD owner edge emits,
breaking the #1975 scoped-impl ownership. It now returns raw text untouched
when there are no mod segments, which also makes the change strictly
additive for every id that has no enclosing module.
- Qualification is scoped to items with no enclosing class/impl. A method
already carries its owner's name, and that owner's id is mod-scoped by the
impl qualifier, so qualifying the method again breaks the same byte-for-byte
agreement. Same-named methods on same-named types in sibling modules
therefore still collapse — a narrower residual than the free-item case fixed
here, and one belonging to the owner edge rather than to this path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV
* fix(storage): bump schema versions for the mod-qualified Rust node ids (#2742)
`INCREMENTAL_SCHEMA_VERSION` 24 -> 25 and `SCHEMA_BUMP` 31 -> 32.
Node ids change for every Rust item inside any `mod` block, and
`#[cfg(test)] mod tests` makes that close to every Rust repository. A pre-v25
index therefore holds ids an incremental top-up cannot reconcile — the old nodes
would simply be stranded — so the reuse gate has to force a full re-analyze. The
qualified name is computed in the parse worker, so a warm parse cache would
likewise replay the old unqualified ids and keep the collapse.
This branch originally claimed v24; #2708 took that number and merged first, so
it is renumbered to v25 here. That is exactly the collision the v29 note in
parse-cache.ts warns about, and re-checking against origin/main at rebase time
rather than at branch time is what caught it. #2708 did not touch `SCHEMA_BUMP`,
so 32 is free.
The version-pin test moves with the bump by design, including the new pre-v25
row in the reuse-gate table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5nWP4BFeBSjbQ3uqAjtxV
* fix(rust): stop mod-qualifying container ids while their owner edges stay bare (#2745 review)
#2742 re-keyed Rust node ids by the enclosing `mod` chain. The mint moved; the
owner-edge anchor did not. `findEnclosingClassInfo` mints a member's owner id
from the container's BARE `nameNode.text` and only follows a qualified shape when
the provider sets `classExtractor.qualifiedNodeId`, which Rust does not.
So every `struct` / `trait` / `enum` / `impl` declared directly inside a `mod`
got a node id that none of its member edges pointed at. Five lines of idiomatic
Rust were enough:
pub mod engine { pub struct Config { pub retries: usize } }
NODE Struct:src/lib.rs:engine.Config
DANGLING HAS_PROPERTY Struct:src/lib.rs:Config -> Property:src/lib.rs:Config.retries
The rows are discarded by the IGNORE_ERRORS COPY retry, so the struct silently
lost every field. A trait impl inside a `mod` additionally dropped its
METHOD_IMPLEMENTS edge outright.
The same gap put `impl a::Inner` inside a `mod` back on the #1975 rake that
`qualifyByEnclosingModScope`'s own docblock warns about. The impl-target branch
deliberately fires only for an UNSCOPED `type_identifier`; the new gate had no
such restriction and picked up the scoped targets that branch had just excluded,
minting `Impl:<file>:outer.a.Inner` against an anchor still reading
`Impl:<file>🅰️:Inner`.
The member side was already excluded via `!enclosingClassInfo`. This adds the
owner side, gated on `MEMBER_OWNER_NODE_TYPES` — derived from
`CLASS_CONTAINER_TYPES`, which is already the single source of "this node type
owns member edges" and already carries an INVARIANT note binding it to
`CONTAINER_TYPE_TO_LABEL`. A language adding a container therefore cannot gain a
mismatched id shape here without also failing that invariant. Keyed purely on
tree-sitter node types, so no language name enters shared ingestion.
`union_item` is listed too: its fields are captured as Property but it is not a
recognized owner, so they carry no HAS_PROPERTY edge and cannot dangle — it is
here so a union's id keeps the same shape as the struct beside it.
Containers still collapse across sibling modules, exactly as before this fix.
That residual belongs to the owner edge, and is not worked around here.
Regression tests use the UNFILTERED `findDanglingEdges(result)`. Every other
dangling assertion in `rust.test.ts` passes `['HAS_METHOD']`, which is precisely
why the HAS_PROPERTY breakage shipped with a green suite. They assert the NODE
id rather than only the edge's anchor, because the anchor was already bare while
the bug was live — an edge-only assertion passes in both builds. All four fail
when the new gate clause alone is reverted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
* fix(rust): resolve modules nested inside an inline mod (#2745 review)
The #2730 self-loop survived one `mod` deeper:
pub mod outer {
pub mod tools { pub fn dispatch() {} }
pub fn dispatch() { tools::dispatch(); }
}
CALLS outer.dispatch -> outer.dispatch <- the #2730 symptom
NODE outer.tools.dispatch <- correct target, unlinked
Two gates were blind to a nested inline module, so the hook refused and the shared
lexical tier bound the call to the enclosing same-name `dispatch`:
`knownModuleNames` was collected by walking `moduleScopeByFile`, which maps a file
to its ROOT `Module` scope only. A `mod` nested inside an inline `mod` binds in the
parent module's scope, so the walk saw depth-1 inline modules and missed every
nested one — `tools` never entered the set and the negative filter rejected the
qualifier before any candidate ran.
`declaresSubmodule` had the same root-only assumption, so even with the name known
the candidate `outer::tools` was never yielded.
Both now read the def index. Names come from every `Namespace` def; inline module
PATHS are derived from the members' `namespacePrefix` rather than from the `mod`
defs, because a `mod` def carries no nesting information of its own — inside
`mod outer { mod tools { … } }` the inner def is `qualifiedName: 'tools'` with NO
`namespacePrefix`, while every def within it is stamped `outer.tools`. A
`Namespace` scope also owns its OWN def rather than its children's, so the scope
tree cannot answer this either: the `mod outer` scope lists `outer`, never `tools`.
Restricted to non-empty prefixes, so this stays a DECLARATION check. Including
file-derived modules would let an undeclared or `cfg`-gated file on disk outrank a
real `use` binding — the regression #2741's review already fixed once. File-backed
submodules therefore keep going through the binding check.
A module with no defs at all is absent from the set, which is harmless: it has no
member for a qualified call to resolve to.
Cost is one pass over an already-resident def index, memoized per resolution pass
on the existing WeakMap — the same order of work as the binding walk it replaces,
and it subsumes it. `isLocalNamespaceBinding` was going to single-source the
duplicated "locally declared submodule" predicate the review flagged; deriving
paths from members removed the second copy outright instead.
Regression fixture covers depth 2 and depth 3, so the fix is depth-agnostic rather
than depth-2 special-cased.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
* fix(rust): let an imported type outrank a same-named module (#2745 review)
Widening the negative filter to inline `mod` names let a type-qualified call
through whenever a module happened to share the type's name. The
crate-root-relative candidate then captured it:
// src/lib.rs
pub mod Buffer { pub fn with_capacity() -> usize { 111 } }
// src/b.rs
use crate::c::Buffer; // the real target lives in c.rs
pub fn call() -> usize { Buffer::with_capacity() }
base: (no CALLS edge — unresolved)
PR: CALLS b::call -> Function:src/lib.rs:Buffer.with_capacity <- fabricated
`ids.ts` states the doctrine this broke: a missing edge is the correct failure
direction for a graph whose consumers include `impact`; a fabricated caller is not.
The base produced the missing edge and the PR produced the fabricated one.
That third candidate is the loosest of the three — a guess at a crate-root-relative
path the caller never wrote, kept for 2015-edition style. In Rust 2018 a bare first
segment resolves in the CALLER's module, so a local binding for that segment
settles the question: it is now skipped when the head names anything non-module in
the caller's own module. Candidates 1 and 2 are untouched, and they run first, so
the legitimate `use crate::tools;` path is unaffected.
The binding lookup goes through `lookupBindingsAt`. A first attempt read
`Scope.bindings` directly and the guard never fired: a `use` binding is finalize
OUTPUT and absent from the scope's own local table, which is exactly the
imported-type case being guarded. Contract I8 in `contract/scope-resolver.ts`
requires that channel anyway.
The regression test asserts the forbidden TARGET rather than an empty edge set, and
separately asserts the module member still exists as a node — otherwise the test
would pass just as well if the call went unresolved for some unrelated reason, or
if the module node disappeared entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
* fix(scope-resolution): try the namespace-prefixed name on the TAGGED keys too (#2745 review)
`resolveDefGraphId` gained a namespace-prefixed retry for the plain qualified key
in this PR, but the five tagged keys above it — template constraints, parameter
types, parameter shape, arity, template arguments — kept composing from the bare
`qualifiedName`.
For a namespace- or `mod`-qualified def those keys are simply dead:
`node-lookup.ts` registers them under the QUALIFIED name (`inner.dispatch#0`)
while this side built `dispatch#0`. The keys exist to separate overloads, so a
mod-scoped overload set was relying on whichever later key happened to catch it.
Verified as a miss rather than a mis-hit before changing anything — an end-to-end
run with a crate-root decoy of the same name and arity binds correctly — so this
is hygiene, not a live bug. Worth doing while the code is open rather than leaving
five keys dead and the behaviour dependent on fallback order.
Both name forms now go through one `lookupTagged` helper, most specific first, so
a sixth tagged key cannot be added with the bare form only. That also removes the
five hand-repeated `qualifiedKey(...)` / `nodeLookup.get(...)` pairs.
Also pins the C++ `EXTENDS` retarget this PR's reorder produces.
`cpp-two-phase-dependent-base-cross-ns-deep` declares a global `Inner` decoy
alongside `ns:🅰️🅱️:Inner`; the base's `qualifiedName` is a bare `Inner` with the
path on `namespacePrefix`, so only the prefixed key separates them, and only if it
runs first. The improvement was riding unasserted in a Rust-scoped PR.
The captures golden covers every `rust-*` fixture, so the three fixtures added by
this review series drift it; regenerated with UPDATE_GOLDEN=1.
Verified: 785 tests across cpp / csharp / rust resolvers and the
callable-id-lockstep unit test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
* fix(rust): keep a mod declared inside a fn from hoisting above the callable (#2745 review)
`fn wrapper() { mod helper { fn dispatch } }` minted
`Function:<file>:helper.wrapper.dispatch@2:8` — the mod segment composed OUTSIDE
the enclosing-callable prefix, inverting the real nesting.
Nothing dangled: the `@line:col` suffix already makes a function-local callable's
id unique, which is also why the mod segment adds no identity in this position. The
path simply read as a lie about the source. It is now skipped rather than
reordered — interleaving two qualifier passes to fix the order would be real
machinery for a shape whose ids are already unique.
Also folds in the three documentation and structure findings from the same review:
- The 4-clause gate is extracted to a named `qualifiesByEnclosingModScope`, matching
the two conditions directly above it in the same function, which were already
named consts.
- `qualifyByEnclosingModScope`'s docblock documented only the impl-target contract
even though the generalized name has had a second, looser caller since #2742. It
now states both, and says which gate belongs to which — that gap is what let the
#1975 scoped-impl regression through in the first place.
- The "cheap rejection BEFORE any index work" comment was no longer true:
`passIndexFor` walks the def index on its first call in a pass. Corrected rather
than left to mislead the next reader into thinking the filter is free. What it
still buys — skipping the per-site candidate search, the part that scales with
the workspace — is stated instead.
Verified: 279 tests across the Rust resolver suite and the Rust scope-resolution
unit tests. Captures golden regenerated for the extended fixture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
* test(storage): move main's SCHEMA_BUMP pin to 32 for the mod-qualified ids (#2745 review)
`#2736` added a pin asserting `SCHEMA_BUMP === 31` on main, which arrived on this
branch through the merge of main while `0062a5c2` had already bumped the constant
to 32. Neither side conflicted textually — the pin and the constant live in
different files — so the merge was clean and the test failed instead.
That is the pin working as designed: it exists so a bump cannot ride along
unnoticed, and this is the fifth time a SCHEMA_BUMP collision has been caught by a
guard rather than by review. Updated to 32 with the reason recorded inline.
`INCREMENTAL_SCHEMA_VERSION` needs no second bump: 25 was introduced by this
unmerged branch, so no released index carries it, and its own pin in
`call-summary-schema-version.test.ts` is already consistent.
Verified: 119 tests across the parse-cache, schema-version, incremental-orchestration
and the two identity suites that arrived with the merge.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
* test(bench): rebaseline the Rust capture fingerprint for the three new fixtures (#2745 review)
CI caught what I missed:
[scope-capture --check] FAIL: rust: capture fingerprint drift
(got 05acbaca..., expected 90fda086...) fixture_count 202
`bench/scope-capture` fingerprints the whole `rust-*` fixture corpus, so the three
fixtures added by this review series drift it. I rebaselined the
`rust-captures-golden` snapshot and stopped there — a new fixture is a call site of
BOTH, and updating only one is how this reached CI red.
This is the same class as PR #2743's headline finding, from the other direction: an
id-shape change makes every synthetic corpus a call site, and the author fixed the
unit-test fixture and missed the bench. Here it is a fixture-count change rather
than an id-shape change, and the review that flagged the #2743 lead as "REFUTED,
bench/ has no Rust node-id corpus" was right about node ids and wrong about the
corpus fingerprint. Noted for the next author in the baseline entry itself.
Verified as pure corpus growth rather than a capture-logic shift: removing ONLY the
three new fixture directories and re-running reproduces the prior fingerprint
exactly (196 fixtures, capture_groups_fp 3432), and restoring them gives the new
one (202, 3556). `emitRustScopeCaptures` is untouched by this series. Scaling 1.022
local / 1.057 CI, well inside the 1.5 budget.
`bench/python-scope` globs `python-*` only and is unaffected; no other bench walks
the Rust corpus. `--check` now PASSes for all 15 languages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GvVxWt1ShhEP8CiMj3b6Z
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): reduce parse-phase memory for huge repos (#1983)
Stop retaining full parse-cache chunks in RAM alongside the merged graph,
slim on-disk shards, defer worker ParsedFile emission for scope-resolver
languages, and add GITNEXUS_DEBUG_HEAP probes for OOM diagnosis.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ingestion): address #2038 tri-review findings (parse-phase memory)
Resolves the confirmed review findings on PR #2038:
- P1: thread exportedTypeMap through the sequential parse path
(processParsingSequential) so a no-worker run over a partially-warm
cache no longer silently drops the sequential-miss files' exported
types. Cache hits made exportedTypeMap.size > 0, suppressing the
end-of-loop buildExportedTypeMapFromGraph rebuild, but the sequential
path never populated the map. Regression test added (fails on the
pre-fix tree, passes after) plus a fully-sequential differential oracle.
- P2: saveParseCache builds its on-disk index from hashes actually
written/copied (writtenKeys), never a usedKeys hash whose shard write
or copy was skipped — no more phantom index entries.
- P2: add a unit test asserting SCOPE_RESOLUTION_LANGUAGES stays in sync
with SCOPE_RESOLVERS (asymmetric drift would lose a language's ParsedFile).
- Backfill cache coverage: loadParseCacheChunk missing/corrupt -> undefined,
pruneCache onDiskKeys branch, slim preserves nodes, saveParseCache
copy-evicted-shard round-trip.
- Cleanups: single-source heap-probe gating via isDebugHeapEnabled();
hoist the per-chunk mkdir in persistParseCacheChunk behind a
process-scoped Set; gate COBOL's unused worker-side ParsedFile
extraction (graph nodes still come from cobolPhase) while keeping
fileCount/progress unconditional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): remove dead worker-side ParsedFile extraction
After #2038 gated worker `ParsedFile` emission behind `!isScopeResolutionLanguage(language)`, and with all 16 SupportedLanguages registered in SCOPE_RESOLVERS, that gate was structurally always true — the worker already produced no ParsedFiles and scope-resolution re-extracts each file from source on the main thread (run.ts). Remove the now-dead machinery:
- Drop both worker `extractParsedFile` call-sites (tree-sitter processFileGroup + the standalone-provider branch) and the `result.parsedFiles.push`. The standalone branch keeps fileCount/onFileProcessed per file. `result.parsedFiles` stays declared but empty (field removal deferred).
- Remove the now-orphaned `scopeSourceKind` var + `ScopeCaptureSourceKind`/`extractParsedFile`/`isScopeResolutionLanguage` imports.
- Delete the consumerless `migrated-languages.ts` (isScopeResolutionLanguage + SCOPE_RESOLUTION_LANGUAGES) and its drift-guard test — parse-worker was their only importer. Also improves AGENTS.md "shared ingestion code must not name languages" compliance.
`extractParsedFile` and the scope-extractor-bridge stay (scope-resolution/run.ts + Vue resolver use them). Behavior-preserving: worker-sequential-parity passes before and after; tsc/eslint clean; no baseline/golden drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ingestion): worker-pool-only parsing; remove sequential parser (#1983)
Completes the #1983 huge-repo parse-OOM effort by making the worker pool
GitNexus's sole parse path.
Parallel serialization (the perf core): workers serialize their ParsedFiles to
a disk store in parallel and stream them back to scope-resolution, so the main
thread no longer re-parses every file (the tree-sitter native-memory leak that
caused the OOM). Adds chunk merge-pipelining + work-proportional chunk sizing so
the pool stays saturated.
Remove the sequential parser: `--workers 0`, `GITNEXUS_WORKER_POOL_SIZE=0`, and
`skipWorkers` now hard-error (no silent degrade — #1741); the small-repo
threshold no longer selects an in-process path; pool creation stays lazy /
cache-miss-gated so warm all-hit runs never spawn workers.
Worker-path parity fixes — removing sequential surfaced two pre-existing gaps
that tiny-fixture tests had masked by running below the worker threshold, both
fixed by carrying per-file metadata as DATA across the worker boundary (never
re-parsing on the main thread, preserving the OOM fix):
- C++: templateConstraints wired into worker node identity (SFINAE overload
disambiguation) + ADL / inline-namespace capture side-channel serialized
onto the ParsedFile.
- Kotlin: companion-scope side-channel serialized the same way (companion /
static dispatch).
Validation: tsc + build clean; full suite green (10,190 pass — the only
deterministic failures were the now-fixed C++/Kotlin worker-path gaps; the 2
remaining full-run failures are pre-existing load flakiness, green in
isolation); cpp-pipeline benchmark stays linear on a 1-worker pool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): wire C static-linkage side-channel + ADL O(1) collect + tri-review cleanups (#1983)
Follow-up to the worker-pool-only refactor, from a tri-review of the parse path.
- C static-linkage side-channel (P1): cProvider had no collect/applyCaptureSideChannel,
so on the now-sole worker path C `static` file-local marks were lost across the worker
boundary -> false cross-file CALLS edges + over-broad #include wildcard visibility on
every C analysis (the Linux kernel is C). Mirror the C++/Kotlin wiring: serialize
`staticNames` per file onto ParsedFile.captureSideChannel and restore it on the main
thread (no re-parse). + a worker-path regression test (the existing c-static-isolation
fixture passed vacuously — its collision resolves via #include before the global
free-call fallback ever consults static-linkage).
- captureSideChannel `kind` discriminant: add `kind:'cpp'`/`kind:'c'` tags + guards
(Kotlin already had one) now that C/C++/Kotlin share the single generic field.
- Perf: collectCppAdlSideChannel scanned the whole argInfoBySite/noAdlSites maps per file
(O(F^2) per sub-batch, ~100M parseSiteKey calls at kernel scale). Add per-filePath
lockstep indexes -> O(1) collect; serialized snapshot byte-identical.
- Cleanups: inline the one-line processParsingWithWorkers wrapper into processParsing;
drop the always-empty WorkerExtractedData.calls/assignments/constructorBindings fields;
remove the voided astCache param from processParsing; refresh stale "sequential
fallback" JSDoc.
Validation: tsc + build clean; cpp 297/297, c 8/8 (incl. the new worker-path
static-linkage guard), typescript + parsedfile-store green; cpp ADL benchmark stays linear.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): index C/C++ #include resolution in finalize (O(n²)→O(n))
Kernel-scale C/C++ analysis ground in finalizeScopeModel because three
per-#include operations each did a full O(F) scan with no index — the
finalize O(n²) that surfaced once the #1983 parse-phase OOM was fixed:
- expand{C,Cpp}WildcardNames: parsedFiles.find() per wildcard edge → O(R·F)
- resolveImportTarget: new Set(allFilePaths) rebuilt per #include
- resolveCImportTarget: suffix-match scanned all workspace paths
Each is replaced with a WeakMap-per-pass index keyed on the stable
parsedFiles/allFilePaths references that scope-resolution run.ts passes
once per pass:
- Map<ScopeId,ParsedFile> for wildcard expansion (c/static-linkage.ts +
cpp/file-local-linkage.ts)
- memoized augmented header set (c/scope-resolver.ts + cpp/scope-resolver.ts)
- basename-bucketed suffix index in resolveCImportTarget (c/import-target.ts),
shared by C and C++ since resolveCppImportTarget delegates to it
Collapses the C/C++ finalize from O(R·F) to O(R+F). Pure-perf, byte-identical
edge output: 962 targeted tests green (490 C + 472 C/C++ scope-resolution);
the basename index preserves the exact endsWith('/'+target) match and the
fewest-path-components-then-lexicographic tie-break.
The kernel's ~25-30k .h headers are classified C++, so both providers must
be fixed. Proven on the Linux kernel: the C finalize completed
(sr-post-finalize lang=c → sr-end lang=c), which the pre-fix run never
reached in 16+ min of grinding.
Build-independent follow-ups (separate from this finalize fix), documented
for later: emitFreeCallFallback same-name buckets (emit phase),
buildGraphNodeLookup + precount global setup, the ParsedFile store-load,
the dart/go/ruby expand-wildcards .find siblings, and the ~26GB
scope-resolution memory floor (full kernel completion needs >~40GB RAM).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(bench): regenerate C scope-capture baseline for the #1983 c-static-linkage-worker fixture
bench/scope-capture/measure.mjs fingerprints emitCScopeCaptures over the
lang-resolution/c-* fixture corpus. The #1983 PR added the
c-static-linkage-worker fixture (caller.c/lib.c/lib.h/local.c — the
worker-path static-linkage side-channel test) but did not regenerate the C
baseline, so `--check` has been red on this branch (main, lacking the
fixture, still matches 0de009b).
Pure fixture-corpus drift — no c/captures.ts or query change branch-vs-main,
existing fixtures' captures byte-identical (c-captures.test.ts 45/45),
scaling stays linear (~0.97). Regenerated: 0de009b -> 39f3a83. Bench now
PASS (14 languages). Unrelated to the finalize O(n²) fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): lower kernel-scale resident memory floor + setup cost
Reduce the scope-resolution resident-memory floor and setup throughput on
huge repos (Linux kernel), the wall that remains after #1983 (parse OOM) and
the finalize O(n^2) fix (b71c77b8). Five units; all preserve byte-identical
edge output (C fixture 177n/255e + c/cpp/cross-file/php/static-linkage suites
green, 619 tests).
U1 (src/cli/analyze.ts): RAM-aware auto heap-cap. Replace the hardcoded
16384MB cap with computeHeapCapMb = max(16384, floor(0.75*effectiveRAM)),
where effectiveRAM = min(os.totalmem(), process.constrainedMemory()) with the
unconstrained-sentinel guard. Add --max-semi-space-size=128 on the respawn.
A user-supplied NODE_OPTIONS heap still wins (no re-exec). Verified: 23973MB
on a 31964MB box, 16384 floor on small machines, cgroup-aware, sentinel safe.
U2 (src/storage/parsedfile-store.ts, .../pipeline/phase.ts): export forceGc()
and call it at the per-language eviction boundary, so a finished language's
ParsedFiles are reclaimed before the next language's store-load instead of
collected lazily under the next pass's allocation pressure (which at cap>=RAM
degrades into swap-thrash). Measured on a real drivers/net/ethernet run:
C 2113->894MB and C++ 1754->1057MB reclaimed at the boundary (no fragmentation
defeat). Answers the plan's Open Question 1.
U3 (src/storage/parsedfile-store.ts): intern def objects by nodeId in the load
reviver so a SymbolDefinition's three serialized copies (localDefs /
scope.ownedDefs / scope.bindings[].def) collapse to one shared object on load.
Per-shard def pool (a def's copies are shard-local). Measured ~42% off the
def-object retained heap (3->1; 1.8M->600k distinct objects on 600k defs).
U4 (.../passes/free-call-fallback.ts): memoize pickUniqueGlobalCallable's
post-filter candidate list per (name, callerFilePath), only when no per-caller
visibility filter applies (the list is then a pure function of name+file), so
repeated free calls of one name from a file reuse the same-name-bucket scan
instead of re-walking a potentially huge bucket per site. The cached array is
read-only-consumed by the .filter()-based arity/overload narrowers. Exported
pickUniqueGlobalCallable + buildGlobalCallableIndex and added an equivalence
test (memoized == un-memoized reference for every (name, file, arity),
including warm-cache repeats and cross-file file-local exclusion).
U5 (.../pipeline/phase.ts): replace the O(L*F) per-language precount + repeated
scannedFiles.filter() with a single O(F) partition-by-language pass; bracket
buildGraphNodeLookup with scope-setup-nodeLookup heap probes so the long setup
is no longer silent.
Plan: docs/plans/2026-06-06-001-perf-kernel-scope-resolution-memory-plan.md
(U6 out-of-core global index deferred). Note: the kernel's full C++ pass floor
(~20k headers + the 8.8GB graph) likely still exceeds 24GB by itself, which is
why U6 remains the only unit that clears the wall.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(test): match OOM-guidance e2e assertions to the U1 reworded hint
The analyze-heap-oom-e2e real-child-OOM test still asserted the pre-U1
wording ('...out of memory.' + a hardcoded 24576 cap). U1 reworded the hint
to mention the auto heap-cap and use a <MB> placeholder, so the three
toContain substrings no longer matched (the assertion at line 62 failed on
all platforms). Update them to the current message. The unit twin
(analyze-heap-respawn) was already updated in 85bfc216; this integration
test was missed by the targeted local run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(lbug): U6a — deterministic id-sorted graph output behind GITNEXUS_SORT_GRAPH_OUTPUT
First increment of U6 (out-of-core scope-resolution). Adds an optional
deterministic ordering of node + relationship CSV rows by their unique graph
id, behind GITNEXUS_SORT_GRAPH_OUTPUT (default OFF = today's graph-insertion
order, byte-identical — the iterator is returned untouched). With the flag ON
the CSV becomes a pure function of the node/edge SET rather than of emit order.
This is the structural enabler for the windowed/out-of-core resolve (U6b-U6d):
csv-generator.ts:518 currently iterates graph.iterRelationships() in insertion
order with NO terminal sort, so any deviation from parsedFiles-order emit would
change bytes. With U6a on, a windowed emit need only reproduce the same edge
SET, not the global insertion order — removing the single largest byte-identical
hazard from every later windowing step.
Verified: default off keeps the existing csv-pipeline suite byte-identical; on,
node rows are id-sorted and output is independent of graph insertion order
(set-build) with the same node/edge set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(storage): U6d foundation — disk-backed scope store + lazy ScopeTree
Adds scope-index-store.ts: persistScopeShards (per-file scope shards via the
proven mapReplacer + def-interning reviver) + DiskBackedScopeTree, a lazy
ScopeTree that serves getScope from a bounded LRU of decoded shards plus a small
resident skeleton (scopeId -> {shard, childIds, parent}). Exports
makeInterningReviver from parsedfile-store for reuse.
This is the contained, highest-risk mechanism of U6d (out-of-core scope
resolution): the emit passes reach the heavy per-Scope binding payload
(~17-20GB on the kernel) ONLY through scopeTree.getScope (a point lookup) and
getChildren — they never read parsed.scopes directly — so moving that payload to
disk behind getScope is transparent. Every consumer reads a Scope BY VALUE, so a
value-faithful disk round-trip is byte-identical to resolution.
Proven in isolation: DiskBackedScopeTree is value-identical to buildScopeTree
for getScope/getChildren/getParent/getAncestors/has/size across multiple files
and after LRU eviction, and preserves the def-identity collapse (ownedDefs[i]
=== binding.def). Nothing wires it yet (the resolution-pipeline integration is
the next increment) — zero production impact; default off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d integration — seal scopeTree to disk before emit (GITNEXUS_DISK_SCOPE_INDEX)
Wires the U6d out-of-core scope index into the live pipeline behind
GITNEXUS_DISK_SCOPE_INDEX (default OFF = byte-identical). When on:
- finalize-orchestrator builds a TransitionalScopeTree (validated, fully
resident) instead of buildScopeTree, so finalize/propagate/resolve are
unchanged.
- After resolve, before emit, run.ts seals it: persists the scopes to a
file-sharded scope-index-store, swaps the model's scopeTree to disk-backed
serving from the inside (the frozen bundle can't be reassigned, but the
wrapper nulls its own resident backing), and drops the heavy Scope.bindings
payload from all THREE holders — the model's tree (seal), the caller's
preExtractedParsedFiles, and run.ts's own parsedFiles (scope-stripped copies
for emit). Emit reads scopes only via scopeTree.getScope (a point lookup,
now disk-backed + LRU) — verified it never reads parsed.scopes.
Purpose: lower the per-language resident PEAK (kernel C pass ~20→~12 GB by
moving the ~8-9 GB scope payload to disk) so the analysis fits on smaller-RAM
machines. At >=24 GB the full kernel already fits with U1-U5 (U2's 8.7 GB
inter-language forceGc reclaim keeps each pass under cap) — empirically
confirmed — so this is the sub-24 GB lever, not needed at 24 GB.
Byte-identical evidence: DiskBackedScopeTree/TransitionalScopeTree return
value-identical scopes vs buildScopeTree (getScope/getChildren/getParent/
getAncestors, across files + after LRU eviction + post-seal); emit reads only
getScope + referenceSites; flag-off (394 tests) and flag-on-resident (91 tests)
resolver suites stay green; an end-to-end A/B on a 212-file C+cpp+rust subset
produced identical 17,444 nodes / 31,343 edges with the seal firing per language
(c: 410→141 MB reclaimed). Kernel-scale peak-drop measurement pending the
in-flight verdict run freeing memory.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d — id-back workspaceIndex so the disk seal can reclaim scopes
The kernel run revealed the contained scopeTree seal didn't lower the heap:
WorkspaceResolutionIndex held Scope OBJECTS (classScopeByDefId / moduleScopeByFile),
built from every ParsedFile and live through emit, so the ~28k module + class
scopes stayed pinned past the seal (sr-seal-pre 17,583 -> sr-seal-post 17,771 MB,
no drop). It was the sole residual Scope-object holder (SemanticModel holds none).
Fix: classScopeByDefId / moduleScopeByFile become id-backed ScopeByKeyView
instances — a ReadonlyMap<K, Scope> facade over a K->ScopeId map + the scopeTree,
whose .get fetches via scopeTree.getScope(id). The index now pins only ids, so
once the tree seals to disk the scopes become collectible. Byte-identical: the
view returns the same Scope the resident tree holds (or a value-identical revived
one in disk mode), and iteration keeps the old insertion order. buildWorkspace
ResolutionIndex takes an optional scopeTree (live pipeline passes it); without it
(unit tests) the legacy direct Scope-object maps are returned unchanged.
Verified byte-identical: 733 tests across workspace-index / imported-return-types
/ c / cpp / cross-file / go / java. Kernel peak-drop re-measurement to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): U6d — precompute exportedCallableByName (fix disk-getScope thrash)
The workspaceIndex id-backing freed the kernel scopes but exposed a throughput
collapse: findExportedDefByName's workspace fallback (walkers.ts:1019) scanned
EVERY module scope's bindings per unresolved free call, and under the U6d
disk-backed scopeTree each module-scope access faulted a shard in from disk —
lib ON went ~1min -> ~7.5min.
Fix: precompute the fallback result once into
WorkspaceResolutionIndex.exportedCallableByName (simpleName -> first module-local
callable def, first-file-wins — the exact semantics the scan returned), built
from the resident module-scope bindings at index-build time. findExportedDefByName
now does an O(1) lookup with zero disk reads.
Result: lib ON ~7.5min -> 21s (cache-warm), byte-identical 17,444/31,343; 758
tests green across workspace-index + c/cpp/cross-file/go/python.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: rename cryptic U-unit codes to descriptive names in comments
The plan-unit shorthand (U3/U4/U6a/U6d/...) was meaningless in the code.
Renamed in comments + test descriptions (no behavior change, byte-identical):
out-of-core scope index (was U6)
deterministic output (was U6a)
disk-backed scope seal (was U6d)
def-object interning (was U3)
free-call candidate cache (was U4)
Also renamed throughout the PR title/summary. Pushed commit messages keep
their original U-codes as historical record.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): durable ParsedFile shards for warm-cache coverage (#2038)
On a warm re-analyze where every chunk is a parse-cache HIT, no parse worker
runs, the run-scoped ParsedFile store is cleared at parse start, and the cached
ParseWorkerResult carries no ParsedFiles (the worker writes them to the store
and empties them from the message). Scope-resolution then found an empty store
and fell back to main-thread extractParsedFile — re-opening the #1983
tree-sitter native-leak OOM the disk store closes (abhigyanpatwari review on
parse-cache.ts).
Fix: workers ALSO write their ParsedFiles to a durable, content-addressed store
(parsedfile-cache/) keyed by chunk hash, mirroring the parse cache's lifecycle
(version-gated by PARSE_CACHE_VERSION, pruned in lockstep to the surviving
keys). On a warm hit the chunk's durable shards are byte-COPIED into the
run-scoped store (no re-parse, no re-serialize -> byte-identical), so
scope-resolution streams them exactly as on a cold run. A coherence gate
re-dispatches the worker whenever a cached chunk's durable shards are missing
(migration / pruned / version-stale) -- never the main-thread extract.
- worker-pool/parse-worker: thread chunkHash through dispatch->job->flush
(incl. split/requeue) so the worker tags its durable shard by content
- parsedfile-store: durable persist / restore / index / prune API (sibling
dir, never cleared per run); content-addressing makes stale reuse impossible
- parse-impl: load durable index, gate the cache hit on durable coverage,
restore on hit, dispatch chunkHash on miss
- run-analyze: prune+save the durable store to the parse cache's surviving keys
- saveParseCache returns its written keys (the durable keepKeys)
Verified on linux/lib: warm preExtractedHits = full coverage (520/207/1, zero
main-thread re-parse), byte-identical cold==warm (17,456n/31,353e), warm 8.5x
faster. New two-run + mixed-mode + coherence-gate regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ingestion): clear stale scope-index-store shards on each seal (#2038)
The disk-backed scope index writes sequential s<n>.json shards into a shared
<storagePath>/scope-index-store/ dir, with the index resetting per
persistScopeShards call. A seal that writes fewer shards than a previous one
(a later language with fewer files, or a re-run of a shrunken repo) left stale
tail shards on disk indefinitely -- never read by the disk-backed tree, but
multi-GB on kernel-scale repos.
Add clearScopeIndexStore() and clear at the start of persistScopeShards: the
previously sealed language has finished emit and been released before the next
seal runs, so its DiskBackedScopeTree never reads those shards again. Unit
tests: a stale prior-run shard is removed, a fewer-files re-seal leaves no tail
shards, and the helper is idempotent.
Addresses abhigyanpatwari review on run.ts (disk hygiene for the
GITNEXUS_DISK_SCOPE_INDEX path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(ingestion): resolve FastAPI include_router(prefix=...) cross-file routes
FastAPI sub-route files declare paths via @router.<verb> while the entry
file mounts the router with app.include_router(<router>, prefix='/x').
Previously both the ingestion-layer Route graph nodes and the group-layer
ExtractedContract URLs lost the cross-file prefix, breaking provider <->
consumer matching.
Ingestion layer:
- parse-worker emits routerIncludes / routerImports + decoratorReceiver
- parsing-processor / parse-impl thread the new fields and aggregate
prefixesByModule across chunks; decorator routes whose receiver is
'router' are duplicated once per matching prefix
- routes.ts joins prefix via normalizeExtractedRoutePath
Group layer:
- HttpLanguagePlugin gains an optional prepareRepo() pre-pass and a
repoContext arg to scan(); python.ts builds prefixesByModule and
falls back to the bare path when no entry matches
- http-route-extractor caches one repoContext per plugin
Tests:
- 3 new http-route-extractor cases (attr / named-import / no-prefix)
- ParseWorkerResult literals in 3 test files updated to the new shape
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(ingestion,group): address PR #1877 review — relative imports, cross-package collisions, host names, ingestion tests
Follow-ups to the FastAPI `include_router(prefix=...)` cross-file fix
based on PR #1877's automated production-readiness review. Three
correctness gaps and one test coverage gap addressed:
1. Relative-import support in the worker regex (FINDING 2)
`FROM_IMPORT_ROUTER_RE` now accepts module paths starting with a
`.` (e.g. `from .calls import router as calls_router`). The
previous `[A-Za-z_][\w.]*` rejected leading dots and silently
dropped every relative-import Shape-B include — a real pattern
from the PR description's own motivating example. The matching
helpers now strip leading dots before keying so absolute and
relative imports collapse to the same module key.
2. Cross-package same-name module collisions (FINDING 3)
Two-tier module keying replaces the previous basename-only key:
• short key — `users` (file basename without `.py`)
• long key — `api/users` (parent dir + stem)
`prefixesByLongKey` is consulted first and only falls back to
`prefixesByShortKey` when no long-key match is available. Both
the ingestion pipeline (parse-impl.ts) and the group extractor
(http-patterns/python.ts) carry the same scheme so the graph
nodes and HTTP contracts agree on which prefix applies.
New protocol field `ExtractedRouterModuleAlias` (parse-worker →
parsing-processor → parse-impl) lets Shape-A
`<host>.include_router(<mod>.router, prefix='/x')` calls promote
to a long key when the same file imports `<mod>` via
`from <pkg> import <mod>`. Without this, `api/users.py` and
`admin/users.py` collided on the basename `users` and the admin
file's routes inherited the `/users` prefix that was only meant
for `api/users.py`.
3. Non-`app` host variable names (FINDING 4)
The group-layer `INCLUDE_ROUTER_*_PATTERNS` queries pinned the
host identifier to the literal `"app"` and dropped every
`application = FastAPI()` / `api = FastAPI()` pattern — the
constraint was redundant given that the call shape
(`include_router` invoked with a router argument and a
`prefix=` keyword) is already specific enough. The pin is
removed; the ingestion regex was already unrestricted.
4. Ingestion-layer regression tests (FINDING 1)
The previous PR added group-layer tests
(`http-route-extractor.test.ts`) but zero in-tree tests for the
ingestion path. Two new suites pin the
worker → parse-impl → routes flow:
- `test/unit/fastapi-router-bindings.test.ts` (23 cases):
`extractFastAPIRouterBindings()` is split into a stand-alone
module so it can be unit-tested without booting a worker
thread, then pinned for regex shape, two-tier key emission,
relative-import support, and negative cases.
- `test/integration/fastapi-prefix-pipeline.test.ts` (5 cases)
plus `test/fixtures/fastapi-prefix-app/` — runs the full
`runPipelineFromRepo()` against a realistic multi-package
fixture (containing both `api/users.py` and `admin/users.py`)
and inspects the resulting `Route` graph nodes for cross-file
prefix joining and absence of cross-package bleed.
Verification
- `npx tsc --noEmit`: pass
- PR-touched test suites (6 files / 117 cases): all green
- `npx prettier --check`: pass on touched files
- `npx eslint`: 0 errors on touched files
Cache / compatibility
The new `routerModuleAliases?` field on `ParseWorkerResult` and
`routerModuleAliases` on `WorkerExtractedData` are optional /
guarded with `?? []`, so historical parse-cache entries continue
to load without forced re-scan.
Refs PR #1877.
* refactor(ingestion): move fastapi-router-bindings out of workers/ — pure module, not a worker
Addresses @magyargergo's `CHANGES_REQUESTED` review on PR #1877:
> Sorry I just found that we are introducing a new worker in the PR.
`gitnexus/src/core/ingestion/workers/fastapi-router-bindings.ts` was a
**pure-function module** — it never imported `worker_threads` or
`parentPort`, never spawned a worker, and was never registered as a
worker entry. It was placed in `workers/` purely because it was split
out of `workers/parse-worker.ts` to make its functions unit-testable
without booting a worker thread (parse-worker is itself the worker
entry and cannot be loaded from the main thread).
To remove the misleading directory placement:
• The implementation moves to
`gitnexus/src/core/ingestion/route-extractors/fastapi-router-bindings.ts`,
alongside the other framework-specific route extractors (`expo`,
`nextjs`, `php`, `laravel`, `middleware`, `response-shapes`).
• `workers/parse-worker.ts` keeps a thin re-export so the worker
entry can keep using `extractFastAPIRouterBindings` directly. The
re-export now carries an explicit comment stating that the imported
file is **not** a worker and that the `workers/` directory
deliberately hosts only true worker entries (`parse-worker.ts`,
`worker-pool.ts`, `quarantine.ts`).
• The new file's leading docstring opens with "NOT A WORKER" and
explains why it exists where it does.
• The unit test (`test/unit/fastapi-router-bindings.test.ts`) is
updated to import from the new path.
No behaviour change. The function body, signatures, and exported types
are identical.
Verification
• `npx tsc --noEmit`: pass
• `npx tsc` (dist rebuild): pass
• `test/unit/fastapi-router-bindings.test.ts` (23 cases): all green
• `test/integration/fastapi-prefix-pipeline.test.ts` (5 cases): all green
• `test/unit/group/http-route-extractor.test.ts` (63 cases): all green
• `npx prettier --check` on touched files: pass
• `npx eslint` on touched files: 0 errors
Refs PR #1877.
* refactor(ingestion): drop parse-worker re-exports; consumers import router types directly from route-extractors
Addresses @magyargergo's two remaining review comments on PR #1877:
1. **`gitnexus/src/core/ingestion/workers/parse-worker.ts:247`** —
"Can you please remove them and update the call sites?"
The `export type { ExtractedRouterInclude, ExtractedRouterImport,
ExtractedRouterModuleAlias } from '../route-extractors/...'` block
in parse-worker.ts is gone. The remaining `import type {…}` is
purely local — used only to type the corresponding fields on
`ParseWorkerResult` below — and the leading comment now says so
explicitly ("this file does NOT re-export them"). The
`extractFastAPIRouterBindings` symbol is also no longer re-exported
from parse-worker.ts; it's still imported here so the worker entry
can call it per file, but downstream consumers must reach it via
`route-extractors/fastapi-router-bindings` directly.
Call sites updated:
- `gitnexus/src/core/ingestion/parsing-processor.ts`
- `gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts`
Both files now `import type { ExtractedRouterInclude,
ExtractedRouterImport, ExtractedRouterModuleAlias }` directly from
`route-extractors/fastapi-router-bindings.js`. The worker types
they still need (`ParseWorkerResult`, `ExtractedToolDef`, etc.)
keep coming from `workers/parse-worker.js`.
The unit + integration tests already imported from the new path,
so no test changes were required.
2. **`gitnexus/src/core/ingestion/parsing-processor.ts:168`** —
suggested simplification:
for (const item of result.routerIncludes ?? []) allRouterIncludes.push(item);
for (const item of result.routerImports ?? []) allRouterImports.push(item);
for (const item of result.routerModuleAliases ?? []) allRouterModuleAliases.push(item);
Applied verbatim. Replaces the previous `if (result.…) for …`
guards. The cache-compat semantics are unchanged — historical
parse-cache entries that lack these fields still load cleanly,
the new form just spells the fallback inline.
No behavior change, no tests touched, no public API change.
Verification
• `npx tsc --noEmit`: pass
• `npx tsc` (dist rebuild): pass
• PR-touched test suites (6 files / 117 cases): all green
• `npx prettier --check` on touched files: pass
• `npx eslint` on touched files: 0 errors
Refs PR #1877.
* refactor(ingestion): hoist fastapi-router-bindings type imports to top of parse-worker.ts
Move the `import type { ExtractedRouterInclude, ExtractedRouterImport,
ExtractedRouterModuleAlias }` block to the top of the file with the
other type imports, and drop the comment that previously sat next to
ExtractedDecoratorRoute.
---------
Co-authored-by: henry <zhangwei2017@unipus.cn>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
* docs: incremental indexing design spec
Captures the design agreed in brainstorming on 2026-05-10:
- Transitive importer closure with public-surface-change optimization
- Git-only change detection (non-git repos: full rebuild as today)
- New default behavior; --force opts out
- New hydratePhase + loadGraphFromLbug primitive
- Iterative closure expansion with parseCache reuse
- incrementalInProgress dirty flag for crash recovery
Prior art: PR #592 (zenprocess), PR #533 (davidbeesley),
PR #1146 (azeemshaik025) — referenced and credited.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(communities): seed Leiden RNG for deterministic community detection
The vendored Leiden algorithm defaults to Math.random for tie-breaking
and randomized walks, which produces non-deterministic community
assignments and modularity values across runs on the same graph.
Pass a seeded mulberry32 RNG (LEIDEN_SEED=0xC0DE) so:
- The same graph always produces the same partition
- Modularity values are reproducible
- Equivalence tests for incremental indexing can compare community
assignments byte-for-byte
This is foundational for the upcoming incremental-indexing feature
(see docs/superpowers/specs/2026-05-10-incremental-indexing-design.md)
where the correctness contract is incremental output ≡ full rebuild
output.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(incremental): change-detection, surface signatures, closure expansion
Three new modules supporting the incremental-indexing pipeline:
* core/incremental/git-diff.ts — getChangedFilesSinceCommit() unions
'git diff lastCommit HEAD' (committed) with 'git status --porcelain'
(dirty tree). Renames flattened to delete(orig) + add(new). Throws
LastCommitMissingError when lastCommit is gone (caller falls back to
full rebuild).
* core/incremental/surface.ts — extractSurfaceSignature() produces a
stable hash of a file's publicly-visible symbols (functions, classes,
methods, interfaces, types, heritage). Body-only edits → same hash.
Signature/heritage changes → different hash. Drives the closure
scoping optimization.
* core/incremental/closure.ts — computeImporterClosure() iterative
fixpoint: parse each closure file, extract surface, query DB
importers, expand. Uses a parseCache so each file is parsed once.
Generic over TParseResult so closure logic is decoupled from the
pipeline's parse representation.
32 unit tests across the three modules. Tests cover edge cases:
clean tree, dirty-only, mixed, renames, deletes, multi-hop cascade,
cycle termination, surface invariance, etc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(lbug): loadGraphFromLbug, queryImporters, deleteAllCommunitiesAndProcesses
Three new primitives in lbug-adapter.ts to support incremental indexing:
* loadGraphFromLbug(graph, unchangedFilePaths) — streams all nodes for
files in the set across every hydratable node table (excludes
Community/Process — graph-wide, regenerated downstream). Then loads
edges where both endpoints belong to loaded nodes, excluding
MEMBER_OF / STEP_IN_PROCESS edges (also graph-wide).
FilePaths chunked at 200 per query to keep statement size bounded
on huge repos. Endpoint-level join filters by source-side filePath
in the query, target-side checked JS-side via the loadedNodeIds set.
* queryImporters(targetFilePath) — returns DISTINCT a.filePath where
a -[IMPORTS]-> b and b.filePath = target. Powers closure expansion:
when a changed file's surface signature changes, all its importers
must be re-parsed.
* deleteAllCommunitiesAndProcesses() — drops Community/Process nodes
(and their edges via DETACH DELETE) at the start of each incremental
run so the communities/processes phases regenerate them from the
fully-merged graph. Required for the 'Leiden runs on full graph'
correctness invariant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(pipeline): hydrate phase + parse-filter for incremental indexing
Wires the incremental-indexing infrastructure into the phase-based
pipeline. Three coordinated changes:
* New hydratePhase (deps: structure) — loads node/edge state for files
OUTSIDE ctx.options.filesToParse from the existing LadybugDB index.
Runs before parse so the parse phase can produce a partial graph
while downstream phases (mro, communities, processes) still see the
full graph. No-op in full-rebuild mode (filesToParse unset).
* PipelineOptions.filesToParse: optional ReadonlySet<string>. When
set, parse phase filters scanned files to this set; hydrate fills
the complement. Set by runFullAnalysis when it detects an eligible
incremental run; never set by callers directly.
* gitnexus-shared PipelinePhase enum: 'hydrate' added so progress
callbacks can report the new phase distinctly from 'structure'.
Phase order: scan → structure → hydrate → markdown,cobol → parse
→ routes,tools,orm → crossFile → scopeResolution → mro → communities
→ processes. Communities (Leiden) still runs on the full graph,
satisfying the correctness invariant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(analyze): incremental orchestrator branch + meta schema
Wires incremental indexing into runFullAnalysis. Highlights:
* RepoMeta schema extended: schemaVersion, surfaceSignatures, and
incrementalInProgress fields. INCREMENTAL_SCHEMA_VERSION = 1.
* core/incremental/file-hash.ts — v1 surface signature: SHA-256 of file
content. v2 will switch to a true surface-only signature (defined in
surface.ts) so body-only edits don't expand the closure. The plumbing
is signature-agnostic so the swap is local.
* core/incremental/orchestrator.ts — eligibility check, closure
computation (uses file-hash as the surface signal), dirty-flag
management, subgraph extraction, signature merge.
* run-analyze.ts adds:
- hasDirtyTree() check on the existing 'lastCommit==HEAD' early-exit
so an uncommitted edit triggers re-index (was a coarse equality
check before).
- incremental branch: try incremental first; fall through to full
rebuild on any setup failure or eligibility miss.
- runIncrementalBranch() — opens existing DB, deletes closure-file
rows + Community/Process, runs pipeline with filesToParse, writes
only the changed-subgraph back, refreshes FTS, updates meta with
new surfaceSignatures and clears the dirty flag.
- Full-rebuild path now populates surfaceSignatures + schemaVersion
in meta.json so the next run is eligible for incremental.
Crash recovery: incrementalInProgress is set BEFORE any DB mutation
and cleared on success by overwriting meta.json. A crash anywhere in
between leaves the flag set, and the next analyze run forces a full
rebuild (cheapest path back to a known-good index).
v1 limitation documented: body-only edits trigger 1-hop closure
expansion (content-hash signal). True surface-only optimization is
deferred to v2 — see design doc for the integration path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(incremental): drop invalid --no-renames=false from git diff
The flag --no-renames=false isn't valid git syntax (it's parsed as a
file path). Git's default rename detection is on; removing the flag
keeps that behavior.
Caught while running an end-to-end smoke test against a small fixture
repo: incremental setup failed with 'Command failed: git diff
--name-status -z --no-renames=false ...'. After the fix, the
incremental path runs cleanly: closure is computed, hydrate phase
loads unchanged-file state from DB, parse phase only re-parses files
in closure, and the writeback updates only changed nodes/edges.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Revert v1 incremental indexing (5 commits)
Reverts the v1 design that parsed only closure files into a fresh
graph and tried to hydrate the rest from DB. Real-repo equivalence
test failed: cross-file resolution operates on partial parse data
(closure files only), so CALLS edges that resolve through unchanged
files silently fall off. Diff against full rebuild on the same
edited state: -50 nodes, -425 edges, -5 communities, -48 processes.
Architecture pivot: switch to PR #533-style content-addressed parse
cache. Pipeline parses every file (cache-served when possible),
giving cross-file resolution full data, with DB writeback then
restricted to changed-file rows.
Reverts:
d4b9de47 fix(incremental): drop invalid --no-renames=false
f35f7634 feat(analyze): incremental orchestrator branch + meta schema
bc039686 feat(pipeline): hydrate phase + parse-filter
98bb893d feat(lbug): loadGraphFromLbug, queryImporters, ...
aa8d7ae3 feat(incremental): change-detection, surface signatures, closure
Kept:
d9e340b0 feat(communities): seed Leiden RNG (foundational)
8235ca36 docs: incremental indexing design spec (will be revised)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(analyze): incremental DB writeback (Option B)
Equivalence-preserving incremental analyze. The pipeline still parses
every file (correctness invariant: cross-file resolution / scope
resolution / MRO / community detection all need full graph data); the
saving comes from selectively replacing only changed-file rows in
LadybugDB instead of wiping and reloading the whole graph.
How it works:
* On every analyze, we hash all source files (SHA-256 of content) and
store the map in meta.json.fileHashes alongside schemaVersion.
* The next run loads the prior map and diffs:
- changed: content hash differs → file's DB rows replaced.
- added: not in prior map → file's DB rows inserted.
- deleted: in prior map but not on disk → file's DB rows dropped.
* If the diff is non-empty AND no --force / no schema mismatch / no
dirty flag, take the incremental path:
- Set incrementalInProgress dirty flag (BEFORE any DB mutation).
- Open existing DB (no wipe).
- deleteNodesForFile() for each changed/added/deleted file.
- deleteAllCommunitiesAndProcesses() — Leiden regenerates these.
- extractChangedSubgraph() from the in-memory ctx.graph: nodes whose
filePath is in the writable set + Community + Process + edges with
at least one endpoint in the writable set (edges entirely between
hydrated unchanged nodes are skipped — already in DB).
- loadGraphToLbug() on the subgraph. Unchanged-file rows in DB
untouched.
- Recreate FTS indexes.
- Update meta with new fileHashes; clear dirty flag.
* Otherwise full-rebuild path runs as before.
Crash recovery: incrementalInProgress is the dirty flag. Set before
destructive ops; cleared on success. Set on next-run startup → forces
full rebuild (cheapest path back to known-good).
Other changes:
* Dirty-tree gate on the existing 'lastCommit==HEAD' early-return:
uncommitted edits no longer slip through as 'already up to date'.
* deleteAllCommunitiesAndProcesses helper in lbug-adapter.
* Skip the embedding cache+restore cycle when willTryIncremental is
true — embeddings stay in DB; re-inserting them would PK-conflict.
End-to-end equivalence verified on this repo (993 files, 24K nodes):
incremental run produces byte-identical {nodes, edges, clusters,
flows} to a full rebuild from the same edited state.
Speedup is currently modest (~5% on this repo) because the parse
phase still runs in full. Parse-cache integration is a separate
follow-up that composes cleanly on top of this work.
See docs/superpowers/specs/2026-05-10-incremental-indexing-design.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(analyze): chunk-level parse cache for full incremental speedup
Composes with the incremental DB writeback (commit 27f3b49d) to deliver
the major-speedup half of incremental indexing. Previously, the parse
phase ran in full on every analyze; the speedup came purely from
selective DB rewriting. With this commit the parse phase also reuses
prior tree-sitter output for chunks whose contents haven't changed.
How it works:
* Cache layer (gitnexus/src/storage/parse-cache.ts):
- File: <repo>/.gitnexus/parse-cache.json. Versioned, atomic write.
- Key: chunk content hash = sha256(sorted(filePath:fileContentHash
for each file in chunk)).
- Value: ParseWorkerResult[] (raw worker output for the chunk,
pre-merge).
- Granularity: per chunk (~20MB byte-budget). A change to one file
invalidates only its chunk — typically 1 of ~50 on a 1000-file
repo (~98% cache hit ratio on a small edit).
* Worker contract (gitnexus/src/core/ingestion/parsing-processor.ts):
- Extracted the chunk-result merge loop into a public
mergeChunkResults() so the same logic applies to live worker
output AND replayed cache entries.
- processParsingWithWorkers / processParsing accept an optional
outRawResults out-parameter that captures worker output before
merging — used by parse-impl to populate the cache after a miss.
* Parse phase wiring (parse-impl.ts):
- For each chunk, compute its content hash (after reading file
contents). Cache hit → mergeChunkResults() on cached results,
skip the worker dispatch entirely. Cache miss → run workers
normally, capture raw results, store under the chunk hash.
- Cache mutations happen in-place on the ParseCache passed via
PipelineOptions.parseCache.
* Lifecycle (run-analyze.ts):
- loadParseCache() before pipeline runs.
- Cache passed via runPipelineFromRepo's PipelineOptions.
- saveParseCache() after the pipeline + DB writeback succeed.
Equivalence verified on this repo (993 files, 24K nodes):
Cold (no cache, full work): 141.1s
Warm cache + 1-file edit, incremental: 63.6s ← 55% speedup
Warm cache + 1-file edit, --force: 71.6s ← 49% speedup
All three runs produce byte-identical {nodes, edges, clusters,
flows}. The cache survives --force (content-addressed = always
correct), so even forced rebuilds get the parse-skip benefit.
Why chunk-level rather than per-file: workers process sub-batches and
emit aggregated ParseWorkerResults. Per-file granularity would require
restructuring the worker contract; chunk-level captures most of the
practical speedup with no worker-side changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* perf(parse-impl): smaller default chunk budget (20MB→2MB) for cache granularity
The parse cache is keyed at chunk granularity. With the previous 20MB
budget, a typical mid-size repo (e.g. this worktree at 9MB total
parseable source) fits in a single chunk — meaning ANY file change
invalidates the whole chunk and re-parses every file.
2MB default produces ~5x more chunks on the same input, so a one-file
edit invalidates ~1/N of cached chunks instead of the whole thing.
Cold-run overhead from more chunks is <5% (one extra serialization
pass per chunk).
Override via GITNEXUS_CHUNK_BYTE_BUDGET env var for benchmarking.
Measured on this repo (~9MB / 887 parseable files):
Cold (no cache): 143s
Warm cache, no source changes: 2s (early-return)
Warm cache + 1-file edit: 81s (~43% off cold)
Speedup is bounded by the scopeResolution phase (~58s flat regardless
of parse cache) and by GitNexus's own auto-writes during analyze
(AGENTS.md / .claude/skills/ etc. mutate between runs and invalidate
chunks containing them). Both are addressable in follow-ups.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* perf(scope-resolution): reuse worker-produced ParsedFile + stabilize chunk order
Two compounding optimizations that drop warm-cache analyze from
~134s to ~38s on a 1000-file repo (72% faster), and cold rebuild
from ~143s to ~86s (40% faster) by short-circuiting work that was
previously re-done.
1. SCOPE-RESOLUTION: REUSE WORKER PARSEDFILE
Previously, the scope-resolution phase re-parsed every file with
tree-sitter on the main thread (~58s on a 1000-file repo) because
worker-produced tree-sitter Trees can't cross the worker MessageChannel.
But the worker ALSO produces a artifact via
, which structured-clones fine — and it's exactly
what scope-resolution would re-derive. Threading those ParsedFiles
through the parse phase () into
( map) lets scope-
resolution skip its extract loop on a per-file basis.
The fast path is bounded only by per file (cheap
graph mutation). On this repo: scopeResolution went from 58s → 5s.
2. MAP-PRESERVING PARSE-CACHE SERIALIZATION
is a
which JSON.stringify collapses to . The first attempt at threading
parsedFiles through the parse cache crashed at runtime with
"importerModule.typeBindings is not iterable" because cached entries
came back as plain objects.
Added a JSON replacer/reviver pair in parse-cache.ts that round-trips
Map and Set instances through tagged plain objects (). Symmetric: save uses replacer, load uses reviver.
3. STABLE CHUNK ORDERING
The byte-budget chunker walked files in filesystem-scan order, which
on Windows isn't guaranteed to be stable across runs. Even with
identical source content, two scans could place files in different
chunks, shifting chunk hashes and causing 100% parse-cache misses.
Added a deterministic alphabetical sort on before
chunking. Chunk membership is now stable across runs, so a single-file
edit invalidates exactly one chunk, not all of them.
Measured on this repo (993 files, 24K nodes):
Cold rebuild: 86s (was 143s)
Warm cache, no source changes: 3s (early-return)
Warm cache + 1-file edit: 38s (was 134s)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(incremental): update spec + AGENTS.md + GUARDRAILS.md for shipped design
- Rewrite docs/superpowers/specs/2026-05-10-incremental-indexing-design.md
to describe the architecture that actually shipped (parse cache +
incremental DB writeback + scope-resolution short-circuit), with the
v1 hydrate-phase post-mortem preserved as historical context.
- AGENTS.md "Keeping the Index Fresh" section: note that incremental
is the new default and --force is the explicit opt-out; mention
the parse-cache file location and that it's safe to delete.
- GUARDRAILS.md Signs: add an "Index seems corrupt or incremental is
misbehaving" entry pointing users to --force as the manual escape
hatch (the dirty flag handles automatic recovery).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(autofix): apply prettier + eslint fixes via /autofix command
* fix(incremental): bugbot review + CI test failures
Bugbot (PR #1479):
- Medium: pruneCache was exported but never called -> cache grew
unbounded. Wire pruneCache into run-analyze before saveParseCache,
using a transient usedKeys Set on ParseCache that the parse phase
populates as it processes chunks.
- Low: willTryIncremental (pre-pipeline) and isIncremental
(post-pipeline) could desync, silently dropping embeddings on
mispredicted runs. Removed the prediction; the embedding cache
now loads unconditionally when shouldLoadCache is true. The
re-insert step gates on the actual isIncremental value to avoid
PK-conflicts when the incremental-writeback path keeps DB rows.
CI test failures:
- cli-e2e #1169 + run-analyze.test.ts #1233: my dirty-tree gate on
the lastCommit==HEAD early-return saw GitNexus's own auto-generated
outputs (.claude/, .cursor/, AGENTS.md, CLAUDE.md) as dirty,
perpetually defeating the up-to-date fast path. Extended the
pathspec exclusion to cover all auto-gen outputs, not just
.gitnexus/.
- ruby field-type disambig: my chunk-stability sort exposed a
pre-existing order-dependency in Ruby cross-file resolution
(`user.address.save -> Address#save` only resolves correctly when
user.rb parses before address.rb in some configurations). Removed
the sort. Filesystem ordering is stable enough in practice that
the parse cache still hits the common case; the pre-existing
fragility is left for a separate fix.
- pipeline-graph-golden: regenerated. Seeded Leiden RNG produces a
partition different from the previous Math.random snapshot.
- staleness `parallel calls` was a CI timing flake; passes locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(incremental): re-insert cached embeddings on incremental path
Bugbot re-review caught: deleteNodesForFile cascades to the
CodeEmbedding table (DELETE WHERE e.nodeId STARTS WITH ...), so
changed-file embedding rows are wiped along with their nodes. The
previous fix gated re-insert on `!isIncremental`, which silently
dropped those embeddings — a regression versus the full-rebuild path's
"preserve embeddings by default" guarantee.
Remove the `!isIncremental` gate. The per-batch try/catch already
handles the unchanged-file PK-conflict case ("some may fail if node
was removed, that's fine") with the same semantics, so re-inserting
the full cached set on incremental works:
- changed-file rows: deleted, then re-inserted from cache (preserved)
- unchanged-file rows: still in DB, re-insert PK-conflicts and is
silently ignored (existing rows are correct)
Cost: re-inserting ~24K embeddings on incremental when only a few
files changed — most are no-op conflicts. Bounded by batch size of
200; ~3-5s overhead. Worth it for correctness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(incremental): address Claude+Bugbot review findings + remove design doc
Addresses CHANGES_REQUESTED review on PR #1479:
1. Remove docs/superpowers/specs/2026-05-10-incremental-indexing-design.md
per maintainer request.
2. BLOCKER (Claude Finding 1, Bugbot Round 3): Stale cross-file edges
between unchanged files. extractChangedSubgraph excluded edges where
both endpoints were unchanged-file nodes — when a barrel/re-export
file changes, cross-file resolution may update CALLS edges between
two unchanged files that would then be silently lost.
Fix: 1-hop importer-closure expansion of the writable set in
run-analyze.ts. Before deleting/rewriting rows, query DB for
importers of every changed/deleted file and add them to the writable
set. Their nodes get deleted+rewritten too, so cross-file's refined
edges land in the DB. Re-added queryImporters to lbug-adapter.ts.
3. BLOCKER (Claude Finding 3): Parse cache key omitted parser version.
After a GitNexus upgrade, the cache silently replays pre-upgrade
ParseWorkerResults against the new schema → wrong CALLS/IMPORTS/
scope edges with no visible signal.
Fix: PARSE_CACHE_VERSION now embeds the gitnexus npm package
version (read at module load via createRequire on package.json).
Format: `${SCHEMA_BUMP}+${PKG_VERSION}` e.g. "1+1.6.4". Any release
that bumps package.json automatically invalidates the on-disk cache.
Mismatched versions fall through to an empty cache (next save
overwrites with the new version baked in).
4. BLOCKER (Claude Finding 2): No automated tests for incremental
behavior. Added 28 unit tests across 3 files:
- incremental-file-hash.test.ts (10 tests)
diffFileHashes classification, computeFileHash determinism,
computeFileHashes batch / missing-file tolerance, sorted output.
- incremental-parse-cache.test.ts (12 tests)
computeChunkHash stability and order-independence, version
prefix format, pruneCache, load/save round-trip on empty /
missing / corrupt / version-mismatched files, AND a Map/Set
round-trip test that pins the JSON replacer/reviver behaviour
(without it, ParsedFile.scopes[*].typeBindings collapses to
{} and downstream `.get()` / iteration throws).
- incremental-subgraph-extract.test.ts (6 tests)
writable-set node inclusion, Community/Process always kept,
edge inclusion when at least one endpoint is writable, MEMBER_OF
edges via graph-wide endpoints, empty subgraph case.
5. Medium (Claude Finding 6): AGENTS.md "Keeping the Index Fresh"
said "only changed files are re-parsed." Imprecise — the pipeline
parses every file every run; the cache skips tree-sitter for chunks
whose contents haven't changed. Reworded to match the design doc.
Test plan still expects:
[x] Typecheck clean
[x] All 28 new unit tests pass
[x] All previously-failing tests still pass on the rebased branch
[x] Equivalence verified locally (incremental ≡ --force, byte-identical
stats on this repo)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(incremental): round 3 review feedback — bounded BFS, atomic meta, integration test, docs
Addresses remaining findings on PR #1479 from Claude's re-review of
commit ad7bd31 + verifies the outstanding Bugbot HIGH severity.
1. F1 — Transitive importer expansion (Claude, was Medium-but-noted).
Previous 1-hop importer expansion missed barrel re-export chains
(A imports C, C re-exports B; when B changes, only C was pulled in
— A was left with potentially-stale CALLS edges to refined targets).
Replaced the single pass with a bounded BFS over the IMPORTS graph
(depth ≤ 4). Catches nested barrel pyramids without ballooning into
a near-full rebuild on monorepos with deep re-export trees. `--force`
remains the escape hatch documented in GUARDRAILS.md for cases that
exceed the bound.
2. F2 — Integration test for incremental orchestration (Claude, BLOCKER,
DoD §2.7). The unit tests added in ad7bd31 covered `diffFileHashes`,
`extractChangedSubgraph`, `computeChunkHash`, `pruneCache`, and the
Map/Set JSON round-trip — but none of them exercised the real
`runFullAnalysis` orchestration. Added gitnexus/test/unit/
incremental-orchestration.test.ts with four end-to-end tests against
a real git-initialized fixture repo + real LadybugDB:
a. First run populates fileHashes + schemaVersion and clears
incrementalInProgress on success.
b. Second run on unchanged state takes the alreadyUpToDate fast
path (early-return).
c. Second run after a source edit takes the incremental path
(not full rebuild) and rotates fileHashes for the touched file
while keeping the dirty flag cleared.
d. A pre-set incrementalInProgress flag forces a full rebuild
that clears it (crash-recovery wire).
These would catch any regression that wires `isIncremental` from a
pre-pipeline prediction (the Bugbot finding from commit 5eb0597) or
accidentally re-gates the embedding re-insert on `!isIncremental`
(the Bugbot finding from commit 60c10f1).
3. F3 — GUARDRAILS.md docs accuracy (Claude, Low). Line 33 still said
"only changed files are re-parsed" — AGENTS.md was already corrected
in ad7bd31 but GUARDRAILS.md was missed. Reworded to match.
4. F5 — Atomic saveMeta (Claude, Medium; vvladescu-tb fork). The dirty
flag (`incrementalInProgress`) travels through meta.json. A crash
mid-write would leave a corrupt meta.json that `loadMeta` would
silently treat as "no prior index", losing the flag and skipping
recovery. Switched to tmp-file + rename matching saveParseCache.
5. Bugbot's "Subgraph edges reference nodes absent from subgraph"
(HIGH severity). Verified as FALSE POSITIVE: `getNodeLabel` in
lbug-adapter.ts derives labels from the node-ID string (parses
the table prefix), not from the in-memory graph. The CSV
generator writes (src_id, dst_id, type) rows without consulting
node objects; `splitRelCsvByLabelPair` routes by ID-derived label;
`COPY ... (from=X, to=Y)` resolves both endpoints against the live
LadybugDB where unchanged-file nodes still exist. No fix needed.
All 213 tests pass locally (including the 4 new integration tests
and the previously-failing CI tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(incremental): address Bugbot round-4 findings (added-file shadow seed + dedupe)
Bugbot review on commit e23e4400 surfaced two new findings against the
incremental writeback in run-analyze.ts:
HIGH — Incremental BFS misses importers of newly added files.
queryImporters() reads the pre-pipeline DB. For a NEWLY ADDED
file there are no IMPORTS rows pointing to it yet, so unchanged
files whose pre-existing import statements now resolve to the
newcomer keep stale CALLS edges pointing at the OLD resolution
target.
LOW — Deleted files double-counted in filesToDelete.
hashDiff.deleted entries can reappear in writableFiles via the
BFS expansion (queryImporters can return a now-deleted path),
so deleteNodesForFile() ran twice for the same file.
Fixes:
- Add gitnexus/src/core/incremental/shadow-candidates.ts: derive
the pre-existing file paths whose JS/TS module-resolution claim
an added file can steal. Pattern catalogue: same-basename/
different-extension, bare-file-beats-directory-index, and
directory-index-beats-bare-file. Emit both POSIX and Windows
separators because the prior fileHashes map may have been
written from either OS.
- In run-analyze.ts, seed the BFS frontier with shadow candidates
that exist in the prior meta.fileHashes. Their importers — found
via queryImporters — get pulled into the writable set so their
CALLS edges re-resolve against the new file.
- Dedupe filesToDelete via Set to avoid the double-call.
Tests: gitnexus/test/unit/incremental-shadow-candidates.test.ts —
8 cases covering each shadow pattern, separator handling, .d.ts as
a single extension token, deduplication, and the no-self-shadow
invariant. All 40 incremental tests (file-hash, parse-cache,
subgraph-extract, shadow-candidates, orchestration) pass locally.
Note on the third Bugbot finding ("Subgraph edges reference nodes
absent from subgraph"): re-anchored from a prior review pass — the
code at subgraph-extract.ts:48 is unchanged. Already verified as a
false positive: getNodeLabel parses labels from ID strings, CSV
write is by ID, and COPY resolves against the live DB.
* chore(autofix): apply prettier + eslint fixes via /autofix command
* test(incremental): exact-equality stats invariant + analyze ≡ analyze --force
Addresses the only remaining Claude production-readiness review finding
on PR #1479 (Low-Medium, test-quality only — Claude itself said it does
NOT block merge, but the central PR claim "incremental ≡ full rebuild"
deserves explicit CI coverage rather than implicit trust).
Changes to gitnexus/test/unit/incremental-orchestration.test.ts:
1) Tighten the existing "comment-only edit takes incremental path" test.
- Replace toBeGreaterThan(0) bounds assertions on stats.files and
stats.nodes with exact toBe(firstMeta) per-field equality across
files / nodes / edges / communities / processes. DoD §2.7 calls
out bounds-only assertions as masking regressions that drop half
the graph; this swap closes that gap.
- Rationale: a comment-only edit must change the file content hash
(driving the incremental path) without changing any graph data.
Therefore every stat MUST be identical to the first run. Anything
else is a regression.
2) New test: incremental output is byte-equivalent to a full rebuild.
- Run analyze → comment-only edit → analyze (incremental writeback)
→ analyze --force (full rebuild from same on-disk state).
- Assert files / nodes / edges / communities / processes are exactly
equal across the incremental and the --force passes.
- This is the PR's central correctness contract, now proven by a
test that exercises the real runtime path end-to-end against a
real on-disk LadybugDB.
All 5 orchestration tests pass locally (52s), including the new
equivalence test — every stat field matches exactly between incremental
and --force on the mini-repo fixture.
tsc --noEmit clean.
* fix(incremental): F1 cross-file edge consistency + F4 stable chunk sort + unit coverage (#1511)
Patch addressing two of the still-open changes-requested findings on PR
#1479, rebased onto the current feat/incremental-indexing head. F3
(parser fingerprint in the cache key), F5 (atomic saveMeta), and F6
(AGENTS.md phrasing) were already handled on the branch, so the
corresponding parts of the original patch were dropped as redundant.
F1 (Blocker) — Cross-file edges between unchanged files
Adds `computeEffectiveWriteSet(graph, toWriteSet)` to
subgraph-extract.ts: a single pass over the new graph's edges that
pulls the unchanged-side file of every writable-boundary-crossing
edge into the write set. run-analyze composes it ON TOP of the
existing importer-BFS expansion and feeds the combined set to BOTH
`deleteNodesForFile` and `extractChangedSubgraph`, so the delete
cascade and the writeback subgraph cover identical files (asymmetry
would leave stale rows or PK-conflict at COPY time). The BFS reads
IMPORTS from the pre-pipeline DB (catches files that *stopped*
importing a changed file); the edge walk reads the new graph
(catches refined CALLS edges the pre-run DB couldn't predict, e.g.
a barrel re-export shifting a symbol from B to D). `extractChangedSubgraph`
stays a pure filter — all expansion is the orchestrator's job.
F4 (Medium) — Restore alphabetical chunk sort
`parseableScanned` is sorted before chunking. Filesystem-scan order
isn't stable enough across runs/platforms (notably macOS APFS) to
keep chunk hashes consistent, so the parse cache thrashes without
it. The pre-existing Ruby cross-file resolution order-dependency the
old comment cited is independent — the sort surfaces it but doesn't
cause it; tracked separately rather than leaving the cache cold.
Tests — incremental-subgraph-extract.test.ts
Locks the F1 invariants: `extractChangedSubgraph` is a pure filter
(includes only the set it's given, plus graph-wide nodes; edges
fire on one writable endpoint), and `computeEffectiveWriteSet`
covers the barrel-re-export scenario, the symmetric edge-into-
changed-file case, the no-boundary-crossed no-op, graph-wide-node
edges, and input-immutability. Supersedes the prior
extractChangedSubgraph-only test file on the branch.
Co-authored-by: Val Vladescu <vvladescu-tb@users.noreply.github.com>
* chore(autofix): apply prettier + eslint fixes via /autofix command
* fix(call-processor): register properties in pre-pass to fix order-dependent field type disambiguation + regenerate golden snapshot
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/2d66666f-861c-432e-a4b0-11f2aefca98a
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix(call-processor): port worker-path property enrichment into the sequential pre-pass
Copilot's pre-pass in 8184439 fixed the Ruby attr_accessor order-dependence,
but it copied the OLD in-loop registration logic, not the canonical worker
path in parse-worker.ts. That left the sequential and worker paths emitting
non-identical Property nodes/symbols for the same source — silently breaking
the `incremental ≡ --force` invariant the moment a repo crosses the worker
threshold between runs.
Two concrete divergences are closed here:
* Node id: worker keys Property as `${file}:${className}.${propName}`
(qualified). Pre-pass was using `${file}:${propName}` (unqualified).
Same source produced different graph ids depending on which path ran.
* Field metadata: worker enriches each routed property with
`provider.fieldExtractor` + `getFieldInfo`, falling back to
`routedFieldInfo.type` for `declaredType` when the routing payload
lacks one (e.g. types discovered from `@address = Address.new`
ctor assignments rather than YARD `@return [Type]`), and propagates
`visibility` / `isStatic` / `isReadonly`. Pre-pass did none of this,
so on the sequential path `resolveFieldAccessType` failed to walk
chains where the type only came from the FieldExtractor.
The pre-pass now mirrors parse-worker.ts:1803-1898 verbatim, with one
deliberate difference: the FieldInfo cache is scoped to a single
`processCalls` invocation rather than module-level (the worker process
is short-lived; the main thread is not, and a module-level cache would
leak state between analyze runs).
Also drops the now-stale "Defer resolution: Ruby attr_accessor properties
are registered during this same loop" comment on `pendingWrites.push` —
the rationale is no longer accurate after Copilot's pre-pass, but the
deferral is still needed so write-access tracking sees inference that
completes during the main loop. Comment updated to reflect that.
Verification:
* `tsc --noEmit`: 0 errors
* test/unit (call-processor, call-routing, field-extraction, ruby-self-call): 224 passing
* test/integration (ruby, ruby-sequential-mixin, pipeline-graph-golden): 137 passing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(call-processor): key fieldInfoCache by filePath:startIndex, not raw byte offset
Claude's review of 255bdf6 caught a real collision in the FieldInfoCache I
added: keying by `classNode.startIndex` alone is a per-file byte offset, so
two files that both begin with a class at byte 0 — extremely common in Ruby /
Python, where files frequently open with `class Foo`, `module Foo` — collide
on the same cache entry. The second file's `getFieldInfo` then returns the
first file's FieldInfo map, producing wrong `declaredType` / `visibility` /
`isReadonly` on its properties.
Same shape as the bug that already exists in parse-worker.ts:377 (also keyed
by `classNode.startIndex` in a module-level map, persistent across files
processed by the same worker). Fixing the symmetric pre-existing leak in
parse-worker.ts is a separate, scoped follow-up — left out of this commit to
keep the fix minimal and reviewable.
Cache map and key are now both string-typed. Composite key
`${context.filePath}:${classNode.startIndex}` keeps the within-file hit rate
(one FieldExtractor.extract() per class regardless of how many
`attr_accessor` lines it has) while eliminating cross-file aliasing.
Verification on the patched HEAD:
* `tsc --noEmit`: 0 errors
* test/unit (call-processor, call-routing, field-extraction, ruby-self-call): 224 passing
* test/integration (ruby, ruby-sequential-mixin, pipeline-graph-golden): 137 passing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Val Vladescu <val.vladescu@thirdbridge.com>
Co-authored-by: Val Vladescu <vvladescu-tb@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
Gergő Magyar