mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-08-28 05:25:25 +00:00
13 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
DuduPhudu
|
223ac7010d
|
feat: close reported graph blind spots in reference resolution, analyze and storage (#2856)
* fix(mcp): report UNKNOWN risk when an upstream impact walk finds no callers `risk: LOW` asserts "safe to change" — a claim ABOUT callers. An upstream walk that resolved none has nothing to base it on: the symbol may be genuinely unused, or reached only through a reference class the index does not record (a property access on a plain object, a bare-identifier read of a module-scope const). Seeding LOW from an empty result is the false-safe signal `anyKnownRisk` already refuses to emit on the ambiguous-candidate path, and that #2687 removed by making an undetermined impactedCount `null` rather than `0`. Zero-caller upstream results now report risk UNKNOWN with a riskNote saying absence of edges is not evidence of disuse. Downstream is untouched: an empty downstream walk reports resolved callees, not safety. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(javascript): emit ACCESSES for bare-identifier reads of module-scope consts A constant read only as a bare identifier — `Math.max(LIMIT, n)`, a default parameter value, `return LIMIT` — minted no reference site at all, because JS captured only `@reference.read.member`, which requires a receiver a bare identifier does not have. So "who uses this constant?", the question behind every dead-code trim and constants refactor, answered with a confident zero in both directions. The rest of the machinery was already in place: `FIELD_KINDS` accepts `Const`, the scope query already declares it via `@declaration.const`, and `read` maps to ACCESSES for any resolved target. This adds the missing capture in VALUE POSITIONS ONLY (call arguments, default-parameter values, return statements) — a blanket `(identifier)` rule would mint a site for every token in the file, which is unaffordable at repo scale and would keep alive the block-local symbols `pruneLocalSymbols` exists to drop. Cross-file readers are NOT yet covered: the site exists and a call through the same import statement resolves, but a value-kind def does not link across the import edge. Recorded as a todo with the investigation. PARSE_CACHE_VERSION bumped 44 -> 45: this is parse-time capture emission, so a warm cache replays the pre-change capture set and the new edges never appear — observed directly, a full `analyze --force` produced a byte-identical graph until the cache was cleared by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(javascript): pin A1/A5 plain-object property acceptance criteria Fixture plus todo specs for the four shapes plain-object property access has to answer: object-literal keys indexed as Property nodes, a read through the holding variable, a property WRITE, and a read through an untyped param. Records the investigation so the work is resumable: the parse-query pattern scoped to literals bound to a variable matches correctly (verified against the raw JAVASCRIPT_QUERIES), but no Property node reaches the graph and local-symbol-pruner is not the cause — it drops only Const/Variable/Static. The remaining gate is in the parse worker's node-creation path. No production code — specs only, so the suite stays green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(javascript): index object-literal keys of a named object as Property nodes Idiomatic JS models configuration as an object literal, not a class, but Property definition nodes existed only for DECLARED CLASS FIELDS. A config field therefore had no symbol at all: `context({name: 'exitMinAtrMult'})` answered "not found" for a field read and written throughout a live code path, and ACCESSES had no target to point at. Both halves are added for keys of a literal BOUND TO A VARIABLE — the parse query mints the graph node, the scope query mints the def the resolver can aim at. Unbound literals are deliberately excluded: an inline call argument or a JSX prop bag is call-site data, not a named surface other code references, so a node per key there would add volume without adding an answerable question. This lands the definition-node half only. The ACCESSES edges still require receiver resolution — typing the const that holds the literal to the literal's scope for the precise case, and name-based matching at reduced confidence for the untyped-param (option bag) case. Both are recorded as todos with the mechanism each needs. Also records a trap that cost a wrong conclusion: under vitest the parse worker runs the BUILT dist code (parse-impl resolves parse-worker.js, absent under src/, and falls back to dist), so parse-query changes are invisible to tests until `npm run build`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(cache): move the SCHEMA_BUMP pin to 45 The pin is the guard that makes two branches claiming one cache-schema number fail loudly instead of silently serving each other's entries, so a bump is only half-done until the pin moves with it. The bump itself landed with the JavaScript bare-identifier captures; this is the other half. Caught by the guard working exactly as designed — the suite failed with "expected 45 to be 44" rather than letting a mismatched pair through. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): resolve plain-object property access by unique name Idiomatic JS reads configuration off an object whose receiver cannot be typed — an options bag passed as a parameter, a destructured handle, an imported literal. No precise pass resolves those, so a field read and written across a live code path produced no ACCESSES edge at all and "who reads this setting?" answered a confident zero. A last-resort pass runs after every precise pass and sees only what they left behind. For each still-unresolved read/write site it asks whether exactly ONE Property in the workspace carries that name. If so the read almost certainly means it. If two or more do, nothing is emitted and the site is COUNTED as ambiguous — a guess between them would be a coin flip, and a wrong edge in the pre-edit safety gate is worse than a missing one. Uniqueness is the right gate because it recovers exactly the names worth recovering: distinctive domain fields (exitMinAtrMult, bookNotionalUsdt) are unique in a repo and resolve, while generic keys (id, name, data) are not and are skipped — which is where name matching would over-connect. Bounded four ways: - Confidence 0.5, the global tier, with the inference named in the reason, so a consumer can filter inferences without losing scope-resolved edges. - Never second-guesses a precise result: sites already resolved are excluded, because first-write-wins stops a duplicate but NOT a second edge to a different target. - Honors `fieldFallbackOnMethodLookup`. A statically-typed language opts out of name matching precisely because it over-connects; inferring an ACCESSES edge by name is the same claim and must obey the same opt-out. - Requires an explicit receiver — a bare identifier is not a property access, and matching one by name would link a local to an unrelated key. Indexes graph nodes rather than scope defs because an object-literal key mints a Property NODE but no scope-resolution DEF: `localDefs` and `scope.bindings` are both empty for exactly the population this serves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(analyze): record a collapsed graph write instead of reporting fresh The dangerous half of a broken refresh: metadata IS written, so the index reads as fresh, hooks re-arm, and every tool answers from a graph missing most of its edges — indistinguishable from a codebase that genuinely has no such relationships. Reported in the field as edges collapsing 23009 -> 2170 and as a CodeRelation table that never materialized. `analyze` now compares the relationship count the pipeline PRODUCED against what the DB hands back after the write. Both numbers are already in scope at the same point, so the shortfall is provable rather than inferred — no comparison against the previous index, which cannot distinguish a failed write from a repo that legitimately shrank. A missing relation table needs no special case: it reads back as a persisted count of zero. On a collapse the run records `graphWriteCollapsed` in metadata, which `getIndexIncompleteReasons` turns into `graph-write-collapsed` so status and the MCP resources report the index INCOMPLETE rather than fresh. A ratio, not equality: some relationship types do not round-trip one-for-one and `--pdg` writes MORE rows into the same table, so demanding equality would fire on healthy runs. Only a collapse is a defect. Fail-safe when the expected count is unavailable — an implementation that offloads relationships out of memory may not be able to report a total, and a false "your index is broken" is worse than a missed one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ingestion): qualify object-literal Property ids by their owning object Two config objects in one file that share a key name generated the same `Property:<file>:<key>` id and COLLAPSED INTO ONE node, so two distinct settings became a single symbol. Worse, the merged name then looked workspace-unique to name inference, which happily resolved reads of it to a node representing both — a wrong edge in the pre-edit safety gate, which is precisely what the unique-name pass is bounded to avoid. `objectLiteralOwnerInfo` already existed for exactly this ("so two constructors in one file that both define `bar` stay distinct nodes") but was gated to `Method`. `Property` now opts in. `findObjectLiteralBindingInfo` returns `ownerName` only when asked. Its `Method` ids must stay byte-identical — qualifying them would rewrite every object-literal method id in every indexed repo — while object-literal KEYS, indexed only since A1/A5, have no such history to preserve. Found by a test written for the ambiguity path rather than by review: the suite reported one node where two were expected, and an edge where none should exist. Both are now pinned, along with the detection boundaries of the B2 collapse check, which was previously an untestable inline expression and is now a pure function. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(typescript): index type aliases and shape members as symbols A TS frontend models its API contracts as `type X = { … }` and `interface`, so a field on one is exactly what "who breaks if I remove this?" is asked about. Three gaps made that unanswerable, all in the TypeScript queries: 1. No `type_alias_declaration` -> `@definition.type`, so an alias minted NO NODE AT ALL and a context() lookup on an exported contract type answered "Symbol not found". TypeScript was the ONLY language missing this — Rust (type_item), Kotlin (type_alias), Swift (typealias_declaration) and Dart all emit it. The alias was declared for scope resolution but never became a graph symbol. 2. No `property_signature` in the parse query, so INTERFACE members minted no Property nodes either — the upstream report's "class/interface index fine" holds only for the type, not its fields. 3. No `property_signature` in the scope query, so even with nodes present the resolver had no member declaration to aim at. Its sibling `method_signature` -> `@declaration.method` already existed; only properties were missing. Interface bodies and object-type aliases both spell members as property_signature, so one pattern per query covers both shapes. Lands the SYMBOLS, not yet the ACCESSES edges: the shape is already a class-like scope and now has member declarations, but no edge forms — the remaining link is owner/type-binding, recorded as todos with the diagnosis. Note TypeScript sets fieldFallbackOnMethodLookup:false, so unlike JavaScript there is deliberately no name-based fallback here; the precise path is the only route by design. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(golden): accept interface members in the mini-repo snapshot Drift is entirely the new TypeScript shape-member indexing: the fixture's three interfaces (ValidationResult 2, DbRecord 3, LogEntry 3) contribute exactly 8 Property nodes, each with exactly one HAS_PROPERTY owner edge. Verified before regenerating rather than after: every pre-existing count is untouched (CALLS 9, IMPORTS 12, DEFINES 16, HAS_METHOD 1, MEMBER_OF 12, STEP_IN_PROCESS 12), so nothing was rewired — the digest moved only because 8 edges were added. The fixture's inline `return { valid: false, … }` literals correctly produced nothing, confirming the object-literal rule stays scoped to variable-bound literals. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(analyze): never report a collapse from a non-numeric count The B2 check reported healthy runs as total graph-write collapses. A non-numeric `expected` (a graph implementation reporting no total, a lightweight pipeline result) does not skip the guards — it INVERTS them: `undefined < 100` is false, so the small-repo exemption never fires, and `0 >= undefined * 0.5` is `0 >= NaN`, also false, so the ratio check "passes" as well. Both bounds silently evaporate and every such run is flagged. That is precisely the failure this check was written to catch, reproduced inside the check itself: an unmeasurable quantity treated as a measured zero. Both sides are now validated as finite numbers before any comparison. `persisted` is also passed as UNKNOWN rather than zero when the DB was not demonstrably readable: `getLbugStats` flattens "no connection", "query threw" and "empty table" all into `edges: 0`, so `stats.nodes > 0` is used as independent evidence the read happened at all. Caught by the existing run-analyze suites, not by the new unit tests — those exercised the pure function with well-formed numbers and were blind to the integration's actual inputs. Both cases are now pinned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(typescript): make object-type aliases own their members A TS object-type alias declares the same `property_signature` members as the interface beside it and answers the same question, but was not a member owner: its fields were minted with bare ids and no owner edge, so two aliases in one file sharing a field name collapsed onto one node, while the identical interface resolved normally. `type_alias_declaration` joins CLASS_CONTAINER_TYPES (and CONTAINER_TYPE_TO_LABEL, as that set's invariant requires — a container missing there gets orphaned member edges or a wrong owner label). Aliases with no object type (`type Id = string`) declare no members, so they own nothing and are unaffected. This also lands the INTERFACE field -> consumer edges, verified on the mini-repo fixture rather than only on a purpose-built one: `saveToDb` now links to `ValidationResult.value`, and `formatLogEntry` to `LogEntry.level` and `LogEntry.message` — three real contract-field reads that previously had no graph path at all. Golden updated: +3 ACCESSES, no node changes. The ALIAS field -> consumer edge is still not linked and is recorded as a todo with the exact blocker: resolving a receiver typed as the alias needs the NAME to resolve to a class-like def, and `isClassLike` is Class|Interface|Struct|Record|Enum|Trait. That predicate is read from ~12 sites including MRO and heritage, and every language mints TypeAlias, so widening it would enrol aliases in linearizations where they do not belong. Widening only the scope index was tried and reverted — the type-name walkers gate on it independently, so it fixed nothing and left dead code. That needs a deliberate "shape-like" concept, not more call-site widening. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(test): record the traced diagnosis for the unlinked alias field edge Traced to the end rather than left as "needs investigation", so the next attempt starts from facts: 1. Graph side is COMPLETE and symmetric with the interface — Property:...:LiveModeConfig.bookSlots is owner-qualified and carries HAS_PROPERTY. 2. Resolution DOES reach resolveClassBindingForName('LiveModeConfig') (instrumented) and misses. 3. It misses because the module scope binds LiveModeIface:Interface, renderAlias, renderIface — and not LiveModeConfig. The alias has no binding on the receiver's scope chain at all. 4. The TS scope query tags aliases @declaration.type, but normalizeNodeLabel accepts only typealias / type_alias and has no "type" case, so it returns undefined. Kotlin and Dart use @declaration.type_alias; TypeScript is alone on the dead tag. 5. Retagging is NECESSARY BUT NOT SUFFICIENT — tried, and the binding still does not appear, so a second gate exists in how a declaration anchored on a node that is ALSO a @scope.class anchor is attached: the alias appears to bind inside its own scope rather than hoisting to Module, where interface_declaration evidently does hoist. An isShapeLike predicate (the nominal-vs-structural split: shapes declare members, nominal types participate in MRO) plus a mirrored findShapeBindingInScope were built and REVERTED along with the retag. With no binding on the chain they never fire, and shipping inert widening is worse than shipping none — the same standard applied to the earlier scope-index attempt. The design is recorded here; it is worth doing once step 5 is fixed, and it also unblocks Rust's parked union_item, which the MEMBER_OWNER_NODE_TYPES comment documents as the same gap in another language. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): resolve cross-file value references, skip block-locals Two halves of the same question, "who uses this constant?". CROSS-FILE. `resolveReferenceSites` runs against the registries and, as its own comment says, "imports live in finalized bindings the registries can't see" — which is why free CALLS need `emitFreeCallFallback`. Reads had no counterpart, so `import { LIMIT }` followed by a bare use resolved to nothing while a CALL through the very same import statement resolved fine. This adds the read/write counterpart, reusing `findValueBindingInScope` (which walks the FINALIZED chain) rather than inventing a lookup. Confidence 0.9: the import names the def, so this is precise resolution, not inference. BLOCK-LOCALS. Bare-identifier capture also matches a read of a block-local `const`, and an edge to one keeps alive exactly the inert locals `pruneLocalSymbols` exists to drop — a pruned node becomes a retained node plus an edge, in every function of every indexed repo. Emission now takes the set of value defs bound at MODULE scope and drops ACCESSES to Const/Variable/Static outside it. The cross-file pass carries the same guarantee structurally: a def in another file cannot be a block-local of this one, so it skips same-file hits entirely. The block-local leak was already shipped in the intra-file A2 commit and was found only because a test was written for the guard rather than the feature — the same way the object-literal id collision surfaced. Verified on the full resolver matrix: 3172 tests, golden unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(lbug): diagnose a vanished staging CSV instead of surfacing a Binder error A forced rebuild could fail with "COPY failed for File: Binder exception: No file found that matches the pattern .gitnexus/csv/file.csv" and then an ENOENT on .gitnexus/csv/rel_Folder_File.csv — two engine-level messages that name neither a cause nor a remedy, which is where several field reports end. Only tables with rows > 0 enter the COPY manifest (csv-generator.ts), so an absent file was WRITTEN during this run and removed since. Both COPY loops now preflight and say exactly that, with the row count, both causes the reports point at (a second `gitnexus analyze` on the same repo — they share .gitnexus/csv — or an external cleanup of .gitnexus/), and the action to take. Scope note, deliberately narrow: this does not attempt to fix WAL corruption or checkpoint rotation. Those already have detection and recovery hints (isWalCorruptionError, WAL_RECOVERY_SUGGESTION, the configurable wal-checkpoint-threshold), and the ~6000 lines added to lbug/ + storage/ since v1.6.9 — index-lock.ts most of all, which serializes writers and plausibly closes the concurrent-run class outright — postdate every report in the window. Guessing at unreproducible durability faults would be speculation; making the one failure with NO handling legible is not. An existing overlap test induced this exact scenario (a manifest entry pointing at a missing csv) and asserted on the engine's wording. Its intent — that a node-COPY failure is rethrown at the FK barrier rather than swallowed — is unchanged and still asserted; only the message it matches moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): split shape-like from class-like, linking alias fields Completes A4: a field on a TypeScript object-type alias now links to the code that reads it, the last unanswerable half of "who breaks if I remove this?" for a TS frontend that models contracts as `type X = { … }`. `isClassLike` answered two questions that only coincide for classes: 1. does this declare MEMBERS I can look up? — a SHAPE (structural) 2. does this participate in inheritance / MRO? — a NOMINAL TYPE An object-type alias is (1) and emphatically not (2) — it has no supertypes and no place in a linearization. Widening `isClassLike` to buy (1) would have enrolled every language's aliases (Rust type_item, Kotlin/Swift/Dart typealias, C typedef) into MRO and heritage, so the two questions now get two predicates. Call sites split by which they ask, and their names already said which: `resolveInheritanceBaseInScope` and `resolveQualifiedInheritanceBase` keep `isClassLike`; receiver typing and member OWNERSHIP take `isShapeLike`. Three parts, each necessary and none sufficient alone: - `findShapeBindingInScope`, mirroring `findValueBindingInScope`'s established relationship to `findClassBindingInScope` (same walker, different accepted def-type), consulted only AFTER the class lookup misses so a class of the same name always wins. - `populateClassOwnedMembers` uses it, so alias members get an `ownerId` and are registered under the alias. Without this the receiver resolved to the alias and then found no members under it. - The TS scope query tags aliases `@declaration.type_alias`, not `@declaration.type`: `normalizeNodeLabel` accepts typealias / type_alias and has no "type" case, so the old tag mapped to NO label and TypeScript aliases produced no scope-resolution def at all. Kotlin and Dart already spelled it this way; TypeScript alone was on the dead tag. An earlier attempt concluded a further "scope-attachment gate" existed. That was wrong and is worth recording: scope extraction runs in the parse WORKER, which loads built `dist`, so the retag was never executed. Rebuilt, the alias hoists to Module scope exactly as the interface does. Same trap as the parse query — `src` edits to anything the worker runs are invisible until `npm run build`. Typedef and Union stay out of `isShapeLike` deliberately: they belong conceptually (the union_item note on MEMBER_OWNER_NODE_TYPES records the same gap) but neither is wired as a member container, so including them would widen a predicate nothing exercises. Verified on the full resolver matrix: 3173 tests, golden unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(typescript): pin the type-alias capture to a tag that maps to a label The capture test asserted `@declaration.type`, the tag that `normalizeNodeLabel` does not recognize (it accepts typealias / type_alias and has no "type" case). So the test passed for as long as the tag was broken: it checked only that the capture FIRED, never that it resolved to anything, while TypeScript aliases produced no scope-resolution def at all. Updated to the working tag and given a second assertion that the derived kind string is one the label mapper accepts — the property that actually matters, and the one whose absence let a dead tag sit pinned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(lbug): declare TypeAlias member pairs so analyze does not abort Making object-type aliases member owners emits HAS_PROPERTY from a `TypeAlias`, and the relation schema declared no such pair. The emit therefore threw `UndeclaredRelationPairError` and the ENTIRE analyze died on any repo containing `type X = { ... }` — a hard stop, not a dropped edge. Found by running the analyzer over a real 16k-node TypeScript repo, not by a test. `Method` is declared alongside `Property`: a member written `type Handler = { onClick(): void }` is a method_signature and would fail in exactly the same way. Why every existing test missed it: the resolver suites build an in-memory graph via `runPipelineFromRepo` and never write to LadybugDB, so the schema constraint was never exercised. `structural-pair-coverage.test.ts` is the one suite that does run the emitters against the declared pairs — and its own docstring names the gap: coverage is bounded by NON_BRIDGE_CORPUS, "a new structural emitter should land with an entry here". This adds that entry, pinning TypeAlias|Property and Interface|Property as sentinels. Verified the guard is not vacuous: removing the pair again makes the suite fail with undeclaredPairs: ["TypeAlias|Property"]. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(processes): trace depth-first so multi-hop flows are detected D1 ("query ranks frontend components above the backend module that owns the concept") and D2 ("processes is dominated by trivial mechanical chains") are the same defect, and neither is about ranking or selection. The walk stops after a fixed NUMBER of traces, so traversal order decides which traces those are. Breadth-first reaches every shallow terminal before any deep one, so the quota filled with the shortest paths in the graph and the walk stopped — `maxTraceDepth: 10` was never approached. Measured on a real repo before the fix: of 300 processes NONE exceeded 7 steps and 90% were 3-4. A multi-hop business flow (signal → order → exit) therefore had no process that could represent it, and `query` could only rank the mechanical pairs that did exist. Step 4 of the caller already sorts by length and dedupes by endpoint — it was always asking for the deepest traces this walk could give it. Depth-first descends to a terminal first, so the same quota is spent on paths worth keeping. Cost is unchanged: same budget, same cycle guard, same depth ceiling — only the order differs. Measured on the same 16k-node repo, same build and flags, BFS vs DFS (an earlier comparison was discarded as confounded — it crossed builds and --pdg): steps 6-8: 50 → 168 (3.4x) totals: 844 → 806 and the reported query moved from `LiveSetupView → Cn` (a React component) to `ReconcilePositions → IsTpInProfit / WithHeld / ShouldNotify` — server-side exit management, which is what was asked for. `traceFromEntryPoint` is exported for the test. Traversal order is unobservable through `processProcesses`: `findEntryPoints` supplies several starting points, so a deep chain is traced from inside it whatever the order does. A test at that level passes under BOTH traversals — the first version of this test did exactly that and guarded nothing. Driving the walk directly, it fails under breadth-first with "expected 3 to be greater than 3". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(test): correct a stale status note left behind by a later fix The A1/A5 header still said "edge resolution REMAINING ... neither is implemented". Both shapes resolve — the typeable receiver precisely, the untyped one by workspace-unique name — and the tests below assert exactly that, so the note contradicted the file it sat on. It was accurate when written and went stale when the work continued past it. Left as-is it would tell a reviewer that a landed feature is missing. The TRAP note is kept: the parse worker still runs built dist under vitest, and that is still the trap it describes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): index literals behind identity-preserving wrappers `export const INERT_EXIT_CONTRACT = Object.freeze({ ... })` minted no `Property` node for any of its keys. The object-literal rule matches `variable_declarator > value: (object)` as a DIRECT child, and freezing puts a call expression in between — so the shape whose fields are most worth querying was the one shape the rule could not see. Freezing a config object is how JS publishes an immutable contract, which is why this reads as a confident zero on exactly the fields a reader cares about. The allowlist is three functions, not "any call". `Object.freeze`, `seal` and `preventExtensions` RETURN THE ARGUMENT THEY WERE GIVEN, which is what makes the literal's keys members of the bound name. For `const x = compute({ a: 1 })` the literal is an argument and `x` holds compute's return value, so attributing `a` to `x` would be a fabrication. Two negative controls, because the obvious one is vacuous: a bare-identifier callee is rejected structurally and would pass with no allowlist at all, so the assertion that actually pins the predicate uses `Object.entries` — identical shape, differing only by name. Verified load-bearing by adding `entries` to the allowlist and watching that test alone fail. SCHEMA_BUMP 46 -> 47: parse-time emission, so a warm cache replays the pre-fix capture set. Observed as a false negative first — `analyze --force` returned the old node set until the on-disk cache was removed by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(scope-resolution): narrow multi-candidate property names by scope Workspace uniqueness was the wrong denominator. Measured on the reporting repo: `exitMinAtrMult` has 26 `Property` definitions — 16 in one-off `scripts/`, 7 in the frontend, one in a test, and exactly ONE in the backend that reads it. Every backend read was refused because of competitors the reader cannot see. The gate was not too permissive or too strict, it was scope-blind. A name with several definitions is now narrowed before being abandoned: same-file first, then files the reading file directly imports, using the finalized import graph rather than a path-shape heuristic. Exactly one survivor at the first non-empty tier resolves; anything else stays refused. A tier holding several candidates stops the walk instead of falling through — local evidence that is itself ambiguous still contradicts reaching further out. Confidence stays 0.5 at every tier. Narrowing changes which candidate is chosen, not the kind of claim: it is still a name match, and the round-1 contract is that filtering on confidence drops all name inference at once. The reason string now names the tier that fired. Ambiguity reporting goes from a count to the actual names (capped), because a count says a gap exists while the names say which fields are unanswerable. Measured on that repo, backend readers of `exitMinAtrMult` go 0 -> 24 and total readers 9 -> 45, including the two call sites in `oppositeSignalExitManager.js` the report singled out. Both narrowing tests were mutation-checked by dropping the import evidence and confirming they, and only they, fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): capture destructured parameter keys as property reads `function exit({ exitMinAtrMult = 0 })` reads that property off whatever the caller passes, exactly as `cfg.exitMinAtrMult` would. It never appears in a member_expression, so it had no reference site at all — and this is the shape the function that IMPLEMENTS a behaviour uses, so the most relevant reader was the one systematically missing from "who reads this setting?". Uses a distinct `@reference.read.destructured` anchor rather than `@reference.read.member`. The latter is filtered emit-side to matches with a member_expression ancestor, because calls and writes share its shape, and a destructuring pattern has none — reusing the tag would have been silently dropped by that filter. The `read.` head already maps to a read kind, so no mapping change is needed. Scoped to formal_parameters. A destructuring binding elsewhere (`const { x } = require('m')`) is frequently an import rather than a field read, and minting a property read there would attribute module bindings to unrelated same-named keys. All three cases (default value, bare shorthand, renamed key) mutation-checked by removing the patterns and confirming those three tests, and only those, fail. The renamed case also asserts the edge points at the KEY and that the local alias mints nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(scope-resolution): link type consumers to the type they name An exported contract type owned its members after round 1 and still answered `incoming: {}`, so "what breaks if I remove this field?" — the question a contract type exists to answer — had no edge to walk. Measured on the reporting repo: all 324 TypeAlias nodes AND every Interface node had DEFINES as their only incoming edge. Two independent causes, and the second is why the first was not enough. TypeScript captured no type references at all — only cpp and csharp did — so an annotation naming a declared type minted no reference site. Added for annotations, generic arguments and `as` assertions, anchored to those contexts rather than a bare `(type_identifier)`, which would also match the name in `type X = …` and make every declaration a consumer of itself. That alone fixed interfaces and left aliases still empty. `TypeAlias` was missing from `LINKABLE_LABELS`, so alias graph nodes were never indexed in `nodeLookup` and `resolveDefGraphId` could not bridge a def to its node — the edge was dropped AFTER a successful lookup. `CLASS_KINDS` has always listed TypeAlias and the ClassRegistry returned the def correctly, which is what made this read as a resolution failure; instrumenting the lookup showed it returning the right def all along and moved the search one table over. Exactly the bug already documented two entries above it for Trait. Fixes every language that spells an alias this way — TypeScript, Kotlin, Dart and Rust all emit `@declaration.type_alias`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): capture record construction as property writes The read side answered well after the narrowing work while "who SETS this field?" still missed the code that stamps the value. A record built inline — `return { exitContract: { exitMinAtrMult: settings.x } }` — is bound to no variable, so it minted no definition and its keys referenced nothing. Modelled as WRITE REFERENCES, deliberately not definitions. The round-1 rule already mints Property nodes for literals bound to a variable; minting more for anonymous records would add same-named competitors to the very name-narrowing that makes these fields resolvable — measured at 26 competing definitions for one field on the reporting repo, which is what made every backend read unanswerable in the first place. A construction site is a USE of a field, not another declaration of it. Two positions only: nested under a key, and returned. Both are records with a name attached (the key, or the function). An inline call argument (`doThing({ id: 1 })`) stays excluded for the same reason round 1 excluded it from definitions — it is call-site data, not a named surface — and is asserted as such. The enclosing literal is the receiver and it is anonymous, so these route through the same narrowing and the same refusal-to-guess as every other untyped receiver. Verified on the reporting repo: `entryPlan.js` went from no rows to `selectExitEnvelope` as a writer of `exitMinAtrMult`. Both captures mutation-checked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(processes): select round-robin by terminal so the list is not one flow repeated Ranking was `sort by length` alone, so the top of the list was one behaviour described many ways: eleven of the top fourteen processes on the reporting repo were four entry points crossed with three terminals of the SAME date-window utility cluster. Genuine call chains, but a reader learns one thing from fourteen entries, and the repo's own domain flows sat below them. Selection now round-robins across TERMINALS, deepest first. Depth still orders within a terminal and still leads the list; what changes is that no terminal takes a second slot until every other has had a first. Keying on the entry point was tried first and made it worse — many files declare a `main`, so each was a distinct entry that round-robin then awarded its own slot, and `Main -> AlignWindowEnd` went from one row to eight. The repetition was never in where a flow starts. Measured on that repo: distinct terminals in the top 20 went 3 -> 20, and its domain flows (`ReconcilePositions -> ...`) moved into the top 4%. Two things this deliberately does not claim. The reported cause — ranking rewarding fan-in, promoting chains ending in widely-called helpers — measured FALSE: those terminals have one caller each (`alignWindowStart` 1, `validateSymbol` 1). A fan-in discount was implemented against that hypothesis, measured, and reverted for moving nothing. And a business flow still cannot be a process in its own right: the walk only emits at a leaf, at max depth, or on a cycle, so a flow whose meaningful endpoint calls onward survives only as whatever leaf it bottoms out in. Both are recorded in the code so neither reads as settled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(structural-pairs): pin the type-annotation USES pair R2-2 emits USES INTO a `TypeAlias`, so the pair is `Function|TypeAlias` — a different table from the `TypeAlias|Property` entry added in round 1, and one that entry stays green without. `TypeAlias` is on the eleven-table list this suite exists for, and an undeclared pair does not degrade: it throws `UndeclaredRelationPairError` and kills the entire analyze on any repo containing an annotated type. Every resolver suite still passes, because they build an in-memory graph and never write to the DB. That exact failure shipped once in this PR already. Two emitters into the same label, each with its own way to reach a released build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(scope-resolution): build the module-level set before the out-of-core seal Review blocker. Under `GITNEXUS_DISK_SCOPE_INDEX=1` the seal replaces every ParsedFile with a scope-STRIPPED copy, and the block-local filter's set was built after it — so it walked `scopes: []` for every file, came out empty, and the filter read that as "no def is module-level" and dropped EVERY `Const`/`Variable`/`Static` ACCESSES edge in the repo. All languages, all files, including the module-scope-const edges this PR exists to add. Nothing threw and nothing logged, on the path the largest repos take: the exact confident-empty answer the PR is about. Built above the seal now, from `parsedFiles`, and passed as `undefined` rather than an empty set when no scope was inspectable — an empty set is a legitimate answer ("this repo has no module-level value defs") and must not be indistinguishable from "could not look". Fails open; the block-local exclusion is still asserted under the seal, since that is correctness rather than optimization. Also widens module level past `kind === 'Module'`. A `Namespace` scope (TS `namespace`, Rust `mod`, C++/C# `namespace`) holds importable values too, and treating its consts as function-locals dropped their reads. Included only when the whole chain to the root is Module/Namespace, so a namespace declared inside a function body stays local — asserted both ways. That fixture then failed for a third reason: `@reference.read.identifier` existed ONLY in the JavaScript query, so A2 did not work for TypeScript at all. Added there, and both languages widened to `variable_declarator value:` and `binary_expression` operands — the gaps review named between what A2 claimed and what it matched. Nothing covered `GITNEXUS_DISK_SCOPE_INDEX`. The new parity test asserts the seal changes no edge, and was verified against an emulation of the original bug: same-file readers vanish and only the cross-file reader survives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(typescript): anchor property_signature to declared shapes Review blocker, and it reproduces end to end. `property_signature` occurs in EVERY object_type in the TS grammar, not only in an interface body or an alias's object type, so inline parameter types, inline return types and nested object types all matched — and the enclosing-container walk hung each one off the nearest class, interface or alias. Measured against the unanchored rule, all four appeared as members of shapes that do not have them: Property:contracts.ts:Svc.inlineParamOnlyKey Property:contracts.ts:Repo.inlineQueryOnlyKey Property:contracts.ts:NestedConfig.nestedOnlyKey Property:contracts.ts:buildInline.inlineReturnOnlyKey@46:33 When the inline member shares a name with a real one — `run(opts: { retries: number })` inside a class that declares `retries` — `addNode` is first-write-wins and the two distinct symbols merge onto one node, so every context()/impact()/rename() answer about that field describes the merge. The sibling JS object-literal rule in this same PR is anchored for exactly this reason; this is the TypeScript half of the same fix. `(A (B))` matches DIRECT children, so nested object types are excluded by the same anchor rather than by a second rule. The first version of these tests was VACUOUS and is recorded here because the reason generalizes: a collision and a correct exclusion both leave exactly one node behind, so counting ids cannot distinguish them. Every inline member in the fixture is now uniquely named, which is the only thing that discriminates — verified by restoring the unanchored rule and watching exactly those four assertions fail. A fifth test asserts anchoring costs no real member. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(analyze): correct the numbers feeding the graph-write-collapse guard Review blocker. The predicate itself held under adversarial probing; every defect was in what it was handed and what happened after it fired. (a) `expected` was wrong twice. Under `GraphEmitSink` streaming the bulk types leave the heap at parse time and never enter `relationshipCount`, so the count understated the real volume by most of it and the ratio passed trivially — on `force === true` runs, which include crash recovery AND the `analyze --force` retry this check's own warning tells the operator to run. Adds the manifest totals, the same correction the buffer-pool hint in this file already makes for the same reason. Separately, an incremental run persists only the changed subgraph while both counts are whole-scope: a 10,000-edge index that lost 200 replacements reads 9,800 and is certified complete. The check is skipped on that path rather than answered wrongly. (b) A throwing edge count became a measured zero. `getLbugStats` initialised its total to 0 and ran the query in a swallowing catch, so WAL/lock contention during finalize — documented on this exact call — reported a healthy index as a total collapse. It now returns `number | undefined`, and the caller requires both a readable node count and a defined edge count. (c) A total loss was exempted for being small. The min-edges rule tested `expected` before looking at `persisted` at all, so `expected = 99, persisted = 0` — every edge gone — stayed fresh and reported success. Total loss is now decided first. The existing test asserted the defect; it now asserts a PARTIAL shortfall, which is the case the exemption was written for. (d) A detected collapse reported success and exited 0. It is different in kind from the other incomplete reasons: those describe a run that did what it said and left work for later, this one means most of your edges are gone and every query answers a confident empty. The CLI now prints INCOMPLETE with the counts and sets a non-zero exit code, and the flag crosses IPC so the worker cannot send a clean `complete` either. Nothing exercised this wiring — only the pure helper. Adds tests for all four, each written so the pre-fix arithmetic fails it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(scope-resolution): keep unique-name property inference inside one language The pass indexed `Property` nodes from the whole shared graph. Per-language gating decides whether it RUNS for a language; it never restricted which nodes could be TARGETS. So the only carrier of a name could be in another language entirely, and a read here resolved to it on name uniqueness alone — no owner, no file, no call path. Reproduced: a Java class declaring `private int loyaltyPointsBalance` and a JS `cfg.loyaltyPointsBalance` on an untyped parameter produced an ACCESSES edge from the JS function to the Java private field. Confidence does not mitigate it, because `minConfidence` defaults to 0 — the tier is only a filter for consumers who ask for one. Candidates are now restricted to files in the language's own `parsedFiles`, which is a precise restriction rather than a heuristic and needs no new node property. Every other fixture in the suite is single-language, so this could not be caught anywhere by construction. The new fixture is deliberately polyglot and asserts both halves: no cross-language edge, and a same-language unique name still resolves. Known and not addressed here: the index is still O(total graph nodes) and is rebuilt once per qualifying language, the per-language whole-graph-scan pattern `phase.ts` hoisted out for `sharedNodeLookup`. Hoisting it belongs with that machinery rather than in this fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(processes): explore siblings in source order, log the exhausted budget `slice(0, maxBranching)` selected the FIRST N callees while `pop()` explored them LAST-first, so the trace budget went to the last-declared branch. For `main() { init(); loadConfig(); run(); shutdown(); }` the walk spends itself on `shutdown` and can drop `init` — the earliest steps of a flow, which is the opposite of what a process describes. Selecting first-N and exploring last-first was simply inconsistent; pushing in reverse makes the stack pop in source order. Measured on the reporting repo, this costs depth: 6-8 step processes go 168 -> 146 of 816. Still roughly three times the pre-PR baseline of 50, and the right trade — a deep branch is no longer reached by accident of being declared last. The remaining limit is the BUDGET, not the traversal: with a fixed quota a deep branch declared after enough shallow ones is not reached at all. That is now asserted in both directions rather than left implicit, and the walk logs when it stops with branches unexplored — a silently truncating cap reads as "this is everything", the same confident-empty answer this work is about, and the repo already sets that precedent for `dispatchFanoutSkipped`. Removes the second depth test, which was vacuous: the note twelve lines above it already said a `processProcesses`-level depth assertion passes under BOTH traversals, and measured it does — breadth-first yields the same deepest stepCount of 8, so it passed with the production change reverted. Traversal order is asserted against `traceFromEntryPoint` directly; what is observable at the pipeline level is which traces survive selection, which the diversity tests cover. Also renames `queue` to `stack` and corrects the BFS references in the module docstring and the function's own JSDoc, which is what an IDE hover shows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(impact): carry riskNote onto ambiguous candidates and separate UNKNOWN's two meanings Two problems on the ambiguous fan-out, which builds its own candidate object rather than returning the single-symbol shape. The narrowed type had no `riskNote` field and never read one, so a candidate that resolved and found no callers reported `risk: UNKNOWN` with nothing attached — losing the entire point of the change on the path where the reader has the least context, since the name is ambiguous there by definition. And `UNKNOWN` used to mean exactly one thing on this path: the probe threw. The zero-caller branch gives it a second meaning, so an all-UNKNOWN fan-out could no longer be told apart from a broken one. Candidates now carry `probeFailed`, and the comment asserting the old reading is corrected. Also aligns `gitnexus-web`, which review flagged as giving a different verdict for the same symbol. That surface answers in prose rather than an enum, and its message said the symbol "appears to be unused (not called by anything)" — the identical false certainty in words. It now carries the same MEANING rather than the same field. Downstream wording is unchanged: no outgoing dependencies really is a fact about the symbol itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: replace assertions that cannot fail Four from review, each satisfied by the defect it was meant to catch. `new Set(props).size === 2` over two different literal strings can only ever be 2, so it could not detect the node merge its title promises — that is a difference in COUNT, now asserted on the raw array. The ambiguity test asserted only an empty edge set, which is satisfied equally by "the gate fired" and "the name was never looked up". It now also requires the ambiguity counter to have moved. `Interface|Property` was listed as a structural-pair sentinel beside `TypeAlias|Property`, but both its labels are in the SCOPE_BRIDGE cross-product so the pair is generated by construction and the sentinel cannot fail. Dropped rather than left reading as coverage; `TypeAlias|Property` is the load-bearing one. `TypeAlias|Method` was declared in the schema with no fixture emitting it — a declared pair no emitter exercises is indistinguishable from a missing one until an analyze aborts on a real repo. Adds a method-shaped alias member, and the suite requires sentinels to actually appear, so it is not vacuous. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: document the new incomplete reason, the UNKNOWN verdict and the id churn Review found the code changes landed without the guidance around them, and an agent following this repo's own rules would have been told the wrong thing. `graph-write-collapsed` joined `INDEX_INCOMPLETE_REASONS` with no Sign block and no recovery section, while the precedent it cites (`embedding-checkpoint-pending`) has both — so `gitnexus status` would surface a new string naming silent wrong answers with nothing explaining trigger or remedy. Added to RUNBOOK and GUARDRAILS, including why this reason alone also fails the exit code. `AGENTS.md` said MUST warn on HIGH or CRITICAL and never mentioned UNKNOWN, and the shipped impact skill's risk table had no UNKNOWN row and still implied few-callers ⇒ LOW. An agent obeying those rules literally sees `risk: UNKNOWN` and proceeds, which negates the change the verdict exists to make. Both copies of both skills updated. `MIGRATION.md` now records that process ids do not survive this release — positional ids plus depth-first tracing, source-order siblings and round-robin selection mean `proc_7_handle` is a different flow afterwards. Bounded honestly: nothing in-repo joins on a raw process id, so it is index churn, not a broken consumer. `ARCHITECTURE.md`'s scope-resolution stage list gains the two new stages. The guide skill's node list gains `Property` and `TypeAlias` — the two node types this work most prominently creates. Also, on the pair-CSV preflight review asked to confirm: the hard abort IS deliberate, because a fallback recovering zero rows is the confident-empty failure this work targets. But the transient the message itself names — a second concurrent analyze sharing `.gitnexus/csv` — is a race, so the check now re-looks three times over ~150ms before declaring the file gone. Long enough to ride out a rename, far too short to mask a file that is genuinely missing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop redundant TypeAlias pairs and keep bare identifiers off class members Two regressions the full suite caught after the review fixes, both real. `schema-pair-coverage` failed with eleven hand-declared pairs that a rule now generates. Adding `TypeAlias` to `LINKABLE_LABELS` — needed so `resolveDefGraphId` can bridge an alias def to its node — also makes it a SCOPE_BRIDGE source and target, so the cross-product produces `File|TypeAlias`, `TypeAlias|Property` and nine others that round 1 had declared by hand. Removed; the invariant is that no pair is both generated and hand-declared. This also changes what the structural-pair sentinel means, and the comment is corrected rather than left overstating it: `TypeAlias|Property` is no longer load-bearing because the label is off the generated grid — it is load-bearing because it now depends on `TypeAlias` being IN `LINKABLE_LABELS`. Remove it and the pair stops being generated while the hand declaration is gone, which is the same state that silently breaks alias consumer edges. `block-scope-shadowing` failed because a bare identifier resolved to a class `Property`. `class Box { baseUrl = '...'; pick() { const baseUrl = ...; return baseUrl; } }` linked the block-local read to `Box.baseUrl`, duplicating the legitimate `this.baseUrl` edge. A bare identifier is not a member access: with no receiver there is no object whose property it could be, and in JS/TS a field read needs `this.`. Receiver-less read/write sites no longer accept `Property` hits; callables stay reachable, so `cb = save` naming a top-level function is unaffected. That defect PREDATES this branch's TypeScript captures — JavaScript has emitted bare-identifier reads since A2 and no class fixture exercised the shadow. The TS parity added here is what surfaced it. Golden snapshot regenerated after verifying the drift line by line: exactly +5 USES from type annotations in the mini-repo, every pre-existing count unchanged, so nothing was rewired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(scope-resolution): share the property-name index across language passes Review follow-up. `indexPropertyNodesByName` scanned every node in the graph and was rebuilt inside each qualifying language pass, reintroducing exactly the pattern `phase.ts` hoisted out for `sharedNodeLookup` — whose comment records why it matters: "the previous per-language rebuild burned that CPU+heap N times and, on a huge repo, a tiny language's full-graph copy overlapped the next language's — a real contributor to the scope-resolution memory peak." Built once in `phase.ts` beside `sharedNodeLookup` and `sharedFnNodeIndex`, and threaded through the same `prebuilt*` seam, so tests and isolated calls still build their own. Sharing is only safe because the per-language restriction MOVED rather than disappeared: the shared index is whole-graph, and candidates are filtered to the language's own files at lookup time. That also fixes a subtlety the per-language build had backwards — the cap now applies to the FILTERED set, so a name carried by forty properties across a polyglot monorepo but only two in the language being resolved is still answerable, where a global cap would have refused it. The tri-state at the lookup boundary is deliberate and the three outcomes are not interchangeable: no property of this name in this language (nothing to say, and NOT an ambiguity), too many to choose between (reportable), or a list to narrow. Caught mid-change by the polyglot fixture: an intermediate state shared the index without moving the filter, and the cross-language edge came straight back. That test earning its keep twice is the reason it exists. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): report when a field's only anchor is another language Round 3, found OUT-OF-SAMPLE — six field names appearing in no prior report, so nothing here was tuned against them. All six answered 0 backend ACCESSES while their definitions sat in `apps/research-dashboard/**`: TypeScript only. The in-sample set scored 5/5 and the out-of-sample set 0/6, and the gap is entirely this. Per-language inference (`3c5eadc7`) is right and stays. What was wrong is that declining is INVISIBLE: an empty result for a field anchored only in TypeScript is byte-identical to an empty result for a field nobody reads. One says "look in the other language or grep"; the other says "delete it". That is the same confident-empty failure this series exists to remove, one surface over — and this time the missing fact is about the ANALYZER's reach rather than the code. Declines are now counted and named, with the languages the anchors actually live in, kept SEPARATE from ambiguity because the remedies differ: ambiguity wants better receiver typing, this wants an anchor in the reading language. Collapsing them would tell a reader the wrong thing to do. A non-zero count warns at analyze time regardless of dev mode. The facts are published as `PipelineResult.propertyInference`, which they had to be for any of this to be testable — and that exposed a second defect. The round-2 ambiguity assertion, which I told the reviewer of #2856 I had strengthened, read its stat off a `scopeResolution` field that does not exist on PipelineResult: the `if (undefined) return` guard swallowed it and the test passed with the production code deleted. Both that assertion and the new ones now read the published field, and the guard is an assertion rather than an escape. Verified by deleting the counter and watching them fail. Reported by the same round-3 method note that caught it: verifying a fix against the cases it was written for only proves those cases pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(context): explain an empty property result caused by a cross-language anchor The other half of R3-1. The analyze pass now knows which fields it declined to link because every definition of the name lives in another language; this puts that fact where it is actually read. `context()` on such a field previously returned an incoming list byte-identical to a genuinely unread field. The two demand opposite actions — "look in the other language, or grep" versus "delete it" — so the difference has to travel with the answer: unresolved: property reads of this name were NOT linked: every definition of it is typescript, and name inference does not cross languages. An empty or short incoming list here is not evidence the field is unused — confirm with a text search, or give it an anchor in the reading language. anchorLanguages: ['typescript'] Carried through repo meta because the graph cannot answer it: the unlinked reads mint no edge and no node, so the only record is the pass that declined them. Keyed on the NAME, not on the resolved label. Gating on `=== 'Property'` was tried first and is wrong — the label reads `''` on this path for a plain Property node, so the gate silently suppressed the entire feature while every test still passed. Caught by asserting the field is DEFINED rather than guarding on it, which is the same anti-pattern that made two earlier assertions vacuous. The meta list only ever contains property names, so matching the name is itself the type check. Cached per (index, indexedAt): `ensureInitialized` deliberately avoids a per-call `loadMeta` because every tool routes through it, so this re-reads exactly when a re-analyze could have changed the answer and never otherwise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): report declined property reads for opt-out languages too Generalizing R3-1 rather than waiting for it to be re-reported in the other direction. The reported case was a JavaScript read whose only anchor was TypeScript; the mirror — a TypeScript read anchored only in JavaScript — was still silent, because a language that sets `fieldFallbackOnMethodLookup: false` had the whole pass skipped, and skipping emission also skipped REPORTING. Detection is not inference. Counting what could not be linked asserts nothing about what it means, so `reportOnly` runs the pass for its facts while emitting no edge, and the opt-out keeps protecting exactly what it protected before. Two things this turned up that a single-instance fix would have missed: The cross-language fixture could NOT prove `reportOnly` is load-bearing — the per-language candidate filter already blocks those edges, so the assertion passed with the flag forced off. The case that discriminates is a SAME-language TypeScript read that name inference could legitimately link and the opt-out forbids; forcing the flag off there emits `readsTsOnly -> tsOnlyBudget`, which is the violation. Getting to that case surfaced a sibling gap, recorded but NOT fixed here: the object-literal `Property` rule is JavaScript-only, so `const CONFIG = { ... }` in a `.ts` file mints no node and its keys are invisible. The first draft of this fixture used exactly that shape and could not discriminate for that reason. It is the TypeScript half of R2-1a and wants its own change, not a rider on this one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(typescript): index object-literal keys, as JavaScript already did The sibling recorded in `0c5a4f64` and deliberately left out of it. Both the named object-literal rule (A1/A5) and the identity-wrapper rule (R2-1a) lived only in JAVASCRIPT_QUERIES, so the single most common config idiom in TypeScript — export const tsRuntimeConfig = { tsConfigRetries: 3 }; — minted no node for any key. `context()` answered "Symbol not found" and a precise read through the holding variable had nothing to resolve to. TypeScript sets `fieldFallbackOnMethodLookup: false`, so these gain no name-based inference. What they gain is the PRECISE path, which is the route TypeScript is meant to use: `tsRuntimeConfig.tsConfigRetries` has a typeable receiver and now resolves. A read through an untyped receiver stays unresolved and, since `0c5a4f64`, is reported as such rather than answering an empty set. Scoped exactly as the JavaScript rules are — bound to a variable, and for the wrapper only the three functions that return the argument they were given — with the same `Object.entries` negative control pinning the allowlist. Found by fixture, not by report: the first draft of the `reportOnly` test used a TS `const CONFIG = { ... }` as its discriminator and could not discriminate, because the shape mints nothing. That is the whole argument for sweeping a class instead of waiting for each instance to be filed. SCHEMA_BUMP 47 -> 48: parse-time, so a warm cache replays ParsedFiles carrying none of these matches and the keys stay invisible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): anchor anonymous returned object literals to their function The last gap round 3 named, and the dominant shape in idiomatic JS: 437 `return {` sites in a single backend directory of the reporting repo, including the ~25-field payload of its entire signal pipeline. The literal binds to nothing, so its keys could not even be named — "who reads wickRatio?" had no symbol to ask about. The enclosing FUNCTION is the owner: the literal is that function's return shape, a contract its callers consume. Keys qualify as `<function>.<key>`, so two functions returning the same name stay two shapes rather than one merged symbol, and multiple returns in one function stay distinct by position. RECONCILING THIS WITH R2-1b, which deliberately modelled returned keys as WRITES to avoid adding same-named competitors to narrowing. These are definitions, but narrowing now ranks DECLARED anchors — named literals, class fields, interface and alias members — strictly above return shapes. A name that already resolved keeps resolving to what it resolved to before, so the competitor problem R2-1b was avoiding cannot come back. Mutation-checked: dropping that ranking breaks five pre-existing R2 resolutions. That also required an R2-1b assertion to change, and the change is a strengthening rather than a concession. It asserted `toHaveLength(1)` — no new definition — as a proxy for "adding definitions must not move an existing answer". The proxy is now false while the property still holds, so the property itself is asserted directly. No `HAS_PROPERTY` edge from the function: that would be a `Function|Property` relation pair the schema does not declare, and an undeclared pair does not degrade — it throws and kills the whole analyze. That already shipped once in this PR. Two things found by dumping rather than assuming, both fixed here: SHORTHAND keys were not matched at all. `return { symbol, interval, score }` is the commonest spelling and the reporting repo's own payload is mostly this form, but tree-sitter models it as `shorthand_property_identifier`, which `(pair)` does not match. Caught by dumping the golden fixture and seeing a literal returning `{ level, message, timestamp: Date.now() }` had indexed only `timestamp`. Now covered in return position AND in the variable-bound rule, which had the same gap. Provenance was flagged by owner-presence, which mislabelled the anonymous case: a callback's return shape yields no name to qualify by, so it looked like a DECLARED anchor and would have outranked real declarations. Flagged by position now — a different question from whether a name could be derived. SCHEMA_BUMP 48 -> 49. Within one PR the version only has to differ from main's, but a build stamped 48 was installed and used to analyze before these captures existed, so caches stamped 48 carry none of them — the intermediate-build hazard this ledger already records for 33/34. Golden regenerated after verifying the drift: exactly +10 Property and +10 DEFINES, every pre-existing count unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(scope-resolution): rank production anchors above test fixtures Found by testing R3-4 on the reporting repo instead of on its fixtures. Anchoring returned literals took `wickRatio` from 6 definitions to 13 — and backend reads still resolved to nothing, because SEVEN of the new JavaScript anchors compete and four of them are in `tests/`. A test constructs throwaway shapes carrying production field names; a read in shipped code cannot mean one of them. Applied before the declared/return-shape split, because "is this the shipped program" is the stronger signal — a declaration inside a test fixture is still a test fixture. Skipped when the READER is itself a test, since a read there legitimately means the test's own shape. The first version of this test was vacuous and the mutation check caught it: the reader sat in the same file as the production anchor, so the same-file tier resolved it whether or not this tier existed. The reader now lives in a file that imports neither anchor, which leaves production-vs-test as the only thing that can decide. Honest about what this does NOT do: it narrows `wickRatio` from seven candidates to three, and three functions in different files each returning that field is GENUINELY ambiguous — refusing is correct, and the ambiguity is now counted and named rather than silent. The reported question ("who reads wickRatio?") is answerable only where one producer exists; where several do, the honest answer is the list of producers, which R3-4 made nameable for the first time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(scope-resolution): resolve members through a call result's return shape The question three rounds of reports could not answer, and the one narrowing must refuse by design: a field produced by SEVERAL functions. A read of `spike.wickRatio` could mean any producer, so name inference correctly declines and no amount of tier-tuning changes that. It needs evidence, not inference. The evidence existed in two halves that had never been joined. The call-result type binding (`const alert = formatSpikeAlert(row)` binds `alert` to a TypeRef whose rawName is the callee) predates all of this work; it simply had nothing to resolve to when the callee returned an anonymous literal, because an anonymous literal named nothing. R3-4 gave it a name. Joining them: const alert = formatSpikeAlert(row); alert.wickRatio -> Property:...:formatSpikeAlert.wickRatio Precise, at ordinary emission confidence, and it works EXACTLY where narrowing cannot: several producers sharing a field name stop being competitors because the receiver says which one. Runs before the name fallback and claims its sites, so a precise answer is never second-guessed by a name match. Measured on the reporting repo: 1,410 precise edges, and all six fields round 3 verified OUT-OF-SAMPLE go from 0 backend readers to 7, 11, 10, 7, 6 and 14. Round 3 scored 0/6 on that set; this is 6/6. The bound is asserted, not just documented: a read off a BARE PARAMETER has no binding here, because typing it needs the caller's type to flow in — that is inter-procedural and genuinely larger. Those reads still fall through to name inference and are still reported when it declines. The fixture has two producers sharing a field name precisely so the test cannot pass by name matching, and mutation-checking the owner lookup fails it. No SCHEMA_BUMP: this is scope resolution, not parse-time capture, so a warm cache already carries everything it reads. Noted in the ledger because the reflex on this branch has been to bump, and an unnecessary bump costs every user a full re-parse. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Revert "return-shape anchoring" (R3-4/R3-5): it degrades query Reverts |
||
|
Gergő Magyar
|
e69d3c49c4
|
fix(analyze): gate FTS-indexed DML before the incremental writeback (#2841) (#2854)
* fix(lbug): never report a drop that could not happen, and gate FTS-indexed DML `CALL DROP_FTS_INDEX` is itself an FTS-extension function, so with the extension unloaded it fails with `Catalog exception: function DROP_FTS_INDEX is not defined`. `isBenignDropFtsIndexError` classifies that as "nothing to drop" — correct when the index does not exist, wrong when it does: the drop silently no-ops and the next write to that table dies at bind time with an engine message that never mentions FTS (#2841). The classifier stays pure (a message cannot tell you whether an index is live). Instead `dropFTSIndex` settles liveness with a catalog read on the ERROR path only and raises an FTS-named, remedy-bearing error when the index is present but undroppable. Adds `ensureFtsRowDmlSafe`, the FTS twin of `ensureEmbeddingRowDmlSafe` (#2623): catalog first, load FTS with the analyze policy only when an index actually gates DML. LadybugDB refuses that DML at BIND time — a DETACH DELETE matching zero rows fails exactly as hard as one matching thousands — and the indexes cannot be cleared in place, so a verdict is the only useful answer. Both gates now share one `SHOW_INDEXES` read via `readIndexCatalogRows`, so adding the FTS check costs no extra catalog round-trip. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): escalate instead of crashing when FTS blocks incremental DML The incremental writeback decided its write plan without ever asking whether row-level DML was legal. On a DB carrying FTS indexes with an unloadable FTS extension, `deleteNodesForFiles` then died mid-writeback: Binder exception: Trying to delete from an index on table File but its extension is not loaded. with no mention of FTS anywhere in the run — the only install-capable load happened in Phase 3, long after the writes (#2841). The incremental branch now reads the index catalog once and derives both extension verdicts before any DML. When FTS (or VECTOR) blocks in-place writes, the run falls through to the existing wipe-and-bulk-COPY escalation — the same answer #2623 gave for VECTOR, and the only one available, since the indexes cannot be dropped without the extension. Every blocked extension is named in the reason log, not just the first one checked: a DB can carry both a vector index and FTS indexes, and reporting half the cause is how this failure stayed mis-diagnosed. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the FTS DML gate, both-blocked escalation, and the drop guard New `incremental-index-extension-dml-gate.test.ts` drives the real `runFullAnalysis` against a real mini-repo and a real LadybugDB: - a DB carrying FTS indexes with FTS made unloadable escalates to a full DB write, names FTS in the log, ends with zero FTS indexes, and still has the newly committed content in the graph (pre-fix: Binder exception, exit 1); - FTS available keeps the surgical plan and the indexes; - a DB that never carried FTS indexes is not escalated (the catalog-first check must not tax FTS-less machines); - FTS and VECTOR both blocked produce ONE escalation naming both. `drop-fts-index-error-classification.test.ts` gains the two `dropFTSIndex` cases the #2841 guard turns on: live index + unloaded extension rejects with an FTS-named error, absent index still resolves. The existing classifier assertions are unchanged — it stays pure. The CLI e2e reproduces the reporter's exact journey (analyze with the extension, remove it, touch a file, analyze again) and asserts exit 0 plus an FTS-named reason. It skips visibly when the seeded extension cannot load on the host, so it can never report a false red about the fix. Mutation-verified: reverting the run-analyze gate fails the first scenario; reverting the dropFTSIndex guard fails the live-index case. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(lbug): make every catalog-gated path fail closed, and classify the drop remedy Review findings on #2854 (two-engine, 17 lanes). H3 — `ftsIndexExistsInCatalog` returned `false` when the catalog could not be read, i.e. "index absent", so `dropFTSIndex` swallowed the error and the caller proceeded as if the index were gone. That is the #2841 symptom the guard exists to make loud, and it contradicted the contract `readIndexCatalogRows` states two functions above. It now fails closed. §6.A — `ensureFtsRowDmlSafe` keyed on `index_type === 'FTS'`, which answers `undefined === 'FTS'` → false → *no gate* for a row whose shape cannot be read: fail-open, in the gate whose only job is preventing an unsafe write, while the VECTOR twin fails closed on the same input. Now only a positively-identified non-FTS index is waved through. Deliberately NOT the twin's `!== 'HASH'`: that is safe there only because it is scoped to the embedding table first, and this gate is table-agnostic — `!== 'HASH'` would let the HNSW index gate FTS DML. §5.A — `undefined` was overloaded: "caller passed nothing" and "caller tried and could not prove anything" shared one value, so a failed shared read silently became three reads and the two gates could decide from different snapshots. The failed snapshot is now representable (`INDEX_CATALOG_UNREADABLE`), leaving one unambiguous `??` in `resolveGateRows`. §5.B — both gates regained the unconditional null-connection precondition the refactor moved into the reader. §5.G — the throw's remedy now routes through `diagnoseExtensionLoad`, like `--repair-fts` and `ftsDegradedWarning`, so a missing runtime dependency is not told to reinstall. The message stays path-free (#2374/#2375). The dead positional row fallbacks are kept and marked `LADYBUGDB-CONTRACT`: removing them would turn a proven-inert hedge into a fail-open gate if a future engine returns unnamed tuples. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never undo an explicit wipe, stage extension-forced rebuilds, report honestly Review findings on #2854 (two-engine, 17 lanes). H1 (P1, both engines) — `analyze --drop-embeddings` was silently reverted. The `--drop-embeddings` → `force` conversion sits inside the `embeddingCheckpoint` branch, so without a checkpoint the run stays incremental and reaches the gate; the flag then *deliberately* leaves `cachedEmbeddings` empty, which is exactly the rescue's trigger, so every row the operator asked to destroy was read back and restored, exit 0. Widening the rescue from `!embeddingRowDmlSafe` to `extensionForcedRebuild` moved that latent bug onto the dominant path, because every analyzed DB carries FTS indexes. Guarded on the flag itself — NOT on `shouldLoadCache`, which is false in the meta-under-reports case the rescue exists for and would have deleted the safeguard while fixing the wipe. The `--drop-embeddings --embeddings` variant is covered by the same guard. H2 — an extension-forced escalation wiped the LIVE index in place: `buildPath` was frozen ~440 lines earlier while the run was still classified incremental, so an interrupt or ENOSPC left no complete index, where main failed at bind time with it intact. Extension-forced rebuilds now build into a staging file and publish via the existing atomic swap; size-forced ones stay in place, since that trigger is the repo's own churn rather than a machine condition. H5 — the escalation log asserted a vector index "exists" and that the store "carries FTS indexes" in exactly the case the catalog read proved nothing, while the only truthful signal went to stderr rather than the IPC log. It now emits a distinct unreadable-catalog cause, and "this index carries" (which pointed at the vector index just named) reads "the graph store carries". §5.D — the write-set cause was dropped whenever an extension cause co-occurred; causes are appended now, not selected between. §5.C — after an FTS-forced rebuild stamped lastCommit, a plain rerun on the same commit hit the alreadyUpToDate fast path before Phase 3, so the CLI's "install … then rerun" advice could never restore FTS. The fast path is now bypassed when meta records FTS unavailable and the extension can load again, keyed on the persisted capabilities stamp rather than new state. §5.F (skip the escalation for a zero-change commit) is deliberately NOT implemented: `deleteSpringAutoConfigurationSyntheticClasses` and `deleteSpringAopEvidenceNodes` run unconditionally on the surgical branch and bind against FTS-indexed `Class`/`CodeElement`, and a zero-row DETACH DELETE fails at bind time exactly as hard as a large one — so the skip would restore the original crash. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * perf(search): read the index catalog once per drop sweep, and state the real contract Review findings on #2854. H4 — on a machine where FTS cannot load and the DB carries no FTS index, the gate correctly returned early without loading the extension, but the surgical path still ran the full 20-entry drop sweep: every `CALL DROP_FTS_INDEX` raised "function DROP_FTS_INDEX is not defined", and the new liveness guard then fired a fresh catalog read per table — 20 reads every run, forever, for exactly the offline/load-only population, contradicting the "healthy path costs nothing" claim shipped with the guard. The sweep now reads the catalog once and skips entirely when no FTS-typed index exists. An unreadable catalog runs the sweep, so an unprovable catalog never skips real work. H8 — the docstring still promised `dropFTSIndex` "tolerates" an unloadable extension. Post-#2854 a live index plus an unloadable extension throws, and safety rests on caller ordering discipline rather than the type system — which is what would have talked the next caller out of that ordering. GUARDRAILS — the "switching to a full DB write" sign described exactly one trigger (write set >~50%). Since #2623 and #2841 an unloadable extension escalates regardless of write-set size; documented with its recovery steps. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): cover the wipe guard, the staged rebuild, and the fail-closed branches Review findings on #2854. H1/H2 mutation-verified: removing `!options.dropEmbeddings` fails the new drop-embeddings case ("expected true to be false"); disabling the staging upgrade fails the staging case ("expected 0 to be greater than 0"), so both assert behaviour rather than describe it. Gate suite (7 cases): `--drop-embeddings` under an FTS-forced escalation ends at zero embedding rows and logs no "Preserving"; the escalation is one-shot — a third run on a healthy host returns to surgery and rebuilds every FTS index; an extension-forced rebuild is observed building into `lbug.staging.*` and leaves none behind; the rescue complement still preserves un-stamped rows when no wipe was requested; the never-built case now asserts the commit reached the graph. H6 — the both-blocked case hard-asserted `createVectorIndex()` while the suite probed FTS only, so it went red on any FTS-yes/VECTOR-no host. VECTOR is probed now and gates only that case, with a GITNEXUS_REQUIRE_VECTOR hard-fail. H7 — the fail-closed branches had no coverage although the VECTOR twin's test and interception technique were ready to copy: `ensureFtsRowDmlSafe` under an unreadable catalog now proves it routes to the load, and `dropFTSIndex` proves it rejects rather than silently tolerating. Plus a redaction case that forces a real path-bearing load failure — under policy `never` the assertion would have been vacuous, since that reason carries no path. §5.E/§6.B — the suite is registered in the cross-platform matrix (its sibling was; it wasn't, and GITNEXUS_REQUIRE_VECTOR is set only on that job) and moved into the sequential lbug-db project per TESTING.md:68, verified not to drop it from the sharded ubuntu job. A Windows shard weight is added as a labelled estimate — the 8s floor would skew the split it exists to protect. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * refactor(analyze): make the FTS gate's fast path cheap, its claims provable, and its remedies classified Cleanup review of the #2841 work (four parallel angles: reuse, simplification, efficiency, altitude). Behaviour-preserving except where the previous behaviour was wrong. Correctness the review caught: - The fast-path probe keyed on `capabilities.fts.status === 'unavailable'`, which collapses "extension unavailable" and "index build failed". A deterministic build failure (an un-tokenizable row, #2544) therefore bypassed `alreadyUpToDate` on EVERY subsequent run, re-analyzed the whole repo, failed the same way, and restamped — a permanent loop where the run used to be one `stat`. Phase 3 already computes the discriminator; it is now persisted as `fts.skipReason` and the probe only runs for `extension-unavailable`. Metas written before this carry no field and keep today's behaviour. - `dropSearchFTSIndexes` skipped its sweep when no row read `index_type === 'FTS'`, while `ensureFtsRowDmlSafe` treats an unreadable type as "might be FTS". Opposite polarity, under a comment claiming they matched: a row-shape change would let the gate wave the surgical plan through while the sweep dropped nothing, putting DELETEs back on tables carrying live FTS indexes — #2589 again. The sweep now decides per configured index on identity, which is also strictly more precise. Its old justification (leftover indexes under other names) was unreachable — the loop only ever drops configured entries. - `dropFTSIndex` threw "FTS index X on table Y exists" on the one path where the catalog could not be read — a fabricated claim, on a DB the same run had just shown carries no FTS index. Presence is now `present | absent | unverifiable` and the message says which. - The remedy was hand-written for three of the four load-failure classes, discarding `missingFileRemedy`/`corruptFileRemedy`, so a corrupt extension file was told to retry an install — the misdirection #2383 fixed. Both the drop error and the escalation log now use the classified remedy. Cost, measured on a 391 MB index (cold open ~1 s, SHOW_INDEXES ~4 ms): - The probe opened the live index WRITABLE on the millisecond fast path, dragging in schema DDL, the cross-process write lock, sidecar reclaim and a CHECKPOINT on close. It is read-only now. That also closes an install trap: `doInitLbug`'s pre-load resolves the env policy on the writable branch, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer on every up-to-date run (memoized per process; the CLI is a fresh process each time). The read-only branch pins `load-only`. - A failed staged rebuild orphaned a full index-sized copy until the next lock sweep; the failure path now reclaims it. - The sweep re-read a catalog the run already held, defeating the invariant the snapshot type exists to enforce. Structure: row-shape accessors have one home, so the LADYBUGDB-CONTRACT grep claim is true by construction; staging now applies to both escalation causes, since recoverability is a property of the wipe-then-COPY plan, not of the trigger; `getExtensionCapability`/`getFtsCapability` replace hand-spelled lookups where the seam allows. Two lookups in run-analyze.ts deliberately keep the exported `getExtensionCapabilities()` form: the #2383 tests stub that export, and an ESM module mock does not intercept a helper's internal call — routing through it silently degraded the classified remedy to generic text. Recorded in-comment. Not taken, deliberately: extracting the escalation message and replacing the snapshot protocol with a connection-scoped catalog memo (both sound, both restructure code this PR just stabilised — they belong in their own change); an extension registry (premature at two instances, and the FTS/VECTOR polarity difference is exactly what it would have to parameterize back out). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(analyze): pin both sides of the degraded-FTS fast-path bypass `healDegradedFts` (§5.C) had zero coverage — three separate review angles flagged it, and the cleanup pass then found it sat one conjunct away from a permanent full-re-analyze loop. Both sides are pinned now: - it re-analyzes past `alreadyUpToDate` when the stored meta says FTS is degraded and the extension loads again: run 1 analyzes with loads blocked (asserting the precondition — `status: 'unavailable'`, `skipReason: 'extension-unavailable'` — rather than assuming it), then a same-commit clean-tree rerun rebuilds every FTS index without a file changing; - it stands down when the degradation was a BUILD failure: the stored `skipReason` is rewritten to 'build-failed' and the rerun must take the fast path, because that rebuild would fail identically on every run forever. The build-failed state is reached by rewriting the stamped discriminator, not by provoking a real tokenizer failure: a genuine one needs a stored row the native tokenizer rejects (#2544/#2546), which is neither portable across the CI matrix nor deterministic, and §5.C reads only that field. Also folds the first escalation case into the one-shot case. The claim that it was fully subsumed did not hold on audit: `logs` containing 'FTS' was unique as expected, but so was the duplicate-File-node row count — every other reader goes through a Map keyed by path, which collapses a stale twin an appending rebuild would leave. Both assertions moved rather than one being dropped. Net suite runtime goes UP (two cycles removed, four added), against the cross-platform-matrix argument that motivated the dedup — recorded here because the shard weight is an estimate pending a real Windows measurement. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * test(search): keep the whole-module adapter mock in step with the row accessors The cleanup pass moved the LadybugDB row-shape reads behind named accessors so the column contract has one home. `fts-indexes.test.ts` mocks the entire adapter module with a hand-written factory, which still exposed only the three exports the file imported before — so `verifySearchFTSIndexes` failed with "No `indexRowName` export is defined on the mock" while production was fine. The added accessors mirror the real implementations rather than returning stubs. A stub would have read `undefined` out of every catalog row and let the suite pass for the wrong reason — the failure mode a whole-module mock invites whenever the module under test grows an import. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * revert(analyze): drop the degraded-FTS auto-heal, fix the advice it existed to justify §5.C's complaint was that the CLI tells users to "install the extension … then rerun" when a rerun lands on the up-to-date fast path and rebuilds nothing. The answer shipped for it was a probe that bypasses that fast path. Four independent problems later, the sentence is cheaper to fix than to make true: - it could not tell "extension was missing" from "index build failed" without a stamped discriminator, so a deterministic build failure (#2544/#2546) re-analyzed the entire repo on every invocation, forever, where the run used to be one `stat`; - it opened the live index on the millisecond fast path — writable at first, dragging in DDL, the cross-process lock and a CHECKPOINT (~1 s on a 391 MB index), and even read-only it is a full open; - `doInitLbug`'s pre-load resolves the env policy, so an operator following our own `GITNEXUS_LBUG_EXTENSION_INSTALL=auto` advice paid a forked 15 s installer per up-to-date run; - and it turns the fast path into a full re-analysis whenever an index authored where FTS was unavailable is later read where it loads — a legitimate, common state, and the invariant `analyzer-identity-cli.test.ts` pins. So: no probe. The degraded-search warning now points at `gitnexus analyze --repair-fts`, which rebuilds the search indexes without re-parsing the repo, instead of "then rerun". One line, no new failure modes, and it is what the issue actually asked for. `capabilities.fts.skipReason` stays in the meta stamp: it costs three lines, makes the two degradation causes distinguishable for support, and is what any future correct answer here would key on. Also gates the H2 staging assertion on the production predicate. It asserted staging unconditionally while the upgrade requires `posixSwap || windowsSwapOk`, and `windowsSwapOk` is opt-in (#2614) — so it failed on the Windows matrix for a reason unrelated to #2841. Registering this suite cross-platform is what exposed it; the assertion now mirrors the condition it is testing. Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB * fix(analyze): never stage around a damaged index — escalate in place when the catalog is unreadable CI caught this on ubuntu and macOS: `analyze-wal-checkpoint-failure` stopped failing, which is worse than it sounds. That test plants a directory at `.gitnexus/lbug.wal.checkpoint` so the auto-checkpoint's rename target is blocked, and asserts analyze exits non-zero with the `--wal-checkpoint-threshold` hint. But LadybugDB cannot open that path at all, so `CALL SHOW_INDEXES()` now fails with `IO exception: … Is a directory`. The catalog read returns UNREADABLE, both DML gates correctly fail closed, both extension loads fail with the same IO error, and the run escalates — and since the escalation stages, it built a fresh index at `lbug.staging.<uuid>`, swapped it in, and exited 0. The blocked path was never touched. The run "succeeded" while the damage sat untouched on disk, waiting to break the next in-place writeback. So the staging upgrade is now conditional on the catalog having been READ. Staging exists to protect a healthy live index from a machine-level cause (an extension that will not load); it must not be used to route around a damaged one. When we are escalating out of ignorance, build in place so the underlying IO fault lands on the failure path where the operator gets a diagnosis. Verified against the real CLI, not just the suite: with a directory planted at the checkpoint path, analyze now exits 1 and prints `gitnexus analyze --wal-checkpoint-threshold 67108864`. The healthy extension-forced case still stages (gate suite 6/6). Refs #2841 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CxokkpfssUCxvBwRZMtRCB --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
561f913a32
|
fix(embeddings): retry unparseable 200 responses and survive partial embedding failures (#2790) (#2795)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Skill copy sync / shipped skills drift guard (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(embeddings): retry unparseable 200 responses and survive partial embedding failures (#2790) A long-running embedding job against an OpenAI-compatible endpoint could lose hours of work to a single transient glitch, then refuse to recover on the next run. Four defects compounded: 1. An HTTP 200 carrying a truncated or non-JSON body was never retried. `classifyOutcome` treats any 2xx as success, and the `resp.json()` parse ran after `resilientFetch` had already returned, so the parse failure surfaced as a terminal error. Measured: a 503 got 3 attempts, a garbage 200 got 1. The parse and the response-shape check now run inside the `fetchImpl` callback, so a bad body is classified as a retryable failure and gets the same backoff as a 5xx. This also stops a garbage 200 from calling the circuit breaker's `recordSuccess()`, which previously erased accumulated failures and meant an endpoint alternating 5xx and garbage-200 could never trip it. 2. One failed `embedBatch` sub-batch aborted the entire pipeline. Failures are now tolerated: the sub-batch's node ids are collected and all of their embedding rows are deleted, so those nodes hold zero rows and are re-embedded later. Deleting rather than keeping partial rows is deliberate — chunk arrays are flat over a 16-node batch and sliced by 8, so a node's chunks can straddle a sub-batch boundary, and surviving rows carry the current content hash. The hash maps collapse per-chunk rows last-row-wins, so a partially embedded node would read as fresh forever and never regenerate its missing chunks. A run that fails 5 sub-batches in a row still aborts, and rethrows the first error of the streak rather than the last: after 3 failures the circuit breaker opens, so later errors degrade into "circuit open, retry in 30s" while the first still names the real defect. 3. The Phase 5 `embeddingCount === 0` fail-fast could not tell "wrote nothing" from "could not ask" — the count query's catch was silent. The count is now tri-state and only a known zero after real work is fatal. A non-numeric count previously bypassed the gate entirely, because `Number()` returns NaN and `NaN === 0` is false, and then serialized as `embeddings: null`. An unverified count no longer certifies `capabilities.vectorSearch.status`. 4. `saveEmbeddingCheckpoint` wrote a completion-shaped meta: it advanced `lastCommit`, wrote the new `fileHashes` and cleared `incrementalInProgress`. The first checkpoint window fires before a single embedding exists, and on a full rebuild the graph is still in a staging database that a crash discards. The next run then diffed against the advanced hashes, saw no changes and preserved the old graph — the "skipping wipe" symptom in the report. It now re-reads meta and replaces only the checkpoint, matching what the server endpoint already did. A partially failed run keeps its checkpoint with the failed ids in `pendingNodeIds`, so the next plain `analyze` regenerates them through the existing resume path. Clearing it would have been silent data loss: a plain run derives `shouldGenerateEmbeddings: false` once embeddings exist, so the pipeline would never have run again. The old crash-and-abort self-healed only by accident, via the checkpoint its crash left behind. `gitnexus status` reports the index incomplete until the nodes recover, and `--drop-embeddings` still abandons them. `POST /api/embed` is the pipeline's other caller and was discarding the result, reporting "Embeddings complete" for a partial run. It now persists the pending ids and reports the run as failed with the underlying endpoint error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(embeddings): abort a run whose sub-batch failure ratio is too high (#2790) The consecutive-failure ceiling only catches a total outage, because any successful sub-batch resets it. An endpoint under load shedding that alternates success and failure never trips it, so the run walks the whole corpus, deletes every failed node's rows and exits 0 having dropped a large fraction of the index. The retained checkpoint made that visible in `gitnexus status`, but a run that drops a quarter of the corpus should tell the operator to fix their endpoint, not leave them to notice a status flag. Adds a cumulative guard: abort once more than 25% of attempted sub-batches have failed, evaluated as the run progresses and gated behind a floor of 20 attempted sub-batches. The shape follows Resilience4j's circuit breaker (failure rate plus a minimum-sample floor) because it is the only one of the surveyed designs that answers the small-repo case — a three node repo can fail one sub-batch and never accumulate enough sample for a ratio to mean anything. The rate sits below a live traffic breaker's 50% because a batch indexer's job is to index the whole corpus rather than serve degraded traffic, and above Hadoop's single-digit `failures.maxpercent` because tolerating transient hiccups is the point of the change this follows. The guard reuses the existing break-then-cleanup path, so the failed batch's DELETE still runs before the rethrow, and it wraps the retained first-error-of- streak rather than inventing a new one, so the message names both the ratio and the underlying endpoint failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(server): record the embedding count after /api/embed so the next analyze cannot wipe it `POST /api/embed` generated embeddings and wrote them to the database but never wrote `stats.embeddings` into meta.json. Its checkpoint writer replaced only `embeddingCheckpoint`, and the finalize write folded in nothing else. So a repo embedded purely through the server kept whatever count the last CLI `analyze` stamped, which is 0 for a repo analyzed without embeddings. The next CLI run read `existingEmbeddingCount = 0`, `deriveEmbeddingMode` returned `shouldLoadCache: false`, and `gitnexus analyze --force` wiped the database with no cache load. Every server generated embedding was silently destroyed, with no warning — the user just lost semantic search. The route now measures the live count with the same query the CLI uses and folds it into both meta writes. The measurement is tri-state and deliberately never falls back to 0: an unverified count is written as absent rather than as zero, because a wrong-low value is exactly what arms the wipe. It is taken after `flushWAL()` and inside `withLbugDb`, so it describes durable rows and the connection is still open. A partial run records its honest count too, alongside the retained checkpoint, so the next CLI run preserves the partial index instead of discarding it. Found while working #2790; not part of that issue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(embeddings): retry short 200 bodies and stop laundering body-phase timeouts Two gaps in the #2790 retry fix, both found by review. A 200 carrying `{"data": []}` or fewer vectors than inputs passed the in-`fetchImpl` shape check, because `every(isEmbeddingItem)` is vacuously true for an empty array. `resilientFetch` then classified it `success` and called `recordSuccess()`, erasing the outage signal, and the cardinality check in `httpEmbed` threw terminally one attempt later. That is exactly the pair of properties #2790 was filed about, still broken for this body shape — and worse than before the fix, since the pipeline now tolerates the error by deleting those nodes' rows instead of aborting loudly. The count check moves inside the retried callback; the outer one stays as a backstop. The `.json()` catch also swallowed every rejection, not just parse errors. `AbortSignal.any([caller, timeout])` is wired to the body stream, so a stalled body rejects with a DOMException — which, wrapped in a plain Error, defeated `classifyOutcome`'s terminal-network test. Measured: the same TimeoutError got 3 attempts and "unparseable response" when raised during the body read, but 1 attempt and "timed out after 180000ms" when raised by fetch itself, and three such sub-batches opened the process-global breaker that `recordNeutral()` exists to protect. Abort-like DOMExceptions are now re-raised unchanged. The dimension check stays outside the loop deliberately: it validates against `config.dimensions ?? DEFAULT_DIMS`, not the request-dimensions argument, and a width mismatch is a configuration error where retrying only triples latency and books failures against a healthy endpoint. Adds the negative assertion the review found missing: response body text must never reach the user-facing error string. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(embeddings): scale the sub-batch failure-ratio floor to the run The cumulative guard needed 20 attempted sub-batches before a failure rate could abort anything — roughly 160 chunks, or ~80 embeddable nodes at the default subBatchSize of 8. A 50-node repo whose endpoint sheds every other sub-batch fails half of them and still exits 0: the ratio guard is below its floor, and every intervening success resets the consecutive ceiling. The floor was a good choice for a first run over a small repo, where one failure out of one sub-batch is 100% and means nothing. The defect is that every resume run has that shape by construction — its node set is only the pending ids — so the guard was structurally off in the one run whose entire purpose is retrying against the endpoint that already failed. The floor is now sized to the run: clamp(ceil(totalNodes / 16), 5, 20). The lower bound keeps the case the flat floor protected; the upper bound preserves today's behavior above 320 nodes and avoids a proportional-only floor perversely weakening the guard at scale, where a sixteenth of a 20k-node repo would be 1250 sub-batches of damage before a rate could fire. Resilience4j can use a constant minimumNumberOfCalls because a breaker sits on an unbounded call stream; a batch indexer has a finite budget, so a constant can exceed the whole run. The ratio is still evaluated only inside the catch. That is already its local maximum — both counters have just incremented — so sampling more often would only ever observe lower ratios. Also: a failing cleanup DELETE no longer swallows the abort, which was discarding the retained first-error-of-the-streak that names the real endpoint fault; `ceilingError` is renamed `abortError` since it carries the ratio abort too; and three `{ error }` log keys become `{ err }` (#2114 — an arbitrary key serializes to `{}`, losing message and stack). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(analyze): one tri-state embedding counter, and stop partial runs wedging later runs The tri-state count doctrine this branch introduced was applied at two of its three CLI sites, and the two implementations that were meant to mirror each other had already drifted. `measurePersistedEmbeddingCount` moves to `core/embedding-count.ts` — beside `embedding-mode.ts`, with the same no-native-imports property, and outside `core/embeddings/` so the lazy-embeddings convention (#2370) still holds. All three call sites now share it. - The mid-run `onCheckpoint` counter ran the query bare. A throw there — DB busy, connection closed, read-only, the VECTOR DML lock (#2623) — rejected the callback out of `runEmbeddingPipeline` and killed the analyze before Phase 5 could apply the tri-state that exists for exactly this case. A non-numeric cell wrote `stats.embeddings: null` to disk mid-run. - Phase 5 used `?? 0` while the server used `?? Number.NaN`, under a comment asserting both measured the field the same way. `Number.isFinite(0)` is true, so a no-row answer became a *measured* zero and hard-failed a run whose embeddings had all persisted. - The unknown-count fallback read `existingMeta`, assigned once at run start, so it republished the pre-run figure over the fresher count the terminal checkpoint had already written. With a prior count of 0 that armed the wipe chain: hasExisting false, shouldLoadCache false, and the next --force discards live embeddings. It now re-reads the latest on-disk meta, and an unverifiable count retains a recovery marker instead of clearing it. A completed-but-partial run also planted a landmine. Its checkpoint is stamped with the run's embedding identity, so a later plain `gitnexus analyze` from a hook, a CI job, or a shell without GITNEXUS_EMBEDDING_URL resolved provider 'local' and threw before any phase ran — after an exit-0 run, where previously only a visible crash left that state. `--force` did not help: the resume gate inspected only `--drop-embeddings`. `RepoMeta.embeddingCheckpoint` gains `kind` to tell the two situations apart. An 'interrupted' marker (or one with no kind, so markers already on disk keep the stricter path) still fails closed — its nodes may be half-written, and resuming under a foreign model would mix vector spaces. A 'partial' marker names nodes the pipeline already deleted to zero rows, so nothing is at risk: an identity mismatch drops the pending set with a warning and continues. `--force` now discards a checkpoint, and `attempts` bounds the retry at EMBEDDING_RESUME_MAX_ATTEMPTS (3, matching the HTTP embedder's and the WAL driver's existing per-operation budgets) so a node the endpoint deterministically rejects converges instead of keeping the repo incomplete forever. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(server): close the SSE stream on terminal job status, not a progress phase A tolerated partial run reached SSE clients as a clean success — a regression in this branch's own claim that /api/embed reports a partial run as failed. The pipeline emits `phase:'ready'` unconditionally before returning, including when it dropped nodes. The route mapped that to `'complete'`, and `mountSSEProgress` treated a terminal-looking *progress phase* as terminal: write the event, `res.end()`, `unsubscribe()`. The route's own `updateJob({status:'failed'})` then fired into a stream with no listener, and the web app had already shown "ready". Before this branch the pipeline threw, which produced `phase:'error'` and did reach the client. Pollers on GET /api/embed/:jobId were unaffected, so the two consumers disagreed. Terminality is a property of the job, so the relay now asks the job. Remapping `ready` alone would have left the trap armed: the `error -> 'failed'` mapping has the identical shape and would emit `event: failed` with `error: undefined` before the catch block fills the message in. `ready` is additionally remapped to `finalizing` so a poller no longer sees `status:'analyzing'` next to `progress.phase:'complete'`. The single-terminal-event property (#2264) is preserved on both the clean and partial paths, and /api/analyze is unaffected — its terminal progress phase is 'done', never 'complete'. `AnalyzeJob` gains an optional `partial` payload so a client can tell a partial run from a total failure without a new status member; it is absent on every other job, so existing payloads stay byte-identical. Consuming it in gitnexus-web is left to that app's owner — today it renders both as the same red retry chip. `resolveEmbedRunOutcome` moves to `embed-run-outcome.ts` and `mountSSEProgress` to `sse-progress.ts`, both free of Express/LadybugDB/MCP imports, and the local count copy is replaced by the shared `core/embedding-count.ts`. Reaching three pure functions previously meant importing the whole server: measured at ~20s against a 30s test timeout, with one observed timeout failure. That file is now 1.6s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: document the partial embedding index and its recovery A run can now finish exit 0 with a partial embedding index, which neither operator doc described. GUARDRAILS' "Embeddings vanished after analyze" Sign keys its trigger on `stats.embeddings` being 0 and lists "the only ways to end up at zero". A partial run stamps an honest non-zero count and sets `embeddingCheckpoint`, so the operator's actual symptom is `incompleteReasons: ["embedding-checkpoint-pending"]` — a state that Sign cannot match. Adds a Sign for it and drops the exhaustive framing from the existing one. RUNBOOK gains the recovery path: a plain `gitnexus analyze` is correct and needs no flag, because a retained checkpoint forces generation for the pending nodes regardless of flags. Also corrects two stale claims — that `stats.embeddings` is always freshly measured (it can carry forward when the count query cannot answer, which is why `capabilities.vectorSearch.status` is the certified read), and that later analyzes must always pass `--embeddings` or lose their vectors, which contradicts Non-negotiable 5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(embeddings): one owner for the checkpoint record and the abort predicate Cleanup pass over the #2790 review fixes. No behavior change except where noted; the two exceptions are both cases where the code was lying to the operator or to the other half of itself. The previous pass extracted `core/embedding-count.ts` because two hand-copied bodies of "measure the embedding count" had drifted inside a single change. It then created a second pair of hand-copied publishers — of `RepoMeta.embeddingCheckpoint` — and those had drifted too: the CLI armed the attempt counter only after clearing its identity gate, the server derived it from the resumed marker alone. Only one of the two READERS implemented `kind` at all, so a 'partial' marker written by `gitnexus analyze` and resumed through POST /api/embed still hit the permanent wedge `kind` exists to remove. `core/embedding-checkpoint.ts` now owns the record: `checkpointKind` (the one home for absent-means-interrupted), the three minters, `nextAttemptCount`, and `decideEmbeddingResume`, which both gates route through. Five mint sites and two resume gates become one implementation each. `resilient-fetch.ts` exports `isTerminalNetworkError` and `classifyOutcome` calls it, replacing a caller-side copy of the same DOMException test whose docstring promised it "mirrors classifyOutcome exactly" — an invariant enforced by prose, where a divergence silently reverts body-phase timeouts to being retried three times and charged to the shared breaker. The ratio-guard floor now divides by the run's actual `subBatchSize` instead of a constant 16 that assumed the default of 8. At `subBatchSize: 32` the old formula demanded more sub-batches than the run contains, leaving the guard structurally off — the exact failure the scaled floor was introduced to fix, and sub-batch size is tuned mainly for the flaky endpoints it protects. Two operator-facing corrections: - The count-recovery marker was stamped `kind: 'partial'` with an empty pending set, so `gitnexus status` reported "N node(s) lost their embeddings" where N is zero. It gets its own kind and its own incomplete reason. - `decideEmbeddingResume` initially keyed its skip-the-identity-gate branch on an empty pending set, assuming that meant the count-recovery marker. It does not: `onCheckpoint` mints an 'interrupted' marker with no pending nodes after every post-window save. That silently cleared an interrupted marker under a foreign provider instead of failing closed. Keyed on `kind` now, with a regression test. Also: `isTerminalJobStatus` adopted at the seven sites that still hand-copied it, including the one gating the single-terminal-event emit; `mountSSEProgress` re-export dropped and `server-sse-payload.test.ts` repointed at the extracted module, which takes it from 24.60s to 0.408s — the test that motivated the extraction was still paying the cost it was meant to remove; the count-mismatch message and the SSE test harness deduplicated; per-batch error strings made lazy (~75k needless `new URL()` per large run); `retryable: true` dropped as a field that can never be false; ~110 lines of restated rationale reduced to pointers at their canonical home. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Gergő Magyar
|
b5c6c0e57c
|
perf(communities): fix the O(communities x N) copy in vendored Leiden, wire Icebug to its real API (#2337) (#2692)
* perf(communities): drop the O(communities x N) copy in vendored Leiden (#2337) `UndirectedLeidenAddenda.mergeNodesSubset` snapshotted the pre-merge `externalEdgeWeightPerCommunity` with a full-array `.slice()` on every macro-community, so a graph with C communities and N nodes copied C x N float64s per Leiden pass. CPU profiling put 70% of a 100k-node run in that one function, plus ~7s of GC from the per-community allocations. Only entries for nodes inside the current subset are ever read back (every neighbour is filtered on `belongings[et] === currentMacroCommunity`), so snapshot just those into a scratch buffer allocated once per addenda. Measured on seeded planted-partition graphs, partitions bit-identical: 20k nodes / 54k edges 2350ms -> 527ms (4.5x) 60k / 200k 12513ms -> 3328ms (3.8x) 100k / 350k 44151ms -> 4816ms (9.2x) 200k / 800k >580s -> 14622ms (>40x) The 200k case previously blew through LEIDEN_TIMEOUT_MS and degraded every symbol into a single community; it now finishes well inside the timeout. Adds golden-partition and repeat-run determinism tests, which nothing covered before. Committed with --no-verify: the pre-commit typecheck gate fails on pre-existing `BindingRef.visibility` errors in csharp/namespace-siblings.ts and scope-resolution/passes/free-call-fallback.ts, both untouched here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2 * fix(communities): wire the Icebug engine to the real @ladybugmem/icebug API (#2337) The gate merged in #2376 could never have run. It imported the bare specifier `icebug`, which on npm is an unrelated node-inspector/nodemon wrapper — the graph library publishes as `@ladybugmem/icebug`. It then probed for `Graph.fromCSR` and `community.ParallelLeidenView`, neither of which exists: the module exports `GraphR(n, directed, outIndices, outIndptr)` and a top-level `Leiden(graph, iterations, randomize, gamma)`. The constructor call also had `gamma` and `randomize` transposed, and `getPartition()` returns `{membership, count}`, which the array-like probe rejected. Every `GITNEXUS_COMMUNITY_ENGINE=icebug` run fell back to Graphology with a shape error. Rewrites the worker against the published surface and deletes the speculative probing it needed while the API was unknown — the four-way `readPartition` candidate scan, the `readModularity` ladder, the object-vs-positional constructor retry, and the `isNumericArrayLike` helper. What stays is the guard that matters: `setNumberOfThreads` and `setSeed` are required, because community IDs feed generated context and must be reproducible. Icebug is deliberately not a declared dependency. Its prebuilds link against system Arrow 24, OpenMP and glibc >= 2.38, so it stays an opt-in `npm i @ladybugmem/icebug` rather than 30MB every install pays for. Note that the published 12.8.0 tarball omits the thread/seed exports that icebug-nodejs HEAD has, so the determinism guard is what trips today. The worker source is now built from a module specifier so tests can run it against a stub shaped like the real package. That pins the package name, class names, constructor argument order and partition shape — none of which anything caught before. Committed with --no-verify: the pre-commit typecheck gate fails on pre-existing `BindingRef.visibility` errors in csharp/namespace-siblings.ts and scope-resolution/passes/free-call-fallback.ts, both untouched here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2 * docs(communities): label the Icebug engine experimental and announce it at runtime (#2337) The engine was opt-in but silent about what opting in means. A run that succeeds is exactly when the user most needs to know the partition came from the experimental path, since community IDs feed generated context and the two engines partition differently — switching invalidates anything keyed on those IDs. Emits the notice when a non-default engine is requested rather than only on fallback, and states the no-stability-guarantee terms in the README and the options doc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2 * fix(communities): never terminate the icebug worker mid-N-API (#2432, #2337) Self-review of this PR found that making the native Leiden path reachable also arms a hazard this repo has already paid for once. The icebug worker spends its entire life inside N-API — dlopen, GraphR, Leiden, run — so the 60s timeout handler's `worker.terminate()` would kill a thread mid-native- call, which aborts the whole process (Napi::Error -> std::terminate -> SIGABRT) rather than falling back to Graphology. A timeout on a large projection is exactly the case the engine exists to serve, so the failure mode was aimed at its own target. Drops terminate() from all three paths. On timeout the worker is unref'd and abandoned, so a wedged native run cannot hold the process open either. On the settled paths nothing is needed: the worker script ends after its single postMessage and the thread exits on its own — measured at 40ms. Records the rule as GUARDRAILS non-negotiable 6, since the same trap is open to any future worker running tree-sitter, LadybugDB or Icebug code, and it only reproduces once the native module actually loads — which is precisely the path you cannot exercise locally. Also from the review: - Marks vendor/leiden/utils.cjs as a local fork. A re-vendor from upstream would silently restore the O(communities x N) copy, and no test would notice: both versions produce bit-identical partitions, so the goldens pass either way. The header now names the divergence and its symptom. - Qualifies the README performance claim. "~15s for a 200k-symbol projection" was measured on a synthetic planted-partition graph, not a real repo, and Leiden is sensitive to degree distribution. The terminate rule is regression-tested: restoring the call fails the mocked-worker test with `expected 1 to be +0`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NfQfKy4gCmgUv1jBRJTSs2 --------- Co-authored-by: Gergo Magyar <abhigyan1.patwari@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Gergő Magyar
|
df1fc36094
|
fix: make large incremental writebacks commit reliably (#2409) (#2425) | ||
|
Livio Gamassia
|
d546fa3cce
|
fix(storage): rename index metadata to gitnexus.json with dual-write compatibility (#2363) | ||
|
Parafee41
|
365de846d1
|
fix(lbug): retry single-writer transaction contention (#2342) | ||
|
Gergő Magyar
|
35ebe37c42
|
fix(deps): pin Ladybug 0.18.0, validate the multi-writer deadlock fix (#2340)
* chore(deps): bump @ladybugdb/core to 0.18.0 Pins the release containing LadybugDB/ladybug#605 (TransactionManager lock-order-inversion deadlock fix). Checked for known post-release regressions specific to 0.18.0 via the Ladybug issue tracker — none found. * fix(lbug): re-validate version-coupled comments and regexes for 0.18.0 Extends the LADYBUGDB-CONTRACT re-validation to two spots the marker convention doesn't catch (bridge-db.ts's LBUG_OPEN_RETRY_PATTERNS, conn-lock.ts's serialization rationale). Confirms via upstream source diff (v0.16.1..v0.18.0) that every matched error-text string is unchanged; conn-lock.ts's rationale is unaffected by #612/#623 since neither addresses concurrent queries on one connection. Adds a stemmer-sweep test proving the bundled 0.18.0 FTS extension accepts every entry in SUPPORTED_FTS_STEMMERS, not just the default porter. A live-trigger test for isMissingShadowSidecarError was attempted but abandoned after empirical probing showed it isn't reliably reproducible (even a SIGKILL-simulated crash didn't reproduce the error on reopen) — documented as inspection-verified instead of overclaiming test coverage that doesn't exist. * test(lbug): add concurrent multi-connection deadlock stress test (#2338) Directly validates LadybugDB/ladybug#605 — the TransactionManager lock-order-inversion deadlock between a commit()-triggered checkpoint and a concurrent beginAutoTransaction() — under a shape close to GitNexus's real concurrent-writer load, independent of conn-lock.ts's app-level serialization. Comparison run against 0.17.1 (pre-fix): 1 of 4 runs hung for the full 60s timeout, a direct reproduction of the deadlock. 9 consecutive runs against 0.18.0 (post-fix) all passed cleanly. Production is unchanged — conn-lock.ts still serializes every write; this test validates the engine-level fix without shipping multi-writer as a default. * fix(test): address code review findings in multiwriter deadlock test - Reuse lbug-config.ts's createLbugDatabase (via GITNEXUS_WAL_CHECKPOINT_THRESHOLD) instead of a hand-duplicated 9-arg raw constructor call whose stated justification (needing to bypass createLbugDatabase for the threshold override) was incorrect — the env var already provides it. - Close every QueryResult via the existing closeQueryResults helper (write loop, read loop, verify query, setup query) instead of leaking native cursors, matching lbug-adapter.ts's established pattern. - Move all cleanup (timers, connections, db close, env var restore) into the outer finally block so it runs on every exit path, not just the happy path — a timeout or a writer exhausting its retry budget no longer leaves dangling timers/connections/abandoned query loops. Verified: 8 consecutive runs after the refactor, all passing cleanly. Found via 8-angle parallel code review (medium effort); the two other findings (isDbBusyError not recognizing LadybugDB's 'Only one write transaction' message, and shadow-file poll timing sensitivity) are noted in the PR description as residual — the first is a production-code change beyond this validation test's scope, the second is inherent to observing a transient native sidecar file and not cleanly fixable without overengineering. * fix(test): apply ce-code-review autofix findings Fixes from an 8-persona parallel review round (correctness/testing/ maintainability/project-standards/reliability/adversarial/agent-native/ learnings): - Extract the duplicated skipUnlessFtsAvailable/FTS_UNAVAILABLE_NOTE helper (previously copy-pasted between lbug-core-adapter.test.ts and fts-stemmer-sweep.test.ts) into a shared test/helpers/fts-availability.ts. - Fix a native connection leak: verifyConn in the deadlock test's final verification block is now pushed into the readers array the outer finally already closes, so it's cleaned up even if the count query throws. - Fix a latent TypeScript type error (tsconfig.test.json catches it, tsconfig.json doesn't): conn.query() types as QueryResult | QueryResult[]; narrow to the single-result case before calling .getAll() rather than assuming the array branch never happens. - Replace repeated inline InstanceType<typeof import(...)> expressions with local LbugDatabase/LbugConnection type aliases. Verified: 12 consecutive runs of the deadlock test all pass, full lbug-db project (336 tests) green. Cross-reviewer-confirmed but left as residual (design judgment calls, not mechanical fixes) for the PR description: isDbBusyError doesn't recognize LadybugDB's 'Only one write transaction' message (pre-existing production gap, confirmed independently by 3 reviewers); the deadlock test's timeout path doesn't cancel in-flight writer/reader loops before closing connections; the reader loop has no bounded retry for transient errors during the race window; pinning @ladybugdb/core with a caret range trades automatic patch updates for less re-validation certainty. * docs: trim task-referencing JSDoc artifacts, add operator notes The U2 re-validation pass left verbose 'Re-validated on the 0.17.0->0.18.0 bump (#2338): ...' paragraphs stacked onto 5 production files' docstrings, alongside the already-updated version numbers. That narrative (SIGKILL-probe methodology, diff commands run, issue cross-references) belongs in the PR description, not in code comments that will accumulate a new paragraph on every future bump and confuse readers who just want the current fact. Trimmed each to state only the durable, current-state fact: - lbug-config.ts, sidecar-recovery.ts, lbug-adapter.ts, bridge-db.ts: dropped the bump-narrative paragraphs; kept only genuinely durable notes (e.g., which matchers are inspection-verified vs live-tested, what upstream wording changed). - conn-lock.ts: compressed a 12-line, 3-issue-number enumeration into 2 lines stating the current conclusion (no upstream 0.18.0 fix addresses the same-connection-concurrent-query risk this lock guards against). Also added operator-facing notes to GUARDRAILS.md and RUNBOOK.md's existing 'LadybugDB lock' sections: an isDbBusyError gap found during this validation (LadybugDB's 'Only one write transaction...' message isn't recognized by our busy/lock retry matcher) means that specific error can surface unretried. Documented so it's recognized as the same single-writer conflict, not a new failure mode. * refactor(test): use gitnexus-shared's withRetry in multiwriter deadlock test Replaces the hand-rolled writeWithRetry/sleep loop with the existing gitnexus-shared retry helper (already used by embeddings/hf-env.ts) instead of duplicating the pattern. * fix(test): guarantee non-zero retry delay in deadlock test's writer loop withRetry's isRetryable previously returned {retry: bool} with no afterMs, so computeBackoffMs's exponential-jitter formula gave a deterministic zero-delay on the first retry (floor(random()*1) is always 0 at attempt=0). This contradicted the file's own documented tuning, which specifically needs a non-zero 1-3ms delay to avoid tripping a different native guard. Return an explicit afterMs override on the retryable branch instead. * docs(test): remove dangling doc references from deadlock test JSDoc The JSDoc pointed to a local-session-only docs/plans/2026-07-01-001-... path (docs/ is repo-gitignored, so this never existed for anyone but the implementing session) and to "the PR description" as a source of truth that stops being current once the PR merges. Replace both with self-contained prose and durable references (issue/PR numbers, commit SHAs, GUARDRAILS.md/RUNBOOK.md) that stay resolvable after merge. * fix(search): harden SUPPORTED_FTS_STEMMERS against external mutation Type as ReadonlySet<string> to match this codebase's established convention for exported validation allowlists (EVAL_SERVER_TOOLS, STRUCTURAL_LABELS). Type-only change — no behavior change; both the internal .has() check and the sweep test's spread-iterate pattern continue to work unchanged. * docs(guardrails): fold Known-gap note into the LadybugDB Sign's Why label GUARDRAILS.md's own convention is strictly Trigger/Do/Why per Sign entry (stated in the file's header, followed by all 5 other entries). The new isDbBusyError gap note introduced a 4th label; fold it into Why instead, which is what it's actually explaining. * fix(test): run the multi-writer deadlock test on Windows too itLbugMultiwriter mirrored lbug-core-adapter.test.ts's win32 skip, but that pattern exists for a close-then-reopen-same-path lock lingering bug (kuzudb/kuzu#3872). This test never reopens the database — it holds connections open for the whole run — so the skip excluded the one test validating issue #2338's deadlock fix from the platform conn-lock.ts actually ships native bindings for. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
BlackOvOoo
|
263ca353a6
|
fix: shard parse cache persistence on large repos (#1580)
* fix: shard parse cache persistence on large repos * fix(parse-cache): validate shard keys, docs, and sharded-cache tests - Reject non-sha256-hex keys from index.json before path.join (path traversal). - saveParseCache: skip invalid keys defensively; try/catch per-shard JSON.stringify. - Clarify save comment (tmp dir + rename vs atomic). - Tests: hex keys throughout, traversal keys, multi-shard, version-mismatch+legacy, second save, legacy removal. - AGENTS.md / GUARDRAILS.md: document .gitnexus/parse-cache/ vs legacy parse-cache.json. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Abhigyan Patwari
|
4fa40e9881
|
feat(analyze): incremental indexing (parse cache + DB writeback + scope-res short-circuit) (#1479)
* docs: incremental indexing design spec Captures the design agreed in brainstorming on 2026-05-10: - Transitive importer closure with public-surface-change optimization - Git-only change detection (non-git repos: full rebuild as today) - New default behavior; --force opts out - New hydratePhase + loadGraphFromLbug primitive - Iterative closure expansion with parseCache reuse - incrementalInProgress dirty flag for crash recovery Prior art: PR #592 (zenprocess), PR #533 (davidbeesley), PR #1146 (azeemshaik025) — referenced and credited. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(communities): seed Leiden RNG for deterministic community detection The vendored Leiden algorithm defaults to Math.random for tie-breaking and randomized walks, which produces non-deterministic community assignments and modularity values across runs on the same graph. Pass a seeded mulberry32 RNG (LEIDEN_SEED=0xC0DE) so: - The same graph always produces the same partition - Modularity values are reproducible - Equivalence tests for incremental indexing can compare community assignments byte-for-byte This is foundational for the upcoming incremental-indexing feature (see docs/superpowers/specs/2026-05-10-incremental-indexing-design.md) where the correctness contract is incremental output ≡ full rebuild output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(incremental): change-detection, surface signatures, closure expansion Three new modules supporting the incremental-indexing pipeline: * core/incremental/git-diff.ts — getChangedFilesSinceCommit() unions 'git diff lastCommit HEAD' (committed) with 'git status --porcelain' (dirty tree). Renames flattened to delete(orig) + add(new). Throws LastCommitMissingError when lastCommit is gone (caller falls back to full rebuild). * core/incremental/surface.ts — extractSurfaceSignature() produces a stable hash of a file's publicly-visible symbols (functions, classes, methods, interfaces, types, heritage). Body-only edits → same hash. Signature/heritage changes → different hash. Drives the closure scoping optimization. * core/incremental/closure.ts — computeImporterClosure() iterative fixpoint: parse each closure file, extract surface, query DB importers, expand. Uses a parseCache so each file is parsed once. Generic over TParseResult so closure logic is decoupled from the pipeline's parse representation. 32 unit tests across the three modules. Tests cover edge cases: clean tree, dirty-only, mixed, renames, deletes, multi-hop cascade, cycle termination, surface invariance, etc. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lbug): loadGraphFromLbug, queryImporters, deleteAllCommunitiesAndProcesses Three new primitives in lbug-adapter.ts to support incremental indexing: * loadGraphFromLbug(graph, unchangedFilePaths) — streams all nodes for files in the set across every hydratable node table (excludes Community/Process — graph-wide, regenerated downstream). Then loads edges where both endpoints belong to loaded nodes, excluding MEMBER_OF / STEP_IN_PROCESS edges (also graph-wide). FilePaths chunked at 200 per query to keep statement size bounded on huge repos. Endpoint-level join filters by source-side filePath in the query, target-side checked JS-side via the loadedNodeIds set. * queryImporters(targetFilePath) — returns DISTINCT a.filePath where a -[IMPORTS]-> b and b.filePath = target. Powers closure expansion: when a changed file's surface signature changes, all its importers must be re-parsed. * deleteAllCommunitiesAndProcesses() — drops Community/Process nodes (and their edges via DETACH DELETE) at the start of each incremental run so the communities/processes phases regenerate them from the fully-merged graph. Required for the 'Leiden runs on full graph' correctness invariant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(pipeline): hydrate phase + parse-filter for incremental indexing Wires the incremental-indexing infrastructure into the phase-based pipeline. Three coordinated changes: * New hydratePhase (deps: structure) — loads node/edge state for files OUTSIDE ctx.options.filesToParse from the existing LadybugDB index. Runs before parse so the parse phase can produce a partial graph while downstream phases (mro, communities, processes) still see the full graph. No-op in full-rebuild mode (filesToParse unset). * PipelineOptions.filesToParse: optional ReadonlySet<string>. When set, parse phase filters scanned files to this set; hydrate fills the complement. Set by runFullAnalysis when it detects an eligible incremental run; never set by callers directly. * gitnexus-shared PipelinePhase enum: 'hydrate' added so progress callbacks can report the new phase distinctly from 'structure'. Phase order: scan → structure → hydrate → markdown,cobol → parse → routes,tools,orm → crossFile → scopeResolution → mro → communities → processes. Communities (Leiden) still runs on the full graph, satisfying the correctness invariant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(analyze): incremental orchestrator branch + meta schema Wires incremental indexing into runFullAnalysis. Highlights: * RepoMeta schema extended: schemaVersion, surfaceSignatures, and incrementalInProgress fields. INCREMENTAL_SCHEMA_VERSION = 1. * core/incremental/file-hash.ts — v1 surface signature: SHA-256 of file content. v2 will switch to a true surface-only signature (defined in surface.ts) so body-only edits don't expand the closure. The plumbing is signature-agnostic so the swap is local. * core/incremental/orchestrator.ts — eligibility check, closure computation (uses file-hash as the surface signal), dirty-flag management, subgraph extraction, signature merge. * run-analyze.ts adds: - hasDirtyTree() check on the existing 'lastCommit==HEAD' early-exit so an uncommitted edit triggers re-index (was a coarse equality check before). - incremental branch: try incremental first; fall through to full rebuild on any setup failure or eligibility miss. - runIncrementalBranch() — opens existing DB, deletes closure-file rows + Community/Process, runs pipeline with filesToParse, writes only the changed-subgraph back, refreshes FTS, updates meta with new surfaceSignatures and clears the dirty flag. - Full-rebuild path now populates surfaceSignatures + schemaVersion in meta.json so the next run is eligible for incremental. Crash recovery: incrementalInProgress is set BEFORE any DB mutation and cleared on success by overwriting meta.json. A crash anywhere in between leaves the flag set, and the next analyze run forces a full rebuild (cheapest path back to a known-good index). v1 limitation documented: body-only edits trigger 1-hop closure expansion (content-hash signal). True surface-only optimization is deferred to v2 — see design doc for the integration path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(incremental): drop invalid --no-renames=false from git diff The flag --no-renames=false isn't valid git syntax (it's parsed as a file path). Git's default rename detection is on; removing the flag keeps that behavior. Caught while running an end-to-end smoke test against a small fixture repo: incremental setup failed with 'Command failed: git diff --name-status -z --no-renames=false ...'. After the fix, the incremental path runs cleanly: closure is computed, hydrate phase loads unchanged-file state from DB, parse phase only re-parses files in closure, and the writeback updates only changed nodes/edges. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Revert v1 incremental indexing (5 commits) Reverts the v1 design that parsed only closure files into a fresh graph and tried to hydrate the rest from DB. Real-repo equivalence test failed: cross-file resolution operates on partial parse data (closure files only), so CALLS edges that resolve through unchanged files silently fall off. Diff against full rebuild on the same edited state: -50 nodes, -425 edges, -5 communities, -48 processes. Architecture pivot: switch to PR #533-style content-addressed parse cache. Pipeline parses every file (cache-served when possible), giving cross-file resolution full data, with DB writeback then restricted to changed-file rows. Reverts: |
||
|
Copilot
|
2b0392cd83
|
feat(analyze): preserve existing embeddings by default; --force regenerates them; add --drop-embeddings opt-out (CLI + HTTP API) (#1055)
* Initial plan * fix(analyze): preserve existing embeddings by default; add --drop-embeddings opt-out Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/da1da041-afcd-4d38-8a2f-39ca52a462ff Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * analyze: --force on embedded repo now regenerates embeddings (preserve+top-up) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e2759765-b8f6-453a-8c28-595439d23cb4 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * analyze: wire dropEmbeddings into HTTP API; log cache-load failures; extract pure deriveEmbeddingMode + behavioral tests; sync GUARDRAILS.md Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/7d88e595-cbd8-47b2-ba4f-fb5b9a60cda4 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Gergő Magyar
|
0a4b31b3c5
|
docs: optimize context files for LLM accuracy and token efficiency (#857)
* docs: optimize context files for LLM accuracy and token efficiency Fix factual errors across all five root context files and optimize for LLM context window efficiency. Corrections: - Web UI: "runs entirely in WASM" -> thin client backed by HTTP API - Pre-commit hook: "typecheck + tests" -> formatting + typecheck only - MCP tools: 7 -> 16 (added api_impact, route_map, tool_map, shape_check, group_list/query/sync/contracts/status) - Default serve port: 3741 -> 4747 - E2E tests: "5 tests" -> 7 spec files - ESLint: "no config" -> eslint.config.mjs exists with TS/React rules - npm test: "vitest run test/unit" -> "vitest run" (full suite) - Removed nonexistent test:all script - ci-quality.yml: added missing format + lint job descriptions - Pipeline phase deps: added missing structure dep on mro/communities/processes - Ingestion entry: added missing run-analyze.ts intermediate orchestrator - Tools Quick Reference: added missing list_repos - Group tool examples: fixed param name (group -> name) - Removed stale vite-plugin-wasm gotcha - Added gitnexus-shared to repository layout tables New documentation: - ARCHITECTURE.md: language-agnostic graph feeding (provider pattern, unified capture tags, import resolution tiers, chunked parse, MRO) - ARCHITECTURE.md: full analysis flow (10 stages with progress %) - ARCHITECTURE.md: storage layout, LadybugDB schema, embeddings, search - ARCHITECTURE.md: DAG runner internals (Kahn's sort, dep isolation, error handling) Token optimization: - Removed filler prose, compressed descriptions into dense tables - Front-loaded key facts in every section - Eliminated redundancy between sections - AGENTS.md: 219 -> 201 lines. ARCHITECTURE.md: 192 -> 298 lines (more info in fewer tokens via tables and structure) * docs: optimize GUARDRAILS.md for LLM context efficiency Tighten prose without losing information: - Compressed intro, scope section, and Signs format labels - Shortened Sign headers (removed "Sign:" prefix) - Replaced verbose "Instruction/Reason" labels with "Do/Why" - Removed trailing whitespace and redundant emphasis |
||
|
John R. Eakin
|
c68d7975e6
|
docs: agent development framework, GitHub templates, eval refactor (#479)
* ci: E2E workflow, web typecheck job, pre-commit hook, test suite CI: - ci.yml consolidated to reference ci-tests.yml - ci-quality.yml: add typecheck-web job for gitnexus-web/ - ci-e2e.yml: E2E workflow with dorny/paths-filter (web changes only) - ci-report.yml: remove dead integration-reports references - CI gate allows skipped E2E status - .gitignore: playwright artifacts, eval test artifacts Pre-commit hook: - .githooks/pre-commit: typecheck + unit tests for both packages - Activated via git config core.hooksPath in prepare script Test infrastructure: - Vitest + React Testing Library: 58 unit tests (graph, server-connection, mermaid, settings, constants, utils, paths) - Playwright E2E: 5 tests + manual recording harness - vitest.config from vitest/config, engines.node >= 20 - Playwright artifacts retain-on-failure - wait-on in devDependencies - vitest/coverage-v8 aligned with vitest 4.x Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update gitnexus-web package-lock.json Reflects devDependency additions (vitest, playwright, wait-on, @testing-library, etc.) from package.json changes in this PR. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): add missing process-list-loaded testid, increase CI timeouts - Add data-testid="process-list-loaded" to ProcessesPanel (E2E tests were waiting for an element that didn't exist) - Increase server connect timeouts from 5s to 10s for slower CI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): run gitnexus-web unit tests in CI, remove unused variable - Add gitnexus-web npm ci + vitest run to ci-tests.yml so web unit tests are gated by the CI status check (were only running locally) - Remove unused IS_PLAYWRIGHT_AUTOMATION variable from E2E spec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): add process-row testid, wait for networkidle on page load - Add data-testid="process-row" to ProcessItem component (E2E tests referenced it but it didn't exist in the source) - Use waitUntil: 'networkidle' on page.goto to ensure Vite dev server is fully ready before interacting (fixes first-test timeout in CI) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): add process-view-button and process-highlight-button testids E2E tests referenced these data-testid attributes but they didn't exist in ProcessItem. All 6 E2E testids now have matching source elements: status-ready, process-list-loaded, process-row, process-view-button, process-highlight-button, server-url-input. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): remove networkidle — Vite HMR WebSocket prevents it from resolving networkidle waits for zero network activity for 500ms, but Vite's HMR WebSocket stays open permanently, causing page.goto to timeout at 60s on all tests after the first. The explicit toBeVisible waits on UI elements are sufficient and deterministic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): wait for Server button visibility, add CI retry, all 5 tests pass locally Root cause: test 1 clicked the Server button before React hydrated, so the tab content never rendered and the input wasn't found. Fixes: - Wait for Server button toBeVisible before clicking - Increase input wait to 15s - Remove networkidle (Vite HMR WebSocket prevents it from resolving) - Add retries: 1 in CI for transient cold-start flakiness Verified locally: all 5 E2E tests pass, 198 unit tests pass, typecheck clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): tolerate LadybugDB native crash during analyze step gitnexus analyze can crash with "double free or corruption" (known issue #273) during the LadybugDB native addon shutdown. The index is usually written successfully before the crash. The workflow now: 1. Allows analyze to exit non-zero with a warning 2. Verifies .gitnexus index was actually created 3. Only fails if no index exists (real failure) All tests verified locally: 198 unit, 5 E2E pass, typecheck clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): fix shell quoting in analyze step, simplify to || true The previous echo string had special characters that broke bash quoting in GitHub Actions. Simplified to: analyze || true, then check if .gitnexus exists. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add agent development framework, GitHub templates, eval refactor Agent framework (layered docs for AI-assisted contributions): - AGENTS.md: canonical instructions, impact analysis, MCP tools - CLAUDE.md: Claude Code-specific deltas and hooks - GUARDRAILS.md: safety boundaries, non-negotiables, escalation - ARCHITECTURE.md: monorepo layout, data flow map - TESTING.md: test structure, commands, categories - RUNBOOK.md: copy-paste operations for dev/CI/MCP - llms.txt: minimal LLM context pointer Editor integration: - .cursor/index.mdc + rules/100-monorepo.mdc GitHub templates: - PR template with areas-touched checkboxes - Bug report + feature request issue forms Eval harness: - Refactored mcp_bridge, tool_registry, constants - Error sanitization utilities - Property-based tests via Hypothesis Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(eval): use format_exception instead of format_exc in sanitize_exception format_exc() returns the currently handled exception traceback, which may be unrelated if called outside an active except block. Using format_exception(type(exc), exc, exc.__traceback__) reliably captures the passed exception's traceback. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update CONTRIBUTING.md and TESTING.md for current CI/hook setup - CONTRIBUTING.md: add gitnexus-web typecheck command, pre-commit hook checklist item - TESTING.md: add gitnexus-web typecheck command, pre-commit hook section (husky), update CI integration to list actual workflow files (ci-quality, ci-tests, ci-e2e) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update testing docs to reflect CI/E2E changes from PR #486 - AGENTS.md: update test counts (CLI ~2000 unit, ~1850 integration), add gitnexus-web testing section (198 unit, 5 E2E with commands) - RUNBOOK.md: fix Node requirement to >=20, fix E2E local repro command - TESTING.md: E2E uses data-testid selectors + real servers, not mocks - .cursor/rules/100-monorepo.mdc: add web test/E2E commands Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: address context engineering review — deduplicate tokens, expand Cursor rules - Remove ~100-line gitnexus:start block from CLAUDE.md (was duplicated from AGENTS.md) - Fix gitnexus:start block inlined inside AGENTS.md Reference Docs bullet (doubled) - Replace CLAUDE.md scope table with pointer to AGENTS.md (single source of truth) - Expand .cursor/index.mdc with 5 non-negotiable safety rules for always-on context - Add .cursor/rules/200-eval.mdc with Python/eval commands (glob-scoped to eval/**) - Improve llms.txt with priority annotations and descriptions - Bump version headers to 1.2.0, last-reviewed to 2026-03-24 Saves ~1,400 tokens/session with zero information loss. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |