cae60c2bd4
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* fix(mcp): reject corrupt query identities and refresh index metadata * fix(mcp): preserve reader and search integrity Keep replaced readers unknown until matching metadata is published. Defer shared-handle refresh until other consumers release the old graph to avoid rebuilding connections against the same stale generation. Propagate identity corruption after both hybrid search lanes settle, so the sibling cannot query after its database session is released. Select the regressions for the OS matrix and retain Python coverage without optional Swift grammars. Validation: 393 focused tests, nine native generation tests and ten race executions across five fresh processes pass. Build and source/test typechecks pass. The early full run passed 26,096 tests and failed eight: seven reproduce on baseline; the mixed-source publication case passes in clean reruns. Windows/macOS native execution remains for CI. Graph analysis maps all edited production symbols; coverage remains partial for the anonymous FTS test callback, manually reviewed. Refs #3571 * fix(lbug): bump @ladybugdb/core to 0.21.2 to stop doubled rows after delete and re-copy LadybugDB 0.21.0 and 0.21.1 return every row of a table that spans more than one native vector twice once the incremental writeback has DETACH DELETEd it and a COPY into an earlier table has run. NodeGroup::scan let a morsel that ended before the group's first live row wrap an unsigned subtraction and rescan to the end of the group (LadybugDB #1085, fixed in 0.21.2). No primary key error fires. The identity guard from #3442 stops the run with "unexpected or duplicate ID", so a Swift edit that pulls every importer into the write set failed on repos with more than 2048 structs. Only unreleased main was affected: 1.6.12 ships 0.18.3. Pin 0.21.2 in package.json, update the six @ladybugdb/core entries in the lockfile, and move coreVersion in the vendored FTS manifest. The FTS extension stays at 0.21.0 because no v0.21.1 or v0.21.2 artifact is published; the pairing assertion and its unit tests pass. Add a regression to the delete-nodes-for-files suite: 2049 structs with CONTAINS edges are loaded, every file is deleted, the graph is loaded again, and each id and edge must appear exactly once. Verification (Linux, Node 22.18): - lbug-delete-nodes-for-files: 11/11 on 0.21.2. With the 0.21.1 native packages swapped in, the new test fails with 4098 rows instead of 2049. - Reporter recipe end to end on the built CLI: incremental analyze exits 0 with no duplicate ids on a Swift-heavy fixture and on a 70-file Python fixture, with and without FTS. On 0.21.1 the Swift fixture exits 1. - A 0.21.1-written database opens and reads under 0.21.2. - prettier, eslint and typecheck:tests are clean on the changed files. - The full lbug-db project had not finished when this was committed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(fts): say what the Windows warm-session branch covers On Windows the repair step calls poolClose, which removes the pool entry before the in-place writer opens. The step therefore checks the backend's cached capability state, not a live pool connection. The comment now says so instead of implying the pool stays warm. No behavior change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * revert: take the @ladybugdb/core 0.21.2 bump out of this PR Reverts |
||
|---|---|---|
| .. | ||
| baseline.json | ||
| measure.cjs | ||
| README.md | ||
| reproduce-search.cjs | ||
| reproduce.cjs | ||
| search-root-cause.md | ||
Incremental node identity reconciliation
For the native cause of #3421/#3423 and regressions covering both missing context results and Unicode Property FTS, see search-root-cause.md.
From gitnexus/:
node --import tsx bench/incremental-write-integrity/measure.cjs --check
node bench/incremental-write-integrity/reproduce.cjs
The first command measures the production reconciliation, including every
single-label scan and comparison of ID, name, path and source range. Construction,
CSV loading, checkpoints and correctness controls are outside the timed region.
Two warm-ups precede seven measured samples. Exact counts and SHA-256 fixture
fingerprints reject empty or incomplete work. --check gates normalized scaling
against the committed budget, rather than a machine-specific time limit.
On Node v24.19.0, Linux x64, Xeon Platinum 8370C, the initial measurement was 583 ms for 16,384 nodes and 2,810 ms for 65,536 nodes per reconciliation. The normalized scaling was 1.206 (budget 2). Analyze performs two reconciliations: after graph COPY/checkpoint and after FTS/embedding work plus a final checkpoint, before registration, freshness metadata or staging publication.
The verifier covers retained nodes as well as the write set. A path/range filter would let a corrupted field hide its own row; a count would miss swapped fields or blank IDs. Folder lifecycle, preserved Community/Process layers and streamed BasicBlock rows require other oracles and are excluded. Relationships and source content are outside this identity check.
Native-only reduction
reproduce.cjs uses only @ladybugdb/core, its native schema/query/COPY API, and
synthetic ASCII CSV files. There is no parser, parse cache, graph adapter, FTS,
relationship table, periodic checkpoint driver or concurrent writer. Compression
is disabled and COPY is serial, matching the production settings.
It loads 8,192 Function rows, checkpoints, deletes 64 rows belonging to the first
and last owners, then copies back those same 64 tuples. An independent tuple-set
oracle validates the complete single-label scan before and after each operation.
It also compares affected scan rows with primary-key lookups after reopening.
Use --keep to retain the synthetic database and CSV files; the JSON output gives
their directory. --require-corruption is a diagnostic assertion, deliberately
not a CI requirement: it should fail when the native defect is fixed.
With the original @ladybugdb/core 0.18.3, this reduction produced:
| Phase | Rows | Incorrect tuples |
|---|---|---|
| Initial COPY + checkpoint | 8,192 | 0 |
| DELETE | 8,128 | 0 |
| Incremental COPY | 8,192 | 3,936 |
| Explicit checkpoint | 8,192 | 3,936 |
| Read-only reopen | 8,192 | 3,936 |
With @ladybugdb/core 0.21.1, the same reduction returns zero incorrect
tuples in every phase. The production reconciliation and negative controls
remain required; this result covers the reduced fixture, not a replay of the
original incident.
For example, the scan returned an empty id at native offset 64, while a
primary-key lookup at that same offset returned
Function:src/owner2.ts:fn64 with the correct name/path/range. Smaller 1,024,
2,048 and 4,096-row fixtures remained healthy in the same three-cycle adapter
probe. The affected tuple count varies with fixture size and scan shape.
This isolates a native scan/lookup discrepancy introduced by incremental COPY into a previously populated table. It survives checkpoint and reopen, but does not establish physical byte loss or the precise C++ defect. The reported Unicode change and FTS build are unnecessary for this reduction; this does not prove that the original incident has exactly the same native cause.
The measurement harness runs selective native writes too. An independent oracle requires the production verifier to reject any inconsistent scan, both before and after checkpoint/reopen; healthy scans must certify their complete identity set. It does not quietly treat the native discrepancy as a successful graph.
Missing-row and swapped-range negative controls separately verify that the check fails closed. Publication tests inject damage after COPY and after FTS, assert unchanged registry/freshness, preserve the live staged baseline, and exercise automatic dirty-index recovery followed by a no-op run. An in-place verification failure keeps a non-FTS dirty phase: an FTS-only repair cannot clear the marker and recertify the inconsistent graph. The same protection applies when FTS returns but analyzer finalization fails before the final identity scan: recovery still rebuilds the graph.