GitNexus/gitnexus/scripts
Gergő Magyar 576e81442e
Some checks failed
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Devcontainer Smoke / Config-transform unit tests (push) Has been cancelled
Devcontainer Smoke / Build devcontainer image (push) Has been cancelled
fix(search): index description field for FTS so doc comments are keyword-searchable (#2300)
* fix(search): index description column for FTS so doc comments are keyword-searchable

Closes #2299. descriptionExtractor (#2286) populates the `description`
column for every symbol table, but FTS only indexed name+content on 5
tables, so doc-comment keywords (Javadoc/KDoc/godoc/Rust ///) were
invisible to BM25 keyword search.

- Add `description` to the Function/Class/Method/Interface FTS indexes
  (File has no description column, left as name+content).
- Add FTS indexes for the remaining EMBEDDABLE_LABELS symbol tables
  (Struct, Enum, Trait, Impl, Macro, Namespace, Constructor, TypeAlias,
  Typedef, Const, Property, Record, Union, Static, Variable).
- createSearchFTSIndexes now drops-then-creates each index so the schema
  change reaches existing DBs on incremental re-analyze and --repair-fts
  (createFTSIndex is idempotent-by-name and would otherwise skip stale
  indexes).

Tests: fts-schema column-subset + coverage guards; drop-before-create
order; e2e doc-comment keyword search (Java class + Rust struct found by
description-only terms). bm25-search assertions derive from FTS_INDEXES.

* fix(review): apply autofix feedback

- Guard the --repair-fts path on FTS-extension availability before
  createSearchFTSIndexes drops-then-creates indexes (P1 regression:
  without the gate, an unavailable extension could drop existing indexes
  then fail to recreate them, leaving the DB index-less). Mirrors the
  analyze path's ftsAvailable gate and fails loudly first.
- Add a re-analyze upgrade integration test: seed an old name+content-only
  DB (no Struct index), run the real createSearchFTSIndexes(), and assert
  description keyword search + the previously un-indexed Struct now resolve.
  Proves drop-then-create upgrades a live stale index end-to-end.

* fix(ci): add loadFTSExtension to --repair-fts test mocks

The R3 review fix added a loadFTSExtension availability gate to the
--repair-fts path, but run-analyze-fts-repair.test.ts mocked the lbug
adapter without that export, so both repair tests threw `No
"loadFTSExtension" export`. Add loadFTSExtension to the two mocks
(returning true to preserve their original intent) and add a dedicated
test proving the guard fails loudly — and does NOT drop any index —
when the extension is unavailable.

* test(fts): run fts-description-search in the sequential lbug-db project

It was the only FTS-index-creating integration test left in the parallel
`default` vitest project; every other ftsIndexes-using test (search-core,
search-pool, augmentation, …) runs in the `lbug-db` project, which forces
fileParallelism: false to avoid LadybugDB native mmap file-lock conflicts
in parallel forks (Windows). Add it to the lbug-db include list and the
default exclude list to match the convention and remove the flake risk.

* test(ci): fail loudly when FTS extension is unavailable, never silently skip

FTS-dependent lbug integration suites (search-core, search-pool,
augmentation, fts-description-search, …) self-skip via ctx.skip() when the
LadybugDB FTS extension can't load, emitting only a console.warn while the
job stays green. That means a broken/missing FTS extension in CI would make
these integration tests silently vanish with no signal — false confidence.

withTestLbugDB now honors GITNEXUS_REQUIRE_FTS=1: when set and the extension
is unavailable, setup() throws instead of skipping, so the suite fails
loudly. The CI test jobs (ubuntu coverage + windows/macOS cross-platform)
set the flag; local/offline runs leave it unset and keep skipping
gracefully. (Verified the extension currently loads on all three runners,
so this is a guard against regression, not a behavior change today.)

* test(ci): run fts-description-search on macOS/Windows cross-platform jobs

The new FTS description-search suite was registered in the sequential
lbug-db vitest project (ubuntu/coverage) but absent from LBUG_NATIVE, so
the macOS/Windows platform-sensitive jobs (which run only the explicit
ALL_CROSS_PLATFORM allowlist via run-cross-platform.ts) never executed it.
The GITNEXUS_REQUIRE_FTS=1 hardening on those jobs guarded the old FTS
fixtures but not the new 20-index/description path. Add the suite to
LBUG_NATIVE so the new path is validated cross-platform too.

Refs #2299.

* fix(search): verify FTS indexes cover description, not just queryability

verifySearchFTSIndexes probed each index with QUERY_FTS_INDEX and treated
'queryable' as 'present'. A stale name+content-only index left on a
pre-#2299 DB stays queryable yet silently misses the description column, so
verification would pass green while doc-comment search stayed broken.

Switch to a single CALL SHOW_INDEXES() that exposes property_names per
index, and report an index as missing when it is absent OR does not cover
its configured columns. Return contract (string[] of table.indexName) is
unchanged, so both run-analyze.ts call sites are untouched. The per-index
string interpolation is gone, so the now-dead safeIdentifier helper is
removed.

The real caller of the live function in tests is bm25-search.test.ts (the
repair test mocks verifySearchFTSIndexes wholesale); its two probe-shaped
cases are rewritten to feed SHOW_INDEXES rows and now assert column
coverage, plus an absent-index case.

Refs #2299.

* test(search): assert description search via the public query surface

The #2299 integration suite only exercised the searchFTSFromLbug helper.
Add a third block that drives the public LocalBackend.callTool('query')
path — which resolves the repo via the registry and routes BM25 through the
pool adapter (a different connection context than the core-adapter helper) —
and asserts a description-only keyword returns the seeded class. Reuses the
existing description-only SEED and production FTS_INDEXES; partial-mocks
repo-manager so listRegisteredRepos points at the test DB while
cleanupOldKuzuFiles and the rest stay real.

Refs #2299.

* test(search): make lbug-core-adapter FTS gate honor GITNEXUS_REQUIRE_FTS

lbug-core-adapter.test.ts has its own per-test FTS gate (skipUnlessFtsAvailable)
that called ctx.skip() whenever the extension could not load — bypassing the
GITNEXUS_REQUIRE_FTS=1 hardening that withTestLbugDB already honors. Since this
file is in LBUG_NATIVE it runs on the ubuntu/macOS/windows jobs that all set
GITNEXUS_REQUIRE_FTS=1, so an FTS regression on a runner would have let these
FTS-primitive tests silently vanish from a green run — the exact gap #2299's
test-infra hardening set out to close.

Make the helper mirror withTestLbugDB: when GITNEXUS_REQUIRE_FTS=1 and the
extension is unavailable, throw (hard fail) instead of skipping. Offline/local
runs (no env var) still skip gracefully.

Refs #2299.
2026-06-25 14:21:44 +01:00
..
bench fix: batch query enrichment, bake FTS extension into CLI image, add FTS memory repro (#2108) 2026-06-09 08:46:46 +01:00
spikes feat(ingestion): M0 — taint/PDG substrate (schema + seams + spikes) (#2080) (#2092) 2026-06-08 18:56:10 +01:00
assert-publish-grammar-coverage.cjs fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111) (#2144) 2026-06-10 14:20:42 +01:00
bench-scope-resolution.ts refactor(ingestion): delete legacy call-resolution DAG + heritage processor (RING4-1, #942) (#2023) 2026-06-04 11:07:37 +01:00
build-tree-sitter-grammars.cjs fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111) (#2144) 2026-06-10 14:20:42 +01:00
build.js fix(build): skip build.js when running outside the monorepo (#1795) (#1816) 2026-05-25 15:31:11 +01:00
cross-platform-tests.ts fix(search): index description field for FTS so doc comments are keyword-searchable (#2300) 2026-06-25 14:21:44 +01:00
install-duckdb-extension.mjs fix: batch query enrichment, bake FTS extension into CLI image, add FTS memory repro (#2108) 2026-06-09 08:46:46 +01:00
run-cross-platform.ts chore(ci): consolidate parity shards and narrow cross-platform matrix (#1798) 2026-05-24 12:10:10 +01:00