GitNexus/gitnexus/test/unit/group/sync.test.ts
WENJIE HUANG 32b5c0e3fc
feat: add IncludeExtractor for C++ cross-repo include tracking (group) (#1156)
* feat: add IncludeExtractor for C++ cross-repo include tracking (group)

* fix: address CodeQL warnings on include-extractor

- Remove unused HEADER_GLOB constant in include-extractor.ts
- Use fs.mkdtempSync for secure temp dir creation in tests
  (CodeQL: 'Insecure temporary file')

* fix(group): close missing ); in manifest-extractor include branch

The 'include' branch in ManifestExtractor.resolveSymbol was missing
the closing ); for the executor() call, causing a syntax error that
broke ESLint, Prettier, and the full test CI on all platforms.

Reported by Claude PR review on #1156.

* chore: drop test/global-setup.ts + test/vitest.d.ts

Upstream removed these in commit 3f0c74fe (ladybugdb 0.16.0 upgrade).
Commit 3f5d21c5 accidentally restored them during a rebase dance.

* style(group): reformat VALID_CONTRACT_TYPES array to satisfy prettier

Adding 'include' pushed the array over prettier's 100-char limit,
so prettier prefers multi-line. Apply the reformat to unbreak
ci-quality/format job.

* fix(include-extractor): address PR #1156 Claude review findings #3-#7

Claude Deep Review raised 7 findings on the IncludeExtractor. #1/#2
(BLOCKERs) were fixed earlier. This commit closes the remaining five.

#3 HIGH  case-sensitive FS -> provider contract-id collision
  Document the deliberate case-folding trade-off on normalizeIncludePath
  (matches C/C++ convention on Windows/macOS; collapses Foo.h & foo.h on
  Linux). Add a unit test pinning the behavior.

#4 HIGH  suffixResolve short-suffix match silently drops cross-repo include
  When a local file ends with the same basename as an external include
  (e.g. local internal/api.h vs. #include "ext/api.h"), suffixResolve
  returned a bogus local hit and suppressed the cross-repo consumer.
  Replace the suffixResolve lookup inside include-extractor with a
  strict isLocalInclude() that only accepts full-path hits via
  SuffixIndex.get / getInsensitive. Callers of suffixResolve elsewhere
  are unaffected. Add 3 unit tests covering the regression.

#5 MEDIUM regex fallback matched #include inside /* ... */
  Strip block comments before running the fallback regex scan.
  Add a unit test.

#6 MEDIUM meta.source was hard-coded to 'tree_sitter'
  Track the actual extraction path with an extractionSource local and
  write it into meta.source so downstream audits can distinguish
  tree-sitter parses from regex fallbacks. Add 2 unit tests.

#7 MEDIUM missing end-to-end coverage
  Add test/integration/group/include-extractor-sync.test.ts with 3
  cases exercising extractor -> syncGroup -> CrossLink (mocked
  contracts, mixed-case/backslash normalization, real temp repos).

Tests: 21 unit + 3 integration, all green.

* fix(lbug): robust Windows lock acquisition for CI integration tests

LadybugDB's `new Database()` raises `Could not set lock on file` from
local_file_system.cpp synchronously inside the constructor — before any
query is issued, so `withLbugDb`'s query-time retry never sees it. On
Windows CI this surfaces as flaky integration tests due to AV-scanner
holds, libuv handle-release lag, and stale `.wal` sidecars from aborted
prior runs.

This change closes the gap at *open time*:

- `openLbugConnection` now wraps `new lbug.Database()` in a bounded
  busy-retry (5x100ms back-off) inside `lbug-config.ts`. Errors that
  exhaust the budget are tagged via `LBUG_OPEN_RETRY_EXHAUSTED` so
  `withLbugDb`'s outer 3x retry skips re-retrying a freshly-exhausted
  path (eliminates the 3x5=15-attempt / ~6s tail latency).
- For recognized test fixtures only (immediate-parent dir matches a
  known prefix AND resolves under `os.tmpdir()`), one final stale-
  sidecar sweep removes `.wal`/`.lock` and retries once. Production
  paths never enter this branch.
- `safeClose` on Windows runs a bounded `fs.open` probe to absorb
  native handle-release lag; logs a warning if the probe exhausts so
  operators can spot AV interference.
- `isDbBusyError` is now defined in `lbug-config.ts` as the single
  source of truth, re-exported from `lbug-adapter.ts` for compatibility.
- New tests cover open-time retry (happy/retry/exhaust/non-busy/tag),
  stale-sidecar sweep (test-fixture-only, production-rejection,
  preserves-original-error), `isTestFixturePath` direct unit suite
  (accept/reject/traversal/nested/trailing-sep), and
  `waitForWindowsHandleRelease` (openable/ENOENT/no-leak).
- The two new test files are added to vitest's existing serialized
  `lbug-db` project (already `fileParallelism: false`).

Closes the chronic Windows CI flake on lbug-touching integration tests
while preserving the existing single-writable-Database-per-process
LadybugDB contract. No public API surface changed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lbug): drop isDbBusyError re-export, import from lbug-config directly

The re-export from lbug-adapter.ts was a transitional convenience — with
the matcher now living in lbug-config.ts, having two import paths for the
same symbol invites future drift. Updated the two real consumers
(lbug-lock-retry.test.ts, lbug-open-retry.test.ts) to import from
lbug-config directly, removed the re-export equality test (now vacuous),
and refreshed the explanatory comment so it no longer references a
re-export pattern that doesn't exist.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lbug): silence benign LadybugDB v0.16.1 schema-init lock warnings on Windows

doInitLbug logs "⚠️ Schema creation warning: ... Could not set lock on
file" on every CREATE NODE TABLE call after the first init on a given
dbPath, on Windows. The lock is internal to LadybugDB v0.16.1 and is
resolved before the table is created — same tolerance pattern as the
existing "already exists" filter. Genuine cross-process lock contention
still surfaces on the next operation through withLbugDb's retry, so
filtering at the schema-init catch only suppresses noise, not signal.

Also extend the safeClose Windows handle-release probe to cover the
.wal sidecar (the previous Database's WAL handle was the slowest to
release, surfacing as the schema-query lock contention) and switch the
probe back to 'r+' so it actually detects exclusive locks.

Test loop in lbug-close-handle-release.test.ts simplified to 10 plain
iterations now that the underlying noise is filtered upstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(lbug): isDbBusyError review fixes

- Drop redundant `could not set lock` term — already subsumed by `lock`.
- Document the intentionally-broad matcher: graph-DB lock-shaped errors
  ("deadlock", "unlock failed", "lock contention", "could not open lock
  file") are all treated as transient. If a non-transient surfaces,
  tighten the matcher rather than raise the retry budget.
- Add positive test cases covering those lock-shaped strings so the
  intent is visible and a future tightening would deliberately break
  these.
- Fix the open-retry back-off comment: max sleep is 100+200+300+400 =
  1000ms (no sleep after the final attempt), not 1.5s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): address PR #1156 follow-up review findings

Addresses two blockers and two mediums from the deep review.

BLOCKER 1: Windows CI ENOTEMPTY in sync.test.ts
  After this PR added writeBridge() to syncGroup, the existing test
  "writes registry to groupDir when skipWrite is false" fails on
  windows-latest. LadybugDB's checkpoint thread briefly outlives
  closeBridgeDb, holding a Win32 lock on bridge.lbug; the test's
  fs.rmSync then fails with ENOTEMPTY. Switched the test cleanup to
  cleanupTempDir from test/helpers/test-db.ts which already tolerates
  EBUSY/EPERM/EACCES/ENOTEMPTY with bounded retries — same pattern
  used elsewhere for LadybugDB-touching tests.

BLOCKER 2: Graph provider absolute-path bug
  extractProvidersGraph queried File.filePath from the LadybugDB graph
  but never stripped the repo root, so provider contract IDs ended up
  as include::/abs/path/foo.h while consumers emitted include::foo.h.
  These never matched through runExactMatch — silently producing 0
  cross-links for any indexed C++ repo (the primary use case).
  Now passes repoPath into extractProvidersGraph and applies
  path.relative(); rows that resolve outside repoPath (stale absolute
  paths from another machine, system headers somehow indexed) are
  dropped instead of polluting the registry.

MEDIUM: `../` relative includes produce spurious noise
  `#include "../foo.h"` is almost always intra-repo, but the suffix
  index can never match a `..`-prefixed path so it became a consumer
  contract no provider could satisfy. Now skipped before matching;
  covers both forward-slash and backslash forms.

MEDIUM: writeBridge error in sync.ts propagates uncaught
  contracts.json is the canonical source of truth and was just written
  successfully when writeBridge runs. A bridge-only failure (disk full,
  schema error, permission denied) shouldn't mask the registry. Wrapped
  writeBridge in try/catch with a logger.warn surfacing the path and
  recovery instructions.

Tests added:
  - extractProvidersGraph repo-relative ID generation (stub Cypher
    executor returns absolute paths)
  - extractProvidersGraph drops rows whose path resolves outside repo
  - `../foo.h` forward-slash skip
  - `..\foo.h` backslash-form skip

Skipped findings:
  - canExtract() removal (#5, low): canExtract is part of the
    ContractExtractor interface; every other extractor implements the
    same `return true` shape. Removing it from IncludeExtractor would
    break the interface contract — keeping for consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): close PR #1156 Codex adversarial findings

Two HIGH findings from the Codex adversarial review on
feat/group-include-extractor:

1. Default-on extraction silently changes existing groups (BLOCKER)
   DEFAULT_DETECT.includes was true, so any pre-existing group.yaml
   that omits the new field would gain a wave of include::* contracts
   on the next sync after upgrade. Flipped to false (opt-in). The
   integration test already declares includes: true explicitly so it
   survives unchanged; the unit extractor tests bypass parseGroupConfig
   entirely; the sync test uses extractorOverride. Only config-parser
   needed regression tests covering omitted/explicit/false variants.

2. IncludeExtractor scans outside the indexed file universe (BLOCKER)
   The extractor was running glob('**/*', { ignore: STANDARD_IGNORES })
   twice with a hand-rolled 9-pattern list, no .gitignore/.gitnexusignore
   honoring, and no max-file-size cap. That meant File:<path> contracts
   could appear for files ingestion would never index, producing
   cross-links group impact cannot fan out to (silent false-negatives).
   Refactored to a single discoverIndexableFiles() helper that mirrors
   walkRepositoryPaths exactly: createIgnoreFilter + getMaxFileSizeBytes,
   one discovery pass shared by provider and consumer paths. Dropped
   STANDARD_IGNORES and SOURCE_GLOB entirely.

   third_party and 3rdparty (the C/C++ vendored-deps conventions) were
   in the local ignore list but not in the canonical DEFAULT_IGNORE_LIST
   used by ingestion. Folded both into the canonical set rather than
   keep a parallel list — the whole point of the Codex finding is that
   two file-discovery implementations drift. Single source of truth.

Tests: 5 new regression tests for the discovery alignment (.gitignore,
.gitnexusignore, max-file-size on both provider and consumer paths)
plus 4 for the opt-in default. All 30 include-extractor tests + the
494-test group suite + ignore-service tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): apply autofix feedback

ce-code-review surfaced 6 safe_auto findings on commit a9936a9b:

- T1 (testing, P2): the sync.ts:174 gate was untested with includes:false.
  Added a sync-level test mirroring the existing thrift-off pattern at
  sync.test.ts:545, asserting zero include contracts when the gate is
  disabled in a real syncGroup call.

- T3 (testing, P3): third_party and 3rdparty entries in DEFAULT_IGNORE_LIST
  had no regression test. Added both to ignore-service.test.ts's
  dependency-directories it.each block.

- M1 (maintainability, P3): discoverIndexableFiles JSDoc lacked a
  fork-warning relative to walkRepositoryPaths. Added a MAINTENANCE
  note explaining why the duplication is tolerated and the contract
  the two implementations must keep.

- M2 (maintainability, P3): thrift-extractor still hand-rolls its
  ignore array with no signal that DEFAULT_IGNORE_LIST additions
  silently do not apply there. Added TODO(#1156-followup) comments
  above both call sites.

- M3 (maintainability, P3): SOURCE_EXTENSIONS duplicated the four
  HEADER_EXTENSIONS entries with no expressed subset relationship.
  Spread HEADER_EXTENSIONS into SOURCE_EXTENSIONS so future header-
  extension additions propagate.

- C1+T4 (correctness+testing, P3, cross-reviewer corroborated):
  discoverIndexableFiles swallowed all fs.stat errors silently,
  including EACCES/EMFILE/EIO. Narrowed the catch to ENOENT (the
  documented benign glob/stat race) and added a logger.warn for
  any other code so operators can spot permission/resource issues.

All 629 tests pass; typecheck + prettier clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): use retryRename in writeContractRegistry to absorb Windows EPERM

`storage.ts:62` used raw `fsp.rename` for the contracts.json atomic swap.
On Windows, AV scanners and concurrent renames briefly hold the
destination handle between rename calls, surfacing as EPERM/EBUSY.
The `insecure-tempfile.test.ts > concurrent writes do not collide`
test was flaking with `EPERM: operation not permitted, rename` on
windows-latest CI.

`bridge-db.ts` already has a battle-tested `retryRename(src, dst, 3)`
helper used at six call sites for exactly this pattern. Reusing it
here keeps the Windows-rename policy single-source-of-truth across
the group package.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(group): drop macro-style #include from consumer contracts

Tree-sitter's `(_) @import.source` wildcard matches the identifier node
of `#include PLATFORM_HEADER`, so the cleaned value `PLATFORM_HEADER`
slipped past the system-header / `..` filters and was emitted as a
permanently orphaned consumer contract (no file is named after a macro
identifier, so no provider can ever match). Add a shape guard that
skips cleaned values lacking both a path separator and an extension
dot, plus regression tests for single and multi-macro files.

Also document `IncludeExtractor.canExtract()` as unused by sync.ts
(gated via `config.detect.includes` instead) and kept solely for
ContractExtractor interface uniformity.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: HuangWenjie <zhoudeng.hwj@alibaba-inc.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 09:31:59 +01:00

1022 lines
34 KiB
TypeScript

import { describe, it, expect, vi, afterEach } from 'vitest';
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import { syncGroup, stableRepoPoolId } from '../../../src/core/group/sync.js';
import { cleanupTempDir } from '../../helpers/test-db.js';
import { _captureLogger } from '../../../src/core/logger.js';
import type {
GroupConfig,
StoredContract,
RepoHandle,
GroupManifestLink,
} from '../../../src/core/group/types.js';
import type { RegistryEntry } from '../../../src/storage/repo-manager.js';
describe('syncGroup', () => {
const makeConfig = (repos: Record<string, string>): GroupConfig => ({
version: 1,
name: 'test',
description: '',
repos,
links: [],
packages: {},
detect: {
http: true,
grpc: false,
thrift: false,
topics: false,
shared_libs: false,
embedding_fallback: false,
workspace_deps: false,
},
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
});
it('returns SyncResult with contracts and cross-links', async () => {
const config = makeConfig({ 'app/backend': 'backend-repo', 'app/frontend': 'frontend-repo' });
const mockContracts: StoredContract[] = [
{
contractId: 'http::GET::/api/users',
type: 'http',
role: 'provider',
symbolUid: 'uid-1',
symbolRef: { filePath: 'src/ctrl.ts', name: 'UserController.list' },
symbolName: 'UserController.list',
confidence: 0.8,
meta: { method: 'GET', path: '/api/users' },
repo: 'app/backend',
},
{
contractId: 'http::GET::/api/users',
type: 'http',
role: 'consumer',
symbolUid: 'uid-2',
symbolRef: { filePath: 'src/api.ts', name: 'fetchUsers' },
symbolName: 'fetchUsers',
confidence: 0.7,
meta: { method: 'GET', path: '/api/users' },
repo: 'app/frontend',
},
];
const result = await syncGroup(config, {
extractorOverride: async () => mockContracts,
skipWrite: true,
});
expect(result.contracts).toHaveLength(2);
expect(result.crossLinks).toHaveLength(1);
expect(result.crossLinks[0].matchType).toBe('exact');
expect(result.crossLinks[0].confidence).toBe(1.0);
expect(result.unmatched).toHaveLength(0);
});
it('reports missing repos', async () => {
const config = makeConfig({ 'app/backend': 'nonexistent-repo' });
const result = await syncGroup(config, {
resolveRepoHandle: async () => null,
skipWrite: true,
});
expect(result.missingRepos).toContain('app/backend');
expect(result.contracts).toHaveLength(0);
});
it('handles empty repos config', async () => {
const config = makeConfig({});
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
expect(result.contracts).toHaveLength(0);
expect(result.crossLinks).toHaveLength(0);
expect(result.missingRepos).toHaveLength(0);
});
it('intra-repo matching works with service field via extractorOverride', async () => {
const config = makeConfig({ 'platform/monorepo': 'monorepo' });
const mockContracts: StoredContract[] = [
{
...makeContract('http::GET::/api/users', 'provider', 'platform/monorepo'),
service: 'services/auth',
},
{
...makeContract('http::GET::/api/users', 'consumer', 'platform/monorepo'),
service: 'services/gateway',
},
];
const result = await syncGroup(config, {
extractorOverride: async () => mockContracts,
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(1);
expect(result.crossLinks[0].from.service).toBe('services/gateway');
expect(result.crossLinks[0].to.service).toBe('services/auth');
});
function makeContract(id: string, role: 'provider' | 'consumer', repo: string): StoredContract {
return {
contractId: id,
type: 'http',
role,
symbolUid: `uid-${repo}-${id}`,
symbolRef: { filePath: `src/${repo}.ts`, name: `fn-${id}` },
symbolName: `fn-${id}`,
confidence: 0.8,
meta: {},
repo,
};
}
it('per-repo extractorOverride receives repo handle and extracts per repo', async () => {
const config = makeConfig({
'app/backend': 'backend-repo',
'app/frontend': 'frontend-repo',
});
const perRepoOverride = async (repo: RepoHandle) => {
if (repo.path === 'app/backend') {
return [makeContract('http::GET::/api/users', 'provider', 'app/backend')];
}
return [makeContract('http::GET::/api/users', 'consumer', 'app/frontend')];
};
const result = await syncGroup(config, {
extractorOverride: perRepoOverride,
resolveRepoHandle: async (_name, groupPath) => ({
id: groupPath,
path: groupPath,
repoPath: '/tmp/' + groupPath,
storagePath: '/tmp/' + groupPath + '/.gitnexus',
}),
skipWrite: true,
});
// per-repo override goes through the initLbug path which will fail
// but the extractorOverride with arity > 0 triggers the else branch
// At minimum, the function should not throw
expect(result).toBeDefined();
});
it('test_syncGroup_closes_only_opened_pools', async () => {
const config = makeConfig({
'app/backend': 'backend-repo',
'app/frontend': 'frontend-repo',
});
const closedIds: string[] = [];
const { vi } = await import('vitest');
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockImplementation(async (id?: string) => {
if (id) closedIds.push(id);
});
try {
await syncGroup(config, {
resolveRepoHandle: async (_name, groupPath) => ({
id: groupPath.replace(/\//g, '-'),
path: groupPath,
repoPath: '/tmp/' + groupPath,
storagePath: '/tmp/' + groupPath + '/.gitnexus',
}),
skipWrite: true,
}).catch(() => {});
// closeLbug must have been called at least once with specific pool ids
expect(closeSpy.mock.calls.length).toBeGreaterThan(0);
expect(closedIds).toContain('app-backend');
expect(closedIds).toContain('app-frontend');
// Every call must have a truthy string id
for (const id of closedIds) {
expect(id).toBeTruthy();
expect(typeof id).toBe('string');
}
// No blanket close (no-arg or empty-string or undefined)
const blanketCalls = closeSpy.mock.calls.filter((args) => args.length === 0 || !args[0]);
expect(blanketCalls).toHaveLength(0);
} finally {
initSpy.mockRestore();
closeSpy.mockRestore();
}
});
it('manifest links in config.links produce cross-links with matchType manifest', async () => {
const links: GroupManifestLink[] = [
{
from: 'app/consumer',
to: 'app/provider',
type: 'http',
contract: 'GET::/api/orders',
role: 'consumer',
},
];
const config: GroupConfig = {
version: 1,
name: 'test',
description: '',
repos: { 'app/consumer': 'consumer-repo', 'app/provider': 'provider-repo' },
links,
packages: {},
detect: {
http: true,
grpc: false,
thrift: false,
topics: false,
shared_libs: false,
embedding_fallback: false,
workspace_deps: false,
},
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
};
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
// ManifestExtractor should inject 2 contracts (provider + consumer) and 1 cross-link
expect(result.contracts).toHaveLength(2);
const manifestLinks = result.crossLinks.filter((cl) => cl.matchType === 'manifest');
expect(manifestLinks).toHaveLength(1);
expect(manifestLinks[0].contractId).toBe('http::GET::/api/orders');
expect(manifestLinks[0].from.repo).toBe('app/consumer');
expect(manifestLinks[0].to.repo).toBe('app/provider');
expect(manifestLinks[0].confidence).toBe(1.0);
// With no DB executors available, UIDs fall back to the deterministic
// synthetic form `manifest::<repo>::<contractId>`.
expect(manifestLinks[0].from.symbolUid).toBe('manifest::app/consumer::http::GET::/api/orders');
expect(manifestLinks[0].to.symbolUid).toBe('manifest::app/provider::http::GET::/api/orders');
// Manifest contracts also participate in runExactMatch; we must not emit a
// duplicate matchType:'exact' cross-link for the same endpoint pair.
const exactForSameContract = result.crossLinks.filter(
(cl) => cl.matchType === 'exact' && cl.contractId === 'http::GET::/api/orders',
);
expect(exactForSameContract).toHaveLength(0);
expect(result.crossLinks).toHaveLength(1);
});
it('runs thrift wildcard matching after exact matching and returns wildcard remaining', async () => {
const config = makeConfig({ 'app/provider': 'provider-repo', 'app/consumer': 'consumer-repo' });
const provider: StoredContract = {
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
type: 'thrift',
role: 'provider',
symbolUid: 'uid-provider-place-order',
symbolRef: { filePath: 'src/provider.ts', name: 'OrderService.PlaceOrder' },
symbolName: 'OrderService.PlaceOrder',
confidence: 0.9,
meta: {},
repo: 'app/provider',
};
const consumer: StoredContract = {
contractId: 'thrift::OrderService/*',
type: 'thrift',
role: 'consumer',
symbolUid: 'uid-consumer-order-service',
symbolRef: { filePath: 'src/consumer.ts', name: 'OrderClient' },
symbolName: 'OrderClient',
confidence: 0.8,
meta: {},
repo: 'app/consumer',
};
const result = await syncGroup(config, {
extractorOverride: async () => [provider, consumer],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(1);
expect(result.crossLinks[0].matchType).toBe('wildcard');
expect(result.crossLinks[0].contractId).toBe('thrift::OrderService/*');
expect(result.crossLinks[0].from.repo).toBe('app/consumer');
expect(result.crossLinks[0].to.repo).toBe('app/provider');
expect(result.unmatched).toEqual([provider]);
});
it('keeps wildcard thrift links to multiple extracted IDL provider methods', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-wildcard-'));
fs.mkdirSync(path.join(tmpDir, 'idl'), { recursive: true });
fs.writeFileSync(
path.join(tmpDir, 'idl', 'order.thrift'),
`namespace java billing.v1
service OrderService {
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
OrderResponse GetOrder(1: string orderId)
}`,
);
try {
const { ThriftExtractor } =
await import('../../../src/core/group/extractors/thrift-extractor.js');
const extractedProviders = (
await new ThriftExtractor().extract(null, tmpDir, {
id: 'provider-repo',
path: 'app/provider',
repoPath: tmpDir,
storagePath: path.join(tmpDir, '.gitnexus'),
})
)
.filter((c) => c.role === 'provider')
.map(
(c): StoredContract => ({
...c,
repo: 'app/provider',
}),
);
const consumer: StoredContract = {
contractId: 'thrift::OrderService/*',
type: 'thrift',
role: 'consumer',
symbolUid: 'manifest::app/consumer::thrift::OrderService/*',
symbolRef: { filePath: 'group.yaml', name: 'OrderService' },
symbolName: 'OrderService',
confidence: 1,
meta: {},
repo: 'app/consumer',
};
const result = await syncGroup(makeConfig({}), {
extractorOverride: async () => [...extractedProviders, consumer],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(2);
expect(result.crossLinks.map((cl) => cl.to.symbolRef.name).sort()).toEqual([
'OrderService.GetOrder',
'OrderService.PlaceOrder',
]);
expect(new Set(result.crossLinks.map((cl) => cl.to.symbolUid)).size).toBe(2);
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
it('matches weak thrift method consumers to namespace-qualified providers during sync', async () => {
const config = makeConfig({ 'app/provider': 'provider-repo', 'app/consumer': 'consumer-repo' });
const provider: StoredContract = {
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
type: 'thrift',
role: 'provider',
symbolUid: 'uid-provider-place-order',
symbolRef: { filePath: 'idl/order.thrift', name: 'OrderService.PlaceOrder' },
symbolName: 'OrderService.PlaceOrder',
confidence: 0.85,
meta: {},
repo: 'app/provider',
};
const consumer: StoredContract = {
contractId: 'thrift::OrderService/PlaceOrder',
type: 'thrift',
role: 'consumer',
symbolUid: 'uid-consumer-place-order',
symbolRef: { filePath: 'src/BillingWorkflow.java', name: 'orderService.PlaceOrder' },
symbolName: 'orderService.PlaceOrder',
confidence: 0.45,
meta: {},
repo: 'app/consumer',
};
const result = await syncGroup(config, {
extractorOverride: async () => [provider, consumer],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(1);
expect(result.crossLinks[0].matchType).toBe('exact');
expect(result.crossLinks[0].contractId).toBe('thrift::OrderService/PlaceOrder');
expect(result.crossLinks[0].from.repo).toBe('app/consumer');
expect(result.crossLinks[0].to.repo).toBe('app/provider');
expect(result.unmatched).toHaveLength(0);
});
it('keeps exact thrift links to extracted IDL and Java providers for same method', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-exact-'));
fs.mkdirSync(path.join(tmpDir, 'idl'), { recursive: true });
fs.mkdirSync(path.join(tmpDir, 'src', 'main', 'java', 'example'), { recursive: true });
fs.writeFileSync(
path.join(tmpDir, 'idl', 'order.thrift'),
`namespace java billing.v1
service OrderService {
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
}`,
);
fs.writeFileSync(
path.join(tmpDir, 'src', 'main', 'java', 'example', 'IfaceOrderHandler.java'),
`package example;
class IfaceOrderHandler implements OrderService.Iface {
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
return new PlaceOrderResponse();
}
}`,
);
try {
const { ThriftExtractor } =
await import('../../../src/core/group/extractors/thrift-extractor.js');
const extractedProviders = (
await new ThriftExtractor().extract(null, tmpDir, {
id: 'provider-repo',
path: 'app/provider',
repoPath: tmpDir,
storagePath: path.join(tmpDir, '.gitnexus'),
})
)
.filter((c) => c.role === 'provider')
.map(
(c): StoredContract => ({
...c,
repo: 'app/provider',
}),
);
const consumer: StoredContract = {
contractId: 'thrift::OrderService/PlaceOrder',
type: 'thrift',
role: 'consumer',
symbolUid: [
'source-scan::thrift',
'consumer',
'OrderService/PlaceOrder',
'src/BillingWorkflow.java',
'orderService.PlaceOrder',
].join('::'),
symbolRef: { filePath: 'src/BillingWorkflow.java', name: 'orderService.PlaceOrder' },
symbolName: 'orderService.PlaceOrder',
confidence: 0.45,
meta: {},
repo: 'app/consumer',
};
const result = await syncGroup(makeConfig({}), {
extractorOverride: async () => [...extractedProviders, consumer],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(2);
expect(result.crossLinks.map((cl) => cl.to.symbolRef.filePath).sort()).toEqual([
'idl/order.thrift',
'src/main/java/example/IfaceOrderHandler.java',
]);
expect(new Set(result.crossLinks.map((cl) => cl.to.symbolUid)).size).toBe(2);
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
it('extracts thrift contracts during real sync when thrift detection is enabled', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-'));
const storageDir = path.join(tmpDir, '.gitnexus');
fs.mkdirSync(path.join(tmpDir, 'services', 'billing', 'idl'), { recursive: true });
fs.mkdirSync(path.join(tmpDir, 'services', 'billing', 'src'), { recursive: true });
fs.mkdirSync(storageDir, { recursive: true });
fs.writeFileSync(path.join(tmpDir, 'services', 'billing', 'package.json'), '{}');
fs.writeFileSync(
path.join(tmpDir, 'services', 'billing', 'src', 'BillingWorkflow.java'),
'package example; class BillingWorkflow {}',
);
fs.writeFileSync(
path.join(tmpDir, 'services', 'billing', 'idl', 'order.thrift'),
`namespace java billing.v1
service OrderService {
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
}`,
);
const config = makeConfig({ 'services/billing': 'billing-repo' });
config.detect.http = false;
config.detect.thrift = true;
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockResolvedValue(undefined);
try {
const result = await syncGroup(config, {
resolveRepoHandle: async (_name, groupPath) => ({
id: 'billing-repo',
path: groupPath,
repoPath: tmpDir,
storagePath: storageDir,
}),
skipWrite: true,
});
expect(result.missingRepos).toHaveLength(0);
expect(result.contracts).toHaveLength(1);
expect(result.contracts[0]).toMatchObject({
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
type: 'thrift',
role: 'provider',
repo: 'services/billing',
service: 'services/billing',
symbolRef: {
filePath: 'services/billing/idl/order.thrift',
name: 'OrderService.PlaceOrder',
},
});
expect(initSpy).toHaveBeenCalledWith('billing-repo', path.join(storageDir, 'lbug'));
expect(closeSpy).toHaveBeenCalledWith('billing-repo');
} finally {
initSpy.mockRestore();
closeSpy.mockRestore();
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
it('does not extract thrift contracts during real sync when thrift detection is disabled', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-off-'));
const storageDir = path.join(tmpDir, '.gitnexus');
fs.mkdirSync(path.join(tmpDir, 'services', 'billing', 'idl'), { recursive: true });
fs.mkdirSync(storageDir, { recursive: true });
fs.writeFileSync(
path.join(tmpDir, 'services', 'billing', 'idl', 'order.thrift'),
`namespace java billing.v1
service OrderService {
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
}`,
);
const config = makeConfig({ 'services/billing': 'billing-repo' });
config.detect.http = false;
config.detect.thrift = false;
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockResolvedValue(undefined);
try {
const result = await syncGroup(config, {
resolveRepoHandle: async (_name, groupPath) => ({
id: 'billing-repo',
path: groupPath,
repoPath: tmpDir,
storagePath: storageDir,
}),
skipWrite: true,
});
expect(result.missingRepos).toHaveLength(0);
expect(result.contracts).toHaveLength(0);
} finally {
initSpy.mockRestore();
closeSpy.mockRestore();
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
it('does not extract include contracts during real sync when includes detection is disabled', async () => {
// PR #1156 Codex follow-up: ce-code-review T1 — verifies the gate at
// sync.ts:174 honors `detect.includes: false`. Mirrors the existing
// thrift-off pattern at sync.test.ts:545.
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-includes-off-'));
const storageDir = path.join(tmpDir, '.gitnexus');
fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
fs.mkdirSync(storageDir, { recursive: true });
fs.writeFileSync(path.join(tmpDir, 'src', 'view.h'), '#pragma once\nclass View {};');
const config = makeConfig({ 'app/cpp-lib': 'cpp-lib-repo' });
config.detect.http = false;
config.detect.grpc = false;
config.detect.thrift = false;
config.detect.topics = false;
config.detect.includes = false;
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockResolvedValue(undefined);
try {
const result = await syncGroup(config, {
resolveRepoHandle: async (_name, groupPath) => ({
id: 'cpp-lib-repo',
path: groupPath,
repoPath: tmpDir,
storagePath: storageDir,
}),
skipWrite: true,
});
expect(result.missingRepos).toHaveLength(0);
expect(result.contracts.filter((c) => c.type === 'include')).toHaveLength(0);
} finally {
initSpy.mockRestore();
closeSpy.mockRestore();
await cleanupTempDir(tmpDir);
}
});
it('dedupes duplicate wildcard cross-links during sync', async () => {
const config = makeConfig({ 'app/provider': 'provider-repo', 'app/consumer': 'consumer-repo' });
const provider: StoredContract = {
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
type: 'thrift',
role: 'provider',
symbolUid: 'uid-provider-place-order',
symbolRef: { filePath: 'src/provider.ts', name: 'OrderService.PlaceOrder' },
symbolName: 'OrderService.PlaceOrder',
confidence: 0.9,
meta: {},
repo: 'app/provider',
};
const duplicateProvider: StoredContract = {
...provider,
confidence: 0.7,
};
const consumer: StoredContract = {
contractId: 'thrift::OrderService/*',
type: 'thrift',
role: 'consumer',
symbolUid: 'uid-consumer-order-service',
symbolRef: { filePath: 'src/consumer.ts', name: 'OrderClient' },
symbolName: 'OrderClient',
confidence: 0.8,
meta: {},
repo: 'app/consumer',
};
const result = await syncGroup(config, {
extractorOverride: async () => [provider, duplicateProvider, consumer],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(1);
expect(result.crossLinks[0].matchType).toBe('wildcard');
});
it('manifest links referencing unknown repos still produce cross-links via synthetic UIDs', async () => {
const links: GroupManifestLink[] = [
{
from: 'app/known',
to: 'app/dangling', // not present in config.repos
type: 'http',
contract: 'POST::/api/missing',
role: 'consumer',
},
];
const config: GroupConfig = {
version: 1,
name: 'test',
description: '',
repos: { 'app/known': 'known-repo' },
links,
packages: {},
detect: {
http: true,
grpc: false,
thrift: false,
topics: false,
shared_libs: false,
embedding_fallback: false,
workspace_deps: false,
},
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
};
const cap = _captureLogger();
try {
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(1);
expect(result.crossLinks[0].matchType).toBe('manifest');
expect(result.crossLinks[0].to.symbolUid).toBe(
'manifest::app/dangling::http::POST::/api/missing',
);
expect(cap.records().some((r) => String(r.msg ?? '').includes('app/dangling'))).toBe(true);
} finally {
cap.restore();
}
});
it('writes registry to groupDir when skipWrite is false', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-write-'));
try {
const config = makeConfig({});
const result = await syncGroup(config, {
extractorOverride: async () => [],
groupDir: tmpDir,
skipWrite: false,
});
expect(result.contracts).toHaveLength(0);
const registryPath = path.join(tmpDir, 'contracts.json');
expect(fs.existsSync(registryPath)).toBe(true);
const registry = JSON.parse(fs.readFileSync(registryPath, 'utf-8'));
expect(registry.version).toBe(1);
expect(registry.contracts).toHaveLength(0);
} finally {
// syncGroup now writes bridge.lbug + WAL/shadow sidecars when
// skipWrite is false. On Windows, LadybugDB's checkpoint thread can
// briefly outlive closeBridgeDb, holding a Win32 lock on the file.
// cleanupTempDir tolerates the documented Windows-native lock codes
// (EBUSY/EPERM/EACCES/ENOTEMPTY) with bounded retries.
await cleanupTempDir(tmpDir);
}
});
describe('workspace_deps integration', () => {
let tmpDir: string;
function makeWsConfig(repos: Record<string, string>, workspaceDeps: boolean): GroupConfig {
return {
version: 1,
name: 'test',
description: '',
repos,
links: [],
packages: {},
detect: {
http: false,
grpc: false,
thrift: false,
topics: false,
shared_libs: false,
embedding_fallback: false,
workspace_deps: workspaceDeps,
},
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
};
}
function writeFileSync(relPath: string, content: string) {
const absPath = path.join(tmpDir, relPath);
fs.mkdirSync(path.dirname(absPath), { recursive: true });
fs.writeFileSync(absPath, content, 'utf-8');
}
afterEach(() => {
vi.restoreAllMocks();
if (tmpDir) fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('workspace_deps: true discovers Rust crate links through syncGroup', async () => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-ws-'));
writeFileSync(
'crate-a/Cargo.toml',
'[package]\nname = "mathlex"\nversion = "0.1.0"\n\n[dependencies]\n',
);
writeFileSync('crate-a/src/lib.rs', 'pub struct Expression {}\n');
writeFileSync(
'crate-b/Cargo.toml',
'[package]\nname = "thales"\nversion = "0.1.0"\n\n[dependencies]\nmathlex = { workspace = true }\n',
);
writeFileSync('crate-b/src/main.rs', 'use mathlex::Expression;\n');
const mockEntries: RegistryEntry[] = [
{
name: 'mathlex',
path: path.join(tmpDir, 'crate-a'),
storagePath: path.join(tmpDir, 'crate-a', '.gitnexus'),
indexedAt: '',
lastCommit: '',
},
{
name: 'thales',
path: path.join(tmpDir, 'crate-b'),
storagePath: path.join(tmpDir, 'crate-b', '.gitnexus'),
indexedAt: '',
lastCommit: '',
},
];
const repoManager = await import('../../../src/storage/repo-manager.js');
vi.spyOn(repoManager, 'readRegistry').mockResolvedValue(mockEntries);
const config = makeWsConfig({ 'parser/mathlex': 'mathlex', 'engine/thales': 'thales' }, true);
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
const manifestLinks = result.crossLinks.filter((cl) => cl.matchType === 'manifest');
expect(manifestLinks).toHaveLength(1);
expect(manifestLinks[0].contractId).toBe('custom::mathlex::Expression');
expect(manifestLinks[0].from.repo).toBe('engine/thales');
expect(manifestLinks[0].to.repo).toBe('parser/mathlex');
});
it('workspace_deps: false skips workspace extraction entirely', async () => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-ws-off-'));
writeFileSync(
'crate-a/Cargo.toml',
'[package]\nname = "mathlex"\nversion = "0.1.0"\n\n[dependencies]\n',
);
writeFileSync('crate-a/src/lib.rs', 'pub struct Expression {}\n');
writeFileSync(
'crate-b/Cargo.toml',
'[package]\nname = "thales"\nversion = "0.1.0"\n\n[dependencies]\nmathlex = { workspace = true }\n',
);
writeFileSync('crate-b/src/main.rs', 'use mathlex::Expression;\n');
const repoManager = await import('../../../src/storage/repo-manager.js');
vi.spyOn(repoManager, 'readRegistry').mockResolvedValue([]);
const config = makeWsConfig(
{ 'parser/mathlex': 'mathlex', 'engine/thales': 'thales' },
false,
);
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
expect(result.crossLinks).toHaveLength(0);
expect(result.contracts).toHaveLength(0);
});
it('discovered workspace links merge with explicit manifest links', async () => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-ws-merge-'));
writeFileSync(
'crate-a/Cargo.toml',
'[package]\nname = "mathlex"\nversion = "0.1.0"\n\n[dependencies]\n',
);
writeFileSync('crate-a/src/lib.rs', 'pub struct Expression {}\n');
writeFileSync(
'crate-b/Cargo.toml',
'[package]\nname = "thales"\nversion = "0.1.0"\n\n[dependencies]\nmathlex = { workspace = true }\n',
);
writeFileSync('crate-b/src/main.rs', 'use mathlex::Expression;\n');
const mockEntries: RegistryEntry[] = [
{
name: 'mathlex',
path: path.join(tmpDir, 'crate-a'),
storagePath: path.join(tmpDir, 'crate-a', '.gitnexus'),
indexedAt: '',
lastCommit: '',
},
{
name: 'thales',
path: path.join(tmpDir, 'crate-b'),
storagePath: path.join(tmpDir, 'crate-b', '.gitnexus'),
indexedAt: '',
lastCommit: '',
},
];
const repoManager = await import('../../../src/storage/repo-manager.js');
vi.spyOn(repoManager, 'readRegistry').mockResolvedValue(mockEntries);
const explicitLinks: GroupManifestLink[] = [
{
from: 'parser/mathlex',
to: 'engine/thales',
type: 'http',
contract: 'GET::/api/parse',
role: 'provider',
},
];
const config: GroupConfig = {
version: 1,
name: 'test',
description: '',
repos: { 'parser/mathlex': 'mathlex', 'engine/thales': 'thales' },
links: explicitLinks,
packages: {},
detect: {
http: false,
grpc: false,
thrift: false,
topics: false,
shared_libs: false,
embedding_fallback: false,
workspace_deps: true,
},
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
};
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
const manifestLinks = result.crossLinks.filter((cl) => cl.matchType === 'manifest');
expect(manifestLinks).toHaveLength(2);
const contractIds = manifestLinks.map((cl) => cl.contractId);
expect(contractIds).toContain('http::GET::/api/parse');
expect(contractIds).toContain('custom::mathlex::Expression');
});
it('discovers Node workspace links through syncGroup orchestrator', async () => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-ws-node-'));
writeFileSync('shared/package.json', '{"name": "@myorg/shared", "version": "1.0.0"}');
writeFileSync('shared/src/index.ts', 'export class Config {}\n');
writeFileSync(
'app/package.json',
'{"name": "@myorg/app", "version": "1.0.0", "dependencies": {"@myorg/shared": "workspace:*"}}',
);
writeFileSync('app/src/index.ts', "import { Config } from '@myorg/shared';\n");
const mockEntries: RegistryEntry[] = [
{
name: 'shared',
path: path.join(tmpDir, 'shared'),
storagePath: path.join(tmpDir, 'shared', '.gitnexus'),
indexedAt: '',
lastCommit: '',
},
{
name: 'app',
path: path.join(tmpDir, 'app'),
storagePath: path.join(tmpDir, 'app', '.gitnexus'),
indexedAt: '',
lastCommit: '',
},
];
const repoManager = await import('../../../src/storage/repo-manager.js');
vi.spyOn(repoManager, 'readRegistry').mockResolvedValue(mockEntries);
const config = makeWsConfig({ 'pkg/shared': 'shared', 'pkg/app': 'app' }, true);
const result = await syncGroup(config, {
extractorOverride: async () => [],
skipWrite: true,
});
const manifestLinks = result.crossLinks.filter((cl) => cl.matchType === 'manifest');
expect(manifestLinks).toHaveLength(1);
const nodeLink = manifestLinks.find(
(cl) => cl.contractId === 'custom::@myorg/shared::Config',
);
expect(nodeLink).toBeDefined();
});
});
});
describe('stableRepoPoolId', () => {
it('returns lowercase name when no collision', () => {
const entry: RegistryEntry = {
name: 'MyRepo',
path: '/a/MyRepo',
storagePath: '/a/MyRepo/.gitnexus',
indexedAt: '',
lastCommit: '',
};
const all = [entry];
expect(stableRepoPoolId(entry, all)).toBe('myrepo');
});
it('appends hash suffix on name collision with different path', () => {
const entry1: RegistryEntry = {
name: 'repo',
path: '/a/repo',
storagePath: '/a/repo/.gitnexus',
indexedAt: '',
lastCommit: '',
};
const entry2: RegistryEntry = {
name: 'repo',
path: '/b/repo',
storagePath: '/b/repo/.gitnexus',
indexedAt: '',
lastCommit: '',
};
const all = [entry1, entry2];
const id1 = stableRepoPoolId(entry1, all);
const id2 = stableRepoPoolId(entry2, all);
expect(id1).toMatch(/^repo-/);
expect(id2).toMatch(/^repo-/);
expect(id1).not.toBe(id2);
});
});