GitNexus/gitnexus/test/integration/lbug-non-ascii-path.test.ts
Gergő Magyar d4449b4ec8
fix(lbug): resolve non-ASCII paths for KuzuDB on Windows (#1811) (#1817)
* fix(lbug): resolve non-ASCII paths to 8.3 short form on Windows (#1811)

KuzuDB's native C++ layer uses ANSI file APIs (fopen) on Windows.
When the repo path contains CJK or other non-ASCII characters, the
UTF-8 bytes from Node.js are misinterpreted as the system's Active
Code Page (e.g. GBK), producing a garbled path — "Error 3: The
system cannot find the path specified."

Add `toNativeSafePath()` which converts non-ASCII paths to their
Windows 8.3 short-name form (all-ASCII) before passing them to the
native layer. Applied to both the database open path and the COPY
CSV paths. No-ops on non-Windows and on all-ASCII paths.

Closes #1811

* test(lbug): add unit + integration tests for non-ASCII path handling (#1811)

- Unit tests for toNativeSafePath: ASCII passthrough, non-Windows
  no-op, Windows short-path conversion, nonexistent-path fallback
- Integration test: full initLbug + loadGraphToLbug round-trip with
  CJK characters in the storage path — runs on all platforms
- Fix toNativeSafePath to reject cmd.exe output containing '?' chars
  (replacement for unrepresentable Unicode in the console code page)
- Register integration test in vitest lbug-db project and
  cross-platform-tests.ts matrix

* chore(autofix): apply prettier + eslint fixes via /autofix command

* feat(lbug): junction fallback, tmpdir CSV staging, pool-adapter coverage (#1811)

U1+U4: toNativeSafePath now tries 8.3 short path → NTFS junction
fallback → diagnostic warning. Junctions target path.dirname(p) and
reconstruct the leaf. Handles EEXIST races. Registers cleanup on
exit/SIGTERM/SIGINT. Orphan scan on first call removes stale
junctions from prior crashes.

U2: loadGraphToLbug redirects csvDir to os.tmpdir() when
storagePath contains non-ASCII on Windows, avoiding non-ASCII
characters in COPY FROM paths entirely.

U3: All 4 createLbugDatabase call sites in pool-adapter.ts now
wrap dbPath with toNativeSafePath.

* fix(test): fix CI failures from toNativeSafePath addition (#1811)

- Fix lbug-non-ascii-path integration test: use CodeRelation (actual
  relationship table name) instead of CALLS
- Add toNativeSafePath to lbug-config.js mocks in pool-wal-recovery
  and lbug-pool-win-fts-probe tests — pool-adapter now imports it

* fix(lbug): sanitize path before cmd.exe shell expansion (CodeQL)

Reject paths containing cmd.exe metacharacters (" % | & < > ^)
before interpolating into the `for %I` short-path command.
Prevents command injection via crafted path names.

* fix(lbug): address code review findings in non-ASCII path implementation

- U1: Use process.exit(0) on Windows instead of process.kill re-raise
  (SIGTERM forcefully kills on Windows, handlers never fire)
- U2: Pass safePath to openWithLockRetry so sidecar sweep targets the
  path KuzuDB actually opened, not the original non-ASCII path
- U3: Skip junction creation in worker threads (isMainThread guard) to
  prevent junction leaks from pool-adapter workers
- U4: Replace existsSync with lstatSync in orphan scan to avoid 30s
  blocking on unreachable UNC network targets

* chore(autofix): apply prettier + eslint fixes via /autofix command

* fix(lbug): correct SIGTERM exit code and run Prettier (#1811)

- Use exit code 143 (SIGTERM) / 130 (SIGINT) on Windows instead of 0
  so termination is not masked as success
- Run Prettier to fix formatting (CI Gate blocker)

* fix(lbug): eliminate CodeQL command-injection taint in tryShortPath

Pass the path via GITNEXUS_SP environment variable instead of
interpolating it into the cmd.exe command string. The FOR loop
reads %GITNEXUS_SP% from the environment, so the command text is
entirely static — no user-controlled data in the shell command.

Also removes CMD_UNSAFE_RE since the env var approach makes
character-level sanitization unnecessary.

---------

Co-authored-by: Test <test@example.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-25 21:28:12 +01:00

80 lines
3 KiB
TypeScript

/**
* Integration Tests: Non-ASCII path handling (#1811)
*
* Verifies that LadybugDB can open a database and run COPY commands when
* the storage path contains CJK (or other non-ASCII) characters.
*
* The primary failure mode is on Windows, where KuzuDB's native layer
* uses ANSI file APIs and the Active Code Page mangles UTF-8 bytes.
* The fix converts paths to 8.3 short-name form on Windows. On
* Linux/macOS the conversion is a no-op since POSIX APIs handle UTF-8
* natively — but locale misconfiguration or filesystem encoding
* mismatches could still surface, so the test runs on all platforms.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import fs from 'fs/promises';
import path from 'path';
import os from 'os';
import { createMinimalTestGraph } from '../helpers/test-graph.js';
let tmpBase: string;
let storagePath: string;
let dbPath: string;
beforeAll(async () => {
// Create a temp directory with CJK characters in the name.
// This reproduces the user's scenario: repo at C:\Project\中文\code
tmpBase = path.join(os.tmpdir(), `gitnexus-lbug-非ASCII路径-${Date.now()}-${process.pid}`);
storagePath = path.join(tmpBase, '.gitnexus');
dbPath = path.join(storagePath, 'lbug');
await fs.mkdir(dbPath, { recursive: true });
});
afterAll(async () => {
// Close the adapter before cleanup to release native file handles.
try {
const adapter = await import('../../src/core/lbug/lbug-adapter.js');
await adapter.closeLbug();
} catch {
// May not have been opened
}
if (tmpBase) {
// Retry cleanup — LadybugDB on Windows holds handles briefly after close.
for (let attempt = 0; attempt < 5; attempt++) {
try {
await fs.rm(tmpBase, { recursive: true, force: true });
return;
} catch {
if (attempt < 4) await new Promise((r) => setTimeout(r, 200 * (attempt + 1)));
}
}
}
});
describe('LadybugDB with non-ASCII storage path (#1811)', () => {
it('initLbug succeeds with CJK characters in the database path', async () => {
const adapter = await import('../../src/core/lbug/lbug-adapter.js');
await expect(adapter.initLbug(dbPath)).resolves.not.toThrow();
});
it('loadGraphToLbug COPY succeeds with CJK characters in CSV paths', async () => {
const adapter = await import('../../src/core/lbug/lbug-adapter.js');
const graph = createMinimalTestGraph();
await expect(adapter.loadGraphToLbug(graph, tmpBase, storagePath)).resolves.not.toThrow();
});
it('data is queryable after loading through non-ASCII paths', async () => {
const adapter = await import('../../src/core/lbug/lbug-adapter.js');
const files = await adapter.executeQuery('MATCH (n:File) RETURN n.id AS id');
expect(files).toHaveLength(2);
const functions = await adapter.executeQuery('MATCH (n:Function) RETURN n.id AS id');
expect(functions).toHaveLength(2);
const rels = await adapter.executeQuery('MATCH ()-[r:CodeRelation]->() RETURN count(r) AS cnt');
expect(rels[0].cnt).toBe(4);
});
});