mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-07 08:26:11 +00:00
* adds an opt-in auto sync and analysis loop for GitNexus * adds an opt-in auto sync and analysis loop for GitNexus,gitnexus watch [init|start|restart|stop|status] * adds an opt-in auto sync and analysis loop for GitNexus,gitnexus watch [init|start|restart|stop|status] * fix: address PR review cleanup * Prettier code style * merge main * fix(watch): protect local repos and cancel active analysis * fix(watch): harden auto-sync lifecycle and locking - validate watch process identity before lifecycle operations\n- serialize registry, analysis, and LadybugDB access with recoverable locks\n- harden clone paths, symlinks, hooks, quarantine, and worker timeouts\n- install procps in the CLI image for reliable Docker watch control\n- add focused regression coverage for lifecycle, locks, clone, and registry behavior * update agents & claude md * merge main * fix(watch): harden auto-sync lifecycle * fix(watch): normalize SSH repo identity paths * fix(watch): normalize SSH repo identity paths * fix(watch): safely cancel analysis across platforms * fix(auto-sync): close worker and group sync failure paths * fix(auto-sync): drop retired allowStale from group sync allowStale was removed from SyncOptions, which broke typecheck and CI on this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): satisfy prefer-const and Prettier in auto-sync The watch timers are assigned exactly once, so prefer-const rejected the deferred `let` declarations. They are only read from `stop()` and the control poll, both of which run after the assignments, so binding them at creation is safe and drops the now-dead undefined guards. Remaining files are formatting only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): make lock identity absolute and stop three fail-open paths Lock owner identity was rendered by `ps -o lstart=` through localtime and the active locale, so the same live process produced a different string under a different TZ. A mismatch reads as PID reuse, so one daemon could reclaim a mutex another still held. Pin TZ=UTC and LC_ALL=C. The owner record also carried no hostname, so a holder on another machine was judged by this kernel's view of its PID — always "stale" — and its lock stolen whenever GITNEXUS_HOME is a shared volume. Record and compare the hostname, as the index lock already does. Ownership verification threw unconditionally on win32, which is reached once per project per tick, so watch reported `running` and then failed every repo forever. POSIX uid/mode cannot be checked there; skip those two assertions and keep the dangerous-root, symlink, containment and internal-root guards. Also: quarantine sweep now refuses a symlinked root instead of deleting through it; an unreadable state file propagates instead of being rewritten as empty state, which used to erase every repo's analyzed commit and failure count; a failed staging cleanup no longer strands a published lock with no release handle; and the concurrency runner settles every worker before surfacing a failure so cancellation cannot orphan a live analyze fork. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): land the deferred review findings Six findings that were deferred from the review backlog, plus the docs they change. Worker heap: admission allowed `floor(availableMemoryGB / 2)` slots while every fork was handed the whole machine's heap cap, so the budget meant nothing as soon as an operator raised max_concurrency. Divide the cap by the repos actually analyzed in parallel. The default single-project path is unchanged. Registration: the parent registered without a branch, so it always took the primary/flat arm and relabelled a pinned branch entry on the branch-fallback path. Reproduce the worker's own resolveBranchPlacement decision instead. Cancellation: requestCancellation cleared the only timer and settled nothing, so a worker wedged past its safe point left the promise pending forever, wedging activeRun and hanging `watch stop`. Add a 5s grace after which the parent stops waiting and releases the IPC channel's hold on its event loop. The child is still never killed — it may be inside native work. overwrite_local_changes: `checkout --force` rewrites tracked files only, so untracked sources survived and were indexed as if they came from the remote. `git clean -fd -e /.gitnexus` after checkout; no -x/-X, so ignored paths and GitNexus's own storage survive. Quarantine: age alone never bounds a repo that fails every tick, since each partial clone is younger than the retention window. Keep the five newest per repo. Validation: repo_git_timeout is now bounded by the lesser of an hour and the sync interval, which is also the guard for the bare-number-means-seconds slip (`600000` meant ~7 days and cleared the timer ceiling). And the remote URL's final segment is validated at config load rather than failing once per tick inside the sync loop. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): release an errored worker, and stop rejecting dotted repo names Three findings from the latest review pass. The 'error' handler settles immediately rather than waiting out the grace, so cleanup() clears the grace timer that would otherwise have released the child. An errored IPC channel does not mean the worker stopped, so release it on that path too — still no kill. The traversal guard tested the raw path for '..', which also rejected an ordinary name like owner/foo..bar that the repository-name rule accepts. Traversal is a whole segment, so test segments. The heap-cap test left two runs and their real timers pending; it now stubs timers and settles both promises. Registration coverage now pins the branch slot rather than leaving it implicit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): validate namespace segments and pin the stopped process identity Replacing the raw-string `..` test with a per-segment one dropped a guard: a segment like `..\..\outside` is not literally `..`, so it passed, and those segments build the clone path — on Windows the backslashes are separators. Hold every namespace segment to the same charset as the repo name, which keeps a separator out of a segment while still allowing an ordinary `foo..bar`. The final segment keeps its own check so a bad repo name keeps its own message. The stop wait polled liveness by pid alone, so a pid reused mid-wait would have it wait on an unrelated process and then report the watch stopped. Compare the process start time recorded for the owner, which also returns sooner. Registration now omits `branch` for a primary index instead of passing it as undefined, so that call keeps the shape it had before this branch. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): ship auto-sync as the remote daemon, reserve gitnexus watch. Keep analyze --watch for local incremental re-index and stop the top-level watch verb from starting a clone/pull loop. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): reject invalid branch refs and verify status identity (#2493) Reject leading slashes and per-component trailing dots in configured branches, and verify the live watch owner before trusting a stored error status. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): reject ownerIds that can escape the watch directory (#2493) Stop interpolating a tampered ownerId into the stop-request filename; only basename-safe values are treated as owners. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): recognize auto-sync in the watch-process identity check (#2493) Stop/status were still looking for a standalone watch token after the command rename, so a live gitnexus auto-sync start process would be refused as unrelated. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(auto-sync): reject boolean max_concurrency instead of coercing it to 1 (#2493) Number(true) is 1, so a YAML boolean would have passed the integer check and silently meant one worker. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): swallow status errors in the watch finally path (#2493) An uncaught updateStatus rejection in finally became an unhandled rejection. Skip the clone-root symlink test on Windows, where directory symlinks need privileges. Align the group-lock comment with fail-closed registry timeouts. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): catch cancelling status-write failures (#2493) Fire-and-forget updateStatus('cancelling') could become an unhandled rejection, the same class as the finally-path status write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): ignore queued interval ticks after stop (#2493) clearInterval does not cancel a timer callback already queued. Guard runSafely on stopping so shutdown cannot start a new un-cancellable run. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): report stored watch status timestamps (#2493) status should show when the watch last entered a state, not when the CLI queried it. The failure-count test still expects 1 after a new commit resets the streak; rename it so that reset is explicit. Co-authored-by: Cursor <cursoragent@cursor.com> * style(auto-sync): apply prettier to starter status logger (#2493) Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: weiyf <weiyf3634@163.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
730 lines
28 KiB
TypeScript
730 lines
28 KiB
TypeScript
import fs from 'node:fs/promises';
|
|
import os from 'node:os';
|
|
import path from 'node:path';
|
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
|
|
|
import {
|
|
extractRepoNameFromRemoteUrl,
|
|
getAutoSyncMutexPath,
|
|
getAutoSyncStatePath,
|
|
getAutoSyncWatchDir,
|
|
getProjectCommitInfoPath,
|
|
loadAutoSyncConfig,
|
|
parseAutoSyncConfig,
|
|
parseBranchCandidates,
|
|
parseDurationMs,
|
|
quarantineAutoSyncPartial,
|
|
resolveConfiguredCloneRoot,
|
|
loadAutoSyncState,
|
|
resetAutoSyncState,
|
|
saveAutoSyncState,
|
|
shouldAnalyzeCommit,
|
|
validateAutoSyncRemoteUrl,
|
|
validateAutoSyncBranchName,
|
|
writeProjectCommitInfo,
|
|
} from '../../src/core/auto-sync/index.js';
|
|
import { acquireFileLock } from '../../src/storage/file-lock.js';
|
|
|
|
describe('auto-sync', () => {
|
|
let tempDir: string;
|
|
let gitnexusHome: string;
|
|
let oldHome: string | undefined;
|
|
|
|
beforeEach(async () => {
|
|
const base = path.join(process.cwd(), '.tmp-test');
|
|
await fs.mkdir(base, { recursive: true });
|
|
tempDir = await fs.realpath(await fs.mkdtemp(path.join(base, 'gitnexus-auto-sync-')));
|
|
gitnexusHome = path.join(tempDir, '.gitnexus');
|
|
await fs.mkdir(gitnexusHome);
|
|
oldHome = process.env.GITNEXUS_HOME;
|
|
process.env.GITNEXUS_HOME = gitnexusHome;
|
|
});
|
|
|
|
afterEach(async () => {
|
|
if (oldHome === undefined) delete process.env.GITNEXUS_HOME;
|
|
else process.env.GITNEXUS_HOME = oldHome;
|
|
await fs.rm(tempDir, { recursive: true, force: true });
|
|
vi.restoreAllMocks();
|
|
});
|
|
|
|
it('places watch runtime artifacts under the watch directory by default', () => {
|
|
expect(getAutoSyncWatchDir(gitnexusHome)).toBe(path.join(gitnexusHome, 'watch'));
|
|
expect(getAutoSyncMutexPath(gitnexusHome)).toBe(
|
|
path.join(gitnexusHome, 'watch', 'watch.mutex'),
|
|
);
|
|
expect(getAutoSyncStatePath(gitnexusHome)).toBe(
|
|
path.join(gitnexusHome, 'watch', 'auto-sync-state.json'),
|
|
);
|
|
expect(getProjectCommitInfoPath(gitnexusHome)).toBe(
|
|
path.join(gitnexusHome, 'watch', 'project_commit_info.txt'),
|
|
);
|
|
});
|
|
|
|
it('refuses to reset state while the watch mutex is held', async () => {
|
|
const statePath = getAutoSyncStatePath(gitnexusHome);
|
|
const infoPath = getProjectCommitInfoPath(gitnexusHome);
|
|
await fs.mkdir(path.dirname(statePath), { recursive: true });
|
|
await fs.writeFile(statePath, '{"kept":true}\n');
|
|
await fs.writeFile(infoPath, 'kept\n');
|
|
const release = await acquireFileLock(getAutoSyncMutexPath(gitnexusHome));
|
|
|
|
try {
|
|
await expect(resetAutoSyncState(gitnexusHome)).resolves.toBe(false);
|
|
await expect(fs.readFile(statePath, 'utf-8')).resolves.toContain('kept');
|
|
await expect(fs.readFile(infoPath, 'utf-8')).resolves.toBe('kept\n');
|
|
} finally {
|
|
await release();
|
|
}
|
|
});
|
|
|
|
it('resets derived state while holding the watch mutex', async () => {
|
|
const statePath = getAutoSyncStatePath(gitnexusHome);
|
|
const infoPath = getProjectCommitInfoPath(gitnexusHome);
|
|
const mutexPath = getAutoSyncMutexPath(gitnexusHome);
|
|
await fs.mkdir(path.dirname(statePath), { recursive: true });
|
|
await fs.writeFile(statePath, '{}\n');
|
|
await fs.writeFile(infoPath, 'derived\n');
|
|
|
|
await expect(resetAutoSyncState(gitnexusHome)).resolves.toBe(true);
|
|
|
|
await expect(fs.access(statePath)).rejects.toThrow();
|
|
await expect(fs.access(infoPath)).rejects.toThrow();
|
|
await expect(fs.access(mutexPath)).rejects.toThrow();
|
|
});
|
|
|
|
it('loads watch_config.yml from GITNEXUS_HOME and normalizes branch candidates', async () => {
|
|
await fs.writeFile(
|
|
path.join(gitnexusHome, 'watch_config.yml'),
|
|
[
|
|
'sync_interval_minutes: 120',
|
|
'max_concurrency: 3',
|
|
'repo_git_timeout: 12s',
|
|
'analyze_timeout: 45m',
|
|
'analyze_failure_threshold: 2',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' group_name: back_end',
|
|
' overwrite_local_changes: true',
|
|
' branches: [test, master, test]',
|
|
' remote_urls:',
|
|
' - git@gitee.com:qts_server/qts_account.git',
|
|
].join('\n'),
|
|
);
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(true);
|
|
if (!loaded.ok) throw new Error('expected config to load');
|
|
expect(loaded.config.configPath).toBe(path.join(gitnexusHome, 'watch_config.yml'));
|
|
expect(loaded.config.syncIntervalMinutes).toBe(120);
|
|
expect(loaded.config.maxConcurrency).toBe(3);
|
|
expect(loaded.config.repoGitTimeoutMs).toBe(12_000);
|
|
expect(loaded.config.analyzeTimeoutMs).toBe(2_700_000);
|
|
expect(loaded.config.analyzeFailureThreshold).toBe(2);
|
|
expect(loaded.config.projects[0]).toMatchObject({
|
|
localPath: '/tmp/repos',
|
|
groupName: 'back_end',
|
|
overwriteLocalChanges: true,
|
|
branches: ['test', 'master'],
|
|
remoteUrls: ['git@gitee.com:qts_server/qts_account.git'],
|
|
});
|
|
});
|
|
|
|
it('defaults repo_git_timeout and max_concurrency and allows empty group_name', async () => {
|
|
await fs.writeFile(
|
|
path.join(gitnexusHome, 'watch_config.yml'),
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' group_name: ""',
|
|
' branch: master',
|
|
' remote_urls:',
|
|
' - git@github.com:owner/repo.git',
|
|
].join('\n'),
|
|
);
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(true);
|
|
if (!loaded.ok) throw new Error('expected config');
|
|
expect(loaded.config.repoGitTimeoutMs).toBe(10_000);
|
|
expect(loaded.config.analyzeTimeoutMs).toBe(300_000);
|
|
expect(loaded.config.maxConcurrency).toBe(1);
|
|
expect(loaded.config.analyzeFailureThreshold).toBe(3);
|
|
expect(loaded.config.projects[0].groupName).toBeUndefined();
|
|
expect(loaded.config.projects[0].overwriteLocalChanges).toBe(false);
|
|
});
|
|
|
|
it('rejects boolean max_concurrency instead of coercing it to 1', () => {
|
|
expect(() =>
|
|
parseAutoSyncConfig(
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'max_concurrency: true',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' branch: master',
|
|
' remote_urls:',
|
|
' - git@github.com:owner/repo.git',
|
|
].join('\n'),
|
|
'/tmp/watch_config.yml',
|
|
),
|
|
).toThrow('max_concurrency must be a positive integer');
|
|
});
|
|
|
|
it('rejects repo_git_timeout values above the Node timer limit', () => {
|
|
expect(() =>
|
|
parseAutoSyncConfig(
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'repo_git_timeout: 2147483648ms',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' branch: main',
|
|
' remote_urls:',
|
|
' - https://github.com/owner/repo.git',
|
|
].join('\n'),
|
|
'/tmp/watch_config.yml',
|
|
),
|
|
).toThrow('repo_git_timeout must not exceed 2147483647ms');
|
|
});
|
|
|
|
it('rejects a repo_git_timeout that exceeds the interval or an hour', () => {
|
|
const config = (timeout: string) =>
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
`repo_git_timeout: ${timeout}`,
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' branch: main',
|
|
' remote_urls:',
|
|
' - git@github.com:owner/repo.git',
|
|
].join('\n');
|
|
|
|
// A bare number means seconds, so this is ~7 days, not 10 minutes.
|
|
expect(() => parseAutoSyncConfig(config('600000'), '/tmp/watch_config.yml')).toThrow(
|
|
'a bare number is interpreted as seconds',
|
|
);
|
|
expect(() => parseAutoSyncConfig(config('600000ms'), '/tmp/watch_config.yml')).not.toThrow();
|
|
});
|
|
|
|
it('rejects analyze_timeout values above half the sync interval', async () => {
|
|
await fs.writeFile(
|
|
path.join(gitnexusHome, 'watch_config.yml'),
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'analyze_timeout: 6m',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' branch: master',
|
|
' remote_urls:',
|
|
' - git@github.com:owner/repo.git',
|
|
].join('\n'),
|
|
);
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(false);
|
|
if (loaded.ok) throw new Error('expected invalid config');
|
|
expect(loaded.message).toContain(
|
|
'analyze_timeout must not exceed half of sync_interval_minutes (5m)',
|
|
);
|
|
});
|
|
|
|
it('rejects invalid analyze_failure_threshold values', async () => {
|
|
await fs.writeFile(
|
|
path.join(gitnexusHome, 'watch_config.yml'),
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'analyze_failure_threshold: 1',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' branch: master',
|
|
' remote_urls:',
|
|
' - git@github.com:owner/repo.git',
|
|
].join('\n'),
|
|
);
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(false);
|
|
if (loaded.ok) throw new Error('expected invalid config');
|
|
expect(loaded.message).toContain('analyze_failure_threshold must be an integer >= 2');
|
|
});
|
|
|
|
it('reports missing config without throwing', async () => {
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded).toEqual({
|
|
ok: false,
|
|
reason: 'missing',
|
|
message: `[auto-sync] Missing config file: ${path.join(gitnexusHome, 'watch_config.yml')}. Auto sync is skipped.`,
|
|
});
|
|
});
|
|
|
|
it('reports invalid config without throwing', async () => {
|
|
await fs.writeFile(path.join(gitnexusHome, 'watch_config.yml'), 'projects: []\n');
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(false);
|
|
if (loaded.ok) throw new Error('expected invalid config');
|
|
expect(loaded.reason).toBe('invalid');
|
|
expect(loaded.message).toContain('[auto-sync] Invalid watch_config.yml:');
|
|
expect(loaded.message).toContain('sync_interval_minutes must be a positive integer');
|
|
expect(loaded.message).toContain('projects must contain at least one project');
|
|
});
|
|
|
|
it('rejects missing, relative, and traversal local_path values at config load', async () => {
|
|
await fs.writeFile(
|
|
path.join(gitnexusHome, 'watch_config.yml'),
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'projects:',
|
|
' - local_path: ../repos',
|
|
' branch: master',
|
|
' remote_urls:',
|
|
' - git@github.com:team/repo.git',
|
|
].join('\n'),
|
|
);
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(false);
|
|
if (loaded.ok) throw new Error('expected invalid config');
|
|
expect(loaded.message).toContain('local_path must be an absolute path');
|
|
});
|
|
|
|
it('hard-fails unsafe configured clone roots', async () => {
|
|
await expect(resolveConfiguredCloneRoot('/')).rejects.toThrow('unsafe auto-sync clone root');
|
|
await expect(resolveConfiguredCloneRoot(os.homedir())).rejects.toThrow(
|
|
'unsafe auto-sync clone root',
|
|
);
|
|
await expect(
|
|
resolveConfiguredCloneRoot(path.join(await fs.realpath(os.tmpdir()), 'repos')),
|
|
).rejects.toThrow('unsafe auto-sync clone root');
|
|
const root = path.join(tempDir, 'repos');
|
|
await expect(resolveConfiguredCloneRoot(`${root}/../repos`)).rejects.toThrow('normalized');
|
|
});
|
|
|
|
it('rejects GitNexus internal directory descendants as clone roots', async () => {
|
|
for (const internalDir of ['groups', 'indexes', 'quarantine']) {
|
|
const root = path.join(gitnexusHome, internalDir, 'repo-root');
|
|
await fs.mkdir(root, { recursive: true });
|
|
|
|
await expect(resolveConfiguredCloneRoot(root)).rejects.toThrow('GitNexus internal directory');
|
|
}
|
|
});
|
|
|
|
it('allows the default GitNexus repos directory as an auto-sync clone root', async () => {
|
|
const root = path.join(gitnexusHome, 'repos');
|
|
await fs.mkdir(root, { recursive: true });
|
|
|
|
await expect(resolveConfiguredCloneRoot(root)).resolves.toEqual(
|
|
expect.objectContaining({
|
|
root,
|
|
quarantineRoot: path.join(gitnexusHome, 'watch', 'quarantine'),
|
|
}),
|
|
);
|
|
});
|
|
|
|
it.skipIf(process.platform === 'win32')(
|
|
'rejects symlinks in configured clone root paths',
|
|
async () => {
|
|
const realRoot = path.join(tempDir, 'real-root');
|
|
const linkRoot = path.join(tempDir, 'link-root');
|
|
await fs.mkdir(realRoot);
|
|
await fs.symlink(realRoot, linkRoot);
|
|
|
|
await expect(resolveConfiguredCloneRoot(linkRoot)).rejects.toThrow('symlink');
|
|
},
|
|
);
|
|
|
|
it('resolves safe configured clone roots and reports quarantine retention', async () => {
|
|
const root = path.join(tempDir, 'repos');
|
|
await fs.mkdir(root);
|
|
|
|
await expect(resolveConfiguredCloneRoot(root)).resolves.toEqual(
|
|
expect.objectContaining({
|
|
root,
|
|
quarantineRoot: path.join(gitnexusHome, 'watch', 'quarantine'),
|
|
quarantineRetentionDays: 14,
|
|
}),
|
|
);
|
|
});
|
|
|
|
it('removes expired quarantine entries while preserving recent and unrelated files', async () => {
|
|
const root = path.join(tempDir, 'repos');
|
|
const quarantineRoot = path.join(gitnexusHome, 'watch', 'quarantine');
|
|
const expired = path.join(quarantineRoot, 'auto-sync-expired-repo');
|
|
const recent = path.join(quarantineRoot, 'auto-sync-recent-repo');
|
|
const unrelated = path.join(quarantineRoot, 'operator-note.txt');
|
|
await fs.mkdir(expired, { recursive: true });
|
|
await fs.mkdir(recent);
|
|
await fs.writeFile(unrelated, 'keep');
|
|
const old = new Date(Date.now() - 15 * 24 * 60 * 60 * 1_000);
|
|
await fs.utimes(expired, old, old);
|
|
|
|
await resolveConfiguredCloneRoot(root);
|
|
|
|
await expect(fs.access(expired)).rejects.toThrow();
|
|
await expect(fs.access(recent)).resolves.toBeUndefined();
|
|
await expect(fs.readFile(unrelated, 'utf-8')).resolves.toBe('keep');
|
|
});
|
|
|
|
it('keeps only the newest quarantine entries per repository', async () => {
|
|
const root = path.join(tempDir, 'repos');
|
|
const quarantineRoot = path.join(gitnexusHome, 'watch', 'quarantine');
|
|
await fs.mkdir(quarantineRoot, { recursive: true });
|
|
const uuid = '00000000-0000-4000-8000-000000000000';
|
|
const entryName = (stamp: string, repo: string) => `auto-sync-${stamp}-4242-${uuid}-${repo}`;
|
|
// Seven ticks of the same failing repo; age alone would keep them all.
|
|
const busy = ['01', '02', '03', '04', '05', '06', '07'].map((n) =>
|
|
entryName(`2026-08-2${n}T00-00-00-000Z`, 'busy-repo'),
|
|
);
|
|
const quiet = ['01', '02'].map((n) => entryName(`2026-08-2${n}T00-00-00-000Z`, 'quiet-repo'));
|
|
for (const name of [...busy, ...quiet]) {
|
|
await fs.mkdir(path.join(quarantineRoot, name), { recursive: true });
|
|
await fs.writeFile(path.join(quarantineRoot, `${name}.README.txt`), 'note');
|
|
}
|
|
|
|
await resolveConfiguredCloneRoot(root);
|
|
|
|
const survivors = await fs.readdir(quarantineRoot);
|
|
for (const name of busy.slice(-5)) {
|
|
expect(survivors).toContain(name);
|
|
expect(survivors).toContain(`${name}.README.txt`);
|
|
}
|
|
for (const name of busy.slice(0, 2)) {
|
|
expect(survivors).not.toContain(name);
|
|
expect(survivors).not.toContain(`${name}.README.txt`);
|
|
}
|
|
// A repo below the cap is untouched.
|
|
for (const name of quiet) expect(survivors).toContain(name);
|
|
});
|
|
|
|
it('falls back to copy and remove when quarantine crosses filesystems', async () => {
|
|
const target = path.join(tempDir, 'partial-repo');
|
|
const quarantineRoot = path.join(gitnexusHome, 'watch', 'quarantine');
|
|
await fs.mkdir(target);
|
|
await fs.writeFile(path.join(target, 'partial.txt'), 'partial');
|
|
vi.spyOn(fs, 'rename').mockRejectedValueOnce(
|
|
Object.assign(new Error('cross-device link'), { code: 'EXDEV' }),
|
|
);
|
|
|
|
const destination = await quarantineAutoSyncPartial(target, quarantineRoot);
|
|
|
|
await expect(fs.readFile(path.join(destination, 'partial.txt'), 'utf-8')).resolves.toBe(
|
|
'partial',
|
|
);
|
|
await expect(fs.access(target)).rejects.toThrow();
|
|
});
|
|
|
|
it('gives concurrent partial clone quarantines unique destinations', async () => {
|
|
const quarantineRoot = path.join(gitnexusHome, 'watch', 'quarantine');
|
|
const first = path.join(tempDir, 'one', 'partial-repo');
|
|
const second = path.join(tempDir, 'two', 'partial-repo');
|
|
await Promise.all([
|
|
fs.mkdir(first, { recursive: true }),
|
|
fs.mkdir(second, { recursive: true }),
|
|
]);
|
|
|
|
const [firstDestination, secondDestination] = await Promise.all([
|
|
quarantineAutoSyncPartial(first, quarantineRoot),
|
|
quarantineAutoSyncPartial(second, quarantineRoot),
|
|
]);
|
|
|
|
expect(firstDestination).not.toBe(secondDestination);
|
|
await expect(fs.access(firstDestination)).resolves.toBeUndefined();
|
|
await expect(fs.access(secondDestination)).resolves.toBeUndefined();
|
|
await expect(fs.access(first)).rejects.toThrow();
|
|
await expect(fs.access(second)).rejects.toThrow();
|
|
});
|
|
|
|
it('rejects group-writable configured clone roots', async () => {
|
|
if (process.platform === 'win32') return;
|
|
const root = path.join(tempDir, 'group-writable-repos');
|
|
await fs.mkdir(root, { mode: 0o770 });
|
|
await fs.chmod(root, 0o770);
|
|
|
|
await expect(resolveConfiguredCloneRoot(root)).rejects.toThrow('group-writable');
|
|
});
|
|
|
|
it('rejects sticky world-writable configured clone roots', async () => {
|
|
if (process.platform === 'win32') return;
|
|
const root = path.join(tempDir, 'sticky-world-writable-repos');
|
|
await fs.mkdir(root);
|
|
await fs.chmod(root, 0o1777);
|
|
|
|
await expect(resolveConfiguredCloneRoot(root)).rejects.toThrow('world-writable');
|
|
});
|
|
|
|
it('creates missing configured clone roots before watch clone work', async () => {
|
|
const root = path.join(tempDir, 'missing-repos');
|
|
|
|
await expect(resolveConfiguredCloneRoot(root)).resolves.toEqual(
|
|
expect.objectContaining({
|
|
root,
|
|
quarantineRoot: path.join(gitnexusHome, 'watch', 'quarantine'),
|
|
}),
|
|
);
|
|
expect((await fs.stat(root)).isDirectory()).toBe(true);
|
|
});
|
|
|
|
it('parses branch strings and arrays with trimming and de-duplication', () => {
|
|
expect(parseBranchCandidates('test, master, test')).toEqual(['test', 'master']);
|
|
expect(parseBranchCandidates(['develop,master', 'develop'])).toEqual(['develop', 'master']);
|
|
});
|
|
|
|
it('rejects unsafe auto-sync branch names', () => {
|
|
expect(() => validateAutoSyncBranchName('feature/good-branch')).not.toThrow();
|
|
expect(() => validateAutoSyncBranchName('foo./bar')).toThrow('trailing-dot');
|
|
expect(() => validateAutoSyncBranchName('/main')).toThrow('must not start');
|
|
expect(() => validateAutoSyncBranchName('-upload-pack=evil')).toThrow('must not start');
|
|
expect(() => validateAutoSyncBranchName('feature bad')).toThrow('whitespace');
|
|
expect(() => validateAutoSyncBranchName('feature..bad')).toThrow('must not contain ".."');
|
|
expect(() => validateAutoSyncBranchName('bad:ref')).toThrow('not allowed');
|
|
expect(() => validateAutoSyncBranchName('feature.')).toThrow('must not end');
|
|
expect(() => validateAutoSyncBranchName('feature/')).toThrow('must not end');
|
|
expect(() => validateAutoSyncBranchName('feature//branch')).toThrow('consecutive');
|
|
expect(() => validateAutoSyncBranchName('feature@{x')).toThrow('must not contain "@{"');
|
|
expect(() => validateAutoSyncBranchName('.hidden')).toThrow('hidden');
|
|
expect(() => validateAutoSyncBranchName('foo/bar.lock')).toThrow('.lock');
|
|
});
|
|
|
|
it('extracts safe repository names from remote URLs', () => {
|
|
expect(extractRepoNameFromRemoteUrl('git@gitee.com:qts_server/qts_account.git')).toBe(
|
|
'qts_account',
|
|
);
|
|
expect(extractRepoNameFromRemoteUrl('git@gitlab.com:team/subgroup/repo-name.git')).toBe(
|
|
'repo-name',
|
|
);
|
|
});
|
|
|
|
it('rejects unsafe repository names without sanitizing them', () => {
|
|
// Rejected by the URL validator now, so the operator learns at config load
|
|
// rather than once per tick from inside the sync loop.
|
|
expect(() => extractRepoNameFromRemoteUrl('git@github.com:team/repo$name.git')).toThrow(
|
|
'repository name must use only',
|
|
);
|
|
expect(() => extractRepoNameFromRemoteUrl('git@github.com:team/repo\\name.git')).toThrow(
|
|
'repository name must use only',
|
|
);
|
|
expect(() => extractRepoNameFromRemoteUrl('git@github.com:team/..')).toThrow('traversal');
|
|
});
|
|
|
|
it('allows only github, gitlab, and gitee SSH SCP remote URLs', () => {
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/repo')).not.toThrow();
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:im-fan/multica.git')).not.toThrow();
|
|
expect(() => validateAutoSyncRemoteUrl('git@gitlab.com:group/subgroup/repo.git')).not.toThrow();
|
|
expect(() =>
|
|
validateAutoSyncRemoteUrl('git@gitee.com:qts-ops/qts-code-engineering.git'),
|
|
).not.toThrow();
|
|
expect(() => validateAutoSyncRemoteUrl('https://github.com/owner/repo.git')).toThrow(
|
|
'must use',
|
|
);
|
|
expect(() => validateAutoSyncRemoteUrl('ssh://git@github.com/owner/repo.git')).toThrow(
|
|
'must use',
|
|
);
|
|
expect(() => validateAutoSyncRemoteUrl('user@github.com:owner/repo.git')).toThrow('must use');
|
|
expect(() => validateAutoSyncRemoteUrl('git@example.com:owner/repo.git')).toThrow(
|
|
'host must be',
|
|
);
|
|
// Traversal is a whole segment; consecutive dots inside a name are not.
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/foo..bar.git')).not.toThrow();
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/../escape.git')).toThrow(
|
|
'traversal',
|
|
);
|
|
// A separator smuggled into a segment is traversal on Windows even though
|
|
// the segment is not literally `..`.
|
|
expect(() => validateAutoSyncRemoteUrl(String.raw`git@github.com:..\..\outside/repo`)).toThrow(
|
|
'traversal',
|
|
);
|
|
expect(() => validateAutoSyncRemoteUrl(String.raw`git@github.com:owner\..\..\x/repo`)).toThrow(
|
|
'traversal',
|
|
);
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/repo.git?ref=main')).toThrow(
|
|
'must not include query strings or fragments',
|
|
);
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/repo.git#main')).toThrow(
|
|
'must not include query strings or fragments',
|
|
);
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/')).toThrow('path must include');
|
|
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner//repo')).toThrow(
|
|
'path must include',
|
|
);
|
|
});
|
|
|
|
it('parses repo git timeout durations', () => {
|
|
expect(parseDurationMs('10s')).toBe(10_000);
|
|
expect(parseDurationMs('2m')).toBe(120_000);
|
|
expect(parseDurationMs('5000ms')).toBe(5000);
|
|
expect(parseDurationMs('10')).toBe(10_000);
|
|
expect(parseDurationMs(10)).toBe(10_000);
|
|
});
|
|
|
|
it('keeps branch compatibility but rejects branch and branches together', async () => {
|
|
await fs.writeFile(
|
|
path.join(gitnexusHome, 'watch_config.yml'),
|
|
[
|
|
'sync_interval_minutes: 10',
|
|
'projects:',
|
|
' - local_path: /tmp/repos',
|
|
' branch: master',
|
|
' branches: [develop]',
|
|
' remote_urls:',
|
|
' - git@github.com:owner/repo.git',
|
|
].join('\n'),
|
|
);
|
|
|
|
const loaded = await loadAutoSyncConfig();
|
|
|
|
expect(loaded.ok).toBe(false);
|
|
if (loaded.ok) throw new Error('expected invalid config');
|
|
expect(loaded.message).toContain('must not set both branch and branches');
|
|
});
|
|
|
|
it('uses commit ids to skip unchanged analyses and retry failed prior analyses', () => {
|
|
expect(shouldAnalyzeCommit({ currentCommit: 'abc', previousAnalyzedCommit: 'abc' })).toBe(
|
|
false,
|
|
);
|
|
expect(
|
|
shouldAnalyzeCommit({
|
|
currentCommit: 'abc',
|
|
previousAnalyzedCommit: 'abc',
|
|
previousStatus: 'failed',
|
|
}),
|
|
).toBe(true);
|
|
expect(shouldAnalyzeCommit({ currentCommit: 'def', previousAnalyzedCommit: 'abc' })).toBe(true);
|
|
});
|
|
|
|
it('saves state atomically and reloads it', async () => {
|
|
const statePath = path.join(tempDir, 'auto-sync-state.json');
|
|
|
|
await saveAutoSyncState(
|
|
{
|
|
'/tmp/repos/qts_account|master': {
|
|
codeCommitId: 'abc',
|
|
analyzedCommitId: 'abc',
|
|
lastAnalyzeStatus: 'success',
|
|
analyzeConsecutiveFailures: 2,
|
|
lastAnalyzeError: 'old error',
|
|
lastSyncTime: '2026-06-30T00:00:00.000Z',
|
|
},
|
|
},
|
|
statePath,
|
|
);
|
|
|
|
await expect(fs.readdir(tempDir)).resolves.not.toContain(
|
|
expect.stringContaining('auto-sync-state.json.tmp'),
|
|
);
|
|
await expect(loadAutoSyncState(statePath)).resolves.toEqual({
|
|
'/tmp/repos/qts_account|master': {
|
|
codeCommitId: 'abc',
|
|
analyzedCommitId: 'abc',
|
|
lastAnalyzeStatus: 'success',
|
|
analyzeConsecutiveFailures: 2,
|
|
lastAnalyzeError: 'old error',
|
|
lastSyncTime: '2026-06-30T00:00:00.000Z',
|
|
},
|
|
});
|
|
});
|
|
|
|
it('returns empty state and reports corrupt state files', async () => {
|
|
const statePath = path.join(tempDir, 'auto-sync-state.json');
|
|
const stderr = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
|
|
await fs.writeFile(statePath, '{not-json', 'utf-8');
|
|
|
|
await expect(loadAutoSyncState(statePath)).resolves.toEqual({});
|
|
|
|
expect(stderr).toHaveBeenCalledWith(
|
|
`[auto-sync] Ignoring corrupt state file: ${statePath}. State will be rebuilt.\n`,
|
|
);
|
|
});
|
|
|
|
it('propagates an unreadable state file instead of overwriting it with empty state', async () => {
|
|
// A directory stands in for any non-ENOENT read failure (EACCES, EIO).
|
|
// Returning {} here would make the next tick persist empty state over
|
|
// every repo's analyzed commit and failure counters.
|
|
const statePath = path.join(tempDir, 'unreadable-state.json');
|
|
await fs.mkdir(statePath, { recursive: true });
|
|
|
|
await expect(loadAutoSyncState(statePath)).rejects.toThrow();
|
|
});
|
|
|
|
it('drops malformed state entries while preserving valid entries', async () => {
|
|
const statePath = path.join(tempDir, 'auto-sync-state.json');
|
|
await fs.writeFile(
|
|
statePath,
|
|
JSON.stringify({
|
|
'/tmp/repos/valid|main': {
|
|
codeCommitId: 'abc',
|
|
analyzedCommitId: 'abc',
|
|
lastAnalyzeStatus: 'success',
|
|
analyzeConsecutiveFailures: 0,
|
|
lastSyncTime: '2026-06-30T00:00:00.000Z',
|
|
},
|
|
'/tmp/repos/invalid|main': {
|
|
codeCommitId: 123,
|
|
analyzeConsecutiveFailures: -1,
|
|
lastSyncTime: null,
|
|
},
|
|
}),
|
|
);
|
|
|
|
await expect(loadAutoSyncState(statePath)).resolves.toEqual({
|
|
'/tmp/repos/valid|main': {
|
|
codeCommitId: 'abc',
|
|
analyzedCommitId: 'abc',
|
|
lastAnalyzeStatus: 'success',
|
|
analyzeConsecutiveFailures: 0,
|
|
lastSyncTime: '2026-06-30T00:00:00.000Z',
|
|
},
|
|
});
|
|
});
|
|
|
|
it('writes project_commit_info.txt atomically', async () => {
|
|
const infoPath = path.join(tempDir, 'project_commit_info.txt');
|
|
|
|
await writeProjectCommitInfo(
|
|
[
|
|
{
|
|
remoteUrl: 'git@github.com:owner/repo.git',
|
|
localPath: '/tmp/repos/repo',
|
|
branch: 'master',
|
|
codeCommitId: 'abc',
|
|
analyzedCommitId: 'abc',
|
|
status: 'success',
|
|
analyzeConsecutiveFailures: 0,
|
|
analyzeFailureThreshold: 3,
|
|
lastSyncTime: '2026-06-30T00:00:00.000Z',
|
|
},
|
|
{
|
|
remoteUrl: 'git@github.com:owner/bad.git',
|
|
localPath: '/tmp/repos/bad',
|
|
branch: 'master',
|
|
codeCommitId: 'def',
|
|
analyzedCommitId: 'abc',
|
|
status: 'threshold_skipped',
|
|
analyzeConsecutiveFailures: 3,
|
|
analyzeFailureThreshold: 3,
|
|
lastAnalyzeError: 'parser crashed',
|
|
lastSyncTime: '2026-06-30T00:00:00.000Z',
|
|
},
|
|
],
|
|
infoPath,
|
|
);
|
|
|
|
const content = await fs.readFile(infoPath, 'utf-8');
|
|
expect(content).toContain('remote: git@github.com:owner/repo.git');
|
|
expect(content).toContain('code_commit: abc');
|
|
expect(content).toContain('analyze_consecutive_failures: 0');
|
|
expect(content).toContain('analyze_failure_threshold: 3');
|
|
expect(content).toContain('status: threshold_skipped');
|
|
expect(content).toContain('last_analyze_error: parser crashed');
|
|
await expect(fs.readdir(tempDir)).resolves.not.toContain(
|
|
expect.stringContaining('project_commit_info.txt.tmp'),
|
|
);
|
|
});
|
|
});
|