mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-06 02:49:56 +00:00
fix(auto-sync): allow self-hosted remotes via allowed_hosts (#3391)
* fix(auto-sync): allow self-hosted remotes via allowed_hosts Auto-sync skipped any remote whose host was not github.com, gitlab.com, or gitee.com. Operators can now name exact extra DNS hosts in watch_config.yml without opening the default set. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3391) - Dial auto-sync DNS names as absolute hosts and store that URL so a later fetch cannot follow a search domain. - Reject ambiguous numeric host spellings; an exact dotted IPv4 the operator listed stays opt-in. - Document allowed_hosts on the root auto-sync contract. Note: pre-existing failure in unit tests that require dist/cli/index.js and parse-worker.js (this worktree has no build); not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Gergő Magyar
GitHub
parent
274ec3df6e
commit
e137daf63b
12 changed files with 515 additions and 44 deletions
|
|
@ -524,6 +524,8 @@ gitnexus auto-sync reset # Clear failure state; leaves clones and in
|
|||
```yaml
|
||||
sync_interval_minutes: 10
|
||||
analyze_timeout: 5m
|
||||
# Extra hosts beyond github.com, gitlab.com, and gitee.com. Exact names only.
|
||||
# allowed_hosts: [gitlab.mycompany.com]
|
||||
projects:
|
||||
- local_path: /absolute/path/to/clones
|
||||
branches: [main, master]
|
||||
|
|
@ -537,7 +539,7 @@ projects:
|
|||
```
|
||||
|
||||
- `sync_interval_minutes` must be at least `5`; `local_path` must be an absolute path. Clones are stored below it as `host/namespace/repo`.
|
||||
- Remote URLs may use SSH SCP or HTTPS and are limited to GitHub, GitLab, or Gitee. The CLI image includes OpenSSH; mount keys yourself. Invalid `watch_config.yml` skips auto-sync immediately. Auto-sync honors `.gitnexusrc` embeddings (HTTP embeddings env still required in the image).
|
||||
- Remote URLs may use SSH SCP or HTTPS. Hosts are github.com, gitlab.com, and gitee.com unless listed in top-level `allowed_hosts` (exact DNS names, no wildcards). The CLI image includes OpenSSH; mount keys yourself. Invalid `watch_config.yml` skips auto-sync immediately. Auto-sync honors `.gitnexusrc` embeddings (HTTP embeddings env still required in the image).
|
||||
- `branches` are tried in order. The legacy `branch` field is supported, but do not set both.
|
||||
- Set per-project `pdg: true` to keep the full control-flow, control/data-dependence, and taint layers current. Untouched configs that omit `pdg` preserve an existing index's mode and cannot silently strip PDG data. Do not paste `pdg: false` from this example onto an existing watch file unless you intend to drop PDG; an explicit `false` opt-out logs a warning before removing existing PDG data. Auto-sync requests atomic incremental publication where supported, so readers keep using the previous graph until a successful update is ready and a failed staged analysis leaves it intact; unsupported paths retain the analyzer's existing in-place behavior.
|
||||
- Analysis runs in an isolated worker; `analyze_timeout` defaults to half of `sync_interval_minutes`, but may be longer (for example, a `30m` analysis timeout with `5` minute polling) up to Node's timer limit. If a polling tick arrives while analysis is active, it is coalesced into one immediate follow-up run using the newest commit. If the parent times out and leaves that worker running, the follow-up is deferred to the next interval so a leftover lock holder is not counted as a hard analyze failure. Timeout and `auto-sync stop` request safe cancellation; a worker in native work exits after reaching a JS-visible safe point. Until then, auto-sync reports `cancelling` or `stopping` and retains ownership so another auto-sync cannot take over, for up to 5 seconds — after that the parent stops waiting and leaves the worker to exit on its own rather than killing it mid-write. This behavior is the same on macOS and Windows. `overwrite_local_changes` defaults to `false`, so a dirty local clone is skipped rather than overwritten; setting it to `true` also deletes untracked files in the clone, while keeping ignored paths.
|
||||
|
|
|
|||
|
|
@ -354,6 +354,7 @@ max_concurrency: 1
|
|||
repo_git_timeout: 10s
|
||||
analyze_timeout: 5m
|
||||
analyze_failure_threshold: 3
|
||||
# allowed_hosts: [gitlab.mycompany.com]
|
||||
projects:
|
||||
- local_path: /abs/path/to/repos
|
||||
branches: [master, main]
|
||||
|
|
@ -368,7 +369,7 @@ projects:
|
|||
- git@gitee.com:owner/repo.git
|
||||
```
|
||||
|
||||
`sync_interval_minutes` must be an integer of at least `5`. `local_path` must be an absolute path without traversal; each remote is cloned below it as `host/namespace/repo`, preventing same-basename repositories from colliding. `remote_urls` may use SSH SCP form (`git@host:owner/repo.git`) or HTTPS (`https://host/owner/repo.git`) for github.com, gitlab.com, or gitee.com. The published CLI image includes `openssh-client` so SSH remotes can clone; mount keys and `known_hosts` yourself. An invalid `watch_config.yml` skips auto-sync immediately with the validation error. `repo_git_timeout` applies to each repo clone/pull and defaults to `10s`; a bare number such as `10` is interpreted as seconds, while `10000ms`, `10s`, and `1m` keep their explicit units. It must not exceed one hour or `sync_interval_minutes`, whichever is smaller — so a bare `600000` is rejected, because it means 600000 seconds rather than milliseconds. `analyze_timeout` applies to each isolated analysis worker and defaults to half of `sync_interval_minutes`, but it is independent of polling and may be longer, up to Node's timer limit (`2147483647ms`). A `5` minute poll with `analyze_timeout: 30m` is valid. Auto-sync analysis honors the cloned repo's `.gitnexusrc` embeddings settings; the CLI image still needs `GITNEXUS_EMBEDDING_URL` or a bind-mounted embedding stack because npm is stripped. A tick that arrives while the previous loop is active never overlaps it: ticks coalesce into one immediate follow-up run, which pulls and analyzes the newest commit. If the parent times out and leaves that worker running, the follow-up is deferred to the next interval so a leftover lock holder is not counted as a hard analyze failure. Timeout and `auto-sync stop` request safe cancellation; a worker already in native work exits after it returns to a JS-visible safe point. While waiting, auto-sync reports `cancelling` or `stopping` and keeps its ownership files so another auto-sync cannot take over. The parent waits up to 5 seconds for the worker to exit; after that it stops waiting, releases its ownership files, and leaves the worker to finish and exit on its own rather than killing it mid-write. `auto-sync stop` uses this same control path on macOS and Windows.
|
||||
`sync_interval_minutes` must be an integer of at least `5`. `local_path` must be an absolute path without traversal; each remote is cloned below it as `host/namespace/repo`, preventing same-basename repositories from colliding. `remote_urls` may use SSH SCP form (`git@host:owner/repo.git`) or HTTPS (`https://host/owner/repo.git`). Hosts are limited to github.com, gitlab.com, and gitee.com unless listed in top-level `allowed_hosts` (exact DNS names, no wildcards). The published CLI image includes `openssh-client` so SSH remotes can clone; mount keys and `known_hosts` yourself. An invalid `watch_config.yml` skips auto-sync immediately with the validation error. `repo_git_timeout` applies to each repo clone/pull and defaults to `10s`; a bare number such as `10` is interpreted as seconds, while `10000ms`, `10s`, and `1m` keep their explicit units. It must not exceed one hour or `sync_interval_minutes`, whichever is smaller — so a bare `600000` is rejected, because it means 600000 seconds rather than milliseconds. `analyze_timeout` applies to each isolated analysis worker and defaults to half of `sync_interval_minutes`, but it is independent of polling and may be longer, up to Node's timer limit (`2147483647ms`). A `5` minute poll with `analyze_timeout: 30m` is valid. Auto-sync analysis honors the cloned repo's `.gitnexusrc` embeddings settings; the CLI image still needs `GITNEXUS_EMBEDDING_URL` or a bind-mounted embedding stack because npm is stripped. A tick that arrives while the previous loop is active never overlaps it: ticks coalesce into one immediate follow-up run, which pulls and analyzes the newest commit. If the parent times out and leaves that worker running, the follow-up is deferred to the next interval so a leftover lock holder is not counted as a hard analyze failure. Timeout and `auto-sync stop` request safe cancellation; a worker already in native work exits after it returns to a JS-visible safe point. While waiting, auto-sync reports `cancelling` or `stopping` and keeps its ownership files so another auto-sync cannot take over. The parent waits up to 5 seconds for the worker to exit; after that it stops waiting, releases its ownership files, and leaves the worker to finish and exit on its own rather than killing it mid-write. `auto-sync stop` uses this same control path on macOS and Windows.
|
||||
|
||||
`pdg` is configured per project. `pdg: true` builds and maintains the full CFG, control-dependence, reaching-definition, and taint layers on both initial and incremental analyses. Auto-sync requests staged atomic incremental publication where the analyzer supports it: the old graph remains available to readers until the replacement succeeds, and analysis errors are recorded while the old graph remains intact. Unsupported paths retain the analyzer's existing in-place behavior. Untouched configs that omit `pdg` preserve the existing index mode and cannot silently strip PDG data. Do not paste `pdg: false` from this example onto an existing watch file unless you intend to drop PDG. An explicit `pdg: false` disables PDG and emits a warning before a successful rebuild removes those layers. `overwrite_local_changes` defaults to `false`; a dirty local clone is skipped with an error log, while `true` allows branch fallback to replace local changes and additionally discards untracked files and directories in the clone after checkout — ignored paths, including GitNexus's own `.gitnexus/` storage, are preserved. `max_concurrency` defaults to `1` and is capped at runtime by `floor(availableMemoryGB / 2)` with a minimum of `1`; the effective value is printed at the start of each loop. Each analysis worker's heap cap is the machine-wide cap divided by the number of repositories analyzed in parallel, so concurrent workers share one memory budget instead of each claiming the whole machine. `analyze_failure_threshold` defaults to `3`, must be at least `2`, and pauses repeated failures only for the same repo branch, commit, and requested PDG mode; a new commit, a PDG mode change, or `gitnexus auto-sync reset` clears the block and allows analysis again. Repositories are registered and added to groups by their full remote identity (`host/namespace/repo`), so repositories with the same basename remain distinct. Use `branches` to try branches in order; legacy `branch` remains supported, but the two fields cannot be set together. If all branches are unavailable or time out, watch logs an error, records the repo status, and skips that repo for the loop. Leave `group_name` empty or omit it to skip group add/sync for that project; otherwise create the group first with `gitnexus group create <name>`. `$GITNEXUS_HOME/watch/project_commit_info.txt` is for inspection only; GitNexus stores machine state separately in `$GITNEXUS_HOME/watch/auto-sync-state.json`.
|
||||
|
||||
|
|
|
|||
|
|
@ -114,6 +114,9 @@ function defaultSyncConfig(localPath: string): string {
|
|||
'repo_git_timeout: 10s',
|
||||
'analyze_timeout: 5m',
|
||||
'analyze_failure_threshold: 3',
|
||||
'# Extra SSH/HTTPS hosts beyond github.com, gitlab.com, and gitee.com.',
|
||||
'# Exact DNS names only; wildcards are rejected.',
|
||||
'# allowed_hosts: [gitlab.mycompany.com]',
|
||||
'projects:',
|
||||
` - local_path: ${localPath}`,
|
||||
' branches: [master, main]',
|
||||
|
|
@ -124,7 +127,7 @@ function defaultSyncConfig(localPath: string): string {
|
|||
' overwrite_local_changes: false',
|
||||
' remote_urls:',
|
||||
' - git@github.com:owner/repo.git',
|
||||
' # HTTPS remotes on github.com, gitlab.com, or gitee.com are also allowed',
|
||||
' # HTTPS remotes are also allowed. Other hosts need top-level allowed_hosts.',
|
||||
' # - https://github.com/owner/public-repo.git',
|
||||
'',
|
||||
].join('\n');
|
||||
|
|
|
|||
|
|
@ -211,7 +211,7 @@ export const en = {
|
|||
'help.command.autoSync.description':
|
||||
'Control scheduled repository clone/pull and analysis from GITNEXUS_HOME/watch_config.yml',
|
||||
'help.autoSync.details':
|
||||
'\nActions: init, start (default), restart, stop, status, reset\nConfiguration: GITNEXUS_HOME/watch_config.yml\nRuntime files: GITNEXUS_HOME/watch/watch.pid, watch.mutex, watch.owner.json, watch.status.json, auto-sync-state.json\nRecovery: mutexes with verified dead owners are reclaimed automatically; invalid or legacy mutexes fail closed and require manual removal after confirming no watch process is running.\nWrites: GITNEXUS_HOME/watch/project_commit_info.txt\nRemote URLs: SSH or HTTPS URLs on github.com, gitlab.com, and gitee.com are allowed. Invalid watch_config.yml skips auto-sync immediately.\nRuns once immediately, then repeats on sync_interval_minutes.',
|
||||
'\nActions: init, start (default), restart, stop, status, reset\nConfiguration: GITNEXUS_HOME/watch_config.yml\nRuntime files: GITNEXUS_HOME/watch/watch.pid, watch.mutex, watch.owner.json, watch.status.json, auto-sync-state.json\nRecovery: mutexes with verified dead owners are reclaimed automatically; invalid or legacy mutexes fail closed and require manual removal after confirming no watch process is running.\nWrites: GITNEXUS_HOME/watch/project_commit_info.txt\nRemote URLs: SSH or HTTPS URLs on github.com, gitlab.com, and gitee.com are allowed. Other hosts need a top-level allowed_hosts list of exact DNS names. Invalid watch_config.yml skips auto-sync immediately.\nRuns once immediately, then repeats on sync_interval_minutes.',
|
||||
'help.command.watch.description':
|
||||
'Ambiguous: use `analyze --watch` for local files, or `auto-sync` for scheduled remotes',
|
||||
'help.watch.details':
|
||||
|
|
|
|||
|
|
@ -199,7 +199,7 @@ export const zhCN = {
|
|||
'help.command.autoSync.description':
|
||||
'控制基于 GITNEXUS_HOME/watch_config.yml 的定时 clone/pull 和分析',
|
||||
'help.autoSync.details':
|
||||
'\n操作：init、start（默认）、restart、stop、status、reset\n配置：GITNEXUS_HOME/watch_config.yml\n运行时文件：GITNEXUS_HOME/watch/watch.pid、watch.mutex、watch.owner.json、watch.status.json、auto-sync-state.json\n恢复：已验证 owner 退出的 mutex 会自动回收；无效或旧版 mutex 会安全拒绝，确认没有 watch 进程运行后再手动删除。\n写入：GITNEXUS_HOME/watch/project_commit_info.txt\n远程地址：允许 github.com、gitlab.com 和 gitee.com 上的 SSH 或 HTTPS 地址。无效的 watch_config.yml 会立即跳过 auto-sync。\n启动后立即运行一次，之后按 sync_interval_minutes 重复。',
|
||||
'\n操作：init、start（默认）、restart、stop、status、reset\n配置：GITNEXUS_HOME/watch_config.yml\n运行时文件：GITNEXUS_HOME/watch/watch.pid、watch.mutex、watch.owner.json、watch.status.json、auto-sync-state.json\n恢复：已验证 owner 退出的 mutex 会自动回收；无效或旧版 mutex 会安全拒绝，确认没有 watch 进程运行后再手动删除。\n写入：GITNEXUS_HOME/watch/project_commit_info.txt\n远程地址：允许 github.com、gitlab.com 和 gitee.com 上的 SSH 或 HTTPS 地址。其他主机需在顶层 allowed_hosts 中列出精确 DNS 名称。无效的 watch_config.yml 会立即跳过 auto-sync。\n启动后立即运行一次，之后按 sync_interval_minutes 重复。',
|
||||
'help.command.watch.description':
|
||||
'含义不明确：本地文件请用 `analyze --watch`，定时远程同步请用 `auto-sync`',
|
||||
'help.watch.details':
|
||||
|
|
|
|||
|
|
@ -16,7 +16,70 @@ const DEFAULT_REPO_GIT_TIMEOUT_MS = 10_000;
|
|||
const DEFAULT_MAX_CONCURRENCY = 1;
|
||||
export const DEFAULT_ANALYZE_FAILURE_THRESHOLD = 3;
|
||||
const MIN_ANALYZE_FAILURE_THRESHOLD = 2;
|
||||
const ALLOWED_REMOTE_HOSTS = new Set(['github.com', 'gitlab.com', 'gitee.com']);
|
||||
const BUILTIN_REMOTE_HOSTS = new Set(['github.com', 'gitlab.com', 'gitee.com']);
|
||||
// Exact DNS names only. The host is a directory under local_path, so wildcards,
|
||||
// ports, and path characters stay out. A label is 1-63 chars and cannot start
|
||||
// or end with a hyphen; the whole name is at most 253 characters.
|
||||
// One trailing dot is stripped before this runs: it marks an absolute lookup,
|
||||
// it is not a different host.
|
||||
const AUTO_SYNC_HOST_PATTERN =
|
||||
/^(?=.{1,253}$)[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?)*$/;
|
||||
|
||||
/** Lowercase, and drop one trailing root dot so `git.` and `git` are one host. */
|
||||
function canonicalAutoSyncHost(host: string): string {
|
||||
return host.trim().toLowerCase().replace(/\.$/, '');
|
||||
}
|
||||
|
||||
function isStrictDottedQuad(host: string): boolean {
|
||||
const labels = host.split('.');
|
||||
if (labels.length !== 4) return false;
|
||||
return labels.every((label) => /^(0|[1-9]\d{0,2})$/.test(label) && Number(label) <= 255);
|
||||
}
|
||||
|
||||
/**
|
||||
* Spellings glibc inet_aton dials as a different address than the token
|
||||
* (`192.168.1` → 192.168.0.1, `0x7f.0.0.1` / `2130706433` → 127.0.0.1).
|
||||
* A strict four-octet address is the address written; listing it is opt-in.
|
||||
*/
|
||||
function isAmbiguousNumericHost(host: string): boolean {
|
||||
const labels = host.split('.');
|
||||
if (labels.some((label) => /^0x[0-9a-f]+$/i.test(label) || /^0\d/.test(label))) return true;
|
||||
return labels.every((label) => /^\d+$/.test(label)) && !isStrictDottedQuad(host);
|
||||
}
|
||||
|
||||
function autoSyncHostProblem(host: string): 'shape' | 'numeric' | null {
|
||||
if (!AUTO_SYNC_HOST_PATTERN.test(host)) return 'shape';
|
||||
if (isAmbiguousNumericHost(host)) return 'numeric';
|
||||
return null;
|
||||
}
|
||||
|
||||
function autoSyncHostProblemMessage(problem: 'shape' | 'numeric'): string {
|
||||
return problem === 'numeric'
|
||||
? 'must not use an ambiguous numeric spelling'
|
||||
: 'must be a DNS hostname';
|
||||
}
|
||||
|
||||
function rewriteAutoSyncRemoteHost(remoteUrl: string, mapHost: (host: string) => string): string {
|
||||
const ssh = /^(git@)([^:\s/]+)(:[^\s]+)$/.exec(remoteUrl);
|
||||
if (ssh) return `${ssh[1]}${mapHost(ssh[2])}${ssh[3]}`;
|
||||
const https = /^(https:\/\/)([^/\s]+)(\/[^\s]+)$/.exec(remoteUrl);
|
||||
if (https) return `${https[1]}${mapHost(https[2])}${https[3]}`;
|
||||
return remoteUrl;
|
||||
}
|
||||
|
||||
/**
|
||||
* Name git should resolve. A trailing dot forces an absolute lookup, so a
|
||||
* search list cannot answer `git` as `git.<domain>` or retry an FQDN under
|
||||
* that domain after NXDOMAIN. A strict IPv4 literal stays undotted: `10.0.0.1.`
|
||||
* is a DNS name, not that address.
|
||||
*/
|
||||
export function absoluteAutoSyncRemoteUrl(remoteUrl: string): string {
|
||||
return rewriteAutoSyncRemoteHost(remoteUrl.trim(), (host) => {
|
||||
const canonical = canonicalAutoSyncHost(host);
|
||||
if (isStrictDottedQuad(canonical)) return canonical;
|
||||
return `${canonical}.`;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* A single clone/pull must fit inside one sync interval and inside an hour.
|
||||
|
|
@ -51,6 +114,11 @@ export interface AutoSyncConfig {
|
|||
analyzeTimeoutMs: number;
|
||||
maxConcurrency: number;
|
||||
analyzeFailureThreshold: number;
|
||||
/**
|
||||
* Extra remote hosts from top-level `allowed_hosts`, already lowercased.
|
||||
* Omitted on hand-built configs; treated as none.
|
||||
*/
|
||||
allowedHosts?: readonly string[];
|
||||
projects: AutoSyncProjectConfig[];
|
||||
}
|
||||
|
||||
|
|
@ -188,6 +256,8 @@ export function parseAutoSyncConfig(content: string, configPath: string): AutoSy
|
|||
errors.push(`analyze_failure_threshold must be an integer >= ${MIN_ANALYZE_FAILURE_THRESHOLD}`);
|
||||
}
|
||||
|
||||
const allowedHosts = parseAllowedAutoSyncHosts(raw.allowed_hosts, errors);
|
||||
|
||||
const rawProjects = raw.projects;
|
||||
if (!Array.isArray(rawProjects) || rawProjects.length === 0) {
|
||||
errors.push('projects must contain at least one project');
|
||||
|
|
@ -221,7 +291,7 @@ export function parseAutoSyncConfig(content: string, configPath: string): AutoSy
|
|||
}
|
||||
for (let urlIndex = 0; urlIndex < remoteUrls.length; urlIndex += 1) {
|
||||
try {
|
||||
validateAutoSyncRemoteUrl(remoteUrls[urlIndex]);
|
||||
validateAutoSyncRemoteUrl(remoteUrls[urlIndex], allowedHosts);
|
||||
} catch (err: unknown) {
|
||||
errors.push(`projects[${index}].remote_urls[${urlIndex}] ${(err as Error).message}`);
|
||||
}
|
||||
|
|
@ -281,11 +351,42 @@ export function parseAutoSyncConfig(content: string, configPath: string): AutoSy
|
|||
analyzeTimeoutMs,
|
||||
maxConcurrency,
|
||||
analyzeFailureThreshold,
|
||||
allowedHosts,
|
||||
projects,
|
||||
};
|
||||
}
|
||||
|
||||
export function parseAutoSyncRemoteIdentity(remoteUrl: string): { host: string; repoPath: string } {
|
||||
function parseAllowedAutoSyncHosts(value: unknown, errors: string[]): string[] {
|
||||
if (value === undefined) return [];
|
||||
if (!Array.isArray(value)) {
|
||||
errors.push('allowed_hosts must be a list of DNS hostnames');
|
||||
return [];
|
||||
}
|
||||
const hosts: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
const entry = value[index];
|
||||
if (typeof entry !== 'string') {
|
||||
errors.push(`allowed_hosts[${index}] must be a DNS hostname`);
|
||||
continue;
|
||||
}
|
||||
const host = canonicalAutoSyncHost(entry);
|
||||
const problem = autoSyncHostProblem(host);
|
||||
if (problem) {
|
||||
errors.push(`allowed_hosts[${index}] ${autoSyncHostProblemMessage(problem)}`);
|
||||
continue;
|
||||
}
|
||||
if (seen.has(host)) continue;
|
||||
seen.add(host);
|
||||
hosts.push(host);
|
||||
}
|
||||
return hosts;
|
||||
}
|
||||
|
||||
export function parseAutoSyncRemoteIdentity(
|
||||
remoteUrl: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): { host: string; repoPath: string } {
|
||||
const trimmed = remoteUrl.trim();
|
||||
if (trimmed.includes('?') || trimmed.includes('#')) {
|
||||
throw new Error('must not include query strings or fragments');
|
||||
|
|
@ -304,26 +405,53 @@ export function parseAutoSyncRemoteIdentity(remoteUrl: string): { host: string;
|
|||
throw new Error('must not include userinfo or a port');
|
||||
}
|
||||
} else {
|
||||
throw new Error('must use an SSH or HTTPS URL on github.com, gitlab.com, or gitee.com');
|
||||
throw new Error(
|
||||
'must use an SSH or HTTPS URL (git@host:owner/repo or https://host/owner/repo)',
|
||||
);
|
||||
}
|
||||
host = host.toLowerCase();
|
||||
assertAutoSyncRemotePath(host, repoPath);
|
||||
host = canonicalAutoSyncHost(host);
|
||||
assertAutoSyncRemotePath(host, repoPath, allowedHosts);
|
||||
return { host, repoPath };
|
||||
}
|
||||
|
||||
/** Canonical `host/owner/repo` key. Strips one trailing `.git`. Throws on an invalid remote. */
|
||||
export function getAutoSyncRepoIdentity(remoteUrl: string): string {
|
||||
const { host, repoPath } = parseAutoSyncRemoteIdentity(remoteUrl);
|
||||
export function getAutoSyncRepoIdentity(
|
||||
remoteUrl: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): string {
|
||||
const { host, repoPath } = parseAutoSyncRemoteIdentity(remoteUrl, allowedHosts);
|
||||
return `${host}/${repoPath.replace(/\.git$/i, '')}`;
|
||||
}
|
||||
|
||||
export function validateAutoSyncRemoteUrl(remoteUrl: string): void {
|
||||
parseAutoSyncRemoteIdentity(remoteUrl);
|
||||
export function validateAutoSyncRemoteUrl(
|
||||
remoteUrl: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): void {
|
||||
parseAutoSyncRemoteIdentity(remoteUrl, allowedHosts);
|
||||
}
|
||||
|
||||
function assertAutoSyncRemotePath(host: string, repoPath: string): void {
|
||||
if (!ALLOWED_REMOTE_HOSTS.has(host)) {
|
||||
throw new Error('host must be one of github.com, gitlab.com, or gitee.com');
|
||||
function isPermittedAutoSyncHost(host: string, allowedHosts?: readonly string[]): boolean {
|
||||
if (BUILTIN_REMOTE_HOSTS.has(host)) return true;
|
||||
if (!allowedHosts) return false;
|
||||
for (const entry of allowedHosts) {
|
||||
if (canonicalAutoSyncHost(entry) === host) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function assertAutoSyncRemotePath(
|
||||
host: string,
|
||||
repoPath: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): void {
|
||||
const problem = autoSyncHostProblem(host);
|
||||
if (problem) {
|
||||
throw new Error(`host ${autoSyncHostProblemMessage(problem)}`);
|
||||
}
|
||||
if (!isPermittedAutoSyncHost(host, allowedHosts)) {
|
||||
throw new Error(
|
||||
'host must be one of github.com, gitlab.com, or gitee.com, or listed in top-level allowed_hosts',
|
||||
);
|
||||
}
|
||||
const pathParts = repoPath.split('/');
|
||||
// Every segment becomes a directory component: the namespace segments build
|
||||
|
|
|
|||
|
|
@ -1,7 +1,10 @@
|
|||
import { extractRepoName } from '../../server/git-clone.js';
|
||||
import { validateAutoSyncRemoteUrl } from './config.js';
|
||||
|
||||
export function extractRepoNameFromRemoteUrl(remoteUrl: string): string {
|
||||
validateAutoSyncRemoteUrl(remoteUrl);
|
||||
export function extractRepoNameFromRemoteUrl(
|
||||
remoteUrl: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): string {
|
||||
validateAutoSyncRemoteUrl(remoteUrl, allowedHosts);
|
||||
return extractRepoName(remoteUrl);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -162,6 +162,7 @@ export async function runAutoSyncOnce(
|
|||
repoName,
|
||||
targetDir,
|
||||
timeoutMs: config.repoGitTimeoutMs,
|
||||
allowedHosts: config.allowedHosts,
|
||||
deps,
|
||||
logger,
|
||||
});
|
||||
|
|
@ -361,7 +362,7 @@ export async function runAutoSyncOnce(
|
|||
repoResult.branch,
|
||||
);
|
||||
await deps.registerRepo(repoResult.targetDir, meta, {
|
||||
name: getAutoSyncRepoIdentity(repoResult.remoteUrl),
|
||||
name: getAutoSyncRepoIdentity(repoResult.remoteUrl, config.allowedHosts),
|
||||
// Omitted rather than passed as undefined, so a primary index is
|
||||
// registered with the same option shape it had before this branch.
|
||||
...(placement.branch ? { branch: placement.branch } : {}),
|
||||
|
|
@ -412,8 +413,8 @@ export async function runAutoSyncOnce(
|
|||
try {
|
||||
membershipAdded = await deps.addRepoToGroup(
|
||||
repoResult.project,
|
||||
getAutoSyncRepoIdentity(repoResult.remoteUrl),
|
||||
getAutoSyncRepoIdentity(repoResult.remoteUrl),
|
||||
getAutoSyncRepoIdentity(repoResult.remoteUrl, config.allowedHosts),
|
||||
getAutoSyncRepoIdentity(repoResult.remoteUrl, config.allowedHosts),
|
||||
);
|
||||
groupMembershipOk = true;
|
||||
} catch (err: unknown) {
|
||||
|
|
@ -490,9 +491,10 @@ export function getConfiguredRepoPath(
|
|||
project: Pick<AutoSyncProjectConfig, 'localPath'>,
|
||||
repoName: string,
|
||||
remoteUrl?: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): string {
|
||||
if (!remoteUrl) return path.resolve(project.localPath, repoName);
|
||||
const identity = getAutoSyncRepoIdentity(remoteUrl);
|
||||
const identity = getAutoSyncRepoIdentity(remoteUrl, allowedHosts);
|
||||
return path.resolve(project.localPath, ...identity.split('/').slice(0, -1), repoName);
|
||||
}
|
||||
|
||||
|
|
@ -550,8 +552,13 @@ async function buildWorkItems(
|
|||
}
|
||||
for (const remoteUrl of project.remoteUrls) {
|
||||
try {
|
||||
const repoName = extractRepoNameFromRemoteUrl(remoteUrl);
|
||||
const targetDir = getConfiguredRepoPath({ localPath: cloneRoot.root }, repoName, remoteUrl);
|
||||
const repoName = extractRepoNameFromRemoteUrl(remoteUrl, config.allowedHosts);
|
||||
const targetDir = getConfiguredRepoPath(
|
||||
{ localPath: cloneRoot.root },
|
||||
repoName,
|
||||
remoteUrl,
|
||||
config.allowedHosts,
|
||||
);
|
||||
const previous = targetOwners.get(targetDir);
|
||||
if (previous !== undefined) {
|
||||
throw new Error(
|
||||
|
|
@ -615,6 +622,7 @@ async function syncFirstAvailableBranch(input: {
|
|||
repoName: string;
|
||||
targetDir: string;
|
||||
timeoutMs: number;
|
||||
allowedHosts?: readonly string[];
|
||||
deps: AutoSyncRunDeps;
|
||||
logger: AutoSyncLogger;
|
||||
}): Promise<
|
||||
|
|
@ -630,6 +638,9 @@ async function syncFirstAvailableBranch(input: {
|
|||
expectedRepoName: input.repoName,
|
||||
quarantineRoot: input.item.cloneRoot!.quarantineRoot,
|
||||
allowAutoSyncSsh: true,
|
||||
...(input.allowedHosts && input.allowedHosts.length > 0
|
||||
? { autoSyncAllowedHosts: input.allowedHosts }
|
||||
: {}),
|
||||
timeoutMs: input.timeoutMs,
|
||||
branch,
|
||||
overwriteLocalChanges: input.item.project.overwriteLocalChanges,
|
||||
|
|
|
|||
|
|
@ -18,7 +18,11 @@ import {
|
|||
assertDirectoryOwnerAndPermissions,
|
||||
quarantineAutoSyncPartial,
|
||||
} from '../core/auto-sync/path-security.js';
|
||||
import { getAutoSyncRepoIdentity, validateAutoSyncRemoteUrl } from '../core/auto-sync/config.js';
|
||||
import {
|
||||
absoluteAutoSyncRemoteUrl,
|
||||
getAutoSyncRepoIdentity,
|
||||
validateAutoSyncRemoteUrl,
|
||||
} from '../core/auto-sync/config.js';
|
||||
|
||||
export { validateGitUrl };
|
||||
|
||||
|
|
@ -179,7 +183,18 @@ export interface CloneOrPullOptions {
|
|||
allowedCloneRoot?: string;
|
||||
expectedRepoName?: string;
|
||||
quarantineRoot?: string;
|
||||
/**
|
||||
* Auto-sync clone/pull. DNS names are stored with one trailing dot so a
|
||||
* resolver search list cannot replace the allowlisted host. A strict IPv4
|
||||
* literal is stored as written.
|
||||
*/
|
||||
allowAutoSyncSsh?: boolean;
|
||||
/**
|
||||
* Extra hosts from watch_config.yml `allowed_hosts`. Honored only together
|
||||
* with `allowAutoSyncSsh`; built-in github.com, gitlab.com, and gitee.com
|
||||
* stay allowed either way.
|
||||
*/
|
||||
autoSyncAllowedHosts?: readonly string[];
|
||||
timeoutMs?: number;
|
||||
branch?: string;
|
||||
overwriteLocalChanges?: boolean;
|
||||
|
|
@ -319,9 +334,15 @@ export function normalizeGitUrlForCompare(url: string): string {
|
|||
}
|
||||
|
||||
/** Same allowlisted repo across SSH and HTTPS, ignoring a trailing `.git`. */
|
||||
function sameAllowlistedAutoSyncRepo(left: string, right: string): boolean {
|
||||
function sameAllowlistedAutoSyncRepo(
|
||||
left: string,
|
||||
right: string,
|
||||
allowedHosts?: readonly string[],
|
||||
): boolean {
|
||||
try {
|
||||
return getAutoSyncRepoIdentity(left) === getAutoSyncRepoIdentity(right);
|
||||
return (
|
||||
getAutoSyncRepoIdentity(left, allowedHosts) === getAutoSyncRepoIdentity(right, allowedHosts)
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
|
|
@ -552,8 +573,11 @@ export async function cloneOrPull(
|
|||
// Always validate the requested URL — the prior shape only ran this in
|
||||
// the code path where the repo was cloned. Now it runs unconditionally,
|
||||
// preventing SSRF / blocked-host bypasses even when targetDir already exists.
|
||||
if (options?.allowAutoSyncSsh) validateAutoSyncRemoteUrl(url);
|
||||
if (options?.allowAutoSyncSsh) validateAutoSyncRemoteUrl(url, options.autoSyncAllowedHosts);
|
||||
else validateGitUrl(url);
|
||||
// Fetch uses remote.origin.url, so the absolute name has to be what is
|
||||
// stored, not only the clone argv. Non-auto-sync clones keep the given URL.
|
||||
const dialUrl = options?.allowAutoSyncSsh ? absoluteAutoSyncRemoteUrl(url) : url;
|
||||
await fs.mkdir(cloneRoot, { recursive: true });
|
||||
if (options?.allowedCloneRoot) {
|
||||
await assertDirectoryOwnerAndPermissions(cloneRoot);
|
||||
|
|
@ -596,20 +620,20 @@ export async function cloneOrPull(
|
|||
let originForCompare = originUrl;
|
||||
if (
|
||||
originUrl &&
|
||||
normalizeGitUrlForCompare(originUrl) !== normalizeGitUrlForCompare(url) &&
|
||||
sameAllowlistedAutoSyncRepo(originUrl, url)
|
||||
normalizeGitUrlForCompare(originUrl) !== normalizeGitUrlForCompare(dialUrl) &&
|
||||
sameAllowlistedAutoSyncRepo(originUrl, url, options?.autoSyncAllowedHosts)
|
||||
) {
|
||||
await runGit(['remote', 'set-url', 'origin', url], safeTarget, {
|
||||
await runGit(['remote', 'set-url', 'origin', dialUrl], safeTarget, {
|
||||
timeoutMs: options?.timeoutMs,
|
||||
});
|
||||
originForCompare = url;
|
||||
originForCompare = dialUrl;
|
||||
}
|
||||
// Confirm the existing clone is actually the same repository the caller
|
||||
// requested. Without this check, a pull would silently succeed against
|
||||
// whatever remote the dir was originally cloned from.
|
||||
await assertRemoteMatchesRequestedUrl(
|
||||
safeTarget,
|
||||
url,
|
||||
dialUrl,
|
||||
options?.timeoutMs,
|
||||
originForCompare ?? undefined,
|
||||
);
|
||||
|
|
@ -617,7 +641,7 @@ export async function cloneOrPull(
|
|||
const runGitImpl = options?.runGitForTest ?? runGit;
|
||||
const gitOpts = {
|
||||
token: options?.token,
|
||||
url,
|
||||
url: dialUrl,
|
||||
timeoutMs: options?.timeoutMs,
|
||||
};
|
||||
// Already at the requested pin? Then there is no switch to make, so do
|
||||
|
|
@ -694,11 +718,11 @@ export async function cloneOrPull(
|
|||
try {
|
||||
const runGitImpl = options?.runGitForTest ?? runGit;
|
||||
const cloneArgs = options?.branch
|
||||
? buildBranchCloneArgs(url, safeTarget, options.branch)
|
||||
: buildCloneArgs(url, safeTarget);
|
||||
? buildBranchCloneArgs(dialUrl, safeTarget, options.branch)
|
||||
: buildCloneArgs(dialUrl, safeTarget);
|
||||
await runGitImpl(cloneArgs, undefined, {
|
||||
token: options?.token,
|
||||
url,
|
||||
url: dialUrl,
|
||||
timeoutMs: options?.timeoutMs,
|
||||
});
|
||||
await assertPostRealpathContainment(cloneRoot, safeTarget);
|
||||
|
|
@ -805,7 +829,8 @@ function resolveGitCredential(options?: { token?: string; url?: string }): strin
|
|||
|
||||
let host: string;
|
||||
try {
|
||||
host = new URL(url).hostname.toLowerCase();
|
||||
// One trailing dot is the absolute-lookup marker, same host (`github.com.`).
|
||||
host = new URL(url).hostname.toLowerCase().replace(/\.$/, '');
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -738,6 +738,88 @@ describe('auto-sync runner', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('clones an allowlisted self-hosted remote under its host path', async () => {
|
||||
const selfHosted: AutoSyncConfig = {
|
||||
...config,
|
||||
allowedHosts: ['gitlab.mycompany.com'],
|
||||
projects: [
|
||||
{
|
||||
localPath: '/tmp/repos',
|
||||
overwriteLocalChanges: false,
|
||||
branches: ['main'],
|
||||
remoteUrls: ['git@gitlab.mycompany.com:group/repo.git'],
|
||||
},
|
||||
],
|
||||
};
|
||||
expect(
|
||||
getConfiguredRepoPath(
|
||||
selfHosted.projects[0],
|
||||
'repo',
|
||||
'git@gitlab.mycompany.com:group/repo.git',
|
||||
selfHosted.allowedHosts,
|
||||
),
|
||||
).toBe('/tmp/repos/gitlab.mycompany.com/group/repo');
|
||||
|
||||
const cloneOrPull = vi.fn(async () => '/tmp/repos/gitlab.mycompany.com/group/repo');
|
||||
const deps: Partial<AutoSyncRunDeps> = withCloneRoot({
|
||||
cloneOrPull,
|
||||
getCurrentBranch: vi.fn(() => 'main'),
|
||||
getCurrentCommit: vi.fn(() => 'commit-2'),
|
||||
runAnalysis: vi.fn(async () => ({ stats: { files: 1 } }) as any),
|
||||
registerRepo: vi.fn(async () => 'repo'),
|
||||
loadState: vi.fn(async () => ({})),
|
||||
saveState: vi.fn(async () => {}),
|
||||
writeCommitInfo: vi.fn(async () => {}),
|
||||
addRepoToGroup: vi.fn(async () => false),
|
||||
syncGroupByName: vi.fn(async () => {}),
|
||||
getAvailableMemoryGB: vi.fn(() => 8),
|
||||
});
|
||||
|
||||
await runAutoSyncOnce(selfHosted, {
|
||||
deps,
|
||||
logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() },
|
||||
});
|
||||
|
||||
expect(cloneOrPull).toHaveBeenCalledWith(
|
||||
'git@gitlab.mycompany.com:group/repo.git',
|
||||
'/tmp/repos/gitlab.mycompany.com/group/repo',
|
||||
undefined,
|
||||
expect.objectContaining({
|
||||
allowAutoSyncSsh: true,
|
||||
autoSyncAllowedHosts: ['gitlab.mycompany.com'],
|
||||
branch: 'main',
|
||||
expectedRepoName: 'repo',
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('skips a self-hosted remote that is not listed in allowed_hosts', async () => {
|
||||
const selfHosted: AutoSyncConfig = {
|
||||
...config,
|
||||
projects: [
|
||||
{
|
||||
localPath: '/tmp/repos',
|
||||
overwriteLocalChanges: false,
|
||||
branches: ['main'],
|
||||
remoteUrls: ['git@gitlab.mycompany.com:group/repo.git'],
|
||||
},
|
||||
],
|
||||
};
|
||||
const cloneOrPull = vi.fn(async () => '/tmp/repos/should-not-clone');
|
||||
const logger = { info: vi.fn(), warn: vi.fn(), error: vi.fn() };
|
||||
const result = await runAutoSyncOnce(selfHosted, {
|
||||
deps: withCloneRoot({
|
||||
cloneOrPull,
|
||||
getAvailableMemoryGB: vi.fn(() => 8),
|
||||
}),
|
||||
logger,
|
||||
});
|
||||
|
||||
expect(result.failed).toBe(1);
|
||||
expect(cloneOrPull).not.toHaveBeenCalled();
|
||||
expect(logger.error).toHaveBeenCalledWith(expect.stringContaining('allowed_hosts'));
|
||||
});
|
||||
|
||||
it('passes watch cancellation controls to the isolated analysis runner', async () => {
|
||||
const controller = new AbortController();
|
||||
const onAnalysisCancellationRequested = vi.fn();
|
||||
|
|
|
|||
|
|
@ -19,10 +19,12 @@ import {
|
|||
resetAutoSyncState,
|
||||
saveAutoSyncState,
|
||||
shouldAnalyzeCommit,
|
||||
getAutoSyncRepoIdentity,
|
||||
validateAutoSyncRemoteUrl,
|
||||
validateAutoSyncBranchName,
|
||||
writeProjectCommitInfo,
|
||||
} from '../../src/core/auto-sync/index.js';
|
||||
import { absoluteAutoSyncRemoteUrl } from '../../src/core/auto-sync/config.js';
|
||||
import { acquireFileLock } from '../../src/storage/file-lock.js';
|
||||
|
||||
describe('auto-sync', () => {
|
||||
|
|
@ -154,6 +156,7 @@ describe('auto-sync', () => {
|
|||
expect(loaded.config.analyzeTimeoutMs).toBe(300_000);
|
||||
expect(loaded.config.maxConcurrency).toBe(1);
|
||||
expect(loaded.config.analyzeFailureThreshold).toBe(3);
|
||||
expect(loaded.config.allowedHosts).toEqual([]);
|
||||
expect(loaded.config.projects[0].groupName).toBeUndefined();
|
||||
expect(loaded.config.projects[0].pdg).toBeUndefined();
|
||||
expect(loaded.config.projects[0].overwriteLocalChanges).toBe(false);
|
||||
|
|
@ -571,6 +574,11 @@ describe('auto-sync', () => {
|
|||
expect(() => validateAutoSyncRemoteUrl('git@example.com:owner/repo.git')).toThrow(
|
||||
'host must be',
|
||||
);
|
||||
expect(() => validateAutoSyncRemoteUrl('git@localhost:owner/repo.git')).toThrow('host must be');
|
||||
expect(() => validateAutoSyncRemoteUrl('git@github.com.evil.com:owner/repo.git')).toThrow(
|
||||
'host must be',
|
||||
);
|
||||
expect(() => validateAutoSyncRemoteUrl('git@..:owner/repo.git')).toThrow('DNS hostname');
|
||||
// Traversal is a whole segment; consecutive dots inside a name are not.
|
||||
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/foo..bar.git')).not.toThrow();
|
||||
expect(() => validateAutoSyncRemoteUrl('git@github.com:owner/../escape.git')).toThrow(
|
||||
|
|
@ -596,6 +604,117 @@ describe('auto-sync', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('accepts a self-hosted remote only when its host is listed in allowed_hosts', () => {
|
||||
const ssh = 'git@gitlab.mycompany.com:group/repo.git';
|
||||
const https = 'https://gitlab.mycompany.com/group/subgroup/repo.git';
|
||||
const allowed = ['GitLab.MyCompany.com'];
|
||||
|
||||
expect(() => validateAutoSyncRemoteUrl(ssh)).toThrow('allowed_hosts');
|
||||
expect(() => validateAutoSyncRemoteUrl(ssh, allowed)).not.toThrow();
|
||||
expect(() => validateAutoSyncRemoteUrl(https, allowed)).not.toThrow();
|
||||
expect(() =>
|
||||
validateAutoSyncRemoteUrl('git@git.gitlab.mycompany.com:group/repo.git', allowed),
|
||||
).toThrow('host must be');
|
||||
expect(() =>
|
||||
validateAutoSyncRemoteUrl('git@mycompany.com:group/repo.git', ['gitlab.mycompany.com']),
|
||||
).toThrow('host must be');
|
||||
expect(() =>
|
||||
validateAutoSyncRemoteUrl('https://user:token@gitlab.mycompany.com/group/repo.git', allowed),
|
||||
).toThrow('userinfo');
|
||||
|
||||
const parsed = parseAutoSyncConfig(
|
||||
[
|
||||
'sync_interval_minutes: 10',
|
||||
'allowed_hosts: [GitLab.MyCompany.com, GitLab.MyCompany.com]',
|
||||
'projects:',
|
||||
' - local_path: /tmp/repos',
|
||||
' branches: [main]',
|
||||
' remote_urls:',
|
||||
` - ${ssh}`,
|
||||
` - ${https}`,
|
||||
].join('\n'),
|
||||
'/tmp/watch_config.yml',
|
||||
);
|
||||
expect(parsed.allowedHosts).toEqual(['gitlab.mycompany.com']);
|
||||
|
||||
expect(() =>
|
||||
parseAutoSyncConfig(
|
||||
[
|
||||
'sync_interval_minutes: 10',
|
||||
'allowed_hosts: ["*.mycompany.com"]',
|
||||
'projects:',
|
||||
' - local_path: /tmp/repos',
|
||||
' branches: [main]',
|
||||
' remote_urls:',
|
||||
' - git@github.com:owner/repo.git',
|
||||
].join('\n'),
|
||||
'/tmp/watch_config.yml',
|
||||
),
|
||||
).toThrow('allowed_hosts[0] must be a DNS hostname');
|
||||
|
||||
expect(() =>
|
||||
parseAutoSyncConfig(
|
||||
[
|
||||
'sync_interval_minutes: 10',
|
||||
'allowed_hosts: gitlab.mycompany.com',
|
||||
'projects:',
|
||||
' - local_path: /tmp/repos',
|
||||
' branches: [main]',
|
||||
' remote_urls:',
|
||||
' - git@github.com:owner/repo.git',
|
||||
].join('\n'),
|
||||
'/tmp/watch_config.yml',
|
||||
),
|
||||
).toThrow('allowed_hosts must be a list of DNS hostnames');
|
||||
});
|
||||
|
||||
it('rejects ambiguous numeric hosts and treats one trailing dot as the same name', () => {
|
||||
const config = (allowed: string, remote: string) =>
|
||||
[
|
||||
'sync_interval_minutes: 10',
|
||||
`allowed_hosts: ["${allowed}"]`,
|
||||
'projects:',
|
||||
' - local_path: /tmp/repos',
|
||||
' branches: [main]',
|
||||
' remote_urls:',
|
||||
` - ${remote}`,
|
||||
].join('\n');
|
||||
|
||||
for (const host of ['192.168.1', '127.1', '0x7f.0.0.1', '0177.0.0.1', '2130706433']) {
|
||||
expect(() =>
|
||||
parseAutoSyncConfig(config(host, 'git@github.com:owner/repo.git'), '/tmp/watch_config.yml'),
|
||||
).toThrow('ambiguous numeric spelling');
|
||||
expect(() => validateAutoSyncRemoteUrl(`git@${host}:group/repo.git`, [host])).toThrow(
|
||||
'ambiguous numeric spelling',
|
||||
);
|
||||
}
|
||||
|
||||
expect(() =>
|
||||
validateAutoSyncRemoteUrl('git@10.0.0.1:group/repo.git', ['10.0.0.1']),
|
||||
).not.toThrow();
|
||||
expect(absoluteAutoSyncRemoteUrl('git@10.0.0.1:group/repo.git')).toBe(
|
||||
'git@10.0.0.1:group/repo.git',
|
||||
);
|
||||
|
||||
const allowed = ['git', 'gitlab.mycompany.com'];
|
||||
expect(() => validateAutoSyncRemoteUrl('git@git.:group/repo.git', allowed)).not.toThrow();
|
||||
expect(getAutoSyncRepoIdentity('git@git.:group/repo.git', allowed)).toBe('git/group/repo');
|
||||
expect(getAutoSyncRepoIdentity('git@gitlab.mycompany.com.:group/repo.git', allowed)).toBe(
|
||||
'gitlab.mycompany.com/group/repo',
|
||||
);
|
||||
expect(absoluteAutoSyncRemoteUrl('git@git:group/repo.git')).toBe('git@git.:group/repo.git');
|
||||
expect(absoluteAutoSyncRemoteUrl('git@git.:group/repo.git')).toBe('git@git.:group/repo.git');
|
||||
expect(absoluteAutoSyncRemoteUrl('https://gitlab.mycompany.com/group/repo.git')).toBe(
|
||||
'https://gitlab.mycompany.com./group/repo.git',
|
||||
);
|
||||
|
||||
const parsed = parseAutoSyncConfig(
|
||||
config('git.', 'git@git:group/repo.git'),
|
||||
'/tmp/watch_config.yml',
|
||||
);
|
||||
expect(parsed.allowedHosts).toEqual(['git']);
|
||||
});
|
||||
|
||||
it('parses repo git timeout durations', () => {
|
||||
expect(parseDurationMs('10s')).toBe(10_000);
|
||||
expect(parseDurationMs('2m')).toBe(120_000);
|
||||
|
|
|
|||
|
|
@ -465,6 +465,20 @@ describe('git-clone', () => {
|
|||
expect(env.GIT_CONFIG_VALUE_1).toBe(expected);
|
||||
});
|
||||
|
||||
it('treats one trailing dot as the same GitHub host and scopes the header to that URL', () => {
|
||||
const env = buildGitEnv(
|
||||
{},
|
||||
{ token: 'ghp_secret123', url: 'https://github.com./owner/repo' },
|
||||
);
|
||||
expect(env.GIT_CONFIG_COUNT).toBe('2');
|
||||
expect(env.GIT_CONFIG_KEY_1).toBe('http.https://github.com./owner/repo.extraHeader');
|
||||
const foreign = buildGitEnv(
|
||||
{},
|
||||
{ token: 'ghp_secret123', url: 'https://github.com.evil.com./owner/repo' },
|
||||
);
|
||||
expect(foreign.GIT_CONFIG_COUNT).toBe('1');
|
||||
});
|
||||
|
||||
it('does not inject a token for a non-github host (defense-in-depth host bind)', () => {
|
||||
const env = buildGitEnv({}, { token: 'ghp_secret123', url: 'https://gitlab.com/owner/repo' });
|
||||
expect(env.GIT_CONFIG_COUNT).toBe('1');
|
||||
|
|
@ -673,6 +687,56 @@ describe('git-clone', () => {
|
|||
}
|
||||
});
|
||||
|
||||
it('dials an allowlisted DNS name absolutely and leaves a strict IPv4 literal unchanged', async () => {
|
||||
const root = await mkControlledRoot('gitnexus-controlled-root-');
|
||||
const cases = [
|
||||
{
|
||||
url: 'git@git:group/repo.git',
|
||||
hosts: ['git'],
|
||||
dial: 'git@git.:group/repo.git',
|
||||
dir: 'short',
|
||||
},
|
||||
{
|
||||
url: 'https://gitlab.mycompany.com/group/repo.git',
|
||||
hosts: ['gitlab.mycompany.com'],
|
||||
dial: 'https://gitlab.mycompany.com./group/repo.git',
|
||||
dir: 'fqdn',
|
||||
},
|
||||
{
|
||||
url: 'git@10.0.0.1:group/repo.git',
|
||||
hosts: ['10.0.0.1'],
|
||||
dial: 'git@10.0.0.1:group/repo.git',
|
||||
dir: 'quad',
|
||||
},
|
||||
] as const;
|
||||
try {
|
||||
for (const item of cases) {
|
||||
const target = path.join(root, item.dir, 'repo');
|
||||
const runGitForTest = vi.fn(async (args: string[]) => {
|
||||
if (args[0] === 'clone') await fs.mkdir(path.join(target, '.git'), { recursive: true });
|
||||
return '';
|
||||
});
|
||||
await cloneOrPull(item.url, target, undefined, {
|
||||
allowedCloneRoot: root,
|
||||
expectedRepoName: 'repo',
|
||||
allowAutoSyncSsh: true,
|
||||
autoSyncAllowedHosts: [...item.hosts],
|
||||
runGitForTest,
|
||||
});
|
||||
expect(runGitForTest.mock.calls[0][0]).toEqual([
|
||||
'clone',
|
||||
'--depth',
|
||||
'1',
|
||||
'--',
|
||||
item.dial,
|
||||
target,
|
||||
]);
|
||||
}
|
||||
} finally {
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// `assertRemoteMatchesRequestedUrl` runs REAL git (it is not injectable), so
|
||||
// these fixtures are real repositories with a matching origin; only the
|
||||
// branch logic under test is driven through `runGitForTest`.
|
||||
|
|
@ -685,6 +749,39 @@ describe('git-clone', () => {
|
|||
return target;
|
||||
};
|
||||
|
||||
it('stores an absolute DNS name on an existing auto-sync origin', async () => {
|
||||
const root = await mkControlledRoot('gitnexus-controlled-root-');
|
||||
const remote = 'git@git:group/repo.git';
|
||||
try {
|
||||
const target = path.join(root, 'repo');
|
||||
await fs.mkdir(target, { recursive: true });
|
||||
await runGit(['init', '--initial-branch=main'], target);
|
||||
await runGit(['remote', 'add', 'origin', remote], target);
|
||||
const runGitForTest = vi.fn(async (args: string[]) => {
|
||||
if (args[0] === 'rev-parse' && args[1] === '--abbrev-ref') return 'main\n';
|
||||
return '';
|
||||
});
|
||||
const options = {
|
||||
allowedCloneRoot: root,
|
||||
expectedRepoName: 'repo',
|
||||
allowAutoSyncSsh: true,
|
||||
autoSyncAllowedHosts: ['git'],
|
||||
branch: 'main',
|
||||
runGitForTest,
|
||||
};
|
||||
await cloneOrPull(remote, target, undefined, options);
|
||||
expect((await runGit(['config', '--get', 'remote.origin.url'], target)).trim()).toBe(
|
||||
'git@git.:group/repo.git',
|
||||
);
|
||||
await cloneOrPull(remote, target, undefined, options);
|
||||
expect((await runGit(['config', '--get', 'remote.origin.url'], target)).trim()).toBe(
|
||||
'git@git.:group/repo.git',
|
||||
);
|
||||
} finally {
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('re-indexing the SAME pinned branch fetches via a safe refspec instead of refusing a dirty tree', async () => {
|
||||
// Analyze writes AGENTS.md / CLAUDE.md / .claude/ into the clone, so the
|
||||
// tree is dirty from its own first run. Routing a same-branch request
|
||||
|
|
@ -1053,11 +1150,11 @@ describe('git-clone', () => {
|
|||
'--branch',
|
||||
'develop',
|
||||
'--',
|
||||
'git@gitlab.com:group/subgroup/repo.git',
|
||||
'git@gitlab.com.:group/subgroup/repo.git',
|
||||
target,
|
||||
],
|
||||
undefined,
|
||||
{ token: undefined, url: 'git@gitlab.com:group/subgroup/repo.git', timeoutMs: 10_000 },
|
||||
{ token: undefined, url: 'git@gitlab.com.:group/subgroup/repo.git', timeoutMs: 10_000 },
|
||||
);
|
||||
} finally {
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
|
|
@ -1239,7 +1336,7 @@ describe('git-clone', () => {
|
|||
|
||||
await fs.writeFile(
|
||||
gitConfig,
|
||||
`[protocol "file"]\n\tallow = always\n[url "${remoteFileUrl}"]\n\tinsteadOf = ${remoteUrl}\n`,
|
||||
`[protocol "file"]\n\tallow = always\n[url "${remoteFileUrl}"]\n\tinsteadOf = ${remoteUrl}\n\tinsteadOf = git@github.com.:team/repo.git\n`,
|
||||
);
|
||||
process.env.GIT_CONFIG_GLOBAL = gitConfig;
|
||||
process.env.GIT_CONFIG_NOSYSTEM = '1';
|
||||
|
|
@ -1422,7 +1519,7 @@ describe('git-clone', () => {
|
|||
}),
|
||||
).rejects.toThrow('offline');
|
||||
await expect(serverGetRemoteOriginUrl(target)).resolves.toBe(
|
||||
'https://github.com/owner/repo.git',
|
||||
'https://github.com./owner/repo.git',
|
||||
);
|
||||
await expect(fs.access(quarantineRoot)).rejects.toThrow();
|
||||
} finally {
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue