GitNexus/gitnexus/test/integration/analyze-embedding-flags-e2e.test.ts
bluerose 89ffa71a52
feat(cli): add --embeddings-baseurl/-model/-auth-token/-dims flags to analyze (#2140)
* feat(cli): add --embeddings-baseurl/-model/-auth-token/-dims flags to analyze

Add four CLI flags to `gitnexus analyze` that configure a custom
OpenAI-compatible HTTP embedding endpoint by setting the
GITNEXUS_EMBEDDING_URL / _MODEL / _API_KEY / _DIMS env vars the HTTP
embedding client already reads. Flags override env vars; env vars keep
working as before. URLs are validated (http/https) and dims must be a
positive integer. Prints "Using custom embedding endpoint: <url>" when
a URL+model pair is configured, and warns when the flags are passed
without --embeddings. The new env keys are added to the analyze
snapshot/restore set so programmatic callers don't leak state. The
non-secret flags are also accepted from .gitnexusrc; the auth token is
intentionally CLI/env-only.

* fix(analyze): set GITNEXUS_EMBEDDING_DIMS from CLI flags before module import

schema.ts reads EMBEDDING_DIMS at module-load time via the static-import
chain (analyze.ts -> run-analyze.ts -> schema.ts). The previous approach
of setting the env var inside analyzeCommandImpl ran AFTER schema.ts had
already loaded with the default 384, causing "Expected: 384, Actual: 4096"
errors when using --embeddings-dims 4096.

Fix: use Commander's preAction hook to set GITNEXUS_EMBEDDING_* env vars
before the lazy import of analyze.ts triggers the schema.ts module load.

* fix(analyze): use hook callback arg instead of this in preAction

Commander v14 passes the command as first argument, not as this binding.

* refactor(cli): rename --embeddings-* analyze flags to singular --embedding-*

Aligns the custom embedding endpoint flags with the existing singular
tuning flags (--embedding-threads/--embedding-device): --embedding-base-url,
--embedding-model, --embedding-auth-token, --embedding-dims. Renames the
derived AnalyzeOptions fields and the .gitnexusrc KEY_SPECS keys to match.
Behavior-preserving; the GITNEXUS_EMBEDDING_* env vars are unchanged.

Refs #2140 review.

* fix(cli): validate and normalize --embedding-dims before module-load reads it

The preAction hook wrote GITNEXUS_EMBEDDING_DIMS unvalidated, so an invalid
value (abc/0/-5/0x10) threw from schema.ts during the lazy import — surfacing
as a raw unhandled rejection on the synchronous program.parse path instead of
a friendly error. And '1e3' slipped through: schema.ts parseInt froze the
vector column at FLOAT[1] while the impl's Number-based check accepted 1000,
so http-client requested 1000-dim vectors against a 1-dim column.

Extract a dependency-free normalizeEmbeddingDims helper (strict /^\d+$/ +
positive, trim-then-validate, canonicalized) shared by both the hook (CLI
path, before module-load) and analyzeCommandImpl (direct-call path). All three
readers — schema.ts, http-client, and this helper — now agree on one value,
and invalid input gets a clean message instead of a crash or a silent mismatch.

Refs #2140 review.

* fix(cli): mask credentials in the custom embedding endpoint confirmation

A base URL with userinfo (http://user:pass@host/v1) or a query token
(?api_key=…) passed the new-URL + http/https validation and was printed
verbatim in the 'Using custom embedding endpoint:' line, leaking the secret
to terminal scrollback and CI logs. Route it through the existing safeUrl()
(now exported from http-client) which strips userinfo + query, keeping
protocol/host/path. Single source of truth — no second sanitizer.

Refs #2140 review.

* fix(cli): drop the ineffective embeddingDims .gitnexusrc key

embeddingDims as a .gitnexusrc key silently did nothing: .gitnexusrc loads in
analyzeCommandImpl, AFTER the lazy import already ran schema.ts's module-load
read of GITNEXUS_EMBEDDING_DIMS, so a config value never sized the vector
column. Remove it (config now fails closed on the key, like the auth token);
URL/MODEL stay as config keys because they're read lazily at runtime. Dims
remains available via --embedding-dims or GITNEXUS_EMBEDDING_DIMS.

Refs #2140 review.

* refactor(cli): narrow the analyze preAction hook to GITNEXUS_EMBEDDING_DIMS

Only DIMS is read at module-load (schema.ts), so only it must be set before the
lazy import. URL/MODEL/API_KEY are read lazily at runtime, so analyzeCommandImpl
is their sole setter — and because the impl's env snapshot is taken AFTER this
hook ran, leaving those three in the hook leaked them past restore. Drop them
from the hook (the impl already sets+restores them), and capture/restore the
pre-hook DIMS baseline via a postAction hook so a CLI --embedding-dims override
no longer leaks into a later in-process program.parseAsync.

Refs #2140 review.

* fix(cli): gate the custom-endpoint confirmation on the embedding flags

The confirmation collapsed into one if/else chain that emits at most one
message reflecting the run's intent. Gating on embeddingsEnabled stops the
'Using custom embedding endpoint' line from printing on every analyze run when
GITNEXUS_EMBEDDING_URL+MODEL merely happen to be set in the environment, and
ordering the '--embeddings absent' note first removes the contradiction where
it printed alongside 'Using custom embedding endpoint'.

Refs #2140 review.

* test(cli): cover the custom embedding endpoint flags

Adds direct-call (analyzeCommandImpl path) coverage the original PR lacked:
URL validation (empty/invalid/non-http), model/token emptiness, dims
validation incl. the 1e3 regression, credential masking in the confirmation
line, confirmation gating (absent --embeddings; ambient env must not trigger
it), CLI-over-env precedence, and the GITNEXUS_EMBEDDING_* snapshot/restore
round-trip. Complements embedding-dims.test.ts and http-client-safe-url.test.ts.

Refs #2140 review.

* test(cli): e2e-cover the --embedding-dims crash path on the real CLI

The dims-validation fix lives in the commander preAction hook, which only
fires on the program.parse path; the direct analyzeCommand() unit tests bypass
it. Add a subprocess e2e (run via tsx, no build) asserting that invalid
--embedding-dims (abc/0/-5/1e3/3.5) produces the friendly flag-named error and
exit 1 — NOT the raw schema.ts module-load throw that the original bug
surfaced. Cases exit inside the hook (no repo/import/pipeline), so they're
deterministic and fast. Updates the unit-suite comment to point at it.

Refs #2140 review.

---------

Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
2026-06-13 14:01:12 +01:00

89 lines
3.7 KiB
TypeScript

/**
* E2E: the analyze `--embedding-dims` validation on the REAL CLI parse path.
*
* The fix for the dims crash lives in the commander `preAction` hook in
* src/cli/index.ts, which must validate the value BEFORE the lazy
* import('./analyze.js') triggers schema.ts's module-load read of
* GITNEXUS_EMBEDDING_DIMS (which throws on a bad value). Direct
* analyzeCommand() unit tests bypass the hook entirely, so this interaction —
* commander hook timing + synchronous program.parse + lazy import + the
* module-load throw — can only be exercised by running the actual binary.
*
* Reliability: every case here is INVALID dims, so the hook prints a friendly
* error and process.exit(1)s *before* the action runs. No repo, no lazy
* import, no pipeline, no DB, no network — just tsx startup + commander parse.
* That makes these deterministic and fast, unlike the full-analyze e2e cases.
*
* Run via tsx (no build step), mirroring test/integration/cli-e2e.test.ts.
*/
import { spawnSync } from 'child_process';
import { createRequire } from 'module';
import os from 'os';
import path from 'path';
import fs from 'fs';
import { fileURLToPath, pathToFileURL } from 'url';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
const testDir = path.dirname(fileURLToPath(import.meta.url));
const repoRoot = path.resolve(testDir, '../..');
const cliEntry = path.join(repoRoot, 'src/cli/index.ts');
const _require = createRequire(import.meta.url);
const tsxPkgDir = path.dirname(_require.resolve('tsx/package.json'));
const tsxImportUrl = pathToFileURL(path.join(tsxPkgDir, 'dist', 'loader.mjs')).href;
let cwd: string;
beforeAll(() => {
// The hook exits before any repo logic, so this dir need not be a git repo.
cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'gn-embed-dims-e2e-'));
});
afterAll(() => {
fs.rmSync(cwd, { recursive: true, force: true });
});
function runAnalyze(args: string[]) {
// Strip any ambient GITNEXUS_EMBEDDING_* so the flag is the sole input.
const env = { ...process.env } as Record<string, string | undefined>;
for (const k of Object.keys(env)) {
if (k.startsWith('GITNEXUS_EMBEDDING_')) delete env[k];
}
// Pre-set the heap cap so analyzeCommand's ensureHeap() wouldn't re-exec
// (would drop the tsx loader). Irrelevant on the invalid-dims path since the
// hook exits first, but harmless and matches the cli-e2e harness.
env.NODE_OPTIONS = `${process.env.NODE_OPTIONS || ''} --max-old-space-size=8192`.trim();
return spawnSync(process.execPath, ['--import', tsxImportUrl, cliEntry, 'analyze', ...args], {
cwd,
encoding: 'utf8',
timeout: 30_000,
stdio: ['pipe', 'pipe', 'pipe'],
env,
});
}
describe('analyze --embedding-dims validation (real CLI parse path)', () => {
it.each(['abc', '0', '-5', '1e3', '3.5'])(
'rejects %j with a friendly error, not a raw module-load crash',
(bad) => {
const result = runAnalyze([cwd, '--embedding-dims', bad]);
const stderr = result.stderr ?? '';
// Non-zero exit (the hook called process.exit(1)).
expect(result.status).toBe(1);
// The friendly, flag-named message surfaced...
expect(stderr).toContain('--embedding-dims must be a positive integer');
// ...and NOT the raw schema.ts throw (env-var-named) that would appear if
// the hook validation were removed and schema.ts crashed during the lazy
// import. This is the regression guard for finding #1.
expect(stderr).not.toContain('GITNEXUS_EMBEDDING_DIMS must be a positive integer');
// No unhandled-rejection / stack-trace leakage either.
expect(stderr).not.toContain('UnhandledPromiseRejection');
expect(stderr).not.toMatch(/^\s+at .+:\d+:\d+/m);
},
30_000,
);
});