mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-08 03:08:13 +00:00
* fix(embeddings): resolve onnxruntime-common under pnpm-strict / pnpm dlx (#307) `@huggingface/transformers` does a bare `import 'onnxruntime-common'` from its shipped `dist/transformers.node.mjs`, but never declares onnxruntime-common in its own `dependencies`. npm's flat node_modules (and pnpm with hoisting) place it on transformers' resolution path by accident; pnpm's isolated store only links a package's declared deps into its scope, so under pnpm-strict / `pnpm dlx` / `pnpx` the import dies with ERR_MODULE_NOT_FOUND before `analyze --embeddings` can run. Declaring onnxruntime-common in gitnexus' own deps (#2074) does not fix this under pnpm: Node resolves the bare specifier from transformers' module scope, not ours, and overrides/resolutions can only re-version an existing edge, never add the missing one (verified against a real `hoist=false` install — the declaration only changes which version wins the hoist, never whether the import resolves). Fix: install a synchronous, in-thread ESM resolution hook (`module.registerHooks`) right before the lazy transformers import that redirects `onnxruntime-common` to the copy gitnexus depends on — but only when the default resolver fails. On npm / hoisted layouts the default resolver succeeds first and the hook never fires, so working setups are unchanged. The hook only intercepts the exact `onnxruntime-common` specifier on failure, so it can never mask an unrelated resolution error; onnxruntime-node's native binding still loads normally from transformers' own scope. `registerHooks` (sync, in-thread, single inline closure) is preferred over the older `module.register` (async, off-thread, now deprecated — DEP0205, removed in Node 26): the redirect is a one-line conditional that needs no worker thread, no separate hook module, and no `data` marshalling. It is available on Node >= 22.15; on older runtimes the helper is a graceful no-op (the gitnexus engines floor is >= 22.0.0, and the import still resolves on hoisted layouts there). Chosen over bundling transformers (the build is tsc-only, and transformers carries native onnxruntime-node + WASM onnxruntime-web assets that bundle poorly). Installation is idempotent, best-effort, and lazy — only on the local-embedding path, so it never affects analysis, the parse workers, or HTTP embedding mode. Validated end-to-end: the compiled resolver fixes a real pnpm `hoist=false` transformers install (ERR_MODULE_NOT_FOUND -> resolved). The separate `@ladybugdb/core` native-binary path under pure `pnpm dlx` is unchanged (#1967 handles that gracefully). Refs #307, #2069 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(embeddings): version-match the onnxruntime-common redirect target (#307) Prefer the onnxruntime-common that onnxruntime-node (the native binding transformers actually loads) depends on, so the redirected copy is version- matched to that binding even under `pnpm dlx` — where gitnexus' npm-style `overrides` block does not apply, because it is honoured only from a root manifest and gitnexus is a transitive dependency there. The walk resolves transformers' main entry (not its `exports`-blocked package.json) -> onnxruntime-node -> its onnxruntime-common, and falls back to gitnexus' own direct dependency when the chain can't be walked. Also corrects the doc comment that claimed the gitnexus copy was already "version-aligned". Addresses a PR #2139 tri-review finding (P2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(embeddings): narrow the onnxruntime-common resolve fallback to absence errors (#307) The resolve closure's `catch` swallowed every error from `nextResolve` and redirected, which would silently paper over a genuinely present-but-broken onnxruntime-common install. Only substitute gitnexus' copy when the specifier is actually absent (ERR_MODULE_NOT_FOUND, or ERR_PACKAGE_PATH_NOT_EXPORTED for an exports-broken copy); rethrow anything else. Adds a test that an unrelated error code rethrows. Addresses a PR #2139 tri-review finding (P3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(embeddings): cover the onnxruntime-common resolver best-effort swallow path (#307) The outer try/catch in ensureOnnxRuntimeCommonResolvable() was untested. A throwing registerHooks spy drives it; the call must not throw (initEmbedder does not guard the return, so a throw would break `analyze --embeddings`). The vitest quirk that surfaced an earlier attempt applies to throwing mock factories, not a throwing spy implementation, so this is testable cleanly. Addresses a PR #2139 tri-review finding (P2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(embeddings): tighten the onnxruntime-common redirect-URL assertion (#307) `/^file:\/\/.*onnxruntime-common/` matched the substring anywhere, so a lookalike path (e.g. `/x/onnxruntime-common-fake/`) would pass. Require an actual `/node_modules/onnxruntime-common/...js` segment so the assertion proves the redirect resolves to the real package, not just a string match. Addresses a PR #2139 tri-review finding (P3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(embeddings): drop the no-op __resetOnnxRuntimeCommonResolverForTests seam (#307) The test helper reloads the resolver via vi.resetModules() + a fresh import(), which already re-initialises the module-level one-shot `attempted` flag to false. The __reset export it then called was therefore a no-op. Remove the test-only export and its call; isolation now rests solely on vi.resetModules(). Addresses a PR #2139 tri-review finding (P3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(embeddings): correct the onnxruntime-common resolver isolation comment (#307) The doc comment claimed the hook "never affects other tools' resolution". Once installed, `module.registerHooks` is process-global and its resolve closure runs for every subsequent resolution — it passes them all through untouched and only substitutes the exact `onnxruntime-common` specifier on genuine absence, at a cost of one string comparison. Also note `registerHooks` is @experimental and requires Node >= 22.15 (graceful no-op below that). Comment-only. Addresses a PR #2139 tri-review finding (P3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
132 lines
4.9 KiB
TypeScript
132 lines
4.9 KiB
TypeScript
import { describe, it, expect, vi, afterEach } from 'vitest';
|
|
|
|
/**
|
|
* Tests for the #307 pnpm-strict / `pnpm dlx` fix: a synchronous in-thread ESM
|
|
* resolution hook (`module.registerHooks`) that redirects @huggingface/
|
|
* transformers' phantom `onnxruntime-common` import to gitnexus' own copy.
|
|
*
|
|
* Each test mocks `node:module` with a chosen `registerHooks` (a spy, or
|
|
* `undefined` to simulate Node < 22.15) so we can assert one-shot installation,
|
|
* graceful degradation, and the redirect/passthrough/rethrow logic of the
|
|
* resolve closure — without mutating the real process loader.
|
|
*/
|
|
|
|
const RESOLVER = '../../src/core/embeddings/onnxruntime-common-resolver.js';
|
|
|
|
/**
|
|
* (Re)load the resolver with a chosen `registerHooks` mocked into node:module.
|
|
* `vi.resetModules()` + the fresh `import()` re-initialises the module-level
|
|
* one-shot guard, so each test gets a pristine resolver with no shared state.
|
|
*/
|
|
async function loadResolver(registerHooks: unknown) {
|
|
vi.resetModules();
|
|
vi.doMock('node:module', async (importOriginal) => {
|
|
const orig = await importOriginal<typeof import('node:module')>();
|
|
return { ...orig, registerHooks };
|
|
});
|
|
return import(RESOLVER);
|
|
}
|
|
|
|
const ctx = { conditions: [], importAttributes: {} } as never;
|
|
const moduleNotFound = (): Error => {
|
|
const e = new Error("Cannot find package 'onnxruntime-common'") as Error & { code: string };
|
|
e.code = 'ERR_MODULE_NOT_FOUND';
|
|
return e;
|
|
};
|
|
|
|
afterEach(() => {
|
|
vi.doUnmock('node:module');
|
|
});
|
|
|
|
describe('ensureOnnxRuntimeCommonResolvable — installation', () => {
|
|
it('installs the resolve hook exactly once (idempotent)', async () => {
|
|
const spy = vi.fn();
|
|
const mod = await loadResolver(spy);
|
|
|
|
mod.ensureOnnxRuntimeCommonResolvable();
|
|
mod.ensureOnnxRuntimeCommonResolvable(); // second call is a no-op
|
|
|
|
expect(spy).toHaveBeenCalledTimes(1);
|
|
expect(typeof spy.mock.calls[0][0].resolve).toBe('function');
|
|
});
|
|
|
|
it('no-ops gracefully when registerHooks is unavailable (Node < 22.15)', async () => {
|
|
const mod = await loadResolver(undefined);
|
|
// Must not throw even though there is no synchronous-hooks API to call.
|
|
expect(() => mod.ensureOnnxRuntimeCommonResolvable()).not.toThrow();
|
|
});
|
|
|
|
it('is best-effort: swallows a registerHooks() failure instead of throwing into the embedder', async () => {
|
|
const mod = await loadResolver(
|
|
vi.fn(() => {
|
|
throw new Error('hook-install-failed');
|
|
}),
|
|
);
|
|
// The call site (initEmbedder) does not guard the return; a throw here would
|
|
// break `analyze --embeddings`. The outer try/catch must absorb it.
|
|
expect(() => mod.ensureOnnxRuntimeCommonResolvable()).not.toThrow();
|
|
});
|
|
});
|
|
|
|
describe('ensureOnnxRuntimeCommonResolvable — resolve hook behaviour', () => {
|
|
/** Install the fallback and return the resolve closure handed to registerHooks. */
|
|
async function captureResolve() {
|
|
const spy = vi.fn();
|
|
const mod = await loadResolver(spy);
|
|
mod.ensureOnnxRuntimeCommonResolvable();
|
|
return spy.mock.calls[0][0].resolve as (
|
|
s: string,
|
|
c: never,
|
|
n: (s: string, c: never) => unknown,
|
|
) => unknown;
|
|
}
|
|
|
|
it('passes a successful default resolution through unchanged (no-op on hoisted layouts)', async () => {
|
|
const resolve = await captureResolve();
|
|
const real = { url: 'file:///real/onnxruntime-common/index.js', shortCircuit: true };
|
|
const next = vi.fn(() => real);
|
|
|
|
const res = resolve('onnxruntime-common', ctx, next);
|
|
|
|
expect(next).toHaveBeenCalledTimes(1);
|
|
expect(res).toBe(real); // the real resolution, NOT a redirect
|
|
});
|
|
|
|
it('redirects onnxruntime-common to the gitnexus copy when default resolution fails', async () => {
|
|
const resolve = await captureResolve();
|
|
const next = vi.fn(() => {
|
|
throw moduleNotFound();
|
|
});
|
|
|
|
const res = resolve('onnxruntime-common', ctx, next) as { url: string; shortCircuit: boolean };
|
|
|
|
expect(res.shortCircuit).toBe(true);
|
|
// The real resolved onnxruntime-common in node_modules (require.resolve runs
|
|
// for real here) — not just any path containing the substring.
|
|
expect(res.url).toMatch(/^file:\/\/.*\/node_modules\/onnxruntime-common\/.*\.js$/);
|
|
});
|
|
|
|
it('never masks an unrelated resolution failure (other specifiers rethrow)', async () => {
|
|
const resolve = await captureResolve();
|
|
const err = moduleNotFound();
|
|
const next = vi.fn(() => {
|
|
throw err;
|
|
});
|
|
|
|
expect(() => resolve('some-other-package', ctx, next)).toThrow(err);
|
|
});
|
|
|
|
it('rethrows when onnxruntime-common fails for a non-absence reason', async () => {
|
|
const resolve = await captureResolve();
|
|
// A present-but-otherwise-broken resolution (not a missing package) must
|
|
// surface, not be silently papered over with gitnexus' copy.
|
|
const err = Object.assign(new Error('bad specifier'), {
|
|
code: 'ERR_INVALID_MODULE_SPECIFIER',
|
|
});
|
|
const next = vi.fn(() => {
|
|
throw err;
|
|
});
|
|
|
|
expect(() => resolve('onnxruntime-common', ctx, next)).toThrow(err);
|
|
});
|
|
});
|