From fb49613a4d423a6a8f98ba3f47a6c59a22e87457 Mon Sep 17 00:00:00 2001 From: DuduPhudu <34869259+ReidenXerx@users.noreply.github.com> Date: Thu, 27 Aug 2026 15:29:05 +0300 Subject: [PATCH] fix(ingestion): ignore emitted Next.js build output, and delete the inert public/build entry (#3018) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(ingestion): ignore emitted Next.js build output, and restore the dead public/build entry `DEFAULT_IGNORE_LIST` contained `.next` — the build CACHE — but not `_next`, the emitted OUTPUT, which are different directories. A Capacitor/Cordova shell copies a built Next.js bundle to `/app/src/main/assets/public/_next/static/`, where no path segment hits the list, so the walker indexed the bundle as source. On a real mobile-wrapped Next.js app that was 256 minified chunk files, and every `Route` node the repo produced pointed at a webpack chunk rather than at source. The filename heuristics did not catch them either: they match `.bundle.`, `.chunk.`, `.generated.` and `.d.ts`, while Next.js emits hashed names like `6862-9d1cdcb99f169a06.js`. Separately, `'public/build'` had been sitting in `DEFAULT_IGNORE_LIST` matching nothing at all. That set is tested one path SEGMENT at a time, and is also read by `isHardcodedIgnoredDirectory(name)`, which receives a bare directory name — so a slash-containing member can never compare equal to anything. Rather than delete the entry and lose its intent, multi-segment paths now live in `DEFAULT_IGNORED_PATH_FRAGMENTS` and are matched against the whole path, so Remix / Laravel Mix asset output is ignored as originally intended. A guard test pins the invariant that made the dead entry possible: no member of the name set may contain a slash. Measured against a production Capacitor-wrapped Next.js app (1558 JS/TS files on disk): 256 newly ignored, none of them under `src/`, and zero files that were previously ignored become indexed. Closes #3007 Co-Authored-By: Claude Opus 5 (1M context) * fix(ingestion): drop the inert public/build machinery, discriminate _next by segment, ignore _next on the web upload path Addresses the review findings on #3018. Remove DEFAULT_IGNORED_PATH_FRAGMENTS, hasIgnoredPathFragment and its shouldIgnorePath branch. The mechanism was correct but unreachable: all four of its match forms put a `/` or end-of-string on both sides of `build`, so a fragment match strictly implies `build` is a whole segment, which the per-segment DEFAULT_IGNORE_LIST loop already catches one branch earlier. Measured over 768,420 generated paths: 65,506 fragment matches, 0 of them decisive, 0 implication violations. `'public/build'` really was an inert member of the name set, but its paths were never unignored — bare `'build'` covered them on both sides — so the entry is deleted rather than relocated, which is the other option #3007 offered. The slash-free guard test stays; it is what stops the next slash-bearing entry from dying the same way. Add negative cases pinning that `_next` matches as a whole path segment. The previous suite could not tell a segment rule from a substring rule: replacing the entry with `normalizedPath.includes('_next')` passed all five tests, while eating `src/_nextgen/index.ts`. Rename the public/build test to what it actually pins — that deleting the inert entry changed no behavior — since it is green on both sides by design. Add `_next` to the web upload filter's EXCLUDED_DIRS. That list is the live browser ingestion path (RepoAnalyzer -> filterRepoFiles -> /api/analyze/upload) and had `.next` but not `_next`, so a Capacitor-wrapped Next.js app uploaded its entire minified tree against the server's 20000-file / 250MB caps for files the analyzer then discards. Co-Authored-By: Claude Opus 5 (1M context) * test(ignore-service): make the single-component set guards able to fail The slash-free guard added for #3007 could not fail. It selected entry lines with startsWith("'") and read only the first quoted token per line, so 'public/build' could return as a backtick string, behind an inline block comment, as a second entry on an existing line, or via .add() and every test stayed green. Prettier and eslint miss the backtick and inline-comment forms too, so CI did not catch them either. U1: remove the duplicate '.serverless' entry so the set can be pinned to one exact number. A Set discarded it, so no ignore behaviour changes. U2/U3: replace the line-based parser with a shared single-pass scanner in test/helpers/ignore-set-source.ts, and extend the guard from DEFAULT_IGNORE_LIST to IGNORED_FILES, ROOT_ARTIFACT_DIRECTORIES and IGNORED_EXTENSIONS, which share the same single-component match contract. The scanner tracks string and comment state together because neither can be removed first: the ignore-list comments quote paths and carry an apostrophe, so matching literals before stripping comments yields phantom slash-bearing entries; and a glob string containing a comment-open sequence makes regex comment-stripping swallow the closing bracket. Only a single pass is correct in both directions. Counts are pinned exactly rather than floored — a floor cannot protect a two-member set and hides a partial parse. Shapes a source parser cannot resolve (spread, interpolation, concatenation, later .add) now throw instead of quietly reporting fewer members, and the parsed names are cross-checked against isHardcodedIgnoredDirectory so parser drift fails without exporting the set. Verified by mutation: all six fail-open spellings now turn the suite red; 187 tests pass, tsc clean. * test(ignore-service): pin that _next prunes the directory, not just its files Every measured benefit of ignoring _next comes from never enumerating the bundle tree, and no file list can observe that: anything under _next is rejected whether the walk pruned the directory or descended and rejected each file. childrenIgnored is the only observation that separates them. The existing build-output tests all call shouldIgnorePath, the leaf predicate, so a refactor moving _next to a shouldIgnorePath-only rule would keep them green while silently restoring the full walk. These assertions close that. Also pins that _next matches as a whole segment (_nextgen and my_next are still walked), and that the `!_next/` negation recovers the directory at any depth — the bare form is the one that works, since `!_next/**` alone never gets tested: childrenIgnored prunes the directory before any descendant pattern is reached. Placed in the .gitnexusignore-negation describe block, which owns mkPath and the tmpdir fixture and is registered in scripts/cross-platform-tests.ts. Verified by mutation: disabling only the pruning branch in childrenIgnored leaves the build-output suite at 26/26 green and turns these assertions red. * test(ignore-service): guard the twin build-output ignore lists against drift _next now lives in two lists in two packages — the analyzer's DEFAULT_IGNORE_LIST and the browser upload filter's EXCLUDED_DIRS — with nothing tying them together. This is the seventh twin-list pair in this repo; the header of receiver-twin-list-drift.test.ts records that the previous ones each shipped a bug when one side moved. Containment runs web -> CLI only, and that is the load-bearing direction: the browser filter decides what the server ever sees, and it reads no .gitnexusignore, so a name it drops that the analyzer would have indexed is silent source loss with no recovery. The reverse is not an error — the analyzer prunes far more aggressively than an upload needs to. .gitnexus is the one exemption and has a mechanism: the walker passes dot: false to glob, so it never enumerates dot-directories. Asserted in both directions so re-adding it to the CLI list or dropping it from the web list both fail. Both sides are source-parsed through the shared helper. DEFAULT_IGNORE_LIST is module-private, and no test in this package imports across the package boundary — every cross-package precedent reads source instead. Also corrects the documentation this PR's comments got wrong: the guard test is cited by path rather than as "below", the unreproducible per-repo percentage is gone, the reason _next is deliberately unanchored is recorded next to the entry (no /_next form matches a root-level _next/static/…), and the upload filter now states that it consults no repository ignore rules — so unlike the CLI, a negation cannot recover what it drops. Verified by mutation: a web-only addition and a CLI removal each turn the guard red. 194 targeted tests pass; tsc clean in both packages. * refactor(test): read the ignore sets with the TypeScript parser, not a hand-rolled scanner The guards read ignore-service.ts as source because the sets are module-private. The first pass hand-rolled a character scanner to do it, and the repo already vendors the right tool: ts.createSourceFile, used this way in literal-collectors, query-determinism-guard, cli-index-help and group/sync-partial-extraction. The scanner had two silent gaps a real parser does not have: - It rejected `${` by substring, but template literals were consumed whole, so that branch could never fire and an interpolated member was accepted as a literal — the exact under-report the file refused to allow. - It took the first `[` after the marker, which on a type-annotated declaration (`readonly string[] = ...`) is the annotation's empty pair. It returned [] with no throw, which would make every assertion in a suite vacuously true. This is the hazard receiver-twin-list-drift.test.ts documents having hit. Reading the declaration node removes both, along with the comment-vs-string ordering problem that motivated the scanner: a parser cannot mistake a comment for a string or a glob's `/*` for a comment-open. Also drops the four pinned exact counts. They were a ratchet — these sets are edited by unrelated PRs, each of which would have failed a count assertion about nothing it touched — and with a real parser the partial-parse hazard they existed to catch cannot happen silently: a member that is not a plain string literal throws. Markers collapse to set names, and the duplicated path-resolution boilerplate moves into the helper the two suites already share. Net 187 deletions against 123 insertions. Verified by mutation: backtick, inline comment, same-line, double-quote, duplicate, interpolation, spread and runtime .add() are all caught; a type-annotated declaration now reads correctly instead of returning empty. 194 tests pass, tsc clean. --------- Co-authored-by: Claude Opus 5 (1M context) Co-authored-by: Gergo Magyar --- gitnexus-web/src/lib/upload-filter.test.ts | 16 +++ gitnexus-web/src/lib/upload-filter.ts | 11 ++ gitnexus/src/config/ignore-service.ts | 23 +++- gitnexus/test/helpers/ignore-set-source.ts | 107 +++++++++++++++ .../test/unit/ignore-build-output.test.ts | 127 ++++++++++++++++++ gitnexus/test/unit/ignore-service.test.ts | 32 +++++ .../unit/upload-filter-ignore-drift.test.ts | 90 +++++++++++++ 7 files changed, 404 insertions(+), 2 deletions(-) create mode 100644 gitnexus/test/helpers/ignore-set-source.ts create mode 100644 gitnexus/test/unit/ignore-build-output.test.ts create mode 100644 gitnexus/test/unit/upload-filter-ignore-drift.test.ts diff --git a/gitnexus-web/src/lib/upload-filter.test.ts b/gitnexus-web/src/lib/upload-filter.test.ts index e97b9ec2c..1943720ba 100644 --- a/gitnexus-web/src/lib/upload-filter.test.ts +++ b/gitnexus-web/src/lib/upload-filter.test.ts @@ -31,6 +31,22 @@ describe('filterRepoFiles', () => { expect(r.droppedCount).toBe(4); }); + it('excludes emitted _next output, including the Capacitor/Cordova copy', () => { + // `.next` was listed but `_next` was not, so a mobile-wrapped Next.js app + // uploaded its whole minified bundle against the server's caps for files + // the analyzer then discards anyway (#3007). + const input = [ + f('repo/android/app/src/main/assets/public/_next/static/chunks/main.js'), + f('repo/ios/App/App/public/_next/static/chunks/framework.js'), + f('repo/_next/static/chunks/x.js'), + f('repo/src/index.ts'), + f('repo/src/_nextgen/index.ts'), + ]; + const r = filterRepoFiles(input); + expect(r.manifest).toEqual(['repo/src/index.ts', 'repo/src/_nextgen/index.ts']); + expect(r.droppedCount).toBe(3); + }); + it('drops files over the per-file size cap', () => { const input = [f('repo/big.bin', MAX_FILE_BYTES + 1), f('repo/small.ts', 10)]; const r = filterRepoFiles(input); diff --git a/gitnexus-web/src/lib/upload-filter.ts b/gitnexus-web/src/lib/upload-filter.ts index a24520f74..9b3fb30db 100644 --- a/gitnexus-web/src/lib/upload-filter.ts +++ b/gitnexus-web/src/lib/upload-filter.ts @@ -22,6 +22,17 @@ export const EXCLUDED_DIRS = new Set([ 'build', 'out', '.next', + // `.next` is the build CACHE, `_next` the EMITTED output — different + // directories. A Capacitor/Cordova shell leaves the emitted bundle at + // `/app/src/main/assets/public/_next/`, so without this the whole + // minified tree is uploaded against the server's file/byte caps only to be + // discarded by the analyzer's own ignore list (#3007). + // + // This pre-filter reads no repository ignore rules, so unlike the CLI walker + // a `.gitnexusignore` negation cannot recover anything dropped here. Names + // added below must therefore stay a subset of the analyzer's own list; see + // `gitnexus/test/unit/upload-filter-ignore-drift.test.ts`. + '_next', '.nuxt', '.cache', 'coverage', diff --git a/gitnexus/src/config/ignore-service.ts b/gitnexus/src/config/ignore-service.ts index 645754427..f1b9e1150 100644 --- a/gitnexus/src/config/ignore-service.ts +++ b/gitnexus/src/config/ignore-service.ts @@ -58,13 +58,33 @@ const DEFAULT_IGNORE_LIST = new Set([ 'obj', 'target', // Java/Rust '.next', + // `.next` is Next.js's build CACHE; `_next` is the EMITTED output, and the two + // are different directories. A Capacitor/Cordova shell copies the emitted + // bundle to `/app/src/main/assets/public/_next/static/…`, where none + // of the path segments hit this list — so a mobile-wrapped Next.js app had its + // shipped bundle indexed as source, and every Route node it produced pointed at + // a webpack chunk rather than code anyone wrote (#3007). + // + // The name is deliberately unanchored. No `/_next` form matches a + // root-level `_next/static/…`, which is the shape the reported repo has, so + // anchoring it would miss the case it was added for. The accepted cost is a + // hand-written directory literally named `_next`; recover one with a bare + // `!_next/` line in `.gitnexusignore`. + '_next', '.nuxt', '.output', '.vercel', '.netlify', '.serverless', '_build', - 'public/build', + // `'public/build'` used to sit here. This set is tested one path SEGMENT at a + // time, and `isHardcodedIgnoredDirectory(name)` takes a bare directory name, + // so a slash-containing member could never match either — it was inert. Its + // paths were never unignored though: bare `'build'` above already prunes + // `public/build/**`, so removing the entry changes no behavior (#3007). + // `test/unit/ignore-build-output.test.ts` keeps the next slash-bearing entry + // in this set — or in IGNORED_FILES, ROOT_ARTIFACT_DIRECTORIES or + // IGNORED_EXTENSIONS — from dying the same way. '.parcel-cache', '.turbo', '.svelte-kit', @@ -95,7 +115,6 @@ const DEFAULT_IGNORE_LIST = new Set([ // remains covered by .gitignore/.gitnexusignore and the unambiguous names. 'monaco-workers', // Monaco editor web-worker bundles generated for browser runtime '.terraform', - '.serverless', // Documentation (optional - might want to keep) // 'docs', diff --git a/gitnexus/test/helpers/ignore-set-source.ts b/gitnexus/test/helpers/ignore-set-source.ts new file mode 100644 index 000000000..04db55dd6 --- /dev/null +++ b/gitnexus/test/helpers/ignore-set-source.ts @@ -0,0 +1,107 @@ +/** + * Reads the bare-name sets in `src/config/ignore-service.ts` out of source. + * + * Those sets are module-private, and exporting them purely to be testable would + * widen a production surface to satisfy a test — the call + * `receiver-twin-list-drift.test.ts` documents. So the guards read the source + * instead, through the TypeScript parser the repo already vendors and already + * uses this way (`literal-collectors.ts`, `query-determinism-guard.test.ts`, + * `cli-index-help.test.ts`). + * + * Using the real parser is what makes the guards trustworthy. A text scanner has + * to decide whether a delimiter opens a comment or sits inside a string, and it + * gets that wrong in both directions here: the ignore-list comments quote paths + * and carry an apostrophe (`Next.js's`), while a glob string such as `'** / *'` + * contains a comment-open sequence. It also has to guess which bracket belongs + * to the declaration rather than to a type annotation. Each of those is a way to + * silently read fewer members — and a guard that quietly stops seeing members is + * the exact defect these guards exist to catch. + * + * `setEntries` therefore refuses anything that is not a plain list of string + * literals, rather than skipping the members it cannot resolve. + */ +import { readFileSync } from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import ts from 'typescript'; + +const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..', '..'); + +/** The analyzer's ignore rules — the sets every guard in this family reads. */ +export const IGNORE_SERVICE_PATH = path.join( + REPO_ROOT, + 'gitnexus', + 'src', + 'config', + 'ignore-service.ts', +); + +/** The browser upload pre-filter, whose excluded-directory set must not drift from the above. */ +export const UPLOAD_FILTER_PATH = path.join( + REPO_ROOT, + 'gitnexus-web', + 'src', + 'lib', + 'upload-filter.ts', +); + +export const readSource = (file: string): string => readFileSync(file, 'utf8'); + +/** + * The string literals `setName` is constructed from, in declaration order. + * + * Throws — never returns a short list — when the declaration is missing or holds + * anything other than plain string literals (a spread, an interpolation, a + * concatenation, a computed value). + */ +export const setEntries = (source: string, setName: string): string[] => { + const sourceFile = ts.createSourceFile( + 'ignore-set-source.ts', + source, + ts.ScriptTarget.Latest, + true, + ); + + let elements: ts.NodeArray | undefined; + const visit = (node: ts.Node): void => { + if ( + elements === undefined && + ts.isVariableDeclaration(node) && + ts.isIdentifier(node.name) && + node.name.text === setName && + node.initializer !== undefined && + ts.isNewExpression(node.initializer) && + node.initializer.arguments?.length === 1 && + ts.isArrayLiteralExpression(node.initializer.arguments[0]) + ) { + elements = node.initializer.arguments[0].elements; + return; + } + ts.forEachChild(node, visit); + }; + visit(sourceFile); + + if (elements === undefined) { + throw new Error(`${setName} is not declared as \`new Set([...])\` — update this test`); + } + + const unresolvable = elements.filter((element) => !ts.isStringLiteral(element)); + if (unresolvable.length > 0) { + throw new Error( + `${setName} holds ${unresolvable.length} member(s) that are not plain string literals ` + + `(first: \`${unresolvable[0].getText(sourceFile)}\`). A source-reading guard cannot resolve ` + + `those, so switch this set to a runtime assertion rather than letting the guard see fewer members.`, + ); + } + + return elements.map((element) => (element as ts.StringLiteral).text); +}; + +/** + * True when `setName` is mutated by `.add(...)` anywhere in `source`. + * + * `setEntries` reads the declaration only, so a member appended afterwards would + * be invisible to it. The guards assert this is false rather than under-reporting. + */ +export const hasRuntimeAdd = (source: string, setName: string): boolean => + new RegExp(`\\b${setName}\\s*\\.\\s*add\\s*\\(`).test(source); diff --git a/gitnexus/test/unit/ignore-build-output.test.ts b/gitnexus/test/unit/ignore-build-output.test.ts new file mode 100644 index 000000000..7e8a87a29 --- /dev/null +++ b/gitnexus/test/unit/ignore-build-output.test.ts @@ -0,0 +1,127 @@ +import { describe, it, expect } from 'vitest'; +import { isHardcodedIgnoredDirectory, shouldIgnorePath } from '../../src/config/ignore-service.js'; +import { + IGNORE_SERVICE_PATH, + hasRuntimeAdd, + readSource, + setEntries, +} from '../helpers/ignore-set-source.js'; + +/** + * Emitted build output must not be indexed as source (#3007). + * + * `.next` (the build cache) was listed but `_next` (the emitted output) was + * not, so a Capacitor/Cordova shell that copies a Next.js bundle into + * `/app/src/main/assets/public/_next/static/` had its shipped bundle + * indexed as source — and every `Route` node the repo produced pointed at a + * webpack bundle instead of code anyone wrote. + */ + +describe('build-output ignores', () => { + it('ignores emitted _next output, including the Capacitor/Cordova copy', () => { + expect(shouldIgnorePath('_next/static/chunks/main.js')).toBe(true); + expect(shouldIgnorePath('.next/server/app/page.js')).toBe(true); + expect( + shouldIgnorePath( + 'android/app/src/main/assets/public/_next/static/chunks/6862-9d1cdcb99f169a06.js', + ), + ).toBe(true); + expect(shouldIgnorePath('ios/App/App/public/_next/static/chunks/framework-abc123.js')).toBe( + true, + ); + }); + + it('does not ignore ordinary source that merely mentions next', () => { + expect(shouldIgnorePath('src/next-steps.ts')).toBe(false); + expect(shouldIgnorePath('src/nextConfig/index.ts')).toBe(false); + expect(shouldIgnorePath('packages/next-auth/src/index.ts')).toBe(false); + }); + + it('matches _next as a whole segment, not as a substring', () => { + // Without these, `normalizedPath.includes('_next')` would satisfy every + // other assertion in this file — the suite could not tell a segment rule + // from a substring rule, and a substring rule would eat real source. + expect(shouldIgnorePath('src/_nextgen/index.ts')).toBe(false); + expect(shouldIgnorePath('packages/my_next/src/index.ts')).toBe(false); + expect(shouldIgnorePath('src/prefix_next.ts')).toBe(false); + }); + + it('keeps public/build ignored after the inert name-set entry was removed', () => { + // NOT a regression test for new behavior — it pins that DELETING the inert + // `'public/build'` entry changed nothing, because bare `'build'` matches + // these as an ordinary segment and always did. Green on both sides of the + // change by design; that is the point. + expect(shouldIgnorePath('public/build/entry.client.js')).toBe(true); + expect(shouldIgnorePath('apps/web/public/build/manifest.js')).toBe(true); + }); + + it('does not ignore public/ or build/-adjacent source outside that pair', () => { + expect(shouldIgnorePath('public/favicon-loader.ts')).toBe(false); + expect(shouldIgnorePath('src/public/api.ts')).toBe(false); + }); + + describe('single-component set guards', () => { + const source = readSource(IGNORE_SERVICE_PATH); + + // Every one of these sets is compared against a single path component, so a + // member containing `/` is dead on arrival — the defect that left + // `'public/build'` inert. + const SET_NAMES = [ + 'DEFAULT_IGNORE_LIST', + 'IGNORED_FILES', + 'ROOT_ARTIFACT_DIRECTORIES', + 'IGNORED_EXTENSIONS', + ] as const; + + it.each(SET_NAMES)('%s holds no slash-bearing member', (setName) => { + expect(setEntries(source, setName).filter((entry) => entry.includes('/'))).toEqual([]); + }); + + it.each(SET_NAMES)('%s holds no duplicate member', (setName) => { + const entries = setEntries(source, setName); + expect(new Set(entries).size).toBe(entries.length); + }); + + it.each(SET_NAMES)('%s is never mutated by .add() after construction', (setName) => { + expect(hasRuntimeAdd(source, setName)).toBe(false); + }); + + it('every IGNORED_EXTENSIONS member starts with a dot', () => { + const entries = setEntries(source, 'IGNORED_EXTENSIONS'); + expect(entries.filter((entry) => !entry.startsWith('.'))).toEqual([]); + }); + + it('reads entries the declaration holds, not text the comments quote', () => { + // The comments in DEFAULT_IGNORE_LIST quote paths and carry an apostrophe + // (`Next.js's`), so a text scanner reads phantom entries out of them — + // several slash-bearing, which would fail the assertion above on correct + // source. Parsing the declaration cannot see comments at all. + const entries = setEntries(source, 'DEFAULT_IGNORE_LIST'); + expect(entries).toContain('_next'); + expect(entries).not.toContain('public/build'); + expect(entries).not.toContain('env/'); + }); + + it('agrees with the runtime set it claims to describe', () => { + // Catches drift between what the guard reads and what the module does, + // without exporting the set. + const entries = setEntries(source, 'DEFAULT_IGNORE_LIST'); + expect(entries.filter((entry) => !isHardcodedIgnoredDirectory(entry))).toEqual([]); + }); + + it('fails loudly when a set is no longer declared as a Set of literals', () => { + expect(() => setEntries(source, 'NOT_A_REAL_SET')).toThrow(/update this test/); + }); + + it('refuses a declaration whose members it cannot resolve', () => { + // A spread, an interpolation, or a concatenation resolves at runtime, not + // in source. Reading the resolvable members and passing is the failure mode + // these guards exist to prevent, so the reader refuses instead. + const poisoned = source.replace( + 'const ROOT_ARTIFACT_DIRECTORIES = new Set([', + 'const ROOT_ARTIFACT_DIRECTORIES = new Set([...OTHER_NAMES,', + ); + expect(() => setEntries(poisoned, 'ROOT_ARTIFACT_DIRECTORIES')).toThrow(/not plain string/); + }); + }); +}); diff --git a/gitnexus/test/unit/ignore-service.test.ts b/gitnexus/test/unit/ignore-service.test.ts index 4fb5536ef..60d6e8f77 100644 --- a/gitnexus/test/unit/ignore-service.test.ts +++ b/gitnexus/test/unit/ignore-service.test.ts @@ -334,6 +334,38 @@ describe('.gitnexusignore negation overrides hardcoded DEFAULT_IGNORE_LIST (#771 expect(filter.childrenIgnored(mkPath('Env'))).toBe(false); }); + // `_next` has to prune the DIRECTORY, not merely reject each file underneath. + // Every measured benefit of ignoring it comes from never enumerating the + // bundle tree, and no file list can show the difference — anything under + // `_next` is rejected either way. `childrenIgnored` is the only observation + // that distinguishes them, so a refactor that moved `_next` to a + // `shouldIgnorePath`-only rule would keep the build-output suite green while + // silently restoring the full walk. + it('prunes emitted _next output as a directory, at any depth', async () => { + const filter = await createIgnoreFilter(tmpDir); + + expect(filter.childrenIgnored(mkPath('_next'))).toBe(true); + expect(filter.childrenIgnored(mkPath('android/app/src/main/assets/public/_next'))).toBe(true); + }); + + it('matches _next as a whole segment, so _nextgen source is still walked', async () => { + const filter = await createIgnoreFilter(tmpDir); + + expect(filter.childrenIgnored(mkPath('src/_nextgen'))).toBe(false); + expect(filter.childrenIgnored(mkPath('packages/my_next'))).toBe(false); + }); + + it('`!_next/` negation unlocks the emitted output directory at any depth', async () => { + // The bare form is the one that works. `!_next/**` alone is a silent no-op: + // `childrenIgnored` prunes the directory before any descendant pattern is + // ever tested, so no file underneath reaches `ignored`. + await fs.writeFile(path.join(tmpDir, '.gitnexusignore'), '!_next/\n'); + const filter = await createIgnoreFilter(tmpDir); + + expect(filter.childrenIgnored(mkPath('_next'))).toBe(false); + expect(filter.childrenIgnored(mkPath('android/app/src/main/assets/public/_next'))).toBe(false); + }); + it('prunes a nested env directory only when pyvenv.cfg identifies a virtual environment', async () => { await fs.mkdir(path.join(tmpDir, 'backend', 'env'), { recursive: true }); await fs.writeFile(path.join(tmpDir, 'backend', 'env', 'pyvenv.cfg'), 'home = python\n'); diff --git a/gitnexus/test/unit/upload-filter-ignore-drift.test.ts b/gitnexus/test/unit/upload-filter-ignore-drift.test.ts new file mode 100644 index 000000000..0eea8b06e --- /dev/null +++ b/gitnexus/test/unit/upload-filter-ignore-drift.test.ts @@ -0,0 +1,90 @@ +/** + * The drift guard for the twin build-output ignore lists (#3007 follow-up). + * + * TWO lists spell "do not index this directory", in two packages: + * + * - `DEFAULT_IGNORE_LIST` — gitnexus `src/config/ignore-service.ts`. The + * analyzer's own list, consulted for every path during the repository walk. + * - `EXCLUDED_DIRS` — gitnexus-web `src/lib/upload-filter.ts`. A client-side + * pre-filter that decides what a browser folder upload sends at all. + * + * `_next` was added to both in the same PR, one commit apart. Before it, neither + * carried the name — so #3007 was a shared omission rather than drift between + * them. What this test guards is the divergence that becomes possible now that + * the same name lives in two places with nothing tying them together. + * + * The containment runs web -> CLI only, and that direction is the load-bearing + * one: the browser filter decides what the server ever sees, so a name it drops + * that the analyzer would have indexed is silent source loss with no recovery — + * this pre-filter reads no `.gitnexusignore`, so a negation cannot bring the + * files back. The reverse direction is not an error: roughly sixty CLI-only + * names exist because the analyzer prunes far more aggressively than an upload + * needs to, and the walker's own `dot: false` already hides dot-directories from + * it. That asymmetry is why equality is not the assertion. + * + * `.gitnexus` is the one deliberate exception, and it has a mechanism rather + * than being an oversight: the CLI walker passes `dot: false` to glob + * (`src/core/ingestion/filesystem-walker.ts`), so it never enumerates + * dot-directories and does not need the name in its list. The browser filter has + * no equivalent and must name it. That exemption is asserted explicitly in both + * directions, so re-adding `.gitnexus` to the CLI list or dropping it from the + * web list both fail loudly. + * + * Both sides are read from source rather than imported. `DEFAULT_IGNORE_LIST` is + * module-private. `EXCLUDED_DIRS` is exported, but `upload-filter.ts` types its + * inputs with the DOM `File` interface, which does not resolve under this + * package's `lib: ["ES2022"] / types: ["node"]` — so importing it here would not + * typecheck. + */ +import { describe, it, expect } from 'vitest'; +import { + IGNORE_SERVICE_PATH, + UPLOAD_FILTER_PATH, + readSource, + setEntries, +} from '../helpers/ignore-set-source.js'; + +const cliNames = () => setEntries(readSource(IGNORE_SERVICE_PATH), 'DEFAULT_IGNORE_LIST'); +const webNames = () => setEntries(readSource(UPLOAD_FILTER_PATH), 'EXCLUDED_DIRS'); + +/** + * Names the browser filter may drop that the analyzer does not list. + * + * Only `.gitnexus`, and only because the CLI walker's `dot: false` makes the + * entry unnecessary there. A name added here must cite a comparable structural + * reason in the walker — this list is not a place to park a failing assertion. + */ +const WEB_ONLY_ALLOWLIST = ['.gitnexus']; + +describe('build-output ignore lists stay in agreement across packages', () => { + it('parses both lists — neither is silently empty', () => { + // Guards the guard: a parser that read nothing would make every containment + // assertion below vacuously true. + expect(cliNames().length).toBeGreaterThan(20); + expect(webNames().length).toBeGreaterThan(5); + }); + + it('every name the browser filter drops is one the analyzer also ignores', () => { + const cli = new Set(cliNames()); + const unmatched = webNames().filter( + (name) => !cli.has(name) && !WEB_ONLY_ALLOWLIST.includes(name), + ); + expect(unmatched).toEqual([]); + }); + + it('carries the documented web-only exemption, and only that one', () => { + // Asserted separately from the containment above so the intent survives if + // that assertion is ever relaxed. + expect(webNames()).toContain('.gitnexus'); + expect(cliNames()).not.toContain('.gitnexus'); + }); + + it('shares the build-output names the reported bug was about', () => { + const cli = new Set(cliNames()); + const web = new Set(webNames()); + for (const name of ['_next', '.next', 'dist', 'build', 'out']) { + expect(web.has(name), `${name} missing from the browser upload filter`).toBe(true); + expect(cli.has(name), `${name} missing from the analyzer ignore list`).toBe(true); + } + }); +});