GitNexus/gitnexus/test/unit/swift-conditional-directive-preprocess.test.ts
Karl Lehenbauer 639eb04b31
fix(swift): preprocess indented conditional directives so class bodies survive parsing (#2771)
* fix(swift): preprocess indented conditional directives so class bodies survive parsing

* fix(swift): make conditional-directive blanking comment-, string- and brace-aware (#2771)

Addresses the review findings on PR #2771. The transform fired
unconditionally, which turned valid Swift into parse errors while missing the
most common shape it was written for.

- The blank/keep decision now consults `blockCommentDepth`, so `  #endif */` —
  the result of commenting out a conditional block — keeps its comment
  terminator. Previously `hasError` went raw=false -> preprocessed=true and the
  rest of the file was swallowed.
- The decision keys on the scanner's brace depth instead of indentation. A
  column-0 `#if` inside a class body is blanked (6 of 7 body shapes previously
  still lost the enclosing declaration) and an indented file-scope directive is
  not — matching what the doc comment already claimed. Bare-CR line endings,
  NBSP/ideographic indentation and a leading BOM are recognized too.
- A group is blanked only when every branch is brace-balanced. An `#if`/`#else`
  that splits a declaration header leaves one unmatched `{` once both branches
  survive, which collapsed five top-level nodes into one and gave unrelated
  types fabricated `NetworkClient.` qualified names. Such a group now degrades
  to the pre-fix behavior.
- Multiline strings honour `\"""` escapes, and a plain `"""` closes even when a
  `#` follows it, so the scanner no longer wedges in string state and silently
  stops blanking for the rest of the file.
- The pound run is counted once per position and skipped. It was quadratic:
  10.6s for one 64k-`#` line, well inside the 512 KB walker limit.
- Extended regex literals (`#/.../#`) no longer open a phantom block comment.
- Directive-free files return early, matching `stripUeMacros`.

Worker parity: `emitSwiftScopeCaptures` and `emitCppScopeCaptures` re-apply
their provider's `preprocessSource` on the parse-cache-miss path — Dart already
did this — and the embedding parse in `ensureAndParse` applies the hook as
well. Before this the worker and the scope-capture/embedding halves analyzed
different programs, turning a consistent degradation into cold-run/warm-run
non-determinism. A new parity test pins the equivalence for every provider that
defines the hook.

SCHEMA_BUMP 37 -> 38: this changes parse semantics, the chunk key hashes raw
on-disk bytes, and `preprocessSource` runs after the key is computed — so a
same-package-version warm cache would replay pre-fix Swift results verbatim,
including across `--force`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(ingestion): apply preprocessSource once in the scope bridge (#2771)

Follow-up cleanup on the review fixes. The previous commit re-applied each
provider's `preprocessSource` inside `emitSwiftScopeCaptures` and
`emitCppScopeCaptures`, mirroring what Dart already did — three copies of the
same rule, and a contract that asked every future emitter to remember it.

`extractParsedFile` is the single funnel every `emitScopeCaptures` caller
passes through (parse worker, scope-resolution run, Vue script extraction), and
it already receives the provider. Applying the hook there on the cache-miss
path covers all three languages and every future one, names no language in
shared code, and drops Dart's unconditional transform on the cache-hit path.
Verified the three emitters use `sourceText` for nothing but the parse, so the
substitution is output-identical — which the parity test asserts directly.

Also from the cleanup pass:

- the parity test derives its language list from the provider registry, so a
  new provider adopting the hook fails until it adds a fixture
- `ensureAndParse` resolves the provider from the language it already computed,
  instead of a second extension table (`getProviderForFile`)
- the preprocessor returns `sourceText` unchanged when no group was blanked,
  which is the common case for files whose only directives are top-level
- `split(/(\r\n|\n|\r)/)` replaces the hand-rolled line splitter, and the
  per-group brace bookkeeping is two scalars instead of an array
- the hint regex is derived from the line regex so the two cannot drift
- unit assertions compare the WHOLE preprocessed file against the expected
  blanking, replacing per-line spot checks; the pipeline tests share one
  `runFixture` helper and `getNodesForFile` in the resolver test helpers
- `LanguageProvider.preprocessSource` documents the real call sites and says
  plainly that the set is not closed — `populateRangeBindings` still hands
  language helpers raw text

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(autofix): apply prettier + eslint fixes via /autofix command

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-01 20:10:58 +00:00

334 lines
9.9 KiB
TypeScript
Raw Permalink Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

import { describe, expect, it } from 'vitest';
import { preprocessSwiftConditionalDirectives } from '../../src/core/ingestion/languages/swift/conditional-directive-preprocess.js';
/**
* Assert the WHOLE output: `source` with exactly `blankedLines` replaced by
* spaces of the same width. Widths come from the source line, so a wrong-length
* blank fails, and an unexpected blank anywhere else fails too.
*/
function expectBlanked(source: string, blankedLines: readonly number[], separator = '\n'): void {
const lines = source.split(separator);
expect(preprocessSwiftConditionalDirectives(source).split(separator)).toEqual(
lines.map((line, index) => (blankedLines.includes(index) ? ' '.repeat(line.length) : line)),
);
}
describe('Swift conditional-directive preprocessing', () => {
it('blanks every directive of a nested group and leaves the top-level one alone', () => {
const source = [
'#if os(macOS)',
'class TopLevel {}',
'#endif',
'class Outer {',
' #if os(iOS) // platform branch',
' enum A { case x }',
'\t#elseif DEBUG && canImport(UIKit) // fallback',
' enum B { case y }',
' #else',
' enum C { case z }',
' #endif // end branch',
'}',
].join('\n');
expectBlanked(source, [4, 6, 8, 10]);
});
it('preserves JavaScript string length and newline count', () => {
const source = '#if DEBUG\nclass Outer {\n #else\n}\n#endif\n';
const rewritten = preprocessSwiftConditionalDirectives(source);
expect(rewritten).toHaveLength(source.length);
expect(rewritten.match(/\n/g)?.length ?? 0).toBe(source.match(/\n/g)?.length ?? 0);
expectBlanked(source, []);
});
it('returns directive-free Swift source unchanged', () => {
const source = 'class Plain {\n var value: Int = 0\n func read() -> Int { value }\n}\n';
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
it('preserves CRLF line endings and offsets', () => {
const source = 'class Outer {\r\n\t#if os(iOS)\r\n\tenum A { case x }\r\n\t#endif\r\n}\r\n';
const rewritten = preprocessSwiftConditionalDirectives(source);
expect(rewritten).toHaveLength(source.length);
expect(rewritten.indexOf('enum A')).toBe(source.indexOf('enum A'));
expectBlanked(source, [1, 3], '\r\n');
});
it('treats a bare carriage return as a line terminator', () => {
const source = 'class Outer {\r #if os(iOS)\r enum A { case x }\r #endif\r}\r';
expect(preprocessSwiftConditionalDirectives(source)).toHaveLength(source.length);
expectBlanked(source, [1, 3], '\r');
});
it('leaves regular and raw multiline string interiors byte-identical', () => {
const source = [
'struct Strings {',
' let regular = """',
' #if os(iOS)',
' #elseif DEBUG',
' #else',
' #endif',
' """',
' #if REAL_DIRECTIVE',
' let between = true',
' #endif',
' let raw = #"""',
' #if raw(iOS)',
' #elseif raw(DEBUG)',
' #else',
' #endif',
' """#',
' let doubleRaw = ##"""',
' #if double-raw-string-data',
' #endif',
' """##',
'}',
].join('\n');
expectBlanked(source, [7, 9]);
});
it('does not let an unterminated multiline string blank later lines', () => {
const source = ['let text = """', ' #if this-is-string-data', ' still string data'].join(
'\n',
);
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
it('leaves non-conditional hash directives untouched', () => {
const source = [
'class Directives {',
' #warning("warning")',
' #error("error")',
' #available(iOS 17, *)',
' #selector(getter: Directives.value)',
' #if DEBUG',
' #endif',
'}',
].join('\n');
expectBlanked(source, [5, 6]);
});
it('keeps nested block comments out of string state and never blanks inside them', () => {
const source = [
'/*',
' #if in-comment',
' /* nested comment */',
' #endif',
'*/',
'let text = """',
' #if in-string',
'"""',
].join('\n');
expectBlanked(source, []);
});
it('keeps a block-comment terminator that shares its line with a directive', () => {
const source = [
'class Foo {',
' /* temporarily disabled:',
' #if DEBUG',
' func f() {}',
' #endif */',
' func g() {}',
'}',
].join('\n');
// Blanking ` #endif */` would un-terminate the comment and swallow `g()`.
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
it('blanks a column-zero directive nested inside a class body', () => {
const source = [
'class Outer {',
' enum A { case x }',
'#if os(iOS)',
' enum B { case y }',
'#endif',
'}',
].join('\n');
expectBlanked(source, [2, 4]);
});
it('leaves an indented directive that is still at file scope intact', () => {
const source = [' #if DEBUG', ' struct Debugged {}', ' #endif', ''].join('\n');
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
it('recognizes non-ASCII indentation and a leading byte-order mark', () => {
const nbspSource = [
'class Outer {',
' #if os(iOS)',
' enum A { case x }',
' #endif',
'}',
].join('\n');
const bomSource = `class Outer {\n #if os(iOS)\n enum A { case x }\n #endif\n}\n`;
expectBlanked(nbspSource, [1, 3]);
expectBlanked(bomSource, [1, 3]);
});
it('refuses to blank a group whose branches split a declaration header', () => {
const source = [
'class NetworkClient {',
' #if swift(>=5.5)',
' func fetch() async {',
' #else',
' func fetch() {',
' #endif',
' perform()',
' }',
'}',
'struct SessionStore {}',
].join('\n');
// Both branch bodies open a brace and only one closes; blanking would leave
// `NetworkClient` unterminated and re-parent `SessionStore` under it.
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
it('blanks nested balanced groups at every level', () => {
const source = [
'class Outer {',
' #if os(iOS)',
' func inner() {',
' #if DEBUG',
' log()',
' #endif',
' }',
' #endif',
'}',
].join('\n');
expectBlanked(source, [1, 3, 5, 7]);
});
it('lets an unbalanced nested group also block its enclosing group', () => {
const source = [
'class Outer {',
' #if os(iOS)',
' func inner() {',
' #if DEBUG',
' if x {',
' #else',
' if y {',
' #endif',
' log()',
' }',
' }',
' #endif',
'}',
].join('\n');
// Both `if` branches survive blanking, so the enclosing branch is +1 too.
// Conservative propagation degrades the whole nest to pre-fix behavior.
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
it('does not wedge on an escaped triple quote inside a multiline string', () => {
const source = [
'class Outer {',
' let text = """',
' escaped \\""" still string data',
' #if in-string',
' """',
' #if REAL_DIRECTIVE',
' func after() {}',
' #endif',
'}',
].join('\n');
expectBlanked(source, [5, 7]);
});
it('closes a plain multiline string whose terminator is followed by a pound', () => {
const source = [
'class Outer {',
' let text = """',
' body',
' """#hashAfterClose',
' #if REAL_DIRECTIVE',
' func after() {}',
' #endif',
'}',
].join('\n');
expectBlanked(source, [4, 6]);
});
it('closes a raw multiline string terminated by extra pounds', () => {
const source = [
'class Outer {',
' let raw = #"""',
' body',
' """##',
' #if REAL_DIRECTIVE',
' func after() {}',
' #endif',
'}',
].join('\n');
expectBlanked(source, [4, 6]);
});
it('does not let an extended regex literal open a phantom block comment', () => {
const source = [
'class Outer {',
' let pattern = #/a/*b/#',
' #if REAL_DIRECTIVE',
' func after() {}',
' #endif',
'}',
].join('\n');
expectBlanked(source, [2, 4]);
});
it('stays linear on a long run of bare pound signs', () => {
// The pre-fix scanner re-walked the whole run at every index: ~10.6s for
// n=64000. The bound is a per-test timeout rather than a measured
// elapsed-time assertion; the fixed scanner runs this in ~1ms.
const source = `class Outer {\n let s = ${'#'.repeat(64000)}\n #if REAL_DIRECTIVE\n func after() {}\n #endif\n}\n`;
expectBlanked(source, [2, 4]);
}, 2000);
it('preserves JavaScript length on a directive carrying non-ASCII comment text', () => {
const source = ['class Outer {', ' #if os(iOS) // 日本語 🔥', ' #endif', '}'].join('\n');
const rewritten = preprocessSwiftConditionalDirectives(source);
// UTF-16 length is preserved; UTF-8 byte length is not (56 -> 48).
// Documented as safe because no consumer slices the original bytes by
// `startIndex` — node-tree-sitter reports UTF-16 code-unit indices.
expect(rewritten).toHaveLength(source.length);
expect(Buffer.byteLength(source, 'utf8')).toBe(56);
expect(Buffer.byteLength(rewritten, 'utf8')).toBe(48);
expectBlanked(source, [1, 2]);
});
it('is idempotent', () => {
const source = ['class Outer {', ' #if os(iOS)', ' enum A { case x }', ' #endif', '}'].join(
'\n',
);
const once = preprocessSwiftConditionalDirectives(source);
expect(preprocessSwiftConditionalDirectives(once)).toBe(once);
});
it('leaves an unmatched directive untouched', () => {
const source = ['class Outer {', ' #endif', ' #if NEVER_CLOSED', '}'].join('\n');
expect(preprocessSwiftConditionalDirectives(source)).toBe(source);
});
});