diff --git a/.github/workflows/ci-tests.yml b/.github/workflows/ci-tests.yml index 642b42d91..411e6baa8 100644 --- a/.github/workflows/ci-tests.yml +++ b/.github/workflows/ci-tests.yml @@ -1055,6 +1055,14 @@ jobs: run: node --import tsx bench/scope-capture/measure.mjs --check working-directory: gitnexus + - name: PL/SQL parse fingerprint + scaling guard + if: ${{ !cancelled() }} + # Build-free: asserts extractPlSqlFacts output over the plsql-* fixtures + # is unchanged (fingerprint) and the warmed ANTLR parse stays linear in + # package-body size (scaling < 1.5). Timings are reported, not gated. + run: node --import tsx bench/plsql-parse/measure.mjs --check + working-directory: gitnexus + - name: Rename output construction scaling guard if: ${{ !cancelled() }} run: node --import tsx bench/rename/measure.mjs --check diff --git a/.prettierignore b/.prettierignore index c85876d08..b067eb682 100644 --- a/.prettierignore +++ b/.prettierignore @@ -1,6 +1,8 @@ dist/ coverage/ gitnexus/vendor/ +# ANTLR4-generated PL/SQL lexer/parser (regenerate, never hand-edit). +gitnexus/src/core/ingestion/plsql/generated/ gitnexus/test/fixtures/ # Line/column-sensitive callable identity and taint regression. gitnexus/test/integration/cfg/fixtures/inline-taint-repo/ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0d9e32b18..58d21c1fc 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -85,7 +85,7 @@ Group-mode `trace` (`gitnexus/src/core/group/cross-trace.ts`) stitches a path th 19 default phases are defined in `gitnexus/src/core/ingestion/pipeline-phases/`, each with explicit `deps` and typed output. `--pdg` adds `taintSummaries` and `callSummaries` (21 total). ``` -scan → structure → [springConfig, markdown, cobol] → parse → [routes, tools, orm] +scan → structure → [springConfig, markdown, cobol, plsql] → parse → [routes, tools, orm] → crossFile → scopeResolution → [springAutoConfiguration, springAop] → pruneLocalSymbols → mro → springAopInheritance → di → communities → processes ``` @@ -97,7 +97,8 @@ scan → structure → [springConfig, markdown, cobol] → parse → [routes, to | `springConfig` | `spring-config.ts` | `structure` | Spring configuration-property nodes and metadata | | `markdown` | `markdown.ts` | `structure` | Section nodes, cross-link edges from .md/.mdx | | `cobol` | `cobol.ts` | `structure` | COBOL program/paragraph/section nodes (regex, no tree-sitter) | -| `parse` | `parse.ts` + `parse-impl.ts` | `structure`, `markdown`, `cobol` | Symbol nodes, IMPORTS/CALLS/EXTENDS edges, extracted routes/tools/ORM queries | +| `plsql` | `plsql.ts` | `structure` | Oracle PL/SQL package/subprogram/trigger/table nodes; CALLS/ACCESSES/IMPORTS edges (ANTLR4) | +| `parse` | `parse.ts` + `parse-impl.ts` | `structure`, `markdown`, `cobol`, `plsql` | Symbol nodes, IMPORTS/CALLS/EXTENDS edges, extracted routes/tools/ORM queries | | `routes` | `routes.ts` | `parse` | Route nodes + HANDLES_ROUTE edges (Next.js, Expo, PHP, decorators, and JS/TS static route sources — see below) | | `tools` | `tools.ts` | `parse` | Tool nodes + HANDLES_TOOL edges | | `orm` | `orm.ts` | `parse` | QUERIES edges (Prisma, Supabase) | @@ -568,7 +569,7 @@ CI auto-discovers the set via `tsx`. No workflow edit required. ## Language-agnostic graph feeding -18 languages → single unified graph. Four abstraction layers: +19 languages → single unified graph. Four abstraction layers: ``` Unified Graph Schema (44 node types, 21 relationship types) diff --git a/README.md b/README.md index 9c0eb862f..ef982e70a 100644 --- a/README.md +++ b/README.md @@ -112,8 +112,8 @@ For Docker, hosted deployments, image verification, or a local frontend, see ## Supported languages -C, C++, C#, COBOL, Dart, Go, Java, JavaScript, Kotlin, Objective-C, PHP, Python, -Ruby, Rust, Swift, TypeScript, Vue and Zig have registered providers. Support +C, C++, C#, COBOL, Dart, Go, Java, JavaScript, Kotlin, Objective-C, Oracle PL/SQL, +PHP, Python, Ruby, Rust, Swift, TypeScript, Vue and Zig have registered providers. Support varies by construct; see the [language inventories](docs/languages/README.md) for measured behavior, exclusions and remaining gaps. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 2d7a0bf90..79aa320e5 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -288,6 +288,7 @@ For local model setup and switching instructions, see [Local Embedding Model](.. | `GITNEXUS_CHUNK_BYTE_BUDGET` | `2097152` (2 MB) | Per-bucket byte budget for parse-cache packing. Files are grouped by `(language, hash(path) mod 128)`; packs inside a bucket are cut at this limit. Smaller = finer-grained invalidation and more dispatch. Default is always 2 MiB and no longer scales with worker count. | Tuning incremental-analyze cache invalidation on monorepos without changing `--workers`. | | `GITNEXUS_NO_GITIGNORE` | unset | When set, skips `.gitignore` parsing. `.gitnexusignore` is still honored. | Indexing a repo whose `.gitignore` excludes files you actually want indexed (e.g., generated code committed for cross-repo lookup). | | `GITNEXUS_SKIP_OPTIONAL_GRAMMARS` | unset | When exactly `1`, skips optional vendored Dart, Proto, Swift, Kotlin and Zig grammar setup. The C grammar and other native dependencies remain required. | Skipping optional language parsers when their prebuilds are unavailable and no source-build toolchain is installed. | +| `GITNEXUS_PLSQL_SEGMENT_DEADLINE_MS` | `10000` | Floor, in ms, of the wall-clock budget for parsing one `/`-terminated PL/SQL segment; the budget grows by 15 ms per line above it. A segment that runs out is recorded as timed out and its definitions come from the regex fallback (low confidence; rename refuses on them). `0` disables the deadline; blank, invalid or negative values use the default. | PL/SQL repositories with very large package bodies on slow or loaded machines, where segments hit the deadline (the `PL/SQL degraded: … at the parse deadline` line `analyze` logs with `NODE_ENV=development`) and lose call edges to the fallback. | | `GITNEXUS_MCP_READ_ONLY` | unset | Set to `1` to expose only proven single-repository read tools and resources; `0` disables the policy and any other value fails startup. | The MCP server runs in an environment where graph mutation, raw Cypher, and cross-repository group routing must be unavailable. | | `GITNEXUS_MCP_ENABLE_REFRESH` | unset | Set to `1` to list the `refresh_index` MCP tool on a local `gitnexus mcp` server (stdio or `--http`). `gitnexus serve` never exposes it, read-only mode hides and rejects it, and any value other than `0` or `1` fails startup. | Agents on a local server can see a stale index but have no shell to run `gitnexus analyze`. | | `GITNEXUS_MCP_REFRESH_MAX_FILES` | `5000` | `refresh_index` refuses a repository whose working tree holds more files than this, counted the way analyze scans it before starting the rebuild. `0` removes the limit. | Raise it when larger repositories should be refreshable over MCP. | diff --git a/docs/languages/README.md b/docs/languages/README.md index 23121f260..3d56f2f14 100644 --- a/docs/languages/README.md +++ b/docs/languages/README.md @@ -1,7 +1,7 @@ # Language Ingestion Gap Inventories These inventories are the starting point for language-specific plans. There is one -`-ingestion-gaps.md` file for each of the 18 languages in +`-ingestion-gaps.md` file for each of the 19 languages in [SCOPE_RESOLVERS](../../gitnexus/src/core/ingestion/scope-resolution/pipeline/registry.ts). Names match provider directories; `jvm` is shared infrastructure, not another language. @@ -27,6 +27,7 @@ receiver examples do not establish whole-language, CFG/PDG or rename completenes | [JavaScript](javascript-ingestion-gaps.md) | Preserve JSDoc container evidence for indexed receivers. | | [Kotlin](kotlin-ingestion-gaps.md) | Inventory companion and inherited-receiver limits. | | [Objective-C](objective-c-ingestion-gaps.md) | Add receiver measurements without weakening dynamic-dispatch boundaries. | +| [PL/SQL](plsql-ingestion-gaps.md) | Measure call and table-access recall on real Oracle package corpora. | | [PHP](php-ingestion-gaps.md) | Retain PHPDoc container types and establish namespace owners. | | [Python](python-ingestion-gaps.md) | Audit constructor-field and module ownership coverage. | | [Ruby](ruby-ingestion-gaps.md) | Isolate YARD parameter bindings before adding recall. | diff --git a/docs/languages/plsql-ingestion-gaps.md b/docs/languages/plsql-ingestion-gaps.md new file mode 100644 index 000000000..87857bcc5 --- /dev/null +++ b/docs/languages/plsql-ingestion-gaps.md @@ -0,0 +1,79 @@ +# Oracle PL/SQL Ingestion Gaps + +Initial inventory for the `experimental` PL/SQL provider. The provider is a +standalone ANTLR4 path (COBOL precedent): `plsql-processor.ts` owns every +structural edge, and the registered resolver runs in +`scopeResolutionEdgeMode: callable-flow-only`, contributing declarations and +the source bindings rename relies on. Nothing below is a whole-language +completeness claim. + +## What is modelled + +| Construct | Graph | Evidence | +| ------------------------------------------------------------ | ------------------------------------------------------------------------ | -------------------------------------------------------------------------- | +| Package spec + body (`.pks`/`.pkb`/`.pck`, or `.sql`) | One `Module`, anchored at the spec, `CONTAINS` from every defining file | `plsql.test.ts` | +| Package procedure/function, nested subprogram, standalone, trigger | `Function`; `isExported` = declared in the spec (package members) | `plsql.test.ts` | +| Calls: local, nested, `pkg.proc`, `schema.pkg.proc` | `CALLS` (`plsql-call`); overloads narrowed by arity and named arguments | `plsql.test.ts` (overload line check) | +| `SELECT`/`INSERT`/`UPDATE`/`DELETE`/`MERGE`/`LOCK`, trigger target | `Record` per table, `ACCESSES` with `reason: plsql-` | `plsql.test.ts`, `plsql-extract.test.ts` | +| SQL*Plus `@@script`, `@script`, `START script` | `IMPORTS` File → File (exact paths only) | `plsql.test.ts`, `external-import-conformance.test.ts` | +| Rename of a package or subprogram | Spec, body, `END` label and qualified call sites together | `rename-integration/plsql/` | + +Deliberately no edge: built-ins (`TRUNC`, `DBMS_OUTPUT.PUT_LINE`) and any call +that does not resolve to a definition in the repository; calls over a database +link; CTE names; `DUAL`; scripts that are not at an exact repository path. + +## Parser robustness + +Grammar: grammars-v4 `sql/plsql` (Apache-2.0) generated with antlr-ng for +antlr4ng; see `gitnexus/src/core/ingestion/plsql/generated/README.md`. + +- A line-preserving pre-pass blanks SQL*Plus client commands (only between + statements), neutralises `&` substitution variables at equal length and + resolves conditional compilation by keeping the first branch of every + `$IF` (nesting-aware). Token positions remain original source positions. +- Files are split into `/`-terminated segments; each is parsed SLL-first, + falls back to LL with error recovery, and has a wall-clock deadline. +- A segment with syntax errors keeps whatever the recovered tree yields and + gains regex-level definitions for any subprogram the tree lost + (`fromFallback`, lower edge confidence, rename refused). +- The parse deadline scales with segment size (10 s floor, 15 ms/line; + `GITNEXUS_PLSQL_SEGMENT_DEADLINE_MS` overrides the floor, 0 disables), so + large valid package bodies are not cut off on a cold DFA cache. +- Sites (calls, accesses) are owned by the innermost enclosing definition by + position, so overloads keep their own edges. +- Same-named packages in different folders stay separate: a body pairs with + the nearest spec, and call ties are broken by path proximity. Rename + families stay within one package instance. + +Spike measurement (801 files / 245k lines from utPLSQL, OraOpenSource Logger, +Alexandria PL/SQL utils, OOS-Utils and Oracle db-sample-schemas): 97.6% of +package/procedure/function/trigger files parse without syntax errors; about +175 ms per 1,000 lines once the ANTLR DFA cache is warm, about 650 ms cold. + +## Planning candidates + +| ID | Evidence | Gap or boundary | Next planning step and acceptance | +| --------- | ----------------------- | --------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| PLSQL-01 | Corpus measurement | Call and table-access recall is proven on fixtures, not measured on large real package bodies. | Hand-label a sample of real packages; report recall/precision per edge kind before promoting from `experimental`. | +| PLSQL-02 | Grammar gaps | `TREAT(x AS t).m()`, newer keywords used as identifiers (`ANNOTATIONS`, `XDB`), some object-type member syntax. | Upstream grammar fixes to grammars-v4; regenerate; re-run the corpus. | +| PLSQL-03 | Ambiguity | An argument-less function call in an expression is indistinguishable from a variable or column. | Resolve against local declarations first; keep rename refusing until then. | +| PLSQL-04 | Dynamic SQL | `EXECUTE IMMEDIATE` sites are recorded but produce no edges; only literal text is kept. | Constant-fold `l_sql := '…'` assignments and parse literal statements as low-confidence edges. | +| PLSQL-05 | Not modelled | Object types (`CREATE TYPE … AS OBJECT`, `UNDER`, member methods), synonyms, views' column lineage. | Decide labels (`Class`/`Method`, `EXTENDS`) and schema impact with maintainers; separate PR. | +| PLSQL-06 | Identity | Tables are keyed by unqualified name; same-named tables in two schemas merge. | Key by `SCHEMA.TABLE` when written and reconcile unqualified references through synonyms / current schema. | +| PLSQL-07 | Performance | The phase runs on the main thread before parse: the corpus above (618 claimed files, 121k lines) takes ~60 s cold in one process; facts are shared with scope extraction within a run but not persisted across runs. | Move segment parsing to workers (one warm ANTLR DFA each); persist extracted facts in the parse cache keyed by content hash so incremental runs skip unchanged files. | +| PLSQL-09 | Overload fan-out | Overloads that differ only by parameter type (e.g. utPLSQL `ut.expect`) cannot be narrowed by arity, so each call gets one 0.5-confidence edge per candidate (3,170 ambiguous sites on the corpus after path-proximity narrowing). | Infer argument types from literals and declared variables/parameters; narrow by declared parameter types. | +| PLSQL-13 | SQL*Plus-only scripts | A `.sql` file holding only SQL*Plus commands (e.g. `EXEC pkg.proc;`), with no Oracle unit or include, is not claimed because `EXEC` is also T-SQL; rename cannot see such a caller. Rename does refuse when a claimed file's SQL*Plus command names the target. | Claim such scripts when they sit in a tree that holds Oracle sources, or parse `EXEC` bodies as call sites. | +| PLSQL-11 | Cross-language rename | Java/Kotlin/XML code that calls procedures by string (`{call pkg.proc(?)}`, MyBatis `CALLABLE`) is not consulted by rename. | Implement the rename interop hooks, or refuse when a non-PL/SQL source mentions the qualified name. | +| PLSQL-12 | Incremental identity | Table `Record` and merged package `Module` nodes are anchored to one file although derived from several. | Add an edit-one-file incremental-analyze test; give synthetic objects a stable anchor if writeback drops them. | +| PLSQL-10 | Impact on tables | `impact` adds `ACCESSES` by default only for `Class`/`Interface` targets, so a table needs `relationTypes: ['ACCESSES', …]` to list its readers and writers. | Agree with maintainers whether `Record` targets should get the same default. | +| PLSQL-08 | CFG/PDG | `collectSourceCfgs` returns no CFGs. | Build statement-level CFGs from the parse tree for `analyze --pdg`. | + +## Code and regression entry points + +- [Provider](../../gitnexus/src/core/ingestion/languages/plsql.ts) and [hooks](../../gitnexus/src/core/ingestion/languages/plsql/). +- [Parser, pre-pass and extraction](../../gitnexus/src/core/ingestion/plsql/). +- [Processor](../../gitnexus/src/core/ingestion/plsql-processor.ts) and [phase](../../gitnexus/src/core/ingestion/pipeline-phases/plsql.ts). +- [plsql.test.ts](../../gitnexus/test/integration/resolvers/plsql.test.ts), + [plsql-extract.test.ts](../../gitnexus/test/unit/plsql-extract.test.ts), + [plsql-preprocess.test.ts](../../gitnexus/test/unit/plsql-preprocess.test.ts), + [rename fixtures](../../gitnexus/test/fixtures/rename-integration/plsql/). diff --git a/eslint.config.mjs b/eslint.config.mjs index 959560892..0c0ecea8b 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -55,6 +55,8 @@ export default [ '**/.stryker-tmp/**', 'gitnexus/reports/mutation/**', 'gitnexus/vendor/**', + // ANTLR4-generated PL/SQL lexer/parser (regenerate, never hand-edit). + 'gitnexus/src/core/ingestion/plsql/generated/**', 'gitnexus-web/src/vendor/**', 'gitnexus/test/fixtures/**', 'gitnexus-web/test/fixtures/**', diff --git a/gitnexus-shared/src/language-detection.ts b/gitnexus-shared/src/language-detection.ts index 583a0fd5f..88c9a413a 100644 --- a/gitnexus-shared/src/language-detection.ts +++ b/gitnexus-shared/src/language-detection.ts @@ -55,6 +55,19 @@ const EXTENSION_MAP: Record = { [SupportedLanguages.Vue]: ['.vue'], [SupportedLanguages.Cobol]: ['.cbl', '.cob', '.cpy', '.cobol'], [SupportedLanguages.Zig]: ['.zig'], + // `.sql` is claimed by content (provider.classifyFileContent), not by extension. + [SupportedLanguages.PlSql]: [ + '.pks', + '.pkb', + '.pck', + '.pls', + '.plb', + '.prc', + '.fnc', + '.trg', + '.tps', + '.tpb', + ], } satisfies Record; // Ensure exhaustiveness /** Pre-built reverse lookup: extension → language (built once at module load). */ @@ -129,6 +142,7 @@ const SYNTAX_MAP: Record = { [SupportedLanguages.Vue]: 'typescript', [SupportedLanguages.Cobol]: 'cobol', [SupportedLanguages.Zig]: 'zig', + [SupportedLanguages.PlSql]: 'plsql', } satisfies Record; // Ensure exhaustiveness /** Non-code file extensions → Prism-compatible syntax identifiers */ diff --git a/gitnexus-shared/src/languages.ts b/gitnexus-shared/src/languages.ts index eec45e612..925859ff9 100644 --- a/gitnexus-shared/src/languages.ts +++ b/gitnexus-shared/src/languages.ts @@ -24,4 +24,6 @@ export enum SupportedLanguages { /** Standalone regex processor — no tree-sitter, no LanguageProvider. */ Cobol = 'cobol', Zig = 'zig', + /** Oracle PL/SQL — standalone ANTLR4 provider (no tree-sitter). */ + PlSql = 'plsql', } diff --git a/gitnexus-shared/src/scope-resolution/language-classification.ts b/gitnexus-shared/src/scope-resolution/language-classification.ts index 2e635c8e2..df5d9a927 100644 --- a/gitnexus-shared/src/scope-resolution/language-classification.ts +++ b/gitnexus-shared/src/scope-resolution/language-classification.ts @@ -16,6 +16,7 @@ * * Added after Ring 1: zig enters as `experimental` (new language * integration; promotion to `production` is a separate governance PR). + * plsql enters as `experimental` (standalone ANTLR4 provider path). */ import { SupportedLanguages } from '../languages.js'; @@ -47,6 +48,7 @@ export const LanguageClassifications: Readonly 9976.2 us per import (50.6x), and Python's from-import submodule-precedence branch, the only spelling that reads context.parsedFiles at all. Fifteen of the seventeen arms cannot observe a context (their hooks declare three or four parameters) and are handed none, so their numbers did not move; which two CAN is now reconciled against SCOPE_RESOLVERS' hook arity rather than asserted in prose. NOTHING ELSE IN THIS FILE COULD HAVE GATED IT, which is why the context arm exists: fingerprints and shape can remain unchanged while dropping context only makes timing faster. The deterministic context arm is therefore the guard for this wiring. The arm is one import per language resolved through resolveOne twice, with and without the pass's parsedFiles, whose two answers must DIFFER and must both match what is recorded. WHAT REMAINS UNMEASURED, narrowed rather than deleted: Python's parsedFileByPath memo is exercised by the five timing arms and cannot be reached by the heap arm at all, because retainedPassBytes requires a probe that MISSES while every path that builds that memo returns a non-null packageTarget \u2014 so no ceiling bounds that Map (one pointer per parsed file, O(files), no depth term) and the contract test's count gate is what holds it to one build per pass. PHP's sole arm carries a representative Composer PSR-4 map, so mapped hits and authoritative misses exercise that production branch directly. And the const tail of PHP's leg is a different ANSWER at the same cost \u2014 it runs the identical candidate gather and localDefs filter and diverges in the last two lines \u2014 so it is gated by count in test/unit/scope-resolution/import-target-index-reuse.contract.test.ts, which stays the gate to read alongside this file.", diff --git a/gitnexus/bench/import-target/measure.mjs b/gitnexus/bench/import-target/measure.mjs index a4d87ba38..1bbee5401 100644 --- a/gitnexus/bench/import-target/measure.mjs +++ b/gitnexus/bench/import-target/measure.mjs @@ -486,6 +486,7 @@ import { cobolScopeResolver } from '../../src/core/ingestion/languages/cobol/sco import { resolveSwiftImportTarget } from '../../src/core/ingestion/languages/swift/import-target.ts'; import { resolveRustImportTarget } from '../../src/core/ingestion/languages/rust/import-target.ts'; import { resolveZigImportInternal } from '../../src/core/ingestion/languages/zig/import-helpers.ts'; +import { resolvePlSqlScriptInclude } from '../../src/core/ingestion/plsql/script-include.ts'; import { resolvePythonImportTarget } from '../../src/core/ingestion/languages/python/import-target.ts'; import { makeJsResolveImportTarget } from '../../src/core/ingestion/languages/javascript/import-target.ts'; import { makeVueResolveImportTarget } from '../../src/core/ingestion/languages/vue/import-target.ts'; @@ -788,6 +789,7 @@ const EXTENSION = { cpp: '.cpp', objc: '.m', zig: '.zig', + plsql: '.sql', }; /** C and C++ resolve `#include` against HEADERS, which reach the resolver * through `resolutionConfig` rather than through `allFilePaths` — see @@ -920,6 +922,9 @@ function uniqueDir(lang, d, i) { // share the other unique arms take from a nested directory comes from the // target instead (see `uniqueTarget`). if (lang === 'zig') return `src/mod${d}`; + // SQL*Plus install trees: one folder per component under `db/`, so a + // script reaches a sibling through one `..` (see `uniqueTarget`). + if (lang === 'plsql') return `db/mod${d}`; throw unwiredLanguage('uniqueDir', lang); } @@ -1015,6 +1020,10 @@ function collideDir(lang, d, i) { // axis that CAN grow — `collideTarget` spells its imports up through the // tree and back down, ~4x the components of the unique arm. if (lang === 'zig') return `src/l0/l1/l2/l3/l4/mod${d}`; + // zig's reasoning: the resolver probes `.has()` on at most four candidates + // (two bases × with/without `.sql`) and never searches, so path depth is + // the only axis its cost can grow along. + if (lang === 'plsql') return `db/l0/l1/l2/l3/l4/mod${d}`; throw unwiredLanguage('collideDir', lang); } @@ -1467,6 +1476,25 @@ function uniqueTarget(lang, { local, r, d, j, dirs }) { if (miss === 1) return `ghost${(r >>> 4) % 97}`; return `../vendor${(r >>> 4) % 97}/missing.zig`; } + if (lang === 'plsql') { + // SQL*Plus `@file` / `START file`: tried against the repository root, then + // the calling script's directory. Every script sits in `db/mod{d}/`, so + // `../mod{n}/…` misses at the root (it climbs out of the repo) and hits + // from the caller — both bases are walked for every local include. The + // `d % 7` slice names an `inner/` that exists nowhere and misses; one + // spelling in three drops `.sql`, which is the appended-extension leg. + // Misses: ORACLE_HOME scripts (`?/rdbms/admin/…`, rejected before any + // probe), a bare vendor script that is in neither base, and a relative + // path to a vendored script outside the corpus. + if (local) { + if (d % 7 === 0) return `../mod${d}/inner/file${j}.sql`; + return (r >>> 3) % 3 === 0 ? `../mod${j % dirs}/file${j}` : `../mod${j % dirs}/file${j}.sql`; + } + const miss = (r >>> 3) % 3; + if (miss === 0) return ['?/rdbms/admin/utlrp.sql', '?/rdbms/admin/catproc.sql'][(r >>> 4) % 2]; + if (miss === 1) return `vendor_patch${(r >>> 4) % 97}`; + return `../vendor${(r >>> 4) % 97}/missing.sql`; + } throw unwiredLanguage('uniqueTarget', lang); } @@ -1700,6 +1728,21 @@ function collideTarget(lang, { local, r, d, j, dirs }) { if (miss === 1) return `ghost${(r >>> 4) % 97}`; return `${up}/vendor${(r >>> 4) % 97}/missing.zig`; } + if (lang === 'plsql') { + // zig's collide design: the same families in the same proportions, spelled + // up to `db/` and back down through `l0/…/l4`. + const up = '../../../../../../l0/l1/l2/l3/l4'; + if (local) { + if (d % 7 === 0) return `${up}/mod${d}/inner/file${j}.sql`; + return (r >>> 3) % 3 === 0 + ? `${up}/mod${j % dirs}/file${j}` + : `${up}/mod${j % dirs}/file${j}.sql`; + } + const miss = (r >>> 3) % 3; + if (miss === 0) return ['?/rdbms/admin/utlrp.sql', '?/rdbms/admin/catproc.sql'][(r >>> 4) % 2]; + if (miss === 1) return `vendor_patch${(r >>> 4) % 97}`; + return `${up}/vendor${(r >>> 4) % 97}/missing.sql`; + } throw unwiredLanguage('collideTarget', lang); } @@ -1931,6 +1974,7 @@ function resolveOne(lang, from, target, pass) { // `null` build config: this arm pins the path walk alone // (see the header); the config legs are gated by their own unit tests. if (lang === 'zig') return resolveZigImportInternal(from, target, allFilePaths, null); + if (lang === 'plsql') return resolvePlSqlScriptInclude(target, from, allFilePaths); if (lang === 'python') { // `from import X` — the spelling the orchestrator actually hands // the provider, and the ONLY one that reads `context.parsedFiles`: a @@ -2243,6 +2287,10 @@ const HEAP_PROBE_TARGET = { // miss after the full component walk, which is the longest leg the resolver // has (a by-name miss returns before any walk). zig: '../vendor0/missing.zig', + // No extension and no `..`: the repository root and the caller's directory + // are each probed with `.sql` appended and as written — four misses, the + // longest leg the resolver has. + plsql: 'vendor0/missing', typescript: 'vendor0/lib/missing', vue: 'vendor0/lib/Missing.vue', cpp: 'vendor0/missing.hpp', @@ -2662,6 +2710,7 @@ const LANG_REGISTRY = { cpp: SupportedLanguages.CPlusPlus, objc: SupportedLanguages.ObjectiveC, zig: SupportedLanguages.Zig, + plsql: SupportedLanguages.PlSql, }; const LANGS = Object.keys(LANG_REGISTRY); /** diff --git a/gitnexus/bench/plsql-parse/README.md b/gitnexus/bench/plsql-parse/README.md new file mode 100644 index 000000000..ddcc94830 --- /dev/null +++ b/gitnexus/bench/plsql-parse/README.md @@ -0,0 +1,38 @@ +# PL/SQL parse-throughput bench + +`measure.mjs` times `extractPlSqlFacts` (SQL\*Plus/conditional-compilation +pre-pass, `/` segment split, ANTLR SLL→LL parse, fact extraction) on a +synthetic package body of 60 and 240 subprograms, after one warm-up pass. + +```sh +node --import tsx bench/plsql-parse/measure.mjs # print JSON +node --import tsx bench/plsql-parse/measure.mjs --check # CI guard +``` + +`--check` gates correctness and shape, not absolute speed: + +- **fingerprint**: sha256 over the facts of every `lang-resolution/plsql-*` + fixture file plus a 20-subprogram synthetic body. Re-baseline + `baselines.json` only for an intended facts change. +- **synthetic bodies** parse with no erroneous or timed-out segment. +- **scaling ratio** `(t_240 / t_60) / (lines ratio)` stays under 1.5. About + 1.0 is linear; a quadratic regression shows up near 4.0. + +## Measurements + +Apple Silicon laptop, Node 25, single thread. + +| Input | Lines | Result | +| ---------------------------------------------------------------------------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------- | +| Synthetic body, 60 subprograms, cold (first parse in the process) | 783 | 677 ms | +| Synthetic body, 60 subprograms, warm | 783 | 125 ms/kLOC | +| Synthetic body, 240 subprograms, warm | 3,123 | 116 ms/kLOC (scaling ratio 0.93) | +| Corpus: utPLSQL, Logger, Alexandria, OOS-Utils, Oracle sample schemas (618 claimed files, one process, one cold start) | 121,292 | 58–62 s end to end through `processPlSql` (about 490 ms/kLOC), 0 segments at the deadline | + +The corpus is not vendored; the processor-level number is a manual +measurement to compare against. It roughly doubles on a machine under memory +pressure, so compare runs taken back to back. Real code costs more per line than the +synthetic body: deeper expressions, more SQL and more LL fallbacks. + +Parsing runs on the main thread inside the `plsql` phase and facts are not +cached across runs (PLSQL-07 in `docs/languages/plsql-ingestion-gaps.md`). diff --git a/gitnexus/bench/plsql-parse/baselines.json b/gitnexus/bench/plsql-parse/baselines.json new file mode 100644 index 000000000..c51fddfd2 --- /dev/null +++ b/gitnexus/bench/plsql-parse/baselines.json @@ -0,0 +1,5 @@ +{ + "_comment": "Baseline for bench/plsql-parse/measure.mjs --check. fingerprint = order-independent sha256 over extractPlSqlFacts output for every lang-resolution/plsql-* fixture file plus a 20-subprogram synthetic package body (correctness gate; re-baseline intentionally on a legitimate facts change). scaling_budget = max allowed (t_large/t_small)/(lines ratio) for 60 vs 240 synthetic subprograms after a DFA warm-up; ~1.0 is linear (0.95 measured), ~4.0 quadratic.", + "fingerprint": "0ccc2c62f8747bb91a8fb34dc73b1335600a629ce92160ae539b851f4e1d4637", + "scaling_budget": 1.5 +} diff --git a/gitnexus/bench/plsql-parse/measure.mjs b/gitnexus/bench/plsql-parse/measure.mjs new file mode 100644 index 000000000..b5e8c7793 --- /dev/null +++ b/gitnexus/bench/plsql-parse/measure.mjs @@ -0,0 +1,173 @@ +/** + * Build-free PL/SQL parse-throughput harness (#3771 acceptance criterion). + * + * Times `extractPlSqlFacts` — pre-pass, segment split, ANTLR SLL/LL parse and + * fact extraction — on a synthetic package body at two sizes, after one + * warm-up pass so the ANTLR DFA cache is populated (each analyze pays the + * cold cost once per process; see README.md for corpus numbers). It reports + * ms/kLOC for each size and a scaling ratio `(t_large/t_small)/(lines ratio)`: + * ~1.0 is linear, ~4.0 is quadratic at these sizes. + * + * The correctness gate is an order-independent sha256 fingerprint over the + * facts of every `lang-resolution/plsql-*` fixture file plus a fixed + * 20-subprogram synthetic body. + * + * Run: `node --import tsx bench/plsql-parse/measure.mjs` (prints JSON). + * `--check` asserts the fingerprint equals baselines.json, the synthetic + * bodies parse without syntax errors or timeouts, and the scaling ratio stays + * under budget; exits non-zero otherwise. Absolute timings are reported, not + * gated: they depend on the machine. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import crypto from 'node:crypto'; +import { fileURLToPath } from 'node:url'; + +import { extractPlSqlFacts } from '../../src/core/ingestion/plsql/extract.ts'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const FIXTURE_ROOT = path.resolve(__dirname, '..', '..', 'test', 'fixtures', 'lang-resolution'); +const BASELINE_PATH = path.resolve(__dirname, 'baselines.json'); +const PLSQL_EXTENSIONS = ['.sql', '.pks', '.pkb', '.pck', '.pls', '.plb', '.prc', '.fnc', '.trg']; + +/** + * The production, size-scaled segment deadline: a parse that stalls ends as a + * timed-out segment, which `--check` rejects, instead of hanging the job. + */ +const OPTIONS = {}; + +const SMALL = 60; +const LARGE = 240; +const REPEATS = 3; + +/** One package body with `n` subprograms that call, branch, read and write. */ +function syntheticBody(n) { + const units = []; + for (let i = 0; i < n; i++) { + const callee = i === 0 ? 'NULL' : `p${i - 1}(p_id => p_id, p_amount => v_total)`; + units.push( + [ + ` PROCEDURE p${i}(p_id IN NUMBER, p_amount IN NUMBER DEFAULT 0) IS`, + ` v_total t_orders.amount%TYPE;`, + ` BEGIN`, + ` SELECT NVL(SUM(amount), 0) INTO v_total FROM t_orders WHERE customer_id = p_id;`, + ` IF v_total > p_amount THEN`, + ` UPDATE t_orders SET status = 'HELD' WHERE customer_id = p_id;`, + ` ELSIF v_total = 0 THEN`, + ` INSERT INTO t_audit (customer_id, note) VALUES (p_id, 'empty ${i}');`, + ` END IF;`, + ` ${callee};`, + ` pkg_log.info('p${i} done');`, + ` END p${i};`, + ].join('\n'), + ); + } + return `CREATE OR REPLACE PACKAGE BODY pkg_bench AS\n${units.join('\n\n')}\nEND pkg_bench;\n/\n`; +} + +function lineCount(text) { + return text.split('\n').length; +} + +function digest(facts) { + const { firstError: _firstError, ...stable } = facts; + return crypto.createHash('sha256').update(JSON.stringify(stable)).digest('hex'); +} + +function collectFixtures() { + const out = []; + for (const entry of fs.readdirSync(FIXTURE_ROOT, { withFileTypes: true })) { + if (!entry.isDirectory() || !entry.name.startsWith('plsql-')) continue; + const stack = [path.join(FIXTURE_ROOT, entry.name)]; + while (stack.length) { + const dir = stack.pop(); + for (const c of fs.readdirSync(dir, { withFileTypes: true })) { + const p = path.join(dir, c.name); + if (c.isDirectory()) stack.push(p); + else if (PLSQL_EXTENSIONS.some((e) => c.name.toLowerCase().endsWith(e))) + out.push({ key: path.relative(FIXTURE_ROOT, p).split(path.sep).join('/'), absPath: p }); + } + } + } + return out.sort((a, b) => a.key.localeCompare(b.key)); +} + +function fingerprint() { + const parts = collectFixtures().map( + (f) => `${f.key}|${digest(extractPlSqlFacts(fs.readFileSync(f.absPath, 'utf8'), OPTIONS))}`, + ); + parts.push(`synthetic-20|${digest(extractPlSqlFacts(syntheticBody(20), OPTIONS))}`); + return crypto.createHash('sha256').update(parts.sort().join('\n')).digest('hex'); +} + +function timeExtract(source) { + let best = Number.POSITIVE_INFINITY; + let facts; + for (let i = 0; i < REPEATS; i++) { + const t0 = performance.now(); + facts = extractPlSqlFacts(source, OPTIONS); + best = Math.min(best, performance.now() - t0); + } + return { ms: best, facts }; +} + +function measure() { + const small = syntheticBody(SMALL); + const large = syntheticBody(LARGE); + const coldStart = performance.now(); + extractPlSqlFacts(small, OPTIONS); + const coldMs = performance.now() - coldStart; + const s = timeExtract(small); + const l = timeExtract(large); + const smallLines = lineCount(small); + const largeLines = lineCount(large); + const perKloc = (ms, lines) => +((ms * 1000) / lines).toFixed(1); + return { + fingerprint: fingerprint(), + cold_ms_small: +coldMs.toFixed(1), + small: { + subprograms: SMALL, + lines: smallLines, + ms: +s.ms.toFixed(1), + ms_per_kloc: perKloc(s.ms, smallLines), + }, + large: { + subprograms: LARGE, + lines: largeLines, + ms: +l.ms.toFixed(1), + ms_per_kloc: perKloc(l.ms, largeLines), + }, + scaling_ratio: +(l.ms / s.ms / (largeLines / smallLines)).toFixed(3), + synthetic_errors: s.facts.segmentsWithErrors + l.facts.segmentsWithErrors, + synthetic_timeouts: s.facts.segmentsTimedOut + l.facts.segmentsTimedOut, + synthetic_calls: l.facts.calls.length, + synthetic_accesses: l.facts.accesses.length, + }; +} + +const result = measure(); +console.log(JSON.stringify(result, null, 2)); + +if (process.argv.includes('--check')) { + const baseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf8')); + const failures = []; + if (result.fingerprint !== baseline.fingerprint) + failures.push( + `fingerprint drift: ${result.fingerprint} != baseline ${baseline.fingerprint} (re-baseline only for an intended facts change)`, + ); + if (result.synthetic_timeouts !== 0) + failures.push( + `synthetic bodies hit the parse deadline in ${result.synthetic_timeouts} segments`, + ); + if (result.synthetic_errors !== 0) + failures.push(`synthetic bodies parsed with ${result.synthetic_errors} erroneous segments`); + if (result.scaling_ratio >= baseline.scaling_budget) + failures.push( + `scaling ratio ${result.scaling_ratio} >= budget ${baseline.scaling_budget} (super-linear parse)`, + ); + if (failures.length > 0) { + for (const f of failures) console.error(`plsql-parse: ${f}`); + process.exit(1); + } + console.error('plsql-parse: fingerprint and scaling within baseline'); +} diff --git a/gitnexus/bench/scope-resolution/README.md b/gitnexus/bench/scope-resolution/README.md index 86a1615f9..7488f3e44 100644 --- a/gitnexus/bench/scope-resolution/README.md +++ b/gitnexus/bench/scope-resolution/README.md @@ -13,12 +13,12 @@ plus callers and helpers. Rust also needs a root module file; Objective-C has a header and implementation per entity. Java adds an isolated producer file and exercises transitive factory returns and repeated unreachable return-name lookups. Swift uses same-module visibility; COBOL exercises -program references and procedure pointers. Other arms import the preceding file and call through its +program references and procedure pointers; PL/SQL exercises package-qualified calls. Other arms import the preceding file and call through its return value. These generated sources exercise the analyzer; they are not builds or representative application benchmarks. Every sample runs in a fresh process through real parsing with one worker and the -production scope-resolution pass. COBOL also runs its standalone structural parser. +production scope-resolution pass. COBOL and PL/SQL also run their standalone structural parsers. Configuration files are written to a temporary workspace and loaded through the resolver's normal configuration hook. No production heap, file-count, or propagation limit is imposed. diff --git a/gitnexus/bench/scope-resolution/baselines.json b/gitnexus/bench/scope-resolution/baselines.json index 5d13bfa00..7f6fa118e 100644 --- a/gitnexus/bench/scope-resolution/baselines.json +++ b/gitnexus/bench/scope-resolution/baselines.json @@ -175,6 +175,22 @@ "moduleTypeBindings": 10010999, "typeBindings": 10032997 }, + "plsql/100/20": { + "fingerprint": "908f30ebfe4287bbb4d4553a869da52708dbf92b0e99548dd58d73f8664bdf3b", + "nodes": 2300, + "edges": 2398, + "crossFileCalls": 198, + "moduleTypeBindings": 0, + "typeBindings": 0 + }, + "plsql/1000/20": { + "fingerprint": "600a7dc54fdf793e40b66c47551215ad5cb4aa3af9d432c283750fd882224dd8", + "nodes": 23000, + "edges": 23998, + "crossFileCalls": 1998, + "moduleTypeBindings": 0, + "typeBindings": 0 + }, "python/100/20": { "fingerprint": "58adb2bc6b7ac24ab3dcd500ec3cd577d3e6853fd1a8e84f658f423677d6df50", "nodes": 2299, diff --git a/gitnexus/bench/scope-resolution/corpus.mts b/gitnexus/bench/scope-resolution/corpus.mts index 5f38eb3ec..df5636ee2 100644 --- a/gitnexus/bench/scope-resolution/corpus.mts +++ b/gitnexus/bench/scope-resolution/corpus.mts @@ -18,7 +18,8 @@ ${previous(i, (p) => ` usePrevious() { return new C${p}().make${p}_0().make${p} }`; // Each arm has one chain of cross-file uses and distinct method names. Swift -// uses same-module visibility; COBOL uses program calls and procedure pointers. +// uses same-module visibility; COBOL uses program calls and procedure pointers; +// PL/SQL uses package-qualified calls. export const generators = { javascript: (i, n) => ({ path: `src/C${i}.js`, content: script(i, n, false, 'js') }), typescript: (i, n) => ({ path: `src/C${i}.ts`, content: script(i, n, true, 'js') }), @@ -175,6 +176,24 @@ ${previous(i, (p) => ` SET TARGET-PTR TO ENTRY 'C${p}'\n CAL PERFORM MAKE-${i}-0 GOBACK. ${members(n, (j) => ` MAKE-${i}-${j}.\n CONTINUE.`)} +`, + }), + plsql: (i, n) => ({ + path: `src/pkg_c${i}.pck`, + content: `CREATE OR REPLACE PACKAGE pkg_c${i} AS +${members(n, (j) => ` PROCEDURE make${i}_${j};`)} + PROCEDURE use_previous; +END pkg_c${i}; +/ +CREATE OR REPLACE PACKAGE BODY pkg_c${i} AS +${members(n, (j) => ` PROCEDURE make${i}_${j} IS BEGIN NULL; END make${i}_${j};`)} + PROCEDURE use_previous IS + BEGIN +${previous(i, (p) => ` pkg_c${p}.make${p}_0;\n pkg_c${p}.make${p}_1;`)} + NULL; + END use_previous; +END pkg_c${i}; +/ `, }), zig: (i, n) => ({ diff --git a/gitnexus/bench/scope-resolution/measure.mts b/gitnexus/bench/scope-resolution/measure.mts index 0ff4fd9e2..1d46d65b1 100644 --- a/gitnexus/bench/scope-resolution/measure.mts +++ b/gitnexus/bench/scope-resolution/measure.mts @@ -18,6 +18,8 @@ import { SupportedLanguages } from 'gitnexus-shared'; import { SCOPE_RESOLVERS } from '../../src/core/ingestion/scope-resolution/pipeline/registry.js'; import { runScopeResolution } from '../../src/core/ingestion/scope-resolution/pipeline/run.js'; import { processCobol } from '../../src/core/ingestion/cobol-processor.js'; +import { processPlSql } from '../../src/core/ingestion/plsql-processor.js'; +import { loadPlSqlParser } from '../../src/core/ingestion/plsql/facts-store.js'; import { generateId } from '../../src/lib/utils.js'; import { parseFilesWithWorkers } from '../../test/helpers/worker-parse.js'; import { corpus, generators } from './corpus.mjs'; @@ -54,6 +56,11 @@ async function sample(language: SupportedLanguages, count: number, methods: numb if (language === SupportedLanguages.Cobol) { processCobol(graph, files, new Set(files.map((file) => file.path))); } + // PL/SQL's structural graph is emitted by its standalone ANTLR parser, loaded on demand. + if (language === SupportedLanguages.PlSql) { + await loadPlSqlParser(); + processPlSql(graph, files, new Set(files.map((file) => file.path))); + } const parseMs = performance.now() - parseStarted; const heapAfterParse = process.memoryUsage().heapUsed; let parsedFileCount = 0; diff --git a/gitnexus/package-lock.json b/gitnexus/package-lock.json index 31f95ea8a..922bd30f5 100644 --- a/gitnexus/package-lock.json +++ b/gitnexus/package-lock.json @@ -26,6 +26,7 @@ "@ladybugdb/core": "0.22.0", "@modelcontextprotocol/sdk": "^1.0.0", "@scarf/scarf": "^1.4.0", + "antlr4ng": "3.0.16", "busboy": "^1.6.0", "chokidar": "^5.0.0", "cli-progress": "^3.12.0", @@ -3073,6 +3074,12 @@ "url": "https://github.com/chalk/ansi-styles?sponsor=1" } }, + "node_modules/antlr4ng": { + "version": "3.0.16", + "resolved": "https://registry.npmjs.org/antlr4ng/-/antlr4ng-3.0.16.tgz", + "integrity": "sha512-DQuJkC7kX3xunfF4K2KsWTSvoxxslv+FQp/WHQZTJSsH2Ec3QfFmrxC3Nky2ok9yglXn6nHM4zUaVDxcN5f6kA==", + "license": "BSD-3-Clause" + }, "node_modules/anynum": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/anynum/-/anynum-1.0.1.tgz", diff --git a/gitnexus/package.json b/gitnexus/package.json index e2af706a5..d0eb35af0 100644 --- a/gitnexus/package.json +++ b/gitnexus/package.json @@ -84,6 +84,7 @@ "@ladybugdb/core": "0.22.0", "@modelcontextprotocol/sdk": "^1.0.0", "@scarf/scarf": "^1.4.0", + "antlr4ng": "3.0.16", "busboy": "^1.6.0", "chokidar": "^5.0.0", "cli-progress": "^3.12.0", diff --git a/gitnexus/scripts/cross-platform-tests.ts b/gitnexus/scripts/cross-platform-tests.ts index bcae19f36..d2c254132 100644 --- a/gitnexus/scripts/cross-platform-tests.ts +++ b/gitnexus/scripts/cross-platform-tests.ts @@ -186,6 +186,7 @@ const LBUG_NATIVE = [ 'test/integration/rename-provider-kotlin.test.ts', 'test/integration/rename-provider-objective-c.test.ts', 'test/integration/rename-provider-php.test.ts', + 'test/integration/rename-provider-plsql.test.ts', 'test/integration/rename-provider-python.test.ts', 'test/integration/rename-provider-ruby.test.ts', 'test/integration/rename-provider-rust.test.ts', diff --git a/gitnexus/scripts/generate-plsql-parser.mjs b/gitnexus/scripts/generate-plsql-parser.mjs new file mode 100644 index 000000000..289cda486 --- /dev/null +++ b/gitnexus/scripts/generate-plsql-parser.mjs @@ -0,0 +1,63 @@ +#!/usr/bin/env node +/** + * Regenerate the ANTLR4 PL/SQL lexer/parser under + * src/core/ingestion/plsql/generated/ from the vendored grammars-v4 sources in + * src/core/ingestion/plsql/grammar/. + * + * npx --yes antlr-ng@1.0.10 is fetched on demand; it is not a runtime dependency. + * + * The grammar's C++ header placeholders are replaced with the antlr4ng base-class + * imports, exactly as grammars-v4's `Antlr4ng/transformGrammar.py` does. + */ +import { execSync } from 'node:child_process'; +import { + mkdtempSync, + readFileSync, + writeFileSync, + rmSync, + readdirSync, + copyFileSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const grammarDir = path.join(root, 'src/core/ingestion/plsql/grammar'); +const outDir = path.join(root, 'src/core/ingestion/plsql/generated'); +const work = mkdtempSync(path.join(tmpdir(), 'plsql-grammar-')); + +try { + for (const name of ['PlSqlLexer.g4', 'PlSqlParser.g4']) { + const text = readFileSync(path.join(grammarDir, name), 'utf8') + .replace( + '// Insert here @header for C++ lexer.', + '@header {import { PlSqlLexerBase } from "./PlSqlLexerBase.js"}', + ) + .replace( + '// Insert here @header for C++ parser.', + '@header {import { PlSqlParserBase } from "./PlSqlParserBase.js"}', + ); + writeFileSync(path.join(work, name), text); + } + // String form runs through the platform shell (cmd.exe / sh) so `npx` + // resolves to npx.cmd on Windows — execFileSync('npx') without a shell + // fails there (same constraint as scripts/build-web.js). + const quote = (arg) => `"${arg}"`; + execSync( + [ + 'npx --yes antlr-ng@1.0.10 -Dlanguage=TypeScript -l true -v false', + `-o ${quote(work)}`, + quote(path.join(work, 'PlSqlLexer.g4')), + quote(path.join(work, 'PlSqlParser.g4')), + ].join(' '), + { stdio: 'inherit' }, + ); + for (const file of readdirSync(work)) { + if (/^PlSql(Lexer|Parser|ParserListener)\.ts$/.test(file)) + copyFileSync(path.join(work, file), path.join(outDir, file)); + } + console.log(`[plsql] regenerated parser into ${path.relative(root, outDir)}`); +} finally { + rmSync(work, { recursive: true, force: true }); +} diff --git a/gitnexus/src/core/ingestion/languages/index.ts b/gitnexus/src/core/ingestion/languages/index.ts index b10822a6f..59331c5d5 100644 --- a/gitnexus/src/core/ingestion/languages/index.ts +++ b/gitnexus/src/core/ingestion/languages/index.ts @@ -31,6 +31,7 @@ import { dartProvider } from './dart.js'; import { vueProvider } from './vue.js'; import { cobolProvider } from './cobol.js'; import { zigProvider } from './zig.js'; +import { plsqlProvider } from './plsql.js'; export const providers = { [SupportedLanguages.JavaScript]: javascriptProvider, @@ -51,6 +52,7 @@ export const providers = { [SupportedLanguages.Vue]: vueProvider, [SupportedLanguages.Cobol]: cobolProvider, [SupportedLanguages.Zig]: zigProvider, + [SupportedLanguages.PlSql]: plsqlProvider, } satisfies Record; /** Get provider by language enum (always succeeds for SupportedLanguages). */ diff --git a/gitnexus/src/core/ingestion/languages/plsql.ts b/gitnexus/src/core/ingestion/languages/plsql.ts new file mode 100644 index 000000000..ef0146a5c --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/plsql.ts @@ -0,0 +1,50 @@ +/** + * Oracle PL/SQL Language Provider + * + * Standalone ANTLR4 provider — no tree-sitter grammar. PL/SQL files are + * parsed with the grammars-v4 PlSql grammar (`../plsql/`) and mapped to the + * graph by `plsql-processor.ts` in the `plsql` pipeline phase, before parse. + * + * Dedicated extensions (.pks, .pkb, .pck, …) are always PL/SQL. `.sql` is + * claimed through `classifyFileContent` only when the file holds an Oracle + * PL/SQL unit or is a SQL*Plus script that includes other scripts + * (`@@file`, `@file.ext`, `START file.ext`), leaving other SQL dialects alone. + */ +import { SupportedLanguages } from 'gitnexus-shared'; +import { defineLanguage } from '../language-provider.js'; +import { PLSQL_EXTENSIONS, isSqlFile, looksLikeOraclePlSql } from '../plsql/file-types.js'; +import { + collectPlSqlSourceCfgs, + emitPlSqlScopeCaptures, + interpretPlSqlImport, + plsqlImportOwningScope, + plsqlReceiverBinding, + plsqlRenameProvider, +} from './plsql/index.js'; + +export const plsqlProvider = defineLanguage({ + id: SupportedLanguages.PlSql, + parseStrategy: 'standalone', + extensions: [...PLSQL_EXTENSIONS], + // Check the extension first: classifiers run before extension lookup for every file. + shouldClassifyFileContent: isSqlFile, + classifyFileContent: (filePath, sourceText) => + isSqlFile(filePath) && looksLikeOraclePlSql(sourceText), + entryPointPatterns: [], + astFrameworkPatterns: [], + treeSitterQueries: '', + typeConfig: { + declarationNodeTypes: new Set(), + extractDeclaration: () => null, + extractParameter: () => null, + }, + exportChecker: () => false, + rename: plsqlRenameProvider, + collectSourceCfgs: collectPlSqlSourceCfgs, + + // ── Scope-resolution hooks ─────────────────────────────────────── + emitScopeCaptures: emitPlSqlScopeCaptures, + interpretImport: interpretPlSqlImport, + importOwningScope: plsqlImportOwningScope, + receiverBinding: plsqlReceiverBinding, +}); diff --git a/gitnexus/src/core/ingestion/languages/plsql/captures.ts b/gitnexus/src/core/ingestion/languages/plsql/captures.ts new file mode 100644 index 000000000..253fd70af --- /dev/null +++ b/gitnexus/src/core/ingestion/languages/plsql/captures.ts @@ -0,0 +1,146 @@ +/** + * `emitScopeCaptures` for Oracle PL/SQL. + * + * Wraps the ANTLR fact extractor (`../../plsql/extract.ts`) and emits the + * parser-agnostic RFC §5.1 vocabulary consumed by the central ScopeExtractor: + * + * package → @scope.module + @declaration.program (→ Module) + * procedure/function → @scope.function + @declaration.function (→ Function) + * trigger → @scope.function + @declaration.function + * spec declaration → @declaration.function (no scope: it has no body) + * call (last part) → @reference.call (+ @reference.arity) + * call qualifier part → @reference.read (the package name in `pkg.proc`) + * END