mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-09-07 08:26:11 +00:00
21 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
ChamHerry
|
681a352006
|
fix(worker): analyze native worker aborts (#1833)
* fix(analyze): avoid native aborts on generated worker bundles Retire timed-out parse workers instead of force-terminating native parser state, and skip Monaco generated worker bundles by default while preserving explicit .gitnexusignore negation overrides. Constraint: Node native tree-sitter bindings can abort the process when a timed-out worker is terminated while inside parser state. Rejected: Falling back to sequential parsing for native stalls | it can move the same native crash onto the main thread. Confidence: high Scope-risk: moderate Directive: Keep timeout recovery from force-terminating workers until they return to JS or exit naturally. Tested: npm test; npx tsc --noEmit; npm run build; targeted analyze on /Users/wangxc/Code/keep; gitnexus detect_changes --scope staged Not-tested: Node 22 LTS runtime and non-macOS platforms * fix(worker): bound retired parser worker lifetimes Keep timeout recovery from immediately terminating workers that may still be inside native parser state, while making terminal pool shutdown own retired worker cleanup so long-lived processes do not accumulate retired threads. Constraint: Claude review on PR #1833 required retiredWorkers cleanup in pool.terminate() and tripBreaker() without regressing no-immediate-terminate timeout safety. Rejected: clearing the retiredWorkers set without terminating | would remove JS bookkeeping while leaking the underlying worker thread. Confidence: high Scope-risk: moderate Directive: Preserve the distinction between recoverable timeout retirement and terminal pool shutdown; do not reintroduce immediate terminate in removeWorkerFromSlot(..., 'retire'). Tested: npx vitest run test/unit/worker-pool-timeout-retire.test.ts; npx vitest run test/unit/worker-pool-timeout-retire.test.ts test/unit/worker-pool-resilience.test.ts test/unit/worker-pool-cumulative-timeout.test.ts test/unit/worker-pool-slot-generation.test.ts; npx tsc --noEmit; npm run build; npx prettier --check src/core/ingestion/workers/worker-pool.ts test/unit/worker-pool-timeout-retire.test.ts ../docs/todo/pr-1833-retired-worker-cleanup-plan.md; npx eslint src/core/ingestion/workers/worker-pool.ts test/unit/worker-pool-timeout-retire.test.ts; gitnexus detect_changes --scope staged. Not-tested: npm test full suite did not complete green in this environment; two runs each had one unrelated test/unit/hooks.test.ts parseHookOutput null failure, and each failed hook test passed when rerun in isolation. * ci: retrigger checks Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: wangxc <wangxc_a_bj@si-tech.com.cn> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Test <test@example.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Minidoracat
|
2b6e7ffbd9
|
fix(php): avoid Blade templates entering PHP analysis (#1790) | ||
|
WENJIE HUANG
|
32b5c0e3fc
|
feat: add IncludeExtractor for C++ cross-repo include tracking (group) (#1156)
* feat: add IncludeExtractor for C++ cross-repo include tracking (group) * fix: address CodeQL warnings on include-extractor - Remove unused HEADER_GLOB constant in include-extractor.ts - Use fs.mkdtempSync for secure temp dir creation in tests (CodeQL: 'Insecure temporary file') * fix(group): close missing ); in manifest-extractor include branch The 'include' branch in ManifestExtractor.resolveSymbol was missing the closing ); for the executor() call, causing a syntax error that broke ESLint, Prettier, and the full test CI on all platforms. Reported by Claude PR review on #1156. * chore: drop test/global-setup.ts + test/vitest.d.ts Upstream removed these in commit |
||
|
Gergő Magyar
|
d3a7ce95a5
|
feat(core): adopt pino structured logger (#1336)
* feat(core): adopt pino structured logger + add no-console eslint forcing function
Adds `pino` as the project-wide structured logger via a thin wrapper at
`gitnexus/src/core/logger.ts` exposing `createLogger(name, opts?)` and a
default `logger` singleton. Migrates the only security-relevant `console.warn`
site (`bridge-db.ts` `openBridgeDbReadOnly` retry-exhaustion path) to
`bridgeLogger.debug({groupDir, err, attempts}, 'msg')`.
Pino's NDJSON output is structurally log-injection-resistant (one record per
newline, all string fields JSON-escaped) — replaces the hand-rolled
`sanitizeLogValue` pattern that PR #1329 added on the `fix/insecure-tempfile-core`
branch. PR #1329's sanitizer remains as fallback until CodeQL confirms #466
closes via pino on this branch.
Also adds an ESLint `no-console: warn` rule scoped to
`gitnexus/src/**/*.ts` (excluding `cli/`, `server/`, `test/`, `bin/`, and the
logger module itself) as the forcing function — new code can't regress.
Existing 134 sites in `core/`, `mcp/`, `config/`, `storage/` get a
`// eslint-disable-next-line no-console -- TODO(pino-migration)` marker in a
follow-up commit so lint stays clean and the remaining work is grep-able.
Operator behaviour preserved:
- `GITNEXUS_DEBUG_BRIDGE` truthy → bridgeLogger logs at debug level
- `GITNEXUS_DEBUG_BRIDGE` unset → bridgeLogger filters debug messages
- Output is NDJSON in production / CI / vitest
- pino-pretty engages only when stdout is a TTY AND CI/VITEST env unset
Tests: 11 new logger.test.ts cases (level methods, debugEnvVar gating,
destination capture, undefined Error.message safety, CR/LF/U+2028/ANSI
single-record invariant). Group test suite (388 tests) passes unchanged.
`--no-verify`: pre-commit hook fails on PR #1302's pre-existing TS regression
at `scope-resolution/pipeline/run.ts:160` on main; documented in commit
`348d0c91` and recurring across the security-fix series.
Refs: #466 (codeql js/log-injection), PR #1329 follow-up.
* chore(lint): baseline-suppress 134 existing console.* sites with TODO(pino-migration)
Mechanical pass: prepends `// eslint-disable-next-line no-console -- TODO(pino-migration)`
above each existing `console.*` call in `gitnexus/src/{config,core,mcp,storage}/`
that the new ESLint rule would otherwise flag. CLI/server are exempt at the
config level (legitimate stdout output).
Zero functional changes. Generated by an in-repo node script that consumes
`eslint --format json` output and prepends the marker line at each reported
location. Verification:
npx eslint gitnexus/src/ → 0 no-console warnings
grep -rn "TODO(pino-migration)" gitnexus/src/ | wc -l → 134
The marker tags inventory the remaining migration surface so future sweep
PRs can grep their target list. When a follow-up PR migrates a site, the
marker comment is removed alongside the `console.*` → `logger.*` swap.
`--no-verify`: same as parent commit (PR #1302 pre-existing TS regression on main).
* refactor(core): complete pino migration — replace all 134 console.* sites + flip ESLint to error
Codebase-wide sweep of every `TODO(pino-migration)` site flagged in commit
|
||
|
azizur100389
|
ee871419e1
|
feat(ingestion): GITNEXUS_INDEX_TEST_DIRS opt-in for __tests__ / __mocks__ (#771) (#1046)
* feat(ingestion): GITNEXUS_INDEX_TEST_DIRS opt-in for __tests__ / __mocks__ (#771) The DEFAULT_IGNORE_LIST hardcodes __tests__ and __mocks__ as auto-filtered directory names. The comment at ignore-service.ts:273 explicitly documents this as intentional — .gitnexusignore negation cannot override hardcoded entries. That default is right for the majority of users, but for Quality Engineering workflows where test files are the primary index target (tracing coverage via CALLS edges), there was no escape hatch short of patching the installed package. Add an opt-in env var mirroring the GITNEXUS_NO_GITIGNORE / GITNEXUS_MAX_FILE_SIZE precedent: `GITNEXUS_INDEX_TEST_DIRS=1` removes __tests__ / __mocks__ from the effective ignore set. Scope is deliberately limited to these two names — the issue asked for these specifically, and other test-adjacent entries (__snapshots__, snapshots, fixtures, .jest) remain auto-filtered unchanged. .gitnexusignore negation semantics are not touched; the env var is the orthogonal escape hatch. Implementation: new `isEffectivelyIgnoredDirectory` helper in ignore-service.ts wraps the `DEFAULT_IGNORE_LIST.has(name)` check with the env-var opt-out. Two call-sites swap: shouldIgnorePath (affects filesystem walker and wiki generator) and createIgnoreFilter.childrenIgnored (affects directory pruning during traversal). `isHardcodedIgnoredDirectory` export unchanged — its contract is "is in the raw list", which remains true for __tests__ / __mocks__ regardless of env var state (locked in by a test). Default behaviour is byte-identical for users who don't set the env var. 9 new unit tests cover default-unset, opt-in-set, scoped scope (other hardcoded entries unaffected), and the scope-discipline guard (future expansion beyond the two named dirs fails loudly). Env state restored by afterEach to prevent leakage. Closes #771. * feat(ingestion): .gitnexusignore negation overrides hardcoded DEFAULT_IGNORE_LIST (#771) Per @magyargergo's review feedback: rather than add a special-case GITNEXUS_INDEX_TEST_DIRS env var to unlock __tests__ / __mocks__, let .gitnexusignore use !pattern negation to override the hardcoded DEFAULT_IGNORE_LIST — mirroring the .gitignore mental model users already know. Implementation: - New private hasExplicitUnignore(ig, rel) helper that walks ancestor segments and uses ignore.test(path)'s `unignored` flag to detect explicit negation. Ancestor-walk is required because .gitignore negation propagates — !__tests__/ implicitly unignores every descendant, but ignore.test() only reports unignored: true on the directly-matched path. - createIgnoreFilter.ignored() and .childrenIgnored() now check hasExplicitUnignore BEFORE applying the hardcoded DEFAULT_IGNORE_LIST. If any ancestor (or the path itself) was explicitly unignored in .gitnexusignore, the hardcoded block is bypassed. - shouldIgnorePath stays pure hardcoded-list — the wiki generator and other callers without per-repo config context keep deterministic behavior. The #771 override lives only inside createIgnoreFilter, which IS called with config. Dropped: - GITNEXUS_INDEX_TEST_DIRS env var (superseded by the more general negation mechanism) - isEffectivelyIgnoredDirectory helper - Associated env-var help text and unit tests Added: - Tip in analyze --help pointing users at .gitnexusignore with !__tests__/ as the example - 8 new unit tests covering default behaviour, directory-level negation, selective overrides, generalisation (!node_modules/), non-leakage across hardcoded entries, standard non-negation rules still layering on top, and preservation of shouldIgnorePath / isHardcodedIgnoredDirectory contracts Default behaviour (no .gitnexusignore or no negation pattern) is byte-identical to pre-#771. Users who want to index an auto-filtered directory add a single !pattern line — no env var, no flag, no re-install. Closes #771. * fix(ingestion): honour re-ignore rules after .gitnexusignore negation (#771) When .gitnexusignore contains both `!__tests__/` and `__tests__/generated/`, the parent negation previously short-circuited and allowed the re-ignored child through. Consult `ig.ignores(rel)` after `hasExplicitUnignore` so a more-specific rule in the same file correctly re-ignores a subset — matching .gitignore's last-match-wins semantics. Adds a compound-pattern test locking this in. |
||
|
ivkond
|
10f8815639
|
fix(ignore): respect negation patterns in .gitnexusignore (#654)
* fix(ignore): respect negation patterns in .gitnexusignore childrenIgnored
childrenIgnored checked `ig.ignores(rel) || ig.ignores(rel + '/')` which
short-circuited on the bare path — directory-only negation patterns like
`!iOS/` were missed because `ig.ignores('iOS')` treats the path as a file.
Now only checks with trailing slash since childrenIgnored is only called
for directories. Bare-name patterns (e.g. `local`) still match per gitignore spec.
Fixes #596
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(ignore): add edge-case for bare `!dir` negation pattern
Verifies that `!iOS` (without trailing slash) also un-ignores the iOS/
directory — confirms the `ignore` package normalizes both `!dir` and
`!dir/` forms consistently when tested with a trailing-slash path.
Addresses non-blocking review suggestion on #654.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(ignore): link ignore package docs for bare-name normalization
Adds references to the `ignore` package documentation in both the
childrenIgnored comment and the bare-negation test, explaining why
`!iOS` (without trailing slash) also re-includes the iOS/ directory.
Addresses non-blocking review suggestion on #654.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Gergő Magyar
|
bf09eab95b
|
feat: configure prettier with pre-commit hook (#563)
* feat: configure prettier with pre-commit hook integration Add prettier, lint-staged, and prettier-plugin-tailwindcss at the repo root with husky pre-commit hook integration. Moves husky from gitnexus/ to root package.json for reliable hook installation. - Root package.json with prepare/format/format:check scripts - .prettierrc with endOfLine:lf and tailwindStylesheet for TW v4 - .prettierignore excluding fixtures, vendor, generated, *.d.ts, *.md - .gitattributes enforcing LF line endings for Windows consistency - Pre-commit hook uses direct node_modules/.bin/ paths (no npx) * style: apply prettier formatting to entire codebase One-time bulk format. No logic changes. Use .git-blame-ignore-revs to skip this commit in git blame. * chore: add .git-blame-ignore-revs for prettier format commit * perf: pre-commit hook runs only tests related to staged files Use vitest --related to scope test execution to tests that import the changed files, instead of running the full suite on every commit. * perf: remove vitest from pre-commit hook, keep in CI only Pre-commit now runs lint-staged + tsc only. Tests run in CI (ci-tests.yml) where they belong — keeps commits fast. * ci: add prettier format check to quality workflow PRs will now fail if code isn't formatted with prettier. |
||
|
Gergő Magyar
|
fd7fb5bf1f
|
feat: unify web and cli ingestion pipeline (#536)
* feat: add server-side ingestion API (POST /api/analyze, SSE progress)
Extract core analysis orchestration from CLI into shared run-analyze.ts
module. Add server-side analyze endpoints so the web app can trigger
ingestion via HTTP instead of running the full pipeline in-browser.
New files:
- src/core/run-analyze.ts — shared runFullAnalysis() orchestrator
- src/server/analyze-job.ts — job manager (single-slot, dedup, SSE events)
- src/server/analyze-worker.ts — forked child process (8GB heap, IPC)
- src/server/git-clone.ts — shallow clone/pull with SSRF protection
API endpoints:
- POST /api/analyze — start analysis (returns 202 + jobId)
- GET /api/analyze/:jobId — poll job status
- GET /api/analyze/:jobId/progress — SSE progress stream
Security: URL validation blocks private IPs and non-HTTP schemes.
Path validation requires absolute paths. Git stderr not leaked to API.
* feat(web): add server-side analyze UI (Phase 2)
Add "Analyze on Server" flow to the web app's Server tab so users
can trigger server-side ingestion from the browser. On completion,
the graph is automatically loaded via the existing connectToServer flow.
New files:
- AnalyzeProgress.tsx — progress bar with phase label, elapsed time, cancel
Modified files:
- backend.ts — startAnalyze(), streamAnalyzeProgress() SSE client
- DropZone.tsx — analyze URL input + button below Connect section
- App.tsx — onServerAnalyze handler wires analyze -> connect flow
* feat: add job cancellation, timeout, and child process tracking (Phase 3)
- DELETE /api/analyze/:jobId — cancel running analysis (SIGTERM to worker)
- 30-minute timeout kills long-running workers automatically
- Child process refs tracked in JobManager for cleanup on shutdown
- dispose() kills all active children on SIGINT/SIGTERM
- Web cancel button now calls server DELETE endpoint
- cancelAnalyze() added to web backend client
* refactor(web): remove browser ingestion pipeline (Phase 4)
Delete 16 duplicated ingestion files, 2 unused service files
(git-clone, zip), and tree-sitter parser-loader from gitnexus-web.
All ingestion now runs server-side via POST /api/analyze.
Deleted (18 files, ~5,000 lines):
- core/ingestion/*.ts (16 pipeline processors)
- core/tree-sitter/parser-loader.ts (WASM tree-sitter loader)
- services/git-clone.ts (isomorphic-git client-side clone)
- services/zip.ts (JSZip extraction)
Simplified:
- DropZone.tsx — server-only (removed ZIP/GitHub tabs)
- ingestion.worker.ts — removed runPipeline/runPipelineFromFiles
- useAppState.tsx — removed pipeline callbacks
- App.tsx — removed handleFileSelect/handleGitClone
- main.tsx — removed Buffer polyfill for isomorphic-git
- types/pipeline.ts — removed PipelineResult/serialize helpers
Kept: cluster-enricher.ts (LLM enrichment, still used by worker)
Dependencies now removable: web-tree-sitter, isomorphic-git,
@isomorphic-git/lightning-fs, jszip (estimated 3-4MB bundle savings)
* refactor(web): sync graph schema from CLI + delete WASM grammars
Sync graph/types.ts and lbug/schema.ts from the CLI (source of truth)
to the web module so the browser LadybugDB can handle all node and
relationship types the server pipeline produces.
Synced types: Route, Tool, Section node labels; HANDLES_ROUTE, FETCHES,
HANDLES_TOOL, ENTRY_POINT_OF, WRAPS, QUERIES relationship types;
description fields on Function/Class/Interface/Method/CodeElement.
Deleted: public/wasm/ directory (14 tree-sitter WASM grammars + core).
Removed deps: web-tree-sitter, isomorphic-git, @isomorphic-git/lightning-fs,
jszip, buffer, @types/jszip (~3-4MB bundle savings).
* feat: create gitnexus-shared package for unified type definitions
Create a new gitnexus-shared package that is the single source of truth
for types shared between the CLI and web modules:
- SupportedLanguages enum (15 languages)
- Graph types: NodeLabel, NodeProperties, RelationshipType, GraphNode, GraphRelationship
- Schema constants: NODE_TABLES, REL_TYPES, REL_TABLE_NAME, EMBEDDING_TABLE_NAME
- Pipeline types: PipelinePhase, PipelineProgress
Both gitnexus (CLI) and gitnexus-web import from gitnexus-shared via
file: dependency. Each package re-exports and extends with platform-specific
additions (CLI: KnowledgeGraph with mutation methods; Web: simpler KnowledgeGraph).
This ensures types can never drift between packages — adding a new
language, node type, or relationship type in gitnexus-shared automatically
propagates to both consumers.
* refactor: import shared types directly from gitnexus-shared at call sites
Replace all re-export patterns with direct imports from gitnexus-shared.
72 files updated across CLI and web:
- SupportedLanguages: 49 CLI files now import from 'gitnexus-shared'
instead of '../config/supported-languages.js'
- GraphNode, GraphRelationship, NodeLabel: 22 CLI + 10 web files now
import from 'gitnexus-shared' instead of local re-export wrappers
- NODE_TABLES: api.ts imports from 'gitnexus-shared'
- PipelineProgress: useAppState.tsx imports from 'gitnexus-shared'
Local types.ts files now only define platform-specific KnowledgeGraph
(CLI has mutation methods, web has add-only). No more re-exports.
* fix: update lock files for gitnexus-shared, remove stale vite polyfills
Add gitnexus-shared@1.0.0 to lock files so npm ci succeeds in CI.
Remove buffer polyfill and global define from vite.config.ts (isomorphic-git was removed).
* fix(security): add write guard to HTTP /api/query, fix CORS proxy bypass
- Add isWriteQuery() check to POST /api/query handler — blocks CREATE,
DELETE, SET, MERGE, DROP, etc. via HTTP API (guard was only in MCP
pool adapter and browser-side, not the HTTP server path)
- Extend CYPHER_WRITE_RE with CALL, INSTALL, LOAD keywords
- Fix CORS proxy subdomain bypass: endsWith('github.com') allowed
'evil-github.com'. Now requires exact match or '.github.com' suffix
* feat(server): enhance /api/search with enrichment, add /api/grep, strip graph content
- POST /api/search: add mode param (hybrid|semantic|bm25), server-side
enrichment returns connections/cluster/processes per result in one call
(collapses 31 sequential HTTP calls to 1 for the agent search tool)
- GET /api/grep: regex search across indexed file contents, eliminates
need to transfer all file contents to browser
- GET /api/graph: strip content field by default (80-95% payload
reduction). Use ?includeContent=true for backward compat
- Add LRU cache invalidation hook point for future caching
* feat(server): add /api/embed endpoint for server-side embedding generation
- POST /api/embed: triggers embedding pipeline via onnxruntime-node
with JobManager for single-slot concurrency, timeout, and dedup
- GET /api/embed/:jobId: poll job status
- GET /api/embed/:jobId/progress: SSE stream with heartbeat, event IDs,
and X-Accel-Buffering:no header for proxy compatibility
- DELETE /api/embed/:jobId: cancel running embedding job
- Maps embedding pipeline phases (ready→complete, error→failed) to
JobManager status conventions
* feat(web): create consolidated BackendClient module
Single HTTP client replacing backend.ts, server-connection.ts, and
worker HTTP helpers. Includes:
- Typed methods: runQuery, search (enriched), grep, readFile, connect
- Generic streamSSE<T> utility extracted from analyze progress pattern
- BackendError with discriminated code field (network/server/client/timeout)
- Embed API: startEmbeddings, streamEmbeddingProgress, cancelEmbeddings
- Search with mode param (hybrid|semantic|bm25) and enrichment
* refactor(web): rewrite Graph RAG tools for backend-only HTTP queries
- Search tool: uses enriched /api/search (1 call replaces 31 sequential queries)
- Cypher tool: removes browser-side embedding; {{QUERY_VECTOR}} routes to
/api/search with mode:'semantic' instead of local transformers.js
- Grep tool: uses /api/grep instead of in-memory fileContents map
- Read tool: uses /api/file instead of fileContents map lookup
- Impact tool: getCallSiteSnippet now async via /api/file
- createGraphRAGTools now accepts GraphRAGBackend interface instead of
7 separate function params + fileContents map
- createGraphRAGAgent simplified to (config, backend, context?)
- Removed imports: embedder, lbug/schema (replaced with gitnexus-shared)
- Net: -205 lines
* refactor(web): delete WASM infrastructure, remove 7 packages (-5242 lines)
Delete browser-side LadybugDB, embeddings, search, and worker:
- gitnexus-web/src/core/lbug/ (adapter, csv-generator, schema, query-result)
- gitnexus-web/src/core/embeddings/ (embedder, pipeline, text-gen, types)
- gitnexus-web/src/core/search/ (bm25-index, hybrid-search)
- gitnexus-web/src/workers/ingestion.worker.ts (828 lines)
- gitnexus-web/src/services/server-connection.ts (merged into backend-client)
- gitnexus-web/src/types/lbug-wasm.d.ts
Remove packages: @ladybugdb/wasm-core, @huggingface/transformers,
comlink, minisearch, vite-plugin-wasm, vite-plugin-top-level-await,
vite-plugin-static-copy
Update vite.config.ts: remove WASM plugins, COOP/COEP headers,
worker config, optimizeDeps exclude
Update imports: App.tsx, DropZone, Header, AnalyzeProgress,
BackendRepoSelector, useBackend → backend-client
* refactor(web): replace Worker/Comlink with direct BackendClient calls
- useAppState: remove Worker instantiation, Comlink.wrap, apiRef.
All queries now go through BackendClient HTTP functions directly.
- Agent runs on main thread (I/O-bound LLM streaming, not CPU-bound)
- initializeAgent: creates GraphRAGAgent with GraphRAGBackend interface
bound to BackendClient methods (runQuery, search, grep, readFile)
- startEmbeddings: calls POST /api/embed + SSE progress instead of
running browser-side transformers.js pipeline
- switchRepo: no longer loads graph into WASM DB or extracts fileContents
- App.tsx: handleServerConnect simplified (no fileContents, no loadServerGraph)
- Delete old backend.ts (replaced by backend-client.ts)
- Net: -396 lines
* fix(web): fix await-in-map build error in agent streaming
Move dynamic import of AIMessage outside .map() callback to avoid
"await can only be used inside an async function" build error.
* fix(web): remove stale apiRef references that broke chat functionality
sendChatMessage referenced apiRef.current (deleted Worker ref) which
would throw TypeError. Replaced with agentRef.current guard since agent
now runs on main thread.
* fix(server): dispose embedJobManager on shutdown, fix job mutation
- Add embedJobManager.dispose() to shutdown handler (was missing,
causing cleanup timer to keep Node process alive)
- Replace direct job.repoName/status mutation with updateJob() to
ensure SSE event emission for initial status change
* fix(server): parameterize Cypher, harden grep, unify SSE endpoints
- Search enrichment: replace string interpolation with executePrepared()
using $nid parameter binding to prevent Cypher injection
- Add executePrepared() to core lbug-adapter (prepare/execute pattern)
- /api/grep: add 200-char pattern length limit (ReDoS protection),
search files on disk instead of loading entire corpus into memory
(constant memory usage regardless of repo size)
- Extract mountSSEProgress() shared helper for SSE streaming — both
analyze and embed endpoints now have consistent heartbeat (30s),
event IDs (reconnection support), and X-Accel-Buffering header
* refactor(web): remove dead code from Worker-era architecture
- Remove loadServerGraph no-op function, interface member, and all consumers
- Remove testArrayParams stub and interface member
- Remove fileContents state from GraphStateProvider (never populated in
server-side architecture)
- Remove forceDevice parameter from startEmbeddings (server-side, no device choice)
- Replace phantom EmbeddingProgress type with inline { phase, percent }
- Replace resolvePathFromContents (needed fileContents Map) with graph-based
file path resolution using filePathIndex built from graph nodes
- Fix: AI citation grounding ([[file.ts:10]]) now works via graph node lookup
instead of broken fileContents-based resolution
* fix(web): use streamAgentResponse for full tool_call/reasoning streaming
Replace naive agent.stream() loop that only handled content chunks with
streamAgentResponse() generator from agent.ts. This properly routes:
- reasoning tokens (before/between tool calls)
- tool_call events (name, args, status)
- tool_result events (completed tool output)
- content tokens (final answer after all tools done)
Previously the onChunk handler for tool_call/tool_result/reasoning was
dead code since the streaming loop only emitted content events.
* fix(web): resolve CI type errors from dead code removal
- Import GraphNode/GraphRelationship from gitnexus-shared in graph.ts
(not re-exported from local types.ts)
- Add Route, Tool entries to NODE_COLORS and NODE_SIZES constants
- Add PipelineResult type to web types/pipeline.ts
- Remove fileContents from CodeReferencesPanel and RightPanel
- Remove testArrayParams and forceDevice from EmbeddingStatus
- Remove forceDevice args from startEmbeddings() calls in App.tsx
- Fix embeddingProgress property accesses for simplified type
* fix(ci): add setup-gitnexus-web action, build shared once per job
- Remove prepare script from gitnexus-shared (tsc not available during
npm ci of consuming packages)
- Create .github/actions/setup-gitnexus-web composite action: builds
gitnexus-shared then runs npm ci for gitnexus-web
- setup-gitnexus action: already builds gitnexus-shared for CLI jobs
- ci-quality typecheck-web: uses setup-gitnexus-web (DRY)
- ci-e2e: uses setup-gitnexus-web (DRY)
- ci-tests: gitnexus-shared already built by setup-gitnexus, just
install web deps without rebuilding
* fix(ci): use prepare script so gitnexus-shared builds during npm ci
Move typescript from devDependencies to dependencies in gitnexus-shared
so the prepare script (tsc) works when npm resolves file: deps during
npm ci. No GHA modifications needed — npm handles the build lifecycle
automatically.
Remove manual gitnexus-shared build steps from setup-gitnexus and
setup-gitnexus-web actions.
* fix(ci): build gitnexus-shared explicitly in setup actions
The file: dependency protocol doesn't reliably run prepare scripts
because devDependencies aren't installed first. Instead of fragile
lifecycle hacks, build gitnexus-shared explicitly in both setup actions:
- setup-gitnexus: npm install && npm run build in gitnexus-shared/
- setup-gitnexus-web: same, before npm ci in gitnexus-web/
- ci-tests: shared already built by setup-gitnexus, web just npm ci
No prepare script, no dist in git, no typescript as a prod dependency.
* fix: remove CALL from CYPHER_WRITE_RE — breaks FTS and vector search
CALL is used by read-only procedures: CALL QUERY_FTS_INDEX(...) and
CALL QUERY_VECTOR_INDEX(...). Adding it to the write guard blocked all
FTS search, causing 3 test failures. The database is opened in read-only
mode as defense-in-depth against write procedures via CALL.
Keep INSTALL and LOAD in the blocklist (genuinely dangerous).
* fix(web): update vercel.json for gitnexus-shared, remove COOP/COEP
- Add installCommand that builds gitnexus-shared before installing
web deps (Vercel doesn't know about the monorepo file: dependency)
- Remove Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy
headers (no longer needed — WASM LadybugDB removed)
* fix(web): update tests for deleted modules
- Delete csv-generator.test.ts (tests deleted WASM-only csv-generator)
- Update security-guards.test.ts: import NODE_TABLES/REL_TYPES from
gitnexus-shared instead of deleted src/core/lbug/schema
- Update server-connection.test.ts: import normalizeServerUrl from
backend-client, remove extractFileContents tests (function deleted)
* fix(e2e): remove Server tab click — UI is now server-only
The DropZone no longer has ZIP/GitHub/Server tabs (browser ingestion
was removed). The server URL input is directly visible on the landing
page. Update e2e test to skip the tab click and go straight to input.
All 5 e2e tests pass locally.
* refactor: use gitnexus-shared for PipelinePhase/PipelineProgress types
CLI was duplicating PipelinePhase and PipelineProgress locally instead
of importing from gitnexus-shared. Updated all consumers to import
directly. Also removed dead code: SerializablePipelineResult,
serializePipelineResult(), deserializePipelineResult().
* fix(server): address PR #536 review — security, race conditions, dead code
- Fix path traversal in POST /api/analyze: split into isAbsolute + normalize check
- Add shared repo lock (activeRepoPaths) preventing concurrent analyze+embed on same repo
- Fix 202 response returning actual job.status instead of hardcoded 'queued'
- Add 30-minute timeout for embedding jobs (was missing unlike analyze jobs)
- Fix DropZone calling startAnalyze without setting backend URL first
- Add SSE reconnect with exponential backoff (3 retries) and Last-Event-ID
- Fix normalizeServerUrl to return base URL (no /api suffix) — clear contract
- Delete dead code: proxy.ts, server-graph-hydration.ts, pipeline.ts re-export barrel
- Update LoadingOverlay to import PipelineProgress directly from gitnexus-shared
* fix(server): fix repo lock key mismatch and embed cancel race
- Use getStoragePath(targetPath) as lock key in analyze handler to match
embed handler's entry.storagePath — keys now always align
- Guard embed completion: don't overwrite 'failed' with 'complete' when
job was cancelled while pipeline was still running
- Remove unused jobType parameter from acquireRepoLock
- Log backend.init() errors instead of silently swallowing
* fix: add gitnexus-shared as a local dependency in package-lock.json
* refactor: move language detection to gitnexus-shared, add syntax highlighting for all 15 languages
Move getLanguageFromFilename() from CLI to gitnexus-shared with COBOL
support added. Add getSyntaxLanguageFromFilename() for Prism-compatible
syntax highlighting covering all 15 code languages plus auxiliary
formats (json, yaml, markdown, html, css, bash, sql, xml).
Refactor CodeReferencesPanel to use shared function instead of a local
30-line switch. Delete dead gitnexus-web/src/config/supported-languages.ts
(web already imports SupportedLanguages from gitnexus-shared).
* feat(web): add first-time user onboarding with auto server detection
Replace the manual "Connect to Server" panel with an automatic onboarding
flow that guides first-time users through starting the GitNexus server.
Server detection:
- useBackend hook polls via setTimeout chain (3s, no overlap)
- Page Visibility API pauses polling when tab is hidden
- SSE heartbeat (/api/heartbeat) for instant disconnect detection
Onboarding UI (OnboardingGuide.tsx):
- Step-by-step flow: copy command → run → auto-connect
- Smart command: shows `gitnexus serve` in dev, `npx gitnexus@latest serve` in prod
- Node.js version auto-detected from package.json via Vite define
- Faux terminal windows with copy-to-clipboard, platform tabs, polling indicator
Transitions (DropZone.tsx):
- Crossfade wrapper with snapshot pattern for smooth phase transitions
- Three phases: onboarding → success (1.2s hold) → loading → graph
- Auto-recovery: falls back to onboarding if server dies or connect fails
Server changes:
- GET /api/heartbeat: SSE endpoint for liveness detection
- GET /api/info: version, launch context, Node.js version
- npm run serve script for local development
- app.disable('x-powered-by') hardening
* feat(web): add repo analysis UI, SSE heartbeat, and review fixes
Repo analysis:
- AnalyzeOnboarding: empty-state card when server has zero repos
- RepoAnalyzer: GitHub URL + Local Folder tabs with browse button
- Header repo dropdown: click project badge to switch repos or analyze new
- DropZone 'analyze' phase integrated into Crossfade transitions
Reliability fixes from 5-agent review:
- Polling: stop scheduling timers when tab hidden, restart on visibility return
- Heartbeat: exponential backoff (1s/2s/4s, 3 retries) prevents graph loss on blip
- RepoAnalyzer: completion timer tracked in ref, cleaned up on unmount
- DropZone: standardized card padding (p-7), heading sizes (text-lg)
Accessibility:
- prefers-reduced-motion global CSS rule (WCAG 2.3.3)
- focus-visible rings on CopyButton
- cursor-pointer on all Header buttons
- Consistent rounded-xl on all dropdowns
Cleanup:
- Deleted dead AnalyzeSheet.tsx (219 LOC) and BackendRepoSelector.tsx (89 LOC)
- Fixed AnalyzeProgress lucide import (lucide-react → @/lib/lucide-icons)
* fix(server): resolve analyze worker fork crash in dev mode
The forked analyze worker was crashing immediately with exit code 1
when running via `npm run serve` (tsx). Two issues:
1. Worker path resolved to `analyze-worker.js` but only `.ts` exists
in the source directory — the `.js` file is only in `dist/`.
2. On Windows, bare `--import tsx` in execArgv fails because Node's
ESM resolver for --import uses the child's CWD, not the parent's
node_modules. Windows also rejects raw paths as `d:` is not a
valid URL scheme.
Fix: detect dev vs prod via `import.meta.url` extension. In dev mode,
resolve `tsx/esm` to an absolute `file://` URL via `pathToFileURL()`
anchored to the parent's `createRequire` context. This works on all
platforms and doesn't depend on the child's CWD or PATH.
Also captures child stderr for better crash diagnostics.
Verified: `POST /api/analyze` with GitHub URL completes successfully
in dev mode (tsx) — status goes from cloning → analyzing → complete.
* fix(server): add worker auto-retry, error handling, and crash diagnostics
Worker resilience:
- Auto-retry up to 2 times with exponential backoff (1s, 2s) on crash
- SSE progress shows "Retrying after crash (1/2)..." during retry
- Captures child stderr for crash diagnostics in failure message
- AnalyzeJob tracks retryCount per job
Server error handling:
- app.listen wrapped in Promise so EADDRINUSE/EACCES propagate cleanly
- serve.ts catches startup errors with friendly messages and exit code 1
- EADDRINUSE gets actionable guidance (stop other process or --port flag)
- Global uncaughtException/unhandledRejection handlers prevent silent exits
- DEBUG=1 env var shows full stack traces
* feat: add e2e tests for onboarding flows, worker retry, and error handling
E2E tests (onboarding.spec.ts — 11 tests):
- Flow 1: OnboardingGuide shown when server unreachable (6 tests)
- Flow 2: Auto-connect with success card, analyze phase for zero repos
- Flow 3: Analyze form — GitHub URL validation, Local Folder tab, tab switching
- Flow 4: Repo dropdown in exploring view (skipped without live server)
Updated server-connect.spec.ts:
- Replaced manual Connect button flow with auto-connect waitForGraphLoaded
Server resilience:
- Worker auto-retry (2 attempts with exponential backoff) on crash
- Friendly error messages for serve startup failures (EADDRINUSE etc.)
- Global uncaughtException/unhandledRejection handlers prevent silent exits
- app.listen wrapped in Promise for proper error propagation
* refactor(shared): enforce exhaustive language coverage via Record types
Replace the if/else chain in getLanguageFromFilename with two exhaustive
Record<SupportedLanguages, ...> maps:
- EXTENSION_MAP: every language → its file extensions
- SYNTAX_MAP: every language → its Prism syntax identifier
Adding a new member to the SupportedLanguages enum without adding it to
both maps now produces a TypeScript compile error:
Property '[SupportedLanguages.NewLang]' is missing in type...
This matches the existing pattern in languages/index.ts (providers table)
which already uses `satisfies Record<SupportedLanguages, LanguageProvider>`.
Three compile-time enforcement points now exist:
1. EXTENSION_MAP in language-detection.ts (file extensions)
2. SYNTAX_MAP in language-detection.ts (Prism syntax identifiers)
3. providers in languages/index.ts (LanguageProvider instances)
* feat(web): load source code from server and scroll to selected line
CodeReferencesPanel now fetches file content via GET /api/file when a
node is selected, instead of showing "Code not available in memory".
- Fetches via readFile() from backend-client when selectedFilePath changes
- Shows loading spinner while fetching
- After content loads, auto-scrolls to the selected node's startLine
- Highlights the selected line range with a cyan left border
- Cancels in-flight fetch if selection changes before it completes
Also: refactored language-detection.ts to use exhaustive Record types
(EXTENSION_MAP and SYNTAX_MAP) so adding a new SupportedLanguages enum
member without implementing extensions/syntax is a compile error.
* feat: buffered file reading for Code Inspector
Server: GET /api/file now supports ?startLine=N&endLine=M for reading
a line range instead of the entire file. Returns { content, startLine,
endLine, totalLines }.
Client: readFile() returns ReadFileResult with metadata. When selecting
a symbol (function, class, method), fetches only ±50 lines around the
symbol's startLine/endLine instead of the full file. File nodes still
fetch the entire file.
SyntaxHighlighter startingLineNumber set from the buffer offset so line
numbers are correct even for partial reads.
* fix: adapt readFile callers to new ReadFileResult return type
tools.ts: readFile comes from GraphRAGBackend interface which returns
Promise<string> (the adapter in useAppState extracts .content), so
revert the { content } destructuring back to plain string assignment.
useAppState.tsx: wrap backendReadFile with { repo } options object
and extract .content to satisfy the GraphRAGBackend interface.
* fix(web): ensure new repos appear in list immediately after analysis
Two fixes:
1. DropZone: handleAnalyzeComplete now passes the repoName through to
connectToServer so the specific newly-analyzed repo loads — not the
server's default first repo.
2. App.tsx: fetchRepos() is now awaited BEFORE handleServerConnect in
both the DropZone and Header flows. This ensures the repo list is
populated before the exploring view renders, so the new repo appears
in the header dropdown immediately without a page reload.
* feat: delete repos, re-analyze with force, select after analysis
Server — DELETE /api/repo:
- Acquires repo lock first (409 if analyze/embed in flight)
- Closes LadybugDB, deletes index + clone dir, unregisters, re-inits
- Lock released in finally block
Server — analyze complete:
- backend.init() must succeed before SSE complete fires
- If backend.init() fails, job is marked failed (not complete)
Web — Header repo dropdown:
- Re-analyze: calls POST /api/analyze with force=true, shows spinning
icon + inline progress bar via SSE
- Delete: acquires lock, aborts any running re-analysis SSE for same
repo, refreshes list, switches to next repo
- After analysis completes: refreshes repo list, connects to the
specific repo by name, loads graph, shows in explorer
- Retry with 1.5s backoff on 404 (server may still be reinitializing)
Type safety:
- err: any → err: unknown + instanceof BackendError in retry loop
- Added missing BackendRepo + BackendError imports in App.tsx
|
||
|
Gergő Magyar
|
d2cd0b676f
|
feat: add COBOL language support with regex extraction pipeline (#498)
* feat: add COBOL language support with regex extraction pipeline Standalone COBOL processor following the markdown-processor.ts pattern: - No LanguageProvider modification — COBOL uses regex, not tree-sitter - No SupportedLanguages enum change — standalone processor pattern New files: - cobol-processor.ts — orchestrator (processCobol, isCobolFile, isJclFile) - cobol/cobol-preprocessor.ts — regex state machine extraction (~888 LOC) - cobol/cobol-copy-expander.ts — COPY statement expansion with circular detection - cobol/jcl-parser.ts — JCL job/step/DD extraction - cobol/jcl-processor.ts — JCL graph node creation Extraction produces: - Module nodes (PROGRAM-ID) - Function nodes (paragraphs) - Namespace nodes (sections) - Property nodes (data items) - CALLS edges (PERFORM intra-file, CALL cross-program) - IMPORTS edges (COPY statements) - CONTAINS edges (section → paragraph hierarchy) Pipeline integration: single processCobol() call in Phase 2.6 54 new tests (33 COBOL + 21 JCL), all 3889 tests pass. * docs: document custom processor pattern in pipeline.ts Add comment block at the custom processor integration point documenting the pattern for future non-tree-sitter language additions. * feat(cobol): enrich graph with EXEC SQL/CICS, ENTRY points, MOVE data flow, PERFORM THRU Maps the remaining 60% of CobolRegexResults to the graph: - EXEC SQL blocks → CodeElement nodes + ACCESSES edges to DB tables - EXEC CICS LINK/XCTL → CodeElement nodes + cross-program CALLS edges - ENTRY points → Constructor nodes (registered for cross-program resolution) - MOVE statements → ACCESSES edges (read/write data flow tracking) - PERFORM THRU → expanded CALLS edges for range targets - File declarations → Record nodes with assignment metadata - Cross-program CALL 2nd pass: resolves unresolved targets after all programs processed * test(cobol): add 26 integration tests with exact assertions + fix CICS resolution bug Integration tests (test/integration/resolvers/cobol.test.ts): - 26 tests covering full COBOL system extraction - ALL assertions use exact toBe(N) — zero fuzzy assertions - Fixtures: CUSTUPDT.cbl, AUDITLOG.cbl, CUSTDAT.cpy, RPTGEN.cbl, RUNJOBS.jcl Bug fix (cobol-processor.ts): - CICS LINK/XCTL cross-program resolution was broken — edges were created with "resolved" reason but pointing to <unresolved> targets - Fix: use cics-link-unresolved / cics-xctl-unresolved suffix pattern matching the existing cobol-call-unresolved pattern - Second-pass resolver now patches both CALL and CICS unresolved edges All 3915 tests pass, 0 failures. * test(cobol): exhaustive 57-test suite with strict exact assertions Complete rewrite of COBOL integration tests using ground-truth approach: dump the full graph, then assert EVERY node and EVERY edge. 57 tests across 9 sections: - Node completeness: Module(3), Function(13), Namespace(2), Property(21), Record(1), CodeElement(8), Constructor(1) — exact sorted arrays - Edge completeness: 22 tests covering every type+reason combination with exact source→target pairs - Cross-program resolution: 6 tests verifying CALL, CICS LINK/XCTL, JCL - COPY expansion: copybook data items in RPTGEN - Section hierarchy: exact paragraph membership per section - Data item ownership: exact per-module breakdown - MOVE data flow: exact read/write pairs - JCL integration: job/step/dataset containment - Grand totals: CALLS(22), CONTAINS(48), IMPORTS(1), ACCESSES(7) Fixture enhancements: - CUSTUPDT.cbl: added INIT-SECTION + PROCESSING-SECTION, PERFORM THRU - AUDITLOG.cbl: added ENTRY "AUDITLOG-BATCH" - RPTGEN.cbl: added EXEC CICS XCTL Zero fuzzy assertions — every expect uses toBe(N) or toEqual([...sorted]). * fix(cobol): add removeRelationship API + single-quote CALL/COPY/ENTRY, PERFORM keyword skip Phase 0A: Add removeRelationship(id) to KnowledgeGraph interface and implementation (trivial Map.delete wrapper). Required for orphan edge cleanup in next commit. Phase 1A (from PR #500 review, modified): - RE_CALL and RE_COPY_QUOTED now match both "double" and 'single' quotes - parseSingleCopyStatement in copy-expander updated for single quotes - PERFORM_KEYWORD_SKIP set prevents UNTIL/VARYING/WITH/TEST/FOREVER from being stored as false-positive perform targets - Sequence number stripping uses /[^0-9 ]/ (preserves numeric seq numbers unlike PR #500's /\S/ which stripped them) - Normalized || to ?? for regex group extraction in copy-expander 5 new graph unit tests, all 57 COBOL integration tests pass. * fix(cobol): RE_ENTRY single-quote + remove orphan unresolved CALLS edges Phase 1B: RE_ENTRY regex now supports both "double" and 'single' quoted ENTRY targets. Uses named intermediates (entryName, usingClause) with ?? operator. USING capture group shifted from [2] to [3]. Phase 1C: Second-pass resolution now collects resolved orphan edge IDs during iteration and removes them after the loop completes, using the new graph.removeRelationship() API. Graph no longer contains phantom <unresolved>: edges alongside their resolved replacements. CALLS count drops from 22 to 18 (4 orphan edges removed). * fix(cobol): Property ID collisions + O(1) Map lookup for MOVE edges Phase 1D+3C (atomic): Property node IDs now use composite key filePath:section:level:name instead of filePath:name. This prevents duplicate data item names in different sections (e.g., STATUS in both WORKING-STORAGE and LINKAGE) from silently colliding. New generatePropertyId() helper ensures both node creation and MOVE edge lookup use the identical key formula. buildDataItemMap() replaces the O(n) findDataItemNode linear scan with O(1) Map lookup, built once per file before MOVE processing. * feat(cobol): MOVE multi-target extraction with OF/IN qualifier filtering MOVE X TO A B C now produces write edges for all targets, not just the first. extractMoveTargets() helper handles OF/IN qualified names (WS-NAME OF WS-RECORD -> target is WS-NAME), subscript stripping (WS-TABLE(I) -> WS-TABLE), and MOVE_SKIP filtering on targets. Data model: CobolRegexResults.moves.to:string -> targets:string[] MOVE CORRESPONDING stays single-target per COBOL standard. Processor MOVE loop now iterates move.targets. * feat(cobol): COPY IN/OF library, pseudotext REPLACING, dynamic CALL, PERFORM TIMES, CICS MAP unquoted Phase 2B: COPY ... IN/OF library-name now captured as metadata in CopyResolution (IN and OF are synonyms per COBOL-85 standard). Phase 2C: COPY REPLACING ==pseudotext== support. Tokenizer handles ==...== delimiters alongside "quoted" strings. Pseudotext forces EXACT type. Two-pass applyReplacing: first pass handles space-containing/ non-identifier pseudotext via global string replace; second pass handles identifier-level LEADING/TRAILING/EXACT. New test file cobol-copy-expander.test.ts with 10 tests. Phase 2E: PERFORM WS-COUNT TIMES no longer produces a false-positive perform target (checks for TIMES keyword after captured identifier). Phase 2F: Dynamic CALL via data item (CALL WS-PROG-NAME without quotes) now emits a CodeElement annotation node with description 'dynamic-call' instead of silently ignoring. Adds isQuoted:boolean to call results. Phase 3A: CICS MAP(WS-MAP-NAME) unquoted identifiers now captured. Phase 3B: Normalized || to ?? in copy-expander (done in Phase 1A). * feat(cobol): nested program support — capture multiple PROGRAM-IDs per file Phase 2D: The state machine now captures all PROGRAM-IDs, not just the first. The primary program name stays in programName; additional nested programs go into nestedPrograms[]. The processor creates separate Module nodes for each nested program, contained by the outer module, and registers them in moduleNodeIds for cross-program CALL resolution. Paragraphs/data items are not yet scoped per-program (attributed to the outer module) — full per-program scoping is a future enhancement that requires END PROGRAM boundary tracking in the state machine. * test(cobol): expand integration tests for all new language features New fixtures: - NESTED.cbl — two PROGRAM-IDs (OUTER-PROG, INNER-PROG) for nested program support testing - COPYLIB.cpy — copybook for pseudotext REPLACING test target Modified fixtures: - CUSTUPDT.cbl — single-quoted ENTRY 'ALTENTRY', multi-target MOVE (WS-AMT TO FIELD-A FIELD-B), dynamic CALL WS-PROG-NAME, COPY COPYLIB with pseudotext REPLACING, LINKAGE SECTION with LS-PARAM - RPTGEN.cbl — PERFORM WS-COUNT TIMES (false-positive guard), unquoted MAP(WS-MAP-NAME), additional data items WS-COUNT WS-MAP-NAME Integration test rewritten with 62 exact assertions covering: - 5 Module, 17 Function, 33 Property, 9 CodeElement, 2 Constructor nodes - Nested program containment (OUTER-PROG -> INNER-PROG) - Dynamic CALL annotation (CodeElement with cobol-dynamic-call) - Multi-target MOVE (UPDATE-BALANCE: 2 reads, 3 writes) - Single-quoted ENTRY (ALTENTRY under CUSTUPDT) - PERFORM TIMES guard (WS-COUNT not in CALLS) - Orphan unresolved edge removal (zero -unresolved edges) - Grand totals: 21 CALLS, 68 CONTAINS, 2 IMPORTS, 10 ACCESSES * fix(cobol): pseudotext REPLACING now applies correctly via isPseudotext flag Root cause: ==PREFIX-== matched /^[A-Z][A-Z0-9-]*$/i (trailing hyphens allowed), routing it to the second-pass EXACT identifier match where PREFIX-RECORD !== PREFIX- failed silently. Fix: Propagate isPseudotext from parseReplacingClause to CopyReplacing interface, then use it in applyReplacing first-pass condition to force global string replacement for all pseudotext entries regardless of whether the content looks like an identifier. Result: COPY COPYLIB REPLACING ==PREFIX-== BY ==WS-==. now correctly transforms PREFIX-RECORD → WS-RECORD, PREFIX-CODE → WS-CODE, etc. * refactor(cobol): per-program scoping via boundary tracking + line-range grouping State machine changes (minimal, ~30 lines): - Add RE_END_PROGRAM regex for END PROGRAM program-name. detection - Replace nestedPrograms[] with programs[] containing startLine/endLine/ nestingDepth metadata for each PROGRAM-ID in the file - Reset division/section/paragraph state on new PROGRAM-ID boundary - EOF finalization flushes remaining stack entries (single-program files) - Programs sorted by startLine (outer before inner) Processor changes: - Uses programs[] with line-range containment to find enclosing parent Module for nested programs (replaces hardcoded nestedParent logic) - programModuleIds Map tracks Module node IDs per program name Fixture: NESTED.cbl now includes END PROGRAM lines for both programs. Integration test: PREFIX-* Property nodes now correctly appear as WS-* after the pseudotext REPLACING fix from the previous commit. * feat(cobol): free-format COBOL support (>>source free) Auto-detects >>SOURCE FREE directive in the first 500 chars and switches to free-format line processing: - No column-position rules (cols 1-6 are program text, not sequence area) - Comments use *> prefix instead of col 7 indicator - No continuation line indicator - Strip inline *> comments - Skip >>SOURCE directive lines preprocessCobolSource() skips col-1-6 stripping for free-format files. Paragraph/section regexes relaxed from fixed 7-space prefix to flexible whitespace with case-insensitivity (/^\s*([A-Z][A-Z0-9-]+)\.\s*$/i). EXCLUDED_PARA_NAMES expanded with COBOL verbs (GOBACK, END-READ, etc.) to prevent false-positive paragraph detection in free-format. Also fixes: entry-point-scoring.ts crash when language is 'cobol' (MERGED_ENTRY_POINT_PATTERNS[language] was undefined → optional chaining). Benchmark on ACAS 3.01 (268 GnuCOBOL free-format programs, 10MB): - Before: 407 nodes, 393 edges (near-empty, only file nodes) - After: 4,297 nodes, 3,612 edges, 542 clusters, 11 flows * fix(cobol): relax data item regexes for free-format (^\s+ to ^\s*) RE_FD, RE_DATA_ITEM, RE_ANONYMOUS_REDEFINES, and RE_88_LEVEL all used ^\s+ which requires at least 1 leading space. In free-format mode, lines are trimmed before processing, so data items like "01 WS-FIELD PIC X." have no leading whitespace after trimming. Changed to ^\s* (zero or more spaces) which works for both fixed-format (indented lines still have spaces) and free-format (trimmed lines). ACAS benchmark (268 GnuCOBOL programs): - Before: 4,297 nodes, 3,612 edges (paragraphs only) - After: 13,832 nodes, 8,615 edges (+ data items, FDs, 88-levels) * feat(cobol): 100% structural feature coverage — GO TO, SCREEN, SD/RD, SORT, SEARCH, CANCEL, Level 66 New extractions: GO TO (CALLS edges), SCREEN SECTION data items, SD/RD alongside FD (Record nodes), SORT/MERGE USING/GIVING (ACCESSES), SEARCH (ACCESSES), CANCEL (CALLS), Level 66 RENAMES (Property), IS EXTERNAL/IS GLOBAL (Property description enrichment). ACAS: 13,951 nodes | 13,193 edges | 685 clusters | 150 flows (+53% edges from new GO TO/SORT/SEARCH/CANCEL extractions) * feat(cobol): enriched CICS extraction — file I/O, dynamic PROGRAM, queues, HANDLE ABEND EXEC CICS blocks now extract: - FILE/DATASET clause: captures VSAM file name (literal or data item ref) for READ/WRITE/REWRITE/DELETE/STARTBR/READNEXT/READPREV → ACCESSES edges - PROGRAM clause: now handles unquoted variable references (dynamic CICS program transfer) → CodeElement annotation with cics-dynamic-program reason - QUEUE clause: captures TS/TD queue names from WRITEQ/READQ → ACCESSES edges - LABEL clause: captures HANDLE ABEND error handler targets → CALLS edges - TRANSID: now handles unquoted variable references CodeElement descriptions enriched with all captured fields (map, program, transid, file, queue, label). CardDemo benchmark: +49 nodes, +33 edges from enriched CICS extraction. * feat(cobol): complete CICS command extraction — all 7 expert recommendations From COBOL expert agent analysis: 1. ENDBR added to isRead file command list 2. LOAD added to PROGRAM edge commands (alongside LINK/XCTL) 3. Two-word commands expanded: WRITEQ/READQ/DELETEQ TS/TD, HANDLE ABEND/AID/CONDITION, START TRANSID 4. Queue reason differentiated: cics-queue-read/-write/-delete 5. RETURN/START TRANSID → CALLS edges to synthetic <transid> target 6. MAP → ACCESSES edges for screen traceability 7. INTO/FROM data fields extracted → ACCESSES edges to data items Also: dataItemMap built before CICS block processing (was declared after), CodeElement descriptions enriched with all captured CICS fields. * test(cobol): strict exhaustive integration tests with exact edgeSet assertions Every edge reason has exact sorted pair assertions via edgeSet(), not just counts. Any change to extraction that adds, removes, or reorders edges will produce a precise, descriptive failure. Updated RPTGEN.cbl fixture with: - GO TO EXIT-PARAGRAPH, SORT USING/GIVING, SEARCH table - EXEC CICS READ FILE INTO, WRITEQ TS QUEUE FROM, SEND MAP FROM - EXEC CICS HANDLE ABEND LABEL, RETURN TRANSID, XCTL PROGRAM(variable) - ABEND-HANDLER and EXIT-PARAGRAPH paragraphs 46 tests covering 24 CALLS + 79 CONTAINS + 18 ACCESSES + 2 IMPORTS edges across 15 distinct edge reason codes, all with exact sorted pair lists. * fix(cobol): address 5 findings from second Claude review (compiler front-end perspective) Finding #2: Numeric sequence numbers now stripped (changed /[^0-9 ]/ to /\S/ in preprocessCobolSource). Lines like "000100 MAIN-PARAGRAPH." now have cols 1-6 blanked so paragraph regex matches correctly. Finding #11: JCL in-stream PROC ordering fixed — pre-register all PROCs into moduleNames before step processing. Steps that EXEC a PROC defined later in the same file now get CALLS edges. Finding #A: PROCEDURE DIVISION USING no longer captures calling-convention keywords (BY, VALUE, REFERENCE, CONTENT, ADDRESS, OF) as parameter names. Finding #C: SORT/MERGE USING/GIVING now captures ALL file references (multi-file), not just the first. Changed from single-match to section extraction with split. Finding #D: Section headers no longer set currentParagraph, preventing PERFORM caller misattribution to Namespace instead of Function nodes. * fix(cobol): address code review findings — ReDoS fix, perf, cleanup P1 CRITICAL — ReDoS in SORT USING/GIVING: Replaced nested-quantifier regex with safe indexOf+substring+split approach. No backtracking possible on crafted input. P2 — readCopy O(M) linear scan: Added copybookByPath reverse Map for O(1) path-to-content lookup. P3 — Dead code removal: Deleted unused RE_SORT_USING and RE_SORT_GIVING constants. P3 — EXCLUDED_PARA_NAMES simplification: Replaced 20 END-* entries with startsWith('END-') prefix check. Auto-covers future END-* verbs. P3 — Misplaced JSDoc on removeRelationship: Fixed comment that described removeNodesByFile instead. Added missing JSDoc to removeNodesByFile. Review agents: architecture-strategist, performance-oracle, security-sentinel, code-simplicity-reviewer. * refactor: add Cobol to SupportedLanguages with parseStrategy: standalone New languages/cobol.ts — standalone regex processor provider with no-op tree-sitter fields. Declares parseStrategy: 'standalone' to distinguish from tree-sitter-based languages. Added parseStrategy: 'tree-sitter' | 'standalone' to LanguageProviderConfig for languages that use their own processor instead of tree-sitter. Removed all 11 'cobol' as any casts — now uses SupportedLanguages.Cobol. Added empty Cobol entries to entry-point-scoring and framework-detection. * fix(cobol): 5 fixes from third Claude review + 3 regression tests Fixes: - Line numbers now 1-indexed in fixed-format (was 0-indexed, off-by-one in jump-to-definition links) - Copybook content preprocessed before COPY expansion (sequence numbers and patch markers in copybooks no longer survive into expanded source) - ENTRY USING filters calling-convention keywords (BY, VALUE, REFERENCE, CONTENT, ADDRESS, OF) — same fix as PROCEDURE DIVISION USING - SORT/MERGE trailing period stripped from USING/GIVING file tokens - Paragraph exclusion uses exact match for SECTION/DIVISION (was substring match that excluded valid names like CROSS-SECTION-ANALYSIS) USING_KEYWORDS moved to module scope for reuse by both PROCEDURE DIVISION USING and ENTRY USING handlers. New unit tests: - ENTRY USING BY VALUE filtering - Paragraph names containing SECTION not excluded - Numeric sequence numbers stripped enabling paragraph detection * fix(cobol): address 6 findings from fourth Claude review + tests Fourth review findings fixed: - New #IV: PERFORM TIMES guard uses perfMatch.index instead of line.indexOf (prevents wrong match when target appears earlier in line) - New #V: 88-level condition values now handle single-quoted literals ('Y' no longer stored with embedded quotes) - New #I: CANCEL edges use two-pass resolution like CALL (no longer silently dropped when target indexed after source) - New #3: Multi-line SORT/MERGE accumulation — sortAccum state variable accumulates lines until period, then extracts USING/GIVING from full statement (95% of production SORT statements span multiple lines) - New #II: PROCEDURE DIVISION USING on split lines — pendingProcUsing flag defers parameter capture to next line if USING not on same line - New #6 (prior): EXCLUDED_PARA_NAMES exact match for SECTION/DIVISION Updated fixture: RPTGEN.cbl SORT now uses multi-line format with GIVING on separate line (period-terminated). New sort-giving integration test. ACCESSES total: 18 → 19 (new sort-giving edge from multi-line capture). * fix(cobol): address 4 findings from fifth Claude review Finding #B (5 reviews old): Section/paragraph node IDs now include enclosing program name to prevent collision when nested programs share section/paragraph names. New findOwningProgramName() helper uses programs[] line ranges to find the innermost enclosing program. Finding #α: pendingProcUsing now reset in the if(procUsingMatch) branch (was only set in else branch, could leak across nested programs). Finding #β: RE_CALL_DYNAMIC uses negative lookbehind (?<![A-Z0-9-]) to prevent false-positive on compound identifiers like WS-CALL OCCURS. Finding #γ: sortAccum flushed at EOF (parallel to flushSelect and pendingFdName EOF cleanup). Prevents silent loss of SORT USING/GIVING relationships in truncated files. * fix(cobol): address findings from reviews 5+6 with full test coverage Review 5 fixes: - #α: pendingProcUsing reset in if(procUsingMatch) branch - #β: RE_CALL_DYNAMIC negative lookbehind prevents WS-CALL false positive - #γ: sortAccum flushed at EOF for truncated files - #B: Section/paragraph IDs include owning program name Review 6 fixes: - #P: sectionNodeIds/paraNodeIds maps use program-scoped keys (PROGNAME:NAME). New scopedParaLookup/scopedCallerLookup helpers. findContainingSection updated with programs parameter. - #Q: RETURNING added to USING_KEYWORDS for COBOL 2002+ - #R: RE_PERFORM matches both THRU and THROUGH via alternation New unit tests (6): - PERFORM THROUGH captures thruTarget - PROCEDURE DIVISION USING RETURNING filters keyword - RE_CALL_DYNAMIC no false-match on WS-CALL compound identifier - Multi-line SORT captures USING/GIVING from continuation lines - PROCEDURE DIVISION USING on split line via pendingProcUsing - Copybook preprocessing strips sequence numbers * fix(cobol): address findings from seventh Claude review + 3 tests Review 7 fixes: - #i: findContainingSection only updates best when lookup succeeds (prevents undefined overwriting valid parent section) - #ii: RE_PROC_SECTION handles segment numbers (SECTION 30.) - #III: procedureUsing now stored per-program on boundary stack entries, propagated to programs[] output. Inner programs no longer overwrite outer program's parameters. - #δ: Dynamic CANCEL (CANCEL variable) now creates CodeElement annotation node, matching dynamic CALL behavior. RE_CANCEL_DYNAMIC with negative lookbehind. cancels[] gains isQuoted field. - #Q: RETURNING added to USING_KEYWORDS (already in prev commit) - #R: PERFORM THROUGH already fixed (THRU|THROUGH alternation) New unit tests: - Nested programs carry per-program procedureUsing - SECTION with segment number detected - Dynamic CANCEL via data item captured with isQuoted=false * feat(cobol): link PROCEDURE DIVISION USING to LINKAGE data items + close 4 findings Finding #10 FIXED: procedureUsing parameters now create ACCESSES edges with reason 'cobol-procedure-using' from Module to matching LINKAGE SECTION Property nodes. This exposes the program's parameter contract in the graph (e.g., AUDITLOG → LS-CUST-ID, AUDITLOG → LS-AMOUNT). Findings closed by expert agent consensus: - #6 COPY IN library: WONTFIX — captured metadata, no universal library-to-directory mapping exists. Field costs nothing and is useful for library queries. - #14 SQL DELETE: WONTFIX — DB2 requires FROM; existing FROM pattern handles it. Bare DELETE would risk false positives. - #E OCCURS DEPENDING ON: WONTFIX — runtime sizing concern, not structural. The static occurs count is sufficient for indexing. All 39 findings from 7 Claude reviews now resolved or closed. * fix(cobol): resolve 48 review findings across 9 review cycles Ninth deep review resolved all remaining COBOL parser gaps identified by 5 specialist agents (COBOL expert, architecture strategist, TypeScript reviewer, security sentinel, code simplicity reviewer). Fixes (P1 — critical): - SELECT OPTIONAL now correctly skips OPTIONAL keyword (C1) - RETURNING params excluded from PROCEDURE DIVISION USING list (C7) - SORT GIVING no longer captures clause keywords as file names (C5) - Extract flushSort() helper eliminating 40-line duplication (S2) - Flush unclosed EXEC blocks at EOF matching SORT/SELECT pattern (S3) - Guard undefined map key in jcl-processor moduleNames (S1) - Add MAX_TOTAL_EXPANSIONS=500 to prevent exponential COPY breadth (S4) Fixes (P2 — important): - Quote-aware stripInlineComment for | and *> in string literals (C2+C3) - Fixed-format literal continuation now handles quoted strings (C6) - PROGRAM-ID detected regardless of division state for siblings (C9) Fixes (P3 — cleanup): - EXEC SQL INTO restricted to INSERT INTO to avoid FETCH false-pos (C8) - Copy expander line numbers fixed from 0-based to 1-based (C11) - Remove dead code: inInStreamProc, fileIsLiteral, expansionDepth (S7-S10) Also fixes 8th-review findings: nested program CONTAINS attribution, multi-PERFORM on same line, INPUT/OUTPUT PROCEDURE IS in SORT, GO TO DEPENDING ON multi-target, MOVE CORR abbreviation, per-program procedureUsing ACCESSES edges. Tests: 145 COBOL tests passing (59 integration + 86 unit) Benchmarks: CardDemo 12,323 nodes/8,893 edges (7.4s) ACAS 14,016 nodes/15,452 edges (9.3s, -9% faster) * docs(cobol): update documentation for ninth review cycle fixes Update all 4 COBOL documentation files to reflect the 16 fixes from the ninth review cycle: - regex-extraction.md: quote-aware comment stripping, SELECT OPTIONAL, RETURNING exclusion, SORT_CLAUSE_NOISE filter, flushSort() helper, GO TO multi-target, PROGRAM-ID division-independent detection - copy-expansion.md: MAX_TOTAL_EXPANSIONS=500 breadth guard, 1-based line numbers, removed expansionDepth/warnedCircular param - deep-indexing.md: GO TO DEPENDING ON, INPUT/OUTPUT PROCEDURE IS, MOVE CORR edge reasons, INSERT INTO restriction, literal continuation - performance.md: updated benchmarks (CardDemo 12,323n/8,893e/7.4s, ACAS 14,016n/15,452e/9.3s), COPY breadth guard * fix(cobol): resolve 10th review findings — nested program edge attribution Fix 6 findings from the 10th review (PR #498 comment #4132201110): #A+#F: All CALL/CANCEL/CICS/ENTRY/SQL/SEARCH/file-declaration edges now use owningModuleId() for nested program attribution instead of the outer program's parentId. Added helper function owningModuleId() to centralize the pattern. #B: Added USING and GIVING to SORT_CLAUSE_NOISE set to prevent MERGE USING + OUTPUT PROCEDURE from capturing clause keywords as file names. #C: INPUT/OUTPUT PROCEDURE regex now captures optional THRU/THROUGH range end paragraph, mirroring RE_PERFORM's THRU support. #D: scopedCallerLookup fallback now uses programModuleIds.get(pgm) instead of parentId, so PERFORM/MOVE/GOTO in nested programs with unresolvable paragraphs fall back to the correct inner module. #E: pendingProcUsing only set when PROCEDURE DIVISION line is NOT period-terminated, preventing false USING expectation. Tests: 145 passing | TypeScript clean * fix(cobol): resolve 10th review findings — nested program edge attribution Fix 6 findings from the 10th review (PR #498 comment #4132201110): #A+#F: All CALL/CANCEL/CICS/ENTRY/SQL/SEARCH/file-declaration edges now use owningModuleId() for nested program attribution instead of the outer program's parentId. Added helper function owningModuleId() to centralize the pattern. #B: Added USING and GIVING to SORT_CLAUSE_NOISE set to prevent MERGE USING + OUTPUT PROCEDURE from capturing clause keywords as file names. #C: INPUT/OUTPUT PROCEDURE regex now captures optional THRU/THROUGH range end paragraph, mirroring RE_PERFORM's THRU support. #D: scopedCallerLookup fallback now uses programModuleIds.get(pgm) instead of parentId, so PERFORM/MOVE/GOTO in nested programs with unresolvable paragraphs fall back to the correct inner module. #E: pendingProcUsing only set when PROCEDURE DIVISION line is NOT period-terminated, preventing false USING expectation. Tests: 145 passing | TypeScript clean * fix(cobol): resolve 11th review findings — final nested program + multi-CALL gaps #1: scopedCallerLookup(null) now uses owningModuleId(lineNum) instead of parentId, fixing PERFORM/MOVE/GOTO before first paragraph in nested programs. #2+#3: CALL and CANCEL extraction now uses matchAll (global flag) to capture multiple occurrences on the same line. Dynamic CALL/CANCEL checked independently instead of in else branch. #4: SORT/MERGE ACCESSES edge IDs now use owningModuleId(sort.line) instead of parentId for nested program correctness. #5: preprocessCobolSource free-format detection now uses first 10 lines (consistent with extractCobolSymbolsWithRegex threshold). #6: EXCLUDED_PARA_NAMES expanded with DISPLAY, ACCEPT, WRITE, READ, REWRITE, DELETE, OPEN, CLOSE, RETURN, RELEASE, SORT, MERGE to prevent false-positive paragraph detection on isolated verbs. Also removed unused GraphNode import from cobol-processor.ts. Tests: 145 passing | TypeScript clean * docs(cobol): deepened full language coverage plan with research findings 3 research agents analyzed Phase 1-2 features and graph value ranking. Key findings: cobol-call-using is #1 edge type (9.2/10); multi-line accumulation is dominant challenge; DECLARATIVES is lowest-risk Phase 2 item; SET TO TRUE covers 80-90% of SET usage. * feat(cobol): implement Phase 1 — high-value data flow edges 4 new extraction features that create new ACCESSES and IMPORTS edges: 1.1: EXEC SQL INCLUDE -> IMPORTS edges with reason 'sql-include' Handles unquoted (SQLCA), quoted ('DBRMLIB.MEMBER'), and underscored (CUST_TBL_DCL) member names. 1.2: CALL USING parameter extraction -> ACCESSES edges Extracts parameters from CALL USING clause, filtering BY/REFERENCE/ CONTENT/VALUE/ADDRESS/OF/LENGTH/OMITTED keywords. Creates 'cobol-call-using' ACCESSES edges (graph value: 9.2/10). 1.4: OCCURS DEPENDING ON -> ACCESSES edges with reason 'cobol-depends-on' Extended OCCURS regex captures DEPENDING ON field with subscript stripping. Creates dependency edge from table to controlling field. 1.5: VALUE clause for standard data items Extracts VALUE from data item clauses: quoted strings with type prefix (X/N/G/B), ALL literals, numerics (incl negative/decimal), and figurative constants. Populates Property node values. Tests: 145 passing (+2 ACCESSES from CALL USING) | TypeScript clean * feat(cobol): implement Phase 2 — DECLARATIVES, SET, INSPECT, EXEC DLI 4 new extraction features for error handling, data flow, and IMS/DB: 2.1: EXEC DLI (IMS/DB) -> CodeElement + ACCESSES edges Accumulates EXEC DLI blocks like EXEC SQL. Parses DLI verbs (GU, GN, ISRT, REPL, DLET, CHKP, SCHD, TERM). Extracts SEGMENT, PCB, INTO/FROM, PSB. Creates dli-{verb} ACCESSES edges to <ims>:segment Record nodes. 2.2: DECLARATIVES / USE AFTER EXCEPTION -> ACCESSES edges Tracks inDeclaratives state. Detects USE AFTER STANDARD EXCEPTION ON file-name. Creates cobol-error-handler ACCESSES edge from handler section to file Record. 2.3: SET statement -> ACCESSES edges Detects SET TO TRUE (80-90% of SET usage) and SET index TO/UP BY/DOWN BY. Creates cobol-set-condition / cobol-set-index write edges + cobol-set-read for identifier values. 2.4: INSPECT -> ACCESSES edges with multi-line accumulator Accumulates INSPECT until period (like SORT). Extracts inspected field + tally counters. Creates cobol-inspect-read/write/tally edges. Form detection: tallying/replacing/converting/combined. Preprocessor: 1398 -> 1597 LOC (+199). Tests: 145 passing. * feat(cobol): implement Phase 3 — completeness fixes 6 partial features fixed to first-class support: 3.1: CALL RETURNING -> ACCESSES write edge (cobol-call-returning) 3.2: SELECT OPTIONAL flag preserved in FileDeclaration + Record node 3.3: ALTERNATE RECORD KEY extraction (matchAll for multiple keys) 3.4: COMMON attribute on nested programs (RE_PROGRAM_ID extended) 3.5: IS EXTERNAL / IS GLOBAL as first-class boolean properties (removed usage string hack) 3.6: AUTHOR / DATE-WRITTEN mapped to Module node description Tests: 145 passing | TypeScript clean * feat(cobol): implement Phase 4 — INITIALIZE + metadata completeness 4.1: INITIALIZE statement -> ACCESSES write edge (cobol-initialize) 4.2: DATE-COMPILED and INSTALLATION paragraphs extracted and mapped to Module node description alongside existing AUTHOR/DATE-WRITTEN All 4 plan phases complete. Coverage: ~95% (up from 71.9%). Tests: 145 passing | TypeScript clean * test(cobol): add 24 unit tests for Phase 1-4 features Coverage for all new extraction features: Phase 1 (8 tests): - EXEC SQL INCLUDE (unquoted, quoted, underscored) - CALL USING (simple, mixed modes, ADDRESS OF, OMITTED) - CALL RETURNING - OCCURS DEPENDING ON - VALUE clause (string, numeric, figurative constant) Phase 2 (10 tests): - EXEC DLI GU/ISRT/SCHD (verb, segment, PCB, INTO, FROM, PSB) - DECLARATIVES USE AFTER EXCEPTION (single + multiple sections) - SET TO TRUE, SET index UP BY - INSPECT TALLYING, INSPECT REPLACING Phase 3-4 (6 tests): - SELECT OPTIONAL flag - ALTERNATE RECORD KEY - PROGRAM-ID IS COMMON - IS EXTERNAL / IS GLOBAL booleans - INITIALIZE extraction - Full programMetadata (AUTHOR, DATE-WRITTEN, DATE-COMPILED, INSTALLATION) Total: 168 tests passing (145 + 24 - 1 removed duplicate) * fix(cobol): use /\r?\n/ split for Windows CRLF compatibility All 4 COBOL source files now split on /\r?\n/ instead of '\n' to handle CRLF line endings on Windows. Previously, trailing \r in lines caused RE_GOTO's $ anchor to fail on multi-line GO TO DEPENDING ON statements, producing only 1 goto edge instead of 4. Files fixed: cobol-preprocessor.ts (2 sites), cobol-processor.ts, jcl-parser.ts, cobol-copy-expander.ts Tests: 168 passing | TypeScript clean * fix(cobol): resolve 12th review — dynamic CALL/CANCEL dedup + trailing anchors #1+#2: Removed incorrect hasQuotedCall/hasQuotedCancel deduplication guards. RE_CALL_DYNAMIC and RE_CANCEL_DYNAMIC require [A-Z] after CALL/CANCEL, so they CANNOT match quoted targets — the guards were both unnecessary and actively harmful, suppressing dynamic CALL/CANCEL in ON EXCEPTION patterns. #3+#5: Changed RE_CALL_DYNAMIC and RE_CANCEL_DYNAMIC trailing anchor from (?:\s|\.) to (?=\s|\.|$) (lookahead). The consuming anchor failed when the identifier was the last token on a physical line. Tests: 168 passing | TypeScript clean * feat(cobol): add CALL accumulator + fix SORT double-statement (#4, #6) Finding #4: Multi-line CALL USING accumulator Added callAccum state variable that accumulates CALL statements spanning multiple physical lines until period or END-CALL is found. Uses flushCallAccum() to re-extract CALL target + USING parameters from the full accumulated statement. This fixes the silent loss of ACCESSES parameter edges when USING appears on lines after CALL. Finding #6: SORT double-statement on same line After flushSort(), the code now falls through to re-check the current line for a new SORT/MERGE start (was previously blocked by the sortAccum === null check evaluating before flushSort ran). Also fixed: used non-global regex for CALL detection test to avoid the classic global regex .test() lastIndex bug. Tests: 168 passing (+1 ACCESSES from multi-line CALL USING) * fix(cobol): resolve 13th review — CICS LOAD, USING extraction, file scoping #1: CICS LOAD unresolved edge no longer silently deleted in second pass. Changed narrow cics-link/cics-xctl check to catch-all pattern: rel.reason?.startsWith('cics-') && rel.reason.endsWith('-unresolved') #2: flushCallAccum USING extraction now stops before COBOL statement verbs (INSPECT, SEARCH, SORT, MERGE, DISPLAY, ACCEPT, MOVE, PERFORM, GO TO, CALL, IF, EVALUATE). Prevents absorbing adjacent statements as false USING parameters in legacy pre-COBOL-85 code without END-CALL. #3: CICS FILE Record nodes now globally-scoped (<cics-file>:FILENAME) instead of per-file-scoped. Enables cross-program CICS file access analysis, consistent with SQL table scoping (<db>:TABLE). #4: callAccum pre-check regex now has (?<![A-Z0-9-]) lookbehind to prevent false activation on compound identifiers like WS-CALL-FLAG. Tests: 168 passing | TypeScript clean * fix(cobol): resolve 14th review — callAccum false paragraph + Area A guard #1: callAccum continuation lines now check for COBOL statement verb starts (GO TO, PERFORM, MOVE, etc.) and paragraph/section headers. If detected, the CALL is flushed as-is and the line processed normally — prevents false paragraph detection and currentParagraph corruption from lines like "WS-ADDR." being treated as paragraphs. #4: callAccum pre-check now guarded by currentDivision === 'procedure' to prevent unnecessary activations in DATA DIVISION. #5: Fixed-format paragraph detection now rejects lines with >7 leading spaces (Area B indentation) as paragraph candidates. Paragraph names in fixed-format must start in Area A (col 8-11, max 7 spaces). Free-format mode is unaffected. Tests: 168 passing | TypeScript clean * fix(cobol): resolve 15th review — callAccum Area A + verb boundary fixes #A: Column-position-aware paragraph detection in callAccum flush. #B: inspectAccum early-flush on paragraph/section/verb headers. #C: Verb boundary \b → (?:\s|$) prevents MOVE-COUNT false flush. * test(cobol): add 17 edge-case regression tests + fix USING verb boundary 17 new tests covering all recurring review patterns: Multi-line CALL USING (7 tests): - Parameters on separate continuation lines (IBM mainframe style) - No absorption of INSPECT/GO TO/paragraphs following CALL - END-CALL scope terminator - Hyphenated identifiers (MOVE-COUNT) not triggering false flush - Dual quoted+dynamic CALL on same line (ON EXCEPTION) Nested program attribution (2 tests): - CALL in inner program within inner line range - PERFORM before first paragraph has null caller CRLF compatibility (1 test): - GO TO DEPENDING ON with \r\n line endings Area A paragraph detection (2 tests): - Area B (>7 spaces) rejected; Area A (7 spaces) accepted SORT/MERGE (1 test): COLLATING SEQUENCE keywords not captured PROCEDURE USING (2 tests): RETURNING excluded, period-terminated Comment stripping (1 test): pipe in quoted string preserved SELECT OPTIONAL (1 test): correct file name, not OPTIONAL keyword Bug fix: USING extraction regex verb terminators changed from \bVERB\b to \bVERB(?=\s|$) in flushCallAccum — prevents truncation on hyphenated identifiers like MOVE-COUNT, PERFORM-LIMIT. Total: 185 tests passing * test(cobol): add 32 comprehensive edge-case regression tests 13 new describe blocks covering all extraction features: - EXEC DLI: no-SEGMENT, multi-line accumulation (2 tests) - SET: multiple targets, DOWN BY, TO numeric (3 tests) - INSPECT: CONVERTING, multiple counters, tallying-replacing, paragraph flush during accumulation (4 tests) - DECLARATIVES: no-STANDARD keyword, I-O mode, post-END paragraphs (3) - COPY REPLACING: pseudotext deletion ==OLD== BY ==== (1 test) - VALUE: hex literal, negative numeric, ALL literal (3 tests) - OCCURS: TO range, fixed-size without DEPENDING ON (2 tests) - Dynamic CALL/CANCEL: end-of-line, multiple CANCELs (3 tests) - EXEC SQL: INCLUDE skips tables, SELECT INTO host vars, host variable extraction (3 tests) - INITIALIZE: target and caller context (1 test) - Nested programs: sibling scoping, PROGRAM-ID without ID DIV (2) - EXEC EOF flush: unclosed EXEC SQL flushed (1 test) - Multi-PERFORM: IF/ELSE dual PERFORM on single line (1 test) - IS EXTERNAL: USAGE not polluted by external flag (1 test) Total: 215 tests passing * fix(cobol): resolve 16th review — CANCEL in CALL block + USING boundary #1: flushCallAccum now extracts CANCEL statements from within CALL ON EXCEPTION blocks. Adds RE_CANCEL + RE_CANCEL_DYNAMIC matchAll passes alongside existing CALL extraction. #2: Added \bCANCEL(?=\s|$) to USING lookahead regex to prevent CANCEL keyword being captured as false USING parameter. #3: Multi-line CALL start now returns immediately to prevent the CALL start line from simultaneously feeding sortAccum/inspectAccum. #6: Division transitions now flush all active accumulators (callAccum, sortAccum, inspectAccum) to prevent state leakage across programs. Also added CANCEL to callAccum flush trigger verb list. Tests: 215 passing | TypeScript clean * refactor(cobol): extract shared verb constants + resolve 17th review Extract COBOL_STATEMENT_VERBS, RE_STATEMENT_VERB_START, and RE_USING_PARAMS as shared constants — eliminates 4 duplicated 25-verb regex patterns. 17th review: #1 flushCallAccum before EXEC entry, #2 inspectAccum verb parity via shared constant. Tests: 215 passing | TypeScript clean * test(cobol): replace all fuzzy assertions with exact toBe checks Replaced 7 toBeGreaterThan/toBeLessThan/toBeGreaterThanOrEqual assertions with exact toBe values: - dataItems.length: >= 3 → toBe(3) - calls.length: >= 1 → toBe(1) - calls[0].line: range check → toBe(10) - programs[].startLine/endLine: comparison → exact values - innerA.endLine/innerB.startLine: comparison → exact values Also added 11 new edge-case tests (accumulator flush on EXEC/division transitions, free-format, CANCEL in CALL block, SORT THRU, verb flush, integration). 226 tests passing — zero fuzzy assertions remain. * fix(cobol): resolve 19th review + 15 accumulator flush tests Fixes: #1: END PROGRAM flushes callAccum/sortAccum/inspectAccum #2: PROGRAM-ID sibling path flushes all accumulators #3: Added COMPUTE/ADD/SUBTRACT/MULTIPLY/DIVIDE/STRING/UNSTRING to COBOL_STATEMENT_VERBS (now 32 verbs) Tests (15 new): - END PROGRAM flush: single + nested programs (2) - PROGRAM-ID sibling flush (1) - Arithmetic verb flush: COMPUTE/ADD/SUBTRACT/MULTIPLY/DIVIDE (5) - String verb flush: STRING/UNSTRING (2) - Arithmetic not captured as false USING params (1) - SORT flushed at END PROGRAM (1) - INSPECT flushed at END PROGRAM (1) - All with exact toBe assertions (2) Total: 239 tests passing | Zero fuzzy assertions * fix(cobol): resolve 20th review — INITIALIZE multi-target + 2 tests Finding 1: INITIALIZE now captures multiple targets with REPLACING clause keyword filtering. Regex changed to lazy match stopping at REPLACING/WITH/period boundary. Targets split on whitespace and filtered against INITIALIZE_CLAUSE_KEYWORDS set. Tests (2 new): - INITIALIZE multi-target: WS-CUSTOMER WS-ORDER WS-LINE-ITEM → 3 - INITIALIZE with REPLACING: only WS-RECORD captured, not keywords Total: 241 tests passing | TypeScript clean |
||
|
Wojciech Guziak
|
8864a0290c
|
feat: add Dart language support (#204) | ||
|
Gergő Magyar
|
7999b6ba7b
|
refactor: SICP-informed LanguageProvider architecture (#488)
* refactor: SICP-informed LanguageProvider architecture for ingestion pipeline Consolidate 16 scattered dispatch surfaces into a single LanguageProvider Strategy interface per language. Processors are now fully language-agnostic — zero SupportedLanguages.X enum access, zero dispatch table imports. Architecture (5-layer DAG, zero circular dependencies): L0: Capability modules (dispatch tables, single source of truth) L1: LanguageProvider interface + createLanguageProvider factory L2: 13 per-language provider files (Strategy objects) L3: Registry with satisfies Record<SL, LP> + pre-built lookup maps L4: Processors (language-agnostic, all behavior via provider.*) Key changes: - Add LanguageProvider interface with 15 properties (6 required, 9 optional) - Create 13 provider files in languages/ + php-helpers.ts - Migrate all processors to getProvider(language) — cached once per scope - Replace heritage if-checks with provider.interfaceNamePattern/heritageDefaultEdge - Replace MRO switch(language) with switch(provider.mroStrategy) - Replace isNodeExported with provider.exportChecker - Move PHP description extraction behind provider.descriptionExtractor - Move Swift implicit imports behind provider.implicitImportWirer - Move PHP route detection behind provider.isRouteFile - Move Kotlin wildcard append behind provider.importPathPreprocessor - Remove deprecated TypeEnvironment.env, add fileScope()/allScopes() - De-export TypeEnv type (module-private) - Pre-build extensionMap, WILDCARD_LANGUAGES, SYNTHESIS_LANGUAGES at load - Remove dead entryPointPatterns/frameworkPatterns from interface - Derive createLanguageProvider config type via Pick/Partial/Omit - Tighten callback types from any to SyntaxNode - Migrate 270+ test call sites from .env to TypeEnvironment API Adding a new language: 3 files (enum + provider + registry line). No processor file touched. Ever. * refactor: clean architecture for LanguageProvider with O(1) AST cache Address all PR #488 review comments and achieve pristine SICP layer separation: Interface redesign: - Split LanguageProvider into Config (input) + Provider (runtime with defaults) - Rename createLanguageProvider → defineLanguage with explicit DEFAULTS constant - Add MroStrategy, ImportSemantics named type aliases for better IDE tooltips - Tighten labelOverride signature: string|null → NodeLabel|null (compile-time safety) - Tighten descriptionExtractor nodeLabel: string → NodeLabel - Un-export LanguageProviderConfig (internal to defineLanguage) CI fixes (all 4 failures resolved): - isNodeExported: add null guard for unknown languages - preprocessImportPath tests: pass getProvider() instead of raw enum - MRO tests: update expected strings to match language-agnostic prefixes Code deduplication: - Extract findDescendant/extractStringContent to ast-helpers.ts (single source of truth) - Unify Kotlin method detection: remove duplicate from extractFunctionName, use provider.labelOverride as single source of truth via findEnclosingFunctionId - extractFunctionName return type: string → NodeLabel Performance (O(1) AST node access): - Add per-file Map-based memoization in parse-worker for parent-chain walks - Cache enclosingClassId, enclosingFunctionId, exportStatus per SyntaxNode - Clear caches before each file parse (not after — handles parse failures) Architecture (pristine languages/ folder): - Move php-helpers.ts → helpers/php.ts (L0 capability, not L2 config) - Create helpers/swift.ts from extracted Swift provider logic - Extract cppLabelOverride AST walk → isCppInsideClassOrStruct in ast-helpers.ts - Extract isPhpRouteFile → helpers/php.ts - All 13 provider files are now pure configuration — zero implementation logic - Ruby: remove no-op namedBindingExtractor assignment (undefined from dispatch table) * refactor: eliminate LANGUAGE_QUERIES, typeConfigs, namedBindingExtractors dispatch tables Phase 1 of L0 dispatch table elimination. Providers now import capabilities directly instead of indexing into redundant Record<SL, T> dispatch tables: - LANGUAGE_QUERIES: providers import named query constants directly (TYPESCRIPT_QUERIES, PYTHON_QUERIES, etc.). Table kept in tree-sitter-queries.ts for call-processor.ts dynamic lookup + test consumers. - typeConfigs: providers import from individual type-extractor files (typescriptConfig from typescript.ts, javaTypeConfig from jvm.ts, etc.). Dispatch table fully removed from type-extractors/index.ts. - namedBindingExtractors: providers import extractors directly from named-binding-extraction.ts (extractTsNamedBindings, etc.). Dispatch table fully removed from import-resolution.ts. Net: -48 LOC of dispatch table indirection. L3 satisfies Record<SL, LP> remains the single exhaustiveness check. * refactor: eliminate exportCheckers, callRouters, importResolvers dispatch tables Phase 2 of L0 dispatch table elimination. All 6 dispatch tables are now gone: - exportCheckers: individual checkers exported directly (tsExportChecker, pythonExportChecker, etc.). isNodeExported uses a local checkersByLanguage map to avoid circular dependency with languages/index.ts. - callRouters: table removed. Providers import noRouting or routeRubyCall directly. noRouting now exported. Dead import removed from call-processor.ts. - importResolvers: resolver functions exported with clean names (resolveTypescriptImport, resolveJavaImport, etc.). Inline lambdas extracted to named exports. Dispatch functions renamed from *Dispatch suffix to clean resolve*Import pattern. Combined with Phase 1, all 6 L0 dispatch tables have been eliminated. L3 satisfies Record<SL, LanguageProvider> is the single exhaustiveness check. Providers are now fully self-contained — each imports its capabilities directly. * perf+refactor: type-env caching, sequential fallback caching, utils.ts split Phase 3 — performance optimizations and barrel cleanup: Type-env parent-walk caching: - Memoize findEnclosingClassName and findEnclosingParentClassName with per-file Map<SyntaxNode, string|undefined> caches - Eliminates O(n*m) repeated child scanning in extractParentClassFromNode - Caches cleared in buildTypeEnv before each file's walk phase Sequential fallback caching: - Add classIdCache + exportCache Maps to parsing-processor.ts - Mirrors the O(1) memoization pattern from parse-worker.ts - Both paths now have identical caching for parent-chain walks Split utils.ts barrel into focused modules: - noise-filter.ts: BUILT_IN_NAMES + isBuiltInOrNoise (167 LOC) - language-detection.ts: getLanguageFromFilename (58 LOC) - utils.ts slimmed to re-exports + yieldToEventLoop + isVerboseIngestionEnabled - Backward compatible — existing imports from utils.ts still work * refactor: rename resolvers/ → import-resolvers/, restructure tests per-concern Directory renames (git mv — history preserved): - src/core/ingestion/resolvers/ → import-resolvers/ (10 files) - test/unit/call-routing.test.ts → call-routing/ruby.test.ts - test/unit/named-binding-extraction.test.ts → named-bindings/csharp.test.ts - test/unit/import-resolution.test.ts → import-resolution/preprocessing.test.ts All 11 import paths updated to reference new import-resolvers/ location. Test imports updated for new subdirectory depth. Note: test/integration/resolvers/ NOT renamed — those tests cover the full ingestion pipeline per-language, not just import resolution. * refactor: eliminate utils.ts barrel — all 33 consumers now import directly Migrated 65 import sites across 33 files to import from the focused source module instead of the utils.ts barrel: - ast-helpers.js: SyntaxNode, extractFunctionName, findEnclosingClassId, etc. - call-analysis.js: inferCallForm, extractReceiverName, countCallArguments, etc. - noise-filter.js: BUILT_IN_NAMES, isBuiltInOrNoise - language-detection.js: getLanguageFromFilename utils.ts reduced to 2 original functions only: - yieldToEventLoop - isVerboseIngestionEnabled Zero re-exports remain. Every import is now direct to its source module. * refactor: create utils/ folder, move all shared utilities, delete utils.ts barrel Final phase of module structure migration: - git mv ast-helpers.ts, call-analysis.ts, noise-filter.ts, language-detection.ts → utils/ subdirectory (history preserved) - Extract yieldToEventLoop → utils/event-loop.ts - Extract isVerboseIngestionEnabled → utils/verbose.ts - Delete utils.ts (zero re-exports, zero functions remain) - Update 38 import paths across source and test files The ingestion/ root is now clean — only processors, capability modules, and the pipeline orchestrator live at the top level. All shared utilities are in utils/, all language-specific helpers in helpers/, all import resolvers in import-resolvers/. * refactor: move findChild from import-resolvers/utils.ts to utils/ast-helpers.ts findChild is a generic AST helper (find first named child by type) — it belongs with the other AST traversal utilities, not in the import resolver module. 4 consumers updated to import from utils/ast-helpers.js. * refactor: split named-binding-extraction.ts into per-language files Rename named-binding-extraction.ts → named-binding-processor.ts (git mv, history preserved), keeping only walkBindingChain for re-export chain resolution. 7 per-language extractor functions moved to named-bindings/ subdirectory: - named-bindings/typescript.ts (extractTsNamedBindings — TS + JS) - named-bindings/python.ts (extractPythonNamedBindings) - named-bindings/kotlin.ts (extractKotlinNamedBindings) - named-bindings/rust.ts (extractRustNamedBindings + collectRustBindings) - named-bindings/php.ts (extractPhpNamedBindings) - named-bindings/csharp.ts (extractCsharpNamedBindings) - named-bindings/java.ts (extractJavaNamedBindings) Each provider now imports its binding extractor from the per-language file. * refactor: eliminate import-resolution.ts — distribute to natural homes Split per-language resolvers into import-resolvers/ per-language files and eliminate the import-resolution.ts catch-all module entirely: Per-language resolvers moved to import-resolvers/: - standard.ts: resolveStandard, resolveJavascriptImport, resolveTypescriptImport, resolveCImport, resolveCppImport - jvm.ts: resolveJavaImport, resolveKotlinImport - go.ts: resolveGoImport - csharp.ts: resolveCSharpImport (helper renamed to Internal) - php.ts, python.ts, ruby.ts, rust.ts: same pattern - swift.ts: new file for resolveSwiftImport Types distributed to their concern directories: - import-resolvers/types.ts: ImportResult, ImportConfigs, ResolveCtx, ImportResolverFn - named-bindings/types.ts: NamedBinding, NamedBindingExtractorFn preprocessImportPath moved to import-processor.ts (its primary consumer). import-resolution.ts deleted — zero catch-all modules remain. * refactor: tighten SPR — eliminate re-exports, dead code, type holes, and redundant patterns 12 review findings resolved across the ingestion layer: Type safety: - CallRouter callNode: any → SyntaxNode (closes type hole) - CaptureMap type alias replaces Record<string, any> - providersWithImplicitWiring filter now type-narrowed (removes ! assertions) - Ruby exportChecker: unnecessary as-cast removed, named export created Architecture: - Circular type dependency eliminated (ImportResolutionContext moved to types.ts) - LANGUAGE_QUERIES residual dispatch replaced with provider.treeSitterQueries - noRouting sentinel deleted — callRouter now properly optional on 12 providers - All 6 re-exports from import-processor/pipeline/languages eliminated Pattern cleanup: - Dead checkersByLanguage table + isNodeExported removed from export-detection - 4 duplicated config interfaces consolidated to language-config.ts - extractCsharpNamedBindings → extractCSharpNamedBindings (casing consistency) Simplification: - import-resolvers/index.ts barrel deleted (dead re-exports) - helpers/ inlined into languages/ (php.ts, swift.ts) — 1 directory removed Verified: tsc --noEmit clean, 3837 tests pass, 0 failures. * refactor: address review — remove LANGUAGE_QUERIES table, type-extractors barrel, fix Windows timeout Review comment fixes (github.com/abhigyanpatwari/GitNexus/pull/488#issuecomment-4117817648): 1. LANGUAGE_QUERIES dispatch table removed from tree-sitter-queries.ts — 5 test files migrated to getProvider(lang).treeSitterQueries — eliminates last parallel dispatch surface 2. type-extractors/index.ts barrel deleted — type-env.ts now imports TYPED_PARAMETER_TYPES from shared.js directly 3. Windows CI timeout fix: afterAll cleanup hook in test-indexed-db.ts now passes explicit 120s timeout to prevent KuzuDB C++ destructor hang from hitting vitest's default 30s testTimeout on Windows Verified: tsc --noEmit clean, 3835 tests pass, 0 failures. * refactor: eliminate chained getProvider property access — assign to variable first All getProvider(lang).property calls now follow the pattern: const provider = getProvider(language); const x = provider.property; 5 source files + 4 test files updated (~35 occurrences). This ensures consistent provider variable usage and avoids repeated lookups in hot paths. * refactor: remove last 4 re-exports from import-resolvers, fix stale CaptureMap comment - Remove `export type { TsconfigPaths }` from standard.ts - Remove `export type { GoModuleConfig }` from go.ts - Remove `export type { ComposerConfig }` from php.ts - Remove `export type { CSharpProjectConfig }` from csharp.ts All 4 types are canonically defined in language-config.ts; zero consumers imported via the resolver re-exports. - Fix stale CaptureMap JSDoc: said "Uses any" but type is SyntaxNode | undefined |
||
|
Gergo Magyar
|
76ed0fa53b |
fix: address PR #409 review findings (P0-P3) and simplify import resolution API
Bug fixes (P0):
- Narrow Go /cmd/ entry-point detection to only match /main.go
- Fix Rust scoped grouped imports (use crate::models::{User, Repo}) resolution
- Filter PHP use function/use const from class-type namedImportMap bindings
Improvements (P1):
- Add C# resolveStandard fallback when .csproj discovery fails
- Change preprocessImportPath return type to string | null with caller guards
- Add Q_SIGNALS/Q_SLOTS (standard plural Qt macros)
- Fix stale "11 supported languages" comment → 13
API simplification (P2):
- Replace buildImportResolvers() factory with const importResolvers table
- Move configs onto ResolveCtx (extends ImportResolutionContext)
- Eliminate tsconfigPaths parameter threading through 6 non-TS resolvers
- Split utils.ts (1,476 lines) into ast-helpers.ts + call-analysis.ts + utils.ts
- Consolidate findChild/findChildByType into single source of truth
- Match multi-file import bindings to files by basename for namedImportMap
Cleanup (P3):
- EMPTY_INDEX returns shared frozen empty array instead of allocating per call
- Type appendKotlinWildcard parameter as SyntaxNode instead of any
- Document call-routing validation requirement on CallRouter type
Tests:
- Add unit tests for preprocessImportPath (13 tests)
- Add integration tests: Rust scoped multi-file, PHP use function/const,
C# without .csproj, Go cmd/ helper scoring (14 tests, 4 fixtures)
All 3579 tests pass.
|
||
|
Gergo Magyar
|
2c17a4642c |
refactor: unify language dispatch with compile-time exhaustive tables
Replace the 120-line if-chain in resolveLanguageImport() and the 7-branch dispatch in extractNamedBindings() with per-language dispatch tables using `satisfies Record<SupportedLanguages, T>` for compile-time exhaustiveness. Key changes: - New import-resolution.ts: buildImportResolvers factory + namedBindingExtractors table + preprocessImportPath with control character rejection - Extracted loadImportConfigs(), createImportEdgeHelpers(), getLabelFromCaptures() to eliminate duplication across import-processor.ts and parse-worker.ts - Added isCppDuplicateClassFunction and getLabelFromCaptures shared helpers - Migrated ENTRY_POINT_PATTERNS and AST_FRAMEWORK_PATTERNS_BY_LANGUAGE to satisfies Record<SupportedLanguages, T> with compile-time exhaustiveness - Added Kotlin entry-point patterns (Android lifecycle, Ktor, MVVM) - Expanded framework detection: Go (Gin/Echo/Fiber/gRPC), Rust (Actix/Axum/ Rocket/Tokio), C++ (Qt), Swift (UIKit/SwiftUI/Vapor), Ruby (Rails/Sinatra) - Fixed Go cmd/ entry-point detection operator precedence bug - Added EMPTY_INDEX frozen sentinel for type-safe memory cleanup - Unified ImportResolutionContext (suffixIndex->index, removed unused dispose()) - Extracted NamedBinding interface (replaced 16 inline occurrences) - Replaced any with SyntaxNode on all tree-sitter node parameters - Added contributor checklist to SupportedLanguages enum 14 files changed, +318/-550 (net -232 lines). All 3550+ tests pass. |
||
|
ivkond
|
fbff6d08c0
|
feat(ingestion): respect .gitignore and .gitnexusignore during file discovery (#231)
* feat(ingestion): respect .gitignore and .gitnexusignore during file discovery Add support for excluding files from indexing based on .gitignore and .gitnexusignore patterns. Previously, GitNexus used only a hardcoded ignore list, causing significant index pollution in repositories with git-ignored directories containing code (e.g., Docker-mounted volumes). Changes: - Add `ignore` package for gitignore-spec pattern matching - Add `loadIgnoreRules()` to parse .gitignore + .gitnexusignore - Add `createIgnoreFilter()` returning glob-compatible IgnoreLike object - Integrate filter into glob's `ignore` option for directory-level pruning - Remove post-glob `.filter()` call (now handled during traversal) The hardcoded DEFAULT_IGNORE_LIST remains as fallback for non-git repos. Closes #228 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ingestion): address review feedback on ignore filtering - Distinguish ENOENT vs EACCES in loadIgnoreRules (warn on permission errors) - Add GITNEXUS_NO_GITIGNORE env var to bypass .gitignore parsing - Fix bare-name pattern matching in childrenIgnored (check both with/without trailing slash) - Rename isIgnoredDirectory to isHardcodedIgnoredDirectory for clarity - Add clarifying comments for design decisions (D2 negation, D3 dot:false redundancy) - Add tests for bare-name patterns, file-glob patterns, EACCES handling, env var Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ingestion): address second round of review feedback - G1: Document GITNEXUS_NO_GITIGNORE in `analyze --help` and log when active - G2: Add comment clarifying path-scurry POSIX normalization contract - G3: Add IgnoreOptions interface — env var now falls back, callers can pass `noGitignore` explicitly for testability and future CLI flag - G4: Add integration test verifying walkRepositoryPaths respects the env var Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(ingestion): gracefully skip files with unavailable tree-sitter grammars Port unsupported language resilience from PR #301 by @jecanore. - Make Kotlin import optional (like Swift) in parser-loader and parse-worker - Add worker-local isLanguageAvailable() with filePath param for tsx distinction - Track and log skipped files per language in both sequential and worker paths - Add skippedLanguages to ParseWorkerResult for worker→main aggregation - Add isLanguageAvailable unit tests Refs: #301, #155, #228 Co-Authored-By: jecanore <juan@housingbase.io> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(e2e): add ignore + language-skip end-to-end test with fixture repo Add a fixture repo (test/fixtures/ignore-and-skip-repo/) with .gitignore, .gitnexusignore, TypeScript source files, and a Swift file to exercise all three features end-to-end: - File discovery: verifies .gitignore excludes data/ and *.log, .gitnexusignore excludes vendor/, source files are discovered - Parsing: verifies TypeScript files produce Function nodes and DEFINES relationships, Swift files are skipped gracefully when grammar is unavailable Add the test to the standalone group in ci-integration.yml and coverage job. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): move ignore-and-skip-e2e test to e2e group per review feedback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(test): use temp directory instead of fixture for e2e ignore test The fixture's .gitignore prevented data/seed.json and debug.log from being committed — these files would be missing after checkout in CI. Switch to creating the entire test structure in a temp directory via beforeAll (matching filesystem-walker.test.ts pattern). This ensures all files exist regardless of git ignore rules. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(test): correct graph API usage in e2e ignore test Use graph.nodes property getter instead of graph.getNodes(), and check Function node filePath instead of non-existent File nodes (File nodes are created by processStructure, not processParsing). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add workflows permission to ci-integration.yml Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: change workflows permission to write per review Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: move workflows permission from ci-integration.yml to ci.yml caller Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): fix Claude workflows for fork PRs, remove misplaced workflows perm Three issues prevented Claude from running on fork PRs: 1. claude-code-review.yml lacked workflows:write — push failed when fork PRs modify .github/workflows/ files 2. claude.yml had no fork PR support — checked out main and couldn't fetch the fork's branch from origin 3. Cleanup step unconditionally deleted branches even when push failed, breaking the concurrent claude.yml workflow Also removes workflows:write from ci.yml's integration job — CI tests don't need that permission. The permission belongs on the claude workflows that push fork branches. Changes: - Add workflows:write to both claude workflow permissions blocks - Add fork PR detection + branch push/cleanup to claude.yml - Add step id to push-fork; cleanup only runs if push succeeded - Pass branch names via env vars to prevent shell injection (security) - Add concurrency groups to prevent race conditions between workflows - Remove misplaced workflows:write from ci.yml integration job * fix(ci): use GitHub API for fork branch refs instead of git push GITHUB_TOKEN cannot have 'workflows' permission — it's only valid for PATs and GitHub Apps. This means git push fails whenever a fork PR modifies .github/workflows/ files. Replace git push with the GitHub REST API (POST/PATCH /git/refs) to create temporary branch refs. The API creates a pointer to the already-existing PR head commit without triggering the workflow file push protection. Similarly, cleanup uses DELETE /git/refs instead of git push --delete. Also removes the invalid 'workflows: write' from permissions blocks. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: jecanore <juan@housingbase.io> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Candido Sales Gomes
|
0999595444
|
feat(ruby): Add Ruby language support for CLI and web (#111) | ||
|
Gary Magyar
|
e046bf734d |
chore: merge upstream main into feat/kotlin-language-support
Resolve conflicts between Kotlin and Swift language support additions. Both languages are now fully supported side by side. |
||
|
jandyx
|
e803e7e9d6 |
feat(swift): add comprehensive Swift/iOS language support
- Enable Swift in supported languages enum - Add tree-sitter-swift parser loading (v0.6.0) - Add .swift file extension mapping - Implement full tree-sitter queries (class, struct, enum, protocol, extension, actor, function, property, init, imports, calls, heritage) - Add Swift export detection (public/open modifiers) - Add Swift/iOS built-in name filtering (~70 entries: stdlib, UIKit, Foundation, GCD, Combine, collection methods) - Add SPM module import resolution (Sources/<Target>/ scanning) - Add iOS/SwiftUI framework path detection with entry point multipliers - Add Swift entry point scoring patterns (UIKit lifecycle, SwiftUI body, Coordinator, AppDelegate/SceneDelegate) - Add Swift test file detection patterns |
||
|
Gary Magyar
|
c37b63ae8b |
feat: add Kotlin language support
Add end-to-end Kotlin parsing, symbol extraction, and visibility detection. Extract findSiblingChild helper into utils.ts for clean AST traversal of Kotlin's modifiers/visibility_modifier sibling pattern. Fix pre-existing duplicate ftsLoaded declaration in kuzu-adapter.ts. Files changed: - supported-languages.ts: add Kotlin enum member - parser-loader.ts, parse-worker.ts: register tree-sitter-kotlin - tree-sitter-queries.ts: add Kotlin queries for classes, interfaces, objects, functions, properties, imports, calls, and heritage - parsing-processor.ts, parse-worker.ts: add Kotlin visibility detection - call-processor.ts, parse-worker.ts: add Kotlin builtins and node types - utils.ts: add .kt/.kts extension mapping and findSiblingChild helper - package.json: add tree-sitter-kotlin dependency |
||
|
Güneş Bizim
|
7883bf2cf0 |
feat(php): add PHP to SupportedLanguages enum and file extension detection
Uncomment PHP = 'php' in both CLI and web SupportedLanguages enums. Add .php, .phtml, .php3/.4/.5/.8 extension detection to getLanguageFromFilename() in both packages. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
abhigyanpatwari
|
46330fa301 | go c c# java python ts js rust now supported | ||
|
abhigyanpatwari
|
e153d30c8f | MCP bridge added |