mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-02 02:11:29 +00:00
195 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Gergo Magyar
|
42c39b5c82 |
fix(storage): never share a graph from a checkout that hides files (#3374)
Trigger: a sparse checkout, a skip-worktree/assume-unchanged entry, or an uninitialized submodule leaves `git status` clean, and the run's indexCoverage.dirtyPaths drops paths with no file hash, so publishSharedGraph published a graph missing those files as the commit graph. seedSharedSlot then copied the seed's lastCommit into the new pointer slot, so the next analyze of a sibling hit the up-to-date fast path without hashing. Fix: add isWorkingTreePristine (storage/git.ts): the unfiltered listWorkingTreeDirtyPaths must be empty (null fails closed) and every gitlink in the index must have a checked-out `.git`. publishSharedGraph requires it instead of isWorkingTreeDirty. seedSharedSlot keeps the seed's lastCommit only when the seed is at HEAD and the checkout is pristine, so a clean sibling still fast-paths onto the shared graph; any other seeded pointer gets an empty lastCommit, like seedFromLocalIndex, and its next run hash-diffs, then re-points at the commit graph on publish. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
416af2a8f6 |
fix(storage): never follow a symlinked checkout .gitnexus (#3374)
A checkout whose `.gitnexus` is a symlink (e.g. `.gitnexus -> ..`) made the shared-store pointer helpers operate on whatever it pointed at: findLegacyLocalIndex listed the target as a "legacy index", so `clean --local-index --force` recursively deleted the checkout's siblings; writeSharedStorePointer wrote store.json/.gitignore there; and removeSharedStorePointer deleted store.json there and could rm -r the target directory. All four now go through probePointerDir, which lstats `.gitnexus` and accepts it only when it is a real directory whose realpath is realpath(checkout)/.gitnexus (mirroring stale-branch-slots.ts). Anything else is left untouched: no legacy index is reported or removed, and no pointer is written or removed. removeLegacyLocalIndex re-probes after sizing, just before its delete loop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
6ad8d7d389 |
feat(storage): share the index store between clones automatically (#3352)
Clones of one repository now share like linked worktrees. A clone whose normalized origin URL matches another registered, still-present clone joins that clone's store, or founds one (keyed on its own path) that the sibling joins on its next analyze. A lone clone keeps its own .gitnexus. Graphs stay keyed by commit and feature key, so clones only ever share a graph built from the same commit with the same settings. `analyze --no-share` now records a lasting opt-out (`shareOptOut` on the registry entry, preserved across re-registration); `--share-with` clears it. The analyze worker reports the storage it wrote over IPC so the server settles a clone's first shared slot. A query-time base-plus-overlay graph stays out: LadybugDB reads one database per query. Instead, private graph copies record whether the filesystem cloned them copy-on-write (sharing unchanged pages on disk) or made a full copy, and `status` reports it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
fcbdd12f22 |
fix(storage): address eighth review round on the shared store (#3374)
- removing a checkout's storage unregisters it and removes its store pointer before deleting the slot directory, which the file lock backend uses for its lock file; withCheckoutSlotLock becomes removeCheckoutStorage, used by remove, clean, and DELETE /api/repo, and analyze --no-share follows the same order - the clean --gc preview skips a slot whose index lock is held, matching what --force would drop Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
e7975a6fb7 |
fix(storage): address seventh review round on the shared store (#3374)
- clean --gc aborts when the registry cannot be read instead of treating every member as orphaned; with no registry file it collects only members whose checkout directory is gone - seeding from a repository-local index re-reads its metadata under the index lock, so the copied graph and the saved metadata match - clean --gc skips a store another collector removed meanwhile, and reports "no shared stores" when stores/ holds only stray files Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
8934d0b55c |
fix(storage): address sixth review round on the shared store (#3374)
- clean, remove, and analyze --no-share remove the checkout's store pointer while still holding the slot's index lock, so an analyze that takes the lock next cannot have its new pointer deleted - clean --gc does not follow a symlink under stores/ (lstat) - removing a store pointer keeps the checkout's .gitnexus directory when it cannot be listed, instead of treating it as empty Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
2cdee6d4a5 |
fix(storage): address fifth review round on the shared store (#3374)
- analyze --no-share leaves the shared store even when routed to a branch sub-index (gate on sharedStore, not placement.branch) - clean --gc aborts a store whose member listing is unreadable instead of treating it as empty, and skips non-directory entries under stores/ - reusing an existing commit graph no longer fails a finished analyze when the redundant private graph cannot be wiped - a store pointer holding JSON null, a number, a string, or an array is treated as invalid - clean, remove, and DELETE /api/repo hold the checkout slot's index lock while deleting the slot - the server analyze launcher waits on the checkout slot the worker actually writes - sync the Factory hook copy; isolate hook tests from storage overrides; use a junction for the Windows worktree alias; the relative GITNEXUS_HOME test now sets a relative home Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
e2a9467171 |
fix(storage): address third review round on the shared store (#3374)
- clean --gc never collects a slot whose index lock is held: an analyze holds it until it registers the checkout, so a seeded but not yet registered slot is busy, not orphaned. - Reclaim compares absolute paths, so a relative GITNEXUS_HOME does not make live commit graphs look unreferenced. - graphPath is followed only into a published <commit>-<featureKey> dir, never .publish-* staging (TS and all three hook copies). - clean --gc fails on an unreadable stores root instead of reporting nothing to collect. - --no-share help states it is for opted-in clones. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
d746675c28 |
fix(storage): address second review round on the shared store (#3374)
- Reject --no-share in a linked worktree before taking any lock or indexing anything. - clean --all and remove also delete each shared checkout's pointer. - Delete an empty store only while also holding its cache lock, and re-check emptiness under it. - A store pointer is trusted only when the slot's own metadata names this checkout; the editable pointer file just says where to look. - Device names with an extension are prefixed on Windows only, so POSIX slot names stay stable. - Test fixtures use the gitnexus-test- prefix the stale-sidecar sweep recognizes; help text and the private-graph label are accurate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
d0894ae4b7 |
fix(storage): address review feedback on the shared store (#3374)
- Hold the slot index lock for the whole server embedding job, not just the graph copy. - --no-share re-registers and deletes the old slot under that slot's index lock, so a running shared analyze cannot re-register it. - clean --gc previews without --force, like every other destructive arm. - status reports a pinned branch index as private. - Slot names prefix Windows device names that carry an extension. - A slot or store named ..<name> is a legal direct child. - Shared-store suites run in the serialized lbug-db vitest project and clear an inherited GITNEXUS_SHARED_STORE. - Doc and test accuracy fixes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
28902962dd |
fix(storage): keep shared-store file reads inside checked paths (#3374)
Address CodeQL js/path-injection and js/file-system-race on shared-store.ts: every filesystem read rebuilds its path under a fixed parent with an inline path.relative barrier, the .git probe is one read (EISDIR marks a directory) instead of stat-then-read, and the graph pointer cache stats and reads through one file descriptor. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
28a770b68d |
fix(storage): address review findings in the shared store (#3352)
- Never publish a graph whose build saw dirty files, and never trust a local-index seed on the up-to-date path; it may hold reverted edits. - Record slot pointers and reclaim under the publish lock, and reclaim right after each publish, so a commit graph is never deleted between publish and pointer save and superseded graphs don't pile up. - clean --gc decides membership from the registry, so opted-in clones and a main checkout without worktrees are not dropped. - Leaving a store re-registers first, so an up-to-date run cannot leave the registry pointing at a deleted slot. - Drop pinned branch summaries when an entry moves into a store slot. - Keep run.cjs and the AGENTS.md runner path inside the checkout. - MCP handles follow the slot's current graph, and branch scoping reads the slot's own metadata. - Cache resolveGraphPath by metadata file identity; look up opted-in entries with canonical registry paths. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
f9e988ebce |
refactor(storage): reuse helpers across the shared-store code (#3352)
One graph-clone helper replaces two copy-then-rename blocks; clean and status reuse formatSlotSize; leaving a store reuses removeSharedStorePointer; withStoreLock is imported from its own module instead of a re-export. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
c86cca40b6 |
feat(storage): adopt existing worktree indexes and report shared-store state (#3352)
A shared checkout gets <repo>/.gitnexus/store.json pointing at its store slot; resolution follows it only when it names that checkout's own slot. An existing local index seeds the slot and is left in place. status (text and --json) and doctor report the store, whether the graph is shared or private, and any leftover local index, which clean --local-index removes while keeping the pointer. With GITNEXUS_SHARED_STORE=off a previously shared checkout indexes into its own .gitnexus again and never writes a commit graph. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
8cdb05c22e |
fix(storage): count a shared checkout's commit graph as its code index (#3352)
A clean shared checkout reads a commit graph and owns no graph file, so the code-index presence check made status report it unindexed and registry validation skip it. The check now follows the slot's validated graphPath. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
8c74a86fc3 |
feat(clean): remove only what no shared-store member references (#3352)
Deleting a shared checkout slot (clean, clean --all, remove, and the server delete route) recounts references under the store's publish lock and deletes commit graphs no member points at, then the store itself once empty. A graph that cannot be deleted (open on Windows) is reported and kept for the next pass. clean --gc also drops member slots whose worktree is gone or no longer resolves to the store. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
f7681b1a55 |
feat(analyze): seed a worktree's graph from the nearest index (#3352)
A new shared slot is seeded from the store's commit graph nearest to HEAD, else from this checkout's or the main checkout's repository-local index (copied under that index's lock, source left in place). The run that follows is up to date or incremental instead of a full build. If a pointed-at shared graph has been removed, analyze falls back to a full build instead of an incremental update over a missing baseline. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
3ca01b4f5a |
feat(analyze): share one immutable commit graph across clean worktrees (#3352)
A linked worktree writes its own slot in the shared store. A new slot is seeded with a pointer to the nearest commit graph, so a clean checkout at an indexed commit takes the up-to-date path and writes no graph. A checkout with local changes gets a copy-on-write private graph before its first write. After a successful run, a clean checkout at HEAD publishes its graph into commits/ under a store lock, or drops it when that commit graph already exists. Commit graphs are never written after publish. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
91c423d107 |
feat(storage): resolve a shared checkout's graph and existing store slot (#3352)
getStoragePaths reads a flat slot's recorded graphPath only for checkout slots under the stores directory; other paths keep <storagePath>/lbug with no I/O. A recorded path outside the store's commit graphs is ignored. resolveStoragePath falls back to an existing store slot for an unregistered checkout, so reads never move to an empty slot. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Gergo Magyar
|
eaf6d24b92 |
feat(storage): resolve the shared sibling-store identity and layout (#3352)
Linked worktrees of one repository resolve to one store under GITNEXUS_HOME/stores/<key>, keyed by the canonical git common dir. The resolver reads the .git entry directly, so hot paths spawn no git. Slot naming moves to a leaf module so storage-resolver and shared-store do not import each other. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Felipe
|
c2ca132620
|
fix: web citation/code panel bugs and serve analyze/route hardening (#3348)
* chore: ignore local Vercel link artifacts
Keep .vercel and env files out of the repo after a local preview link.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): repair citation chips, code panel line math, stale agent state
Audit findings in the web client, each verified against the source:
- RightPanel: citation chips ([[path:10-20]], [[Class:Foo]]) were wired to a
stub resolver that always returned null, so clicking any citation did
nothing. Expose resolveFilePath from useAppState and use it.
- CodeReferencesPanel: graph startLine/endLine are 1-based but were treated
as 0-based, so the highlighted range and scroll target were off by one
line; AI citation cards always rendered "code not available" because the
snippet loader was a stub. Fetch per-citation snippets via /api/file.
- useAppState: sendChatMessage read llmSettings.activeProvider outside its
deps (stale provider capabilities after switching provider);
initializeAgent trapped projectName at '' for callers without an override
(system prompt labelled the codebase "project"); the embeddings 409 dedup
matched a message the server never sends for same-repo jobs.
- tools.ts impact: for path targets every symbol defined in the file shares
the filePath, so the disambiguation always picked the first row and could
analyze an arbitrary symbol while reporting a file impact. Prefer the File
node.
- useSigma: the layout timeout called stop() but never kill(), leaking one
ForceAtlas2 Web Worker plus four graph listeners per completed layout.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): release repo lock on cancel, IPC-first worker cancel, route hardening
Audit findings in gitnexus/src, each verified against the source:
- analyze-launch: cancelJob marks the job failed before the worker exits,
so the exit handler's terminal early-return skipped releaseLockOnce and
the repo stayed locked ("Another job is already active") until restart.
Release on terminal exit and when forkWorker bails on a terminal job.
- analyze-job: cancellation now sends { type: 'cancel' } over IPC first and
signals only after a 15s grace. On Windows child.kill('SIGTERM') is a
forceful termination, so leading with it could kill the worker inside a
LadybugDB write. Mirrors core/auto-sync/analysis-worker-launch.
- api resolveRepo: a job that FAILED during the hold-queue wait fell through
to the { __timedOut } sentinel ("taking longer than expected") instead of
404; only /api/repo checked the sentinel, so graph/query/search/file/grep/
embed/delete crashed on entry.storagePath with a 500 after a 5 minute hang.
Return null on failed jobs and check the sentinel in every consumer.
- api processes/process/clusters/cluster: resolve ?repo= through the HTTP
resolver (documented policy on resolveRegisteredRepoEntry) and pass the
registered absolute path to the backend; add the standard rate limiter.
The raw param previously reached the MCP resolver, which runs a
cwd-relative realpathSync probe + registry refresh on a bare-name miss
and accepts unambiguous partial names.
- api body handling: Express 5 leaves req.body undefined without a JSON
content type, turning "Missing X" 400s into TypeError 500s; body-parser
4xx errors (malformed JSON, over-limit) were also reported as 500.
- /api/file: the lexical path.relative check cannot see symlinks; re-check
containment on realpath so a cloned repo containing evil -> /etc/passwd
cannot read outside the root.
- repo-manager unregisterRepo: used the lenient reader, so a transient read
error (EBUSY/EPERM racing another process's atomic rename) turned into
writing [] and deregistering every repo. Use the strict-if-present reader.
- clean --branch: compared registry paths with raw path.resolve instead of
the canonical registryPathEquals used everywhere else (macOS /private/var,
Windows short names / drive-letter case) and reported indexed branches as
not indexed.
Tests: cancelJob IPC-before-signal contract; /api/file symlink escape (403)
and in-repo symlink (200), skipped where the host cannot create symlinks.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore: keep example env files visible and ignore local Cursor config
.env* also hid gitnexus/.env.example and eval/.env.example. .vercel was already ignored. The web app's .cursor/ stays local, including its MCP file.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add realtime execution ops dashboard for Vercel monitoring.
Expose /api/ops snapshots over the local serve process and a ?view=ops SPA panel so analyze/embed jobs can be watched live from the hosted web UI.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: address gitnexus-check review on cite/embed/resolve paths
Pass req into resolveRepo for process/cluster routes, tighten same-repo embed 409 handling, guard empty citation paths, fix snippet retry races, and drop the lone-File ambiguity fallback.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ops): harden CORS, redact paths, and bound ops streams
Restrict Vercel CORS to exact production hosts, omit raw repo paths/URLs from the unauthenticated ops feed, rate-limit and cap SSE connections, and fix dashboard SSE/poll edge cases.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): stabilize citation fetches and file impact matching
Retry cancelled snippet loads without duplicate in-flight reads, cap range-less citation downloads, and make impact file matching unique-suffix-aware with a synthetic File target when LIMIT drops the File node.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web/ops): close gitnexus-check review threads on SSE and redaction
Cap citation reads, reconnect ops on applied server URL, skip SSE onError after abort, and strip URL query/fragment from public repoName.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ops): abort SSE on poll fallback and harden repoName parsing
Prevent dual SSE+poll after a failed safety snapshot, skip overlapping poll ticks, ignore aborted streamSSE onError, and basename Windows drive-letter URLs so ops never leaks path segments.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ops/web): close gitnexus-check threads on SSE budget and credential leak
Keep finite SSE retries across short 200s, strip backend URL userinfo before ?server=, and redact progress messages on the public ops feed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3348)
Restore 0-based GraphNode line math, redact public job poll/error fields, fix omit-?repo= 400, hold the analyze lock across cancel-during-settle, and restore the Vercel shared compile.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cli): contain leftover-slot reclaim to the slot and keep --stale status honest
Preview and force now share one branches/ containment rule, nested junctions cannot walk a sibling index, and a mid-loop git failure no longer claims leftovers were not deleted after a successful rm.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(cli): share leftover-slot helpers without changing reclaim behavior
Pull the rolling I/O pool and owned-cwd storage lookup into one place so clean --stale/--branch and leftover listing stop restating the same ownership and concurrency paths.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(autofix): apply prettier + eslint fixes via /autofix command
* Address PR review feedback (#3348)
Keep the omitted-repo snapshot instead of re-listing, stop citation and ops races, redact full public repo URLs, and restore fake timers in teardown.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address remaining PR review feedback (#3348)
Store graph node citation lines as 0-based offsets and correct the default-port origin comment.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address remaining PR review feedback (#3348)
Redact public SSE progress text, stop citation append retries from
cancelling in-flight reads, and keep the ops dashboard from showing a
stale snapshot or clearing a failed Connect.
Note: pre-existing failure in incremental-index-extension-dml-gate and other lbug/env unit tests not addressed by this PR.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address remaining PR review feedback (#3348)
Prune citation snippets when AI refs are cleared, and poll /api/ops at 2s so the fallback stays under the 60/min limit.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): apply prettier class order for format check (#3348)
CI quality/format runs root-only npm ci, so prettier-plugin-tailwindcss
sorts scrollbar-thin without the web Tailwind catalog.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3348)
- Require a unique suffix match for graph-backed citation paths so
ambiguous names like index.ts no longer open the first graph file.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3348)
- Require a path-component boundary so unique citation suffixes cannot match filename substrings like myindex.ts
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): redact filesystem paths in ops text
Unauthenticated /api/ops and poll replay worker errors. URLs were
scrubbed but home-directory and Windows paths still leaked.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): omit repoPath from public SSE frames
/api/ops lists job ids, so the unauthenticated progress stream
must not replay the analyzed filesystem path.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): treat embed lock 409 as a busy error
Analyze and embed share the same lock string. Mapping that 409 to
embedding hid an in-flight analyze as a successful embed start.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): bound impact File path suffix matches
Unbounded endsWith let lib/foo.ts select src/mylib/foo.ts. Require
an exact path or a unique /suffix, matching resolveUniqueIndexedPath.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): keep canceled analyze slot until exit
Marking failed before the worker exited let a second POST start
cloneOrPull against a LadybugDB file still being written.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): skip Windows SIGTERM on job dispose
child.kill('SIGTERM') is TerminateProcess there. Ask over IPC first
and leave the 15s grace timer to SIGKILL.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): let process routes skip the analyze hold
GET /api/processes and /api/clusters always waited up to 300s.
?awaitAnalysis=false fails fast; default still waits like /api/repo.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(web): cover public ops and analyze SSE user flows
Lock the unauthenticated dashboard and analyze complete/fail/cancel paths so a leaked repoPath, token, or home path cannot ship unnoticed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3348)
Keep caller cancel reasons over the worker's generic IPC, skip publish while cancel is pending, and redact scp-style remotes on the public ops feed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address remaining PR review feedback (#3348)
Release the analyze slot when a worker fails to spawn, and omit branch refs from the unauthenticated ops feed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): do not reuse an analyze job that is pending cancel
A dying same-repo job still occupies the single slot; 202-reuse would
attach a new client to a cancel in flight.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): hold the analyze lock until the worker exits after cancel
Cancel error IPC used to drop the repo lock while the child was still
checkpointing. Abort settle immediately on pending cancel so the slot
is not held for a 60s disk poll.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): redact known repo paths with spaces in public ops text
Known repoPath/repoUrl literals are replaced first so a clone dir with
spaces cannot leak past the whitespace-bounded path regex.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(server): return the live job on analyze and embed DELETE
Hard-coding failed made clients retry immediately and 409 while the
child still occupied the slot. resolveRepo now returns not-found as
soon as that job fails instead of waiting out the hold timeout.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(web): drive public analyze and ops through a live gitnexus serve
Spawn the real backend and observe requests instead of intercepting
them, so slot occupancy, redaction, and reconnect stay honest.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(web): reuse code-panel helpers and drop dead UI aliases
Citation fetches already had selectedNodeFileRange and snippetRepoKey;
the impact File suffix filter already handled exact paths.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(autofix): apply prettier + eslint fixes via /autofix command
* Address PR review feedback (#3348)
- Skip createJob reuse after cancel IPC is consumed while the child remains
- Hold the repo lock until exit when complete IPC races a pending cancel
- Scrub full remote URLs before known repoUrl prefixes in public ops text
- Reject unique impact File suffix matches from a truncated LIMIT 10 page
- Make live e2e helpers bound probes, clean up failed startups, and wait out the cancel slot
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): open live e2e pages on the Vite host CI actually bound
Absolute 127.0.0.1:5173 navigation refused on Actions because wait-on
and Vite use localhost (often ::1). Honor FRONTEND_URL when set, else
pick the first of localhost / 127.0.0.1 that answers.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3348)
Hold the analyze lock until worker exit when cancel aborts settle, and assert GitLab failure chrome does not leak host or path.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(web): keep live analyze e2e under the analyze rate limit
POST /api/analyze allows 10 requests per minute per IP. The slot-free
helper re-POSTed every 400ms while a cancelled worker was exiting, spent
that budget, and the lock test's hold request got 429 instead of 202.
- postAnalyze waits out a 429 using the RateLimit reset and retries
- slot polling backs off to 2s and leaves a small POST budget for callers
- the slot probe is a clone that fails before any worker fork, so the
probe itself no longer holds the slot after reporting failed
- the lock test holds the slot with a real local analyze
- token and GitLab tests wait for a free slot before posting from the UI
- request fetches carry a timeout; teardown signals the serve process group
- an empty FRONTEND_URL falls back to the default base URL
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Address PR review feedback (#3348)
- ops view: a `?server=` link no longer auto-connects to another origin
while a deploy token is held; it prefills and waits for Connect
- analyze completion: a local-path run reconnects by the path this client
submitted, so duplicate basenames stay collision-safe without repoPath
on the public SSE frame
- stale slot cleanup: revalidate each nested directory (lstat + realpath)
right before readdir, so a mid-cleanup junction swap aborts instead of
walking an outside tree; list phases run sequentially so the slot-I/O
cap is global
- e2e: 429 backoff honours the caller deadline; the unreachable-backend
ops test navigates through the resolved frontend URL
- drop the unused `repo` member from the clean integration helper
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(server): reconnect after analyze by an opaque repo id
Public job views and the SSE terminal frame no longer carry repoPath, and
repoName is not unique, so a post-analyze reconnect by name could load a
same-named sibling. The server now issues `repoId`: an HMAC of the
canonical registry path under a per-process random key. It is set on
complete jobs (ops view, analyze poll, SSE terminal frame) and matches
the new `id` on `GET /api/repos` entries. The web client resolves it to
the exact entry path on completion; unknown ids fall back to the name.
This covers URL clones and folder uploads, and replaces the local-path
only fallback.
With reconnect off the label, public `repoName` for a branch-pinned URL
clone is the repository name, not the `<repo>__<branch slug>` registry
name, so the requested branch stays off the unauthenticated feed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Address PR review feedback (#3348)
- stale slot cleanup: a descendant that vanishes before its unlink is
treated as removed instead of aborting the reclaim
- e2e teardown: escalate to SIGKILL on the process group when the live
backend ignores SIGTERM for 5s
- ops view: drop the dead initial value CodeQL flagged
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Address PR review feedback (#3348)
- public redaction: a known repoPath now also consumes its descendant
tail, so `<repoPath>/src/secret.ts` becomes `[path]` instead of
`[path]/src/secret.ts`; a same-prefix sibling is left to the path scrub
- e2e: the cancel test waits for the analyze slot the previous failed
local-path job still holds; `fetchOps` carries the request timeout
- docs: SSE terminal payload and RepoAnalyzer `onComplete` describe
`repoId` resolution
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Address PR review feedback (#3348)
- RepoAnalyzer: drop a completion that resolves after unmount, so a slow
/api/repos lookup cannot switch repos after the sheet was dismissed
- e2e: select the local-path input by test id (the placeholder differs on
Windows); the slot probe's job wait honours the caller's deadline
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(server): assert the public SSE terminal frame in analyze-api
The #2790 terminality tests still expected `repoPath` on the terminal
frame. This PR replaced it with the opaque `repoId`, so assert that shape
and that the analyzed path never appears in the stream.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(web): wait for the analyze slot between duplicate-repo setup runs
The server now keeps the single analyze slot until the worker exits, even
after its job reports complete. repo-path-identity posted the second
duplicate's analyze immediately and got 409 in CI. Use the shared
slot-aware POST, which also waits out a 429.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
||
|
Adam B.
|
7376347058
|
feat(ingestion): add tRPC procedure detection and MCP chain/route surfacing (#3339)
* feat(ingestion): tRPC pattern detection — procedures, curried calls, route extraction
Three changes to properly index tRPC router files:
1. HOC-in-pair patterns: detect procedures like
'create: procedure.mutation(async ({ input }) => {...})'
as named Function nodes (pair > call_expression > arguments > arrow_function)
2. Curried call detection: capture 'workflow(db)(input)' chained calls
where call_expression.function is itself a call_expression
3. tRPC route extraction: new route-extractors/trpc.ts detects
.query()/.mutation()/.subscription() procedures, maps to /trpc/* routes
with prefix inference from router variable names
4. Function/Const dedup: structural check skips Const nodes when
variable_declarator value is arrow_function/function_expression
5. Process-route linking: match by (filePath, methodName) instead of
filePath only, preventing shared flows across procedures in same router
(cherry picked from commit eced15e60a39a181a48bff8c5bbe278d26aa53b8)
* fix(route-extractors): line-by-line scanner for chained tRPC procedures
Original regex only matched direct patterns (list: proc.query()) but not
chained patterns (create: proc.input(z.object({...})).mutation()). Only
41/256 routes were detected on Jurialis.
Rewrote extractTrpcRoutes() as a line-by-line scanner that:
- Detects procedure keys starting with *Procedure builders
- Tracks currentProcedure forward until terminal .query()/.mutation()
- Handles .input() chaining naturally
- Deduplicates via seen Set on procedurePath
Result: 245 routes from 30 routers (was 41), 256 total after re-index.
(cherry picked from commit 49edf953e93e6b67b77932e866fcbfb8d089b1f3)
* feat(mcp): expose tRPC chains via context/query tools
Eight fixes to make the tRPC route->procedure->workflow->sub-workflow chain
visible through the standard MCP tools (context, query) that AI agents use,
instead of requiring raw Cypher queries.
- A context: order incoming/outgoing CALLS test-last, raise LIMIT 30->100
- B query: batched ENTRY_POINT_OF lookup, surface route URL on processes
- C query: mark process_symbols entry point with is_entry_point=true
- D entry-point-scoring: skip UTILITY_PATTERNS (get*/set*) penalty for
symbols in tRPC router files so framework boost (3.0x) is preserved
- E fts-schema: index Route nodes so /trpc/* URLs are keyword-searchable
- F tools: bump max_symbols default 10->25 to fit procedure->workflow chain
- G context: optional chain_depth (0-3) param walks CALLS edges in both
directions and returns layered chain field
- H LadybugDB bug workaround: WHERE r.type IN [...] silently drops edges
on relationship properties; replace with OR chains (7 occurrences in
local-backend.ts, pdg-impact.ts, graph-queries.ts)
Validated on Jurialis: setProviderCap procedure now appears as caller of
setProviderCapWorkflow, /trpc/cabinet.setProviderCap Route is searchable,
chain_depth=3 returns the layered call graph.
(cherry picked from commit c24df0adce91e80f191a5c5d2e8b14e9da677366)
* feat(mcp): surface is_entry_point + routes in context, raise query limit
context() is the mandatory pre-edit tool (AGENTS.md). Until now an agent
had to issue a separate query() call just to learn whether its symbol is
a process entry point or which HTTP route it handles. These two fields
make context() self-sufficient for the bmad-dev flow.
- context: query STEP_IN_PROCESS now returns p.entryPointId; new
ENTRY_POINT_OF lookup attributes routes only from processes where the
symbol is the entry point (middle steps do not own the route) plus any
direct Route->symbol handler edge (tRPC procedures outside any Process)
- context: new top-level is_entry_point (true only) and routes[] fields
({url, method?}), emitted only when non-empty to keep the diff additive
- query: raise default limit 5->10 so dense domains (tRPC action router
with 20 chains, data-export with 9 flows) surface more of their flow
set without an explicit param
(cherry picked from commit 004f5b0ba985fcef3dfac5e67a2f2f7262184215)
* docs(mcp): document chain_depth, routes, and entry-point flag; neutralize examples in comments
* fix(mcp): address code-review findings on tRPC entry points, queries, and dispatch
- entry-point scoring: optional-chain framework detection and normalize
path separators before router-pattern matching (P1 crash on .js routers)
- tRPC extractor: emit controllerName as null (callers resolve via
lookupClassByName; a router object stringified as name corrupted lookups)
- route dedup: key seenRoutes by method:url so GET/POST pairs survive
- legacy TS queries: drop curried-call patterns (arity-corrupting for
overload resolution) and require non-array callee on pair member calls
- registry-primary TS query: mirror the non-array-callee predicate on the
new pair member-expression patterns (fixes query compile error)
- group tool port: forward chain_depth into per-tool context args
* fix(mcp): nested tRPC router paths, controller-less route binding, query chain_depth
- trpc extractor: brace-depth nesting stack composes sibling router paths (bare router() import style included); merge-prefix dot normalization; strict publicProcedure allowlist gate; drop phantom router metadata
- call-processor: bind controller-less tRPC routes to same-file handlers via exact single-match symbol lookup; ambiguous or missing handlers skipped
- query/group query: optional chain_depth (0-3) enriches ranked processes with their context chain; fix _computeContextChain layer docstring
- tree-sitter TS/JS scope queries capture string-key function declarations; tRPC pattern scoring narrowed to server router files
- parse-cache schema bump to v103 for extractor and route-binding changes
- add trpc route extractor regression tests (9 cases incl. sibling nesting and merge prefix)
* Address PR review feedback (#3339)
Close remaining review threads: compact tRPC keys, comment-safe terminals,
quoted-key identifier HOC captures, group-query default alignment, and
LadybugDB label scalars on context chains.
* chore(autofix): apply prettier + eslint fixes via /autofix command
* test: pin PARSE_CACHE schema bump to 103 (PR #3339 review fixes)
* fix(ingestion): bind same-name tRPC handlers and surface HANDLES_ROUTE
Same-name procedures resolve by startLine, the extractor keeps nested paths
through multiline schemas, and context/query UNION HANDLES_ROUTE for leaf
procedures that never become Process entries.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Clamp group query bounds, drop HOC pair false positives, emit tRPC
terminal lines and hyphenated quoted keys, cap chain BFS concurrency,
and restrict the router utility exemption to accessor names.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Score JS/JSX tRPC routers like TypeScript, ignore inner db.query and
unrelated .merge calls, mask regex braces, and assert clamp tests invoke
the query mock.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(autofix): apply prettier + eslint fixes via /autofix command
* fix(ingestion,mcp): emit-side HOC callback guards + MCP schema prettier (PR #3339 round 2)
* Address PR review feedback (#3339)
Treat `/` after return-style keywords as a regex, drop the synthetic
appRouter path prefix, and bind the create mutation via an inline callback.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Qualify query() docs so is_entry_point is promised only when the entry
symbol is among the search hits.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Emit tRPC terminals only at procedure paren depth, drop the filename
prefix on bare appRouter = router(), mask regex after if (), and cap
groupQuery member fan-out.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
De-duplicate context-chain BFS nodes reached through multiple frontier
edges, and bind tRPC identifier callbacks (`.mutation(handler)`) to the
handler symbol instead of the procedure key.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Apply LIMIT 50 after DISTINCT neighbors in the context-chain BFS, and
treat the official lowercase `procedure` builder as a tRPC procedure key.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Keep the later duplicate tRPC object-literal key so route binding matches
the handler JavaScript actually ships.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Compose same-file identifier-mounted tRPC subrouters so admin: adminRouter
emits the live admin.list path instead of an unprefixed /trpc/list.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Memoize tRPC identifier-mount paths so a depth-N chain stays linear, and
gate it with the build-free measure.mjs / baselines.json harness.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Compose identifier mounts without phantom /trpc URLs, bind wrapped and
multiline handlers, fail-close missing bench budgets, and reject query
page bounds before search.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Reject advertised MCP query bounds in group mode and chain_depth
instead of clamping or accepting non-integers.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Reject invalid groupContext chain_depth and ignore non-callable
same-file tRPC handler candidates.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Memoize live tRPC mount paths so the depth-N chain bench stays linear,
and render invalid group/MCP bounds without throwing.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Keep t.merge('prefix.', namedRouter) procedures live by recording a
zero-hop mount so the unmounted-router drop does not hide them.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback (#3339)
Recognize type-annotated `const adminRouter: AppRouter = t.router(`
bindings so identifier mounts still compose.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
Gergő Magyar
|
620fd18a5c
|
feat(cli): reclaim leftover per-branch indexes after branch delete (#3338)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* feat(storage): classify leftover per-branch index slots Operators need a shared enumerator for deleted-branch leftovers before clean --stale or doctor can reclaim or report them. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(storage): reclaim a per-branch slot and empty branches/ Named clean --branch now shares one rm-then-registry helper so the last leftover slot can drop the empty branches directory, and a failed rm still keeps the retryable summary. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(cli): add clean --stale to reclaim leftover branch indexes Operators can drop per-branch slots whose recorded branch is gone without remembering each name, while a git-list failure stays a no-op. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(cli): report leftover branch indexes in doctor Operators can see cwd orphaned per-branch slots and their size, then reclaim them with clean --stale, without doctor deleting anything. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): keep unreadable branch slots out of stale reclaim A stat error other than ENOENT/ENOTDIR must not look like a missing directory, or clean --stale --force drops the registry row and leaves the slot on disk. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(cli): keep leftover-slot display helpers in the CLI layer Preview and doctor share one size formatter and an i18n path for registry-only rows, so storage no longer owns display copy. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): keep leftover reclaim moving after a registry drop failure Catch removeBranchIndex rejections so --stale continues, match doctor registry rows through canonicalizePath, and size leftover slots sequentially. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): match leftover-slot registry rows with canonicalizePath Use the repo-manager path contract so clean --stale and --branch still see registry-only leftover rows when cwd and the stored path differ. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): contain leftover-slot deletes and re-check live heads Refuse symlink and junction escapes under branches/, unlink slot links instead of removing through them, and skip --stale --force when a name is a local head again. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3338) - Bound listLocalHeads spawnSync with GIT_PATH_LIST_MAX_BUFFER. - Clarify that doctor leftover reporting is cwd-only, not registry-wide. - Drop the MCP/serve assumption from clean --stale delete failures. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3338) - Describe disk-only leftover slots in --stale help, not only recorded branches. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): stop doctor reclaim copy when heads cannot be listed Doctor was naming clean --stale for leftover rows even when git cannot list local heads, which is a no-op. Print the retry-git message instead (#3337). Co-authored-by: Cursor <cursoragent@cursor.com> * revert: drop Unreleased changelog notes from this branch Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): keep live branch pins when a tag shares the name %(refname:short) disambiguates against tags, so clean --stale treated still-local heads as leftover. Fail closed on obstructed slots and unlistable branches/ directories. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3338) Bound leftover-slot listing, revalidate paths immediately before delete, and keep registry rows when a stray disk-only directory claims a recorded branch. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Parafee41
|
ee2feb7a5b
|
fix(cli): announce explicit registry alias changes (#3334)
* fix(cli): announce explicit registry alias changes * fix: handle async rename observer failures * Address PR review feedback (#3334) - Invoke onRename after withRegistryLock releases so observer I/O cannot stall the registry - Spy the rejecting observer and prove lock release via re-entrant registerRepo Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Parafee41
|
795cf0e151
|
fix(swift): resolve inherited protocol extension calls (#3309)
* fix(swift): resolve inherited protocol extension calls * fix(scope): gate inherited implicit receiver lookup * fix(swift): resolve call result types by exact callee * test(swift): align cache and local call expectations * fix(scope): reconcile replay diagnostics * fix(swift): preserve exact callable return types * fix(swift): capture throwing async call results * test(swift): refresh capture golden * test(swift): refresh scope capture baseline * fix(swift): require explicit callable returns * fix(scope): preserve duplicate return metadata * chore(scope): align index documentation * Address PR review feedback (#3309) Stamp only protocol/class-extension members (nested QN + SPM buckets), arity-narrow implicit-this across MRO, and keep Swift type peeling out of shared workspace-index via stripTypePreservingDecoration. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3309) Stamp extension members even when the extension declares a nested type, keep inherited class members ahead of protocol-extension defaults, and report replay-only interface-dispatch fan-out drops. Note: pre-existing failure in gitnexus tsc against an older gitnexus-shared dist not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3309) Walk inherited implicit-this owners nearest-first so a nearer override wins, and tighten Swift owner-stamp tests. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
ba39d5c009
|
feat(analyze): expose process-detection budget overrides (#3324)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
* feat(analyze): expose process-detection budget overrides (#3313) Operators can raise or lower process count, branching, trace depth, and the entry-point candidate pool via CLI, .gitnexusrc, or GITNEXUS_* without changing shipped defaults. A budget-only change re-detects flows on the next analyze without --force. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): say invalid budget flags still honor env A rejected --max-processes value was described as falling back to the built-in default even when GITNEXUS_MAX_* still won the next precedence tier. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(analyze): share process-detection defaults and skip unused walks Keep DEFAULT_CONFIG aligned with the budget resolver and count symbols only when maxProcesses is still dynamic. Co-authored-by: Cursor <cursoragent@cursor.com> * style(analyze): wrap process-detection budget files for prettier Co-authored-by: Cursor <cursoragent@cursor.com> * docs(analyze): name the real process-detection default formula Co-authored-by: Cursor <cursoragent@cursor.com> * docs(analyze): stop calling maxProcesses*2 a hard trace quota Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): say invalid env budget tokens fall back to defaults Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): recertify process-detection after in-place FTS abort (#3324) Persist processDetection.uncertified on the in-place FTS dirty stamp when the budget mismatched so a flagless retry cannot keep rewritten flows. Qualify .gitnexusrc fail-fast copy and tighten related tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(analyze): skip live dirty stamp on atomic incremental (#3324) POSIX atomic incremental mutates a staging copy, so stamping live incrementalInProgress before swap made a crash force-rebuild a healthy index. Align analyze --help with CLI > .gitnexusrc > env > default. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(changelog): drop the atomic-incremental dirty-stamp note The code fix stays; Unreleased no longer lists that recovery change. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): survive FTS SIGSEGV in --limit e2e CREATE_FTS_INDEX can kill the setup analyze on some WSL hosts (status null). Rebuild with --skip-fts and skip BM25-only query --limit cases unless GITNEXUS_REQUIRE_FTS=1. Refs #3324 Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): mark update-check child at import Writing refresh-started from fetch() raced a 30s poll against cold tsx boot on a loaded default-project worker. Refs #3324 Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3324) Isolate default-budget FTS crash-marker tests from GITNEXUS_MAX_* env, assert uncertify-before-FTS order and deferred flow detection on park recovery, drop the dangling "then" from entry-point help, and correct stale streamGraphEmit docs without skipping the process-detection stamp. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(changelog): drop Unreleased process-detection notes Keep the #3313 / #3322 code; Unreleased changelog matches main until release. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Abhinav Pandey
|
0ede6ae501
|
fix(rust): respect Cargo target boundaries in name fallback (#3294)
* fix(rust): respect Cargo target boundaries in name fallback * fix(rust): read Cargo sources through validated file descriptors * fix(rust): require matching imports for crate-root guesses * fix(rust): require Cargo root identity for cross-target imports * fix(rust): enforce Cargo identity across fallback imports * fix(rust): keep Cargo membership on typical derive/macros (#3294) Abort only genuine unknown expansion, ignore Cargo artifact layouts instead of every `target` path segment, and require a covering import for unanswered crate-root name fallback so #3253 still holds on ordinary Rust sources. Co-authored-by: Cursor <cursoragent@cursor.com> * test(rust): gate Cargo target membership in CI (#3253) Add a parse-dispatch-rounds-style bench so derive/macro abort, target-path globs, and superlinear membership walks fail in CI instead of staying graph-invisible. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(rust): verify Cargo macro and re-export evidence * style(bench): format Cargo membership benchmark * fix(rust): require imports for cross-file root guesses --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Gergő Magyar
|
21a52af1d4
|
fix(lbug): ship FTS per-platform and recover in-place native aborts (#3274)
* fix(lbug): pin Ladybug core so Dependabot cannot ship a skewed FTS artifact The extension version is a separate upstream constant. Ignore daily core bumps and fail the pairing gate when the committed manifest does not name the installed core. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): make doctor and CI FTS gates resolve the packaged artifact Doctor and the REQUIRE_FTS file gates still treated an empty ~/.lbdb as unavailable, which would turn three CI jobs red once analyze stops installing into that tree. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name native-abort and tuple-missing so analyze cannot mis-advise The CLI summary's trailing else treated every unknown skip reason as a missing extension. New crash and platform causes must get their own remedies, not a network-install hint. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): delete the dead read-path FTS index create ensureFTSIndex had no production callers and swallowed read-only CREATE_FTS_INDEX failures, which hid the only signal that a reader tried to write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): vendor per-platform FTS artifacts so analyze needs no host install Keyword search depended on a CDN fetch into ~/.lbdb. Shipping the five published tuples inside the package makes air-gapped and ignore-scripts installs load the same artifact the publish gate checksums. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): load the packaged FTS artifact before any network install Analyze still required a CDN fetch into ~/.lbdb even when the package already shipped the file. FTS now path-loads the vendored tuple first and records source labels so a later truncated home copy cannot steal the diagnosis. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): diagnose a core/extension version skew instead of a missing runtime A structurally valid FTS artifact whose path version disagrees with the packaged pin must name both versions, not prescribe VC++ or OpenSSL. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): stamp an FTS phase so repair stays usable after an in-place abort A native CREATE_FTS_INDEX abort leaves no skip reason; the next run infers it from the dirty flag, and --repair-fts must not treat that phase as a half-written graph. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): park an in-place FTS crash WAL without wiping the graph An FTS abort after a successful checkpoint must reopen the live index on macOS, Windows, and Linux. Staging never parks the live WAL; readers keep today's large-WAL refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): refuse read-only opens of an FTS-poisoned WAL MCP and serve cannot repair a leftover in-place abort. Fail before the native open and name --repair-fts, on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): name a vendor-neutral Windows OpenSSL prerequisite OQ1 is unanswered here so GitNexus does not ship OpenSSL DLLs. Windows FTS now asks for a system OpenSSL 3 runtime instead of Git Bash PATH. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): inject the FTS vendor root and redact it on HTTP and MCP Path-loaded artifacts no longer vary with HOME. Tests pass an injected vendor tree and assert search warnings never leak a filesystem path. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): document load-only as the global FTS install default Analyze still overrides to auto. Packaged per-platform artifacts load before any network install on macOS, Windows, and Linux. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(lbug): format the FTS install-policy README table Prettier does not run on Markdown in pre-commit, so the U10 table wrap needs its own formatting commit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): skip FTS CREATE after a persisted native abort A recovered analyze run was retrying CREATE_FTS_INDEX from skipReason alone. Keep that skip until --repair-fts, fail closed on unsupported tuples, and honor the checkpoint warrant for park/repair. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): honor checkpoint flushed warrant and align FTS tests with packaged vendor A no-op CHECKPOINT must not satisfy the FTS park warrant, and CI still asserted HOME-only FTS isolation after analyze started path-LOADing the packaged artifact. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): accept a nonempty incremental write set in the #2790 recovery check FTS-phase recovery can incremental-add files (changed=0, added=1). That is not the #2790 empty-diff wipe skip. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): compare FTS home versions to the core pin and tighten the publish filename gate Ladybug's ~/.lbdb/extension directory is the runtime/core version; treating it as the artifact version false-diagnosed skew. The publish guard now rejects a path-escaping filename the same way the fetch script does. Co-authored-by: Cursor <cursoragent@cursor.com> * test(lbug): seed FTS e2e fixtures from the packaged vendor artifact A machine with no ~/.lbdb copy should still run the vendor-survivorship cases; the seed no longer depends on HOME or a network install. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Keep in-place FTS abort evidence after persist so a second CREATE abort cannot fail-open readers, and close the CLI, loader, embed, and e2e gaps the review called out. Note: full npm test hit Ladybug worker-pool startup failures under memory pressure; tsc and 180 targeted unit tests passed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3274) Run the vendored-path symlink guard on the OS matrix, put e2e HOME fixtures on Ladybug's real extension layout, pin the embed crash-WAL gate before the writable open, and let analyze writers park through missing-shadow recovery. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(lbug): keep --repair-fts CI green after vendored-first FTS Never-installed warning fixtures must not inspect a packaged vendor binary, and a failed dirty restamp must not abort an otherwise successful --repair-fts run. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): give the #1169 analyze e2e the same 90s Windows budget as its sibling The first #1169 persist-meta case was still on a 60s spawn/it budget and was killed banner-only on windows-latest after the FTS warning fixture no longer failed the shard first. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): reweight Windows shards after the FTS e2e grew Vendored-first HOME fixtures pushed fts-extension-e2e to ~6 minutes on windows-latest, so the old 146s weight packed it with skills-e2e and blew the 20-minute watchdog. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
mengkaka
|
79543c8f83
|
feat(storage): add configurable index storage and content retention tiers (#3060)
* feat(storage): add configurable index storage and content retention tiers Rebase #3060 onto current origin/main. Keep GITNEXUS_STORAGE_PATH, GITNEXUS_STORAGE_ROOT, and GITNEXUS_CONTENT_RETENTION, and fold in main's FTS skip, embed-session, and help-text updates. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3060) Keep legacy registry rows on the local storage fallback, resolve symlinks before the destructive-path guard, and align hook lookup with CLI branch slugs, branch-slot metadata, and longest-path match. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3060) Only list swept upload directories after a successful removal so callers cannot treat a permission or transient rm failure as gone. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3060) Document that getStoragePath may consult registered storage while this module still does not mutate the global registry. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(storage): close review findings for external indexes and retention Re-inspect ownership under the analyze lock, fail-closed when the registry file is missing, and keep skip-git hook discovery plus retention fields on HTTP/MCP list surfaces. /api/file stays 410 unless contentRetention is full. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * Address PR review feedback (#3060) Treat lock-only index dirs as empty, honor HTTP --force storage policy, and prefer registered plus branch-aware slots in hooks and augment. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3060) Keep hook fallbacks inside the current worktree, compare foreign-local slots canonically, and make storage fixtures survive ownership validation. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix macOS hook test expecting realpath'd registry paths. resolveHookRepo returns the written registry path, not a filesystem realpath, so the assertion must match that. * Address gitnexus-check warnings on hook install docs and slot tests. The Cursor troubleshooting list omitted registry-query.cjs, and the writable-slot test only checked that isDirectory exists instead of that the path is a directory. * Align the HTTP catalog source-scan with skippable resolveRepo validation. resolveRepo lists fresh repos with validate: options.validateStorage !== false so DELETE can skip prune; the test still required a literal validate: true. * Harden storage path sinks so CodeQL path-injection and ReDoS alerts clear. Contain every filesystem probe inside the resolved storage slot with the inline path.relative idiom, reject filesystem-root slots, and trim slot basenames in linear time. * Settle bridge stamps before writing so CI size/mtime matches stay stable. LadybugDB can still flush into bridge.lbug after close+rename; persist whole-millisecond mtimes and wait for consecutive stats to agree so a freshly written pair matches. * Type the settled bridge stat as fs.Stats so tsc does not see bigint. Awaited<ReturnType<typeof fsp.stat>> collapsed the bigint overload and broke prepare/typecheck on CI. * Keep the bridge mtime stamp exact so same-size swaps still fail the pair check. Co-authored-by: Cursor <cursoragent@cursor.com> * Wrap the bridge stamp predicate so prettier --check stays green. Co-authored-by: Cursor <cursoragent@cursor.com> * Require a quiet interval before stamping a settled bridge file. Co-authored-by: Cursor <cursoragent@cursor.com> * Reuse shared storage and settle helpers instead of local copies. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
ceaff27c1e
|
fix(parse-cache): retire a chunk whose durable generation could not be reset (#3271)
* fix(parse-cache): retire a chunk whose durable generation could not be reset #3200 skipped the parse-cache write when `prepareDurableParsedFileChunk` failed, but the chunk hash was already in `usedKeys` from the lookup. When a previous generation existed on disk — reachable because the coherence gate re-dispatches a chunk whose `.v8` shard is live but whose durable shards are unreadable — `saveParseCache` copied that old shard forward, and the durable prune, which keeps exactly the saved keys, retained the mixed directory. The next run then served a warm hit out of a directory the previous run had already decided it could not account for. Retire the hash instead of only skipping the write: - `ParseCache.staleKeys` is a transient set that `saveParseCache` filters out of its key list. Filtering at save is what makes it survive the post-parse key merges in run-analyze (#2106 sibling fold, unreadable-meta retention), and it reaches both stores at once because the durable prune keeps exactly the keys `saveParseCache` returns. - The hash is retired at the reset-failure site, which runs unconditionally. The parse-cache write branch sits behind `rawResults.length > 0`, so a chunk whose worker round returns nothing would never have been retired there. - Worker-quarantined chunks get the same treatment for the same reason: they also reach the save with no in-memory entry, which is what triggers the copy-forward. That branch was previously unreachable when the worker died on the chunk and returned no results. - Guard the durable prune's non-survivor `fs.rm`. The causes that break the reset break that delete too, and it sat outside the validation try — one undeletable directory aborted the loop and cost every remaining chunk its index entry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): apply review findings Retire a chunk only when a generation nobody cleared is still on disk. `prepareDurableParsedFileChunk` is rm-then-mkdir, and the catch could not tell the two apart: an rm that succeeded before a failing mkdir leaves NO directory, so the workers recreate it and write a clean generation. Retiring there discarded a good `.v8` for no safety gain — and under a correlated failure (an empty durable index turns every chunk into a re-dispatched miss, then a descriptor burst rejects the resets en masse) it would have wiped both shared stores for every branch, where the pre-#3204 posture cost only the writes. `durableChunkHasStaleShards` is the discriminator. Finish the delete guard on the path that runs before it. The staged→live overlay in `mergeStagedDurableParsedFileStore` awaited `replaceDurableChunkDir` unguarded, so on the cold-rebuild path one undeletable directory threw out of the merge before the prune ever ran — the durable index was never rewritten and a retired chunk kept its directory. Same log-and-continue treatment, plus best-effort handling of the two `.replacing` backup removals. Aggregate the prune's delete-failure warning: a store-wide cause hits every non-survivor, and one line per directory buries the message that matters. Tests: - Guard the chmod-based prune test with the repo's `skipIf` for root/Windows and assert the directory survived, so it cannot pass vacuously where the delete succeeds. - Add a two-chunk control: one chunk's reset fails, and the sibling must stay warm through the next run. One chunk plus a global spawn marker could not tell "retires the failing chunk" from "retires everything". - Add the rm-succeeded/mkdir-failed case, which must NOT retire. - Model both post-parse merges in the R4 test (the sibling fold re-adds the key, the unreadable-meta fallback unions `entries`), and move the in-memory `entries` assertion to a direct helper test — the sharded path never populates `entries`, so the old assertion proved nothing. - Type the cache factory as `ParseCache`; the `staleKeys` assertions were TS2339 and `?? false` read as a pass regardless. - Register the store test in the cross-platform filesystem list. Correct two comments that still described a quarantined chunk by the premise this fix disproves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(review): stop swallowing the backup removal in replaceDurableChunkDir Swallowing that `fs.rm` manufactured the very hazard this PR removes. When a non-empty `${to}.replacing` survives, the following `fs.rename(to, backup)` cannot overwrite it and is suppressed as "dest was missing", so `backedUp` stays false and the `fs.cp(from, to)` fallback merges the staged generation INTO the live directory — old shards alongside new, which the prune then indexes as one valid survivor. Let it throw; the per-entry guard added to `mergeStagedDurableParsedFileStore` already stops one such chunk from costing the others their prune. The post-publish backup cleanup stays best-effort, where an undeletable leftover really is litter. Also: - Make the sibling-isolation test perform the run its title claims. It asserted index membership and stopped; an index entry does not exercise the warm-hit path, so it would have passed even if the sibling re-dispatched. Each chunk now runs alone so the single spawn marker names which one re-parsed. - Correct two comments that outran the implementation: retirement is gated on shards actually surviving, and an undeletable directory is dropped from the index rather than removed from disk. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
azizur100389
|
3f5ca8cdb7
|
fix(storage): guard stale file-lock reclamation (#3234)
* fix(storage): guard stale file-lock reclamation * fix(storage): close lock recovery failure paths * Address PR review feedback (#3234) - Flush the lock-child stderr diagnostic before process.exit - Treat explicit NaN timeouts as the default ceiling - Document non-retryable guard timeouts on the worker IPC contract Co-authored-by: Cursor <cursoragent@cursor.com> * fix(storage): stop mislabeling live lock waits as orphan recovery A brief peer inspect must not attach guardPath or send operators to RUNBOOK delete steps. Refuse lock-free embeddings sync, stop --watch only on a true guard timeout, and drop an unreadable self-created guard before failing closed. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * Address PR review feedback (#3234) - Verify lock/guard absence before degrading a denied main-lock create - Launch the third contender from unlinkSync, not the dead rename path - Document group-lock timeouts for unrecoverable guard leftovers Co-authored-by: Cursor <cursoragent@cursor.com> * style: prettier index-lock reclaim guard tests Co-authored-by: Cursor <cursoragent@cursor.com> * fix(storage): treat O_EXCL as the lock-file presence check CodeQL flagged existsSync-then-wx on analyze.lock. Create with wx first and only reclaim unreadable leftovers after grace, so a successor is never unlinked from a lost race. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Parafee41
|
68eca0ced8
|
fix: map deleted files to indexed symbol ranges (#3269)
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
azizur100389
|
f8036ac349
|
feat(analyze): add explicit FTS opt-out (#3205)
* feat(analyze): add explicit FTS opt-out * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(analyze): treat flag and env FTS opt-out as one mode Avoid a same-commit rebuild when only the skipReason discriminator changes, and advertise disablement on dirty meta before leftover indexes are wiped. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
auyua9
|
a4769439dd
|
fix(parse): retain metadata-only diff files (#3251)
* fix(parse): retain metadata-only diff files * Address PR review feedback (#3251) Keep detect_changes honest on C-quoted, TAB-terminated, and ambiguous diff --git dests, and fail closed when a header cannot be parsed. Co-authored-by: Cursor <cursoragent@cursor.com> * Simplify parseDiffHunks dest-prefix helper (#3251) Drop the unused a/ prefix arm and the redundant empty-parse unparsed-header check. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(parse): ignore +++ inside hunks and accept mixed-quoted git headers Stop treating hunk-body lines as file headers, and parse independently C-quoted src/dest tokens on diff --git. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: ming <silverchris@foxmail.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Abhinav Pandey
|
2220f4d851
|
fix(resolution): label fallback guesses and preserve export visibility (#3190)
* fix(resolution): distinguish name guesses and preserve export visibility * test(go): keep method enrichment fixture in one package * fix(resolution): address split review edge cases and evidence reporting * fix(exports): recognize imported and expression-local receivers * fix(resolution): align export and target evidence with language scope * fix(ingestion): preserve lexical import provenance through resolution * test(ci): rebalance Windows shards from measured slow suites * Address PR review feedback (#3190) - Label constructor unique-name guesses as global-name-fallback and run language vetoes - Tighten Go qualified, Rust crate::, and Ruby class-reopen fallback guards - Ignore for-loop shadowed CommonJS receivers and exclude guesses from the resolved-call census - Refresh FinalizeOutput and hook docs for lexical binding scopes Co-authored-by: Cursor <cursoragent@cursor.com> * Tighten review-feedback leftovers on fallback visibility. Qualified Go calls still respect export and test-package rules, nested Rust src/ stays a module segment, and top-level conditional this.x is treated as CommonJS. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * Address PR review feedback (#3190) - Distinguish Swift package prefixes when comparing target modules - Document lexical import binding and handledSites refusal marking - Drop the stale ci-scope-parity workflow claim and prototype-safe export verdicts Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3190) Supply caller source on Ruby visibility cases so they exercise the named allow branches instead of the missing-text bypass. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(resolution): label unique constructor types as name guesses A workspace-unique class hit in findClassBindingInScope was treated as an in-scope bind, so Go/JS constructor-form sites skipped the guess label and the Go unexported veto. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(resolution): keep qualified constructors precise after unique-name split Bare constructor unique-name hits stay guesses so Go can veto an unexported type. A written qualifier is now carried as rawQualifiedName so `new pkg.Foo()` and `models.Box[T]{}` can still recover the unique class without that veto. Co-authored-by: Cursor <cursoragent@cursor.com> * test(bench): rebaseline Go/Java scope-capture fingerprints for constructor qualifiers Generic Go composite literals and qualified Java `new pkg.Foo()` now carry @reference.qualified-name on existing constructor matches. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(resolution): treat import-reached unique constructors as precise C++ #include and Rust re-exports do not mint a lexical class binding. A unique type in an imported file (or imported directory) is therefore a real bind, not a name guess, so those CALLS edges stay import-resolved. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(resolution): require named or resolved imports for constructor precision Bare Go Box[T]{} is not package-qualified, and a sibling-file import of a different name is not constructor visibility. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Navid EMAD
|
506432017f
|
fix(zig): model callable-value references, and stop reporting their absence as exact (#3219)
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Scorecard / Scorecard analysis (push) Waiting to run
* fix(impact): stop reporting 'exact' over unmodelled callable-value references
A function named in VALUE position — `bridge.accessor(Element.getNamespaceUri,
null, .{})`, `{ onClick: handler }`, a comparator handed to a sort — is
registered somewhere rather than called. The registration is modelled (a
`value-ref` site becomes a USES edge; Kythe `ref` vs `ref/call`, Joern
METHOD_REF), but the invocation THROUGH the stored value is not: it happens
later via a struct field, a registry lookup or comptime reflection.
impact()/context() nonetheless reported such a target as `epistemic: 'exact'`.
For lightpanda-io/browser that meant the DOM `Element.namespaceURI` accessor
came back with two internal callers, LOW risk and a claim of completeness —
worse than no answer, because 'exact' tells the reader not to look further.
tools.ts defines 'lower-bound' as "the walk provably missed callers", which is
precisely this case.
computeEpistemicBoundary now probes for inbound USES edges stamped with the
value-ref reason and hedges when it finds any, contributing a boundary note and
a new `causes.callableValueReferences` (unit: distinct referrer symbols).
Read from the graph, not from index metadata: unlike a dropped receiver — which
leaves no edge to find and therefore needs a persisted summary — a value
reference IS in the graph. So the signal needs no re-index and no analyzer
change, and it works on indexes written before this commit.
The cause gets its own slot rather than joining `dispatchBoundary`: a value
referrer is neither an implementation nor an interface-level consumer, and the
two differ in what the reader should do about them — a dispatch boundary is
irreducible, a callable value usually becomes traceable once the provider
models the store/load that carries it.
Language-neutral: every provider that emits a `value-ref` capture participates.
The writer and the reader now share VALUE_REF_EDGE_REASON, because drift
between them would fail silently — the probe would match nothing and every
answer would go back to claiming certainty.
* fix(zig): emit `value-ref` for a callable named in value position
`mapReferenceKindToEdgeType` has handled the registration-vs-invocation case
since #2437, and TypeScript, JavaScript and C++ all emit `value-ref`. Zig
emitted zero — so a Zig function handed somewhere as a VALUE was absent from
the graph entirely.
That is not a corner: Zig's JS bridge is built out of this one shape,
pub const namespaceURI = bridge.accessor(Element.getNamespaceUri, null, .{});
and `bridge.{accessor,function,indexed,…}` appears 2,047 times across 257 files
in lightpanda-io/browser — the project's whole JS<->Zig surface, none of it
reaching the graph. `impact` on `Element.getNamespaceUri`, which IS the DOM
`Element.namespaceURI` accessor, answered with its two in-file callers.
Three query rules, tagging `@reference.value-ref` on: a bare identifier in
argument position (`bridge.accessor(_tagName, …)`), a qualified one
(`bridge.accessor(Element.getNamespaceUri, …)`), and a const binding initialiser
(`pub const defaultHandler = onReset;`). Everything downstream already existed —
no new edge type, no schema change, no capture-machinery change, no baseline
edited.
One grammar detail is load-bearing: in tree-sitter-zig, call arguments are
DIRECT children of `call_expression` — there is no `arguments` node, only
builtins have one — so the callee has to be consumed explicitly by `function:`.
Without that binding the same rule also matches the callee of `foo(bar)` and
mints a USES edge shadowing the call's own CALLS edge. There is a test for it.
The rules are deliberately broad — `js.Bridge(Element)` and `register(count)`
match too — because the callable gate in the property-dispatch pass
(Function/Method/Constructor only) is the filter, the same design that keeps
TypeScript's `{ port: DEFAULT_PORT }` from registering anything. Measured on the
real corpus: all 3,169 emitted edges land on a Method (3,146) or Function (23).
Deliberately NOT modelled: the terminal invoke. The value reaches
`Accessor.init` -> a struct field -> `Factory.zig` reflection (`inline for`,
`@typeInfo`) -> `Caller.zig`'s `@call(.auto, func, args)` over `func: anytype`.
Resolving that needs comptime evaluation. The point is to stop dropping the
reference; the shortfall is now reported as `epistemic: "lower-bound"` by the
preceding commit instead of being papered over.
Measured on lightpanda-io/browser (698 .zig files), wiped index both times:
30,222 nodes / 71,070 edges -> 30,222 nodes / 74,229 edges. The delta is
entirely `USES` (3,169 after vs 10 before, and every one is a value-ref); CALLS,
ACCESSES, IMPORTS, HAS_METHOD, DEFINES and the rest are unchanged — purely
additive, no node invented.
The fixture joins the existing `zig-idioms` corpus rather than adding a new
one, because `bench/receiver-resolution` uses `test/fixtures/lang-resolution` as
its `--check` corpus. That gate, `scope-capture` (15 languages),
`zig-cross-file-resolution` and every other bench `--check` pass unchanged.
* fix(review): resolve qualified value references through their owner, and stop
hedging on registrations the analyzer already followed
Five findings from the review bot on #3219; four valid, all addressed.
1. QUALIFIED VALUE REFERENCES RESOLVED BY TAIL NAME (the serious one).
`bridge.accessor(Element.getNamespaceUri, …)` was resolved with
`findCallableBindingInScope(site.inScope, site.name, …)`, which never sees
the receiver and gives LOCAL bindings precedence. Reproduced:
const Element = @This();
pub fn getThing(...) // main.getThing
pub const JsApi = struct {
fn getThing(...) // JsApi.getThing
pub const thing = bridge.accessor(Element.getThing, null, .{});
};
emitted `USES JsApi → JsApi.getThing` — a WRONG edge, which is worse than
the missing edge this PR set out to fix.
`resolveValueRefTarget` now resolves a site carrying an explicit receiver
through `findClassBindingInScope` → `findOwnedMember` (the machinery
`receiver-bound-calls` already uses), gated on `CALL_TARGET_TYPES` because
`findOwnedMember` also answers with fields. A bare site keeps the lexical
walk, which is what an unqualified name means. When the owner cannot be
resolved the site is DECLINED rather than falling back: declining costs a
reference, falling back mints a confident edge to the wrong function, and
the missing reference is now reported as `lower-bound` anyway.
Cost on lightpanda-io/browser: value-ref edges 3,169 → 2,799 (−12%). Those
370 were tail-name coincidences, not registrations — all 94 `Element.zig`
`JsApi` entries survive, the cross-container case resolves and is correctly
attributed (`IntersectionObserverEntry.JsApi` → `IntersectionObserverEntry.
getTarget#0`, not the enclosing observer), and both acceptance probes are
unchanged: `getNamespaceUri` 7/3/LOW/lower-bound, `getTagNameLower`
31/10/HIGH/exact.
2. A FAILED PROBE READ AS "NO BOUNDARY". `.catch(() => [])` turned an
unanswerable query into count 0 and no note, so `exact` could be published
on the strength of a question that was never asked. It now returns `null`
and emits a boundary note. This file's own `loadMeta` comment states the
rule: a probe failing must never read as certainty.
3. NOT EVERY VALUE REFERENCE IS AN UNMODELLED INVOCATION. Where
`emitPropertyDispatchCalls` sweep 2 synthesized the dispatch (reason
`property-dispatch`), the walk did NOT provably miss the caller, so hedging
was noise over an answer that was computed — and a signal that fires on
every JS/TS hook table stops carrying information. A second probe excludes
those targets. Zig never sets a property key, so the motivating case is
untouched. The exclusion is symbol-level, not per-edge, because the graph
does not record which registration produced which synthesized call; that
residual is documented at the call site.
4. `LIMIT 50` SILENTLY UNDERSTATED THE COUNT. `rows.length` over a capped row
set published a ceiling as the documented symbol count. Replaced with
`COUNT(DISTINCT other.id)` — bounded work without a bounded answer, the
shape `countByType` in the same file already uses.
5. THE TEST DID NOT PIN THE CALLER COUNT its own comment promised. Added
`expect(result.impactedCount).toBe(2)`.
New regression tests: qualified references bind the written owner and not the
nearer lexical match; an unresolvable receiver emits nothing rather than a
wrong edge; a dispatch-modelled registration stays `exact`; a probe that cannot
run hedges instead of claiming certainty.
Known limitation, pinned by a test rather than left implicit: when a `@This()`
alias's NAME differs from its container's (`const Element = @This();` inside
`main.zig`), the receiver does not resolve and the reference is declined. The
provider has `rewriteZigThisAlias` for this, but it is applied to type nodes
and extending it to reference receivers would change existing CALL-site
behaviour. Lightpanda's `const Foo = @This();` in `Foo.zig` convention makes
the names coincide, which is why the corpus is unaffected.
* fix(review): bump the parse-cache schema and resolve module-owned value references
Review round 2 on #3219. Four inline items, all reproduced against the
worktree before deciding.
P1 — the new captures could stay INERT on a warm parse cache. Adding
`@reference.value-ref` rules to `ZIG_SCOPE_QUERY` changes
`ParsedFile.referenceSites`, which is a PARSE-TIME fact, but `SCHEMA_BUMP`
stayed at 93. A repo indexed before this branch and re-analyzed after it
replays the old, empty site list for every unchanged `.zig` file — `--force`
included, since shards are content-addressed — so no USES edge is emitted, the
boundary probe measures a real zero, and `impact` on a registered accessor goes
back to `epistemic: "exact"`. #3399 un-fixed on the incremental path most users
are on, with every cold-run test still green. DECISIONS D1-1's "zero changes to
… the schema" conflated the graph schema with the cache schema.
Bumped 93 -> 98, not 94: #3190 claims 94 and #3179 claims 94 through 97 in one
PR. `incremental-parse-cache.test.ts` re-pinned, with 93-97 added to the taken
list. Re-check against origin/main and open PRs immediately before merging.
P2 — a qualified value reference through a MODULE was declined with no hedge.
R1-2 resolves a written receiver with `findClassBindingInScope`, which requires
`isClassLike`; a namespace-only `@import` handle is not class-like, so
const dom_utils = @import("dom_utils.zig"); // no `@This()` in that file
pub const comparator = bridge.accessor(dom_utils.compare, null, .{});
resolved to nothing. That is not the conservative half of R1-2's trade-off: a
declined site emits NO edge, so there is nothing for the probe to read and
`impact` on `compare` reports `exact`. Silence, not a hedge — and the pass
comment claiming otherwise was wrong on this path.
`resolveValueRefTarget` now tries the second kind of owner a qualified name can
have. `findNamespaceValueRefTarget` reads the file's `namespace` import edges
for the handle and the target module's own `origin: 'local'` module-scope
bindings for the member — the same channel `receiver-bound-calls` Case 1
already trusts for `dom_utils.compare()`, with Case 1's three guards for Case
1's reasons: `isNamespaceNameShadowed`, local-origin bindings only, and two
distinct defs under one name resolve nothing. `CALL_TARGET_TYPES` gates module
owners exactly as it gates container owners. Language-neutral: it reads generic
namespace import edges, names no language.
Still declined, deliberately: a receiver this index knows under no name at all
(the `@This()`-alias case, owners outside the workspace). There the alternative
is a confident edge to a lexically-nearer function the source did not name.
P2 — `impact-callable-value-references.test.ts` was in neither vitest list.
It opens a real engine via `withTestLbugDB(poolAdapter: true)`, so TESTING.md
puts it in the `lbug-db` include list and the `default` exclude list; it was in
neither, so `default` also collected it into the parallel pool. Added next to
its `impact-epistemic-lower-bound` sibling in both arrays.
P3 — DECISIONS.md was stale against HEAD and embedded host paths. D2-3 still
described the `LIMIT 50` that R1-5 removed and D1-3 still described the
receiver-blind resolution that R1-2 replaced; both now carry explicit
"superseded by" pointers. The `~/code/...` and mise-node paths are replaced
with placeholders.
Two smaller corrections the new cause made necessary:
- `formatImpactResult`'s `lower-bound` header hard-coded "callers binding via
DI / dynamic dispatch", which now contradicts the value-reference bullet
printed directly under it. The bullets carry the cause; the header only
states that the count is a floor.
- `tools.ts` said a `causes.callableValueReferences` of 0 means "nothing was
missed". The dispatch exclusion is symbol-level, not edge-level, so a target
with both a followed registration and an unfollowed escape also reads 0. The
docs now say a 0 means "no unfollowed registration was proven".
Fixture: `src/webapi/dom_utils.zig` (namespace-only) plus three cases in
`Element.zig` — the module-qualified registration, a non-callable module member,
and a `u8` parameter shadowing the handle. The shadow case was verified to FAIL
with the guard disabled, so it is not passing for an unrelated reason.
Gates: `tsc --noEmit` clean, `npm run build` clean, prettier clean.
`resolvers/` 3,630 passed / 3 skipped; `unit/scope-resolution` 2,010 passed;
`impact-callable-value-references` 7 passed under `lbug-db`;
`incremental-parse-cache` 40 passed; eval formatters 104 passed. Bench --check
all PASS with no baseline edited: receiver-resolution, zig-cross-file-resolution,
scope-emission, callable-value-flow, scope-capture (15 languages), python-scope.
* fix(review): guard the class receiver against a shadowing binding, and pin the dispatchability partition
Review round 3 (`gitnexus-check` bot on `cf53bbaa`). Three findings, each
reproduced against the code before deciding.
R3-1 (Error, valid, REPRODUCED) — a CLASS receiver could resolve through a
shadowing value binding. `findClassBindingInScope` is a class-only walk: it
filters the scope chain by `isClassLike`, so it steps over a nearer binding that
is a value and keeps climbing — and past the chain entirely, into a
qualified-name fallback that answers with the unique workspace definition of the
name. A `u8` parameter named `Ticker`, in a file that neither declares nor
imports the `Ticker` container another file defines, therefore emitted
`register(Ticker.fire)` as a confident USES edge to that container's method.
That is exactly the wrong-edge failure R1-2 exists to prevent, arriving through
the class channel instead of the lexical one.
Fixed with `isOwnerNameShadowedBySomethingElse` — a sibling of
`isNamespaceNameShadowed` with one extra clause. The plain namespace guard could
NOT be reused: a container is often its own local declaration
(`fn make() { const Local = struct {…}; register(Local.go); }`), and reading that
binding as its own shadow suppresses precisely the resolutions this path exists
to make — the #2723 mistake, one channel over. So a scope that binds the name
answers immediately, and the answer is "not shadowed" only when one of that
scope's own bindings IS the def just resolved.
Both halves are pinned and both were verified to fail when the guard is
weakened: the parameter case fails with no guard at all, the local-container
case fails with the plain `isNamespaceNameShadowed`.
R3-2 (Warning; mechanism correct, unreachable today; fragility fixed instead) —
the dispatch exclusion could suppress an unfollowed registration. The bot is
right about the code: sweep 2 synthesizes CALLS only for a registration whose
site carried a `propertyKey`, while the exclusion zeroes the note on ANY inbound
`property-dispatch` CALLS edge. It is not reachable in the current rule set, and
the reason is measured rather than assumed: `@reference.value-ref` is emitted by
exactly three languages — JavaScript (2 rules), TypeScript (2), Zig (3) — every
JS/TS rule also captures `@reference.property-key` (both are object-literal
shapes) and no Zig rule does. A dispatchable registration is therefore always a
JS/TS one, an undispatchable one always a Zig one, and they cannot meet on one
symbol.
Rejected: splitting the edge `reason` into dispatchable / undispatchable. It is
the precise fix, but it is a graph-content change that churns whichever side
keeps the old literal — the Zig, TypeScript and probe suites all pin
'scope-resolution: value-ref' by hand as a drift canary — and it buys nothing
against a case no rule can produce.
What was actually wrong is that the exclusion's soundness rested on a
coincidence recorded nowhere, in files nobody reading `local-backend.ts` would
open. Fixed at both ends: the exclusion site now states the invariant, the three
facts it rests on and the two options for when it breaks; and
`value-ref-dispatchability.test.ts` fails the day it does — a JS/TS rule for a
bare callback argument, a Zig rule that grows a key, or a fourth language
emitting `value-ref` at all. Verified to fire (adding a property key to a Zig
value-ref rule fails the Zig case), and its rule splitter has its own guard test
so the suite cannot pass vacuously. `ZIG_SCOPE_QUERY` is exported for that test
only.
R3-3 (Nit, valid) — a test comment claimed the wrong epistemic result. The
`declines a qualified reference whose receiver cannot be resolved` case said the
shortfall shows up as `lower-bound`. It does not: with no edge there is no
evidence and the target stays `exact`. The pass docstring was corrected in round
2 and this comment was missed. It now says the decline costs the reference AND
the hedge, and why that is still the right trade.
Gates: tsc --noEmit clean, npm run build clean, prettier clean.
`test/integration/resolvers` 3,632 passed / 3 skipped (70 files);
`test/unit/scope-resolution` 2,015 passed (120 files);
`impact-callable-value-references` 7 passed under `lbug-db`. Bench --check:
receiver-resolution, zig-cross-file-resolution, scope-capture (15 languages),
scope-emission PASS with no baseline edited; callable-value-flow failed once on
its TIMING budget (2.006 > 1.9) with a byte-identical fingerprint, then passed
twice at 1.788 / 1.813 — machine load, not a regression.
* fix(review): let a file's own `@import` outrank the workspace-wide class fallback
Found by running the local preflight harness before pushing rather than after.
One of its five findings is a real hole in R2-2; the rest are documentation that
overclaimed.
R4-1 (valid, REPRODUCED) — the container channel preempted the file's own
`@import`. R2-2 added the module channel as a FALLBACK after
`findClassBindingInScope`, and that order is wrong: `findClassBindingInScope`
does not stop at the scope chain. When its `isClassLike` walk misses — and a
namespace handle binds a Module, so it always misses — it falls back to
`scopes.qualifiedNames`, a workspace-wide index, and answers with the unique def
of that name anywhere in the repo. A container named `dom_utils` in a file
`Element.zig` never imports therefore captured
`bridge.accessor(dom_utils.compare, …)`, binding
`Method:src/webapi/decoy.zig:dom_utils.compare#2` while the module channel that
would have answered correctly was never reached. R3-1's shadow guard cannot
catch it: the import binds at MODULE scope, which that guard treats as the floor.
Fixed by trying the module channel FIRST. An `@import` written in this file is
the strongest available statement about what the name means here and outranks a
global uniqueness guess; when the handle is not an import of this file the
channel answers nothing and the container path runs exactly as before. Pinned by
`decoy.zig` and a strengthened assertion on the existing module-owner test,
which fails on the old order.
R4-2 — the cause documentation named shapes nothing captures. `tools.ts`
illustrated `callableValueReferences` with "a callback argument", "a stored
function pointer" and `qsort(xs, n, sz, compareItems)`. Only Zig captures a call
argument or a const initialiser; JS/TS capture only object-literal property
values, and C has no value-ref rule, so the `qsort` example is counted in no
language. Both cause blocks now name the captured shapes and say that a bare
JS/TS callback argument is not among them, so a 0 does not rule it out.
R4-3 — the same block exempted itself from the re-index caveat this PR proves it
needs. "Read from the graph, so it needs no index-time metadata" is true of the
probe and false of the edges: an index built before a language emitted these
captures has none and reports 0 — the warm-cache failure R2-1 bumped
SCHEMA_BUMP for. It now says to re-analyze before reading a 0 as measured.
R4-4 — the dispatchability canary was narrower than its own promise. Its header
claimed it fails on "a fourth language emitting value-ref at all"; it reads
`languages/<dir>/query.ts`, so Vue — which owns no query and borrows
`emitTsScopeCaptures` / `emitJsScopeCaptures` — emits value-refs while the
assertion lists three languages, and a capture synthesized in code is invisible
to it. The case now asserts on query OWNERS, which is what it checks and is
sound because a delegating language inherits the rules it borrows; the header
states the synthesized-capture gap instead of letting a green tick imply it away.
R4-5 — the BARE docstring described a lexical walk that is not one.
`findCallableBindingInScope` applies the callable predicate WHILE walking, so a
nearer parameter or local is stepped over: the defect R3-1 fixed on the container
channel, unguarded here, pre-existing since #2437 and reachable in JS/TS. Out of
scope for #3399, so behaviour is unchanged and the sentence now says what the
walk does rather than implying a guarantee it does not give.
Gates: tsc --noEmit clean, npm run build clean, prettier clean.
`test/integration/resolvers` 3,632 passed / 3 skipped (70 files);
`test/unit/scope-resolution` 2,015 passed (120 files);
`impact-callable-value-references` 7 passed under `lbug-db`. Bench --check:
receiver-resolution, zig-cross-file-resolution, scope-capture (15 languages),
scope-emission, callable-value-flow all PASS with no baseline edited.
* fix(review): honour the hub opt-in for value references, so `hub.fn` means one thing
Review round 5 (`gitnexus-check` bot on `1c7c05ff`). One finding, valid and
reproduced before fixing.
`findNamespaceValueRefTarget` accepted only `ref.origin === 'local'`. R2-2
recorded that as deliberate — "the `namespaceExportsIncludeImportedNames` hub
opt-in is a provider decision this language-neutral pass does not make" — and
that reasoning was wrong twice over. Zig sets the flag
(`languages/zig/scope-resolver.ts:41`, measured on ghostty and tigerbeetle before
it landed), and the pass does have the provider in scope: `runScopeResolution`
takes one and already forwards several of its hooks to other passes.
The result was the exact asymmetry R2-2 argued against in its own first
paragraph. A Zig hub declares nothing — every name it publishes it imported — so
requiring a local declaration declines every member reached through one:
// hub.zig
pub const scale = @import("dom_utils.zig").scale;
// Element.zig
const hub = @import("hub.zig");
pub fn callsThroughTheHub(v: u8) u8 { return hub.scale(v); } // resolved
pub const scaled = bridge.accessor(hub.scale, null, .{}); // declined
One name meaning two different things depending on whether a `(` follows it.
Fixed by forwarding `provider.namespaceExportsIncludeImportedNames` into the pass
and consulting the published channel when it is set — the same question
`receiver-bound-calls` Case 1 asks, through the same `lookupBindingsAt` read
`findExportedDefIncludingImportedNames` performs for the CALL form. The pass
still names no language; it asks the provider, which is the sanctioned hook.
Precedence is unchanged where it mattered: a locally declared member still wins
over a republished one, two distinct defs under one name still resolve nothing,
and `CALL_TARGET_TYPES` still gates the answer — pinned by `hub.DEFAULT_NS`, a
re-exported CONSTANT, which stays unregistered. Languages that do not opt in are
unaffected: the parameter defaults to `false`, and passing `false` was verified
to fail the new hub test, so it is load-bearing. The fixture republishes a member
(`scale`) that nothing else uses, so the hub assertion cannot be satisfied by an
edge another case emitted.
Gates: tsc --noEmit clean, npm run build clean, prettier clean.
`test/integration/resolvers` 3,634 passed / 3 skipped (70 files);
`test/unit/scope-resolution` 2,015 passed (120 files);
`impact-callable-value-references` 7 passed under `lbug-db`. Bench --check:
receiver-resolution, zig-cross-file-resolution, scope-capture (15 languages),
scope-emission, callable-value-flow all PASS with no baseline edited.
Also recorded in DECISIONS.md: `/autofix` returned "No successful autofix run"
because the `PR Autofix` run on `1c7c05ff` failed at `actions/upload-artifact`
with a 403 from GitHub's artifact storage, not because of anything in this
branch. This push triggers a fresh run.
* fix(review): inspect the module scope in the owner-shadow guard, and check the TSX suffix
The `gitnexus-check` pass on `bd6e577e` carried three findings of its own, and I
answered the later `1c7c05ff` pass without noticing them. Recording that as a
process failure too: bot reviews are per-head, and a later pass does not
necessarily repeat an earlier one's findings.
R6-1 (Error, valid, REPRODUCED) — the shadow guard stopped one rung short.
`isOwnerNameShadowedBySomethingElse` returned `false` on reaching the module
scope, justified as "a container declared there IS the binding, and the caller
already resolved it". That holds when the owner came from the scope chain and
fails when it came from the workspace-wide qualified-name fallback:
// Gauge.zig — never imported by Element.zig
const Gauge = @This();
pub fn read(self: *Gauge) u8 { … }
// Element.zig
const Gauge = @import("dom_utils.zig").DEFAULT_NS; // NOT a container
pub const level = bridge.accessor(Gauge.read, null, .{}); // → Gauge.zig's read
`findClassBindingInScope` steps over the module-scope binding because it is not
class-like, answers from `scopes.qualifiedNames`, and the guard waved it through.
Worth recording: the first fixture attempt did NOT reproduce. A local
`const Gauge: u8 = 3;` also claims the workspace qualified name `Gauge`, leaving
two candidates, and the fallback refuses to guess between two — so the shape
defeated itself. Binding the name by IMPORT claims no qualified name, the
fallback stays unique, and it fires. A negative result on the first shape was not
evidence the finding was wrong.
Fixed by inspecting the module scope as the last rung instead of skipping it.
The identity exemption is what makes that safe where `isNamespaceNameShadowed`
cannot do it (#2723: a namespace import writes its own name into the module scope
and would read as its own shadow) — the binding that IS the owner exempts itself,
and only a binding to something else answers `true`. `lookupBindingsAt` is
consulted at that scope and only there, because an imported alias lives in the
finalized channel rather than in `scope.bindings`.
R6-2 — the hub re-export finding, already fixed in `5d8fe9d8`; the same defect
restated on the later head.
R6-3 (valid, fixed) — the dispatchability canary omitted the TSX suffix.
`getTsScopeQuery` analyzes a `.tsx` file with `TYPESCRIPT_SCOPE_QUERY +
TSX_JSX_QUERY_SUFFIX`, and the test read only the base, so a `value-ref` rule
added to the suffix would be emitted in TSX analysis with the canary green. The
suffix is now exported and concatenated into the check; verified load-bearing by
adding an unkeyed `jsx_expression` value-ref rule to it, which fails the
TypeScript case.
Gates: tsc --noEmit clean, npm run build clean, prettier clean.
`test/integration/resolvers` 3,635 passed / 3 skipped (70 files);
`test/unit/scope-resolution` 2,015 passed (120 files);
`impact-callable-value-references` 7 passed under `lbug-db`. Bench --check:
receiver-resolution, zig-cross-file-resolution, scope-capture (15 languages),
scope-emission, callable-value-flow all PASS with no baseline edited.
* perf(bench): gate callable-value reference resolution on linear scaling
`resolveValueRefTarget` (#3399) replaced one lexical walk with four
channels, and the last of them — a qualified receiver resolved through
`findClassBindingInScope` — falls back to `scopes.qualifiedNames`, a
WORKSPACE-WIDE index consulted once per site. Keyed that is O(1); scanned
it is O(files) per site, and a registration table that costs O(sites)
today costs O(sites x files) tomorrow. Nothing in the suite can see that:
the fixtures are single-file, and the corpus it actually matters on is
lightpanda-io/browser, where `bridge.{accessor,function,…}` appears 2,047
times across 257 files.
`bench/value-ref-resolution/measure.mjs` builds two synthetic Zig corpora
of identical shape 4x apart in file count, and times ONLY the per-site
resolution loop — extraction, `reconcileOwnership` and finalize are setup.
`linear_factor` is `(t_large/t_small)/(N_large/N_small)`: measured
1.00-1.09 across runs, with `us_per_site` flat at 1.4-1.5 between the arms.
Correctness comes first, because a timing gate alone is satisfied by a fast
wrong answer: exact site/resolved/declined counts per arm plus an
order-independent sha256 over every (site -> resolved target) pair. The
corpus exercises all four channels — CONTAINER, NAMESPACE, HUB and BARE —
and carries two DECLINE controls per module (a non-callable namespace
member, a non-callable bare argument), so a widened callable gate moves
`declined` instead of hiding inside the timing.
Verified load-bearing rather than assumed: making the qualified lookup scan
a workspace-sized collection leaves the fingerprint IDENTICAL and takes
`linear_factor` to 3.752 against a slack of 1.375 — the regression class
this exists for is exactly the one no correctness gate can see.
Zig is the corpus because it is the only language whose provider sets
`namespaceExportsIncludeImportedNames`, so it is the only one that can
exercise the hub channel at all; the pass itself names no language.
`resolveValueRefTarget` is exported for the bench. Timing
`emitPropertyDispatchCalls` instead would fold the signal into edge
emission, and re-implementing the channel order in the bench would pin the
bench's idea of the function rather than the function.
* feat(zig): index every build package in the repo, not only the root one
Zig already had workspace setup — `loadZigBuildConfig` has parsed
`build.zig.zon` `.path` deps, the root `build.zig`'s named modules and each
build module's own `addImport` table since #1432, threaded through
`ScopeResolver.loadResolutionConfig` exactly as tsconfig is for TypeScript.
What it did not have is the part `tsconfigFor` supplies: per-package scope.
`loadZigBuildConfig` reads `<repoRoot>/build.zig{,.zon}` and nothing else, so
a repo laying its packages out as `packages/<name>/build.zig` — no root build
files at all — got `null`, and EVERY bare `@import("<module>")` in it went
unresolved. Cross-file resolution silently degraded to relative imports.
Measured on a two-package probe before this change: `config = null`,
`@import("core")` from `packages/app/src/main.zig` → `null`.
`loadZigWorkspaceIndex` discovers packages the way `findTsconfigFiles`
discovers configs — one bounded breadth-first walk that skips the hardcoded
ignore set — and `zigPackageFor` is the `tsconfigFor` analogue: the nearest
enclosing package governs a file, deepest-first, with no fall-through to an
outer package. Fall-through is what makes a vendored dependency's
`@import("config")` resolve to the outer repo's `config` module, the same
failure `loadTsconfigIndex` documents for a package declaring no `baseUrl`.
`resolveZigImportInternal` is UNCHANGED — impact analysis puts it at HIGH risk
with 6 dependents, and it does not need to move: it is handed one package's
config, and which config it receives is the only difference. Its 48 existing
tests pass untouched. A nested package's paths are rebased to repo-relative at
load time (`.path = "../core"` → `packages/core`, which the root-relative
reading rejects outright as an escape); the ROOT package keeps its raw
spelling, which is what `parseZigBuildZon` promises and its tests pin, so a
single-package repo is byte-identical.
`loadImportConfigs` — which runs unconditionally for every repo, Zig or not —
keeps calling the root-only loader, so no non-Zig repo pays for the walk. That
is the split TypeScript already has between the cheap `loadTsconfigPaths` and
the repo-walking `loadTsconfigIndex`, which `loadResolutionConfig` reaches only
during a language pass.
The `zig-monorepo` fixture carries the discriminating case rather than only the
happy path: `tool` binds the alias `core` to its OWN `src/core.zig`, so a
repo-wide flattened module map — the shape a workspace index invites — would
point `measure` at `packages/core`, a confident edge into a package `tool` does
not depend on. That is worse than the unresolved import this fixes, and only
per-package scoping keeps them apart.
Tests: 7 unit cases pinning the index (including `loadZigBuildConfig` answering
`null` on the same fixture, side by side) and 3 integration cases pinning the
EDGES — cross-package CALLS from the module root AND from a non-root file, and
`tool` not crossing over. Verified load-bearing: restricting the index to the
root package fails all three.
Second consumer caught by the integration test and fixed with it:
`populateZigWorkspaceStaticGating` reads the same `resolutionConfig`, per file
because two files of that pass can belong to different packages.
Gates: build clean, `tsc --noEmit` clean, prettier clean, eslint 0 errors.
`test/integration/resolvers` 3,768 passed / 4 skipped, `test/unit/scope-resolution`
2,015 passed. Bench `--check` with NO baseline edited: `receiver-resolution`
(whose corpus is `test/fixtures/lang-resolution`, where the fixture lands),
`zig-cross-file-resolution`, `value-ref-resolution`, `scope-capture` (15
languages), `import-target`, `callable-value-flow`, `scope-emission`.
* perf(zig): dequeue the package walk by head index, not `shift()`
`findZigPackageDirs` pushes children while it drains the queue, which keeps
the array in a mode where `Array.prototype.shift()` memmoves the whole
remainder rather than taking V8's left-trimming fast path — so the walk is
quadratic in the frontier, bounded only by `ZIG_SCAN_MAX_DIRS` (20,000).
Measured at that bound rather than estimated from the move count: 53 ms at
fan-out 4 and 81 ms at fan-out 20, against 0.8 ms with a head index — 66-106x,
and paid before a single config is read.
FIFO order is unchanged, and the package ordering never depended on the walk
anyway: `loadZigWorkspaceIndex` sorts by (dir length desc, localeCompare).
Memory is unchanged too — the entries were already retained by the pushes;
`shift()` only dropped the head.
`findTsconfigFiles` and `loadNodeWorkspacePackages` carry the same walk with
the same bound and are equally affected. Deliberately NOT fixed here: they are
pre-existing, outside this PR's diff, and reach TypeScript/Node import
resolution, which nothing in this change set covers. Filed separately.
Reported by gitnexus-check on
|
||
|
mengkaka
|
4154b63131
|
feat(indexing): add Objective-C semantic indexing support (#3179)
* docs: add Objective-C fork provider notes * feat(objective-c): add deterministic provider and grammar * feat(objective-c): finalize provider MVP * fix(objective-c): harden provider integration * fix(objective-c): normalize bare macro markers * docs(objective-c): integrate provider documentation * fix(objective-c): harden resolution and header classification * fix(objective-c): complete provider follow-ups * fix: address Objective-C review follow-ups * chore: format Objective-C grammar sources * fix(objective-c): harden review follow-ups * Address PR review feedback (#3179) Keep Objective-C chunking and macro recovery aligned with the grammar, and stop Community MEMBER_OF edges from leaking into symbol context. Co-authored-by: Cursor <cursoragent@cursor.com> * Address follow-up review on ObjC chunking and language fallback. Keep preprocessor directive text from changing file-scope brace depth, group real ivar nodes, skip header modifiers, and restore Rakefile/Gemfile detection through getLanguageFromFilename. Co-authored-by: Cursor <cursoragent@cursor.com> * Parse Objective-C headers with the objc grammar in embeddings. ensureAndParse and structural extraction now use the same content classifier as ingest, including method snippets from .h files, so Protocol/Category/Class chunks are not re-parsed as C++. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) Keep file-scope macro elision off C line splices and @interface/@protocol/@implementation bodies, and attach ivar attributes to the following instance variable when chunking. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(bench): rebaseline Objective-C CSV emit * feat(objective-c): add workspace resolution and linear emit benches Plain .h files are classified as C++, so the ObjC pass could not resolve #import of those headers. Load a C/C#-style workspace once per pass, and keep protocol-candidate USES linear. Refs #3179 Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) - Compare LadybugDB labels() as a scalar when excluding Community MEMBER_OF edges. - Walk superclass members, skip file-static C sibling defs, and ignore comments in ObjC header/macro scans. Note: pre-existing failure in objective-c-provider integration (worker-pool ready timeout) not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) Emit Objective-C declaration captures so compilation-unit siblings can share header/implementation bindings, and keep class vs protocol visibility groups distinct. Note: pre-existing failure in worker-pool startup (GITNEXUS_WORKER_READY_TIMEOUT_MS) not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3179) Emit every comma-separated property/ivar declarator, and count @interface after a multiline block comment closes so in-declaration macros stay intact. Note: pre-existing failure in worker-pool startup (GITNEXUS_WORKER_READY_TIMEOUT_MS) not addressed by this PR. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: ximengkai <ximengkai@soyoung.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Gergő Magyar
|
376ed3bb4a
|
perf(lock): probe this process's own start time once (#3222)
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
* perf(lock): probe this process's own start time once `acquireFileLock` stamps the owner file with the acquiring process's start time so a later reclaimer can tell a live owner from pid reuse. That value cannot change while we are running, but it was re-probed on every acquisition — and on Windows the probe is a `powershell.exe` spawn plus a `Get-CimInstance Win32_Process` WMI query, which is the single most expensive step in taking an uncontended lock. Add `readProcessStartTimeCached` and make it the default reader in `acquireFileLock` and `resolveWatchDeps`. Only this process's own pid is cached: - A foreign pid is always re-probed. That process can exit and its pid be reused, which is precisely what the stamp exists to detect. - A failed probe is not cached. `acquireFileLock` throws when the start time is empty, so caching one transient failure would leave the process unable to take a lock for the rest of its life. `readProcessStartTime` itself is unchanged and still probes every call, so the existing timezone-pinning regression test keeps exercising the real `ps` invocation instead of passing off a cached value. Behavior is otherwise identical: same probe, same string, same stamp. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
96132bd13a
|
perf(scope-resolution): stop re-scanning the ParsedFile store once per language (#3211)
* perf(scope-resolution): stop re-scanning the ParsedFile store once per language Scope resolution calls `loadParsedFilesForPaths` once per language, and every call walks every shard in the store. The skip decision needs the envelope's path listing, and that listing is only trustworthy after the payload digest has been checked -- so a pass that wants 50 Python files still opens and SHA-256s all 413 shards / 301MB of a TypeScript-dominated store to prove it can skip them. A pass wanting a SINGLE file costs 335ms. The cost scales with language count, not with the files that language has, so a polyglot repo pays it worst. `tryLoadV8Cache` now returns the listing it already parsed for that skip decision, and the store memoizes it per run. Later passes skip on the memoized listing without reopening the file. Measured on a 2234-file, 3-language repo, min-of-5: before python 411ms typescript 2872ms javascript 507ms = 3834ms after python 415ms typescript 2805ms javascript 248ms = 3484ms -350ms here, roughly -250ms per additional language elsewhere. The first pass is unchanged by construction -- it is what populates the memo. End-to-end the graph is byte-identical: 51,288 nodes / 163,094 edges / 2106 clusters / 759 flows on a true incremental run. Keyed on size+mtime as well as name. Shard names are content-addressed, so a name collision across different content should be impossible, but that invariant lives in the parse-cache keying rather than here and one stat per shard is a few ms against the hundreds this saves. The memo holds one store directory at a time, so a new repo in a long-lived MCP process drops the previous set instead of accumulating. The failure mode a listing memo introduces is a FALSE SKIP: a pass concludes a shard holds nothing it wants and those files silently never reach the graph -- an exit-0 wrong answer, not a crash. The new test walks four passes with disjoint wants over one store, plus a shard written after the memo is warm; it fails when the skip is forced. Also records the full scopeResolution breakdown in bench/. The headline is that `emit` is 7161ms of the 14.7s phase and ~21% of the edit loop, spread across a fan of passes with no hot inner loop -- so the win there is not running them for unchanged files, which is a design rather than a patch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Address PR review feedback (#3211) Strengthen the shard-listing memo test so a later miss asserts fs.open and v8.deserialize never run for the skipped shard. Key-set checks alone still passed if the memo never skipped. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
9bf307123e
|
feat: notify users when a newer gitnexus version is available (#3175)
* feat(core): add cache-first npm update-check service Shared fail-open checker: validated 24h cache under GITNEXUS_HOME, acquireFileLock-guarded refresh, monotonic publication, hardened registry fetch (no credentials, private-address redirects refused, body-capped), strict x.y.z comparator, install-eligibility classification, and an unref'd refresh scheduler for long-lived processes. Extracts getGlobalDir into storage/global-dir.ts with a repo-manager re-export (no caller changes). Co-authored-by: Cursor <cursoragent@cursor.com> * feat(cli): notify on available updates via stderr and doctor One i18n'd stderr line on interactive invocations when the validated cache holds a newer version (TTY-gated, CI/opt-out/eligibility-gated, hook and help/version command identities excluded). Stale cache spawns a detached hidden __update-check refresh child so command exit latency is unchanged. doctor prints the cached latest version when known. Dockerfile.cli sets GITNEXUS_NO_UPDATE_NOTIFIER=1. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(mcp): emit one stderr update notice per process per version Process-scoped adapter in mcpCommand (stdio and --http), dynamically imported after the stdout sentinel, started only after connect, fully catch-isolated. Arms the shared refresh scheduler with cleanup on process exit. Protocol payloads stay free of update state (R15). Co-authored-by: Cursor <cursoragent@cursor.com> * feat(serve): expose update state on /api/info Serve-scoped controller owns an in-memory update snapshot: one staleness evaluation after listen, then the shared unref'd scheduler, stopped on close/shutdown. /api/info reads only the snapshot and gains optional latestVersion/updateAvailable fields for eligible installs; the existing three fields are byte-compatible. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(web): dismissible update-available banner from /api/info Fetches server info after backend connect and on reconnect, renders a fixed banner in the exploring view only when updateAvailable is true and the version is undismissed, hides while the reconnect banner is active, and fails open on fetch errors. role=status + aria-live with a keyboard-focusable dismiss; dismissal persists per version in localStorage. Copy in en/zh-CN common.json with version interpolation. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(cli): document update notifications and opt-outs Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): apply review findings and simplify pass Review: gate the detached refresh spawn on a live lock-owner probe so parallel CLI invocations coalesce to one refresh child (validated P2, three-reviewer agreement); poll /api/info on a slow cadence while exploring so post-load server-side discoveries surface (validated P1); add a monotonic sequence guard so overlapping server-info fetches commit in order. Simplify (behavior-preserving): shared truthy-env/opt-out/freshness helpers in update-cache.ts, shared cachedUpdateNoticeLine for CLI and doctor, extracted install-eligibility core with per-process memo, memoized registry parsing, single evaluation per scheduler cycle, cache-only startup evaluate in serve, flattened MCP exit handler, shared bottom-banner shell, storage keys in ui-constants. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(update-notifier): address residual review tickets on this PR Stop the lock-busy 1ms scheduler spin, replace clock-skewed cache entries, move the outbound URL guard into core, and extract the serve update controller. Pin the startup/guard/single-flight/MCP/CLI contracts those tickets called out. Fixes #3167 #3168 #3169 #3170 #3171 #3172 #3173 #3174 Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) Fetch the npm /latest document instead of the full packument so the 64KiB cap can succeed, and treat reused lock PIDs as stale so refresh is not suppressed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) Register the CLI spawn suite on the OS matrix, pin MCP opt-out env, and compare versions without IEEE-754 rounding. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(cli): add gitnexus update install and versioned command banners Give an explicit Claude/Codex-style upgrade (`npm i -g gitnexus@version`) and print `GitNexus <Name> (version)` on every command so the running build is obvious without silent self-update. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) - Document the pinned install as npm i -g gitnexus@<x.y.z>, not a copyable @version tag - Wait for wall-clock-future cache repair to publish before asserting - Restore the stdout spy if the TTY notice assertions fail Co-authored-by: Cursor <cursoragent@cursor.com> * fix(update-notifier): keep last known latestVersion on a failed refresh A later offline check was wiping the pin and hiding a known update for 24h. gitnexus update still treats a failed live fetch as checkFailed. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#3175) - Word update.current so a newer-than-latest install is not called the latest stable version. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): hide the detached update-check spawn on Windows The refresh child was spawned without windowsHide, so Windows CI could stall before writing the cache and then fail cleanup with EBUSY. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Parafee41
|
47c2799ba7
|
fix(analyze): add cold parser rebuild option (#3153) | ||
|
Garrett Griffin
|
3c2b14aff5
|
feat(zig): mark CALLS edges inside comptime-false branches as staticGated (#3161)
* feat(zig): static-gating analysis module + fixture (ported from feat/zig-static-gated-edges-v2) Squashes c6fe922c, 2f3c8e9e, fab088f4, 9b58af74, aef2ae83, 86b892ef, d5657861, f3780b3a: file-local comptime bool constants, cross-file flag resolution via the @import alias map, re-aliased const chains, == / != against known bools, else / else-if branch awareness. The module is self-contained; the hooks that call it land in the next commit. * feat(zig): stamp static-gated call sites through the scope pipeline Wires the ported gating module into the scope-resolution pipeline that now emits every Zig CALLS edge (PR #1432), replacing the parse-worker / call-processor hooks of the original branch, which targeted the legacy DAG path the merged provider no longer uses. Data flow, one new fact carried end to end: emitZigScopeCaptures stamps `@reference.static-gated` on a call capture whose anchor lies in a statically dead range (body of `if (CONST_FALSE)`, else of `if (CONST_TRUE)`), via the module's new `collectZigStaticGatedRanges` (line/col ranges, because a Capture keeps no node) scope-extractor marker -> `ReferenceSite.staticGated` buildReference -> `Reference.staticGated` references-to-edges, -> `GraphRelationship.staticGated` on the free-call-fallback, emitted CALLS edge (both emit paths, plus edges.ts (tryEmitEdge*) the generic bridge) local-backend impact -> `staticGated` on impact frontier edges Same marker idiom as Go's `@reference.callee-position` / `embedded-pointer`: zero-range, present or absent, so every ungated site's capture set is byte-identical and no other language changes. SCHEMA_BUMP 92 -> 93: parse-time captures changed. Cross-file constants (`if (cfg.FOO)` with `cfg = @import("cfg.zig")`) are NOT stamped yet: the module resolves them through `lookupBoolsForPath`, but the capture emitter runs per file in the parse worker with only `{ path, content }`, so it cannot see the sibling source. The two positive cross-file cases in zig-static-gating.test.ts are `it.skip` with that reason; the negative ones pass unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG * feat(graph): add staticGated edge property Adds an optional `staticGated?: boolean` field to `GraphRelationship` that flags edges originating in code branches known at index time to be unreachable in production — e.g. `if (CONST_FALSE)` blocks where the condition reduces to a comptime-known `false`. Schema + persistence wiring: - `gitnexus-shared/src/graph/types.ts` — additive optional field on `GraphRelationship`; absent edges read identically to live ones. - `gitnexus/src/core/lbug/schema.ts` — `staticGated BOOLEAN` column on the `CodeRelation` REL table. - `gitnexus/src/core/lbug/csv-generator.ts` — appends a `staticGated` column (0/1) to the `relations.csv` written for bulk COPY ingest. - `gitnexus/src/core/lbug/lbug-adapter.ts` — fallback per-row `MATCH ... CREATE` insert reads the optional column and threads it into the relationship properties. No language has populated this field yet — the Zig hookup lands in the next commit. Existing DBs need a re-index for the new column to appear; existing readers are unchanged because the field is optional and absent on every other language's edges. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit c8cd5efe27a77a5d1b3f05e3f4e89069b5c6b19e) * fix(zig): gate bare literal branches; AND the flag over deduplicated free-call sites PR #3161 review, two findings: 1. `stampZigStaticGating` returned early when the file declared no boolean constants, but `collectZigStaticGatedRanges` also folds bare literals, so `if (false) { foo(); }` in a constant-free file went unstamped. The early return is gone; the range walk runs for every file. 2. `emitFreeCallFallback` deduplicates CALLS edges per (caller, callee) and wrote `staticGated` from whichever site it met first, so a callee reached from one live site and one dead site was gated or not by traversal order. Emission is now deferred to the end of each file's sites and the flag is the AND over every site that collapsed into the edge: one live site keeps the edge live. The other emit path keys its dedup on the site range and was not affected; `collapseByCallerTarget` in the generic bridge would have the same shape but no language that sets the marker opts into it. Fixture + tests: `gated_bare_literal`, `live_and_gated_same_callee` (live site first) and `gated_then_live_same_callee` (dead site first) in zig-static-gating.test.ts. All 70 resolver suites (3,603 tests) pass with the shared emitter change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ciQ3MTjpQXkCoNntZR9zG * test(zig): move the SCHEMA_BUMP pin to 93; rebaseline emit fingerprints for the staticGated column Three CI failures on |
||
|
Navid EMAD
|
932d937085
|
feat: add Zig language support (#1432)
* feat: add Zig language support
Adds a Zig LanguageProvider grounded in @tree-sitter-grammars/tree-sitter-zig 1.1.2.
The grammar's published peerOptional `tree-sitter@^0.22.1` is suppressed via an
npm `overrides` entry that aliases the peer to the bundled `tree-sitter@0.21.x`;
load-time smoke testing confirmed ABI compatibility.
v1 capabilities:
- .zig file detection + Prism syntax mapping
- Top-level + nested function_declaration as Function/Method
- struct/enum/union (anonymous in the grammar — owner name resolved from the
enclosing variable_declaration in class/field/method extractors)
- container_field as struct/union fields and enum variants
- top-level const/var as Variable nodes
- free + member call_expression as @call edges
- @import("./foo.zig") local-file resolution; std and external packages
return empty (no ghost edges)
- pub keyword detection for export checking
- no heritage hooks (Zig has no inheritance; queries never emit @heritage.*)
Generic extractor changes (backward-compatible):
- field-extractors/generic.ts and method-extractors/generic.ts: empty
`bodyNodeTypes` falls back to the type declaration node itself as its own
body container — needed because Zig's struct_declaration directly contains
its container_field children. (The `extractOwnerName` hook this commit
originally introduced now exists upstream; Zig just configures it.)
Out of scope (deferred):
- usingnamespace, build.zig.zon package graph, comptime/anytype
- scope-resolution hooks (emitScopeCaptures, interpretImport, …): Zig is
classified `experimental` and uses the generic fallback resolution path
- cross-package imports (std, deps)
Tests:
- new fixture test/fixtures/sample-code/simple.zig
- Zig describe block in tree-sitter-languages integration test
- simple.zig added to parsing.test.ts fixture-existence list
- Zig added to ingestion-utils detection unit test
- Zig smoke case in parser-loader-abi.test.ts
- Zig grammar registered in the grammar-literal validation gate
* feat(zig): integrate build.zig.zon resolution + Union label from PR 1096
Ports the additive pieces of grgisme's standalone Zig provider PR
(https://github.com/abhigyanpatwari/GitNexus/pull/1096) onto the rebased
provider:
- build.zig.zon `.path` dependency resolution for bare-name
@import("pkg") (parseZigBuildZon / loadZigBuildZon in
language-config.ts, resolveZigImportInternal in import-resolvers/zig.ts,
wired through ImportConfigs.zigBuildZon). `.url` deps and
repo-escaping paths return null cleanly. 13 unit tests.
- `union(enum)` containers now produce `Union` nodes (not Struct):
'Union' added to ClassLikeNodeLabel + CLASS_LIKE_LABELS, the Zig query
tags @definition.union, CONTAINER_TYPE_TO_LABEL maps union_declaration
to 'Union'. The label was already plumbed graph-wide on main.
- /^build$/ entry-point pattern (build.zig).
- zig-basic lang-resolution fixture + resolvers integration test.
Adapted to current main while porting:
- labelOverride relabels container-nested fns Function → Method
(mirrors isKotlinClassMethod); the structure phase no longer derives
Method from the legacy method-extraction path for plain
@definition.function captures.
- IMPORTS/CALLS edges require scope-resolution hooks
(emitScopeCaptures / interpretImport) since the legacy DAG removal;
Zig does not implement them yet, so the integration test documents
that with a skipped import-edge case. The resolver itself is wired
into the resolver factory and becomes live when the hooks land.
Not ported: named-bindings extractor (the legacy namedBindingExtractor
API no longer exists) and the bespoke field extractor (the generic
factory's extractOwnerName / empty-bodyNodeTypes hooks cover Zig).
Co-authored-by: Garrett Griffin-Morales <grgisme@gmail.com>
* feat(zig): scope-resolution hooks — IMPORTS and CALLS edges (Ring 3)
Implements the registry-primary scope-resolution path for Zig, the
prerequisite for cross-file edges since the legacy DAG removal. Adds the
standard per-language stack under languages/zig/:
- query.ts: scope query (containers as Class scopes, blocks, functions),
declarations (container anchors placed on the container node itself so
the def lands in its own Class scope and the name binding auto-hoists
to the parent — populateClassOwnedMembers needs the class-like def
among the class scope's ownedDefs), @import statements (#eq?-gated
builtin), parameter/constructor type bindings, and call/constructor
reference sites. The grammar is required lazily (optionalDependency).
- captures.ts: emitZigScopeCaptures — groups query matches, drops the
plain-variable group for container/import bindings (their dedicated
rules bind the name), and relabels container-nested fns
@declaration.function → @declaration.method (labelOverride parity).
- interpret.ts: namespace-kind imports (const x = @import("…")) and
type bindings — self-parameter convention marks the receiver, Zig
sigils (*, ?, [], error unions, const) stripped from type names while
dotted qualifiers (mod.T) are preserved for Case-3 namespace-prefix
receiver dispatch.
- simple-hooks.ts: parameter bindings stay function-local (Go
rationale), local-over-import merge precedence, bounds-check arity
(always 'unknown' today — no synthesized arity metadata).
- scope-resolver.ts: emit-side wiring; build.zig.zon threads through
loadResolutionConfig into the same resolveZigImportInternal the legacy
resolver config wraps. fieldFallbackOnMethodLookup off (statically
typed). Registered in SCOPE_RESOLVERS.
The resolvers integration test un-skips the import-edge case and gains
CALLS assertions: free call (main → helper) and receiver-bound method
dispatch through a namespace-qualified constructor
(var p = pioneer.Pioneer{…}; p.tick() → main → tick).
* fix(zig): Union is class-like + missing-grammar warning (review pass)
Self-review findings on the Zig branch:
- scope/walkers.ts `isClassLike` and finalize-algorithm's
CALLABLE_OR_TYPE_LIKE did not include 'Union': a `union(enum)`
container's methods got no ownerId from populateClassOwnedMembers, so
method dispatch on union receivers silently dropped. Widened both
sets; the zig-basic fixture gains a Tag method + a CALLS assertion
(main → isEnergy) that fails without the widening (verified by
reverting).
- optional-grammars.ts now lists tree-sitter-zig with an npm `probe`
(it is an optionalDependency, not vendored): users with .zig files
and no prebuild get the standard one-line stderr warning instead of
a silently degraded index.
- Deduplicated the container-method predicate: `isZigContainerMethod`
+ ZIG_CONTAINER_TYPES now live once in languages/zig/captures.ts and
feed both the provider labelOverride and the scope-capture relabel.
- README language matrices: Zig row now claims Type Annotations,
Constructor Inference, and Config (build.zig.zon) — all true since
the scope-resolution hooks landed.
* fix(zig): anchor @declaration.variable to the binding identifier
`(variable_declaration (identifier) @declaration.name)` matched EVERY
identifier child of the node, so `const first = target;` also declared a
phantom local named `target` in the enclosing block. That phantom shadowed
the real function for later references (and starved callable-value-flow
seeds of a target). The `.` anchor pins the pattern to the first named
child — the bound name.
Regression test in resolvers/zig.test.ts pins the capture set.
* feat(zig): callable-value-flow captures + main's per-language conformance gates
Post-rebase catch-up: since this branch forked, main added three "every
registered language must appear here" tests. Each needs a Zig entry:
- callable-value-flow (#2522): Zig now emits `@callable-flow.*` facts via
`synthesizeCallableFlowCaptures` (ZIG_CALLABLE_CAPTURE_OPTIONS in
zig/captures.ts). tree-sitter-zig's `call_expression` carries arguments
as direct children with no wrapper node, which the shared helper could
not decompose, so this adds a language-neutral `extractCallArguments`
hook (mirror of `extractFunctionParameters`; `undefined` = shared path).
Zig joins the provider matrix as 'matrix' with a real assign→copy→
argument→invoke case.
- external-import-conformance (#2953): `@import("std")` beside a decoy
`src/std.zig` resolves to nothing; the decoy stays reachable via the
relative spelling. Zig holds the property (no suffix fallback), so it is
a case, not a KNOWN_GAPS entry.
- import-target-index-reuse contract (#2909): membership-probe-only
fixture (minimumScans: 0, same shape as Rust).
* fix(zig): address gitnexus-check review findings
One commit per the bot's list so each item is easy to check off:
- parser-loader: Zig row gains `userSkippable: true`, so
`GITNEXUS_SKIP_OPTIONAL_GRAMMARS` (=1 or a list naming `zig`) disables it
at analyze time like swift/dart/kotlin — as `optional-grammars.ts` already
documented. Covered in parser-loader-skip-optional.test.ts.
- export-detection: `zigExportChecker` stops at the first declaration it
reaches. A non-`pub` fn inside `pub const T = struct {…}` was reported
exported because the walk continued up to the wrapper.
- import-resolvers/zig: a `.path = "."` dep normalizes to '' and no longer
grows a leading slash (`/src/main.zig` could never match).
- language-config: build.zig.zon parsing strips `//` comments string-aware
(a `//` inside `.url = "https://…"` survives) and matches braces while
skipping string literals, so a commented-out `.path` cannot declare a dep
and a `}` in a comment/string cannot truncate the block.
- method-extractors/configs/zig: the leading `self` receiver is excluded
from `parameters` (Rust parity). Fixing that exposed a worse bug: the
`parameters` node is a plain child of `function_declaration`, not a
`parameters:` field, so `childForFieldName('parameters')` was always null
and every Zig method had no parameters, no receiver and `isStatic: true`.
One `zigParameterList` helper now feeds all three readers.
- variable-extractors/configs/zig: container (`struct`/`enum`/`union`) and
`@import` bindings are skipped via the same predicate the scope captures
use (`isZigContainerOrImportBinding`), instead of the comment merely
claiming they were.
- tree-sitter-languages.test.ts: the "missing grammar" case now forces the
absent-binding path through the loader's runtime opt-out on a fresh module
instead of passing vacuously when the package is installed.
New: test/unit/zig-extractors.test.ts (exports, receiver/parameters,
variable guard); zig-import-resolver.test.ts gains the `.` dep, comment and
brace cases.
The `createFieldExtractor` heads-up needs no change: the added branch is
unreachable for every existing config (none has empty `bodyNodeTypes`).
* fix(zig): address second gitnexus-check review pass
- import-resolvers/zig: `..` above the repository root now returns null
instead of aliasing a same-named root file (`../bar.zig` from `main.zig`
is not `bar.zig`); the stale "extension is stripped and re-added" comment
is corrected to what the code does.
- variable-extractors/configs/zig `extractType`: read the `type:` field only.
The positional fallback returned the INITIALIZER of `const f = target;`
as its type and gave up on compound annotations (`*Foo`, `?[]const u8`).
The comment claiming 1.1.2 has no `type` field on variable_declaration
was wrong (verified by AST dump) — and it is what led the review to
suspect the callable-flow `extractAssignment` callback, which was
already correct for `extern var f: T;`.
- receiver detection: only a FIRST parameter named `self` is the receiver.
`emitZigScopeCaptures` tags first-position parameters
(`@type-binding.first-parameter`), `interpretZigTypeBinding` requires the
tag as well as the name, so `zigReceiverBinding` no longer turns
`fn f(a: u32, self: T)` into an instance method.
- resolvers/zig.test.ts: both suites `describe.skipIf(!zigAvailable)`
(Swift/Dart pattern) — the grammar is an optionalDependency.
- tree-sitter-languages.test.ts: the Zig parsing case gates on
`isLanguageAvailable` instead of a catch-all `return`, so an installed
grammar that fails to load fails the test; comment no longer calls Dart
and Swift npm optionalDependencies (they are vendored).
- test/helpers/literal-collectors: `DIR_LANG` gains `zig`, so literals under
`languages/zig/**` are validated against the Zig grammar alone rather than
against every grammar.
- walkers.ts `isShapeLike` doc: Union IS included (via isClassLike, wired by
Zig's union member container); Typedef remains the only deferred one.
- language-config `ZigBuildZonConfig.pathDeps` doc: values are the raw
`.path` strings; the resolver normalizes.
Tests: zig-import-resolver (+1), zig-extractors (+3).
* fix(zig): address third gitnexus-check review pass
- language-config `parseZigBuildZon`: the `.dependencies = .{` header and
the per-entry `.<name> = .{` headers are matched only OUTSIDE string
literals (per-offset string mask + `matchZonHeader`). A `.name` or
`.description` value spelling `.dependencies = .{ .fake = .{ .path = … } }`
used to be taken as the block and returned the fake dep instead of the
real top-level one.
- import-resolvers/zig: an absolute import (`@import("/foo.zig")`) returns
null. The path walker skipped every empty component, so the leading `/`
vanished and `/foo.zig` resolved as importer-relative `src/foo.zig` — an
in-repo edge for an import Zig rejects as outside the module path.
- tree-sitter-languages.test.ts: the Zig parsing case gates on the PACKAGE
being installed (`createRequire().resolve`, minus a deliberate
`GITNEXUS_SKIP_OPTIONAL_GRAMMARS` opt-out), not on `isLanguageAvailable`,
which is false for absent AND for installed-but-broken bindings — so a
load failure (ABI mismatch, bad export) now fails the test instead of
skipping it, as the comment already claimed.
- walkers.ts `isShapeLike` doc: `Union` sits in `isClassLike` because that
is the label set the ownership walkers consult, not because unions
inherit — Zig has no inheritance and no heritage hooks. The previous
wording ("inheritance-capable owner") said otherwise.
- Not re-fixed (already addressed in the second pass, findings carried
over): "receiver = any parameter named self" — `interpretZigTypeBinding`
only sources a first-position parameter as `self`; `zigReceiverBinding`'s
doc now states that invariant. "`DIR_LANG` has no zig entry" — it does.
Extended one level out: `BASENAME_LANGS` (`zig.ts`) and `PREFIX_LANGS`
(`ZIG_`) map to the Zig grammar too, so extractor configs and the
export-detection set are validated against Zig alone. That immediately
caught a dead `childForFieldName('parameters')` in
method-extractors/configs/zig `zigParameterList` (there is no such field;
the named-child lookup was already the one doing the work) — removed.
Tests: zig-import-resolver (+2: absolute path, header inside a string);
both fail on the previous code.
* fix(zig): address fourth gitnexus-check review pass
- language-config: `parseZigBuildZon` only accepts a `.path` that is a
DIRECT field of a dependency entry. Nested blocks inside the entry body
are blanked (string-aware, offsets preserved) before the `.path` regex
runs, and a match starting inside a string literal is rejected, so
`.dep = .{ .url = "…", .meta = .{ .path = "x" } }` no longer becomes a
path dep. Regression test in zig-import-resolver.test.ts (fails on the
previous code).
- tree-sitter-languages test: the "grammar is absent" case now drives the
loader's real `source.load()` catch branch — `node:module` is stood in
with a `createRequire` whose require throws MODULE_NOT_FOUND for
`@tree-sitter-grammars/tree-sitter-zig` and delegates everything else —
instead of the `GITNEXUS_SKIP_OPTIONAL_GRAMMARS` opt-out, which has its
own test. It also asserts the opt-out flag is NOT set and that other
grammars still load (non-fatal optional failure).
Not re-fixed:
- "Return type is read from the wrong tree-sitter field": tree-sitter-zig
1.1.2 has NO `return_type` field on function_declaration — the type after
`)` is the `type` field (AST dump: `builtin_type "i32" field=type`; the
proposed `childForFieldName('return_type')` is null for every function).
A pin test in zig-extractors.test.ts asserts both the grammar fact and
that `returnType` is extracted (`void`, `!*Counter`).
- "`DIR_LANG` has no zig entry": it does (added in the first pass and
answered again in the third); the finding is carried over unchanged.
* fix(zig): address fifth gitnexus-check review pass
- tree-sitter-languages test: the Zig "functions, structs, enums, and
imports" case now asserts the `import.source` capture for
`const std = @import("std");` (the fixture's only import), so a query
change that drops Zig import matching fails it instead of passing
unchanged.
Not re-fixed:
- "Return type is read from the wrong tree-sitter field": carried over from
the fourth pass unchanged. tree-sitter-zig 1.1.2 has no `return_type`
field on function_declaration; the return type IS the `type` field, and
the pin test added in the fourth-pass commit
(zig-extractors.test.ts, `childForFieldName('return_type')` is null,
`returnType` = `void` / `!*Counter`) proves it.
- "`DIR_LANG` has no zig entry": carried over unchanged for the third time;
the entry exists since the second-pass commit.
* feat(zig): export fn visibility, opaque containers, named test blocks, member ownership
Ports the parts of upstream PR #305 (closed, unmerged) that our Zig
provider lacked, plus two gaps found while porting.
- `export fn` / `export var` (C-ABI linkage, never `pub`) are exported;
the pub/export predicate is now shared by the export checker and the
method/variable extractors' visibility (`hasZigVisibilityKeyword`).
- `const H = opaque { … }` is a Struct-labelled container (it may own
methods, never fields) in both the structure queries and the scope
query; ZIG_CONTAINER_TYPES is the single source for the extractor
configs.
- `test "name" { … }` blocks are Function nodes named by the string
node WITH quotes, so `test "add"` beside `fn add` cannot merge onto
Function:<file>:add; `test_declaration` joins FUNCTION_NODE_TYPES and
the Zig method config names it in the enclosing-function walk, so
calls inside a test attribute to the test. Anonymous `test {}` and
decl-tests `test add {}` are scopes without a node (an empty-name hook
result stops the walk instead of falling through to the identifier of
the function under test).
- Empty container bodies (`struct {}`, `opaque {}`) no longer mint a
nameless Property: tree-sitter-zig 1.1.2 recovers them as a
container_field with a MISSING identifier; #not-eq? guards in both
queries and the field extractor drop it.
- Owner walk (`findEnclosingClassInfo`): an anonymous container bound
by the enclosing `variable_declaration` takes the binding identifier,
same shape as the Go `type_spec` branch. Before this NO Zig member
had an owner — zero HAS_METHOD / HAS_PROPERTY edges for Zig.
Not ported from #305, deliberately: `builtInNames` (a bare-name call-site
drop filter; `alloc`/`free`/`append`/`print` are the most common user
method names in Zig and `std.*` receivers are already external via the
import binding), `usingnamespace` (removed in Zig 0.15), `@cImport`,
`build.zig` ignore, and the web-app changes.
* test(zig): absent-grammar case owns GITNEXUS_SKIP_OPTIONAL_GRAMMARS
The loader parses the opt-out variable lazily once per module copy, so
under `GITNEXUS_SKIP_OPTIONAL_GRAMMARS=zig` (or `all`) — a supported way
to run — the fresh loader took the opt-out branch and the "not the
opt-out path" assertion failed before the absent-binding path ran.
Clear the variable for the fresh module and restore it in `finally`,
instead of returning early: the branch stays exercised in every
environment. Verified: fails on the previous code under `=zig`, passes
with and without the variable now.
* fix(zig): declare the Union relation pairs — analyze aborted on any union
`gitnexus analyze` exited 1 on every Zig repository that declares a
`union` (including the zig-basic fixture itself): the member-ownership
commit made `union_declaration` a MEMBER_OWNER, so HAS_PROPERTY /
HAS_METHOD edges are emitted FROM a Union node, but `Union` was not in
LINKABLE_LABELS, so the schema's scope-bridge cross product never
generated a `FROM Union` pair and LadybugDB rejected the edge
(`labelPair: "Union|Property"`). Resolver tests stayed green because
they never write to the DB.
- `Union` joins LINKABLE_LABELS (also bridges `Tag{…}` constructor
references); the three hand-written `→ Union` target pairs move to the
generated half, per the STRUCTURAL_PAIR_DDL rule.
- structural-pair-coverage gains an optional-grammar corpus with
zig-basic (`Union|Property`, `Union|Method` sentinels), skipped when
the grammar is absent.
- Rust `union_item` note updated: the three gates it cited are widened.
Note for reviewers: the DDL fingerprint changes (#2808), so existing
indexes are rebuilt on next analyze.
* feat(zig): resolve path deps through the dep's build.zig and src/root.zig
The bare-name resolver only knew `src/<name>.zig` and `src/main.zig`.
`zig init` has written `src/root.zig` for libraries since 0.12, so the
default library layout never resolved. Now: the root the dep's own
build.zig declares (`b.addModule("<name>", .{ .root_source_file =
b.path("…") })`, name-matched module first), then src/root.zig,
src/<name>.zig, src/main.zig. `normalizeZigDepPath` is shared by the
loader and the resolver.
* feat(zig): Const/Variable defs, member imports, receiver typing, generic type constructors
Coverage gaps found by indexing idiomatic Zig against the branch:
- Const / Variable nodes: ZIG_QUERIES had no @definition.const /
@definition.variable, so `pub const VERSION`, error sets and type
aliases were absent and zigVariableConfig never ran. Rules are gated on
the literal `const` / `var` keyword — tree-sitter-zig 1.1.2 parses
statement assignments (`x = 5;`, `x += 1;`, `_ = expr;`) as keyword-
less `variable_declaration`s, and the scope query minted a phantom
local per assignment and one `_` per discard. Container and @import
bindings are skipped via `shouldSkipDefinitionCapture`.
- Imports: `const X = @import("x.zig").X` (named / alias), `const X =
ns.X` where `ns` is an @import binding of the file (promoted to a
named import), and `pub usingnamespace @import(...)` (wildcard, with
`expandsWildcardTo`). All three lost the file-level IMPORTS edge.
- Receiver typing: `var x: T = undefined` / decl literals `const x: T =
.init()` (annotation), `var c = T.init()` / `mod.T.init()` (call
return), `List(u8){}` (instantiation literal); `normalizeZigTypeName`
drops the comptime argument list.
- Generic type constructors `fn List(comptime T: type) type { return
struct {…}; }`: the returned container is a Struct/Union/Enum named
after the fn, owns its members (HAS_METHOD / HAS_PROPERTY), binds in
the module scope beside the Function def, and is emitted ahead of it
so a named import binds the type.
- `export` vs `pub`: `visibility` is now `pub`-only (Zig-module fact);
`isExported` keeps `pub|export` (visible outside the unit, as C's
external linkage). `export fn` without `pub` is not reachable from
other Zig files.
- Extractor configs share `zigContainerName`; ast-helpers' owner walk
learns the type-constructor shape.
Tests: zig-idioms fixture (10 resolver cases), extractor/interpret unit
cases for each rule.
* fix(zig): address sixth gitnexus-check review pass
- Windows absolute `.path` deps (`C:\x`, `C:/x`) return null from
`normalizeZigDepPath` like POSIX ones; a `/`-only check let them
through as repo-relative.
- `parseZigBuildZon` accepts the `.dependencies = .{` header only at
brace depth 1 (a direct field of the file's `.{`), so a same-named
field nested in an earlier struct cannot hijack the block.
- `importsExecuteWhereWritten: false` on the provider: `@import` is
compile-time name lookup (as C `#include`, Rust `use`); a body-level
`@import` is no longer marked `runsOnlyWhenCalled`.
- Namespace imports record the MODULE as `importedName`
(`zigModuleNameOf`: last path segment without `.zig`), per the shared
contract; the local handle stays `localName`.
- Keyword-less `<ident> = @import(…)` (`_ = @import("x.zig")` in a test
block) is a `side-effect` import: file edge, no binding. Only
`const`/`var` declarations bind a name or feed alias promotion.
- `extractZigFunctionName` doc: an empty name is falsy, so the enclosing-
function walk skips the test node and continues to the File; it does
not "end" there.
Not re-fixed: "DIR_LANG has no zig entry" — carried over for the fourth
pass in a row; `test/helpers/literal-collectors.ts` has had `zig` in
`DIR_LANG` (line 91) and `BASENAME_LANGS` since the second-pass commit.
Regression tests: absolute-path spellings, nested `.dependencies`
decoy, namespace/side-effect interpretation, function-scoped `@import`
not deferred (all four fail on the previous source).
* fix(zig): address seventh gitnexus-check review pass
- The scope query's `@import` binding rules are keyword-gated (`"const"` /
`"var"`, first-child anchored) like every other binding rule, and the
keyword-less `<ident> = @import(…)` statement has its own
`@import.side-effect` rule. Tree-sitter queries cannot express "no
keyword child", so that rule also matches the keyword shapes and
`emitZigScopeCaptures` drops those (they are the binding rules'
matches). Behaviour is unchanged from the sixth-pass fix — the existing
side-effect test covers it — the query text now carries the guard the
finding asked for.
Not re-fixed: "DIR_LANG has no zig entry" — fifth pass in a row;
`test/helpers/literal-collectors.ts` has had `zig` in `DIR_LANG` since
the second-pass commit. Left for a human reviewer to close.
* fix(zig): resolve @import of the repo's own build.zig modules (F3)
Bare-name imports were resolved through build.zig.zon path deps only, so
the module a repo's ROOT build.zig declares for itself —
`b.addModule("lightpanda", .{ .root_source_file = b.path("src/lightpanda.zig") })`,
imported by name from 378/567 Lightpanda files — never produced an IMPORTS
edge, and nothing reached through `lp.X` resolved. A repo with a build.zig
but no build.zig.zon got no resolution config at all.
- language-config: `parseZigRootModules` (static scan of the root build.zig:
`addModule("<name>", …root_source_file = b.path("<p>.zig")…)`, and
`createModule`/`addModule` bindings named via `addImport("<name>", m)` or
`.imports = &.{ .{ .name, .module = m } }`; generated / `.url` / computed
modules are skipped) → `ZigBuildZonConfig.rootModules`.
- `loadZigBuildZon` → `loadZigBuildConfig`: reads the zon AND the root
build.zig; null only when neither contributes.
- resolver: root modules are consulted before path deps; std/builtin/root
still never resolve.
- fixtures: zig-idioms gains a Lightpanda-shaped root module (+ decoy
`addOptions().createModule()`); new zig-rootmodule (build.zig, no zon).
Corpus (Lightpanda): IMPORTS 3014→3389 (378 edges to src/lightpanda.zig,
was 0), CALLS 13885→13989, ns.f() 79.3%→83.3%,
param.m() type=ns-qualified 43→46/417.
* fix(zig): import every @import in expression position; resolve @import("x").f()
Both query sets only saw `@import` as the value of a const/var or under
`usingnamespace`, so an @import in any other position produced no file
edge: Lightpanda's `pub const Interfaces = .{ @import("a.zig"), … }`
registration table (288 modules), call arguments
(`CounterEnum("size", @import("ArenaPool.zig").BucketSize)`), comparison
operands (`JsApi == @import("x.zig").JsApi`) and member-call receivers
(`try @import("dump.zig").root(...)`) — 417 of 3,401 in-repo import pairs
had no IMPORTS edge, and the 80 inline-receiver calls resolved 0 times.
Scope query: a catch-all `@import.inline` rule matches every `@import`
builtin; `emitZigScopeCaptures` drops the ones a binding rule (or the
keyword-less side-effect rule) already claimed (by string-node id) so a
bound import is never doubled, emits the rest as side-effect imports once
per distinct source per file, and binds a member-call receiver as a
namespace import whose local name is the builtin's own text — the
`@reference.receiver` text on that call is identical, so the shared
namespace-receiver lookup (Case 1) resolves the member in the imported
module.
ZIG_QUERIES: the three variable_declaration/usingnamespace-anchored
`@import` rules collapse into the same single builtin rule (the structure
phase only skips import matches; one match per builtin keeps
tree-sitter-languages' exact-capture assertion intact).
Lightpanda corpus (zig-corpus-check, before → after): IMPORTS 3014 → 3426,
in-repo pairs missing 417 → 5 (4 under a default-ignored `cache/` dir, 1 a
commented-out import the census regex counts), `@import(..).f()` 0/80 →
72/80 (the 8 left are `@import("root")` and non-import builtins the census
mislabels), CALLS 13885 → 13959; every other line unchanged.
* feat(zig): model file-structs — a file with top-level fields is a Struct named after the file
In Zig every file is a struct; one that declares top-level fields is an
instantiable type whose name is the file stem (`Page.zig` declares `Page`,
`@typeName` agrees), and its top-level `fn`s taking `self` are its methods.
Lightpanda spells 413 of 567 files this way and, before this, `page.getArena()`
on a `page: *Page` parameter resolved 23 of 993 times (2.3 %) — `impact` on
`Page.getArena` reported 0 callers for 159 call sites, and 2,395 top-level
fields were ownerless Property nodes.
Definition phase: `((source_file (container_field …)) @definition.struct)` +
the class extractor names it from the file path (`zigContainerName(source_file,
filePath)`); top-level fns/fields are owned through the new
`LanguageProvider.resolveFileTypeOwner` hook (consulted by
`findEnclosingClassInfo` when the walk reaches the tree root, and by the
method/field extractors' owner lookup) — ids become `Method:<file>:Page.getArena#0`
/ `Property:<file>:Page.session` with HAS_METHOD / HAS_PROPERTY edges.
Scope phase: `emitZigScopeCaptures` emits a Class scope over the whole file
(same range as the Module scope, nested under it — the pair `canParentScope`
already admits) plus a Struct def anchored on it; member NAME bindings are
hoisted back to the Module scope by `zigBindingScopeFor` so `Page.init()`
(namespace member) keeps working, while ownedDefs stay in the Class scope so
`populateClassOwnedMembers` stamps the owner. The file-level `const Page =
@This();` alias no longer mints a Const (it would shadow the Struct); `@This()`
aliases in type position (`self: *SigHandler` in Sighandler.zig, nested
`Self`) are rewritten to the container name so receivers resolve. A namespace
import of a `.zig` file gets a NAMED twin of the file stem so `x: *Page` in the
importer binds the type as well as the module.
Shared, additive: `resolveFileTypeOwner` hook; `filePath` threaded to
`ClassExtractionConfig.extractName` / `extractOwnerName`; nameless
`definition.struct` passes `getLabelFromCaptures` like `definition.class`
already did (extractor synthesizes the name).
Lightpanda corpus (zig-corpus-check): param receivers typed by a file import
23/993 → 993/993; `self.m()` 99.2 → 100 %; annotated locals 11.6 → 23.2 %;
CALLS 13,885 → 15,689; HAS_METHOD 1,477 → 8,003; ownerless Property 2,395 →
276; Function/Method 8,378/1,518 → 2,562/7,330; no row regressed.
Fixture `zig-filestruct` (Page/Session/Sighandler/util) + unit tests pin the
shape, the stem naming, the alias rewrite, the namespace twin and the
unchanged namespace-file behaviour.
* test(zig): expression-position import case sees the file-struct type twin
* fix(zig): stop reading member calls `x.f(arg)` as direct calls named `f`
tree-sitter-zig spells `field_expression` as `object:`/`member:`; the shared
callable-flow reader only knew `property`/`field`/`method`, so every Zig
member call collapsed to a DIRECT call named after the member and the
solver fanned each argument out to every same-named callable
(4,761 cap warnings on Lightpanda, `Global.deinit -> Global.deinit`
self-loops through `pub const release = deinit;`).
Shared (grammar-neutral, receiver-gated):
- `memberParts` also reads `member` (only C/C++ `offsetof_expression` and
JS `class_body` expose that field, without a receiver field).
- A member call is a field-stored-callable invoke only when a MEMBER store
(`o->run = handler`, `self.f = target`) or a declared callable-typed field
is visible — a same-named plain binding no longer gates it.
- `direct-callee-name` requires a direct designator: `.init(x)`,
`' '.join(x)`, `string.Join(x)` name no callee to seed by simple name.
Zig: formals are numbered without the leading `self`, so `r.run(target)`
joins `cb` and yields `Runner.run -> target`.
Goldens for python/csharp regenerated: the only drift is the dropped
`direct-callee-name` on `' '.join(...)`, `text.strip().ljust(...)`,
`string.Join(...)`.
Lightpanda: cap-warnings 4761 -> 2, cvf self-loops 10 -> 0,
CALLS 13885 -> 13857 (28 removed, all callable-value-flow: 10 self-loops,
17 same-name fan-out, 1 lost `on -> TypeErased.start`; +1 correct
`Arena.alloc -> allocator`).
* fix(zig): bind container field types so `self.field.m()` resolves (F5)
A container's field types were never bound on its Class scope: the scope
query's `container_field` rule captured only the name, and
`emitZigScopeCaptures` synthesized no `@type-binding.field` group. The
compound resolver reads member types from that scope
(`typeOfMemberOnClass` → `classScope.typeBindings.get(field)`), so
`self.session.name()`, `self.counter.incr()` — Lightpanda's dominant
cross-object call shape — resolved 9 of 2803 times (0.3 %).
- query.ts: capture `type: (_)? @declaration.field-type` on
`container_field` (enum variants have none).
- captures.ts: per typed field, push a `@type-binding.field` group (name =
field, type = the type text, `@This()` aliases rewritten to the container
name like parameter types); anonymous inline containers are skipped. The
binding lands on the container's Class scope — the file's Class scope for
a file-struct — since `zigBindingScopeFor` hoists only declaration names.
- query.ts/captures.ts: `const page = self.page;` / `var s = self.session;`
one-level field aliases become `@type-binding.alias` bindings whose "type"
is the RHS path; the resolver's member-alias branch re-resolves it as a
receiver chain. Import aliases (`const Counter = counter.Counter;`) are
dropped — they are named imports.
- interpret.ts: `@type-binding.field` → 'annotation',
`@type-binding.alias` → 'assignment-inferred' (an annotation on the same
binding wins).
Corpus (Lightpanda, zig-corpus-check): self.field.m() 11/2803 (0.4 %) →
1433/2803 (51.1 %); ident.m() bound=local-field-access 29/1902 (1.5 %) →
307/1902 (16.1 %); chain.m() 146/5022 (2.9 %) → 642/5022 (12.8 %);
CALLS 15838 → 18066. self.m() / free f() / ns.f() unchanged.
Tests: unit (zig-extractors) — one @type-binding.field per typed field with
sigils stripped and aliases rewritten; the binding hosted on the container's
Class scope (file-struct: the file's Class scope, not Module) with the
written spelling as declaredSpelling; field aliases bound to the RHS path
and never for import aliases. Integration (zig-idioms `holder.zig`,
zig-filestruct `Page.zig`): `viaField → incr` ×2 into counter.zig,
`viaAlias → get/twice`, `sessionName → name` / `sessionLabel → name` into
Session.zig. All fail without the change. The optional-payload capture
`if (self.opt) |c| c.incr()` is not asserted (F6).
* fix(zig): `pub const X = @import(…)` at file scope republishes X (reexportsName)
Lightpanda's `lightpanda.zig` is one long list of `pub const Arena =
@import("Arena.zig");`, and most files name their types through it (`const
lp = @import("lightpanda"); const Arena = lp.Arena;`, `arena: *lp.Arena`).
The scope side treated those bindings as plain imports of the hub file, so
the hub never published the names it re-exports and a third file's `const
Arena = lp.Arena;` (promoted to a named import of `Arena` from the hub) found
nothing.
`emitZigScopeCaptures` now marks named/alias import groups whose declaration
is a file-level `pub const` — the `@import(...).X` form, the alias promotion
`pub const Bar = ns.Bar`, and the file-struct type twin of `pub const Arena =
@import("Arena.zig")` — and `interpretZigImport` sets the shared contract's
`reexportsName: true` on them (the Python `__init__.py` shape, consumed by
`buildReexportClosures`). Private and fn-local bindings stay unflagged.
Not covered here: a receiver ANNOTATED with the dotted hub path (`arena:
*lp.Arena`) — Case 3 of the receiver-bound pass looks the member up with
`findExportedDef`, which only sees locally declared names; following
re-exports there is a shared change left for a follow-up.
* fix(zig): type receivers through `const X = <type expr>;` aliases (F7)
`const LocalAlias = Local;`, `const T2 = Thing;` (alias of an alias/import)
and `const B = util.List(u8);` (an INSTANTIATED generic type constructor)
were plain `@declaration.variable` bindings, so `LocalAlias.mk()`,
`var l = LocalAlias.mk(); l.go()`, `T2.make()`, `B.init()`, `B{}` and
`var x: B` all typed nothing (review repro r3-flow b1..b9; Lightpanda:
`pub const Proto = HtmlElement;` x68, `const Allocator = std.mem.Allocator`
x104, `pub const KeyIterator = GenericIterator(...)`, fn-local
`const R = ...(...)`).
Model: a `@type-binding.alias` binding of the alias NAME to the value's type
text — Rust's `let x = y` / JS's `const B = Foo`, source
'assignment-inferred' — NOT a TypeAlias def. Reasons: (1) the shared
machinery already chains typeBindings (`followChainedRef` in the extractor,
`followChainPostFinalize` after propagation), so `var l = LocalAlias.mk()`
and `var x: B` reach the target through the alias with no new shared code;
(2) nothing shared follows a `TypeAlias` def to its target — `isShapeLike`
only makes the alias itself a member owner (TS object-type aliases) — so a
relabel would have needed language-named shared code; (3) graph node ids
are UNCHANGED: every alias stays `Const:<file>:X`. `normalizeZigTypeName`
already drops the comptime arguments, so `util.List(u8)` binds `util.List`
and resolves through the namespace import (Case 3). The identifier /
member shapes also take `var` (`var cur = orig; cur.go()` — the cursor
idiom, same binding as Rust's `let x = y`).
Heuristic, stated as such: a CALL value is kept only when the callee's last
identifier is TitleCase (Zig's naming convention for types), because the
grammar cannot tell `util.List(u8)` from `util.makeThing()` and the latter
belongs to the call-return rules; the call-return group is dropped for the
same TitleCase shape so the two never race on match order. Import bindings
(`const Stack = @import("x.zig").Stack`), promoted namespace-member aliases,
enum/decl literals (`.foo`) and the `type:` annotation of
`var b: T = undefined;` are excluded.
Not done: the two-hop `pub const bridge = js.Bridge(T); bridge.accessor()`
chain. `js.Bridge` is a Function that RETURNS `bridge.Builder(T)` (a call,
not a container), so the alias binds `js.Bridge`, Case 3 finds a Function
with no members in js.zig, and Case 3b is skipped for a namespace head.
Following that hop needs a namespace-member return-type route in shared
code (or a Zig `resolveQualifiedReceiverMember` hook that re-implements
member lookup without the model); left for a follow-up.
Corpus (Lightpanda, `harness/zig-corpus-check.mjs`): CALLS 15838 -> 16051
(+213, 0 removed), `ident.m() bound=local-alias` 6/63 -> 36/63,
`local-call` 159 -> 176, `module/unknown` 110 -> 116, `local-other`
271 -> 273; `self.m()`, `free f()`, `ns.f()` unchanged or up.
* fix(zig): one alias rule set — F7's alias rules subsume F5's field-access alias rules
* fix(zig): type locals through try/catch/orelse, return types and payload captures
F6 of the gitnexus-check review. Three gaps in the value flow that types a
local receiver, all measured on Lightpanda:
1. `@type-binding.call-return` needed the `call_expression` as the DIRECT
value child, so `const p = try Page.init(…)` (2,551 sites), `… catch
return` (410) and `… orelse return` typed nothing. The rule is now one
keyword-gated declaration match; `emitZigScopeCaptures` unwraps `try`,
`catch`, `orelse` and parentheses (`zigUnwrapValue`) and decides what the
value types (`zigCallReturnTypeOf`): a module-level receiver still names
the type (`Counter.init()` → Counter, Rust `Foo::new()`); a free call
binds the callee name (`makeThing`); a member call on a fn-LOCAL receiver
(parameter / local / payload — Zig forbids shadowing, so "declared in the
fn" is exact) binds the compound `node.asElement()` the shared resolver
walks to the method's return type — instead of typing `el` as `Node`. A
TitleCase callee (`List(u8)`) is a type constructor and binds nothing.
2. No `@type-binding.return` existed. `fn make() !*Thing` now binds
`make ↦ Thing` in the enclosing scope (Module for free fns, the container's
Class scope for methods, where the compound resolver reads it). Builtins,
`type`, `@TypeOf(…)` and comptime type parameters (`?*T`) bind nothing;
`@This()` / `Self` returns name the container. `normalizeZigTypeName` now
strips the error union BEFORE the payload's sigils, so
`Allocator.Error!*Page` → `Page` (it used to leave `*Page`).
3. Payload captures had no binding at all. `populateZigRangeBindings`
(registered as `populateRangeBindings`) types `for (items) |it| / |*it|`,
`for (items, 0..) |it, i|`, `if (opt) |v|`, `if (call()) |v|`,
`while (it.next()) |x|` from the SUBJECT's written type minus one layer
(`[]T` element, `?T` payload) — declining when the layer is not visible
(`ArrayList(T)`) — and the same projection for `const t = items[i]` /
`opt.?` / `ptr.*`. `catch |err|` and `switch` prongs are skipped.
Corpus (Lightpanda, gate before → after): CALLS 15838 → 17853;
`ident.m() bound=local-try/catch/orelse` 11/1298 → 584/1298;
`local-call` 159/1282 → 398/1282; `payload` 82/1085 → 233/1085;
`other-recv:call_expression` 5/991 → 457/991; `local-other` 271 → 364;
`self.m()` 100 %, `free f()` 97.5 %, `ns.f()` 83.4 % unchanged; nothing down.
Not covered: `const t = ns.f()` (a namespace fn's return type across files —
Case 3 has no path from a namespace head to a callable's return binding),
and expression receivers (`items[i].run()`, `o.?.run()`).
* fix(zig): resolve leftover fixture merge markers (Page.zig)
* fix(zig): reconcile F6 value inference with F7 aliases and F5 field bindings
- A fn-local TitleCase receiver (`const R = generic.List(u8); var l =
R.init();`) is a type alias (F7), not a value local: `R.init()` names the
type `R` like `Counter.init()` does at module level, so `l` chains
R → util.List → push. F6's local-receiver rule now excludes TitleCase heads.
- The F6 unit helper only collects the value-inferred / return kinds it
owns; F5 field and F7 alias bindings for the same names are asserted in
their own suites.
* fix(zig): give function-local and anonymous containers an identity (F8)
`const R = struct {…}` declared inside a fn (Lightpanda's reflection.zig
has ~20, one per builder) all collapsed onto one `Struct:<file>:R` with one
`R.get`; anonymous containers (`std.sort.pdq(…, struct { fn lessThan … }
.lessThan)`, `const byte_size = struct { fn it … }.it;`, `?struct { min,
max }` field types) had no identity at all, so their fns were OWNERLESS
Methods (`Method:<file>:lessThan#3`) that collided across a file.
`zigContainerName` now yields the graph IDENTITY on both phases:
- function-local named: `<enclosing callable>$<name>` — `Reflect.string$R`
(Java local-class `$` chain; `populateClassOwnedMembers` leaves it whole);
- anonymous: `<host>$<ordinal>` — `build$1`, `Outer$1`, `Page$1`
(javac's `Outer$1` numbering per host, in source order);
- a `test` host is keyed `test@L<line>` (its string does not survive the
class extractor's qualified-name normalization).
`zigContainerBindingName` keeps the spelling code writes (`R`) for scope
bindings and `@This()` alias rewrites (`@declaration.binding-name`).
Structure phase: bare `(struct|enum|union|opaque_declaration)` rules mint the
local/anonymous nodes via the class extractor; `shouldSkipDefinitionCapture`
keeps exactly one rule per container (`zigContainerAnchor`); a new
grammar-neutral `resolveContainerTypeOwner` provider hook lets the shared
owner walk name a container from context, so `Method:<file>:Reflect.string$R
.get#0` and its HAS_METHOD source agree by construction. Scope phase: the
wrapper group splits name/binding-name for locals and anonymous containers
get synthesized `@declaration.<kind>` defs (`is-synthetic`).
Lightpanda: ownerless Methods 14 → 0, fns without a node 55 → 0, ownerless
Properties 276 → 5, HAS_METHOD 8003 → 8074, HAS_PROPERTY 7275 → 7403,
CALLS 15838 → 15857, Struct 1905 → 2199; no resolution bucket dropped.
* fix(zig): re-add the implicit receiver on the call side so both method-call spellings reach the callback formal
`extractFunctionParameters` sliced the leading `self` off the formals, which
lined up `r.run(target)` (target@0 ↔ cb@0) but lost the explicit spelling
`Runner.run(&r, target)` (&r@0, target@1 ↔ cb@0): the callback never joined
its formal and `run → target` was missing (PR #1432 review by koriyoshi2041).
Formals are numbered once per function while the receiver differs per call
shape, so the fix lives in `extractCallArguments`: keep `self` as formal 0
and prepend the receiver as actual 0 when the callee is a member call on a
VALUE receiver — chain head is a fn-local name that is not TitleCase, the
same value-vs-type rule F6 uses. Namespace / type / decl-literal receivers
(`Runner.init(cb)`, `helpers.apply(cb)`, `List(u8).init`, `.init(cb)`) get
no prepend. Known residual gap, documented: a module-level value receiver
(`global_runner.run(cb)`) is not fn-local and still misses.
Tests: the F2 integration case now asserts both spellings plus a namespace
call; the unit contract pins `self@0, cb@1` and the per-call actual index.
* fix(zig): address eighth gitnexus-check review pass
- Named dependency modules: `parseZigBuildModuleRoots` scanned
`addModule("<name>", .{ … })` with a `[^}]*` regex, so a nested field
before `.root_source_file` (`.imports = &.{ .{ … } }`) ended the match
at the inner `}` and demoted the module to an unnamed fallback — the
first exe/lib root in the file then answered `@import("<name>")`. The
named lookup now walks the balanced `addModule(…)` argument list
(same scanner as `parseZigRootModules`, comment-stripped, string-aware);
the unnamed fallbacks are unchanged. Regression test in
`zig-import-resolver.test.ts`.
- Type-position `@import`: `var x: @import("m.zig").T = undefined;` was
read as an import binding of `x` on both sides — the query rules match
the `type:` child like a value, and `isZigContainerOrImportBinding`
scanned every named child — so `x` was never declared and became a
named import of `T`. The helper now skips the `type:` field, and
`emitZigScopeCaptures` drops binding-rule matches whose `@import` sits
in the annotation (`isZigTypePositionImport`) without claiming the
source, so `x` binds as a variable and the file edge survives as a
side-effect import. Regression tests in `zig-extractors.test.ts`
(variable extractor + scope captures).
- Union in the class-capture skip guard: the parse-worker's inline
class-like predicate lacked `Union`, so a `Union` definition bypassed
`shouldSkipClassCapture` unlike every other `ClassLikeNodeLabel`.
Added the label; no Zig behavior changes (Zig defines no skip hook), so
no test.
- File-owned method ids in `findEnclosingFunctionId`: the arity lookup
used `findEnclosingClassNode` while the owner came from the file-owner
aware `cachedFindEnclosingClassInfo`, so a Zig file-struct's top-level
fn produced `Method::Page.get` without the `#<arity>` suffix. It now
uses `findEnclosingClassNodeOrFileOwner`, the definition-phase lookup.
Consistency fix only: `ParseWorkerResult.calls` / `.assignments` (the
sole consumers of this id) are merged but not read since #942 — CALLS
edges come from the scope pipeline, whose ids were already right — so
no observable graph change and no test.
Not re-fixed:
- `test/helpers/literal-collectors.ts` `DIR_LANG` has no `zig` entry
(raised for the seventh time): the entry exists (`zig:
SupportedLanguages.Zig`, added by the second-pass commit), so
`languages/zig/**` literals are already validated against the Zig
grammar alone; documented in the PR body since the fifth pass.
* fix(zig): keyword-gate the constructor type-binding rules
The three `@type-binding.constructor` rules (`const p = T{…}`, `mod.T{…}`,
`List(u8){…}`) matched any `variable_declaration` with an identifier and a
`struct_initializer`, keyword or not — and tree-sitter-zig 1.1.2 parses a
re-assignment `p = T{…};` (and `_ = T{…};`) as the same node type. So an
assignment minted a constructor binding for `p` in its own block, and one
for `_`. Zig's static typing makes the extra binding redundant (`p`
already carries its type from its declaration: annotation, constructor or
inferred value), so it cost little, but it declared nothing and stood out
against every other binding rule (`@declaration.variable`, the import
rules, the call-return rules), which are keyword-gated for exactly this
shape. Split each rule into `"const" .` / `"var" .` variants, like the
call-return rules.
Regression test in `zig-extractors.test.ts`: `p = T{…}`, `q = mod.T{…}`,
`l = List(u8){}` and `_ = T{…}` after their declarations yield only the
three declaration bindings (fails on the previous query). The zig,
callable-value-flow, grammar-literal and tree-sitter-languages suites are
unchanged.
Raised twice by gitnexus-check (passes on 2026-08-18 12:42 and 12:56).
* fix(zig): re-baseline the callable-flow capture fingerprints, keep `await f<T>(x)` a direct callee
The `benchmarks (GITNEXUS_BENCH)` CI job gates two capture fingerprints
that this branch's shared callable-flow change (3e62b99a) moved without
re-baselining: `bench/python-scope/baseline-fingerprint.txt` and four
languages in `bench/scope-capture/baselines.json` (csharp, cpp,
typescript, kotlin). Both `--check` runs pass on origin/main and failed
on this branch; every other language matched its baseline on the same
run.
Drift, verified by dumping the canonical matches on both trees:
- csharp / kotlin / python: exactly the intended change — a MEMBER call
(`string.Join(x)`, `.forEach { }`, `' '.join(x)`, `.ljust(w)`) no
longer carries `direct-callee-name`; the argument fact is unchanged.
- cpp: `choice.select(1)` (cpp-deleted-overload) drops an INDIRECT
invoke + its synthetic `@reference.call.free` that were gated only by
the same-named free `select` binding; the site is a genuine method
call already captured as `@reference.call.member`. capture_groups_fp
4605 -> 4601.
- typescript: `await svc.verify<T>(x)` (member) and `initializer()(cb)`
(call-of-call) drop the name as intended. But `await verifyToken<T>(x)`
— a DIRECT call — lost it too, because tree-sitter-typescript parses
`await f<T>(x)` as `call_expression(function: await_expression(f),
type_arguments, …)` and the new direct-designator gate saw an
await_expression, not `f`. `wrappedExpression` now unwraps
`await_expression` (no named field, so the field-based unwrap missed
it), restoring parity with main for the direct spelling while the
member spelling stays nameless. Regression test added; it fails
without the unwrap on both assertions.
Gates run locally: scope-capture --check (15 languages), python-scope
--check, import-target --check, tsc, eslint, prettier, the callable-flow
/ golden / tripwire / resolver test files (33 files), full suite with
coverage (82.9/71.5/89.1/86.4 vs 26/23/28/27 thresholds).
* test(zig): fail CI when the optional Zig grammar is absent
`@tree-sitter-grammars/tree-sitter-zig` is an optionalDependency, so every
Zig suite gates on `isLanguageAvailable(Zig)` and the ABI load-smoke accepts
a clean load failure for an optional grammar. Both are the right contract for
a platform with no prebuild, and together they leave a hole: if the grammar
never installed on any CI runner, this PR would merge with all eight Zig
resolver suites plus the structure-phase suite reported green-by-skip, having
never executed the native Zig parser once.
Close it with the `GITNEXUS_REQUIRE_FTS` idiom already used for the FTS
suites. `GITNEXUS_REQUIRE_ZIG=1` declares "this runner has a prebuild, the
grammar MUST be here", and a missing grammar becomes a failure instead of a
skip. tree-sitter-zig@1.1.2 publishes prebuilds for {darwin,linux,win32}-
{x64,arm64}, so the flag is set on two required jobs that all run on covered
platforms: the sharded ubuntu `tests` job and the three-OS `abi-assert` job.
- test/helpers/optional-grammar.ts: the registry mapping a language to its
require-variable, plus `describeGrammarPresence`, a presence assertion that
FAILS when required-but-absent. Deliberately a separate test rather than
flipping the suites from skip to fail: a skipped suite reports success, so
only a failing test can turn "Zig never ran" into a red job.
- parser-loader-abi.test.ts: the optional exemption is revoked for a language
the environment declares required, so an ABI-broken Zig binding fails the
smoke instead of passing as a clean absence.
- optional-grammar-gate.test.ts: pins the two ways the gate could silently
never fire — reading a variable name CI does not set, or accepting a value
CI does not write.
Nothing changes for a run that leaves the variable unset: local runs and any
future prebuild-less platform still skip. Verified both directions --
`GITNEXUS_REQUIRE_ZIG=1` alone: 149 passed, 0 skipped; with
`GITNEXUS_SKIP_OPTIONAL_GRAMMARS=1` forcing the grammar away it fails 2 tests
with an actionable message; with the skip flag but no require flag it is
green-by-skip exactly as before.
Addresses the test-only blocker in the gitnexus-check review of #1432.
* fix(zig): type the optional-grammar gate by grammar key, not language
The gitnexus-check finding on parser-loader-abi.test.ts:155 is right, and
none of the gates caught it: `tsconfig.json` includes only `src/**/*`, so
neither `tsc --noEmit` nor CI typechecks the test tree, and vitest strips
types without checking them. Confirmed with a scoped tsc run over the file:
`error TS2345: Argument of type 'string' is not assignable to parameter of
type 'SupportedLanguages'`.
Not fixed with the proposed `key as SupportedLanguages` cast, which would
assert something false: `listGrammarSources()` yields one row per SOURCES
entry, including variants like `typescript:tsx` that are not enum members.
`isOptionalGrammarRequired` now takes the grammar KEY it is really given,
and the registry keeps a `satisfies Partial<Record<SupportedLanguages,
string>>` so every key we write is still pinned to a real language.
Two new cases cover the failure mode the type error was pointing at — a
registry key that can never match what the ABI smoke passes, leaving the
gate configured-looking and permanently inert: every OPTIONAL_GRAMMAR_ENV
key must be a key `listGrammarSources()` yields and must be marked optional
there, and an unregistered variant (`typescript:tsx`) must not be required
even with the variable set.
Same blind spot, two more latent errors in files this PR adds, both fixed:
`Parser.Language` is not an exported member (use the `setLanguage` parameter
type, as parser-loader-abi.test.ts already does), and the `ParsedImport`
filter did not narrow the union, so `localName` was read through a `!` on an
arm that has no such property — now a type predicate. The one remaining
error under the same probe, in `resolvers/callable-value-flow.test.ts:319`,
predates this branch (authored 2026-07-17, on main) and is left alone.
structural-pair-coverage's optional-grammar case switches from
`it.concurrent.each` to `it.concurrent.for`: only `for` passes the test
context as a second argument (`each`'s callback is `(...args: T[])`), and
that context carries the dynamic `skip()` the per-language gate calls.
Behaviour is unchanged — grammar present: 10 passed; grammar forced away:
9 passed, 1 skipped.
Whole test tree typechecking is a separate, much larger job: the same probe
over `test/**` minus fixtures reports 734 pre-existing errors across the
repo. Out of scope here.
* fix(zig): address tenth gitnexus-check review pass
- `normalizeZigDepPath`: normalize backslashes BEFORE the absolute-path
check. A UNC dep (`\\server\share\dep`) used to slip past the check and
normalize to the repo-relative `server/share/dep`; root-relative `\dep`
had the same hole. Both now return null. Regression case added to the
absolute-spellings test with the files those misreadings would resolve.
- `bindPayloads`: a pointer capture `for (pages) |*p|` now records `*Page`
(declaredSpelling) instead of `Page` — the `*` is an anonymous payload
child before the identifier. Method dispatch is unchanged (`rawName`
strips the sigil), but a deref projection `const q = p.*;` now sees the
pointer layer. New fixture fn `viaPtrCaptureDeref` + assertion; fails on
the previous code (verified by stashing the src fix).
- `optional-grammar-gate.test.ts`: renamed the `typescript:tsx` case — the
key IS a registry row; what makes it inert is the missing gate entry. Now
also asserts a key no registry yields.
- `structural-pair-coverage.test.ts`: header updated — ten tables (not
eleven) are absent from every rule's target side; `Union` left the set
when Zig made it linkable.
- `language-classification.ts`: doc comment now names zig in the
experimental set (added after Ring 1).
Not re-fixed (invalid findings):
- "owner-hook contract wired to an undeclared variable": stale-diff read —
`findEnclosingClassInfo` declares `resolveFileTypeOwner` /
`resolveContainerTypeOwner` as optional parameters (ast-helpers.ts:905,
917) and parse-worker threads them at every call site; tsc compiles clean.
- "optional Zig grammar added unconditionally to the parsing fixture
suite": the cited block only `fs.readFile`s the committed fixture file to
assert it is non-empty — no parser or grammar load is involved.
* fix(scope-resolution): mark construction-site CALLS edges in reason (opt-in), enable for Zig
PR #1432 human review, item 2: a Zig struct literal `T{ .f = x }` (no
parens) is modelled as a CALLS edge to the type — the Rust `T { .. }` /
Go `T{}` shape — and nothing on the edge told it apart from an invocation
(`get_next_spawn → SpawnRequest` from seven `return SpawnRequest{ … }`).
`ScopeResolver.markConstructionSites` (default off): when set, the edge
emitted for a `callForm === 'constructor'` site gets ` (constructor)`
appended to its reason, in both emit paths — `local-call (constructor)` /
`import-resolved (constructor)` in the free-call fallback and
`scope-resolution: call (constructor)` in the reference bridge. The Zig
resolver opts in. `Reference` gains an optional `callForm`, copied from
the site by `buildReference`, so the bridge can see the form.
Why `reason` and not a property or edge type: relationships carry no
arbitrary properties, a new column changes the relation DDL and moves
SCHEMA_FINGERPRINT, and `reason` is the channel the IMPLEMENTS `-pointer`
receiver form already uses. Why opt-in: the unsuffixed strings are a
pinned contract asserted verbatim by the other language suites
(php/cpp constructor calls expect exactly `import-resolved`); every
non-Zig edge stays byte-identical.
Tests: `references-to-edges-call-form.test.ts` pins both vocabularies
and the default-off behaviour; `zig.test.ts` asserts
`Reflect.string → Accessor` / `Reflect.url → Accessor` carry
`local-call (constructor)` next to a plain invocation, and that every
marked edge targets a Struct.
* feat(zig): track qualified struct literals (`mod.T{…}`) as construction sites
PR #1432 re-test (issue comment on
|
||
|
glier
|
b15ff2d888
|
feat(ingestion): mint Destination nodes from AsyncAPI 3.x documents (#3140)
* feat(ingestion): read AsyncAPI 3.x documents into broker addresses
Adds a format-driven reader that turns the `operations[]` entries of an
AsyncAPI 3.x document into (broker, address, direction) triples, plus the
protocol-to-broker map behind it. Nothing consumes it yet.
The reader lives outside `frameworks/spring/` on purpose, like
`destination-key.ts` and for the same reason: an AsyncAPI document is a
published artifact emitted by generators across several language toolchains
and written by hand as often as generated. The entry criterion is therefore
the document format -- a root `asyncapi` key -- and never the generator.
AsyncAPI 2.x is refused under its own countable reason rather than mapped.
Its `publish`/`subscribe` are inverted relative to 3.x `send`/`receive`, so
a naive mapping reverses every direction in the async graph while leaving it
connected: nothing fails, the arrows simply point the wrong way. A silent
skip would be indistinguishable from "this service publishes no document",
which is the one thing the refusal count has to be able to tell us.
The broker is read twice over -- from the operation's bindings and from its
channel's server protocol -- and the two readings must agree. A destination
keyed on the wrong broker joins a stranger, and with the document
contradicting itself there is no way to tell which reading is right, so the
operation is refused rather than decided by a coin flip.
An unmapped protocol passes through as its own literal instead of being
dropped, because `destinationNodeKey` takes a plain string precisely so a
non-Spring caller can attest to a broker Spring has no member for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(cli): add --asyncapi-spec, an explicit path to AsyncAPI documents
Threads an `asyncApiSpecPath` option from the CLI, the server analyze
endpoint, and the programmatic entry through to `PipelineOptions`. Nothing
reads it yet; the reader added in the previous commit is still unwired.
Shaped deliberately after `springActuatorPath`, the existing option for an
out-of-band artifact: an explicit local path, accepting a directory or a
single file, resolved against the repository root so a committed
`docs/asyncapi` and an absolute cache populated by something else are both
natural, and `undefined` keeping the feature entirely off. Mirroring that
option rather than inventing a mechanism is what lets a downstream consumer
point the reader at documents fetched out of band without patching a file
here.
`analyze --watch` REJECTS the flag, exactly as it rejects --spring-actuator.
The watcher reacts to source changes and nothing watches a document
directory, so honouring it there would read the documents once and then
serve a stale answer for the rest of the session -- worse than refusing,
because it looks like it worked.
Additive only: 49 inserted lines, no deletions and no modified lines. Every
new interface member is optional and every forward is an object-literal
spread of an undefined value, so with the option unset the analyzer takes
byte-identical paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(ingestion): mint Destination nodes from AsyncAPI documents
Wires the reader into the destinations phase. With `--asyncapi-spec` set,
every `send` operation emits PUBLISHES_TO and every `receive` emits
CONSUMES_FROM against the ordinary resolved `Destination` node -- same key,
same `address` property -- so a document and a source site that name one
address on one broker land on ONE node and the two halves of a conversation
meet. Verified end to end: an address named only in a document is minted
with the right broker and direction, and an address a source site already
resolved stays a single node with its own `literal` provenance while the
same address on a different broker stays separate.
This claims only what a document states -- that the service talks to that
address, on that broker, in that direction -- and never which method does
it. The addresses in one document partition by (broker, action) into buckets
that usually hold more than one operation, so any assignment past a bucket
of size one is a heuristic, and a wrong one attaches a real address to the
wrong handler: a false connection wearing the clothes of a resolved one. The
edge therefore starts at the document, not at a callable. That is weaker
than a source-derived edge and worth having anyway, because it is available
where the source supplies nothing at all -- a programmatically registered
listener, a broker with no patterns here, a language whose messaging idiom
nobody has taught this codebase yet.
Documents are read even when the source pass found no messaging, which is
why the early return had to move: a repository whose brokers are invisible
to the patterns is precisely the case a published document covers, and an
early return keyed on source sites skipped the documents exactly there.
Their counters are kept in their own block rather than folded into the
existing ones. `refusalsByReason` is the denominator of the SOURCE
unresolved fraction, and a mistyped specification directory must not be able
to make the source look worse than it is. The block is absent -- not zeroed
-- when no path was configured, so "not asked for" stays distinguishable
from "asked for and found nothing"; those need different answers from an
operator and one zero cannot say which happened.
The direction assertion is the one that matters and it is pinned by type,
not by existence: inverting the mapping in the source tree fails exactly one
test, because every other assertion passes identically under both readings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: document --asyncapi-spec and why step 4 of the cascade stays empty
Adds the flag to both READMEs and to the three byte-identical copies of the
CLI skill, which a sync test pins together.
Also rewrites the note on the `specification` seam in the address cascade.
It said "nothing supplies it today", which was true and is now misleading:
a reader exists, and the hook is still unsupplied because of a decision
rather than for want of one.
A document names addresses; it does not name the method that uses one. To
hand an address to a particular candidate something must choose which of the
document's operations belongs to it, and the only division both sides agree
on -- (broker, action) -- leaves buckets that usually hold more than one
operation. On a real generated document exactly one bucket of four was
unambiguous. Every assignment past a bucket of size one is a heuristic, and
a wrong one puts a REAL address on a joining node under the wrong site: a
false connection wearing the clothes of a resolved one, which is the outcome
the keying rule exists to prevent.
The note also records the two things that would change that and are not
heuristics -- a document carrying the implementing symbol, or a
configuration source answering the `${key}` the candidate already recorded
-- and that the second wants its own resolver, since what it needs is the
placeholder key rather than the candidate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): refuse the document shapes that would forge a join
Four ways a conformant AsyncAPI document could mint a Destination that
connects two services which have said nothing about each other. Every one is
reachable from ordinary 3.x vocabulary, not from malformed input, and each is
now a countable refusal.
A PARAMETERIZED ADDRESS is a pattern, not a place. Two services that publish
`{env}.orders` share a template; one deploys with env=prod and the other with
env=staging, and keying on the template text merges them into a single node
with a publisher on one side and a subscriber on the other. This is the
document-side twin of `overridable-config-default`, which argues the same
thing about `${key:default}` in source. A channel declaring `parameters` is
the specification's own statement that its address is a template, so the
detector is a reading rather than a guess; the `{` test catches generators
that template without declaring.
ANY BINDINGS KEY WAS TAKEN AS A PROTOCOL. AsyncAPI allows `bindings` to be a
Reference Object, so the map's own key can be `$ref` -- and passed through,
that becomes half of a join key carrying no broker information at all. Two
services that both reference shared bindings and both name `orders` then land
on one node, defeating the broker-in-key rule that keeps `kafka orders` and
`rabbit orders` apart. A broker must now be spelled like a protocol name.
A BROKER CONTAINING A SPACE COLLIDES, because the node key joins with one:
("kafka orders", "x") and ("kafka", "orders x") are the same key. That was
latent while every broker came from Spring's closed union. This module is the
first caller to feed the shared helper text that a document wrote, which is
exactly the condition under which it stops being latent, so it is closed here
-- at the producer -- rather than by changing an encoding that `routeNodeKey`
shares.
THE ADDRESS WAS TRIMMED, while the source cascade keeps an address exactly as
written so `" orders "` stays its own node. Two producers of one key held
opposite whitespace policies and the document side erred toward joining.
Also: fold the transport-security protocol variants (`kafka-secure`,
`secure-mqtt`, `wss`, `stomps`, `https`) onto their base protocol. The
`amqp`->`rabbit` argument already in this file demands it -- AsyncAPI's server
vocabulary distinguishes them and its bindings vocabulary does not, so a
secured cluster's own document was being read as self-contradictory. Treat a
channel with no `servers` as available on all of the document's servers, which
is the specification's default and was costing every single-server document
its destinations. Bound the address and operation-id lengths, because
`generateId` concatenates rather than hashes, and bound total operations
across the run rather than only per document.
Read each file through ONE handle for both the size gate and the read, as
`actuator-runtime.ts` does and for the reason its comment gives (CodeQL
js/file-system-race): re-resolving the path lets a swapped file bypass the
cap, and the out-of-band cache this option reads is written by other tooling
by definition. Open it with O_NONBLOCK: the type check that rejects a FIFO is
unreachable without it, because opening a FIFO for reading blocks in open(2)
until a writer appears -- found by writing the test first and watching it time
out rather than fail.
Count symlinked entries and walk truncation instead of dropping them in
silence. A symlinked cache and a wrong path were producing identical results.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(analyze): rebuild when documents are configured, and report what was read
An AsyncAPI document is external to git freshness in exactly the way an
Actuator snapshot is: replacing one moves no commit and dirties no file. The
option's own README paragraph advertises an absolute cache written by other
tooling, and on the second run of that workflow the already-up-to-date fast
path fired, no document was ever opened, and the previous run's addresses were
served as current. Measured, not reasoned: editing a document and re-running
printed "Already up to date" and left the old address in the graph; with this
change the same probe re-reads and the new address replaces it.
So an enabled run forces a rebuild and dropping the option forces one more, to
clear document-derived evidence -- the treatment `springActuatorPath` already
gets, for the same reason. Only the FLAG is recorded in index metadata, not
the path: Actuator retains its inputs so future scans keep excluding them,
whereas a committed document is deliberately NOT excluded (it wants its real
`File` node), so there is nothing to retain and recording the path would put
an operator's directory layout into metadata for no consumer.
That also settles a defect it would have been tempting to patch separately. A
synthetic `File` node for an out-of-tree document carries a path that is in no
write set and is not covered by `isGraphWideNode`, so an incremental writeback
dropped the node while keeping its edges -- which then COPY against a row that
was never written, and fail into an IGNORE_ERRORS retry that reports success.
A forced rebuild has no incremental subgraph to get that wrong.
Distinguish the two meanings of `resolution: 'specification'`. That value
belongs to the address cascade and means a CODE candidate was resolved through
the step-4 hook; a node minted from a document has no code site and now says
`asyncapi-document`. Reusing one string would leave a query that groups by
provenance unable to separate an address a document states from one a document
was used to resolve, and only the second is a claim about source.
Report what was read. The stats block was justified on the grounds that an
operator must be able to tell a mistyped directory from a repository with no
documents -- and nothing surfaced it, so the justification was aspirational. A
configured path that yields nothing, or a walk that hit a bound, now warns
unconditionally, as `spring-auto-configuration.ts` does for the same class of
input. The phase summary carries the refusal breakdown rather than only the
totals, because the unresolved fraction is the number this work is judged on
and a bare count says how big the gap is without saying what would close it.
Tests for the three wiring lines that were individually deletable with a green
suite, following the templates already in the repository: a row in the
`--watch` rejection table, the CLI-threading assertion beside the Actuator
one, and the shipped-skill fragment that pins the flag in all three copies.
Also pin `filePath: ''` on a spec-minted destination -- the half of the keying
rule that stops a shared node becoming collateral damage of one document's
next change -- and the in-repo `File` branch, which was dead-code-able.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): close the join-forging paths a second review round found
The `$ref` exclusion added last round fixed one instance of a class and left
the class open. A `bindings` map key of `x-scs-function` -- an ordinary
Specification Extension, which generators emit -- still became the broker, so
two unrelated services carrying one vendor annotation and one address landed
on ONE node whose broker half said nothing about any broker. And
`{ kafka: {}, x-internal: {} }` read as two brokers, losing a conformant
document and reporting it as self-contradictory, which also made any document
author a one-line saboteur of their own cross-service links.
So the two readers are now separate functions with opposite defaults, and the
header says why they must be. `servers[].protocol` is a FIELD DECLARED to hold
a protocol: an unrecognized value there is the document's own claim and passes
through, because refusing it would lose a destination the document states
plainly. A `bindings` MAP KEY is not that -- the specification puts `$ref` and
`x-` in the same namespace -- so a non-protocol key is the EXPECTED case and
only AsyncAPI's binding vocabulary may answer. The syntactic test that was
applied to both was the right rule for one of them.
The walk fix from last round introduced something worse than it reported. A
shared abort flag meant depth exhaustion in ONE branch terminated the whole
traversal, so ten good documents beside a twelve-deep unrelated subtree were
kept or lost depending on whether that subtree sorted before or after them.
Truncation and budget-exhaustion are now separate: depth returns from its own
branch, and only a genuinely global bound stops the walk.
Rewriting `protocol.ts` dropped the whitespace check from the server-protocol
path and made the node-key collision reachable again. The test written for
that collision last round caught it within the minute; the comment now records
that it was learned twice.
Everything else measured this round:
- The broker is the THIRD string that reaches a graph identifier, and it was
unbounded while the header claimed there were two. A one-megabyte protocol in
a document satisfying every other cap was measured producing a gigabyte of
resident identifier strings, because `generateId` concatenates rather than
hashes and the phase mints one id per node and per edge.
- The run-wide operation budget counted ACCEPTED operations, reproducing at the
run level the exact defect the per-document cap was corrected for last round:
a run whose every operation is refused never decrements it. Both now count
operations EXAMINED, and `operation-cap` sets `truncated` -- it is a bound
that stopped the operation count, which is what that flag is documented to
mean.
- The channel-inherits-all-servers rule ran per operation. Hoisted: it depends
only on the servers.
- A subdirectory that cannot be listed is counted rather than dropped, so a
mixed-permission cache cannot report a clean, complete read.
- The read LOOPS, like the Actuator reader this claims to follow. A single read
was never short across seven hundred probes on APFS, but POSIX permits it and
FUSE mounts with `direct_io` -- the deployment this option targets -- return
short counts. A document truncated at a line boundary still parses, so the
failure is silent: operations vanish with `refusals: {}`.
- `parameters: {}` no longer refuses a literal address; an empty container
states nothing and generators emit them.
- A channel that is itself a Reference Object gets its own reason instead of
`no-address`, which was telling operators their documents omit addresses when
the reader simply stops one hop short.
- A multi-protocol document resolves from its operation's own bindings; only
when those are silent does an inherited multi-protocol server set refuse, and
under `ambiguous-server-default` rather than a reason that says the document
contradicts itself. It does not.
- HTTP and WebSocket are refused for destination minting. For a broker the
topic is the namespace; for HTTP the host is, so keying on the path alone
would make every service exposing `/events` one node. A `Route` already
models an HTTP endpoint, with its method in the key.
- The sniff window is a parse gate, not a read gate, and four kilobytes refused
a good document behind a licence header.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ingestion): pin the wiring and the reporting that were deletable
Three lines could be deleted with the whole suite green, and each of them
makes the feature partly or wholly inert: the forward from `run-analyze` into
`PipelineOptions`, the forced rebuild while documents are configured, and the
cleanup rebuild when the option is dropped. None is visible one layer up,
where the CLI test asserts on a mock's arguments.
One integration test closes all three. It drives the real `runFullAnalysis`
against a real repository and asserts, in order: the enabled run does not take
the up-to-date fast path and logs the rebuild; the destination reaches the
graph, which only happens if the option is forwarded; a document edited with
the tree clean and the commit unchanged is re-read; dropping the option
rebuilds once and removes the document-derived evidence; and the run after
that is up to date again -- the "rebuilds once" half, which rests on the
metadata being written as a fresh literal rather than merged, and which
nothing pinned.
The document lives OUTSIDE the repository on purpose. That is the workflow the
option is documented for, and it is the only one where the hazard exists:
editing a tracked file dirties the tree and forces a rebuild anyway, so an
in-repo fixture would pass with the freshness fix reverted. Verified by
reverting both: deleting the forward fails on the empty destination list,
deleting the forced rebuild fails on the missing log line.
Also pinned, each because deleting the code it covers left the suite green:
the `parameters` half of the templated-address refusal (its old test supplied
a braced address too, so the `{` half alone satisfied it); the phase actually
forwarding `symlinksSkipped`; the unconditional warning, whose whole argument
is that a tally nobody can see is not a tally -- captured through the
repository's own `_captureLogger`; a `.yml` document; a character device,
which is the case the `isFile` check exists for and which the FIFO test does
not reach; and the bound that stops a walk.
Reject an empty `--asyncapi-spec` at the CLI. It resolved to the repository
root and walked the whole tree, defeating this module's own rule that there is
no glob-based auto-discovery -- and the HTTP entry point already rejected the
identical value. Two doors onto one option must not hold different rules.
Surface the flag in the MCP context resource beside `spring_actuator`. It
matters more there than for its neighbour: Actuator annotates nodes the source
pass already found, while document reading mints destinations and edges with
no code site, and nothing said where they came from.
Log the configured path relative to the repository. The same change refuses to
persist that path to index metadata because it would record an operator's
directory layout; holding that rule for metadata and not for logs was holding
it in one place.
Both test files now clean up their temporary directories.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* style(ingestion): apply the repository's prettier contract
`quality / format` runs `npx prettier --check .`, and three files added by this
branch were not formatted to it. No behaviour changes: the reader's line
breaks and two test literals move, nothing else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ingestion): release the mini-repo handle the document test allocated
`setupMiniRepo` documents that the caller owns cleanup, and every other test
in this file calls `repo.cleanup()` in its `finally`. The AsyncAPI document
test removed only the document directory, so each run left a temporary
repository behind.
The two owners are separate on purpose: the document directory is a SIBLING of
the repository, placed outside the working tree so that editing it cannot
dirty the tree and force a rebuild on its own. The repo's cleanup therefore
does not reach it, and both calls belong in the same block.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): stop partial server evidence from reading as unanimous
Six review findings, every one a way this reader could name a broker the
document does not name. They share a shape: something is DROPPED rather than
refused, the remaining evidence agrees with itself, and an operation is
attributed with confidence to a broker its document never settled on. A wrong
broker is half a join key, so it does not produce a missing edge -- it produces
an edge to a stranger, reported as a fact.
CAPPED SERVER MAPS. A channel with no `servers` inherits all of them, and that
map is capped at 1,000. A document whose first thousand servers are Kafka and
whose thousand-and-first is JMS read as unanimously Kafka, because unanimity
was tested on the slice. Counted `server-cap` and set `truncated`, but neither
stopped the attribution. The inherited path now refuses under
`capped-server-default` -- checked BEFORE agreement, since a subset agrees with
itself for free.
ROOT SERVER REFERENCE OBJECTS. The Servers Object patterned field is
`Server Object | Reference Object`, so `{ $ref: '#/components/servers/prod' }`
is conformant. Reading `protocol` off the raw value dropped every one: an
all-reference document had no protocol at all, and -- worse -- a MIXED set lost
its disagreeing half and became unanimous. One hop is now followed, through
`#/servers` and `#/components/servers`; anything else is refused under
`unresolved-server-reference` rather than skipped.
CHANNEL BINDINGS. Only the operation's bindings were read. A conformant channel
carrying `bindings: { kafka: {} }` with no operation binding was dropped as
`protocol-unknown` while the document said plainly which broker it meant, and a
disagreement between the two levels was invisible. Both are read; a conflict is
`protocol-disagreement`.
EMPTY `servers`. "If `servers` is absent or empty, this channel MUST be
available on all the servers defined in the Servers Object" -- one sentence,
both cases. A zero-iteration loop returned `explicit: true`, which blocked the
inherited fallback and dropped valid operations.
POINTER DECODING ORDER. RFC 6901 percent-decodes the fragment BEFORE splitting
on `/`. The raw token was tested for a separator first, so `#/channels/orders%2Fv1`
passed a check it should have failed and then decoded into two segments -- a
pointer addressing `channels.orders.v1` was read as a channel named `orders/v1`,
inventing a channel the document never declared. A malformed escape is now
refused rather than resolved against its undecoded text. `~1` still resolves; it
is the pointer's own escape and belongs after segmentation.
THE SNIFF WINDOW. A fixed window decides by where the root key sits rather than
whether it is there, so every window is a false negative waiting for a longer
preamble -- 4 KiB was replaced by 64 KiB for that reason and inherited the same
defect. The whole text is scanned; it is already bounded and already in memory,
and the gate exists to skip the PARSE, which is the expensive half. A leading
UTF-8 BOM is stripped before both sniff and parse.
Twelve of the fourteen new tests were run against the unfixed reader and all
twelve failed; the other two are controls that must pass either way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(ingestion): simplify AsyncAPI pointer and binding resolution
Decode each $ref once, union binding evidence, and stop walking capped
server maps whose brokers are unused on the inherit path.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
ChunxueLi
|
131d9f93fd
|
feat(route): resolve vendor-derived Spring mapping annotations by suffix (#2883)
* feat(route): resolve vendor-derived Spring mapping annotations by suffix Frameworks commonly wrap Spring's built-in annotations with company-specific variants (e.g. Winning Health's @WinPostMapping wraps @PostMapping). The annotation definition lives in a binary JAR — not in source — so the meta-annotation cannot be read statically. Add resolveSpringAnnotationAlias(): resolves custom annotations by naming suffix (WinPostMapping → PostMapping → POST). This matches the universal Java convention of naming derived annotations with the base name as a suffix. Works for any vendor prefix, not just one company. The fix is in springAnnotationHttpMethods() (spring-shared.ts), which both the ingestion extractor (spring.ts) and the group extractor (java.ts) call. A single-function change propagates to both layers automatically. Zero configuration: no .gitnexusrc, no annotation allowlist. If an annotation name ends with a known Spring mapping suffix, it inherits that annotation's HTTP semantics. False-positive risk is negligible. Tests: 16 new unit tests covering resolveSpringAnnotationAlias directly, springAnnotationHttpMethods with aliased annotations, end-to-end extractSpringRoutes with vendor annotations, and ingestion/group parity. Existing route tests (260) continue to pass. * fix(route): address review findings — class-level aliases, registered prefixes P1: class-level @WinRequestMapping now gets the same prefix/constraint semantics as @RequestMapping — all five class-level exact-match sites (spring.ts phase-1 collect, typeRequestMethods, typeClassPrefixes; group http-patterns java.ts typeRequestMethods + type-level branch) route through the new shared isClassLevelMappingAnnotation predicate, and the hard-coded 'RequestMapping' argument in springAnnotationHttpMethods calls is replaced with the actual annotation name so alias resolution applies. P2: suffix-only alias matching accepted unrelated annotations (@AuditPostMapping emitted a phantom POST /audit). Alias resolution now requires a REGISTERED vendor prefix — 'Win' by default, extendable via GITNEXUS_SPRING_VENDOR_PREFIXES=Win,Acme without a rebuild. Tests: negative e2e for the unregistered-suffix phantom route, vendor class-prefix parity e2e, predicate unit matrix, env-registration test. * style: prettier * chore: drop accidental gitnexus-shared/dist worktree symlink from prettier commit Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback (#2883) Wire vendor Spring mapping aliases into Kotlin ingestion and group extraction, restore GITNEXUS_SPRING_VENDOR_PREFIXES after the env test, and stamp spring.route-bindings so existing indexes rebuild. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(route): honor Kotlin vendor aliases and prefix freshness Parse Kotlin RequestMapping method arrays in the shared Spring helper, bump spring.route-bindings, and rebuild when registered vendor prefixes change. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: ChunxueLi <mecoloud@users.noreply.gitee.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> |
||
|
Yayler
|
3aa62be717
|
feat: add gitnexus auto-sync for scheduled remote clone and analyze (#2493)
* adds an opt-in auto sync and analysis loop for GitNexus * adds an opt-in auto sync and analysis loop for GitNexus,gitnexus watch [init|start|restart|stop|status] * adds an opt-in auto sync and analysis loop for GitNexus,gitnexus watch [init|start|restart|stop|status] * fix: address PR review cleanup * Prettier code style * merge main * fix(watch): protect local repos and cancel active analysis * fix(watch): harden auto-sync lifecycle and locking - validate watch process identity before lifecycle operations\n- serialize registry, analysis, and LadybugDB access with recoverable locks\n- harden clone paths, symlinks, hooks, quarantine, and worker timeouts\n- install procps in the CLI image for reliable Docker watch control\n- add focused regression coverage for lifecycle, locks, clone, and registry behavior * update agents & claude md * merge main * fix(watch): harden auto-sync lifecycle * fix(watch): normalize SSH repo identity paths * fix(watch): normalize SSH repo identity paths * fix(watch): safely cancel analysis across platforms * fix(auto-sync): close worker and group sync failure paths * fix(auto-sync): drop retired allowStale from group sync allowStale was removed from SyncOptions, which broke typecheck and CI on this PR. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): satisfy prefer-const and Prettier in auto-sync The watch timers are assigned exactly once, so prefer-const rejected the deferred `let` declarations. They are only read from `stop()` and the control poll, both of which run after the assignments, so binding them at creation is safe and drops the now-dead undefined guards. Remaining files are formatting only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): make lock identity absolute and stop three fail-open paths Lock owner identity was rendered by `ps -o lstart=` through localtime and the active locale, so the same live process produced a different string under a different TZ. A mismatch reads as PID reuse, so one daemon could reclaim a mutex another still held. Pin TZ=UTC and LC_ALL=C. The owner record also carried no hostname, so a holder on another machine was judged by this kernel's view of its PID — always "stale" — and its lock stolen whenever GITNEXUS_HOME is a shared volume. Record and compare the hostname, as the index lock already does. Ownership verification threw unconditionally on win32, which is reached once per project per tick, so watch reported `running` and then failed every repo forever. POSIX uid/mode cannot be checked there; skip those two assertions and keep the dangerous-root, symlink, containment and internal-root guards. Also: quarantine sweep now refuses a symlinked root instead of deleting through it; an unreadable state file propagates instead of being rewritten as empty state, which used to erase every repo's analyzed commit and failure count; a failed staging cleanup no longer strands a published lock with no release handle; and the concurrency runner settles every worker before surfacing a failure so cancellation cannot orphan a live analyze fork. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): land the deferred review findings Six findings that were deferred from the review backlog, plus the docs they change. Worker heap: admission allowed `floor(availableMemoryGB / 2)` slots while every fork was handed the whole machine's heap cap, so the budget meant nothing as soon as an operator raised max_concurrency. Divide the cap by the repos actually analyzed in parallel. The default single-project path is unchanged. Registration: the parent registered without a branch, so it always took the primary/flat arm and relabelled a pinned branch entry on the branch-fallback path. Reproduce the worker's own resolveBranchPlacement decision instead. Cancellation: requestCancellation cleared the only timer and settled nothing, so a worker wedged past its safe point left the promise pending forever, wedging activeRun and hanging `watch stop`. Add a 5s grace after which the parent stops waiting and releases the IPC channel's hold on its event loop. The child is still never killed — it may be inside native work. overwrite_local_changes: `checkout --force` rewrites tracked files only, so untracked sources survived and were indexed as if they came from the remote. `git clean -fd -e /.gitnexus` after checkout; no -x/-X, so ignored paths and GitNexus's own storage survive. Quarantine: age alone never bounds a repo that fails every tick, since each partial clone is younger than the retention window. Keep the five newest per repo. Validation: repo_git_timeout is now bounded by the lesser of an hour and the sync interval, which is also the guard for the bare-number-means-seconds slip (`600000` meant ~7 days and cleared the timer ceiling). And the remote URL's final segment is validated at config load rather than failing once per tick inside the sync loop. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): release an errored worker, and stop rejecting dotted repo names Three findings from the latest review pass. The 'error' handler settles immediately rather than waiting out the grace, so cleanup() clears the grace timer that would otherwise have released the child. An errored IPC channel does not mean the worker stopped, so release it on that path too — still no kill. The traversal guard tested the raw path for '..', which also rejected an ordinary name like owner/foo..bar that the repository-name rule accepts. Traversal is a whole segment, so test segments. The heap-cap test left two runs and their real timers pending; it now stubs timers and settles both promises. Registration coverage now pins the branch slot rather than leaving it implicit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(watch): validate namespace segments and pin the stopped process identity Replacing the raw-string `..` test with a per-segment one dropped a guard: a segment like `..\..\outside` is not literally `..`, so it passed, and those segments build the clone path — on Windows the backslashes are separators. Hold every namespace segment to the same charset as the repo name, which keeps a separator out of a segment while still allowing an ordinary `foo..bar`. The final segment keeps its own check so a bad repo name keeps its own message. The stop wait polled liveness by pid alone, so a pid reused mid-wait would have it wait on an unrelated process and then report the watch stopped. Compare the process start time recorded for the owner, which also returns sooner. Registration now omits `branch` for a primary index instead of passing it as undefined, so that call keeps the shape it had before this branch. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): ship auto-sync as the remote daemon, reserve gitnexus watch. Keep analyze --watch for local incremental re-index and stop the top-level watch verb from starting a clone/pull loop. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): reject invalid branch refs and verify status identity (#2493) Reject leading slashes and per-component trailing dots in configured branches, and verify the live watch owner before trusting a stored error status. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): reject ownerIds that can escape the watch directory (#2493) Stop interpolating a tampered ownerId into the stop-request filename; only basename-safe values are treated as owners. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): recognize auto-sync in the watch-process identity check (#2493) Stop/status were still looking for a standalone watch token after the command rename, so a live gitnexus auto-sync start process would be refused as unrelated. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(autofix): apply prettier + eslint fixes via /autofix command * fix(auto-sync): reject boolean max_concurrency instead of coercing it to 1 (#2493) Number(true) is 1, so a YAML boolean would have passed the integer check and silently meant one worker. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): swallow status errors in the watch finally path (#2493) An uncaught updateStatus rejection in finally became an unhandled rejection. Skip the clone-root symlink test on Windows, where directory symlinks need privileges. Align the group-lock comment with fail-closed registry timeouts. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): catch cancelling status-write failures (#2493) Fire-and-forget updateStatus('cancelling') could become an unhandled rejection, the same class as the finally-path status write. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): ignore queued interval ticks after stop (#2493) clearInterval does not cancel a timer callback already queued. Guard runSafely on stopping so shutdown cannot start a new un-cancellable run. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(auto-sync): report stored watch status timestamps (#2493) status should show when the watch last entered a state, not when the CLI queried it. The failure-count test still expects 1 after a new commit resets the streak; rename it so that reset is explicit. Co-authored-by: Cursor <cursoragent@cursor.com> * style(auto-sync): apply prettier to starter status logger (#2493) Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: weiyf <weiyf3634@163.com> Co-authored-by: Gergő Magyar <gergomagyar@icloud.com> Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Gergő Magyar
|
5a1e5c8803
|
feat(kotlin): ingest Spring HTTP routes as decoratorRoutes (#3133)
* feat(kotlin): ingest Spring HTTP routes as decoratorRoutes (#3130) Kotlin RestControllers now emit analyze Route nodes, handler attribution, and folded constant paths on the ingestion decoratorRoutes channel, matching Java Spring without changing Java or Python extractors. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review): reject abstract/sealed Kotlin RestControllers and keep Java brace parsing Code review required fail-closed admission for abstract and sealed classes, and Kotlin-owned method=[...] translation so Java extractors do not start emitting routes from bracket arrays. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(kotlin): note ingestion-side Spring decoratorRoutes wiring The module comment still described Kotlin as group-layer-only. Ingestion now uses the same constant-fold hooks as Java. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kotlin): keep empty Spring path arrays and trailing commas (#3133) Empty [] / arrayOf() class or method paths are no prefix, not a skip, matching the group-layer contract. Trailing commas in annotation argument lists no longer fail parseSpringAnnotationArguments. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(kotlin): flatten empty Spring path classification Match class-level empty [] / arrayOf() handling to the method-level branch so the fail-closed path is not nested. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Carter LaSalle
|
f34daea86a
|
fix(routes): connect decorator routes to their handler function (#2865)
* fix(routes): connect decorator routes to their handler function
A Route node's only relationship was HANDLES_ROUTE from its FILE. The graph knew
a route existed and which file declared it, but not which function implemented
it. Two consequences on a 12.4k-file repository with 162 FastAPI routes:
- Every decorated handler was indistinguishable from dead code. Its sole edge
was DEFINES, so a reachability query reported it unreferenced even though the
framework invokes it on every request.
- `route_map` / `api_impact` could only answer at file granularity, and
`processes.ts` routed every route through its `routesWithoutHandlerByFile`
fallback instead of keying by handler.
Two halves of one gap, both already designed for and neither wired:
1. `ExtractedDecoratorRoute.handlerName` is documented as "captured at extraction
where the decorated definition node is in hand", and `resolveRouteHandlerSymbols`
already consumes it to stamp `handlerSymbolId`. Only the Spring extractor ever
set it, so for every decorator-routed framework — FastAPI, Flask, NestJS — it
arrived undefined and 0 of 162 routes carried a handler. A route decorator's
parent IS the decorated definition, so the name is in hand: add
`decoratedDefinitionName` and thread it through. It climbs consecutive
decorators so stacked forms (`@router.get(...)` over `@requires_auth`) resolve,
caps the climb so a malformed tree cannot loop, and returns undefined rather
than guessing — the routes phase already treats a missing name as
"fall back to file-level".
2. With a handler symbol resolved there is finally something to point an edge at.
Emit a definition-level HANDLES_ROUTE alongside the file-level one. The sibling
decorator overlay already does exactly this: `pipeline-phases/tools.ts` anchors
HANDLES_TOOL on the definition the decorator sat on, not its file. Routes were
the outlier.
Kept as one change because the edge is inert without the symbol — emitted from a
branch lacking part 1 it produces zero edges, since `handlerSymbolId` is empty.
Additive, and both existing consumers are unaffected:
`group/extractors/http-route-extractor.ts` types its query `(handlerFile:File)`;
`manifest-extractor.ts` matches an untyped `(handler)` but takes `LIMIT 1` ordered
by `handler.id`, and `File:…` sorts before `Function:…`, so its selected row is
unchanged.
Direction is Function → Route, matching how every other overlay attaches
(MEMBER_OF → Community, STEP_IN_PROCESS → Process, HANDLES_TOOL → Tool: the symbol
is the source). That also keeps it free of schema risk — `Function|Route` is
already declared by the ATTACHMENT rule in `lbug/schema.ts`
(`DEFINITION_ANCHOR_LABELS × ATTACHMENT_TARGET_LABELS`), which that file documents
as deliberate headroom for this case. Route → Function would have needed a new
hand-listed pair, and an undeclared pair aborts `analyze` outright — a failure
that file records having hit four separate times.
Verified on a FastAPI fixture (edges 9 → 11):
api.py (File) -> GET /widgets, POST /widgets [unchanged]
list_widgets (line 10) -> GET /widgets [new]
create_widget (line 15) -> POST /widgets [new]
On the 12.4k-file repository: 161 of 162 routes now resolve to their handler
function, up from 0. The single abstention is `uniqueSymbolId` correctly refusing
to guess where the name is not uniquely resolvable in its file.
`npx tsc --noEmit` clean; schema-pair coverage and route suites pass (196 tests).
* fix(routes): harden decorator handler attribution (#2865)
Keep definition-level route links correct across warm caches and malformed symbol lookups, and avoid per-route group-sync scans. Move Python AST ownership behind the language provider and add end-to-end regression coverage.
Note: full npm test could not complete in this container due unrelated worker startup failures and a stalled retry; targeted route suites, typecheck, format, and lint passed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(routes): reuse per-file symbol lookup and drop duplicate warm-cache test
Share extract()'s CONTAINING_QUERY memo with the graph provider path, resolve each route handler once, and fold the decorator-edge warm-cache assertions into the existing FastAPI composed-route round-trip.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Carter LaSalle <carterlasalle@gmail.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
glier
|
f7a58cf188
|
feat(ingestion): capture Spring handler annotation arguments and template publishes (#3128)
* feat(ingestion): capture Spring handler annotation arguments and template publishes
Non-HTTP handler recognition already resolves the annotation NAME through
imports, aliases, use-site targets and package visibility. What it never
captured is the annotation's ARGUMENTS, so the destination a listener binds to
was invisible: `@KafkaListener(topics = ...)` and `@RabbitListener(queues = ...)`
name it with different attributes, and a producer names it by position. The
publishing side was missing entirely, which left every messaging edge
one-directional by construction.
Consumer side: `SpringNonHttpHandlerAnnotationFact` gains an optional
`args?: readonly { name?: string; text: string }[]`. `name` is optional because
positional and named arguments are genuinely different shapes, not because it
is sometimes unknown.
Producer side: `KafkaTemplate.send`, `RabbitTemplate.convertAndSend`,
`JmsTemplate.convertAndSend` and `StreamBridge.send` for both languages.
Arguments are captured as SYNTAX, never as a resolved address. At capture time
imports are not final, a sibling file's constants do not exist yet and
configuration has not been read — the same reason annotation-name resolution
was deferred. Resolution belongs to a later phase; doing it here would be a
layering error that happens to work on simple inputs.
The parse cache schema moves 82 -> 83. Both new facts ride the existing
worker -> main side channel, which is replayed verbatim from
`ParsedFile.captureSideChannel`, so a warm v82 cache would skip the workers and
hand back annotation facts with no `args` and an empty producer list. Measured
on the fixture app: a warm all-cache-hit run (`usedWorkerPool=false`,
`reparsedFileCount=0`) reproduces 6 Java and 7 Kotlin producer facts from the
store alone — exactly the state a pre-change cache would have served as zero.
Tests cover both languages across literal, constant and configuration-key
destinations, and pin the PREVIOUS behaviour too: handlers captured before are
still captured, and shapes that must not produce a fact still do not.
* test(ingestion): cover the handler and template shapes capture left unpinned
Auditing the argument capture against its own definition of done turned up
three annotations and three templates that work but that nothing asserts, so a
regression in them would land silently.
Handler side: `@EventListener` and `@ServiceActivator` were only ever checked
for RECOGNITION, never for arguments, in either language, and Kotlin
`@RabbitListener` appeared in no argument test at all. Both annotations carry an
address just as `topics` and `queues` do — an event listener names it as a type
and an integration endpoint names it as a channel — so leaving them unpinned
left a third of the handler family covered by nothing.
Producer side: Kafka was the only template whose destination was written three
ways. Rabbit, JMS and the stream bridge each appeared with a single spelling, so
nothing said that a constant or a configuration-bound name produces a fact for
them too. The same fixtures pin the negative that `RabbitTemplate.send` and
`JmsTemplate.send` stay unrecognized, since only the method that belongs to the
template counts.
Each test asserts the previous behaviour alongside the new one: the handler is
still recognized and still named the same, and only then are its arguments
checked. A test that looked at arguments alone would keep passing if recognition
itself broke.
Verified against the parent commit that this is coverage, not repair: every
Java and Kotlin fixture in the suite produces a byte-identical capture side
channel on both revisions once arguments and producer facts are set aside.
Mutating `StreamBridge` out of the template table, and making Kotlin annotation
arguments return nothing, each fail the new tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): stop Spring capture from inventing arguments and receivers
Five defects in the capture-time Spring messaging facts, all of which put
data that is wrong — not data that is missing — into a durable store.
Facts built from recovered syntax. After a syntax error tree-sitter keeps
parsing by guessing boundaries, so the tree stays well formed while
describing text nobody wrote. An unterminated `kafkaTemplate.send(TOPIC,`
absorbed the next method's source and offered it as two more arguments;
`@KafkaListener(topics = "orders", groupId =` reported a `groupId` whose
value was an empty `{}` borrowed from the method body. Both now fail
closed: a producer call with an unparsed argument list yields no fact at
all, and an annotation with one reports no arguments. Neither carries a
state that could mean "published somewhere unreadable", so the choice was
between silence and a plausible lie.
Arguments were not normalized though the receiver beside them was. The
receiver already collapsed a wrapped chain to one spelling; the argument
kept its newlines and the ENCLOSING block's indentation, so the same
constant compared unequal to itself at two nesting depths, and again in a
CRLF checkout. Receiver and argument now share one normalizer.
That normalizer damaged multi-line literals. Its doc comment promised to
keep the rewrite away from nested string literals, and delivered that only
for single-line ones: a Java text block or Kotlin raw string whose newline
sat next to a dot lost the newline, changing the value. The normalizer is
now literal-aware, which makes the promise true for both.
The receiver name match accepted only one decoration. Matching the type
name as a suffix recognized `orderKafkaTemplate` and dropped
`kafkaTemplateDlq`, `kafkaTemplateV2`, `kafkaTemplate2`, `KAFKA_TEMPLATE`,
`kafka_template`, `streamBridge2`, `STREAM_BRIDGE`, and `rabbitTemplate1` —
including the `static final` constant spelling, which is exactly the shape
this capture exists to find. The name is now folded on `_`/`$` and matched
as a substring. The bare-identifier gate still runs first and is what keeps
`config.get("a.kafkaTemplate")`, `templates["k"]`, and `getTemplate()` out.
Ownership was attributed one level too deep. With no boundary types the
ancestor walk passed through a nested type body, so a publish in the field
initializer of a class declared inside a method was attributed to that
method, which may never run it. The identical construct at the top level of
a class already yielded no fact; a type body is now a boundary so the rule
reads the same at every depth, while a publish in a METHOD of a nested or
anonymous type is still attributed to that method.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(ingestion): read Kotlin handler annotation arguments on evidence
Kotlin asked for annotation arguments unconditionally, for every annotated
function with any annotation, while Java made the same decision in two
passes and paid only for callables that carry a handler annotation.
Measured on 200 annotated NON-handler functions in one file, the Kotlin
side-channel payload went from 41069 bytes to 78797 — a doubling, crossing
the worker boundary and landing in the durable store, for data no consumer
reads today.
The reason Kotlin had no prefilter is real and is preserved: an import
alias (`EventListener as SpringEvent`) gives a handler annotation a local
name no list can contain, so discarding CALLABLES by simple name would lose
them before the post-import resolver runs. That argument covers capturing
the annotation; it does not cover reading its arguments, because the alias
is not a mystery at capture time. The import header states both the local
name and the FQN it stands for, so the existing relevance predicate can be
asked about the IMPORTED name and the answer carried back to the alias.
Kotlin now runs Java's two passes, with that alias set widening the first
one. Every annotated function still produces a fact with the same name and
use-site target as before — the non-handler payload is 41069 bytes again,
byte for byte what it cost before arguments existed — while handlers, and
handlers reached only through an alias, keep their arguments.
Also corrects two comments that described behavior the code did not have.
The Java capture claimed an economy Kotlin was not making; it now describes
both languages. The argument opt-in on both DI modules claimed it kept
argument text off the wire, but every DI fact already carries the
annotation's full source text — what the opt-in avoids is a second, parsed
copy, and the comment now says so.
The test file is renamed: `spring-handler-annotation-arguments` differed
from the pre-existing `spring-annotation-arguments` by one word in the
middle, though they cover different mechanisms — an AST capture versus a
text parser. It is now `spring-argument-fact-capture`, after the module it
exercises.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ingestion): correct the argument-text contract the normalizer outgrew
Both `SpringNonHttpHandlerAnnotationFact.args` and `SpringArgumentFact.text`
promised the value stays "exactly as written". That was true when the field was
added and stopped being true in the same branch, when argument text started
going through `normalizeSpringFactText` so that one destination written across
two lines would not compare unequal to the same reference on one line.
A consumer reading only the interface would have assumed a source spelling the
fact does not retain — and the indentation such a consumer would have seen is
the enclosing block's, not a property of the expression at all.
Both docs now state the single rewrite and its reason, and still say plainly
that nothing is resolved. Reported by the review bot on #3128; the claim was
introduced by this branch, not inherited.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* style(ingestion): apply Prettier to the four files CI flagged
`quality / format` runs `prettier --check .` and four files from this branch had
drifted: two line-width wraps and two of the opposite kind, where a call fits on
one line. No behaviour change — tsc clean, the four affected suites still pass
126 tests.
Worth noting why the pre-commit hook did not catch it: lint-staged formats
staged files, but a rebase replays commits without running hooks, so anything
that only becomes unformatted relative to a moved base slips through. Checking
the whole diff against `prettier --check` before pushing is the reliable step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ingestion): publish Kotlin named arguments through a Kotlin template
Kotlin forbids named arguments when the callee is a Java method: parameter
names are not guaranteed to survive into bytecode, so the compiler refuses
`kafkaTemplate.send(topic = ..., data = ...)` for the Spring `KafkaTemplate`
imported from `org.springframework.kafka.core`. Every named-argument example
in this feature was written that way, which asserted capture on source that
could never compile.
The path itself is real and stays covered. The classifier matches on the
receiver's NAME, so a template declared in Kotlin is recognized exactly like
the Spring one, and named arguments to it are legal. Each affected example now
declares that template and publishes through it; the assertions are unchanged
except for the one receiver spelling they name.
The fixture's `publishWithNamedArguments` had no test reading it at all, so it
carried the illegal shape into an app fixture for nothing. It is removed, and
the two pipeline expectations that counted its publish drop a row.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): withhold a broker when the receiver name matches two templates
Widening the receiver-name rule from a suffix to a substring — needed to accept
`kafkaTemplateDlq`, `KAFKA_TEMPLATE`, and the rest — also let ONE receiver
satisfy TWO signatures. `KafkaTemplate` and `StreamBridge` both publish through
`send`, `RabbitTemplate` and `JmsTemplate` both through `convertAndSend`, so
`streamBridgeKafkaTemplate.send(...)` matched twice and the loop returned
whichever came first in the list: kafka, by declaration order alone.
The receiver's TYPE is deliberately never resolved here, so nothing in this
module can rank the two matches. Neither the longest match, nor the last one,
nor the order of the signature list is evidence about the bean: that name reads
equally as a KafkaTemplate fronted by a stream binding or a StreamBridge named
after the broker behind it. Publishing one of them as the template turned an
unanswered question into a definite attribution a consumer has no way to
distinguish from a resolved one — a publish routed to the wrong broker.
An ambiguous receiver now yields no fact at all. That costs a rare publish,
stays recoverable by a later phase that owns type information, and is the
failure this capture already prefers everywhere else. Both outcomes are pinned:
three receivers naming two templates yield nothing, while decorated names that
merely look long (`orderStreamBridge`, `streamingKafkaTemplate`) still resolve.
The `typeName` contract said "suffix", which the same widening had made false.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ingestion): correct two argument contracts this change set invalidated
Sweeping the Spring capture comments for claims the feature commits outgrew
turned up two more, both about what a MISSING argument list means.
`SpringNonHttpHandlerAnnotationFact.args` promised that absence means the
annotation was written without an argument list. Reading Kotlin arguments on
evidence gave absence a second cause: Kotlin still produces a fact for every
annotated function — it has no name prefilter, so an import alias cannot hide a
handler — but reads arguments only for callables carrying a handler annotation,
so a non-handler fact has no arguments however its annotation was written. Java
produces facts for handler-bearing callables only, so there the old reading
still holds. The field now states both causes and which language has which.
`SpringArgumentFact.name` said a template call gives its destination by
position. That is true of Java, which has no named arguments, but Kotlin names
call arguments whenever the callee is declared in Kotlin, and this module
captures the key when it does — the reason the field exists for calls at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ingestion): make the Kotlin argument reader refuse recovered syntax itself
`kotlinValueArgumentFacts` is exported and already has a caller in another
module, and its contract said the caller MUST reject a recovered list first.
Both callers did. But a guard that every future caller has to remember is the
same fragility this change set exists to remove — the Java twin is safe only
because it is module-private with one call site.
It now returns `null` for a recovered list, so the decision is unavoidable at
the type level, and each caller answers in the way its fact requires: a producer
call drops the whole fact, having no state for "published somewhere unreadable",
while an annotation reports no arguments and collapses into the marker form.
Both say "nothing here to resolve", which is true.
No behaviour change — the same 126 tests across the three affected suites pass,
including the truncated-annotation and truncated-call cases that pin the
fail-closed path.
Raised as hardening in the maintainer review of #3128.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
|
||
|
MyShining
|
c217a0f257
|
feat(spring): import optional Actuator runtime data with Kotlin JVM mapping (#3107) |