mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-10-04 02:31:36 +00:00
b565c7c990
* feat(ingestion): resolve FastAPI include_router(prefix=...) cross-file routes
FastAPI sub-route files declare paths via @router.<verb> while the entry
file mounts the router with app.include_router(<router>, prefix='/x').
Previously both the ingestion-layer Route graph nodes and the group-layer
ExtractedContract URLs lost the cross-file prefix, breaking provider <->
consumer matching.
Ingestion layer:
- parse-worker emits routerIncludes / routerImports + decoratorReceiver
- parsing-processor / parse-impl thread the new fields and aggregate
prefixesByModule across chunks; decorator routes whose receiver is
'router' are duplicated once per matching prefix
- routes.ts joins prefix via normalizeExtractedRoutePath
Group layer:
- HttpLanguagePlugin gains an optional prepareRepo() pre-pass and a
repoContext arg to scan(); python.ts builds prefixesByModule and
falls back to the bare path when no entry matches
- http-route-extractor caches one repoContext per plugin
Tests:
- 3 new http-route-extractor cases (attr / named-import / no-prefix)
- ParseWorkerResult literals in 3 test files updated to the new shape
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(ingestion,group): address PR #1877 review — relative imports, cross-package collisions, host names, ingestion tests
Follow-ups to the FastAPI `include_router(prefix=...)` cross-file fix
based on PR #1877's automated production-readiness review. Three
correctness gaps and one test coverage gap addressed:
1. Relative-import support in the worker regex (FINDING 2)
`FROM_IMPORT_ROUTER_RE` now accepts module paths starting with a
`.` (e.g. `from .calls import router as calls_router`). The
previous `[A-Za-z_][\w.]*` rejected leading dots and silently
dropped every relative-import Shape-B include — a real pattern
from the PR description's own motivating example. The matching
helpers now strip leading dots before keying so absolute and
relative imports collapse to the same module key.
2. Cross-package same-name module collisions (FINDING 3)
Two-tier module keying replaces the previous basename-only key:
• short key — `users` (file basename without `.py`)
• long key — `api/users` (parent dir + stem)
`prefixesByLongKey` is consulted first and only falls back to
`prefixesByShortKey` when no long-key match is available. Both
the ingestion pipeline (parse-impl.ts) and the group extractor
(http-patterns/python.ts) carry the same scheme so the graph
nodes and HTTP contracts agree on which prefix applies.
New protocol field `ExtractedRouterModuleAlias` (parse-worker →
parsing-processor → parse-impl) lets Shape-A
`<host>.include_router(<mod>.router, prefix='/x')` calls promote
to a long key when the same file imports `<mod>` via
`from <pkg> import <mod>`. Without this, `api/users.py` and
`admin/users.py` collided on the basename `users` and the admin
file's routes inherited the `/users` prefix that was only meant
for `api/users.py`.
3. Non-`app` host variable names (FINDING 4)
The group-layer `INCLUDE_ROUTER_*_PATTERNS` queries pinned the
host identifier to the literal `"app"` and dropped every
`application = FastAPI()` / `api = FastAPI()` pattern — the
constraint was redundant given that the call shape
(`include_router` invoked with a router argument and a
`prefix=` keyword) is already specific enough. The pin is
removed; the ingestion regex was already unrestricted.
4. Ingestion-layer regression tests (FINDING 1)
The previous PR added group-layer tests
(`http-route-extractor.test.ts`) but zero in-tree tests for the
ingestion path. Two new suites pin the
worker → parse-impl → routes flow:
- `test/unit/fastapi-router-bindings.test.ts` (23 cases):
`extractFastAPIRouterBindings()` is split into a stand-alone
module so it can be unit-tested without booting a worker
thread, then pinned for regex shape, two-tier key emission,
relative-import support, and negative cases.
- `test/integration/fastapi-prefix-pipeline.test.ts` (5 cases)
plus `test/fixtures/fastapi-prefix-app/` — runs the full
`runPipelineFromRepo()` against a realistic multi-package
fixture (containing both `api/users.py` and `admin/users.py`)
and inspects the resulting `Route` graph nodes for cross-file
prefix joining and absence of cross-package bleed.
Verification
- `npx tsc --noEmit`: pass
- PR-touched test suites (6 files / 117 cases): all green
- `npx prettier --check`: pass on touched files
- `npx eslint`: 0 errors on touched files
Cache / compatibility
The new `routerModuleAliases?` field on `ParseWorkerResult` and
`routerModuleAliases` on `WorkerExtractedData` are optional /
guarded with `?? []`, so historical parse-cache entries continue
to load without forced re-scan.
Refs PR #1877.
* refactor(ingestion): move fastapi-router-bindings out of workers/ — pure module, not a worker
Addresses @magyargergo's `CHANGES_REQUESTED` review on PR #1877:
> Sorry I just found that we are introducing a new worker in the PR.
`gitnexus/src/core/ingestion/workers/fastapi-router-bindings.ts` was a
**pure-function module** — it never imported `worker_threads` or
`parentPort`, never spawned a worker, and was never registered as a
worker entry. It was placed in `workers/` purely because it was split
out of `workers/parse-worker.ts` to make its functions unit-testable
without booting a worker thread (parse-worker is itself the worker
entry and cannot be loaded from the main thread).
To remove the misleading directory placement:
• The implementation moves to
`gitnexus/src/core/ingestion/route-extractors/fastapi-router-bindings.ts`,
alongside the other framework-specific route extractors (`expo`,
`nextjs`, `php`, `laravel`, `middleware`, `response-shapes`).
• `workers/parse-worker.ts` keeps a thin re-export so the worker
entry can keep using `extractFastAPIRouterBindings` directly. The
re-export now carries an explicit comment stating that the imported
file is **not** a worker and that the `workers/` directory
deliberately hosts only true worker entries (`parse-worker.ts`,
`worker-pool.ts`, `quarantine.ts`).
• The new file's leading docstring opens with "NOT A WORKER" and
explains why it exists where it does.
• The unit test (`test/unit/fastapi-router-bindings.test.ts`) is
updated to import from the new path.
No behaviour change. The function body, signatures, and exported types
are identical.
Verification
• `npx tsc --noEmit`: pass
• `npx tsc` (dist rebuild): pass
• `test/unit/fastapi-router-bindings.test.ts` (23 cases): all green
• `test/integration/fastapi-prefix-pipeline.test.ts` (5 cases): all green
• `test/unit/group/http-route-extractor.test.ts` (63 cases): all green
• `npx prettier --check` on touched files: pass
• `npx eslint` on touched files: 0 errors
Refs PR #1877.
* refactor(ingestion): drop parse-worker re-exports; consumers import router types directly from route-extractors
Addresses @magyargergo's two remaining review comments on PR #1877:
1. **`gitnexus/src/core/ingestion/workers/parse-worker.ts:247`** —
"Can you please remove them and update the call sites?"
The `export type { ExtractedRouterInclude, ExtractedRouterImport,
ExtractedRouterModuleAlias } from '../route-extractors/...'` block
in parse-worker.ts is gone. The remaining `import type {…}` is
purely local — used only to type the corresponding fields on
`ParseWorkerResult` below — and the leading comment now says so
explicitly ("this file does NOT re-export them"). The
`extractFastAPIRouterBindings` symbol is also no longer re-exported
from parse-worker.ts; it's still imported here so the worker entry
can call it per file, but downstream consumers must reach it via
`route-extractors/fastapi-router-bindings` directly.
Call sites updated:
- `gitnexus/src/core/ingestion/parsing-processor.ts`
- `gitnexus/src/core/ingestion/pipeline-phases/parse-impl.ts`
Both files now `import type { ExtractedRouterInclude,
ExtractedRouterImport, ExtractedRouterModuleAlias }` directly from
`route-extractors/fastapi-router-bindings.js`. The worker types
they still need (`ParseWorkerResult`, `ExtractedToolDef`, etc.)
keep coming from `workers/parse-worker.js`.
The unit + integration tests already imported from the new path,
so no test changes were required.
2. **`gitnexus/src/core/ingestion/parsing-processor.ts:168`** —
suggested simplification:
for (const item of result.routerIncludes ?? []) allRouterIncludes.push(item);
for (const item of result.routerImports ?? []) allRouterImports.push(item);
for (const item of result.routerModuleAliases ?? []) allRouterModuleAliases.push(item);
Applied verbatim. Replaces the previous `if (result.…) for …`
guards. The cache-compat semantics are unchanged — historical
parse-cache entries that lack these fields still load cleanly,
the new form just spells the fallback inline.
No behavior change, no tests touched, no public API change.
Verification
• `npx tsc --noEmit`: pass
• `npx tsc` (dist rebuild): pass
• PR-touched test suites (6 files / 117 cases): all green
• `npx prettier --check` on touched files: pass
• `npx eslint` on touched files: 0 errors
Refs PR #1877.
* refactor(ingestion): hoist fastapi-router-bindings type imports to top of parse-worker.ts
Move the `import type { ExtractedRouterInclude, ExtractedRouterImport,
ExtractedRouterModuleAlias }` block to the top of the file with the
other type imports, and drop the comment that previously sat next to
ExtractedDecoratorRoute.
---------
Co-authored-by: henry <zhangwei2017@unipus.cn>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
|
||
|---|---|---|
| .. | ||
| call-routing | ||
| group | ||
| import-resolution | ||
| integrations | ||
| mcp | ||
| model | ||
| named-bindings | ||
| scope-resolution | ||
| shadow | ||
| workers | ||
| ai-context.test.ts | ||
| analyze-api.test.ts | ||
| analyze-community-skills-gate.test.ts | ||
| analyze-embeddings-limit.test.ts | ||
| analyze-heap-respawn.test.ts | ||
| analyze-job.test.ts | ||
| analyze-lbug-checkpoint-threshold.test.ts | ||
| analyze-no-stats-bridge.test.ts | ||
| analyze-respawn-progress-terminal.test.ts | ||
| analyze-wal-error.test.ts | ||
| analyze-worker-pool-size.test.ts | ||
| analyze-worker-timeout.test.ts | ||
| api-file-route.test.ts | ||
| api-graph-streaming.test.ts | ||
| api-query-readonly-wiring.test.ts | ||
| api-readonly-wiring.test.ts | ||
| ast-cache.test.ts | ||
| ast-utils.test.ts | ||
| binding-accumulator.test.ts | ||
| blade-template-routes.test.ts | ||
| bm25-search.test.ts | ||
| call-attribution-issue-1166.test.ts | ||
| call-extraction.test.ts | ||
| call-form.test.ts | ||
| call-processor.test.ts | ||
| calltool-dispatch.test.ts | ||
| chunker.test.ts | ||
| cli-commands.test.ts | ||
| cli-i18n.test.ts | ||
| cli-index-help.test.ts | ||
| cli-message.test.ts | ||
| cobol-copy-expander.test.ts | ||
| cobol-preprocessor.test.ts | ||
| cohesion-consistency.test.ts | ||
| community-processor.test.ts | ||
| compatible-stdio-transport.test.ts | ||
| cors.test.ts | ||
| cpp-ue-preprocessor.test.ts | ||
| cross-file-impl.test.ts | ||
| cross-file.test.ts | ||
| csv-escaping.test.ts | ||
| cursor-hook.test.ts | ||
| dart-import-resolver.test.ts | ||
| dart-type-extractor.test.ts | ||
| deferred-resolution-profile-wiring.test.ts | ||
| deferred-resolution-profile.test.ts | ||
| detect-changes-worktree.test.ts | ||
| doctor-format.test.ts | ||
| embedder.test.ts | ||
| embedding-chunking.test.ts | ||
| embedding-config.test.ts | ||
| embedding-pipeline.test.ts | ||
| entry-point-scoring.test.ts | ||
| env.test.ts | ||
| esm-extension-resolution.test.ts | ||
| eval-formatters.test.ts | ||
| eval-server-bind-restriction.test.ts | ||
| exact-search.test.ts | ||
| expo-routes.test.ts | ||
| extract-element-type-from-string.test.ts | ||
| extract-generic-type-args.test.ts | ||
| fastapi-router-bindings.test.ts | ||
| fetch-reason-parsing.test.ts | ||
| field-extraction.test.ts | ||
| format-elapsed.test.ts | ||
| framework-detection.test.ts | ||
| git-clone.test.ts | ||
| git-utils.test.ts | ||
| git.test.ts | ||
| graph.test.ts | ||
| group-service-not-found.test.ts | ||
| has-method.test.ts | ||
| heritage-extraction.test.ts | ||
| heritage-map.test.ts | ||
| heritage-processor.test.ts | ||
| hf-env.test.ts | ||
| hooks.test.ts | ||
| http-embedder.test.ts | ||
| hybrid-search.test.ts | ||
| ignore-service.test.ts | ||
| impact-batching-grouping.test.ts | ||
| impact-confidence.test.ts | ||
| impact-pagination.test.ts | ||
| import-processor.test.ts | ||
| import-resolver-factory.test.ts | ||
| incremental-file-hash.test.ts | ||
| incremental-orchestration.test.ts | ||
| incremental-parse-cache.test.ts | ||
| incremental-shadow-candidates.test.ts | ||
| incremental-subgraph-extract.test.ts | ||
| index-repo-command.test.ts | ||
| ingestion-utils.test.ts | ||
| jcl-parser.test.ts | ||
| kotlin-scope-captures.test.ts | ||
| kotlin-static-marker.test.ts | ||
| language-skip.test.ts | ||
| laravel-route-extraction.test.ts | ||
| lazy-action.test.ts | ||
| lbug-adapter-wal-schema.test.ts | ||
| lbug-checkpoint-lifecycle.test.ts | ||
| lbug-checkpoint.test.ts | ||
| lbug-config-wal.test.ts | ||
| lbug-embedding-hashes.test.ts | ||
| lbug-extension-loader.test.ts | ||
| lbug-native-check.test.ts | ||
| lbug-native-safe-path.test.ts | ||
| lbug-pool-win-fts-probe.test.ts | ||
| lbug-readonly-error.test.ts | ||
| local-backend-maxbuffer.test.ts | ||
| local-cli-subprocess.test.ts | ||
| logger.test.ts | ||
| max-file-size.test.ts | ||
| mcp-stdout-sentinel.test.ts | ||
| mcp-wal-feedback.test.ts | ||
| method-extraction.test.ts | ||
| method-props.test.ts | ||
| mro-processor.test.ts | ||
| noise-filter.test.ts | ||
| parse-diff-hunks.test.ts | ||
| parse-impl-chunk-concurrency.test.ts | ||
| parse-impl-deferred-extraction.test.ts | ||
| parse-impl-e1-emission-shape.test.ts | ||
| parse-impl-env-reads.test.ts | ||
| parse-impl-fallback.test.ts | ||
| parse-impl-progress-monotonic.test.ts | ||
| parse-impl-worker-lazy-cache.test.ts | ||
| parser-loader.test.ts | ||
| parsing-worker-fallback.test.ts | ||
| phase-timer.test.ts | ||
| php-namespace-extraction.test.ts | ||
| php-template-scope.test.ts | ||
| pipeline-exports.test.ts | ||
| pipeline-runner.test.ts | ||
| platform-capabilities.test.ts | ||
| pool-wal-recovery.test.ts | ||
| process-processor.test.ts | ||
| publish.test.ts | ||
| query-fts-parameterization.test.ts | ||
| query-params.test.ts | ||
| rate-limit.test.ts | ||
| receiver-extraction.test.ts | ||
| registry-primary-flag.test.ts | ||
| rel-csv-split.test.ts | ||
| repo-manager-ensure-ignore-readonly.test.ts | ||
| repo-manager-finalize-invariant.test.ts | ||
| repo-manager.test.ts | ||
| resolve-enclosing-owner.test.ts | ||
| resources.test.ts | ||
| route-tool-detection.test.ts | ||
| ruby-self-call.test.ts | ||
| run-analyze-fts-repair.test.ts | ||
| run-analyze.test.ts | ||
| safe-parse.test.ts | ||
| schema.test.ts | ||
| security.test.ts | ||
| semantic-chunk-search.test.ts | ||
| sequential-language-availability.test.ts | ||
| server-cors-stack.test.ts | ||
| server-validation.test.ts | ||
| server.test.ts | ||
| setup-antigravity.test.ts | ||
| setup-codex.test.ts | ||
| setup-jsonc.test.ts | ||
| setup.test.ts | ||
| shape-check.test.ts | ||
| shared-type-extractors.test.ts | ||
| sibling-clone-drift.test.ts | ||
| sidecar-recovery.test.ts | ||
| skill-gen.test.ts | ||
| skip-git-cli.test.ts | ||
| staleness.test.ts | ||
| stdout-silence.test.ts | ||
| structure-processor.test.ts | ||
| suffix-index-ambiguity.test.ts | ||
| symbol-resolver.test.ts | ||
| symbol-table.test.ts | ||
| text-generator.test.ts | ||
| tool-direct-cli.test.ts | ||
| tool-process-linking.test.ts | ||
| tools.test.ts | ||
| topological-sort.test.ts | ||
| transitive-include-closure.test.ts | ||
| tree-sitter-queries.test.ts | ||
| type-env.test.ts | ||
| utils.test.ts | ||
| variable-extraction.test.ts | ||
| vue-sfc-extractor.test.ts | ||
| wal-checkpoint-driver.test.ts | ||
| web-ui-serving.test.ts | ||
| wiki-flags.test.ts | ||
| wiki-grouping-batch.test.ts | ||
| wiki-llm-client.test.ts | ||
| wiki-mermaid-sanitizer.test.ts | ||
| wildcard-synthesis.test.ts | ||
| worker-pool-cumulative-timeout.test.ts | ||
| worker-pool-options.test.ts | ||
| worker-pool-resilience.test.ts | ||
| worker-pool-slot-generation.test.ts | ||
| worker-pool-timeout-retire.test.ts | ||
| worker-pool-transferlist.test.ts | ||
| worker-pool-windows-quarantine.test.ts | ||