Commit graph

9 commits

Author SHA1 Message Date
Gergo Magyar
cddbe135c0
fix(scope): follow namespace factory return types and record untyped drops
`import mod as rp; x = rp.make(); x.m()` lost the receiver type even when
`make` is declared `-> C`: `x` was bound to the raw dotted name `rp.make`,
and nothing followed a dotted name to a function's return type. The inline
`rp.make().m()` failed the same way in the compound resolver, which read
`ns.f()` only as a construction.

Resolve `ns.f` to `f`'s return annotation when `ns` is a namespace import,
the fact `propagateImportedReturnTypes` already mirrors for a named import.
The post-finalize chain follow uses it for bindings and the compound
resolver for inline calls. Only sites that emitted nothing change: the
full lang-resolution corpus emits the identical 2,838 CALLS/ACCESSES edges.

Record the drops that left `impact` at "exact":
- a plain receiver bound to a namespace function with no usable return type
  (`x = ns.untyped()`);
- an outer call whose `file:line:col` key a resolved inner site already
  claimed (`ns.untyped().m()`, `s.repo.Delete(id)`). Only a duplicate
  capture of the same member hides a drop now.

Receiver baseline: call drops 128 -> 151, reads 44 -> 48, ten shape-arm
cells INVISIBLE-GAP -> VISIBLE-GAP; every new drop has no CALLS edge to its
member (BASELINE.md lists them). The python.factory-qualified accuracy gap
is removed; the python-scope fingerprint and capture golden gain the new
fixture only.

Fixes #3498

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:22:29 +01:00
samuel mbabhazi
0e7b3d6fad
fix(mcp): report cross-function explain lines 1-based (#3635)
* fix(mcp): report cross-function explain lines 1-based

The interprocedural branch of explain passed the endpoints' and hops'
graph startLine through unchanged. Those are 0-based (tree-sitter rows),
so every cross-function source, sink and hop sat one line above the
function's declaration in an editor, while the intra-procedural findings
of the same response were already 1-based. source.line, sink.line and
hops[].line now go through toDisplayLine like context, query and impact
(#2377), and the response note says these are declaration lines.

The interproc integration block asserts the exact fixture lines: handle on
line 7 of source.ts, runIt on line 6 of sink.ts.

Fixes #3491

* fix(review): remove repaired accuracy allowance

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-20 08:54:52 +01:00
Abhigyan Patwari
565e50276d
fix(routes): resolve Express registrations made through helper parameters (#3629)
* fix(routes): resolve Express registrations made through helper parameters (#3487)

#3603 made Express receivers provenance-proven, which removed the receiver-
name false positives but also every registration on a function parameter:
a parameter has no binding value, so `app.get(...)` inside a helper never
resolved. On GitNexus's own server that dropped `* /api/mcp`
(`mountMCPEndpoints(app, ...)`) and `GET /` (`registerWebUI(app, ...)`),
and the two SSE progress routes registered by
`mountSSEProgress(app, '/api/.../progress', jobs)` were never found at all
(#3487, item a).

Named function values in the route facts now carry an identity and the
binding id of each parameter (a TypeScript `this` parameter takes no
position, and a parameter written anywhere in the body, nested closures
included, takes none either). Calls to plain functions and to namespace or
default import members are recorded as invocations, keeping only the
arguments that can carry an allocation or a literal path; arguments after a
spread have no position. The resolver resolves a registration whose
receiver, literal path or mounted child is a parameter through each call
site of its function, once per site and in the helper's file. This is one
hop only: a site argument that is itself a parameter does not resolve. When
no call site is recorded, the call is evaluated as written, so an unknown
mount path still suppresses its child router instead of exposing its paths
at the root.

Only calls whose callee is named like a function that registers on its
parameters are evaluated, which keeps resolution at its previous cost on
repositories with many helper calls. A function value's invalidation key now
includes its identity, so a property write on one function
(`Button.displayName = ...`) no longer disables every other. The group
layer shares the resolver, so it gets the same routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): forward mounted-child parameters and aliased helper imports

Review follow-ups for the helper-parameter forwarding:

- A helper that forwards only its mounted router (`function mountChild(child)
  { app.use('/api', child) }`) was never resolved, because the forwarded
  function was chosen from the receiver and path alone. Any parameter among
  the receiver and arguments now selects it, as long as all come from one
  function.
- A helper called through an aliased import (`{ mountSSEProgress as
  mountProgress }`) was filtered out before its call sites were recorded,
  since the filter compared only the local name. It now also accepts the
  imported name.
- A write to a same-named local declared in a nested block (`{ let app;
  app = other; }`) discarded the outer parameter. A write to a binding that
  a nested block, function or catch clause declares no longer counts.
- The fixture's `registerWebUI` uses its static directory.

The release tool-accuracy gate now passes `routes.helper-registration`, so
its known-gap allowance is removed. Its expected file moves from the caller
(`api.ts`) to the helper (`progress.ts`): an Express route belongs to the
file of its `app.get` call, which is also where its handler and response
keys are, as for every directly registered route.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): treat destructuring reassignment as a parameter write

`[app] = [other]` reassigns `app`, but the write check compared the whole
target text (`[app]`) with the parameter name, so the parameter stayed
forwardable and later registrations used the call-site argument. A
destructuring target now writes each name it binds; a member target still
writes only a property. Also drops an unused local flagged by CodeQL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): follow helpers exported under another name

`export { mountRoutes as setup }` records the export as a binding, so the
call-site filter knew the helper only as `mountRoutes` and skipped callers
importing `setup`. An export of a binding to a named function now adds its
exported name too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
2026-10-18 22:13:10 +01:00
Abhigyan Patwari
aa33a6e94c
fix(routes): link base-URL fetch calls made through wrapper chains (#3628)
* fix(routes): link base-URL fetch calls made through wrapper chains (#3488)

GitNexus's own web client calls `fetchWithTimeout(`${_backendUrl}/api/repos`)`,
which reaches fetch through `resilientFetch` and a local `fetchImpl` alias.
`api_impact` found none of those consumers, for three independent reasons:

- normalizeFetchURL rejected any URL that did not start with `/`, so a
  leading base-URL binding dropped the call. It now strips a leading
  `${...}` run followed by `/`, with the same rules as the group layer's
  consumer normalizer (#3111); the helper moves to route-path.ts so both
  layers share it. A path whose every segment is a param (`${base}/${id}`,
  `${base}/${id}.json`) is rejected, since it would match every route of
  that depth. Interpolations are collapsed brace-balanced before the query
  split, so `${repo ? `?repo=${repo}` : ''}` no longer ends the path, and
  one glued to the last static segment is treated as a query or suffix.
- The wrapper call collector required a `/`-leading literal, so the same
  template never became a wrapper candidate.
- Wrapper detection only looked five AST levels deep for a call to an
  identifier named `fetch`. Arrow-const wrappers sit deeper than that, and
  `resilientFetch` calls an alias of `globalThis.fetch`. Detection now walks
  the whole definition and accepts `globalThis`/`window`/`self.fetch` and
  aliases whose value names fetch (through `??`, `||`, `?:`), read from the
  AST rather than the text. It records whether the first parameter reaches
  fetch's URL argument and which callees receive that parameter. The routes
  phase closes the wrapper set over that forwarding: `fetchWithTimeout` is a
  wrapper because it forwards its URL to `resilientFetch`. A callee declared
  in the caller's file or imported from a relative module must carry a URL
  in exactly that module; a package-imported name is followed only when
  every recorded function of that name carries one. A fetch caller whose
  first parameter is not the URL (`track(event)`) or a same-named function
  elsewhere (an IPC `request(channel)`) lends nothing.

When `[param]` matches several sibling routes (`/api/analyze/upload` and
`/api/analyze/:jobId`), the fetch now links to the route whose segments
match by kind instead of the first one registered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): match wrapper calls by declaring module and decline ambiguous fetch matches

Review follow-ups for #3488:

- A propagated wrapper's name went into a repo-wide set, so a literal call
  to an unrelated same-named function in another module was taken as a
  fetch. Wrapper call candidates now carry the module declaring a bare
  callee when the file declares or imports it relatively, and must reach a
  wrapper defined in exactly that module. Package and member callees keep
  the name match.
- An alias declared in a nested helper (`const fetchImpl = globalThis.fetch`)
  marked an outer call to a same-named parameter as a fetch. An alias now
  counts only where its declaring function encloses the call.
- A first parameter redeclared inside the function (`{ const url = ...;
  fetch(url) }`) was taken as the URL. A shadowed parameter is no longer
  followed.
- A fetch whose `[param]` ties between sibling static routes
  (`/api/kinds/foo`, `/api/kinds/bar`) linked to the first registered one.
  A tie now links none.
- The fixture's `resilientFetch` returns the last response instead of
  rejecting with `undefined`, and its client encodes the repo query value.

The release tool-accuracy gate now passes `api.template-wrapper`, so its
known-gap allowance is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): keep call-site wrapper names and ignore helpers that never run

- A wrapper call through a renamed relative import (`import { request as
  send }`) stored the exported name, so a configured `fetchWrappers` entry
  naming the call-site `send` no longer matched. The call keeps its
  call-site name and carries the declared name separately for the module
  lookup.
- A nested helper bound to a local that is never referenced again (`const
  later = () => resilientFetch(url)`) made its enclosing function forward
  its URL. Calls inside such a helper are ignored; a helper that is called
  or passed on, such as a retry callback, still counts.
- The fixture's `resilientFetch` no longer retries after a timeout or abort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): let a nested parameter shadow a fetch alias

A function parameter between a fetch alias and a call shadows the alias
there (`function send(fetchImpl) { return fetchImpl(event) }` beside an
outer `const fetchImpl = globalThis.fetch`), so that call no longer marks
the enclosing function as a fetch wrapper. The fixture's retry helper also
stops returning a stale 5xx response after a later attempt throws.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): drop fetch aliases whose name is redeclared with another value

A block that redeclares an alias name with a non-fetch value (`{ const
fetchImpl = other; fetchImpl('/x') }`) let the inner call read as a fetch.
A name declared anywhere in the function with a value that does not name
fetch is no longer an alias, which covers every block-scoped shadow without
lexical scope tracking.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(routes): ignore uncalled nested declarations and shadowed global objects

- An uncalled nested `function helper() { fetch('/x') }` made its enclosing
  function a fetch wrapper. Nested function declarations that are never
  referenced are now ignored, as unused function-valued locals already were.
- `window.fetch(url)` through a parameter or local named `window` (or
  `self`, `globalThis`) was read as the platform fetch. Such a root is now
  that value's method. A bare `fetch` parameter is still taken as fetch,
  since injecting a fetch implementation is the common reason for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(bench): refresh the receiver-resolution baseline for the fetch fixture

The new `fetch-base-url-wrapper` fixture in `test/fixtures/lang-resolution`
adds one unresolved TypeScript chain-field call to the committed check
corpus (127 to 128 call drops). Regenerated with
`measure.mjs --update-baseline` after a fresh build and cleared parse caches;
no other counter moved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
2026-10-18 22:11:37 +01:00
azizur100389
702c917377
fix(python): retain nullable factory receiver types (#3595)
* fix(python): retain nullable factory receiver types

* test(accuracy): remove repaired Python gap allowances

* test(bench): refresh Python capture fingerprint

* fix(python): type `None or call()` assignments like `call() or None`

`None or make_user()` always evaluates to the call, but only the
`call() or None` order matched, so the variable lost the factory's
return type. Add the mirrored boolean pattern, matching the two
conditional-expression orders the PR already supports.

Fixture gains `none_or` (resolves to User.save) and `none_and` (stays
unresolved: `None and x` is always None). The capture golden changes only
for the edited fixture, and the python-scope baseline is re-recorded
(`measure.mjs --check` passes, scaling 1.021).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:39:40 +01:00
Gergő Magyar
bd20dde825
fix(taint): report inline and factory handler flows (#3612)
* fix(taint): cover inline and returned handlers (U8)

Join expression-handler CFG summaries to distinct graph functions without changing existing anonymous-owner identities. Keep named function expressions private to their lexical scope.

Related: #3489

* fix(ci): encode inline handler taint paths (U8)

Use codec-safe generated display names while preserving positional function identity. Verify persisted hop continuity and remove the repaired inline-flow accuracy allowance.

* fix: keep anonymous names unbound (U8)

* test: pin inline callback captures (U8)

* ci: run inline taint guard (U8)

* test: align inline handler contracts (U8)

Pin callback-owned call edges while preserving nested callable IDs and module-level HOC attribution. Assert repaired inline flows and the retained captured-closure limitation.

* ci: run native inline proofs on all OSes (U8)

Register real-worker identity/CFG and persisted explain cases on the existing Windows/macOS matrix. Charge explicit scheduling estimates without changing watchdogs or runtime behavior.

* test(taint): reject all enclosing factory sources

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-17 20:22:58 +00:00
Gergő Magyar
8fa8e64388
fix(rename): extend safe refactoring and flow coverage across languages (#3557) 2026-10-13 08:38:17 +03:00
Gergő Magyar
191a954bf6
fix(python): resolve calls through module-returning accessors (#3542)
Some checks failed
Trivy Image Scan / Trivy (gitnexus-cli) (push) Has been cancelled
Trivy Image Scan / Trivy (gitnexus-web) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Gitleaks / gitleaks (push) Has been cancelled
Publish / Classify release event (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Publish / RC guard (marker + release-PR skip) (push) Has been cancelled
Publish / ci (push) Has been cancelled
Publish / Publish to npm (push) Has been cancelled
Publish / Build & Push RC Docker images (push) Has been cancelled
* test(accuracy): retire repaired Python accessor gaps

* fix(python): connect module accessor resolution

* fix(python): resolve calls through module-returning accessors

Bind `_pc().target()` and `m = _pc(); m.target()` to the module's
function when a local accessor returns an imported module, and count
capped-out dropped names in the impact lower bound.

Closes #3497

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor(python): share module-accessor position key (#3542)

- Reuse the capture-side positionKey in the receiver namespace resolver
  instead of a second line:col helper plus an inline template.
- Note why the synthetic { moduleScope, scopes: [] } input is sufficient
  for createNamespaceTargetCache (sealed disk-backed ParsedFile shape).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor(python): reuse subtype-dispatch positionKey (#3542)

- Export the existing line:col positionKey from subtype-dispatch instead of
  defining an identical copy in module-accessors, so both Python capture
  side channels key positions through one helper.
- Apply prettier to the Function-scope indexing line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): unique test titles and rebaseline Python benches (#3542)

- impact-epistemic-lower-bound: `%j` rendered null, NaN and Infinity all as
  `null`, so the test-completeness audit rejected the duplicate identities.
  Use `%s` so every parameterized case has a distinct title.
- python-scope: capture fingerprint changes with the new module-accessor
  capture facts (golden captures were already updated).
- receiver-resolution: +4 Python call drops (125 -> 129). The two-step
  `x = f(); x.m()` form is now counted as an unresolved call instead of
  vanishing: fastapi-depends `router.get/post` (unknown, x3) and
  python-method-chain-binding `user.get_city()` (in-program; `User` has no
  `get_city`, so the site is genuinely unresolvable).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:41:51 +01:00
Gergő Magyar
4756e8a820
fix(release): gate publication on accuracy and paired evaluation (#3503)
* test(release): add fixed-answer native tool accuracy

* fix(release): gate stable artifacts on pinned paired evaluations

Apply review findings #1-4: gate Docker publication, grade in-flight retries, preserve repository context, and execute negative controls in CI.

* fix(eval): verify native context and resolve harness security findings

Inspect Claude's first startup reminder for ordinary repository guidance.
Bound launcher-option matching and escape Markdown backslashes and pipes.
Exclude only the parsed synthetic accuracy corpus from CodeQL; keep the
benchmark harness and evaluator under security analysis.

Validated focused Python and TypeScript regressions plus core typecheck.

* fix(eval): wake the EC2 runner and prove paired evaluator execution

Bootstrap the existing dedicated instance from a protected hosted job, bound
native runner pickup, and stop only instances started by this run. Keep paid
sessions inside the fixed stop window and add a runner-only dispatch.

Require a contained six-cell prepare/session/oracle/report canary in CI,
including a failed repair that remains in the measured denominator.

* fix(ci): reuse existing configuration and disambiguate accuracy cases

Remove the new AWS credential/settings path and release opt-in variable. Bound native runner pickup with the existing GitHub token and retain private EventBridge startup until its existing mechanism can be reused. Reuse current workflow environment names and evolution schedule controls.

Name malformed-observation cases explicitly so the execution audit distinguishes null from missing and actually tests invalid arrays.

* fix(review): isolate release builds and preserve benchmark evidence

* fix(release): gate RC publication on paired quality and manage EC2 lifecycle

* fix(ci): clarify release shell redirects and build script

* fix(eval): remove temporary patch sinks after capture

* fix(bench): retire tool-accuracy allowances repaired on main

Main at 50aa4be3b repaired three #3487 route checks (#3505) and all four
#3499 Python scope checks (#3502, #3504). The ratchet correctly failed the
merged head with stale allowances, so remove them and keep the fixed answers
as release protection: 18/31 pass, 13 recorded gaps remain.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(release): gate RCs on accuracy only and keep paid runs for stable

Follow the cost split proposed in discussion #3493: fixed-answer tool
accuracy on every RC, paired agent runs before stable releases and on a
weekly schedule against the latest RC.

RC publication no longer prepares a bundle, calls release-evaluation.yml
and waits for a paid run of up to 21 hours on the shared EC2 lock. Every
release attaches this run's accuracy report; stable npm publication still
requires exact-commit paired evidence. release-evaluation.yml drops its
workflow_call path and the now-unused candidate bundle module.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(release): address PR #3503 review feedback

- Page through recent release-evaluation runs instead of reading only
  the newest 50, so valid stable evidence cannot be pushed off page one.
- Bound each runner-pickup jobs request by the remaining deadline.
- Replace non-finite floats at any depth in failure receipts so the
  incomplete report still serializes with allow_nan=False.
- Clarify that the summary.cap case has 501 names over 1001 sites.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(review): harden candidate build and close review gaps

- Mount the candidate checkout's .git read-only inside the build sandbox,
  so lifecycle scripts cannot plant git config (e.g. core.fsmonitor) that
  host git later runs in that checkout during actions/checkout cleanup.
  The real-Bubblewrap canary now also attempts that write.
- Type tool-accuracy observations instead of using explicit any.
- Cover the fail-closed branches reviewers found untested: unsafe or
  missing evidence artifacts, mismatched task pins, malformed repo/SHA,
  an instance that stops again during bootstrap, and unbalanced GitNexus
  guidance markers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(release): describe RC accuracy gate and stable evidence step

- CONTRIBUTING now states that RCs are gated by CI (including the
  fixed-answer tool-accuracy check) without a paid run, and that stable
  publication needs a passing Release evaluation for the exact commit
  within seven days.
- The oracle-control fixture passes tarfile's data filter only where it
  exists (3.11.4+), matching the declared Python 3.11 floor.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor(bench): pin tool-accuracy fixture line anchors in one place

run.ts bucketed rename edits and explain findings by line literals that
expectations.ts duplicated. Export FIXTURE_ANCHORS (file, line, pinned text)
from expectations.ts, build the fixed answers and run.ts classifiers from it,
and fail loudly (runner and unit test) when a fixture line no longer holds
the pinned text. Expected answers and observations are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(bench): fail closed when ripgrep is missing from the tool-accuracy run

rename's text-search pass shells out to rg and only logs a degraded warning
when it is absent, so a runner without ripgrep scored a degraded rename
(homonym.ts:1 missing) and still passed. Check rg --version before indexing,
record it in accuracy.json source.externalTools, and install ripgrep in the
ci-tests tool-accuracy job and the docker release-gate (same if: condition).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor(eval): split baseline guidance, share bwrap preamble, drop dead bare option, harden patch cleanup

- Move the baseline (no-GitNexus) guidance scrubbing out of proposer_sandbox.py
  into baseline_guidance.py, with its unit tests.
- Expose real_directory, runtime_mount_args and bwrap_base_args publicly so
  release_build no longer imports private helpers or re-assembles the shared
  bubblewrap preamble; the candidate build command is byte-identical.
- Remove the unused run_claude `bare` parameter and its --bare/--tools branches.
- capture_patch: a failed temporary-directory removal no longer discards the
  read patch or replaces an earlier error; it is attached as a note, or warned
  on stderr when there was no earlier error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ec2-runner): keep stop observing through transient describe failures

- stop() treats a failed or timed-out describe-instances call as unknown
  state and keeps observing until the stop deadline; a persistent failure
  is still raised at the deadline with the last command error. Unusable
  states and identity mismatches still fail at once (new CommandError
  subclass separates command failures from semantic ones).
- Readiness CLI has one paid mode: --job-name NAME. --evolve is removed;
  skill evolution passes its job name, and tests tie the watchdog command
  to the paid job's actual name in both EC2 workflows.
- Jobs-API requests never get a timeout larger than the remaining pickup
  budget (previously rounded up to 1s past the deadline).
- Document that the shared concurrency group keeps only one pending run,
  so a queued release evaluation can be replaced and must be confirmed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(eval): record verified EC2 lifecycle configuration

The README still described the AWS settings as assumed and unverified.
The OIDC role, environment settings and runner_only lifecycle were
configured and verified live on 2026-10-09.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(release-eval): grade both runtimes against one pinned task toolchain

Candidate and stable each staged task dependencies (node_modules,
gitnexus-shared/dist) from their own runtime checkout, so the same v1.6.12
task source was graded with different TypeScript, Vitest and LadybugDB
versions and a per-task solve delta also measured dependency drift.

- Check out and build the pinned task commit once (tasks-base) and point
  every task's repo at it for both runtimes; the runtime under test reaches
  sessions only through --gitnexus-root. prepare requires --task-repo at the
  single pinned task commit (new task-sha subcommand resolves it).
- Retain sandbox_dependency_content_digest per measured row and make
  release_gate fail when any task's cells, across candidate and stable, do not
  share one recorded dependency digest.
- Move the task-pin comparison into validate_report(task_pins=...) and use
  it from release_report check and release_gate.
- Cover the gate CLI with a missing and a corrupt candidate report.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(ci): warn that a pending release evaluation can be replaced

GitHub keeps one pending run per concurrency group, so a newer queued
EC2 run cancels a pending release evaluation. Operators must confirm the
startup job ran before relying on the evidence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(eval): prove hidden oracles against the grading toolchain

Release evaluation now grades every task with dependencies built at the
pinned task commit (TypeScript 5.9.3, Vitest 4.1.11), but the oracle
controls still ran against the current checkout's toolchain. The Ubuntu
containment job now builds the pinned task toolchain and points the
controls at it. All seven controls pass locally against that build.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(ci): pin the duplicated EC2 lifecycle jobs together

Both EC2 workflows carry their own copy of the live-verified start,
check, readiness and stop jobs. A reusable workflow would change that
verified job structure, so instead a test requires the copies to stay
identical apart from the job display name and the paid-job dependency.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(release-eval): accept stable evidence only from main's current evaluator

Stable publish validated downloaded agent evidence against the release
commit's eval/ tree and never tied a report to the run that produced it.
Evidence from an older evaluator stayed acceptable for seven days after
graders changed, while task changes on main made every fresh evaluation
mismatch the tag. The first invalid artifact also hid older valid runs.

The composite action now extracts eval/ from main's head and runs the
validator there with locked base dependencies only (no dev extras or
project build). A report counts only when its harness_sha is the head_sha
of its trusted main run and the compare API shows no change under eval/,
the release-evaluation workflow or the pinned agent CLI between that
harness and the evaluator; a missing, diverged or 300-file comparison
fails closed. Runs are tried newest first, rejected runs are skipped
with a bounded reason list, every gh call has a timeout, and task pins
use validate_report's task_pins check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(eval): break the sandbox import cycle and resolve code-scanning alerts

- baseline_guidance is now pure text handling with its own GuidanceError;
  the baseline mount builder lives in proposer_sandbox again and converts
  that error to SandboxError, so imports run one way only.
- task_assets uses the public real_directory name; the unused private
  alias is gone.
- The oracle-control fixture always extracts with tarfile's data filter
  and skips on Pythons that lack it, instead of extracting unfiltered.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(release-eval): run candidate lifecycle scripts offline; reject smuggled report fields

- build_candidate now downloads locked dependencies with --ignore-scripts,
  then runs every lifecycle script (dependency installs, prepare, build)
  in a fresh network namespace, so candidate code cannot reach host-local
  services. Verified locally that the current head and the pinned task
  commit build this way; the real-Bubblewrap canary now asserts only
  loopback is visible to a lifecycle script.
- validate_report requires the evidence document to equal its
  field-whitelisted rebuild, so per-run or top-level fields outside the
  published schema are rejected before the publisher copies the file.
- The sandbox test suite also covers a no-GitNexus sandbox refusing
  unbalanced guidance markers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(eval): describe the offline candidate lifecycle

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(release-eval): pin downloads to the npm registry; require dependency binding

Address review feedback on PR #3503:
- The candidate's online install phase now refuses lockfile entries that
  resolve outside https://registry.npmjs.org/ (other than in-checkout
  workspace links) and refuses shipped .npmrc files, so a candidate
  cannot steer npm at loopback, private or metadata endpoints.
- Every measured row must carry a 64-hex task-dependency digest, so a
  single report cannot be accepted with its grading toolchain unbound.
- Oversized JSON integers mark a measurement untrustworthy instead of
  crashing report generation.
- Baseline guidance stripping recognises headings with up to three
  leading spaces (CommonMark).
- Remove a stale comment about the old private helper name.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(eval): parse CommonMark ATX headings; compare evidence by exact JSON

- Baseline guidance stripping now parses headings as CommonMark ATX
  headings (0-3 leading spaces, optional closing # run, tabs), so a
  GitNexus section heading like '## GitNexus rules ##' is removed too.
- Evidence validation compares the serialized rebuild with the report,
  so a retyped value (true for 1, 2.0 for 2) no longer passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(eval): drop continuations of every CommonMark list item in baseline guidance

A removed GitNexus instruction left its indented continuation behind when
the item used '+' or an ordered marker (1. / 1)). The scrubber now
recognises every CommonMark list-item marker.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(eval): isolate candidate MCP from agent task writes (#3503)

Run MCP in a nested Bubblewrap boundary with read-only task, graph,
registry and runtime mounts, private state and isolated processes/network.
Keep task writes on the agent's built-in tools and remove mutating MCP grants.

Add startup/tool mutation canaries and preserve explicit unsafe diagnostics.

Validation: focused pytest 138 passed, 16 skipped. Full locked evaluator
996 passed, 29 skipped; real containment canaries require Ubuntu CI.
Note: eight pre-existing comparator-reuse failures reproduce on the
unchanged head due to missing os.supports_dir_fd support. One process-control
timeout failed in the full run and passed alone and on the unchanged head.

* fix(eval): close baseline mount overlap and test symlink evidence (#3503)

Reject supplied no-MCP mounts that cover forbidden GitNexus paths, including ancestor mounts and lexical aliases. Keep ordinary dependency mounts available.

Exercise evidence symlink rejection with a real valid-report target and retain dangling-link coverage.

Validation: 1026 evaluator tests passed, 29 skipped; targeted mount tests 36 passed and evidence tests 34 passed. Removing the symlink guard in memory makes the repaired regression fail. Ruff and diff checks passed.

Note: eight pre-existing comparator-reuse failures remain in this environment because os.supports_dir_fd lacks the required os.lstat support.

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 08:26:08 +01:00