The two argument-selection cost groups (r-scope-naming-argument,
r-definition-anchors) were copy-pasted with an absolute 2500 ms limit and
never asserted what the query kept, so an empty parse passed.
- Move the shapes and a size-ratio check (N vs 4N, fastest of three runs,
floored small run) into test/helpers/r-argument-scaling.ts; linear work is
~4x, a quadratic query ~16x, independent of machine speed.
- Assert the exact match count and naming capture at both sizes.
- r-definition-anchors now parses through parseSourceSafe (the 100 K+ char
inputs hit the Windows native-binding limit documented in safe-parse.ts).
- r-scope-naming-argument: classNames no longer filters on /^[A-Z]/; it selects
by @declaration.class and gains lowercase positive and negative cases.
75eaac75 only had integration coverage. Add populateRNamespaceImports cases
for each never-bind class (unexported, undefined at top level, dotted,
dotted sibling of the same tail) plus a positive control in the same package.
Deleting the resolveRNamedImport skip now fails six unit tests.
library()/require()/importFrom() fan-out and the R6 HAS_METHOD set were only
checked with length >= 1 / >= 2, so an extra-edge regression passed. Compare
sorted target lists instead.
- Replace the vacuous ".Rprofile/.Renviron are not indexed" test (no fixture
contained such files) with an r-non-indexed fixture holding .Rprofile,
.Renviron, .Rmd and .qmd that define functions next to a plain .R positive
control; the Rmd/qmd bodies are valid R so routing them to the R parser would
fail the test.
- Add getLanguageFromFilename negatives for those names.
- Add R to the query-compilation smoke list.
- Assert R's LanguageClassifications value ('experimental'), which the docs
claim but nothing checked.
populateRNamespaceImports synthesised a named import for every
importFrom(localpkg, name). Names the provider does not export, does not
define at top level, spells with a dot, or defines beside a dotted S3
method of the same tail never resolve, yet the unresolved import counted as
an explicit name claim and suppressed the documented 0.5-confidence
global-name-fallback edge.
Skip the import when resolveRNamedImport returns null, so such names create
no phantom claim and their calls keep the fallback.
@eagleoutice/tree-sitter-r is scoped, so the tree-sitter- name filter in the
bundle preparation step skipped it. It was neither bundled nor peer-patched:
installing the packed tarball warned that its tree-sitter ^0.21.0 peer
conflicts with the bundled 0.25.1, and the packaged native-parser smoke
failed because the grammar had no sample.
Recognise grammars by package basename, exact-pin and bundle the R grammar,
add it to AUDITED_PEERS, give the smoke an R sample, and let the readiness
report parse scoped names from AUDITED_PEERS.
Upstream took SCHEMA_BUMP 133 to 137, so the R provider bump moves to 138.
The pin test and the taken-versions list are updated to match. The import-target
benchmark keeps both the R and Rust arms in the heap-budgeted and context
language lists, and the tree-sitter readiness tests keep the R grammar checks
against the reworked report (R is now a peer-patch grammar and the held-grammar
counts changed).
* test(accuracy): retire repaired Python accessor gaps
* fix(python): connect module accessor resolution
* fix(python): resolve calls through module-returning accessors
Bind `_pc().target()` and `m = _pc(); m.target()` to the module's
function when a local accessor returns an imported module, and count
capped-out dropped names in the impact lower bound.
Closes#3497
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* refactor(python): share module-accessor position key (#3542)
- Reuse the capture-side positionKey in the receiver namespace resolver
instead of a second line:col helper plus an inline template.
- Note why the synthetic { moduleScope, scopes: [] } input is sufficient
for createNamespaceTargetCache (sealed disk-backed ParsedFile shape).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* refactor(python): reuse subtype-dispatch positionKey (#3542)
- Export the existing line:col positionKey from subtype-dispatch instead of
defining an identical copy in module-accessors, so both Python capture
side channels key positions through one helper.
- Apply prettier to the Function-scope indexing line.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ci): unique test titles and rebaseline Python benches (#3542)
- impact-epistemic-lower-bound: `%j` rendered null, NaN and Infinity all as
`null`, so the test-completeness audit rejected the duplicate identities.
Use `%s` so every parameterized case has a distinct title.
- python-scope: capture fingerprint changes with the new module-accessor
capture facts (golden captures were already updated).
- receiver-resolution: +4 Python call drops (125 -> 129). The two-step
`x = f(); x.m()` form is now counted as an unresolved call instead of
vanishing: fastapi-depends `router.get/post` (unknown, x3) and
python-method-chain-binding `user.get_city()` (in-program; `User` has no
`get_city`, so the site is genuinely unresolvable).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(indexing): document native pool exhaustion recovery (U3)
Publish the reviewed recovery runbook and collapsed-write guardrails for #3526.
* docs(indexing): explain native buffer pool sizing (U3)
* fix(indexing): reject collapsed staged graph writes (U2)
* fix(indexing): abort native buffer exhaustion (U1)
* test(indexing): cover exhausted and collapsed graph writes
* ci(tests): split ubuntu coverage into 4 shards
Coverage shard 1/3 runs 20-25 minutes across branches and now hits the
25-minute job timeout (cancelled on this PR and on unrelated branches),
which skips coverage-merge and fails "every test executed" plus every
platform compatibility gate downstream. A fourth shard brings each slice
well under the cap. Shard job names are not required checks; the merge
job globs coverage-blob-*, so no other wiring changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(lbug): remove branching from buffer-exhaustion tests
Split the collapsed-staged-graph case into first-time and existing-index
tests sharing one helper, so every assertion runs unconditionally instead
of behind `if (hasExistingIndex)`. Parameterize the first-COPY exhaustion
case with its statement matcher (reusing isRelationshipCopy) rather than a
ternary on the phase name. Assertions are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Use the existing snapshot capture ceiling for buffered materialization so
valid graphs remain usable on filesystems without reflinks. Preflight both
release runtimes before paid sessions and exercise the real copy boundary
with a 513 MiB regression and the native release smoke.
* fix(deps): make tree-sitter peer installs clean
Keep the eight audited lockfile peer ranges consistent with the bundled tree-sitter 0.25.1 runtime and apply the existing manifest preparation during postinstall. Ordinary and strict npm installs resolve without peer warnings.
* docs: explain audited tree-sitter peer maintenance
* fix(deps): pin audited tree-sitter peers to 0.25.1
* fix(deps): normalize audited peers to the exact runtime
* docs: clarify exact tree-sitter runtime peer pins
* test(deps): cover exact peer pins and existing-install migration
Verify exact 0.25.1 peers, repeatable preparation, and migration from the previously widened peer metadata.
* fix(deps): support pnpm symlinks in bundle preparation (#3555)
Follow package symlinks and pnpm sibling dependency containers. Deduplicate real paths to avoid revisiting aliases or cycles while preserving the audited manifest and duplicate-instance checks.
Validated with strict npm ci, both typechecks, npm pack, a real pnpm layout, and a pnpm-installed tarball parsing all 19 grammars on main and worker threads.
* test(deps): cover pnpm layout and symlink cycles (#3555)
Exercise real package symlinks, transitive sibling dependencies, a self-cycle, metadata drift before writes, and rejection of distinct duplicate grammar instances.
The original script fails the pnpm regressions; the updated script passes all 10 bundle tests.
* chore(autofix): apply prettier + eslint fixes via /autofix command
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
lint-staged 15 pulls braces via micromatch (GHSA-vfj7-8cjw-p6xm, high).
lint-staged 16 uses picomatch instead, clearing the remaining npm audit
findings at the repo root. Node floor (>=20.17) is met by CI (22/24).
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(deps): upgrade native tree-sitter to 0.25.1
* fix(tree-sitter): preserve sync loading and update verified capture baselines
Route C# scope queries through the synchronous native grammar loader so the compiled analyze module remains require-compatible on Node 22.18. Refresh only the audited empty-Python-module and renamed-PHP-fixture captures, and pin the runtime assertion to 0.25.1.
Validated 479 targeted tests, all 16 scope-capture benchmark cases, Python capture/import guards, compiled C# worker parsing, build, typecheck and formatting. Performance budgets are unchanged.
* fix(tree-sitter): refresh upstream vendors and preserve parser semantics
* fix(ci): refresh prebuilds and support Windows ARM
Refresh macOS, Linux ARM, and Windows x64 grammar binaries from the successful native CI builds. Pin node-gyp 12.4.0 so the Windows ARM jobs recognize Visual Studio 2026.
Use local Git filtering for E2E to avoid PR-files API timeouts on generated parser diffs.
Validation: 143 focused tests, Actionlint, and all 16 scope-capture benchmark checks passed. Full npm test run started; native database suites still running.
* fix(ci): refresh Windows ARM grammar prebuilds
Import all seven Windows ARM grammar artifacts from successful native build run 37965761760. Artifact filenames and SHA-256 hashes verified; all 42 native build jobs passed.
Validation: 31 prebuild coverage tests passed.
* fix(bench): stabilize scope capture timing guard
Warm each input size seven times and use 21 median samples while retaining isolated timing blocks. Yield between languages so native finalizers release allocations outside the measured sections. Keep all fixture sizes, fingerprints, scaling formulas, and budgets unchanged.
Validation: all 16 language guards passed with a 2 GB heap limit (1.38 deep-chain ratio; 1518 MB peak RSS). Linear and quadratic controls passed their expected guard outcomes. Formatting and graph change analysis passed. Full npm test is running without failures so far.
* fix(parser): reduce TypeScript capture traversal cost
* fix(test): await parser worker readiness before teardown
* fix(parser): load C# synchronously in workers
* fix(grpc): resolve native proto detections against their own file
With the native proto grammar active, sibling .proto files that declare
the same short service name under different packages tied in the
directory-based resolveProtoConflict heuristic and both providers were
dropped. A proto-origin detection already knows its defining file, so
use that file's entry before falling back to the heuristic.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(packaging): assert native grammars load from the installed tarball
The packaged-install smoke only ran `gitnexus --version`, so a grammar
that failed to load from the published package (bundled runtime, patched
peer manifests, vendored prebuilds, C# sync loader, Protobuf wrapper)
would ship silently; the dev-tree ABI smoke also accepts vendored
grammars that report unavailable. Parse a sample for every registered
grammar plus the native Protobuf plugin from the installed package, on
the main thread and in a worker thread.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(tree-sitter): track compatibility workarounds instead of 0.25 readiness
The daily readiness report still targeted tree-sitter@0.25.0, so after the
upgrade it would keep reporting "Blocked" and never close its tracking issue.
Repurpose it to track what we carry for the bundled runtime: derive the target
from the package.json pin, list each pack-time peer patch from
prepare-tree-sitter-bundle.cjs with its drop condition (bump, wait for an
upstream release, transitive, or intentionally pinned), and reword the
buckets. The workflow adopts the issue opened under the old title.
Also update the grammar-update-monitor header and auto-PR text, which still
described the 0.21.1 runtime and a tree-sitter-c hold.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(dart): assert both nested labels target the loop; fix Zig call-shape doc
The nested-label CFG test only jumped to `outer` and checked reachability,
which still passes when a jump is retargeted to the inner loop (its exit
reaches the outer header). Jump to both labels and assert the exact
`continue` edges; dropping the inner label now fails the test.
Update the Zig callable-flow comment: call_expression now wraps actuals in a
required `arguments:` field.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Set the supported ONNX installer skip policy inside the cleared build
sandbox so optional CUDA downloads do not abort offline lifecycle scripts.
Keep CPU binaries, network containment, and all grading checks intact.
Strengthen command construction and real Bubblewrap lifecycle coverage.
Targeted tests: 114 passed, 15 skipped. Full core and real containment
validation deferred to hosted CI with maintainer approval.
* test(release): add fixed-answer native tool accuracy
* fix(release): gate stable artifacts on pinned paired evaluations
Apply review findings #1-4: gate Docker publication, grade in-flight retries, preserve repository context, and execute negative controls in CI.
* fix(eval): verify native context and resolve harness security findings
Inspect Claude's first startup reminder for ordinary repository guidance.
Bound launcher-option matching and escape Markdown backslashes and pipes.
Exclude only the parsed synthetic accuracy corpus from CodeQL; keep the
benchmark harness and evaluator under security analysis.
Validated focused Python and TypeScript regressions plus core typecheck.
* fix(eval): wake the EC2 runner and prove paired evaluator execution
Bootstrap the existing dedicated instance from a protected hosted job, bound
native runner pickup, and stop only instances started by this run. Keep paid
sessions inside the fixed stop window and add a runner-only dispatch.
Require a contained six-cell prepare/session/oracle/report canary in CI,
including a failed repair that remains in the measured denominator.
* fix(ci): reuse existing configuration and disambiguate accuracy cases
Remove the new AWS credential/settings path and release opt-in variable. Bound native runner pickup with the existing GitHub token and retain private EventBridge startup until its existing mechanism can be reused. Reuse current workflow environment names and evolution schedule controls.
Name malformed-observation cases explicitly so the execution audit distinguishes null from missing and actually tests invalid arrays.
* fix(review): isolate release builds and preserve benchmark evidence
* fix(release): gate RC publication on paired quality and manage EC2 lifecycle
* fix(ci): clarify release shell redirects and build script
* fix(eval): remove temporary patch sinks after capture
* fix(bench): retire tool-accuracy allowances repaired on main
Main at 50aa4be3b repaired three #3487 route checks (#3505) and all four
#3499 Python scope checks (#3502, #3504). The ratchet correctly failed the
merged head with stale allowances, so remove them and keep the fixed answers
as release protection: 18/31 pass, 13 recorded gaps remain.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(release): gate RCs on accuracy only and keep paid runs for stable
Follow the cost split proposed in discussion #3493: fixed-answer tool
accuracy on every RC, paired agent runs before stable releases and on a
weekly schedule against the latest RC.
RC publication no longer prepares a bundle, calls release-evaluation.yml
and waits for a paid run of up to 21 hours on the shared EC2 lock. Every
release attaches this run's accuracy report; stable npm publication still
requires exact-commit paired evidence. release-evaluation.yml drops its
workflow_call path and the now-unused candidate bundle module.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(release): address PR #3503 review feedback
- Page through recent release-evaluation runs instead of reading only
the newest 50, so valid stable evidence cannot be pushed off page one.
- Bound each runner-pickup jobs request by the remaining deadline.
- Replace non-finite floats at any depth in failure receipts so the
incomplete report still serializes with allow_nan=False.
- Clarify that the summary.cap case has 501 names over 1001 sites.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(review): harden candidate build and close review gaps
- Mount the candidate checkout's .git read-only inside the build sandbox,
so lifecycle scripts cannot plant git config (e.g. core.fsmonitor) that
host git later runs in that checkout during actions/checkout cleanup.
The real-Bubblewrap canary now also attempts that write.
- Type tool-accuracy observations instead of using explicit any.
- Cover the fail-closed branches reviewers found untested: unsafe or
missing evidence artifacts, mismatched task pins, malformed repo/SHA,
an instance that stops again during bootstrap, and unbalanced GitNexus
guidance markers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(release): describe RC accuracy gate and stable evidence step
- CONTRIBUTING now states that RCs are gated by CI (including the
fixed-answer tool-accuracy check) without a paid run, and that stable
publication needs a passing Release evaluation for the exact commit
within seven days.
- The oracle-control fixture passes tarfile's data filter only where it
exists (3.11.4+), matching the declared Python 3.11 floor.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* refactor(bench): pin tool-accuracy fixture line anchors in one place
run.ts bucketed rename edits and explain findings by line literals that
expectations.ts duplicated. Export FIXTURE_ANCHORS (file, line, pinned text)
from expectations.ts, build the fixed answers and run.ts classifiers from it,
and fail loudly (runner and unit test) when a fixture line no longer holds
the pinned text. Expected answers and observations are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(bench): fail closed when ripgrep is missing from the tool-accuracy run
rename's text-search pass shells out to rg and only logs a degraded warning
when it is absent, so a runner without ripgrep scored a degraded rename
(homonym.ts:1 missing) and still passed. Check rg --version before indexing,
record it in accuracy.json source.externalTools, and install ripgrep in the
ci-tests tool-accuracy job and the docker release-gate (same if: condition).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* refactor(eval): split baseline guidance, share bwrap preamble, drop dead bare option, harden patch cleanup
- Move the baseline (no-GitNexus) guidance scrubbing out of proposer_sandbox.py
into baseline_guidance.py, with its unit tests.
- Expose real_directory, runtime_mount_args and bwrap_base_args publicly so
release_build no longer imports private helpers or re-assembles the shared
bubblewrap preamble; the candidate build command is byte-identical.
- Remove the unused run_claude `bare` parameter and its --bare/--tools branches.
- capture_patch: a failed temporary-directory removal no longer discards the
read patch or replaces an earlier error; it is attached as a note, or warned
on stderr when there was no earlier error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(ec2-runner): keep stop observing through transient describe failures
- stop() treats a failed or timed-out describe-instances call as unknown
state and keeps observing until the stop deadline; a persistent failure
is still raised at the deadline with the last command error. Unusable
states and identity mismatches still fail at once (new CommandError
subclass separates command failures from semantic ones).
- Readiness CLI has one paid mode: --job-name NAME. --evolve is removed;
skill evolution passes its job name, and tests tie the watchdog command
to the paid job's actual name in both EC2 workflows.
- Jobs-API requests never get a timeout larger than the remaining pickup
budget (previously rounded up to 1s past the deadline).
- Document that the shared concurrency group keeps only one pending run,
so a queued release evaluation can be replaced and must be confirmed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(eval): record verified EC2 lifecycle configuration
The README still described the AWS settings as assumed and unverified.
The OIDC role, environment settings and runner_only lifecycle were
configured and verified live on 2026-10-09.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(release-eval): grade both runtimes against one pinned task toolchain
Candidate and stable each staged task dependencies (node_modules,
gitnexus-shared/dist) from their own runtime checkout, so the same v1.6.12
task source was graded with different TypeScript, Vitest and LadybugDB
versions and a per-task solve delta also measured dependency drift.
- Check out and build the pinned task commit once (tasks-base) and point
every task's repo at it for both runtimes; the runtime under test reaches
sessions only through --gitnexus-root. prepare requires --task-repo at the
single pinned task commit (new task-sha subcommand resolves it).
- Retain sandbox_dependency_content_digest per measured row and make
release_gate fail when any task's cells, across candidate and stable, do not
share one recorded dependency digest.
- Move the task-pin comparison into validate_report(task_pins=...) and use
it from release_report check and release_gate.
- Cover the gate CLI with a missing and a corrupt candidate report.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(ci): warn that a pending release evaluation can be replaced
GitHub keeps one pending run per concurrency group, so a newer queued
EC2 run cancels a pending release evaluation. Operators must confirm the
startup job ran before relying on the evidence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(eval): prove hidden oracles against the grading toolchain
Release evaluation now grades every task with dependencies built at the
pinned task commit (TypeScript 5.9.3, Vitest 4.1.11), but the oracle
controls still ran against the current checkout's toolchain. The Ubuntu
containment job now builds the pinned task toolchain and points the
controls at it. All seven controls pass locally against that build.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(ci): pin the duplicated EC2 lifecycle jobs together
Both EC2 workflows carry their own copy of the live-verified start,
check, readiness and stop jobs. A reusable workflow would change that
verified job structure, so instead a test requires the copies to stay
identical apart from the job display name and the paid-job dependency.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(release-eval): accept stable evidence only from main's current evaluator
Stable publish validated downloaded agent evidence against the release
commit's eval/ tree and never tied a report to the run that produced it.
Evidence from an older evaluator stayed acceptable for seven days after
graders changed, while task changes on main made every fresh evaluation
mismatch the tag. The first invalid artifact also hid older valid runs.
The composite action now extracts eval/ from main's head and runs the
validator there with locked base dependencies only (no dev extras or
project build). A report counts only when its harness_sha is the head_sha
of its trusted main run and the compare API shows no change under eval/,
the release-evaluation workflow or the pinned agent CLI between that
harness and the evaluator; a missing, diverged or 300-file comparison
fails closed. Runs are tried newest first, rejected runs are skipped
with a bounded reason list, every gh call has a timeout, and task pins
use validate_report's task_pins check.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(eval): break the sandbox import cycle and resolve code-scanning alerts
- baseline_guidance is now pure text handling with its own GuidanceError;
the baseline mount builder lives in proposer_sandbox again and converts
that error to SandboxError, so imports run one way only.
- task_assets uses the public real_directory name; the unused private
alias is gone.
- The oracle-control fixture always extracts with tarfile's data filter
and skips on Pythons that lack it, instead of extracting unfiltered.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(release-eval): run candidate lifecycle scripts offline; reject smuggled report fields
- build_candidate now downloads locked dependencies with --ignore-scripts,
then runs every lifecycle script (dependency installs, prepare, build)
in a fresh network namespace, so candidate code cannot reach host-local
services. Verified locally that the current head and the pinned task
commit build this way; the real-Bubblewrap canary now asserts only
loopback is visible to a lifecycle script.
- validate_report requires the evidence document to equal its
field-whitelisted rebuild, so per-run or top-level fields outside the
published schema are rejected before the publisher copies the file.
- The sandbox test suite also covers a no-GitNexus sandbox refusing
unbalanced guidance markers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* docs(eval): describe the offline candidate lifecycle
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(release-eval): pin downloads to the npm registry; require dependency binding
Address review feedback on PR #3503:
- The candidate's online install phase now refuses lockfile entries that
resolve outside https://registry.npmjs.org/ (other than in-checkout
workspace links) and refuses shipped .npmrc files, so a candidate
cannot steer npm at loopback, private or metadata endpoints.
- Every measured row must carry a 64-hex task-dependency digest, so a
single report cannot be accepted with its grading toolchain unbound.
- Oversized JSON integers mark a measurement untrustworthy instead of
crashing report generation.
- Baseline guidance stripping recognises headings with up to three
leading spaces (CommonMark).
- Remove a stale comment about the old private helper name.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(eval): parse CommonMark ATX headings; compare evidence by exact JSON
- Baseline guidance stripping now parses headings as CommonMark ATX
headings (0-3 leading spaces, optional closing # run, tabs), so a
GitNexus section heading like '## GitNexus rules ##' is removed too.
- Evidence validation compares the serialized rebuild with the report,
so a retyped value (true for 1, 2.0 for 2) no longer passes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(eval): drop continuations of every CommonMark list item in baseline guidance
A removed GitNexus instruction left its indented continuation behind when
the item used '+' or an ordered marker (1. / 1)). The scrubber now
recognises every CommonMark list-item marker.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(eval): isolate candidate MCP from agent task writes (#3503)
Run MCP in a nested Bubblewrap boundary with read-only task, graph,
registry and runtime mounts, private state and isolated processes/network.
Keep task writes on the agent's built-in tools and remove mutating MCP grants.
Add startup/tool mutation canaries and preserve explicit unsafe diagnostics.
Validation: focused pytest 138 passed, 16 skipped. Full locked evaluator
996 passed, 29 skipped; real containment canaries require Ubuntu CI.
Note: eight pre-existing comparator-reuse failures reproduce on the
unchanged head due to missing os.supports_dir_fd support. One process-control
timeout failed in the full run and passed alone and on the unchanged head.
* fix(eval): close baseline mount overlap and test symlink evidence (#3503)
Reject supplied no-MCP mounts that cover forbidden GitNexus paths, including ancestor mounts and lexical aliases. Keep ordinary dependency mounts available.
Exercise evidence symlink rejection with a real valid-report target and retain dangling-link coverage.
Validation: 1026 evaluator tests passed, 29 skipped; targeted mount tests 36 passed and evidence tests 34 passed. Removing the symlink guard in memory makes the repaired regression fail. Ruff and diff checks passed.
Note: eight pre-existing comparator-reuse failures remain in this environment because os.supports_dir_fd lacks the required os.lstat support.
---------
Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
rIsCallableVisibleFromCaller rejects every dotted qualified name, but the
free-call fallback keys defs by the text after the last dot, so a call spelled
helper.util never reaches the hook and binds in the qualified-name pass instead.
Pin that for same-package, cross-package, library() and no-import callers; the
edges are unchanged with the hook disabled.
The linear-regex parser refused any pattern whose text contained "(?<", so
[(?<] and \(?<x were dropped as look-behind. Reject at the group the parser
reaches, which is never inside a class or after a backslash. A real look-behind
or named group is still refused, after any class, with the same linear parse.
S3method(generic, class, method) registers method, not generic.class, as the S3
method. parseRNamespaceExports ignored the third argument and so marked a
function called generic.class public while the registered method stayed
non-exported. Use the third argument when it is a plain name, else generic.class.
R matches named arguments first, so in setClass(a = "Q", "B", Class = "A") the
class is A. The naming-argument scan stopped at the first unnamed argument and
named both B and A. Keep scanning the list, once and cached, and drop the choice
when an argument spelled with the formal appears. Replaces the pinned known-limit
test and the known-limit wording in the file header and the provider guide.
getRTopLevelMethodOwnerName counted string-valued arguments, so a named where= or
valueClass= string became the setMethod class, and extractRefClassMethods took the
first string, so contains = "Base" ahead of Class = "Acc" named Base. Match the
signature argument with R's rule (named first, then positional into the remaining
formals) and the RefClass name with the Class formal.
declaredClassName and the new("Class") constructor scanner took the first
argument, so setClass(contains = "Base", Class = "Derived") anchored roxygen
types to Base and new(name = "x", "A") typed nothing. Select the class with
getRClassNameArgument (isRNamingArgument(arg, 'Class')), the choice the scope
query, definition hook and field extractor already make.
Embed the module path in the child-process script as a file URL, and
replace the chmod(0o000) tests with injected EACCES read failures so the
unreadable-means-truncated behaviour is exercised on every OS.
An unreadable DESCRIPTION left RPackageConfig.truncated false, so a
qualifier naming the package it declares was classed external and its
call site dropped. The sibling hidesUndiscoveredPackage case already
treats an unreadable DESCRIPTION as unknown; do the same here and cover
it with a test.
Tighten three weak test assertions: the escaped-bracket export pattern
now asserts its exact match results, the S4 anchors test proves each
generic is defined exactly once, and the worker owner-absence test
fails on any defined ownerId, including an empty string.
extractS4Fields and getRTopLevelPropertyOwnerName named the owner of a
setClass/setRefClass slot from the first string-valued argument, so
setClass(contains = "Base", Class = "A", representation(x = "numeric"))
attached slot x to Base. Select the owner with isRNamingArgument(arg, 'Class'),
the same choice the scope query and definition hook make.
Both owner paths share one helper so the field metadata and the ownerNameHint
agree.
