GitNexus/eval/tests
Neutron c2d9012b58
Some checks are pending
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Gitleaks / gitleaks (push) Waiting to run
Publish / Classify release event (push) Waiting to run
Publish / RC guard (marker + release-PR skip) (push) Blocked by required conditions
Publish / ci (push) Blocked by required conditions
Publish / Publish to npm (push) Blocked by required conditions
Publish / Build & Push RC Docker images (push) Blocked by required conditions
Scorecard / Scorecard analysis (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-cli) (push) Waiting to run
Trivy Image Scan / Trivy (gitnexus-web) (push) Waiting to run
fix(lbug): guard read queries against runaway native memory (#3696) (#3749)
* fix(lbug): guard read queries against runaway native memory (#3696)

A single impact query was reported to allocate ~184 GB of native heap on a
66k-symbol index. That memory is outside the buffer pool and V8, so neither
bufferManagerSize nor --max-old-space-size bounds it, and the Node binding
cannot interrupt a running query.

- Add a process-wide memory guard around every pooled read query. While a
  query runs it samples RSS every 200 ms and, on macOS, the physical
  footprint for queries running over 1 s (RSS excludes compressed and
  swapped pages there). Past the budget it names the running queries on
  stderr and exits with code 70 instead of letting the host run out of
  memory. GITNEXUS_QUERY_MAX_RSS_MB sets the limit (0 disables); the
  default is max(8 GiB, 25% of RAM).
- Add GITNEXUS_QUERY_TRACE=1: one stderr line per query with duration,
  rows and memory, so a reporter can identify the query that blows up.
- Cap the impact BFS frontier query at 100k rows per depth level. A capped
  level stops the walk, marks the result partial, and cannot report a
  LOW/MEDIUM risk.

* fix(ci): initialize shards after fork approval

GitHub approval increments the attempt number without running any jobs. Allow initial timing snapshot creation on attempt 2 only when the first attempt required approval and had zero jobs. Keep real retries fail-closed on missing frozen input.

* fix(impact): qualify both capped risk scores

Apply the same conservative cap policy to risk and riskSharedAxes, retain proven high warnings, and strengthen frontier-limit regression assertions. Correct the memory-budget test description.

* fix: skip query phases after deadline

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
2026-10-28 14:08:17 +00:00
..
fixtures test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
__init__.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
bench_fixtures.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
conftest.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_baseline_guidance.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_ce_plugin_runtime.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_check_test_execution.py fix(ci): require every test to execute across the CI matrix (#3479) 2026-10-06 07:34:17 +03:00
test_comparator_reuse.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_ec2_workflows.py fix(eval): reuse reference settings for PR cost previews (#3738) 2026-10-27 09:17:08 +00:00
test_errors.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_evolution_runner.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_evolution_runner_ready.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_evolve.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_mcp_bridge.py feat(eval): evolve review skills against historical PRs 2026-09-04 05:32:31 +00:00
test_measure_evolution_cost.py feat(eval): Add bounded packed-scheduler primitives and offline replay benchmarks (#3206) 2026-09-08 08:22:12 +01:00
test_mock_provider.py test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
test_model_gateway.py fix(eval): require finite gateway startup budgets 2026-09-05 11:03:23 +00:00
test_offline_session_integration.py test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
test_offline_sweep_integration.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_oracle_assets.py fix(eval): grade real C include provenance (#3611) 2026-10-17 15:03:42 +01:00
test_parse_run_id.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_pr_evaluation.py fix(eval): reuse reference settings for PR cost previews (#3738) 2026-10-27 09:17:08 +00:00
test_process_control.py feat(mcp): reduce response size and query overhead (#3621) 2026-10-18 17:32:09 +01:00
test_promotion_apply.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_property_based.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_proposer_sandbox.py fix(eval): keep candidate code out of agent comparisons (#3715) 2026-10-25 05:01:08 +00:00
test_provider_usage.py feat(eval): record provider-native usage at the gateway instead of inferring it after translation (#3220) 2026-09-08 18:22:04 +01:00
test_provider_usage_capture.py test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
test_release_build.py fix(eval): build release dependencies offline without CUDA downloads (#3541) 2026-10-10 10:05:17 +01:00
test_release_evaluation_smoke.py fix(eval): materialize release graphs before paid sessions (#3556) 2026-10-10 15:24:09 +00:00
test_release_evidence.py fix(eval): keep candidate code out of agent comparisons (#3715) 2026-10-25 05:01:08 +00:00
test_release_gate.py fix(eval): keep candidate code out of agent comparisons (#3715) 2026-10-25 05:01:08 +00:00
test_release_preflight.py fix(eval): materialize release graphs before paid sessions (#3556) 2026-10-10 15:24:09 +00:00
test_release_report.py fix(eval): keep candidate code out of agent comparisons (#3715) 2026-10-25 05:01:08 +00:00
test_reuse_round_trip.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_review_corpus.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_review_scoring.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_runner_hardening.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_sanitized_graph.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_session_progress.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_sweep_finalization.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_task_assets.py fix(eval): materialize release graphs before paid sessions (#3556) 2026-10-10 15:24:09 +00:00
test_tool_scripts.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_workflow_bench.py fix(lbug): guard read queries against runaway native memory (#3696) (#3749) 2026-10-28 14:08:17 +00:00
test_workflow_bench_evolution.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_workflow_bench_sessions.py fix(eval): retain large session measurements (#3722) 2026-10-25 18:30:17 +00:00