GitNexus/eval/tests
Gergő Magyar cb5a8e242f fix(eval): bind the resolved node binary into the sandbox, not a hardcoded host path (#2607)
Surfaced by the first real workflow_dispatch run on the self-hosted runner
(https://github.com/abhigyanpatwari/GitNexus/actions/runs/29836411744):
every session failed with error_kind infra-error, error_detail "bwrap:
execvp /usr/local/bin/node: No such file or directory", tripping the
outage-streak breaker after 5 consecutive failures.

sanitized_graph.py and runner_sessions.py invoke the sandboxed graph CLI
at the fixed path /usr/local/bin/node. _runtime_mount_args only binds
/usr, /bin, /lib, /lib64 wholesale, so that path resolves correctly when
node happens to live under /usr/local/bin on the host -- true on
GitHub-hosted runner images, but not on a self-hosted runner, where
actions/setup-node installs into its own tool-cache directory instead
(outside all four bound trees, so invisible to the sandbox regardless of
what PATH says on the host).

Fix lives entirely in the mount construction: resolve `node` via
shutil.which (correctly picks up wherever actions/setup-node put it,
since its tool-cache dir is already on PATH by the time this runs) and
bind it read-only to the same fixed sandbox path the two call sites
already expect. Neither call site needed to change. Backward compatible
with GitHub-hosted runners, where this resolves to the same path and
binds a harmless no-op self-mount.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 15:39:56 +01:00
..
__init__.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
conftest.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_ce_plugin_runtime.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_errors.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_evolve.py feat(eval): run the skill-evolution loop online (#2571) 2026-07-20 05:37:07 +01:00
test_mcp_bridge.py fix: start MCP bridge correctly when using npx (#1114) 2026-04-27 18:19:02 +01:00
test_oracle_assets.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_parse_run_id.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_process_control.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_promotion_apply.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_property_based.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_proposer_sandbox.py fix(eval): bind the resolved node binary into the sandbox, not a hardcoded host path (#2607) 2026-07-21 15:39:56 +01:00
test_runner_hardening.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_sanitized_graph.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_task_assets.py fix(eval): size the buffered-fallback budget for the real graph index 2026-07-20 14:42:50 +00:00
test_tool_scripts.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_workflow_bench.py fix(eval): self-hosted skill-evolution runner + sandbox Python 3 trust fix (#2600) 2026-07-21 14:40:20 +01:00
test_workflow_bench_evolution.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_workflow_bench_sessions.py fix(eval): stop using --bare for arms that need Skill or MCP tools (#2584) 2026-07-20 21:26:11 +01:00