GitNexus/eval/tests
Gergo Magyar f0cdc9e776 fix(eval): close the runtime-cap gap and pin the reuse directory
Both were left open on #3207 as approach decisions rather than nits.

Runtime cap: run-evolution.sh computed the budget in its own
`uv run python -c` and passed a number, so the script's remaining
provenance work and the CLI's own startup were spent by nobody and charged
to the sweep — out of the upload reserve the cap exists to protect. The
script now passes --max-runtime-from-instance-window and evolve reads
/proc/uptime itself, on the line after it starts the clock the budget is
measured against, so no interval exists to lose. Also removes an
interpreter start from the script and lets --dry-run print the real argv.

Reuse directory: _real_child_directory lstat-checked `transcripts` and
returned its pathname, so a concurrent writer could rename the directory
and leave a symlink before the name was used again — O_NOFOLLOW guards
only the leaf. Every artifact is now resolved against a held descriptor:
_open_real_directory opens with O_DIRECTORY|O_NOFOLLOW (check and open in
one syscall), and _open_regular / _copy_owner_only take dir_fd. The reuse
path is therefore POSIX-only; _require_openat says so and fails closed,
which the runner already treats as "run a paid cell". _resolved_directory
still tolerates a symlinked reuse root, unchanged and still tested.

evolution._require_directory_chain is still lstat-per-component. It guards
a different surface (candidate overlay reads) that neither review raised,
so it is left alone rather than widened into here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 17:39:32 +00:00
..
__init__.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
conftest.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_ce_plugin_runtime.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_comparator_reuse.py fix(eval): close the runtime-cap gap and pin the reuse directory 2026-09-07 17:39:32 +00:00
test_errors.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_evolve.py fix(eval): close the runtime-cap gap and pin the reuse directory 2026-09-07 17:39:32 +00:00
test_mcp_bridge.py feat(eval): evolve review skills against historical PRs 2026-09-04 05:32:31 +00:00
test_model_gateway.py fix(eval): require finite gateway startup budgets 2026-09-05 11:03:23 +00:00
test_oracle_assets.py fix(eval): close CI and remaining review gaps 2026-09-05 10:29:58 +00:00
test_parse_run_id.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_process_control.py fix(eval): close CI and remaining review gaps 2026-09-05 10:29:58 +00:00
test_promotion_apply.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_property_based.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_proposer_sandbox.py fix(eval): address the second round of PR review feedback (#3207) 2026-09-07 17:14:58 +00:00
test_review_corpus.py fix(eval): cut skill-evolution wall clock without shrinking the gate 2026-09-07 14:21:14 +00:00
test_review_scoring.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_runner_hardening.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_sanitized_graph.py fix(eval): cut skill-evolution wall clock without shrinking the gate 2026-09-07 14:21:14 +00:00
test_session_progress.py fix(eval): cut skill-evolution wall clock without shrinking the gate 2026-09-07 14:21:14 +00:00
test_task_assets.py Address PR review feedback (#2785) 2026-09-04 18:59:32 +00:00
test_tool_scripts.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_workflow_bench.py fix(eval): address the second round of PR review feedback (#3207) 2026-09-07 17:14:58 +00:00
test_workflow_bench_evolution.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_workflow_bench_sessions.py fix(eval): address the second round of PR review feedback (#3207) 2026-09-07 17:14:58 +00:00