GitNexus/eval/tests
Gergo Magyar b97ad89f38 fix(eval): address PR review feedback (#3207)
- aggregate: count admissible rows directly instead of subtracting the
  execution and evidence counters, which double-charged a row that is both
  a session error and invalid review evidence and could report UNUSABLE for
  an arm holding real measurements.
- run_proposer: bound the session timeout by what is left of
  --max-runtime-seconds, so clearing the sweep minimum cannot start a
  full-length session past the instance window.
- comparator reuse: hold one O_NOFOLLOW descriptor for the size check,
  digest and copy, and prove it is the inode that was checked, closing the
  swap window a concurrent writer of the reuse directory had.
- Drive the review-artifact mount assertion through run_arm and the
  clone-template assertion through run_cell, instead of rebuilding the
  expected values in the tests (also removes the CodeQL unnecessary lambda).
- Assert the workflow invokes run-evolution.sh rather than that its YAML
  mentions --max-runtime-seconds, which only appears in a comment.
- Correct the parse_review_output failure-mode claim: the fold was empty
  artifacts reported as "not valid UTF-8 JSON"; a never-created file raised
  FileNotFoundError.
- prettier: wrap the over-long readFileSync call flagged by PR autofix.

Note: pre-existing failure in tests/test_model_gateway.py::test_locked_litellm_translates_messages_to_offline_responses (local LiteLLM proxy never becomes ready in this environment) not addressed by this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 17:02:37 +00:00
..
__init__.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
conftest.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_ce_plugin_runtime.py feat(skills): GitNexus Engineering Tool Kits (#2566) 2026-07-19 15:07:24 +01:00
test_comparator_reuse.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_errors.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_evolve.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_mcp_bridge.py feat(eval): evolve review skills against historical PRs 2026-09-04 05:32:31 +00:00
test_model_gateway.py fix(eval): require finite gateway startup budgets 2026-09-05 11:03:23 +00:00
test_oracle_assets.py fix(eval): close CI and remaining review gaps 2026-09-05 10:29:58 +00:00
test_parse_run_id.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_process_control.py fix(eval): close CI and remaining review gaps 2026-09-05 10:29:58 +00:00
test_promotion_apply.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_property_based.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_proposer_sandbox.py fix(review): close the artifact path in every layer that gates it 2026-09-07 14:21:15 +00:00
test_review_corpus.py fix(eval): cut skill-evolution wall clock without shrinking the gate 2026-09-07 14:21:14 +00:00
test_review_scoring.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_runner_hardening.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_sanitized_graph.py fix(eval): cut skill-evolution wall clock without shrinking the gate 2026-09-07 14:21:14 +00:00
test_session_progress.py fix(eval): cut skill-evolution wall clock without shrinking the gate 2026-09-07 14:21:14 +00:00
test_task_assets.py Address PR review feedback (#2785) 2026-09-04 18:59:32 +00:00
test_tool_scripts.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_workflow_bench.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00
test_workflow_bench_evolution.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_workflow_bench_sessions.py fix(eval): address PR review feedback (#3207) 2026-09-07 17:02:37 +00:00