GitNexus/eval/tests
Gergő Magyar 62f86ea068
feat(mcp): reduce response size and query overhead (#3621)
* docs(mcp): document focused response filters

* feat(mcp): add opt-in response section and relation filters

* test(mcp): verify response filters and compatibility

* test(mcp): add deterministic response-size benchmark

* docs(skills): teach focused MCP calls and safe filter use

* docs(skills): update debugging response-filter guidance

* docs(skills): update exploration response-filter guidance

* docs(skills): explain focused MCP response options

* docs(plugin): mirror focused debugging guidance

* docs(plugin): mirror focused exploration guidance

* docs(plugin): mirror focused MCP tool guide

* docs(cursor): mirror focused debugging guidance

* docs(cursor): mirror focused exploration guidance

* test(mcp): give filter cases unique identities

Use named cases to keep array inputs intact and let the CI execution audit distinguish every malformed-filter test.

* perf(mcp): batch BM25 symbol hydration

* test(mcp): serialize native hydration coverage

* test(eval): publish child PID atomically

* test(mcp): cover hydration on native platforms

---------

Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com>
Co-authored-by: Abhigyan Patwari <126312502+abhigyanpatwari@users.noreply.github.com>
2026-10-18 17:32:09 +01:00
..
fixtures test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
__init__.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
bench_fixtures.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
conftest.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_baseline_guidance.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_ce_plugin_runtime.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_check_test_execution.py fix(ci): require every test to execute across the CI matrix (#3479) 2026-10-06 07:34:17 +03:00
test_comparator_reuse.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_ec2_workflows.py feat(eval): request PR cost comparisons by label (#3622) 2026-10-18 11:00:08 +01:00
test_errors.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_evolution_runner.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_evolution_runner_ready.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_evolve.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_mcp_bridge.py feat(eval): evolve review skills against historical PRs 2026-09-04 05:32:31 +00:00
test_measure_evolution_cost.py feat(eval): Add bounded packed-scheduler primitives and offline replay benchmarks (#3206) 2026-09-08 08:22:12 +01:00
test_mock_provider.py test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
test_model_gateway.py fix(eval): require finite gateway startup budgets 2026-09-05 11:03:23 +00:00
test_offline_session_integration.py test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
test_offline_sweep_integration.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_oracle_assets.py fix(eval): grade real C include provenance (#3611) 2026-10-17 15:03:42 +01:00
test_parse_run_id.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_pr_evaluation.py feat(eval): request PR cost comparisons by label (#3622) 2026-10-18 11:00:08 +01:00
test_process_control.py feat(mcp): reduce response size and query overhead (#3621) 2026-10-18 17:32:09 +01:00
test_promotion_apply.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_property_based.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_proposer_sandbox.py fix(eval): build release dependencies offline without CUDA downloads (#3541) 2026-10-10 10:05:17 +01:00
test_provider_usage.py feat(eval): record provider-native usage at the gateway instead of inferring it after translation (#3220) 2026-09-08 18:22:04 +01:00
test_provider_usage_capture.py test(eval): run the benchmark offline against a scripted provider (#3235) 2026-09-09 12:34:00 +01:00
test_release_build.py fix(eval): build release dependencies offline without CUDA downloads (#3541) 2026-10-10 10:05:17 +01:00
test_release_evaluation_smoke.py fix(eval): materialize release graphs before paid sessions (#3556) 2026-10-10 15:24:09 +00:00
test_release_evidence.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_release_gate.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_release_preflight.py fix(eval): materialize release graphs before paid sessions (#3556) 2026-10-10 15:24:09 +00:00
test_release_report.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_reuse_round_trip.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_review_corpus.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_review_scoring.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_runner_hardening.py fix(release): gate publication on accuracy and paired evaluation (#3503) 2026-10-10 08:26:08 +01:00
test_sanitized_graph.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_session_progress.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_sweep_finalization.py fix(eval): sweep evidence handling and measurement health, with guarded comparator reuse (#3207) 2026-09-08 09:25:45 +00:00
test_task_assets.py fix(eval): materialize release graphs before paid sessions (#3556) 2026-10-10 15:24:09 +00:00
test_tool_scripts.py docs: agent development framework, GitHub templates, eval refactor (#479) 2026-03-25 06:48:41 +00:00
test_workflow_bench.py fix(eval): grade real C include provenance (#3611) 2026-10-17 15:03:42 +01:00
test_workflow_bench_evolution.py fix(eval): make evolution evidence valid and bounded 2026-09-05 10:08:35 +00:00
test_workflow_bench_sessions.py fix(eval): account for background completion turns (#3604) 2026-10-16 17:03:28 +00:00