mirror of
https://github.com/abhigyanpatwari/GitNexus.git
synced 2026-11-16 09:22:00 +00:00
* feat(eval): add cost diagnostics and trials * style(eval): format retrieval case fixtures * fix(eval): address review and CI failures Refs #3773. Validate locked uv commands, model aliases and oversized budgets. Stop the fake MCP server before releasing its listener and replace the wall-clock staleness assertion with a concurrent result check. Note: pre-existing failure in test_oracle_assets.py reference-checkout type resolution not addressed by this PR. * feat(eval): publish detailed run measurements Collect per-sweep provider ledgers and publish sanitized measurements in the Actions summary and artifacts. Validate experiment tasks with the runner and withhold incomplete token totals. Refs #3773. Note: pre-existing failure in test_oracle_assets.py reference-checkout type resolution not addressed by this PR. * fix(eval): validate inputs and quoted commands Reject non-mapping task YAML before reading fields. Attribute literal quoted CLI queries while preserving Bash attribution for shell operators and expansions. Refs #3773. * fix(eval): count empty filtered query results * fix(eval): handle alternate event content --------- Co-authored-by: Gergo Magyar <gergomagyar0@gmail.com> |
||
|---|---|---|
| .. | ||
| fixtures | ||
| helpers | ||
| integration | ||
| unit | ||
| utils | ||