Follow-up to PR #28117. Three behavior changes + one new workflow,
addressing the team's concerns on the original review:
1) Apply auto-close to ALL open PRs, not just those over a week old.
- close_low_quality_prs.py: --min-age-days default flipped from 7 to
0. The flag is preserved as an opt-in safety net for one-off
backfill runs that want to spare very-young PRs, but the daily
scheduled sweep now closes external-author PRs as soon as Greptile
scores them <4/5.
- close_low_quality_prs.yml: workflow_dispatch input default also
flipped to 0; doc comments updated.
2) Apply auto-close to draft PRs too.
- close_low_quality_prs.py: removed the skip-draft branch in
evaluate_pr. Drafts are NOT a free pass — the team's intent is
'open PR count == PRs internal collaborators need to action on',
so a draft Greptile scored 2/5 still belongs in the closed bucket.
Authors who genuinely need a long-lived draft can attach the 'wip'
opt-out label, which is unchanged.
- The 'skip-draft' action is gone; the 'wip' label still skips.
3) Address the 'OSS contributors cannot reopen a bot-closed PR' wrinkle.
GitHub does NOT let an external (non-write-access) contributor
reopen a PR that was closed by a bot or maintainer (long-standing
limitation). The original PR's close-comments told contributors to
'Reopen the PR — I'll re-evaluate automatically', which is broken
for the very audience this triage targets. Two changes:
a) Reword every close-comment (Greptile sweep + Agent Shin PR
close + Agent Shin issue close + PR template) to recommend:
- Open a new PR with the updated branch (primary path).
- Or comment '@agent-shin reconsider' on the closed PR for a
re-evaluation that, on pass, reopens the PR via the bot's
GH_TOKEN write access.
b) Add the @agent-shin reconsider workflow:
- .github/workflows/triage_reconsider.yml: new
'issue_comment'-triggered workflow. Authorizes only the
PR/issue author or an internal collaborator
(OWNER/MEMBER/COLLABORATOR), gated via a step output so
unauthorized commenters never reach the destructive steps.
Globally gated on AGENT_SHIN_ENABLED='true' (positive form,
matching the test_github_triage_workflows guardrail
patterns).
- triage_with_llm.py: --reconsider mode. On a closed PR/issue,
re-runs the LLM judge (or linked-issue regex short-circuit)
and:
- on pass: reopens via reopen_pr/reopen_issue + posts a
'Re-evaluated and reopened' comment.
- on fail: leaves closed and posts a 'still missing X'
comment so the contributor can iterate again.
Reconsider-on-open is a no-op ('skip-not-closed').
Internal-author + bot-account skips still take priority over
reconsider.
4) Greptile-on-closed-PRs question: the team asked whether Greptile can
re-review a closed PR. Greptile's docs don't address this and we
shouldn't promise behavior we can't verify, so the new close-comment
wording does NOT instruct contributors to 're-request greptile on
the closed PR'. Instead it points them at the new-PR path (which
Greptile definitely reviews) or the @agent-shin reconsider trigger
(which re-runs the LiteLLM-side rubric judge, not Greptile).
Tests: 93 passing (was 59).
- test_github_close_low_quality_prs.py: replaced 'skip drafts' test
with 'closes drafts when score is low' + 'closes brand-new PR when
min_age=0' + 'no skip when min_age=0'. The 'skip too young'
assertion is preserved as opt-in.
- test_github_triage_with_llm.py: 6 new TestTriageOrchestration cases
for reconsider mode (skip-not-closed on open, reopen on pass,
still-failing comment on fail, linked-issue short-circuit reopen,
skip internal author in reconsider, reopen-issue on pass) + a new
TestCloseCommentText class that pins the user-facing 'open a new
PR' + '@agent-shin reconsider' wording.
- test_github_triage_workflows.py: added triage_reconsider.yml to
the destructive-gate guardrail table; AGENT_SHIN_ENABLED is its
own destructive gate (no separate per-run flag needed).
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
PR template:
- Make the rubric explicit at the top: link an issue, OR provide a clear
problem description + expected vs. actual + visual QA proof.
- Add dedicated sections for each piece so the bot has a deterministic
shape to read.
- Keep the existing 'Linear ticket' section for internal contributors
(they're exempt from the auto-triage rubric).
Bug report template:
- Split 'What happened?' into 'Actual behavior' + 'Expected behavior'.
- Make logs/screenshot a required textarea.
- Warning banner at the top tells external contributors that incomplete
reports will be auto-closed (with re-evaluation on reopen).
Feature request template:
- Require a concrete use case + example in the motivation field, not just
a one-liner pitch.
- Same auto-triage warning banner.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
