- close_low_quality_prs.py: Treat author_association API lookup failures
as internal (fail-safe) so transient errors don't cause internal
contributors' PRs to be auto-closed.
- triage_with_llm.py: Update summary heading from 'Would post comment:'
to 'Posted comment:' since this branch only runs after the comment
has already been posted.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
- close_low_quality_prs.yml: only workflow_dispatch with close=true (and
AGENT_SHIN_ENABLED=true) actually closes PRs. Scheduled runs are always
dry-run, matching the safety invariant documented for triage_pr/issue.
- triage_with_llm.py: textwrap.dedent on an f-string with multi-line
interpolated bodies fails because the body's 2nd+ lines start at column 0,
making the common-indent zero. Dedent the static template first, then
.format() the title/body in.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
- close_low_quality_prs.py now filters by GitHub author_association via
the REST API: PRs from OWNER / MEMBER / COLLABORATOR (and bot accounts)
are skipped with a new 'skip-internal' summary bucket.
- close_low_quality_prs.yml now defaults workflow_dispatch close=false,
and ignores 'close=true' unless the new repo variable
AGENT_SHIN_ENABLED is set to 'true'. Scheduled runs are dry-run only
until the team flips that switch.
- Updated unit tests: one new test asserting internal authors are
skipped, and an autouse fixture treats unspecified test PRs as
external so the rest of the suite still exercises the close path.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Adds a new triage flow that evaluates external pull requests and issues
against the project's contribution rubric and, when configured to do so,
auto-closes non-conforming ones with an explanatory comment. Contributors
can update + reopen to be re-evaluated.
Scope:
- Internal BerriAI contributors (author_association OWNER/MEMBER/COLLABORATOR)
and bot accounts are skipped entirely.
- 'Fixes #1234' / 'Resolves https://github.com/.../issues/N' in the PR body
short-circuits to PASS without burning LLM tokens.
- LLM judge returns structured JSON (verdict, missing[], explanation);
parser tolerates markdown fences and embedded JSON.
- LLM errors NEVER close PRs/issues — failure surfaces as 'skip-llm-error'.
Safety:
- pull_request_target / issues triggers are FORCED dry-run in the workflow;
only manual workflow_dispatch with close=true (and AGENT_SHIN_ENABLED=true)
takes destructive action.
- Default mode writes verdicts to GITHUB_STEP_SUMMARY only — no public
comments until the team flips the AGENT_SHIN_ENABLED repo variable.
- LLM uses an OpenAI-compatible endpoint (model and base URL configurable
via repo variables; key via OPENAI_API_KEY secret).
Files:
- .github/scripts/triage_with_llm.py - judge orchestrator + CLI
- .github/workflows/triage_pr_with_llm.yml
- .github/workflows/triage_issue_with_llm.yml
- tests/test_litellm/test_github_triage_with_llm.py - 33 unit tests
End-to-end validated against four real PRs (#28117 internal collaborator,
#28108 bot, #28129 'Fixes #28128', #28116 no linked issue) and issue
#28132 with a stubbed LLM judge: each path produces the expected action.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Adds .github/scripts/close_low_quality_prs.py and a daily workflow that
closes PRs which:
- are open for at least 7 days, and
- carry a most-recent greptile-apps review with Confidence Score <4/5,
- and are not drafts or opt-out-labeled ('do not close', 'wip', etc.).
Each closure posts an explanatory comment telling the contributor how to
bring the PR back (rebase, re-request greptile, reopen at 4+/5). The
4/5 bar is already documented in the PR template
(.github/pull_request_template.md), so this just enforces it.
Tested with a dry run against the live BerriAI/litellm backlog of 1000
open PRs: 100 candidates identified, 598 PRs pass the bar (4+/5), 186
are too young, 97 are drafts, 19 lack any Greptile review and are left
alone.
Workflow defaults to closing 25 PRs/run as a safety net and supports
workflow_dispatch with overrides (close=false for a dry run, custom
min_age_days/min_score/limit).
18 unit tests cover score extraction (HTML/markdown/plain text, login
variants, multi-review picks latest) and per-PR evaluation (drafts,
opt-out labels, age, missing/passing/failing scores).
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Add a Python script that detects duplicate issues using title similarity
(difflib.SequenceMatcher) and closes them via the gh CLI. Two-tier system:
- 0.6 threshold: informational comment via existing wow-actions step
- 0.85 threshold: auto-close with comment, label, and not_planned reason
Includes a workflow_dispatch workflow for one-time batch scans and
integrates auto-close into the existing check_duplicate_issues workflow
for newly opened issues.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add auto issue labelling for llm_translation
* Add logging
* make the formatting better
* make the formatting better
* make the formatting better
* make the formatting better
