diff --git a/skills/strix-ci-setup/SKILL.md b/skills/strix-ci-setup/SKILL.md index 8ba8b200a..3efa2f843 100644 --- a/skills/strix-ci-setup/SKILL.md +++ b/skills/strix-ci-setup/SKILL.md @@ -53,9 +53,10 @@ jobs: Then tell the user to add two repository secrets: `STRIX_LLM` (model id, e.g. `openai/gpt-5.4`) and `LLM_API_KEY` (the provider key). Do not create these values yourself. Notes: -- In CI/headless runs Strix automatically scopes to the PR's changed files (`--scope-mode auto`). If diff resolution fails, keep `fetch-depth: 0` or pass `--diff-base origin/main`. +- In CI/headless runs Strix automatically scopes to the PR's changed files (`--scope-mode auto`). If diff resolution fails, keep `fetch-depth: 0` or set `--diff-base` to the PR's actual base branch — use `origin/${{ github.base_ref }}` in GitHub Actions rather than a hard-coded `origin/main`, since repos use different default branches. - Exit codes: `0` pass, `2` vulnerabilities found (fails the job), `1` setup error. - The runner needs Docker (default GitHub-hosted Ubuntu runners have it). +- **Size the budget so the scan completes — don't let it fail open.** A `0` exit means "no validated vulnerabilities in what was analyzed"; if `--max-budget` is hit before the diff is fully covered, the scan wraps up early and can exit `0` on code it never finished testing. For a `quick` diff-scoped PR scan `--max-budget 10` is usually ample, but for large diffs raise it, and don't treat a green build as a guarantee of full coverage. To be strict, fail the build unless `strix_runs//run.json` reports a completed (non-budget-truncated) status. ### Optional: upload findings to GitHub code scanning @@ -75,10 +76,13 @@ Any pipeline works the same way — install, set the two env vars, run headless: ```bash curl -sSL https://strix.ai/install | bash -strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base origin/main --max-budget 10 +# Resolve the PR's base branch (fall back to the remote default, then main). +DIFF_BASE="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-}" +DIFF_BASE="origin/${DIFF_BASE:-$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null | sed 's#^origin/##' || echo main)}" +strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base "$DIFF_BASE" --max-budget 10 ``` -Gate the pipeline on the exit code. Schedule `standard` scans nightly and `deep` scans for release candidates. +Gate the pipeline on the exit code (see the budget/fail-open caveat above — give the scan enough budget to finish). Schedule `standard` scans nightly and `deep` scans for release candidates. --- diff --git a/skills/strix-fix-findings/SKILL.md b/skills/strix-fix-findings/SKILL.md index b25a0021e..a08749224 100644 --- a/skills/strix-fix-findings/SKILL.md +++ b/skills/strix-fix-findings/SKILL.md @@ -39,13 +39,15 @@ After fixing, re-scan scoped to the fixed area and confirm the finding is gone. **OSS CLI:** ```bash -# Re-test just the changed files (fast) -strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base origin/main --max-budget 5 +# Re-test just the changed files (fast). Resolve the repo's real default +# branch instead of assuming origin/main (many repos use master/develop). +DIFF_BASE=$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo HEAD~1) +strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base "$DIFF_BASE" --max-budget 5 -# Or re-test with the original finding as focus +# Or re-test with the original finding as focus (no diff base needed) strix -n -t ./ --instruction "Verify the SQL injection in app/api/search.py is fixed. Original PoC: " --max-budget 5 ``` -Exit code `0` = clean; `2` = findings remain (read the new `strix_runs//vulnerabilities/` and iterate). +Exit codes: `2` = findings remain (read the new `strix_runs//vulnerabilities/` and iterate); `0` = clean **for what was analyzed**. Before trusting a `0`, confirm the run wasn't cut short — check `run.json` for a completed status and that the budget wasn't exhausted mid-scan (a too-low `--max-budget` can end verification early). Give verification enough budget to finish, and prefer re-running the specific PoC as the ground-truth signal. **Cloud:** rerun with the same config and re-poll, then confirm the finding no longer appears: ```bash diff --git a/skills/strix-pentest/SKILL.md b/skills/strix-pentest/SKILL.md index 197203c26..c7111c2a4 100644 --- a/skills/strix-pentest/SKILL.md +++ b/skills/strix-pentest/SKILL.md @@ -90,10 +90,12 @@ Scans take minutes (`quick`) to hours (`deep`). Run them in the background and p ### Exit codes (headless) -- `0` — scan completed, no vulnerabilities +- `0` — finished with no validated vulnerabilities **in what was analyzed** - `1` — fatal error (missing env vars, Docker down, bad config) - `2` — vulnerabilities found +A `0` is not proof of full coverage: if `--max-budget`/`--max-turns` is reached before the scan completes, it wraps up early and still exits `0`. When you need assurance the scan finished, give it enough budget and check `strix_runs//run.json` for a completed (non-truncated) status rather than trusting the exit code alone. + ### Reading results Artifacts land in `strix_runs//`: