mirror of
https://github.com/usestrix/strix.git
synced 2026-10-28 06:23:07 +00:00
docs(skills): harden CI/verify guidance per review
- CI + verify: resolve the PR/default base branch instead of hard-coding origin/main (repos use master/develop; GitHub Actions uses github.base_ref) - Document the budget/fail-open caveat: exit 0 means clean for what was analyzed, not full coverage; size budget to complete and check run.json
This commit is contained in:
parent
8192498a74
commit
56f08d52d1
3 changed files with 16 additions and 8 deletions
|
|
@ -53,9 +53,10 @@ jobs:
|
|||
Then tell the user to add two repository secrets: `STRIX_LLM` (model id, e.g. `openai/gpt-5.4`) and `LLM_API_KEY` (the provider key). Do not create these values yourself.
|
||||
|
||||
Notes:
|
||||
- In CI/headless runs Strix automatically scopes to the PR's changed files (`--scope-mode auto`). If diff resolution fails, keep `fetch-depth: 0` or pass `--diff-base origin/main`.
|
||||
- In CI/headless runs Strix automatically scopes to the PR's changed files (`--scope-mode auto`). If diff resolution fails, keep `fetch-depth: 0` or set `--diff-base` to the PR's actual base branch — use `origin/${{ github.base_ref }}` in GitHub Actions rather than a hard-coded `origin/main`, since repos use different default branches.
|
||||
- Exit codes: `0` pass, `2` vulnerabilities found (fails the job), `1` setup error.
|
||||
- The runner needs Docker (default GitHub-hosted Ubuntu runners have it).
|
||||
- **Size the budget so the scan completes — don't let it fail open.** A `0` exit means "no validated vulnerabilities in what was analyzed"; if `--max-budget` is hit before the diff is fully covered, the scan wraps up early and can exit `0` on code it never finished testing. For a `quick` diff-scoped PR scan `--max-budget 10` is usually ample, but for large diffs raise it, and don't treat a green build as a guarantee of full coverage. To be strict, fail the build unless `strix_runs/<run>/run.json` reports a completed (non-budget-truncated) status.
|
||||
|
||||
### Optional: upload findings to GitHub code scanning
|
||||
|
||||
|
|
@ -75,10 +76,13 @@ Any pipeline works the same way — install, set the two env vars, run headless:
|
|||
|
||||
```bash
|
||||
curl -sSL https://strix.ai/install | bash
|
||||
strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base origin/main --max-budget 10
|
||||
# Resolve the PR's base branch (fall back to the remote default, then main).
|
||||
DIFF_BASE="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-}"
|
||||
DIFF_BASE="origin/${DIFF_BASE:-$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null | sed 's#^origin/##' || echo main)}"
|
||||
strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base "$DIFF_BASE" --max-budget 10
|
||||
```
|
||||
|
||||
Gate the pipeline on the exit code. Schedule `standard` scans nightly and `deep` scans for release candidates.
|
||||
Gate the pipeline on the exit code (see the budget/fail-open caveat above — give the scan enough budget to finish). Schedule `standard` scans nightly and `deep` scans for release candidates.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -39,13 +39,15 @@ After fixing, re-scan scoped to the fixed area and confirm the finding is gone.
|
|||
|
||||
**OSS CLI:**
|
||||
```bash
|
||||
# Re-test just the changed files (fast)
|
||||
strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base origin/main --max-budget 5
|
||||
# Re-test just the changed files (fast). Resolve the repo's real default
|
||||
# branch instead of assuming origin/main (many repos use master/develop).
|
||||
DIFF_BASE=$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo HEAD~1)
|
||||
strix -n -t ./ --scan-mode quick --scope-mode diff --diff-base "$DIFF_BASE" --max-budget 5
|
||||
|
||||
# Or re-test with the original finding as focus
|
||||
# Or re-test with the original finding as focus (no diff base needed)
|
||||
strix -n -t ./ --instruction "Verify the SQL injection in app/api/search.py is fixed. Original PoC: <poc>" --max-budget 5
|
||||
```
|
||||
Exit code `0` = clean; `2` = findings remain (read the new `strix_runs/<run>/vulnerabilities/` and iterate).
|
||||
Exit codes: `2` = findings remain (read the new `strix_runs/<run>/vulnerabilities/` and iterate); `0` = clean **for what was analyzed**. Before trusting a `0`, confirm the run wasn't cut short — check `run.json` for a completed status and that the budget wasn't exhausted mid-scan (a too-low `--max-budget` can end verification early). Give verification enough budget to finish, and prefer re-running the specific PoC as the ground-truth signal.
|
||||
|
||||
**Cloud:** rerun with the same config and re-poll, then confirm the finding no longer appears:
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -90,10 +90,12 @@ Scans take minutes (`quick`) to hours (`deep`). Run them in the background and p
|
|||
|
||||
### Exit codes (headless)
|
||||
|
||||
- `0` — scan completed, no vulnerabilities
|
||||
- `0` — finished with no validated vulnerabilities **in what was analyzed**
|
||||
- `1` — fatal error (missing env vars, Docker down, bad config)
|
||||
- `2` — vulnerabilities found
|
||||
|
||||
A `0` is not proof of full coverage: if `--max-budget`/`--max-turns` is reached before the scan completes, it wraps up early and still exits `0`. When you need assurance the scan finished, give it enough budget and check `strix_runs/<run>/run.json` for a completed (non-truncated) status rather than trusting the exit code alone.
|
||||
|
||||
### Reading results
|
||||
|
||||
Artifacts land in `strix_runs/<run-name>/`:
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue