From f50e6e139968b567fc936ed0a2c0ab1f390b2ad7 Mon Sep 17 00:00:00 2001 From: utku Date: Sat, 24 Oct 2026 21:21:56 -0700 Subject: [PATCH 1/2] Add Bandit source-aware testing playbook --- strix/skills/README.md | 3 +- .../coordination/source_aware_whitebox.md | 2 + strix/skills/custom/source_aware_sast.md | 16 ++- strix/skills/tooling/bandit.md | 131 ++++++++++++++++++ 4 files changed, 150 insertions(+), 2 deletions(-) create mode 100644 strix/skills/tooling/bandit.md diff --git a/strix/skills/README.md b/strix/skills/README.md index 87a48ddfa..605e0ac2a 100644 --- a/strix/skills/README.md +++ b/strix/skills/README.md @@ -40,7 +40,8 @@ The skills are dynamically injected into the agent's system prompt, allowing it Notable source-aware skills: - `source_aware_whitebox` (coordination): white-box orchestration playbook -- `source_aware_sast` (custom): semgrep/AST/secrets/supply-chain static triage workflow +- `source_aware_sast` (custom): semgrep/Bandit/AST/secrets/supply-chain static triage workflow +- `bandit` (tooling): Python AST triage, configuration/suppression review, scan completeness, and source-to-runtime validation - `dependency_cve_scanning` (custom): trivy-based SCA workflow for reporting known dependency CVEs via `create_dependency_report` - `npx_confusion` (custom): npx/npm exec/bunx fallback and adjacent package-runner identity confusion, with runner-specific registry and reporting gates - `semantic_confusion` (vulnerabilities): cross-boundary parser, normalization, and representation mismatch analysis diff --git a/strix/skills/coordination/source_aware_whitebox.md b/strix/skills/coordination/source_aware_whitebox.md index d8849e638..d416fb085 100644 --- a/strix/skills/coordination/source_aware_whitebox.md +++ b/strix/skills/coordination/source_aware_whitebox.md @@ -23,6 +23,8 @@ Increase white-box coverage by combining source-aware triage with dynamic valida ## Source-Aware Triage Stack - `semgrep`: fast security-first triage and custom pattern scans +- `bandit`: Python AST triage when Python source is present; load the `bandit` + skill for scan-completeness checks and source-to-runtime validation - `ast-grep` (`sg`): structural pattern hunting and targeted repo mapping - `tree-sitter`: syntax-aware parsing support for symbol and route extraction - `gitleaks` + `trufflehog`: complementary secret detection (working tree and history coverage) diff --git a/strix/skills/custom/source_aware_sast.md b/strix/skills/custom/source_aware_sast.md index fee2fd921..3c04d5212 100644 --- a/strix/skills/custom/source_aware_sast.md +++ b/strix/skills/custom/source_aware_sast.md @@ -1,6 +1,6 @@ --- name: source-aware-sast -description: Practical source-aware SAST and AST playbook for semgrep, ast-grep, gitleaks, and trivy fs +description: Practical source-aware SAST and AST playbook for semgrep, Python Bandit, ast-grep, gitleaks, and trivy fs --- # Source-Aware SAST Playbook @@ -78,6 +78,20 @@ semgrep scan --config auto --json --output /workspace/.source-aware/semgrep-auto If diff scope is active, restrict to changed files first, then expand only when needed. +## Python-Specific Triage + +When the source map includes Python, use `load_skill(["bandit"])` and add a +scoped Bandit pass alongside Semgrep. Bandit is already installed in the +sandbox. The playbook covers JSON output, scan errors, configuration and +`# nosec` suppressions, and the source-to-runtime handoff. + +Keep each service's artifacts separate. Check scanned-file metrics and +`errors[]` before describing coverage; an empty result list or a zero exit +status can coexist with skipped files. Treat rule matches as hypotheses, +trace the caller and closest control, and dynamically validate before reporting. +Python authorization and business-logic paths still need manual review even +when both scanners return no candidates. + ## AST-Grep Structural Mapping Use `sg` for structure-aware code hunting: diff --git a/strix/skills/tooling/bandit.md b/strix/skills/tooling/bandit.md new file mode 100644 index 000000000..15ad58524 --- /dev/null +++ b/strix/skills/tooling/bandit.md @@ -0,0 +1,131 @@ +--- +name: bandit +description: Python source triage with Bandit, structured output, suppression review, and source-to-runtime validation without promoting scanner matches to confirmed vulnerabilities +--- + +# Bandit Python Triage + +Use Bandit when the source map includes Python. It is installed in the sandbox +alongside Semgrep. Bandit recognizes Python AST patterns; it does not establish +interprocedural taint flow, runtime reachability, or authorization coverage. + +Official references: +- https://bandit.readthedocs.io/en/latest/man/bandit.html +- https://bandit.readthedocs.io/en/latest/config.html +- https://bandit.readthedocs.io/en/latest/formatters/json.html + +## Scoped First Pass + +Run from the Python repository root. For a monorepo, scan each relevant Python +service and give each one its own artifact directory. In a diff-scoped review, +start with the changed Python files and expand into their callers and controls +only as needed. + +```bash +ART=/workspace/.source-aware/python +mkdir -p "$ART" +bandit --version > "$ART/bandit.version" 2>&1 +bandit_status=0 +bandit -r . --format json \ + --exclude './.venv,./venv,./node_modules,./build,./dist' \ + > "$ART/bandit.json" 2> "$ART/bandit.log" || bandit_status=$? +printf '%s\n' "$bandit_status" > "$ART/bandit.exit" +``` + +Keep the unfiltered first pass: severity/confidence thresholds can hide useful +leads. To narrow a follow-up, select relevant test IDs with `--tests B307,B602` +or prioritize with `--severity-level medium --confidence-level high`; save that +run separately and record its filters. + +## Configuration and Suppressions + +- Recursive scans can discover a repository's `.bandit` INI file. Inspect its + `tests`, `skips`, and exclusions before interpreting the result. +- YAML/TOML configuration requires an explicit `-c`, for example + `bandit -r . -c pyproject.toml --format json`. Inspect the configuration first; + a project's lint policy is not necessarily a sufficient security review. +- `# nosec` suppresses matches. Review suppressed security-sensitive paths with + a separate `--ignore-nosec` pass using the same scope and exclusions. Record + the justification instead of assuming suppression proves safety. +- A JSON baseline supplied with `--baseline` hides existing matches. Do not use + it as repository-wide coverage evidence, or add skips just to obtain a clean + report. + +## Read Findings and Coverage Separately + +Exit code `1` can mean Bandit found candidates, not that execution failed. Do not +hide the status with `--exit-zero` or an unconditional `|| true`. + +Inspect all of the following before claiming a pass completed: + +| Evidence | Interpretation | +| --- | --- | +| Exit status and stderr | Distinguish reported matches from configuration or execution failures | +| `errors[]` | Files that could not be scanned, including syntax errors; mark their coverage incomplete even if exit status is zero | +| Per-file entries in `metrics` | Which files were analyzed; `_totals` alone does not demonstrate expected file coverage | +| `results[]` | Candidate locations, not a list of verified vulnerabilities | + +Missing/invalid JSON, unexpected status, skipped expected files, or no analyzed +Python files require investigation. An empty `results` array does not prove the +repository is secure. Preserve version, target paths, configuration, exclusions, +and filters with the output. + +For each candidate retain `test_id`, `filename`, `line_number`, `line_range`, +`issue_severity`, `issue_confidence`, and any `issue_cwe`. Read the referenced +function and callers locally. Raw JSON can contain source snippets and secrets; +share only the minimal redacted evidence needed for validation. + +## Prioritize by Trust Boundary + +| Signal | Follow-up needed | +| --- | --- | +| `B307` (`eval`) | Trace the evaluated value back to an attacker-controlled entrypoint; distinguish literal expressions from untrusted input | +| `B602` (`subprocess` with a shell) | Determine which command bytes an attacker controls and which validation runs before execution | +| `B301` / `B506` (pickle / YAML loading) | Identify the input's origin, loader, and deployment preconditions before attempting deserialization validation | +| `B608` (SQL construction) | Inspect the final database call and parameter binding; a string-construction match alone is insufficient | + +Import-only warnings, example credentials, and constant inputs are not enough to +establish an exploit. Conversely, `shell=False` does not rule out argument +injection, and an unreported wrapper is not proven safe. Keep manual source +review and the relevant vulnerability skill in the workflow. + +## Source-to-Runtime Handoff + +For each plausible candidate, record: + +1. **Location:** repository-relative file, line, test ID, and dangerous operation. +2. **Path:** entrypoint, caller chain, attacker-controlled value, and closest + validation/authorization control. Inspect each reachable caller separately. +3. **Preconditions:** account/role, deployment settings, and required input form. +4. **Validation:** a bounded runtime test in the authorized target, the observed + effect, and a negative control. Deduplicate with Semgrep only when the actual + path and broken control are the same. + +For example, `bandit --tests B307 --format json example.py` flags both calls here: + +```python +import ast + + +def parse_value(text): + return eval(text) + + +def constant_value(): + return eval("42") + + +def literal_value(text): + return ast.literal_eval(text) +``` + +`parse_value` is a lead only if untrusted input reaches it. `constant_value` +matches the same rule without establishing an attacker-controlled expression. +`literal_value` is a negative control for B307, not proof against resource +exhaustion or other vulnerability classes. Do not execute the suspicious code +merely to collect scanner output; Bandit can analyze these functions statically. + +Load the matching vulnerability skill for dynamic validation. Promote a +candidate to `create_vulnerability_report` only after the exploit is confirmed. +If execution is unavailable or blocked, retain the hypothesis and explain the +coverage limitation instead of reporting a verified vulnerability. From b199b32dfd130049158493baac851efd6a61ce00 Mon Sep 17 00:00:00 2001 From: utku Date: Sat, 24 Oct 2026 21:26:21 -0700 Subject: [PATCH 2/2] Clarify Bandit coverage and preserve scan failures --- strix/skills/custom/source_aware_sast.md | 7 ++-- strix/skills/tooling/bandit.md | 53 +++++++++++++++++------- 2 files changed, 41 insertions(+), 19 deletions(-) diff --git a/strix/skills/custom/source_aware_sast.md b/strix/skills/custom/source_aware_sast.md index 3c04d5212..9b982d26f 100644 --- a/strix/skills/custom/source_aware_sast.md +++ b/strix/skills/custom/source_aware_sast.md @@ -85,9 +85,10 @@ scoped Bandit pass alongside Semgrep. Bandit is already installed in the sandbox. The playbook covers JSON output, scan errors, configuration and `# nosec` suppressions, and the source-to-runtime handoff. -Keep each service's artifacts separate. Check scanned-file metrics and -`errors[]` before describing coverage; an empty result list or a zero exit -status can coexist with skipped files. Treat rule matches as hypotheses, +Keep each service's artifacts separate. Check per-file metrics against +`errors[]` and the expected scope before describing coverage: files that fail +parsing can still have metrics, and an empty result list or a zero exit status +can coexist with skipped files. Treat rule matches as hypotheses, trace the caller and closest control, and dynamically validate before reporting. Python authorization and business-logic paths still need manual review even when both scanners return no candidates. diff --git a/strix/skills/tooling/bandit.md b/strix/skills/tooling/bandit.md index 15ad58524..03afb54a0 100644 --- a/strix/skills/tooling/bandit.md +++ b/strix/skills/tooling/bandit.md @@ -17,25 +17,43 @@ Official references: ## Scoped First Pass Run from the Python repository root. For a monorepo, scan each relevant Python -service and give each one its own artifact directory. In a diff-scoped review, -start with the changed Python files and expand into their callers and controls -only as needed. +service separately. The command below scans the whole service. For a diff-scoped +review, replace `-r .` with explicit changed Python file paths; Bandit does not +infer Git diff scope. If that list is empty, skip the pass instead of falling +back to a full-tree scan. Expand into callers and controls only as needed. +Review the exclusions against the source map before running. ```bash -ART=/workspace/.source-aware/python -mkdir -p "$ART" -bandit --version > "$ART/bandit.version" 2>&1 -bandit_status=0 -bandit -r . --format json \ - --exclude './.venv,./venv,./node_modules,./build,./dist' \ - > "$ART/bandit.json" 2> "$ART/bandit.log" || bandit_status=$? -printf '%s\n' "$bandit_status" > "$ART/bandit.exit" +( + set -eu + ART_ROOT=/workspace/.source-aware + mkdir -p "$ART_ROOT" + ART=$(mktemp -d "$ART_ROOT/bandit-python.XXXXXX") + printf 'Bandit artifacts: %s\n' "$ART" + pwd -P > "$ART/source-root.txt" + bandit --version > "$ART/bandit.version" 2>&1 + bandit_status=0 + bandit -r . --format json \ + --exclude '*/.venv/*,*/venv/*,*/node_modules/*,*/build/*,*/dist/*' \ + > "$ART/bandit.json" 2> "$ART/bandit.log" || bandit_status=$? + printf '%s\n' "$bandit_status" > "$ART/bandit.exit" + case "$bandit_status" in + 0|1) ;; + *) cat "$ART/bandit.log" >&2; exit "$bandit_status" ;; + esac +) ``` -Keep the unfiltered first pass: severity/confidence thresholds can hide useful -leads. To narrow a follow-up, select relevant test IDs with `--tests B307,B602` -or prioritize with `--severity-level medium --confidence-level high`; save that -run separately and record its filters. +Each run prints its artifact directory, records the source root, and preserves +the scanner status. Findings can still make the scanner exit `1`; the shell +block accepts `0`/`1` so the JSON can be inspected, and propagates other statuses. +This does not classify coverage: check `errors[]` even when the block succeeds. + +Keep the default severity/confidence thresholds for the first pass; filtering +can hide useful leads. To narrow a follow-up, select relevant test IDs with +`--tests B307,B602` or prioritize with +`--severity-level medium --confidence-level high`. Record the filters and any +configuration restrictions with that run. ## Configuration and Suppressions @@ -47,6 +65,7 @@ run separately and record its filters. - `# nosec` suppresses matches. Review suppressed security-sensitive paths with a separate `--ignore-nosec` pass using the same scope and exclusions. Record the justification instead of assuming suppression proves safety. + `--ignore-nosec` does not restore excluded files or skipped test IDs. - A JSON baseline supplied with `--baseline` hides existing matches. Do not use it as repository-wide coverage evidence, or add skips just to obtain a clean report. @@ -62,7 +81,7 @@ Inspect all of the following before claiming a pass completed: | --- | --- | | Exit status and stderr | Distinguish reported matches from configuration or execution failures | | `errors[]` | Files that could not be scanned, including syntax errors; mark their coverage incomplete even if exit status is zero | -| Per-file entries in `metrics` | Which files were analyzed; `_totals` alone does not demonstrate expected file coverage | +| Per-file entries in `metrics` | Files with recorded metrics, including files that failed parsing; cross-check against `errors[]` and the expected scope | | `results[]` | Candidate locations, not a list of verified vulnerabilities | Missing/invalid JSON, unexpected status, skipped expected files, or no analyzed @@ -74,6 +93,8 @@ For each candidate retain `test_id`, `filename`, `line_number`, `line_range`, `issue_severity`, `issue_confidence`, and any `issue_cwe`. Read the referenced function and callers locally. Raw JSON can contain source snippets and secrets; share only the minimal redacted evidence needed for validation. +Rule severity and match confidence are separate from demonstrated impact and +the final vulnerability severity. ## Prioritize by Trust Boundary