scriptoriumadmin/strix
1
0
Fork 0
mirror of https://github.com/usestrix/strix.git synced 2026-10-01 02:03:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Run confirmed finding fixes as native agents in the scan sandbox

Browse source
This commit is contained in:
Jonathan Singer 2026-09-30 12:55:01 -04:00
parent 17894006b4
commit 83176652f5
26 changed files with 1053 additions and 978 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

190
docs/fix-preparation.md
View file

@ -1,175 +1,31 @@
# Fix preparation
# Fixing findings during a scan
The workflow is **repair → review → reviewed patch**. Both agents use Strix's existing
agent loop, native filesystem and shell tools, and saved conversations. They share
one persistent sandbox. The assignments live in `strix/agents/prompts/fix_repair.jinja` and
`fix_review.jinja`, with shared workspace instructions in `fix_workspace.jinja`.
- The assessment investigates and validates an issue, then saves its vulnerability report.
- A confirmed report with an actionable source-backed candidate starts a Fix agent immediately. Unconfirmed reports, duplicate reports, and explicit candidate blockers do not start one.
- Each finding gets a Git worktree in the scan's existing sandbox. Fixes run concurrently; the original checkout remains available for assessment and attack chaining.
- One native Strix agent implements the complete fix, adds a regression test, runs it and relevant existing customer unit tests, runs applicable build/lint/type checks, and reviews the change. Test selection and recovery belong to the agent.
- The agent calls `agent_finish(outcome="done")` or `agent_finish(outcome="blocked")`. The controller enforces **300 total model turns per finding**, including resumed execution and candidate revisions. It does not start a fresh agent after exhaustion.
- The controller binds completion to the final source checkpoint. Only a completed, nonempty patch becomes an artifact. Blocked, interrupted, or capped work produces no deliverable patch.
- Assessment completion publishes the security report. Fixes may continue in the same sandbox; execution and sandbox cleanup finish after all Fix tasks stop. Scan cancellation and the shared model budget also stop fix work.
- In the hosted app, successful fixes become available for **user-initiated draft PR creation** on the issue. Incomplete patches are not shown. Internal diagnostic logs and terminal status remain available to operators.
Repair receives the finding, evidence, affected locations, suggested remediation,
and available reproduction details. It makes a minimal fix, adds a regression test
using the repository's framework, and hands test locations, commands, results, and
failed approaches to review. Once it understands the affected path, it starts the
change rather than expanding the investigation. It preserves legitimate behavior,
not the behavior that enables the vulnerability. Once its focused regression passes,
repair hands off rather than expanding into the full customer suite.
Review receives the finding, patch, repair summary, and command history. It runs
the customer's relevant existing unit tests and the regression test, then judges
whether the change addresses the issue without obvious regressions. It challenges
the repair's central assumption with the strongest plausible bypass and checks
legitimate behavior. Required tests must pass, exercise the actual security decision,
and include any helpers needed to reproduce them in the delivered patch. The reviewer
can make small corrections and rerun affected tests. Optional hardening is follow-up
work; a remaining path to the reported attack is not optional. Existing customer unit
tests remain mandatory: start with the changed component and its direct consumers.
Run the full suite only when small or justified by broad effects, explaining the
reason before starting. Finish once the relevant tests pass, the attack is blocked,
and legitimate use works; additional reassurance alone is not a reason to expand.
## Implementation
Both agents use documented setup and targeted recovery, avoid repeating failed
experiments without a new hypothesis, and hand off or report a blocker when they
cannot progress. Unrelated failures are investigated enough to establish a baseline
and then documented, without taking on repair of the entire test environment. Required
validation that remains blocked is reported as a blocker. Test commands must retain
their actual exit status. These are
agent instructions, not a separate controller that selects or interprets tests.
- `strix/tools/reporting/tool.py`: persists the finding and its confirmed/unconfirmed validation status.
- `strix/report/state.py`: notifies the scan only after successful finding persistence.
- `strix/fix/scan.py`: deduplicates tasks, creates independent worktrees, preserves turn counts, and joins/cancels tasks during scan teardown.
- `strix/fix/session.py`: borrows the scan sandbox with a worktree-specific filesystem root and process ownership. Cleanup never stops another agent's processes.
- `strix/agents/prompts/fix.jinja`: the single Fix assignment; shared workspace guidance is in `fix_workspace.jinja`.
- `strix/fix/runtime.py`: uses `build_strix_agent`, `run_agent_loop`, native tools, persisted sessions, and usage hooks. There is no separate reviewer or custom conversation loop.
- `strix/fix/prepare.py`: checks source identity and the completed patch, then exports successful artifacts.
- Pro supplies progress/result callbacks. The app registers the inline attempt, stores successful artifacts privately, and creates draft PRs using its existing repository integration. Neither starts another fix sandbox.
The existing fix hooks also warn when the same completed command, directory, shell,
exit status, and process output recur three times within twelve recent completed
commands. Timing and chunk IDs are excluded from the comparison. Changed results
reset that command's history; native patch calls reset the window. Running commands
and `write_stdin` polling are excluded. The warning asks the agent to change approach
or hand off; it never blocks a tool, waives tests, or decides the review outcome.
The `single_agent` result contract records the Fix agent's completion, command history, final file manifest, and source digest. It does not claim independent verification. Commands include diagnostic failures and superseded attempts; the agent's final summary explains which tests passed and any optional follow-ups.
## Completion and handoffs
## Standalone OSS command
Before preparation, a source-backed scan report must supply paired `fix_before` /
`fix_after` edits in `code_locations` plus `fix_verification`, or an explicit
`fix_candidate_blocker` with a reason. Prose in `fix_pr_body` alone is not a code
handoff. This also applies to external tests with an attached repository. Black-box
findings without repository source can still be reported without code locations.
The verification note describes draft reasoning and testing gaps; the scanner does
not need to implement or test the fix before reporting the vulnerability.
`strix fix --finding findings.json --finding-id FINDING_ID --repo /path/to/repo`
The blocker is stored as `fix_candidate.blocker`. It preserves the finding and
explains why preparation cannot start. Updating with a blocker withdraws old edits;
new paired locations and verification replace it. Hosted callers persist the whole
candidate and its digest together, and candidate changes invalidate old preparations
and revoke outstanding callbacks.
The standalone command uses the same single-agent implementation in its own sandbox, because no live scan exists to borrow. It preserves the supplied checkout and writes private outputs outside the repository by default. Only successful runs export a patch and archive. `--max-agent-turns` and the legacy `--max-repair-turns` can lower the turn cap; they cannot raise it above 300. Old request fields are accepted for compatibility, but reviewer limits no longer control a second agent.
Agents finish through Strix's `agent_finish` tool:
- Repair: `done` starts review; `blocked` stops and preserves work.
- Review: `approved` finishes; `changes_requested` resumes repair with feedback;
`blocked` stops and explains the missing prerequisite or failed required tests.
Each agent retains its own conversation across handoffs. Test selection and
interpretation belong to the reviewer. Code checks source identity, requires a
nonempty patch, and ensures delivery matches the final workspace approved by review.
Reviewer corrections are included in that workspace. Changes after approval block
delivery; they do not automatically start another repair.
Malformed completion calls return the native tool error to the same agent so it can
correct the call. The logging hook accepts non-JSON error text without crashing or
mistaking it for successful completion. There is no additional retry loop.
## Files and evidence
- `strix/fix/prepare.py`: routes repair and review decisions.
- `strix/fix/runtime.py`: supplies assignments to native Strix agents, routes outcomes,
and records tool results and usage.
- `strix/fix/workspace.py`: stages source and sanitized Git metadata in the sandbox,
then exports changes to the host's artifact mirror.
The public `strix.fix.runtime.run_isolated_fix_preparation()` entry point takes a
request and a clean Git checkout. It creates a job-owned clone and artifact mirror;
the supplied checkout is never edited. It uses the configured native sandbox
backend (Docker in OSS; registered cloud backends work for hosted callers).
The agents execute customer code only inside the sandbox. The host mirror is used
for artifact construction. Changes are saved when an agent completes or is
interrupted. Interrupted runs retain useful work without claiming approval.
Native shell and filesystem tools resolve relative paths from the same staged
repository root. Temporary checkpoint archives live under the sandbox's Git metadata
and are excluded from exported source.
The artifact contains the patch, changed files, `execution.json`,
`agent-sessions.json`, and `tool-results.jsonl`. Logs stay outside repository source.
Command records retain the output returned by native tools, including their output
limits. Agents can redirect lengthy test output to a sandbox file and inspect it
with the native tools. Command exit codes are evidence for review, not proof of
security or coverage by themselves.
## Budgets and delivery
Repair defaults to 400 turns and review to 250, counted across continuations rather
than reset on each handoff. Optional `max_repair_turns` and `max_review_turns` override
the respective limit. The legacy `max_agent_turns` overrides both defaults; an explicit
role limit takes precedence. Existing native turn warnings tell fix agents to finish
their current work and hand off or decide, preserving partial work. Normal scan limits
and warnings are unchanged. The configurable job deadline still defaults to 7,200 seconds. An optional
`max_budget_usd` applies across both agents using SDK usage estimates. The legacy
request field `max_repair_attempts` is accepted but does not control this loop.
New results use `validation_mode: agent_review`. They contain the review decision,
summary, final patch identity, and command history. The app delivers approved
results as draft PRs and includes the review and testing limitations. Completion
`open_items` become reported gaps and `final_recommendations` become follow-up notes,
including on approved results. The CLI and draft PR show both; PRs put them before
the command history. Historical
`native_tests` and `paired` records remain readable by the app's compatibility code;
new runs do not produce those proof structures.
## Run from the OSS CLI
Use the same configured model and Docker environment as a normal Strix scan:
```bash
strix fix --repo ./repo --finding strix_runs/my-scan/vulnerabilities.json \
--finding-id vuln-0001 --output ./fix-result/result.json
```
A file containing one finding or a `FixCandidateV1` also works. Findings need their
recorded `fix_candidate.source_identity`; the command does not guess which revision
an old finding described. The checkout must be clean and at that recorded commit.
This first CLI version supports Git sources, not restoration of uploaded archives.
Automation and benchmarks can pass the existing request format:
```bash
strix fix --repo ./repo --request request.json --output ./fix-result/result.json
```
`--workspace` is an alias for `--repo`. `--artifact` overrides the archive path;
`--max-repair-turns`, `--max-review-turns`, the legacy `--max-agent-turns`, `--timeout`,
and `--max-budget` override request budgets.
Outputs are result JSON, a readable Markdown review, a patch, and the full ZIP
artifact. Without `--output`, they go in a new `~/.strix/fixes/fix-…` directory
outside the source checkout. If that location is itself inside the repository,
choose an external directory with `--output`. Explicit output directories can be
shared: result files and the ZIP are private from creation (0600 on POSIX) and
published atomically. Existing directory permissions are left unchanged.
Exit codes: 0 approved, 2 incomplete/blocked/stale, 1 startup or input failure,
130 interrupted. Interruptions save any checkpointed work in the ZIP archive.
Partial patches and their limitations are retained when review cannot approve.
The CLI does not push changes or publish PRs.
## Hosted integration and credentials
The hosted runner in `strix-pro` restores authorized source, calls this exact OSS
entry point, and sends the result to the app. The app owns account permissions and
publishing through the connected Git provider. Neither supplies a separate repair
or review implementation.
Fix requests cannot select environment variables from the runner. The removed
credential forwarding option accepts legacy empty lists only; nonempty lists fail
validation. No host credential names or prefix blocklists are needed. Customer test
credentials are not injected by this feature; tests needing them must report the
missing setup accurately.
## Local checks
`make test-fix-reliability` exercises the actual Strix loop and native SDK tools
with scripted model responses and local fixture tests. It covers handoffs, reviewer
corrections, blocked or interrupted work, and artifact integrity. It does not make
live model calls or evaluate patch quality; the benchmark covers those questions.
Uploaded archives and ambiguous multiple-repository findings cannot currently start an automatic worktree fix: the candidate must identify one Git source and its exact revision.

4
strix/agents/prompt.py
View file

@ -21,13 +21,13 @@ _PROMPT_DIRNAME = "prompts"
CACHE_POINT = "<cache_point>"
def render_fix_prompt(*, review: bool, workspace_root: str) -> str:
def render_fix_prompt(*, workspace_root: str) -> str:
"""Render a fix assignment without loading scan-only skills."""
env = Environment(
loader=FileSystemLoader(get_strix_resource_path("agents", _PROMPT_DIRNAME)),
autoescape=select_autoescape(enabled_extensions=(), default_for_string=False),
)
template = "fix_review.jinja" if review else "fix_repair.jinja"
template = "fix.jinja"
return str(env.get_template(template).render(workspace_root=workspace_root))

22
strix/agents/prompts/fix.jinja Normal file
View file

@ -0,0 +1,22 @@
Fix the confirmed vulnerability in your assigned worktree and validate the result.
Use the supplied finding, evidence, and available scan setup details.
Make the smallest complete change that follows repository conventions and preserves
legitimate behavior. Add a focused regression test exercising real application
behavior. Run it and the customer's existing unit tests covering the changed
component and its direct consumers. Both are required. Run applicable lint,
typecheck, or build checks.
Inspect the final change for remaining paths to the reported attack. Correct
problems and rerun affected tests. Keep unrelated hardening as follow-up work.
You have at most 300 turns total. Use documented setup and targeted recovery; do
not repeat failed approaches without new evidence. If required tests are missing
or cannot run or pass, report blocked. Incomplete fixes are not delivered.
Call agent_finish with outcome done only when the fix is complete and required
checks pass, or blocked when you cannot finish. Summarize the change, test paths,
commands, actual results, and limitations. Put optional follow-ups in
final_recommendations.
{% include "fix_workspace.jinja" %}

23
strix/agents/prompts/fix_repair.jinja
View file

@ -1,23 +0,0 @@
Fix the reported vulnerability with the smallest complete change that follows
repository conventions. Preserve legitimate behavior, but do not preserve behavior
that enables the vulnerability. Treat suggested remediation as guidance; broader
hardening is follow-up work unless needed to stop the reported attack.
Once you understand the affected path, make the change. Add a focused regression
test using the repository's existing framework and run it. Exercise the real
security decision without building a larger test environment than necessary.
Once the focused regression passes, hand off to review. Review owns the existing
customer unit tests; do not expand into the full unit, integration, or end-to-end
suite before handing off.
Hand the patch, test locations, commands, results, and remaining blockers to review.
Include failed approaches so the reviewer can continue without repeating your
investigation.
Call agent_finish with result_summary and outcome done when the patch is ready for
review, or blocked when you cannot continue. Put unresolved limitations and required
actions in open_items, and optional follow-ups in final_recommendations. Preserve
useful work.
{% include "fix_workspace.jinja" %}

30
strix/agents/prompts/fix_review.jinja
View file

@ -1,30 +0,0 @@
Independently review whether this patch stops the reported attack. Challenge the
repair's central assumption: could an attacker still achieve the same harm with
different inputs or through an allowed branch? Check the strongest plausible bypass
and legitimate behavior through the relevant application code.
Run the new regression and the customer's existing unit tests covering the changed
component and its direct consumers. These tests are required. Run the full suite
only when it is small or the change has broad effects that require it; explain why
broader testing is necessary before starting it.
Do not mock away the security decision being checked. Confirm that helpers needed
to run the regression are included in the patch.
Finish when these tests pass, the reported attack is blocked, and legitimate use
still works. Do not expand testing merely for additional reassurance.
If existing tests assert the vulnerable behavior,
update their expectations while preserving meaningful coverage; do not simply
dismiss their failures.
Make small corrections directly and rerun affected tests; send larger
corrections back to repair. Keep separate hardening as follow-up work, but do not
classify a remaining path to the reported attack as optional. If required tests
are missing or cannot pass, explain the blocker.
Call agent_finish with result_summary and outcome approved, changes_requested, or
blocked. Report actual test results, unresolved limitations, and any required
customer actions. Put limitations and required actions in open_items, and optional
follow-ups in final_recommendations.
{% include "fix_workspace.jinja" %}

10
strix/agents/prompts/fix_workspace.jinja
View file

@ -1,15 +1,17 @@
Both agents share one persistent sandbox. Use the repository's documented runtime
Your worktree is inside the scan sandbox; other agents use different directories.
Keep edits and test resources in your assigned worktree. Never change the original
assessment checkout or stop another agent's services. Use distinct ports for services you start. Use the repository's documented runtime
and test setup. Install needed dependencies, but avoid turning unrelated
infrastructure failures into another development project. Attempt a targeted
recovery; if still blocked, preserve the work and explain what is needed.
recovery; if still blocked, stop and explain what is needed.
Investigate unrelated test failures only enough to establish whether they occur
without the fix, then document them. Do not repair the repository's entire test
environment. If required validation remains blocked, preserve the patch and report
environment. If required validation remains blocked, stop and report
the blocker rather than claiming approval.
Do not repeat an experiment without a new hypothesis or a relevant change. When
attempts stop producing useful evidence, simplify the approach, hand off, or report
attempts stop producing useful evidence, simplify the approach or report
a blocker.
Preserve test exit codes. For lengthy output, capture the test's status before

14
strix/agents/prompts/system_prompt.jinja
View file

@ -328,7 +328,7 @@ ROOT AGENT ROLE:
1. **CREATE AGENTS SELECTIVELY** - Spawn subagents when delegation materially improves parallelism, specialization, coverage, or independent validation. Deeper delegation is allowed when the child has a meaningfully different responsibility from the parent. Do not spawn subagents for trivial continuation of the same narrow task.
2. **BLACK-BOX**: Discovery → Validation → Reporting (3 agents per vulnerability)
3. **WHITE-BOX**: Discovery → Validation → Reporting with an initial fix candidate. The later preparation phase repairs and verifies the candidate.
3. **WHITE-BOX**: Discovery → Validation → Reporting with an initial fix candidate. A dedicated Fix agent starts automatically after a confirmed report and implements/tests the candidate.
4. **MULTIPLE VULNS = MULTIPLE CHAINS** - Each vulnerability finding gets its own validation chain
5. **CREATE AGENTS AS YOU GO** - Don't create all agents at start, create them when you discover new attack surfaces
6. **ONE JOB PER AGENT** - Each agent has ONE specific task only
@ -372,9 +372,19 @@ If valid → Spawns "Auth Reporting Agent" (creates the vulnerability report
with the initial fix candidate: code_locations fix_before/fix_after
+ fix_pr_body)
↓
STOP - the scan preparation phase handles repository-wide repair and verification
STOP - a dedicated Fix agent starts automatically for the confirmed report.
It fixes and tests in its own worktree while the assessment continues.
```
CONFIRMED FINDINGS AND FIXES:
- Set validation_status="confirmed" only after validation establishes the issue.
Use "unconfirmed" for source concerns with unresolved evidence gaps.
- Reporting supplies the finding and initial fix suggestion; the automatically
started Fix agent owns implementation, regression testing and customer unit tests.
- Keep assessment code unchanged for continued testing and attack chaining.
- Finish the assessment when security work is complete. Fix agents may still run;
finish_scan publishes the report and the runtime handles their eventual cleanup.
CRITICAL RULES:
- **NO FLAT STRUCTURES** - Always create nested agent trees

2
strix/core/execution.py
View file

@ -425,6 +425,8 @@ async def respawn_subagents(
]
candidates: list[tuple[str, str, str | None, dict[str, Any]]] = []
for aid, status, md in agents_snapshot:
if "fix_task" in md.get("skills", []):
continue # Fix tasks restore their own workspace, prompt and turn allowance.
if not interactive and status not in {"running", "waiting"}:
continue
if coordinator.parent_of.get(aid) is None or aid == root_id:

43
strix/core/runner.py
View file

@ -8,7 +8,7 @@ import io
import json
import logging
import uuid
from collections.abc import Callable
from collections.abc import Awaitable, Callable
from pathlib import Path
from typing import TYPE_CHECKING, Any
@ -43,6 +43,7 @@ from strix.core.inputs import (
)
from strix.core.paths import run_dir_for, runtime_state_dir
from strix.core.sessions import open_agent_session
from strix.fix.scan import FixSink, ScanFixes
from strix.report.state import get_global_report_state
from strix.runtime import session_manager
from strix.telemetry import set_scan_phase
@ -198,6 +199,8 @@ async def run_strix_scan(
status_sink: StatusSink | None = None,
mcp_connection_requests: list[McpConnectionRequest] | None = None,
mcp_status_sink: McpStatusSink | None = None,
fix_sink: FixSink | None = None,
assessment_sink: Callable[[dict[str, Any]], Awaitable[None]] | None = None,
) -> RunResultBase | None:
"""Run or resume one Strix scan against a sandbox.
@ -356,6 +359,7 @@ async def run_strix_scan(
sessions_to_close: list[SQLiteSession] = []
mcp_registry: McpRegistry | None = None
fixes: ScanFixes | None = None
try:
targets = scan_config.get("targets") or []
@ -490,6 +494,23 @@ async def run_strix_scan(
skills=skills,
)
report_state = get_global_report_state()
if report_state is not None and local_sources and not interactive:
fixes = ScanFixes(
session=sandbox_session,
coordinator=coordinator,
parent_id=root_id,
scan_id=scan_id,
state_dir=state_dir,
local_sources=local_sources,
hooks=hooks,
event_sink=event_sink,
sink=fix_sink,
)
report_state.fix_finding_callback = fixes.notify
for finding in report_state.get_existing_vulnerabilities():
fixes.notify(finding)
child_agent_builder = make_child_factory(
scan_mode=scan_mode,
is_whitebox=is_whitebox,
@ -593,7 +614,10 @@ async def run_strix_scan(
if isinstance(final, str):
try:
parsed = json.loads(final)
scan_completed = bool(isinstance(parsed, dict) and parsed.get("scan_completed"))
scan_completed = bool(
isinstance(parsed, dict)
and (parsed.get("scan_completed") or parsed.get("review_completed"))
)
except (ValueError, TypeError):
scan_completed = False
elif isinstance(final, dict):
@ -607,6 +631,16 @@ async def run_strix_scan(
scan_id,
str(final)[:300],
)
if report_state is not None and report_state.scan_results:
if assessment_sink is not None:
try:
await assessment_sink(report_state.scan_results)
except Exception:
logger.exception("Could not publish assessment before fix completion")
if fixes is not None:
report("Assessment complete · Fixes in progress")
await fixes.wait()
report_state.save_run_data(mark_complete=True)
return result # noqa: TRY300
except BudgetExceededError as exc:
logger.info("Scan %s stopped: %s", scan_id, exc)
@ -641,6 +675,11 @@ async def run_strix_scan(
await coordinator.set_status(root_id, "failed")
raise
finally:
if fixes is not None:
await fixes.close()
report_state = get_global_report_state()
if report_state is not None:
report_state.fix_finding_callback = None
configure_spill_writer(None)
# Settle descendants before closing sessions: on a clean finish a child
# can still be mid-turn, and closing its session underneath it crashes it.

11
strix/fix/contracts.py
View file

@ -152,6 +152,7 @@ class ReportedCheck(ContractModel):
class FindingContext(ContractModel):
validation_status: Literal["confirmed", "unconfirmed"] | None = None
title: str = ""
description: str = ""
evidence: str = ""
@ -187,6 +188,8 @@ class FixCandidateV1(ContractModel):
data = self.model_dump(mode="json")
if self.finding is None:
data.pop("finding", None) # Preserve digests for stored legacy candidates.
if self.finding is not None and self.finding.validation_status is None:
data["finding"].pop("validation_status", None)
if self.blocker is None:
data.pop("blocker", None)
if data.get("reproduction") and data["reproduction"].get("command"):
@ -221,7 +224,7 @@ class FixPreparationRequestV1(ContractModel):
@property
def repair_turn_limit(self) -> int:
return self.max_repair_turns or self.max_agent_turns or 400
return min(self.max_repair_turns or self.max_agent_turns or 300, 300)
@property
def review_turn_limit(self) -> int:
@ -285,7 +288,8 @@ class FileManifestEntry(ContractModel):
class FixPreparationResultV1(ContractModel):
version: Literal["1"] = "1"
validation_mode: Literal["agent_review"] = "agent_review"
validation_mode: Literal["agent_review", "single_agent"] = "single_agent"
completion: RepairOutcome | None = None
prepared_source_digest: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$")
state: PreparationState
stop_reason: str
@ -372,6 +376,9 @@ def candidate_from_legacy_report(
draft_edits=edits,
reproduction=ReproductionSpec(instructions=reproduction) if reproduction else None,
finding=FindingContext(
validation_status=cast(
"Literal['confirmed', 'unconfirmed'] | None", report.get("validation_status")
),
title=str(report.get("title") or ""),
description=str(report.get("description") or report.get("technical_analysis") or ""),
evidence=str(report.get("evidence") or report.get("poc_description") or ""),

218
strix/fix/prepare.py
View file

@ -14,19 +14,15 @@ from pathlib import Path
from typing import Literal
from strix.fix.contracts import (
BlockerKind,
CheckResult,
FileManifestEntry,
FixCandidateV1,
FixPreparationAttempt,
FixPreparationRequestV1,
FixPreparationResultV1,
PreparationBlocker,
PreparationState,
RepairOutcome,
RepairStatus,
VerificationDecision,
VerifierResult,
)
@ -51,10 +47,6 @@ RepairAgent = Callable[
[PreparationContext, list[CheckResult]],
Awaitable[RepairOutcome],
]
IndependentVerifier = Callable[
[PreparationContext, list[CheckResult]],
Awaitable[VerifierResult],
]
SourceVerifier = Callable[[PreparationContext], Awaitable[bool]]
EvidenceReader = Callable[[], Awaitable[list[CheckResult]]]
@ -221,194 +213,70 @@ async def _verify_source(context: PreparationContext) -> bool:
return status_process.returncode == 0 and not status_output.strip(b"\x00")
def _result(
context: PreparationContext,
*,
state: PreparationState,
reason: str,
started: float,
checks: list[CheckResult] | None = None,
verifier: VerifierResult | None = None,
gaps: list[str] | None = None,
manifest: list[FileManifestEntry] | None = None,
diff_summary: str = "",
artifact_ref: str | None = None,
attempt_history: list[FixPreparationAttempt] | None = None,
blocker: PreparationBlocker | None = None,
) -> FixPreparationResultV1:
return FixPreparationResultV1(
state=state,
validation_mode="agent_review",
prepared_source_digest=(
verifier.source_digest
if verifier is not None
else context.feedback[-1].repair.source_digest
if context.feedback
else None
),
stop_reason=reason,
source_identity=context.candidate.source_identity,
candidate=context.candidate,
candidate_digest=context.candidate.digest(),
final_file_manifest=manifest or [],
artifact_ref=artifact_ref,
changed_files=[entry.path for entry in manifest or []],
diff_summary=diff_summary,
checks=checks or [],
verifier=verifier,
attempt_history=attempt_history or [],
gaps=gaps or [],
blocker=blocker,
attempts=context.attempt,
elapsed_seconds=time.monotonic() - started,
)
async def prepare_fix( # noqa: PLR0915 - thin orchestration and cleanup
async def prepare_fix( # noqa: PLR0911
request: FixPreparationRequestV1,
workspace: Path,
*,
repair: RepairAgent,
verify: IndependentVerifier,
manifest_builder: ManifestBuilder = build_git_manifest,
source_verifier: SourceVerifier = _verify_source,
evidence_reader: EvidenceReader | None = None,
cancelled: CancellationCheck = lambda: False,
) -> FixPreparationResultV1:
"""Run repair/review conversations; agents own setup, tests and corrections."""
"""One native agent owns implementation and testing; export successful work only."""
started = time.monotonic()
context = PreparationContext(request=request, workspace=workspace, candidate=request.candidate)
checks: list[CheckResult] = []
verifier: VerifierResult | None = None
repair_turns = review_turns = 0
completion: RepairOutcome | None = None
async def finish(
state: PreparationState,
reason: str,
*,
blocker: PreparationBlocker | None = None,
gaps: list[str] | None = None,
) -> FixPreparationResultV1:
nonlocal checks
if evidence_reader:
checks = await evidence_reader()
manifest, summary, artifact = await manifest_builder(workspace)
return _result(
context,
async def finish(state: PreparationState, reason: str) -> FixPreparationResultV1:
manifest: list[FileManifestEntry] = []
summary, artifact = "", None
if state is PreparationState.READY:
manifest, summary, artifact = await manifest_builder(workspace)
return FixPreparationResultV1(
state=state,
reason=reason,
started=started,
checks=checks,
verifier=verifier,
manifest=manifest,
stop_reason=reason,
source_identity=context.candidate.source_identity,
candidate=context.candidate,
candidate_digest=context.candidate.digest(),
completion=completion,
prepared_source_digest=(
completion.source_digest if completion and state is PreparationState.READY else None
),
final_file_manifest=manifest,
changed_files=[entry.path for entry in manifest],
diff_summary=summary,
artifact_ref=artifact,
attempt_history=context.feedback,
blocker=blocker,
gaps=gaps,
)
async def execute() -> FixPreparationResultV1: # noqa: PLR0911, PLR0912 - terminal outcomes
nonlocal checks, verifier, repair_turns, review_turns
if cancelled():
raise PreparationCancelledError
if context.candidate.blocker:
return await finish(
PreparationState.BLOCKED,
context.candidate.blocker.reason,
gaps=[context.candidate.blocker.reason],
)
if not await source_verifier(context):
return await finish(
PreparationState.STALE,
"The repository no longer matches the finding source.",
blocker=PreparationBlocker(
kind=BlockerKind.SOURCE,
summary="The repository no longer matches the finding source.",
user_action="Refresh the finding against the current repository revision.",
),
)
# Location/snippet interpretation belongs to repair. Exact source identity is checked above.
while repair_turns < request.repair_turn_limit and review_turns < request.review_turn_limit:
if cancelled():
raise PreparationCancelledError
context.attempt += 1
verifier = None
record = FixPreparationAttempt(
attempt=context.attempt,
repair=RepairOutcome(status=RepairStatus.INCOMPLETE, summary="Repair started."),
workspace_digest=await workspace_digest(workspace),
)
context.feedback.append(record)
record.repair = await repair(context, checks)
repair_turns += max(1, record.repair.turns_used)
checks = await evidence_reader() if evidence_reader else record.repair.command_results
record.checks = list(checks)
record.workspace_digest = await workspace_digest(workspace)
manifest, _, _ = await manifest_builder(workspace)
if record.repair.status is not RepairStatus.COMPLETE:
return await finish(
PreparationState.BLOCKED,
record.repair.summary,
blocker=record.repair.blocker,
gaps=record.repair.gaps,
)
if not manifest:
return await finish(
PreparationState.BLOCKED,
"Repair completed without a deliverable patch.",
)
if cancelled():
raise PreparationCancelledError
# Review can investigate even incomplete validation and run the missing checks itself.
verifier = await verify(context, checks)
review_turns += max(1, verifier.turns_used)
record.verifier = verifier
if evidence_reader:
checks = await evidence_reader()
record.checks = list(checks)
if verifier.blocker:
return await finish(
PreparationState.BLOCKED,
verifier.summary,
blocker=verifier.blocker,
gaps=verifier.gaps,
)
if verifier.decision is VerificationDecision.REJECTED:
record.repair.gaps.extend(verifier.gaps or [verifier.summary])
continue
if verifier.decision is not VerificationDecision.VERIFIED:
return await finish(PreparationState.BLOCKED, verifier.summary, gaps=verifier.gaps)
# Test selection, failures, reruns and coverage belong to the reviewer.
# Review may correct the patch. Approval binds to its final snapshot,
# not the earlier repair checkpoint.
if verifier.source_digest != await workspace_digest(workspace):
return await finish(
PreparationState.BLOCKED,
"The deliverable changed after review; the approved patch cannot be delivered.",
)
return await finish(
PreparationState.READY,
"Independent review approved the draft PR. See the review for validation results.",
gaps=verifier.gaps,
)
return await finish(
PreparationState.BLOCKED,
"The agent turn budget was reached; partial work was retained.",
checks=await evidence_reader()
if evidence_reader
else (completion.command_results if completion else []),
attempts=1 if completion else 0,
elapsed_seconds=time.monotonic() - started,
)
try:
async with asyncio.timeout(request.timeout_seconds):
return await execute()
if cancelled():
raise PreparationCancelledError # noqa: TRY301
if context.candidate.blocker:
return await finish(PreparationState.BLOCKED, context.candidate.blocker.reason)
if not await source_verifier(context):
return await finish(PreparationState.STALE, "The finding source no longer matches.")
completion = await repair(context, [])
if cancelled():
raise PreparationCancelledError # noqa: TRY301
if completion.status is not RepairStatus.COMPLETE:
return await finish(PreparationState.BLOCKED, completion.summary)
manifest, _, _ = await build_git_manifest(workspace)
if not manifest:
return await finish(PreparationState.BLOCKED, "The agent produced no patch.")
if completion.source_digest != await workspace_digest(workspace):
return await finish(PreparationState.BLOCKED, "Source changed after completion.")
return await finish(PreparationState.READY, completion.summary)
except PreparationCancelledError:
return await finish(PreparationState.FAILED, "Fix preparation was cancelled.")
except TimeoutError:
return await finish(PreparationState.FAILED, "Fix preparation exceeded its time limit.")
except Exception as error:
logging.getLogger(__name__).exception(
"Fix preparation failed during attempt %s", context.attempt
)
return await finish(
PreparationState.FAILED,
f"Fix preparation stopped after {type(error).__name__}; partial work was retained.",
)
logging.getLogger(__name__).exception("Fix preparation failed")
return await finish(PreparationState.FAILED, f"Fix stopped after {type(error).__name__}.")

175
strix/fix/runtime.py
View file

@ -51,8 +51,6 @@ from strix.fix import (
PreparationContext,
RepairOutcome,
RepairStatus,
VerificationDecision,
VerifierResult,
build_git_manifest,
build_git_patch,
prepare_fix,
@ -90,12 +88,11 @@ def _output_text(text: str, *, max_chars: int | None = _MAX_TOOL_OUTPUT_CHARS) -
class _FixHooks(ReportUsageHooks):
"""Use Strix usage hooks and retain native tool evidence without deciding test success."""
def __init__(self, environment: _RuntimeEnvironment, *, review: bool = False) -> None:
self.max_turns = environment.max_review_turns if review else environment.max_repair_turns
def __init__(self, environment: _RuntimeEnvironment) -> None:
self.max_turns = min(environment.max_repair_turns, 300)
super().__init__(model=load_settings().llm.model or "", max_turns=self.max_turns)
self.environment = environment
self.review = review
self.turns = 0
self.turns = environment.turns_used
self.completion_digest: str | None = None
self._recent_commands: deque[tuple[str, int, str]] = deque(maxlen=_REPEAT_WINDOW)
self._repetition_warning = False
@ -110,8 +107,12 @@ class _FixHooks(ReportUsageHooks):
raise BudgetExceededError("The configured LLM cost budget was reached.")
if self.turns >= self.max_turns:
raise MaxTurnsExceeded("The agent turn budget was reached.")
self.turns += 1
self._sync_scan_budget()
await super().on_llm_start(context, agent, system_prompt, input_items)
self.turns += 1
self.environment.turns_used = self.turns
if self.environment.turn_sink:
self.environment.turn_sink(self.turns)
if self._repetition_warning:
input_items.append(
{
@ -127,6 +128,12 @@ class _FixHooks(ReportUsageHooks):
)
self._repetition_warning = False
def _sync_scan_budget(self) -> None:
shared = self.environment.scan_hooks
if shared is not None:
self.set_max_budget_usd(shared.max_budget_usd)
self._budget_policy = shared.budget_policy
def _track_repetition(self, command: dict[str, Any], exit_code: int, output: str) -> None:
identity = json.dumps(
(
@ -148,35 +155,32 @@ class _FixHooks(ReportUsageHooks):
self._recent_commands.clear()
def _turns_used(self, _context: RunContextWrapper[dict[str, Any]], /) -> int:
# SDK usage starts over when review sends repair feedback; our counter does not.
return self.turns
# SDK usage starts over when a persisted Fix task resumes; our counter does not.
return self.turns + 1
def _turn_warning(
self, _context: RunContextWrapper[dict[str, Any]], /, turns_used: int, stage: int
) -> str:
action = (
"Complete the essential tests and decide approved, changes_requested, or blocked."
if self.review
else "Finish the patch and focused regression, then hand off results and blockers."
)
action = "Finish the fix and required tests, or report blocked."
urgency = ("Begin wrapping up.", "Wrap up now.", "Finish immediately.")[stage]
return (
f"[Fix turn budget] {turns_used}/{self.max_turns} turns used across all handoffs. "
f"[Fix turn budget] {turns_used}/{self.max_turns} total turns used. "
f"{urgency} {action} Do not start new investigations. If required validation is "
"incomplete, report it honestly; do not claim approval. Call agent_finish with "
"result_summary and outcome. Partial work is retained if the budget is reached."
"result_summary and outcome. Incomplete fixes will not be delivered."
)
async def on_llm_end(
self, context: RunContextWrapper[dict[str, Any]], agent: Agent[Any], response: ModelResponse
) -> None:
await super().on_llm_end(context, agent, response)
self.environment.usage.record(
agent_id=str(context.context["agent_id"]),
agent_name=agent.name,
model=load_settings().llm.model,
usage=response.usage,
)
self._sync_scan_budget()
await super().on_llm_end(context, agent, response)
async def on_tool_end(self, context: Any, agent: Any, tool: Any, result: Any) -> None: # noqa: ARG002 - SDK keyword signature.
if not isinstance(context, ToolContext):
@ -204,7 +208,7 @@ class _FixHooks(ReportUsageHooks):
if not isinstance(completion, dict):
return
completion = cast("dict[str, Any]", completion)
if completion.get("agent_completed") and completion.get("outcome") == "approved":
if completion.get("agent_completed") and completion.get("outcome") == "done":
await env.checkpoint()
self.completion_digest = env.validated_digest
return
@ -263,8 +267,14 @@ class _RuntimeEnvironment:
initialized: bool = False
base_commit: str = ""
validated_digest: str | None = None
max_repair_turns: int = 400
max_review_turns: int = 250
max_repair_turns: int = 300
turns_used: int = 0
turn_sink: Callable[[int], None] | None = None
scan_hooks: ReportUsageHooks | None = None
parent_id: str | None = None
event_sink: Callable[[str, Any], None] | None = None
scan_context: dict[str, object] = field(default_factory=dict)
resume: bool = False
max_budget_usd: float | None = None
cancelled: Callable[[], bool] = lambda: False
usage: LLMUsageLedger = field(default_factory=LLMUsageLedger)
@ -344,9 +354,7 @@ class _RuntimeEnvironment:
if not self.initialized:
return
# Git metadata is excluded from source export and stays within the SDK workspace root.
archive = (
Path(self.sandbox_workspace) / ".git" / f"strix-checkpoint-{self.execution_id}.tar"
)
archive = Path(self.sandbox_workspace) / f".strix-checkpoint-{self.execution_id}.tar"
result = await self.session.exec(
"python",
"-c",
@ -364,6 +372,7 @@ class _RuntimeEnvironment:
)
content = await self.session.read(archive)
apply_checkpoint(self.workspace, content.read())
await self.session.exec("rm", "-f", "--", str(archive), shell=False, timeout=30)
self.validated_digest = await workspace_digest(self.workspace)
@ -439,24 +448,20 @@ class _Completion:
class _FixAgent:
"""A task adapter around the standard Strix agent, session and lifecycle."""
def __init__(self, environment: _RuntimeEnvironment, *, review: bool = False) -> None:
def __init__(self, environment: _RuntimeEnvironment) -> None:
self.environment = environment
self.agent_id = f"{environment.execution_id}-{'review' if review else 'repair'}"
self.outcomes = (
["approved", "changes_requested", "blocked"] if review else ["done", "blocked"]
)
self.hooks = _FixHooks(environment, review=review)
self.agent_id = environment.execution_id
self.outcomes = ["done", "blocked"]
self.hooks = _FixHooks(environment)
self.session = open_agent_session(
self.agent_id, environment.workspace.parent / "fix-agents.db"
)
settings = load_settings()
self.agent = build_strix_agent(
name="Independent fix reviewer" if review else "Fix repair agent",
name="Fix agent",
is_root=False,
base_tools=[think],
instructions_override=render_fix_prompt(
review=review, workspace_root=environment.sandbox_workspace
),
instructions_override=render_fix_prompt(workspace_root=environment.sandbox_workspace),
chat_completions_tools=uses_chat_completions_tool_schema(
settings.llm.model or "", settings
),
@ -479,7 +484,7 @@ class _FixAgent:
self.context = {
"coordinator": environment.coordinator,
"agent_id": self.agent_id,
"parent_id": environment.execution_id,
"parent_id": environment.parent_id or "fix-standalone",
"sandbox_session": environment.session,
"completion_outcomes": self.outcomes,
"interactive": False,
@ -489,17 +494,31 @@ class _FixAgent:
start_turns = self.hooks.turns
self.hooks.completion_digest = None
env = self.environment
await env.coordinator.register(self.agent_id, self.agent.name, env.execution_id)
await env.coordinator.register(
self.agent_id,
self.agent.name,
env.parent_id or "fix-standalone",
skills=["fix_task"],
task="Implement and test the confirmed finding",
)
await env.coordinator.attach_runtime(
self.agent_id,
session=self.session,
task=asyncio.current_task(),
resumable=False,
)
await env.coordinator.mark_running(self.agent_id)
try:
remaining = self.hooks.max_turns - start_turns
if remaining <= 0:
return _Completion(
"blocked", "The agent turn budget was reached; partial work was retained.", 0
"blocked",
"The agent turn budget was reached; no incomplete fix will be delivered.",
0,
)
result = await run_agent_loop(
agent=self.agent,
initial_input=_untrusted_prompt_data(payload),
initial_input=[] if env.resume else _untrusted_prompt_data(payload),
run_config=_run_config(env),
context=self.context,
max_turns=remaining,
@ -507,6 +526,7 @@ class _FixAgent:
agent_id=self.agent_id,
interactive=False,
session=self.session,
event_sink=env.event_sink,
hooks=self.hooks,
)
completion = getattr(result, "final_output", None)
@ -529,17 +549,18 @@ class _FixAgent:
)
return _Completion(
"blocked",
"The agent stopped without a completion outcome; partial work was retained.",
"The agent stopped without a completion outcome; "
"no incomplete fix will be delivered.",
self.hooks.turns - start_turns,
)
except (MaxTurnsExceeded, BudgetExceededError):
return _Completion(
"blocked",
"The agent budget was reached; partial work was retained.",
"The agent budget was reached; no incomplete fix will be delivered.",
self.hooks.turns - start_turns,
)
finally:
await env.checkpoint()
await env.coordinator.set_status(self.agent_id, "completed")
async def close(self) -> None:
self.session.close()
@ -557,11 +578,7 @@ class ManagedRepairAgent(_FixAgent):
"repository_root": self.environment.sandbox_workspace,
"network_allowed": self.environment.network_allowed,
"requested_checks": [c.model_dump(mode="json") for c in context.request.checks],
"review_feedback": (
context.feedback[-2].verifier.model_dump(mode="json")
if len(context.feedback) > 1 and context.feedback[-2].verifier
else None
),
"scan_context": self.environment.scan_context,
}
)
return RepairOutcome(
@ -571,62 +588,8 @@ class ManagedRepairAgent(_FixAgent):
summary=completion.summary,
gaps=completion.open_items,
notes=completion.recommendations,
turns_used=completion.turns,
turns_used=self.hooks.turns,
command_results=self.environment.repair_checks[first_command:],
source_digest=self.environment.validated_digest,
blocker=PreparationBlocker(
kind=BlockerKind.EXTERNAL_CONFIGURATION,
summary=completion.summary,
user_action=completion.summary,
)
if completion.outcome == "blocked"
else None,
)
class ManagedIndependentVerifier(_FixAgent):
def __init__(self, environment: _RuntimeEnvironment) -> None:
super().__init__(environment, review=True)
async def __call__(
self, context: PreparationContext, checks: list[CheckResult]
) -> VerifierResult:
environment = self.environment
manifest, _, _ = await build_git_manifest(context.workspace)
patch = (await build_git_patch(context.workspace, manifest)).decode(errors="replace")
first_command = len(environment.repair_checks)
completion = await self.run(
{
"finding": _finding_assignment(context),
"repair": context.feedback[-1].repair.model_dump(
mode="json", exclude={"command_results"}
),
"repository_root": environment.sandbox_workspace,
"network_allowed": environment.network_allowed,
"diff": patch[:150_000],
"diff_truncated": len(patch) > 150_000,
"changed_files": [entry.model_dump(mode="json") for entry in manifest],
"requested_checks": [c.model_dump(mode="json") for c in context.request.checks],
"checks": [_command_preview(c, max_chars=2000) for c in checks],
}
)
extra_checks = environment.repair_checks[first_command:]
approved = completion.outcome == "approved"
return VerifierResult(
decision=(
VerificationDecision.VERIFIED
if approved
else VerificationDecision.REJECTED
if completion.outcome == "changes_requested"
else VerificationDecision.INCONCLUSIVE
),
summary=completion.summary,
turns_used=completion.turns,
gaps=completion.open_items or ([] if approved else [completion.summary]),
notes=completion.recommendations,
review_basis="execution"
if any(c.status is CheckStatus.PASSED and c.exit_code == 0 for c in extra_checks)
else "code_review",
source_digest=self.hooks.completion_digest,
blocker=PreparationBlocker(
kind=BlockerKind.EXTERNAL_CONFIGURATION,
@ -691,7 +654,7 @@ async def run_fix_preparation(
cwd=context.workspace,
timeout=30,
)
matches = not status.strip()
matches = environment.resume or not status.strip()
if matches and not environment.initialized:
await environment.initialize()
return matches
@ -727,7 +690,6 @@ async def run_fix_preparation(
json.dumps(
{
"repair": await repair.session.get_items(),
"review": await reviewer.session.get_items(),
}
),
)
@ -742,32 +704,23 @@ async def run_fix_preparation(
return manifest, summary, str(destination)
environment.max_repair_turns = request.repair_turn_limit
environment.max_review_turns = request.review_turn_limit
environment.max_budget_usd = request.max_budget_usd
environment.cancelled = cancelled
await environment.coordinator.register(environment.execution_id, "Fix preparation", None)
repair = ManagedRepairAgent(environment)
reviewer = ManagedIndependentVerifier(environment)
try:
result = await prepare_fix(
request,
environment.workspace,
repair=repair,
verify=reviewer,
evidence_reader=environment.current_checks,
manifest_builder=build_artifact,
source_verifier=verify_source,
cancelled=cancelled,
)
return result.model_copy(update={"cost_usd": environment.usage.total_cost})
except asyncio.CancelledError:
# Save the checkpoint before the public entry point removes its temporary clone.
await build_artifact(environment.workspace)
raise
finally:
await repair.close()
await reviewer.close()
async def run_isolated_fix_preparation(

333
strix/fix/scan.py Normal file
View file

@ -0,0 +1,333 @@
"""Run finding-scoped Fix tasks in the live scan sandbox."""
from __future__ import annotations
import asyncio
import contextlib
import hashlib
import io
import json
import logging
import shutil
import subprocess
from collections.abc import Awaitable, Callable
from pathlib import Path
from typing import TYPE_CHECKING, Any
from strix.fix.contracts import FixCandidateV1, FixPreparationRequestV1, FixPreparationResultV1
from strix.fix.runtime import _RuntimeEnvironment, run_fix_preparation
from strix.fix.session import WorktreeSession
from strix.fix.workspace import git_metadata_archive
from strix.utils.secret_files import open_secret_file
if TYPE_CHECKING:
from agents.sandbox.session import BaseSandboxSession
from strix.core.agents import AgentCoordinator
from strix.core.hooks import ReportUsageHooks
logger = logging.getLogger(__name__)
FixSink = Callable[
[str, dict[str, Any], FixPreparationResultV1 | None, Path | None], Awaitable[bool | None]
]
class ScanFixes:
def __init__(
self,
*,
session: BaseSandboxSession,
coordinator: AgentCoordinator,
parent_id: str,
scan_id: str,
state_dir: Path,
local_sources: list[dict[str, Any]],
hooks: ReportUsageHooks,
event_sink: Any = None,
sink: FixSink | None = None,
) -> None:
self.session, self.coordinator, self.parent_id = session, coordinator, parent_id
self.scan_id, self.directory = scan_id, state_dir / "fixes"
self.directory.mkdir(parents=True, exist_ok=True, mode=0o700)
self.directory.chmod(0o700)
self.path = self.directory / "tasks.json"
self.records: dict[str, Any] = (
json.loads(self.path.read_text()) if self.path.exists() else {}
)
self.sources = [
Path(s["source_path"]).resolve()
for s in local_sources
if s.get("source_path") and (Path(s["source_path"]) / ".git").exists()
]
self.source_roots = {
Path(
s["source_path"]
).resolve(): f"/workspace/{s.get('workspace_subdir') or Path(s['source_path']).name}"
for s in local_sources
if s.get("source_path")
}
self.hooks, self.event_sink, self.sink = hooks, event_sink, sink
self.tasks: dict[str, asyncio.Task[None]] = {}
self.loop = asyncio.get_running_loop()
self.closed = False
self.base = f"/workspace/.strix-fixes/{hashlib.sha256(scan_id.encode()).hexdigest()[:16]}"
self._source_lock = asyncio.Lock()
self._staged: set[str] = set()
def _save(self) -> None:
temporary = self.path.with_suffix(".tmp")
with open_secret_file(temporary) as stream:
stream.write(json.dumps(self.records).encode())
temporary.replace(self.path)
def notify(self, report: dict[str, Any]) -> None:
# Reporting callbacks can execute on a worker thread.
self.loop.call_soon_threadsafe(self._schedule, report)
def _schedule(self, report: dict[str, Any]) -> None:
if self.closed:
return
finding_id = str(report["id"])
try:
candidate = FixCandidateV1.model_validate(report.get("fix_candidate"))
eligible = (
not report.get("deletion")
and report.get("validation_status") == "confirmed"
and candidate.finding is not None
and candidate.finding.validation_status == "confirmed"
and not candidate.blocker
and bool(candidate.draft_edits)
and candidate.source_identity is not None
)
except ValueError:
eligible = False
candidate = None
digest = candidate.digest() if eligible and candidate else None
previous = self.records.get(finding_id, {})
if (
digest
and previous.get("digest") == digest
and (finding_id in self.tasks or previous.get("status") != "running")
):
return
old_task = self.tasks.get(finding_id)
if old_task and not old_task.done():
old_task.cancel()
if not digest or not candidate:
if previous:
previous["status"] = "obsolete"
previous.pop("artifact", None)
self._save()
return
used = int(previous.get("turns", 0))
if used >= 300:
return
resume = previous.get("digest") == digest and previous.get("status") == "running"
self.records[finding_id] = {"digest": digest, "turns": used, "status": "running"}
self._save()
self.tasks[finding_id] = asyncio.create_task(
self._run(report, candidate, resume=resume, previous=old_task),
name=f"fix-{finding_id}",
)
async def _emit(
self,
stage: str,
report: dict[str, Any],
result: FixPreparationResultV1 | None = None,
artifact: Path | None = None,
) -> bool:
if self.sink is None:
return True
return await self.sink(stage, report, result, artifact) is not False
async def _stage_base(self, source: Path) -> str:
key = hashlib.sha256(str(source).encode()).hexdigest()[:16]
base = f"{self.base}/repositories/{key}"
async with self._source_lock:
if key not in self._staged:
# Scan metadata can be read-only. Use a sanitized local Git object
# store for worktree bookkeeping, never alter the assessment checkout.
exists = await self.session.exec(
"test", "-d", f"{base}/.git", shell=False, timeout=30
)
if exists.exit_code:
archive = f"{self.base}/repository-{key}.tar"
await self.session.exec("mkdir", "-p", base, shell=False, timeout=30)
data = await asyncio.to_thread(git_metadata_archive, source)
await self.session.write(Path(archive), io.BytesIO(data))
await self._exec("tar", "--no-same-owner", "-xf", archive, "-C", base)
await self._exec("rm", "-f", archive)
self._staged.add(key)
return base
async def _exec(self, *argv: str) -> None:
result = await self.session.exec(*argv, shell=False, timeout=120)
if result.exit_code:
raise RuntimeError(f"Workspace command failed: {argv[0]}")
async def _run( # noqa: PLR0912, PLR0915
self,
report: dict[str, Any],
candidate: FixCandidateV1,
*,
resume: bool,
previous: asyncio.Task[None] | None,
) -> None:
finding_id, digest = str(report["id"]), candidate.digest()
key = hashlib.sha256(f"{finding_id}:{digest}".encode()).hexdigest()[:24]
agent_id = f"fix-{key}"
root = f"{self.base}/worktrees/{key}"
directory = self.directory / key
artifact = directory / "prepared-fix.zip"
borrowed: WorktreeSession | None = None
base: str | None = None
result: FixPreparationResultV1 | None = None
started = False
try:
if previous:
await asyncio.gather(previous, return_exceptions=True)
if not await self._emit("started", report):
return
started = True
if len(self.sources) != 1 or candidate.source_identity is None:
raise ValueError("Fix requires one identified Git source checkout") # noqa: TRY301
source = self.sources[0]
commit = candidate.source_identity.value
directory.mkdir(parents=True, exist_ok=True)
mirror = directory / "source"
if not mirror.exists():
await asyncio.to_thread(_clone_revision, source, mirror, commit)
base = await self._stage_base(source)
exists = await self.session.exec("test", "-d", root, shell=False, timeout=30)
if resume and exists.exit_code:
raise RuntimeError( # noqa: TRY301
"The previous fix workspace is unavailable; no automatic restart"
)
if exists.exit_code:
await self._exec(
"git",
"-C",
base,
"-c",
"core.hooksPath=/dev/null",
"worktree",
"add",
"--detach",
root,
commit,
)
borrowed = WorktreeSession(self.session, root, agent_id)
def turns_used(turns: int) -> None:
# One cumulative allowance per finding, including candidate revisions.
self.records[finding_id]["turns"] = turns
self._save()
request = FixPreparationRequestV1(
scan_id=self.scan_id,
finding_id=finding_id,
candidate=candidate,
network_allowed=True,
max_agent_turns=300,
)
environment = _RuntimeEnvironment(
workspace=mirror,
sandbox_session=borrowed,
sandbox_workspace=root,
base_commit=commit,
initialized=True,
execution_id=agent_id,
coordinator=self.coordinator,
parent_id=self.parent_id,
turns_used=self.records[finding_id]["turns"],
turn_sink=turns_used,
scan_hooks=self.hooks,
event_sink=self.event_sink,
resume=resume,
scan_context={
"assessment_source": self.source_roots[source],
"reproduction": report.get("poc_script_code", ""),
},
)
result = await run_fix_preparation(
request,
mirror,
sandbox_session=borrowed,
runtime_environment=environment,
artifact_path=artifact,
)
await self._emit(
"finished", report, result, artifact if result.state == "ready" else None
)
except asyncio.CancelledError:
raise
except Exception:
logger.exception("Fix task %s stopped", finding_id)
finally:
if started and result is None:
with contextlib.suppress(Exception):
await self._emit("finished", report)
if borrowed:
with contextlib.suppress(Exception):
await borrowed.pty_terminate_all()
if base:
with contextlib.suppress(Exception):
await self._exec("git", "-C", base, "worktree", "remove", "--force", root)
with contextlib.suppress(Exception):
await self.coordinator.set_status(agent_id, "completed")
record = self.records[finding_id]
if record.get("digest") == digest and record.get("status") != "obsolete":
record["status"] = "done" if result and result.state == "ready" else "stopped"
if record["status"] == "done":
record["artifact"] = str(artifact)
self._save()
# Successful artifacts survive; unfinished source never becomes a deliverable.
shutil.rmtree(directory / "source", ignore_errors=True)
if result is None or result.state != "ready":
artifact.unlink(missing_ok=True)
async def wait(self) -> None:
await asyncio.sleep(0) # Drain report callbacks queued by worker threads.
while pending := [task for task in self.tasks.values() if not task.done()]:
await asyncio.gather(*pending, return_exceptions=True)
self.closed = True
async def close(self) -> None:
self.closed = True
for task in self.tasks.values():
if not task.done():
task.cancel()
await asyncio.gather(*self.tasks.values(), return_exceptions=True)
def _clone_revision(source: Path, mirror: Path, commit: str) -> None:
subprocess.run( # noqa: S603
[
shutil.which("git") or "/usr/bin/git",
"clone",
"--local",
"--no-checkout",
"--",
str(source),
str(mirror),
],
check=True,
capture_output=True,
timeout=120,
)
subprocess.run( # noqa: S603
[
shutil.which("git") or "/usr/bin/git",
"-c",
"core.hooksPath=/dev/null",
"checkout",
"--detach",
commit,
],
cwd=mirror,
check=True,
capture_output=True,
timeout=60,
)

125
strix/fix/session.py Normal file
View file

@ -0,0 +1,125 @@
"""A worktree-scoped view of a scan sandbox; it never owns the backend."""
from __future__ import annotations
import copy
import uuid
from typing import TYPE_CHECKING, Any
from agents.sandbox.session import BaseSandboxSession
if TYPE_CHECKING:
from pathlib import Path
class WorktreeSession(BaseSandboxSession):
def __init__(self, parent: BaseSandboxSession, root: str, task_id: str) -> None:
self.parent = parent
self.state = copy.deepcopy(parent.state)
self.state.manifest.root = root
self.state.manifest.entries = {}
self.state.session_id = uuid.uuid5(parent.state.session_id, task_id)
self.task_id = task_id
self.processes: set[int] = set()
def _command(self, command: tuple[Any, ...]) -> list[str]:
return [
"env",
f"STRIX_FIX_TASK={self.task_id}",
"sh",
"-c",
'cd -- "$1"; shift; exec "$@"',
"sh",
self.state.manifest.root,
*map(str, command),
]
async def _exec_internal(self, *command: Any, timeout: float | None = None) -> Any:
return await self.parent.exec(
*self._command(command),
shell=False,
timeout=min(timeout or 600, 600),
)
def supports_pty(self) -> bool:
return self.parent.supports_pty()
async def pty_exec_start(self, *command: Any, **kwargs: Any) -> Any:
# Intentional native sandbox shell, never host execution.
prepared = self._prepare_exec_command( # nosec B604
*command,
shell=kwargs.pop("shell", True),
user=kwargs.pop("user", None),
)
kwargs["timeout"] = min(kwargs.get("timeout") or 600, 600)
update = await self.parent.pty_exec_start(
*self._command(tuple(prepared)),
shell=False,
**kwargs,
)
if update.process_id is not None:
self.processes.add(update.process_id)
return update
async def pty_write_stdin(self, *, session_id: int, **kwargs: Any) -> Any:
if session_id not in self.processes:
raise ValueError("That process belongs to another agent")
return await self.parent.pty_write_stdin(session_id=session_id, **kwargs)
async def read(self, path: Path, **kwargs: Any) -> Any:
return await self.parent.read(self.normalize_path(path), **kwargs)
async def write(self, path: Path, data: Any, **kwargs: Any) -> None:
await self.parent.write(self.normalize_path(path, for_write=True), data, **kwargs)
async def running(self) -> bool:
return await self.parent.running()
async def hydrate_workspace(self, _data: Any) -> None:
raise RuntimeError("A borrowed worktree cannot restore the scan workspace")
async def persist_workspace(self) -> Any:
raise RuntimeError("The scan owns workspace persistence")
async def stop(self) -> None:
await self.pty_terminate_all()
async def shutdown(self) -> None:
await self.pty_terminate_all()
async def pty_terminate_all(self) -> None:
# Background services inherit this task marker too. Never call the
# parent session's terminate_all(), which would kill assessment tools.
await self.parent.exec(
"python",
"-c",
_STOP_OWN_PROCESSES,
self.task_id,
shell=False,
timeout=15,
)
self.processes.clear()
_STOP_OWN_PROCESSES = r"""
import os, pathlib, signal, sys, time
marker = b"STRIX_FIX_TASK=" + sys.argv[1].encode()
def owned():
found = []
for path in pathlib.Path("/proc").glob("[0-9]*/environ"):
try:
if marker in path.read_bytes().split(b"\0"):
found.append(int(path.parent.name))
except (OSError, ValueError):
pass
return found
for sig in (signal.SIGTERM, signal.SIGKILL):
for pid in owned():
try:
os.kill(pid, sig)
except ProcessLookupError:
pass
if sig == signal.SIGTERM:
time.sleep(0.2)
"""

5
strix/fix/workspace.py
View file

@ -75,6 +75,10 @@ def git_metadata_archive(workspace: Path) -> bytes:
(clone / "config").write_text(config)
output = io.BytesIO()
with tarfile.open(fileobj=output, mode="w") as archive:
# Packed refs can leave refs/ empty; Git still requires the directory.
refs = tarfile.TarInfo(".git/refs")
refs.type, refs.mode = tarfile.DIRTYPE, 0o755
archive.addfile(refs)
for path in sorted(clone.rglob("*")):
relative = path.relative_to(clone)
if relative.parts[0] in {"hooks", "logs"} or not path.is_file():
@ -160,6 +164,7 @@ changed = set(
git("diff", "--name-only", "--no-renames", "-z", base).decode().split("\0")
) - {""}
changed |= current - original
changed.discard(str(pathlib.Path(sys.argv[3]).relative_to(root)))
manifest = []
with tarfile.open(sys.argv[3], "w") as archive:
for index, name in enumerate(sorted(changed)):

27
strix/interface/fix_cli.py
View file

@ -40,9 +40,13 @@ def _parser() -> argparse.ArgumentParser:
parser.add_argument(
"--artifact", type=Path, help="Patch/log archive; defaults beside the result."
)
parser.add_argument("--max-agent-turns", type=int, help="Shared override for both agents.")
parser.add_argument("--max-repair-turns", type=int, help="Repair turns across handoffs (400).")
parser.add_argument("--max-review-turns", type=int, help="Review turns across handoffs (250).")
parser.add_argument("--max-agent-turns", type=int, help="Maximum total Fix turns (up to 300).")
parser.add_argument(
"--max-repair-turns",
type=int,
help="Maximum total Fix turns (300); larger values are capped.",
)
parser.add_argument("--max-review-turns", type=int, help=argparse.SUPPRESS)
parser.add_argument("--max-budget", type=float, help="Combined LLM cost budget in USD.")
parser.add_argument("--timeout", type=int, help="Whole-job timeout in seconds.")
return parser
@ -110,7 +114,9 @@ async def _preflight() -> None:
def _summary(result: FixPreparationResultV1) -> str:
lines = ["# Fix preparation", "", f"Status: {result.state.value}", "", result.stop_reason]
if result.verifier:
if result.completion:
lines.extend(["", "## Fix", "", result.completion.summary])
elif result.verifier:
lines.extend(["", "## Review", "", result.verifier.summary])
elif result.attempt_history:
lines.extend(["", "## Repair", "", result.attempt_history[-1].repair.summary])
@ -127,6 +133,8 @@ def _summary(result: FixPreparationResultV1) -> str:
if gaps:
lines.extend(["", "## Remaining work", "", *dict.fromkeys(gaps)])
notes = list(result.attempt_history[-1].repair.notes) if result.attempt_history else []
if result.completion:
notes.extend(result.completion.notes)
if result.verifier:
notes.extend(result.verifier.notes)
if notes:
@ -141,11 +149,12 @@ async def _execute(
result = await run_isolated_fix_preparation(request, repo, artifact_path=artifact)
write_secret_text(output, result.model_dump_json(indent=2) + "\n")
write_secret_text(output.with_suffix(".md"), _summary(result))
with (
zipfile.ZipFile(artifact) as archive,
open_secret_file(output.with_suffix(".patch")) as stream,
):
stream.write(archive.read("changes.patch"))
if result.state is PreparationState.READY:
with (
zipfile.ZipFile(artifact) as archive,
open_secret_file(output.with_suffix(".patch")) as stream,
):
stream.write(archive.read("changes.patch"))
return result

22
strix/report/state.py
View file

@ -80,6 +80,7 @@ UPDATABLE_REPORT_FIELDS = frozenset(
"assumptions",
"counterevidence",
"confidence",
"validation_status",
"confidence_rationale",
"severity_change_conditions",
"fix_effort",
@ -238,6 +239,7 @@ class ReportState:
self._saved_vuln_ids: set[str] = set()
self.caido_url: str | None = None
self.fix_finding_callback: Callable[[dict[str, Any]], None] | None = None
self.vulnerability_found_callback: Callable[[dict[str, Any]], None] | None = None
self.vulnerability_updated_callback: Callable[[dict[str, Any]], None] | None = None
self.vulnerability_deleted_callback: Callable[[dict[str, Any]], None] | None = None
@ -357,6 +359,7 @@ class ReportState:
http_exchange_ids: list[str] | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
validation_status: str = "unconfirmed",
finding_class: str | None = None,
dependency_metadata: dict[str, str] | None = None,
fix_candidate: dict[str, Any] | None = None,
@ -421,6 +424,7 @@ class ReportState:
report["fix_verification"] = fix_verification.strip()
if fix_pr_body:
report["fix_pr_body"] = fix_pr_body.strip()
report["validation_status"] = validation_status
report["finding_class"] = (finding_class or "dynamic").strip().lower()
if dependency_metadata:
report["dependency_metadata"] = dependency_metadata
@ -442,6 +446,7 @@ class ReportState:
scarf.finding(severity, cwe=cwe, is_cve=bool(cve))
self.save_run_data()
self._notify_fix(report)
return report_id
def _deleted_vulnerability_reports(self) -> list[dict[str, Any]]:
@ -558,6 +563,7 @@ class ReportState:
)
self.save_run_data()
self._notify_fix(report)
return report
def delete_vulnerability_report(
@ -639,9 +645,17 @@ class ReportState:
except OSError:
logger.exception("could not remove %s", md_path)
self._notify_fix({**report, "deletion": entry})
logger.info("Deleted vulnerability report %s - %s", report_id, report.get("title"))
return report
def _notify_fix(self, report: dict[str, Any]) -> None:
if self.fix_finding_callback:
try:
self.fix_finding_callback(dict(report))
except Exception:
logger.exception("Could not schedule fix for %s", report.get("id"))
def get_existing_vulnerabilities(self) -> list[dict[str, Any]]:
return list(self.vulnerability_reports)
@ -740,9 +754,11 @@ class ReportState:
self.run_record["scan_results"] = self.scan_results
logger.info("Updated scan final fields")
self.save_run_data(mark_complete=True)
posthog.end(self, exit_reason="finished_by_tool")
scarf.end(self, exit_reason="finished_by_tool")
self.run_record["assessment_completed_at"] = datetime.now(UTC).isoformat()
self.save_run_data(mark_complete=self.fix_finding_callback is None)
if self.fix_finding_callback is None:
posthog.end(self, exit_reason="finished_by_tool")
scarf.end(self, exit_reason="finished_by_tool")
def record_mcp_connections(self, names: list[str]) -> None:
"""Note the MCP servers this run connected, and persist it.

13
strix/tools/finish/tool.py
View file

@ -152,7 +152,11 @@ async def finish_scan(
2. Writes the four narrative sections to the scan record.
3. Marks the scan completed and stops execution.
**This is a terminal action, not a status probe.** Whatever you pass
Fix agents (marked fix_task) may still be running. This publishes the
completed assessment; the runner waits for fixes before sandbox cleanup.
Never stop or poll Fix agents merely to publish the assessment.
**This is a terminal action for assessment, not a status probe.** Whatever you pass
is persisted VERBATIM as the final, customer-facing report and then
execution stops. There is no draft mode and no second chance: never
submit placeholder, provisional, or "checking if done" text in any
@ -164,7 +168,7 @@ async def finish_scan(
**Pre-flight checklist (mandatory — do not skip):**
1. **Call ``view_agent_graph`` first.** Inspect every entry in the
summary. If ANY agent is in ``running`` / ``waiting`` state,
summary. If any assessment agent is in ``running`` / ``waiting`` state,
you MUST NOT call ``finish_scan`` yet —
wrap them up first via ``send_message_to_agent`` (ask them to
finish), ``wait_for_agents`` (block until their report
@ -317,6 +321,11 @@ async def finish_scan(
parent_id = inner.get("parent_id")
if coordinator is not None and parent_id is None and me is not None:
active_agents = await coordinator.active_agents_except(me)
active_agents = [
entry
for entry in active_agents
if "fix_task" not in coordinator.metadata.get(entry["agent_id"], {}).get("skills", [])
]
if active_agents and coordinator.reserve_stopped:
active_agents = []
else:

19
strix/tools/reporting/tool.py
View file

@ -12,7 +12,7 @@ import json
import logging
import re
from pathlib import Path, PurePosixPath
from typing import TYPE_CHECKING, Any, cast
from typing import TYPE_CHECKING, Any, Literal, cast
from agents import RunContextWrapper, function_tool
@ -447,6 +447,7 @@ _FIX_CANDIDATE_FIELDS = frozenset(
"evidence",
"fix_verification",
"fix_candidate_blocker",
"validation_status",
}
)
@ -563,6 +564,13 @@ def _collect_update_changes( # noqa: PLR0912, PLR0915
if value is not None:
changes[name] = value
validation_status = fields.get("validation_status")
if validation_status is not None:
if validation_status not in {"confirmed", "unconfirmed"}:
errors.append("validation_status must be confirmed or unconfirmed")
else:
changes["validation_status"] = validation_status
confidence = clean_optional(fields.get("confidence"))
if confidence is not None:
confidence = confidence.lower()
@ -978,6 +986,7 @@ async def _do_create( # noqa: PLR0911 - explicit validation and persistence out
confidence_rationale: str | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
validation_status: Literal["confirmed", "unconfirmed"] = "unconfirmed",
fix_candidate_blocker: FixCandidateBlocker | None = None,
agent_id: str | None = None,
agent_name: str | None = None,
@ -1074,6 +1083,7 @@ async def _do_create( # noqa: PLR0911 - explicit validation and persistence out
"assumptions": assumptions,
"counterevidence": counterevidence,
"confidence": confidence,
"validation_status": validation_status,
"confidence_rationale": confidence_rationale,
"severity_change_conditions": severity_change_conditions,
"fix_effort": fix_effort,
@ -1188,6 +1198,7 @@ async def create_vulnerability_report(
confidence_rationale: str | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
validation_status: Literal["confirmed", "unconfirmed"] = "unconfirmed",
fix_candidate_blocker: FixCandidateBlocker | None = None,
) -> str:
"""File a vulnerability report — one report per fully-verified finding.
@ -1508,6 +1519,9 @@ async def create_vulnerability_report(
you checked. Distinguish executed checks from reasoning and
name testing gaps. Repair and independent testing happen later;
this field does not claim the draft is verified.
validation_status: Record the validation agent's conclusion: confirmed only
when the vulnerability is established, unconfirmed for unresolved source
concerns. Confirmed source-backed reports start a dedicated Fix agent.
fix_candidate_blocker: With repository source attached, provide either
code_locations containing paired fix_before/fix_after edits and
fix_verification, or this object with a concrete reason why you
@ -1615,6 +1629,7 @@ async def create_vulnerability_report(
fix_verification=fix_verification,
fix_pr_body=fix_pr_body,
fix_candidate_blocker=fix_candidate_blocker,
validation_status=validation_status,
agent_id=agent_id,
agent_name=agent_name,
)
@ -1650,6 +1665,7 @@ async def update_vulnerability_report(
http_exchange_ids: list[str] | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
validation_status: Literal["confirmed", "unconfirmed"] | None = None,
fix_candidate_blocker: FixCandidateBlocker | None = None,
contextual_cvss_reasoning: str | None = None,
) -> str:
@ -1774,6 +1790,7 @@ async def update_vulnerability_report(
"fix_verification": fix_verification,
"fix_pr_body": fix_pr_body,
"fix_candidate_blocker": fix_candidate_blocker,
"validation_status": validation_status,
"contextual_cvss_reasoning": contextual_cvss_reasoning,
}
if http_exchange_warning and all(value is None for value in fields.values()):

62
tests/test_fix_cli.py
View file

@ -2,7 +2,6 @@
from __future__ import annotations
import asyncio
import importlib
import json
import os
@ -13,7 +12,7 @@ from pathlib import Path
from typing import Any
import pytest
from agents import ModelResponse, RunConfig
from agents import RunConfig
from agents.sandbox import SandboxRunConfig
from strix.fix import (
@ -52,7 +51,7 @@ def test_cli_role_budget_overrides(tmp_path: Path) -> None:
]
)
loaded = fix_cli._load_request(args)
assert (loaded.repair_turn_limit, loaded.review_turn_limit) == (400, 250)
assert (loaded.repair_turn_limit, loaded.review_turn_limit) == (300, 250)
def _local_runtime(monkeypatch: pytest.MonkeyPatch, tmp_path: Path, model: ScriptedModel) -> None:
@ -97,7 +96,9 @@ def test_cli_runs_shared_workflow_and_preserves_original_checkout(
if blocked
else [*suite_commands(), finish("approved", "Existing and regression tests passed.")]
)
model = ScriptedModel([*patch(), finish("done")], review)
model = ScriptedModel(
[*patch(), *(review if blocked else [*suite_commands(), finish("done", "tests passed")])]
)
_local_runtime(monkeypatch, tmp_path, model)
output = tmp_path / "result.json"
@ -117,14 +118,15 @@ def test_cli_runs_shared_workflow_and_preserves_original_checkout(
assert code == (2 if blocked else 0)
result = json.loads(output.read_text())
assert result["state"] == ("blocked" if blocked else "ready")
assert result["changed_files"]
assert "safe" in output.with_suffix(".patch").read_text()
assert bool(result["changed_files"]) is not blocked
assert output.with_suffix(".patch").exists() is not blocked
assert ("customer database" if blocked else "tests passed") in output.with_suffix(
".md"
).read_text()
with zipfile.ZipFile(output.with_suffix(".zip")) as artifact:
assert "files/tests/test_security.py" in artifact.namelist()
assert "tool-results.jsonl" in artifact.namelist()
if not blocked:
with zipfile.ZipFile(output.with_suffix(".zip")) as artifact:
assert "files/tests/test_security.py" in artifact.namelist()
assert "tool-results.jsonl" in artifact.namelist()
assert _git(workspace, "status", "--porcelain") == ""
assert _git(workspace, "rev-parse", "HEAD") == commit
assert "unsafe" in (workspace / "app.py").read_text()
@ -157,7 +159,7 @@ def test_stale_request_delivers_explanation_without_running_agents(
)
assert json.loads(output.read_text())["state"] == "stale"
assert not model.inputs["repair"]
assert output.with_suffix(".patch").read_text() == ""
assert not output.with_suffix(".patch").exists()
def test_dirty_checkout_is_preserved_and_never_sent_to_agents(
@ -187,42 +189,6 @@ def test_dirty_checkout_is_preserved_and_never_sent_to_agents(
assert not model.inputs["repair"]
@pytest.mark.asyncio
async def test_interruption_exports_partial_work_before_removing_temporary_clone(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
workspace, _ = _workspace(tmp_path)
commit = existing_suite(workspace)
waiting = asyncio.Event()
class PausedModel(ScriptedModel):
async def get_response(self, *args: Any, **kwargs: Any) -> ModelResponse:
if not self.responses["repair"]:
waiting.set()
await asyncio.Event().wait()
return await super().get_response(*args, **kwargs)
model = PausedModel(patch(), [])
_local_runtime(monkeypatch, tmp_path, model)
output = tmp_path / "partial.zip"
task = asyncio.create_task(
fix_runtime.run_isolated_fix_preparation(
_request(commit),
workspace,
artifact_path=output,
)
)
try:
await asyncio.wait_for(waiting.wait(), timeout=10)
finally:
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
with zipfile.ZipFile(output) as artifact:
assert b"return 'safe'" in artifact.read("files/app.py")
assert _git(workspace, "status", "--porcelain") == ""
def test_finding_selection_is_required_before_preflight(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
@ -264,7 +230,7 @@ def test_cli_outputs_and_in_progress_archive_are_private_in_shared_directory(
shared.mkdir()
shared.chmod(0o777)
output = shared / "result.json"
model = ScriptedModel([*patch(), finish("done")], [*suite_commands(), finish("approved")])
model = ScriptedModel([*patch(), *suite_commands(), finish("done")])
_local_runtime(monkeypatch, tmp_path, model)
original_writestr = zipfile.ZipFile.writestr
writes_checked: list[str] = []
@ -318,7 +284,7 @@ def test_default_outputs_allow_repeated_runs_from_inside_the_repository(
monkeypatch.chdir(workspace)
for attempt in range(2):
model = ScriptedModel([*patch(), finish("done")], [*suite_commands(), finish("approved")])
model = ScriptedModel([*patch(), *suite_commands(), finish("done")])
with monkeypatch.context() as runtime_patch:
_local_runtime(runtime_patch, tmp_path / f"attempt-{attempt}", model)
assert fix_cli.run_fix(["--request", str(request_path), "--repo", "."]) == 0

293
tests/test_fix_completion.py
View file

@ -6,11 +6,12 @@ import json
import shlex
import sys
import zipfile
from pathlib import Path
from typing import Any
from types import SimpleNamespace
from typing import TYPE_CHECKING, Any
import pytest
from agents import Model, RunConfig
from agents import Agent, Model, RunConfig, RunContextWrapper
from agents.exceptions import MaxTurnsExceeded
from agents.items import ModelResponse
from agents.sandbox import SandboxRunConfig
from agents.tool import CustomTool
@ -22,14 +23,21 @@ from openai.types.responses import (
ResponseOutputText,
)
import strix.core.hooks as hooks_module
from strix.config.models import _completed_stream_event
from strix.core.hooks import BudgetExceededError, ReportUsageHooks
from strix.fix import PreparationState
from strix.fix import runtime as fix_runtime
from strix.fix.runtime import _FixHooks
from strix.interface.fix_cli import _summary
from tests.test_fix_reliability import environment, existing_suite
from tests.test_fix_runtime import _request, _workspace
if TYPE_CHECKING:
from pathlib import Path
def call(name: str, **arguments: Any) -> ResponseFunctionToolCall:
return ResponseFunctionToolCall(
type="function_call", name=name, call_id=name, arguments=json.dumps(arguments)
@ -65,8 +73,8 @@ def suite_commands() -> list[Any]:
class ScriptedModel(Model):
def __init__(self, repair: list[Any], review: list[Any]) -> None:
self.responses = {"repair": repair, "review": review}
def __init__(self, repair: list[Any], review: list[Any] | None = None) -> None:
self.responses = {"repair": repair, "review": review or []}
self.inputs: dict[str, list[Any]] = {"repair": [], "review": []}
self.tools: set[str] = set()
@ -156,225 +164,106 @@ async def scenario(
@pytest.mark.asyncio
async def test_native_agents_review_executes_customer_and_regression_tests(tmp_path, monkeypatch):
model = ScriptedModel([*patch(), finish("done")], [*suite_commands(), finish("approved")])
result, env = await scenario(tmp_path, monkeypatch, model)
async def test_single_agent_implements_runs_both_test_suites_and_exports(tmp_path, monkeypatch):
model = ScriptedModel([*patch(), *suite_commands(), finish("done", "Both suites passed")])
result, _env = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
assert result.attempts == 1
assert {"exec_command", "apply_patch", "agent_finish"} <= model.tools
assert not {"create_agent", "finish_scan", "record_coverage", "run_command"} & model.tools
assert len(result.checks) == 4
assert all(c.exit_code == 0 for c in result.checks)
assert result.validation_mode == "single_agent"
assert not model.inputs["review"]
assert result.verifier is None
assert result.completion.turns_used == 5
assert all("Ran 1 test" in c.output for c in result.checks[-2:])
assert result.prepared_source_digest == result.verifier.source_digest == env.validated_digest
assert (tmp_path / "fix-agents.db").exists()
with zipfile.ZipFile(tmp_path / "prepared.zip") as archive:
assert "files/tests/test_security.py" in archive.namelist()
assert len(json.loads(archive.read("execution.json"))) == 4
sessions = json.loads(archive.read("agent-sessions.json"))
assert sessions["repair"]
assert sessions["review"]
assert b"agent_finish" in archive.read("tool-results.jsonl")
assert {"exec_command", "apply_patch", "agent_finish"} <= model.tools
assert not {"create_agent", "finish_scan", "record_coverage"} & model.tools
with zipfile.ZipFile(tmp_path / "prepared.zip") as artifact:
assert "files/tests/test_security.py" in artifact.namelist()
assert "Both suites passed" in _summary(result)
@pytest.mark.asyncio
async def test_reviewer_corrections_are_validated_and_delivered(tmp_path, monkeypatch):
async def test_agent_corrects_failed_test_in_same_conversation(tmp_path, monkeypatch):
model = ScriptedModel(
[*patch("incorrect"), finish("done")],
[
*suite_commands(),
*patch(),
*suite_commands(),
finish("approved", "Corrected patch; both suites now pass."),
],
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
assert any(c.exit_code == 1 for c in result.checks)
assert all(c.exit_code == 0 for c in result.checks[-2:])
assert result.prepared_source_digest != result.attempt_history[0].repair.source_digest
with zipfile.ZipFile(tmp_path / "prepared.zip") as archive:
assert b"return 'safe'" in archive.read("files/app.py")
@pytest.mark.asyncio
async def test_review_feedback_resumes_both_sessions(tmp_path, monkeypatch):
model = ScriptedModel(
[*patch("incorrect"), finish("done"), *patch(), finish("done")],
[
*suite_commands(),
finish("changes_requested", "The regression fails: return the safe value."),
*suite_commands(),
finish("approved"),
],
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
assert result.attempts == 2
assert "The regression fails" in json.dumps(model.inputs["repair"][-1])
assert "changes_requested" in json.dumps(model.inputs["review"][-1])
@pytest.mark.asyncio
async def test_invalid_finish_outcome_is_corrected_through_native_tool(tmp_path, monkeypatch):
model = ScriptedModel(
[*patch(), finish("approved"), finish("done")], [*suite_commands(), finish("approved")]
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
assert "Choose an outcome" in json.dumps(model.inputs["repair"][-1])
@pytest.mark.asyncio
async def test_missing_finish_summary_is_corrected_without_crashing(tmp_path, monkeypatch):
model = ScriptedModel(
[*patch(), call("agent_finish", outcome="done"), finish("done")],
[*suite_commands(), call("agent_finish", outcome="approved"), finish("approved")],
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
for role in ("repair", "review"):
error = json.dumps(model.inputs[role][-1])
assert "result_summary" in error and "Field required" in error
with zipfile.ZipFile(tmp_path / "prepared.zip") as archive:
assert b"Field required" in archive.read("tool-results.jsonl")
@pytest.mark.asyncio
async def test_completion_limitations_and_recommendations_survive_approval(tmp_path, monkeypatch):
limitation = "Production integration still requires customer credentials."
note = "Consider wider integration coverage."
model = ScriptedModel(
[
*patch(),
call(
"agent_finish",
outcome="done",
result_summary="Patch ready.",
open_items=["Reviewer must run existing tests."],
final_recommendations=["Repair follow-up."],
),
],
[
*suite_commands(),
call(
"agent_finish",
outcome="approved",
result_summary="Tests passed.",
open_items=[limitation],
final_recommendations=[note],
),
],
[*patch("still unsafe"), *suite_commands(), *patch(), *suite_commands(), finish("done")]
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY
assert result.verifier.gaps == result.gaps == [limitation]
assert result.verifier.notes == [note]
assert result.attempt_history[0].repair.notes == ["Repair follow-up."]
assert "Reviewer must run existing tests." in json.dumps(model.inputs["review"][0])
assert limitation in _summary(result)
assert note in _summary(result)
assert any(c.exit_code != 0 for c in result.checks)
assert all(c.exit_code == 0 for c in result.checks[-2:])
assert not model.inputs["review"]
@pytest.mark.asyncio
@pytest.mark.parametrize("limited_role", ["repair", "review"])
async def test_role_budget_is_cumulative_across_feedback(tmp_path, monkeypatch, limited_role):
model = ScriptedModel(
[*patch("incorrect"), finish("done"), *patch(), finish("done")],
[
*suite_commands(),
finish("changes_requested", "Correct the return value."),
*suite_commands(),
finish("approved"),
],
)
result, _ = await scenario(
tmp_path,
monkeypatch,
model,
repair_turns=5 if limited_role == "repair" else 20,
review_turns=4 if limited_role == "review" else 20,
)
@pytest.mark.parametrize("end", ["blocked", "limit"])
async def test_blocked_or_capped_agent_discards_patch(tmp_path, monkeypatch, end):
model = ScriptedModel([*patch(), finish("blocked", "Database unavailable")])
result, _ = await scenario(tmp_path, monkeypatch, model, turns=2 if end == "limit" else 10)
assert result.state is PreparationState.BLOCKED
assert result.attempts == 2
assert "budget" in result.stop_reason
assert len(model.inputs[limited_role]) == (5 if limited_role == "repair" else 4)
assert model.responses[limited_role] # The budget stopped execution, not a scripted completion.
resumed_input = json.dumps(model.inputs[limited_role][3])
assert (
f"4/{5 if limited_role == 'repair' else 4} turns used across all handoffs" in resumed_input
)
assert "in-progress work is discarded" not in resumed_input
assert result.final_file_manifest
assert not result.final_file_manifest
assert not (tmp_path / "prepared.zip").exists()
assert len(model.inputs["repair"]) <= (2 if end == "limit" else 3)
@pytest.mark.asyncio
async def test_blocked_tests_keep_patch_without_reopening_repair(tmp_path, monkeypatch):
async def test_native_lifecycle_retries_invalid_outcome(tmp_path, monkeypatch):
model = ScriptedModel([*patch(), finish("approved"), *suite_commands(), finish("done")])
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY
assert result.completion.turns_used == 6
@pytest.mark.asyncio
async def test_resume_consumes_remaining_turn_allowance(tmp_path):
env = environment(tmp_path / "source", tmp_path)
env.turns_used = 299
env.max_repair_turns = 500
saved = []
env.turn_sink = saved.append
hooks = _FixHooks(env)
context = RunContextWrapper(context={})
await hooks.on_llm_start(context, Agent(name="fix"), "", [])
with pytest.raises(MaxTurnsExceeded):
await hooks.on_llm_start(context, Agent(name="fix"), "", [])
assert saved == [300]
@pytest.mark.asyncio
async def test_completion_recommendations_are_in_summary(tmp_path, monkeypatch):
model = ScriptedModel(
[*patch(), finish("done")],
[
shell("exit 1"),
finish("blocked", "Customer unit tests require an unavailable database."),
],
*patch(),
*suite_commands(),
call(
"agent_finish",
outcome="done",
result_summary="Fixed and tested",
final_recommendations=["Run the nightly suite"],
),
]
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.BLOCKED
assert result.attempts == 1
assert result.final_file_manifest
assert result.checks[-1].exit_code == 1
assert "Run the nightly suite" in _summary(result)
@pytest.mark.asyncio
async def test_budget_interruption_saves_partial_patch(tmp_path, monkeypatch):
model = ScriptedModel([*patch(), shell("pwd")], [])
result, _ = await scenario(tmp_path, monkeypatch, model, turns=2)
assert result.state is PreparationState.BLOCKED, result.model_dump_json()
assert result.final_file_manifest
assert result.verifier is None
assert "budget" in result.stop_reason.lower()
async def test_fix_respects_live_scan_budget_without_double_counting(tmp_path, monkeypatch):
@pytest.mark.asyncio
async def test_plain_prose_uses_native_lifecycle_recovery(tmp_path, monkeypatch):
model = ScriptedModel(
[*patch(), "All done", finish("done")], [*suite_commands(), finish("approved")]
recorded = []
state = SimpleNamespace(
get_total_llm_cost=lambda: 2.0, record_sdk_usage=lambda **kwargs: recorded.append(kwargs)
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
assert "lifecycle tool" in json.dumps(model.inputs["repair"][-1])
@pytest.mark.asyncio
async def test_patch_changed_after_approval_is_not_delivered_as_ready(tmp_path, monkeypatch):
original = fix_runtime._FixHooks.on_tool_end
async def change_after_finish(hooks, context, agent, tool, result):
await original(hooks, context, agent, tool, result)
if hooks.completion_digest:
(Path(hooks.environment.sandbox_workspace) / "app.py").write_text(
"def result():\n return 'changed after approval'\n"
)
monkeypatch.setattr(fix_runtime._FixHooks, "on_tool_end", change_after_finish)
model = ScriptedModel([*patch(), finish("done")], [*suite_commands(), finish("approved")])
result, env = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.BLOCKED, result.model_dump_json()
assert "changed after review" in result.stop_reason
assert result.verifier.source_digest != env.validated_digest
@pytest.mark.asyncio
@pytest.mark.parametrize("chat_tools", [True, False])
async def test_native_filesystem_patch_is_shared_with_reviewer(tmp_path, monkeypatch, chat_tools):
monkeypatch.setattr(fix_runtime, "uses_chat_completions_tool_schema", lambda *_: chat_tools)
production_patch = (
"*** Begin Patch\n*** Update File: app.py\n@@\n"
"- return 'unsafe'\n+ return 'safe'\n*** End Patch"
)
model = ScriptedModel(
[call("apply_patch", patch=production_patch), patch()[1], finish("done")],
[*suite_commands(), finish("approved")],
)
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
assert all(c.exit_code == 0 for c in result.checks)
monkeypatch.setattr(hooks_module, "get_global_report_state", lambda: state)
env = environment(tmp_path / "source", tmp_path)
shared = ReportUsageHooks(model="test", max_budget_usd=10)
env.scan_hooks = shared
hooks = _FixHooks(env)
context = RunContextWrapper(context={"agent_id": "fix", "parent_id": "root"})
shared.set_max_budget_usd(1)
with pytest.raises(BudgetExceededError):
await hooks.on_llm_end(
context,
Agent(name="fix"),
ModelResponse(output=[], usage=Usage(requests=1), response_id=None),
)
assert len(recorded) == 1

217
tests/test_fix_preparation.py
View file

@ -13,7 +13,6 @@ from unittest.mock import AsyncMock
import pytest
from strix.fix.contracts import (
BlockerKind,
CandidateLocation,
CheckResult,
CheckStatus,
@ -27,8 +26,6 @@ from strix.fix.contracts import (
ReproductionSpec,
SourceIdentity,
SourceIdentityKind,
VerificationDecision,
VerifierResult,
candidate_from_legacy_report,
)
from strix.fix.locations import AnchorStatus, anchor_location
@ -141,13 +138,12 @@ async def test_explicit_candidate_blocker_does_not_start_agents(tmp_path: Path)
"blocker": {"reason": "Affected source is unavailable."},
}
)
repair, review = AsyncMock(), AsyncMock()
result = await prepare_fix(_request(candidate), workspace, repair=repair, verify=review)
repair = AsyncMock()
result = await prepare_fix(_request(candidate), workspace, repair=repair)
assert result.state is PreparationState.BLOCKED
assert result.stop_reason == candidate.blocker.reason
assert result.attempts == 0
repair.assert_not_awaited()
review.assert_not_awaited()
async def _noop_repair(
@ -209,18 +205,6 @@ async def _fixture_command(workspace: Path, command: CommandSpec) -> CheckResult
)
async def _verified(
context: PreparationContext,
_checks: list[CheckResult],
) -> VerifierResult:
return VerifierResult(
decision=VerificationDecision.VERIFIED,
source_digest=context.feedback[-1].repair.source_digest,
summary="The fix addresses the finding.",
review_basis="code_review",
)
def test_candidate_from_legacy_report_preserves_draft_and_checks() -> None:
candidate = candidate_from_legacy_report(
{
@ -310,12 +294,10 @@ async def test_prepare_fix_rejects_wrong_source_commit(tmp_path: Path) -> None:
_request(_candidate("0" * 40)),
workspace,
repair=_noop_repair,
verify=_verified,
)
assert result.state is PreparationState.STALE
assert result.blocker is not None
assert result.blocker.kind is BlockerKind.SOURCE
assert result.stop_reason == "The finding source no longer matches."
@pytest.mark.asyncio
@ -327,7 +309,6 @@ async def test_prepare_fix_rejects_dirty_workspace(tmp_path: Path) -> None:
_request(_candidate(commit)),
workspace,
repair=_noop_repair,
verify=_verified,
)
assert result.state is PreparationState.STALE
@ -383,190 +364,54 @@ def test_candidate_keeps_full_finding_without_inventing_reproduction() -> None:
@pytest.mark.asyncio
async def test_agent_tests_are_reused_without_controller_execution(tmp_path: Path) -> None:
async def test_agent_tests_are_reused_without_controller_execution(tmp_path):
workspace, commit = _workspace(tmp_path)
outcome = None
async def repair(context, checks):
nonlocal outcome
outcome = await _noop_repair(context, checks)
return outcome
result = await prepare_fix(
_request(_candidate(commit)), workspace, repair=repair, verify=_verified
)
result = await prepare_fix(_request(_candidate(commit)), workspace, repair=_noop_repair)
assert result.state is PreparationState.READY
assert result.validation_mode == "agent_review"
assert result.checks == outcome.command_results
assert result.validation_mode == "single_agent"
assert result.completion.status == RepairStatus.COMPLETE
assert result.verifier is None
assert result.checks == result.completion.command_results
assert "Ran 1 test" in result.checks[0].output
assert result.prepared_source_digest == result.verifier.source_digest
@pytest.mark.asyncio
async def test_review_can_request_more_than_two_repairs(tmp_path: Path) -> None:
@pytest.mark.parametrize("stop", ["blocked", "exception", "cancel"])
async def test_failed_fix_never_exports_partial_work(tmp_path, stop):
workspace, commit = _workspace(tmp_path)
async def review(context, checks):
if context.attempt < 4:
return VerifierResult(
decision=VerificationDecision.REJECTED,
summary="Inspect sibling path",
gaps=["Inspect sibling path"],
)
assert context.feedback[-2].verifier.summary == "Inspect sibling path"
return await _verified(context, checks)
result = await prepare_fix(
_request(_candidate(commit)), workspace, repair=_noop_repair, verify=review
)
assert result.state is PreparationState.READY
assert len(result.attempt_history) == 4 # Legacy max_repair_attempts=2 is not a loop cap.
@pytest.mark.asyncio
@pytest.mark.parametrize("defect", ["missing_unit", "failed", "missing_request"])
async def test_agent_approval_owns_test_evidence_without_controller_retries(
tmp_path: Path, defect: str
) -> None:
workspace, commit = _workspace(tmp_path)
async def repair(context, checks):
result = await _noop_repair(context, checks)
if defect == "missing_unit":
result.command_results = [
c for c in result.command_results if c.name != "existing suite"
]
elif defect == "missing_request":
result.command_results = [c for c in result.command_results if c.name != "compile"]
else:
check = result.command_results[0]
check.status, check.exit_code = CheckStatus.FAILED, 1
return result
request = _request(_candidate(commit))
request.max_agent_turns = 2
result = await prepare_fix(request, workspace, repair=repair, verify=_verified)
# Deliberately scripted approval: test policy is the reviewer's responsibility.
# This tests routing, not whether a real reviewer ought to approve this evidence.
assert result.state is PreparationState.READY
assert result.final_file_manifest
assert result.attempts == 1
assert not result.attempt_history[0].repair.gaps
@pytest.mark.asyncio
async def test_incomplete_validation_can_be_finished_by_reviewer(tmp_path: Path) -> None:
workspace, commit = _workspace(tmp_path)
evidence = []
async def repair(context, checks):
result = await _noop_repair(context, checks)
evidence.extend(result.command_results)
evidence[1].status = CheckStatus.FAILED
return result
async def review(context, checks):
assert checks[1].status is CheckStatus.FAILED
command = CommandSpec(name="existing suite", argv=checks[1].argv, purpose="unit")
evidence[1] = await _fixture_command(workspace, command)
return await _verified(context, checks)
async def read_evidence():
return list(evidence)
async def agent(context, checks):
outcome = await _noop_repair(context, checks)
if stop == "exception":
raise RuntimeError("test interruption")
return outcome.model_copy(update={"status": RepairStatus.BLOCKED})
export = AsyncMock()
result = await prepare_fix(
_request(_candidate(commit)),
workspace,
repair=repair,
verify=review,
evidence_reader=read_evidence,
repair=agent,
manifest_builder=export,
cancelled=lambda: stop == "cancel",
)
assert result.state is PreparationState.READY
assert result.checks[1].status is CheckStatus.PASSED
assert result.state is not PreparationState.READY
assert not result.final_file_manifest
assert result.artifact_ref is None
export.assert_not_awaited()
@pytest.mark.asyncio
@pytest.mark.parametrize("stop", ["budget", "blocked", "exception", "timeout", "cancel"])
async def test_interruptions_preserve_partial_patch_without_approval(
tmp_path: Path, stop: str
) -> None:
workspace, commit = _workspace(tmp_path)
cancel = False
async def repair(context, checks):
nonlocal cancel
result = await _noop_repair(context, checks)
if stop == "budget":
result.status = RepairStatus.BUDGET_EXHAUSTED
elif stop == "blocked":
result.status = RepairStatus.BLOCKED
elif stop == "exception":
raise RuntimeError("provider error")
elif stop == "timeout":
await asyncio.sleep(10)
else:
cancel = True
return result
async def review(*_args):
raise AssertionError("Stopped repair must not be approved")
result = await prepare_fix(
_request(_candidate(commit)).model_copy(
update={"timeout_seconds": 1 if stop == "timeout" else 30}
),
workspace,
repair=repair,
verify=review,
cancelled=lambda: cancel,
)
assert result.state in {PreparationState.BLOCKED, PreparationState.FAILED}
assert result.final_file_manifest
@pytest.mark.asyncio
async def test_review_mutation_blocks_delivery_without_controller_repair_loop(
tmp_path: Path,
) -> None:
async def test_completed_agent_cannot_deliver_changed_checkpoint(tmp_path):
workspace, commit = _workspace(tmp_path)
async def review(context, checks):
result = await _verified(context, checks)
if context.attempt == 1:
(workspace / "review.tmp").write_text("temporary")
return result
async def agent(context, checks):
outcome = await _noop_repair(context, checks)
(workspace / "app.py").write_text("changed after completion")
return outcome
async def repair(context, checks):
assert context.attempt == 1
return await _noop_repair(context, checks)
result = await prepare_fix(
_request(_candidate(commit)), workspace, repair=repair, verify=review
)
result = await prepare_fix(_request(_candidate(commit)), workspace, repair=agent)
assert result.state is PreparationState.BLOCKED
assert "deliverable changed" in result.stop_reason
assert result.attempts == 1
assert result.final_file_manifest
@pytest.mark.asyncio
async def test_empty_deliverable_does_not_start_another_repair(tmp_path: Path) -> None:
workspace, commit = _workspace(tmp_path)
async def repair(context, _checks):
assert context.attempt == 1
return RepairOutcome(status=RepairStatus.COMPLETE, summary="Done.")
async def review(*_args):
raise AssertionError("An empty artifact cannot be delivered")
result = await prepare_fix(
_request(_candidate(commit)), workspace, repair=repair, verify=review
)
assert result.state is PreparationState.BLOCKED
assert result.attempts == 1
assert "without a deliverable patch" in result.stop_reason
assert not result.final_file_manifest
def test_new_command_metadata_does_not_change_existing_finding_digest(tmp_path: Path) -> None:

11
tests/test_fix_repetition.py
View file

@ -16,8 +16,7 @@ from tests.test_fix_reliability import environment
@pytest.mark.asyncio
@pytest.mark.parametrize("review", [False, True])
async def test_repeated_native_command_warns_then_agent_can_finish(tmp_path, monkeypatch, review):
async def test_repeated_native_command_warns_then_agent_can_finish(tmp_path, monkeypatch):
# Interleaved commands and different SDK chunk IDs must not hide the repeated read.
repeated = [
shell("cat app.py"),
@ -26,14 +25,10 @@ async def test_repeated_native_command_warns_then_agent_can_finish(tmp_path, mon
shell("ls tests"),
shell("cat app.py"),
]
model = ScriptedModel(
[*patch(), *([] if review else repeated), finish("done")],
[*(repeated if review else []), *suite_commands(), finish("approved")],
)
model = ScriptedModel([*patch(), *repeated, *suite_commands(), finish("done")])
result, _ = await scenario(tmp_path, monkeypatch, model)
assert result.state is PreparationState.READY, result.model_dump_json()
role = "review" if review else "repair"
assert any("[Repeated command]" in json.dumps(turn) for turn in model.inputs[role])
assert any("[Repeated command]" in json.dumps(turn) for turn in model.inputs["repair"])
# The warning does not rewrite command evidence or replace required test execution.
assert all(check.exit_code == 0 for check in result.checks)
assert all("Ran 1 test" in check.output for check in result.checks[-2:])

2
tests/test_fix_runtime.py
View file

@ -108,7 +108,7 @@ def test_run_fix_preparation_requires_sandbox() -> None:
def test_role_budgets_default_and_legacy_override() -> None:
request = _request("a" * 40)
assert (request.repair_turn_limit, request.review_turn_limit) == (400, 250)
assert (request.repair_turn_limit, request.review_turn_limit) == (300, 250)
request.max_agent_turns = 100
assert (request.repair_turn_limit, request.review_turn_limit) == (100, 100)
request.max_repair_turns = 180

54
tests/test_runner_teardown.py
View file

@ -91,3 +91,57 @@ async def test_a_live_child_is_settled_before_sessions_close(
task = child_task["t"]
assert task.done(), "the child task was left running past scan teardown"
assert task.cancelled(), "the child was not cancelled cleanly on a finish"
@pytest.mark.asyncio
async def test_assessment_publishes_before_fixes_end_and_sandbox_teardown(monkeypatch, tmp_path):
_wire_runner(monkeypatch, tmp_path)
events = []
class State:
fix_finding_callback = None
def __init__(self):
self.scan_results = {"scan_completed": True}
def get_existing_vulnerabilities(self):
return []
def save_run_data(self, **_):
events.append("save")
class Fixes:
def __init__(self, **_):
pass
def notify(self, _):
pass
async def wait(self):
events.append("fixes finished")
async def close(self):
events.append("fixes closed")
async def assessment(_):
events.append("assessment published")
async def root(**_):
return types.SimpleNamespace(final_output={"scan_completed": True})
async def cleanup(*_):
events.append("sandbox deleted")
monkeypatch.setattr(runner, "get_global_report_state", State)
monkeypatch.setattr(runner, "ScanFixes", Fixes)
monkeypatch.setattr(runner, "run_agent_loop", root)
monkeypatch.setattr(session_manager, "cleanup", cleanup)
await runner.run_strix_scan(
scan_config={"targets": [], "scan_mode": "deep"},
scan_id="scan",
image="image",
local_sources=[{"source_path": str(tmp_path)}],
assessment_sink=assessment,
)
assert events.index("assessment published") < events.index("fixes finished")
assert events.index("fixes finished") < events.index("sandbox deleted")

106
tests/test_scan_fixes.py Normal file
View file

@ -0,0 +1,106 @@
"""Exercise live-scan worktree isolation with native tools and scripted inference."""
from __future__ import annotations
import asyncio
from pathlib import Path
from unittest.mock import AsyncMock
import pytest
from agents import RunConfig
from agents.sandbox import SandboxRunConfig
from strix.core.agents import AgentCoordinator
from strix.core.hooks import ReportUsageHooks
from strix.fix import FindingContext
from strix.fix import runtime as fix_runtime
from strix.fix.scan import ScanFixes
from strix.fix.session import WorktreeSession
from tests.test_fix_completion import ScriptedModel, finish, patch, suite_commands
from tests.test_fix_reliability import LocalSandbox, existing_suite
from tests.test_fix_runtime import _git, _request, _workspace
def setup(tmp_path):
source, _ = _workspace(tmp_path)
commit = existing_suite(source)
parent = LocalSandbox(tmp_path / "sandbox")
coordinator = AgentCoordinator()
fixes = ScanFixes(
session=parent,
coordinator=coordinator,
parent_id="root",
scan_id="scan",
state_dir=tmp_path / "state",
local_sources=[{"source_path": str(source)}],
hooks=ReportUsageHooks(model="test", max_turns=1000),
)
fixes.base = str(tmp_path / "sandbox" / "fixes")
candidate = _request(commit).candidate
candidate.finding = FindingContext(validation_status="confirmed", title="Unsafe result")
report = {
"id": "finding",
"validation_status": "confirmed",
"fix_candidate": candidate.model_dump(mode="json"),
}
return fixes, report, source, parent
@pytest.mark.asyncio
async def test_parallel_fixes_use_worktrees_without_modifying_scan_source(tmp_path, monkeypatch):
fixes, report, source, parent = setup(tmp_path)
models = {}
def config(env):
model = models.setdefault(
env.execution_id, ScriptedModel([*patch(), *suite_commands(), finish("done")])
)
return RunConfig(
model=model, sandbox=SandboxRunConfig(session=env.session), tracing_disabled=True
)
monkeypatch.setattr(fix_runtime, "_run_config", config)
fixes.notify(report)
fixes.notify({**report, "id": "another-finding"})
await fixes.wait()
assert len(models) == 2
assert all(record["status"] == "done" for record in fixes.records.values()), fixes.records
assert all(Path(record["artifact"]).exists() for record in fixes.records.values())
assert _git(source, "status", "--porcelain") == ""
assert "unsafe" in (source / "app.py").read_text()
assert parent.state.manifest.root == str(tmp_path / "sandbox")
assert not list((tmp_path / "sandbox/fixes/worktrees").glob("*/app.py"))
assert not list((tmp_path / "state/fixes").glob("*/source"))
@pytest.mark.asyncio
async def test_unconfirmed_duplicate_and_exhausted_findings_do_not_start_agent(tmp_path):
fixes, report, _, _ = setup(tmp_path)
fixes._run = AsyncMock()
fixes.notify({**report, "validation_status": "unconfirmed"})
await asyncio.sleep(0)
fixes._run.assert_not_called()
fixes.notify(report)
fixes.notify(report)
await fixes.wait()
assert fixes._run.await_count == 1
fixes.closed = False
fixes.records["finding"]["turns"] = 300
fixes.records["finding"]["status"] = "running"
fixes.tasks.clear()
fixes.notify(report)
await fixes.wait()
assert fixes._run.await_count == 1
@pytest.mark.asyncio
async def test_worktree_process_cleanup_never_terminates_parent_sessions(tmp_path):
parent = LocalSandbox(tmp_path)
parent.exec = AsyncMock()
parent.pty_terminate_all = AsyncMock()
child = WorktreeSession(parent, str(tmp_path / "fix"), "fix-one")
await child.pty_terminate_all()
parent.pty_terminate_all.assert_not_called()
assert parent.exec.call_args.args[3] == "fix-one"
with pytest.raises(ValueError, match="another agent"):
await child.pty_write_stdin(session_id=123, chars="kill")