strix/tests
ian-at-strix 5a30961852
feat(llm): scrub images from every request to a text-only model (#1466)
* feat(llm): scrub images from every request to a text-only model

Replace images at the MCP call before any result_transform, give load_skill the
same agent_browser screenshot swap as the system prompt, and add a
call_model_input_filter that turns any image left in the input into a tool error
while chaining any filter already set.

* chore: fix the ruff and bandit failures on main
2026-10-07 23:59:19 +03:00
..
__init__.py Add configurable token / cost usage limits (#576) 2026-06-22 11:17:08 -04:00
conftest.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_agent_factory_shell.py feat(llm): keep images away from text-only models (#1465) 2026-10-07 23:11:58 +03:00
test_agent_factory_tool_arguments.py Treat literal 'null'/'none' strings as absent for optional tool args (#1164) 2026-08-25 13:21:12 -04:00
test_agent_graph_coordination.py fix(agents): stop parents waiting on finished non-interactive children 2026-09-02 22:11:12 +03:00
test_agent_tool_registration.py fix: stop the run after finish_pr_review (#1451) 2026-10-05 16:25:33 -04:00
test_api_spec.py feat: support API specs and Postman collections as targets (#866) 2026-08-03 21:07:44 -07:00
test_api_spec_targets.py feat: support API specs and Postman collections as targets (#866) 2026-08-03 21:07:44 -07:00
test_auth_cli.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_budget_pause_policy.py fix(budget): parked agents count as active; park never overwrites a stop 2026-09-30 03:13:40 +03:00
test_caido_bootstrap.py perf: bootstrap Caido concurrently with the scan start (#1143) 2026-08-21 12:09:59 -07:00
test_caido_handle.py perf: bootstrap Caido concurrently with the scan start (#1143) 2026-08-21 12:09:59 -07:00
test_cli_fail_on.py fix(cli): let --fail-on through when no terminal is attached 2026-10-05 00:37:27 +03:00
test_cli_mcp_config.py Add MCP server support (#1137) 2026-08-24 14:00:16 -04:00
test_cli_resume_picker.py feat(cli): add -r for --resume and a title column to the resume picker 2026-10-06 07:51:00 +03:00
test_cli_target_list.py feat(cli): strix cloud — managed platform CLI (login, scans, billing, and the rest of the API) (#1177) 2026-09-01 14:39:34 -04:00
test_cloud_cli.py fix(cloud): print top-up instructions on 402 and guide oversize or archive --source (#1242) 2026-09-02 15:26:37 -04:00
test_cloud_cli_runtime.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_idempotency.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_payment_proxy.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_session.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_source_upload.py fix(cloud): print top-up instructions on 402 and guide oversize or archive --source (#1242) 2026-09-02 15:26:37 -04:00
test_cloud_wallet.py Link CLI wallet (#1222) 2026-09-01 16:00:32 -04:00
test_codex_auth.py feat(tui): replace Textual with a Go/Bubble Tea interface (#941) 2026-08-03 19:23:07 -07:00
test_codex_streaming.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_compaction.py fix(llm): pass LLM_EXTRA_HEADERS through ModelSettings so they reach the agent loop (#937) 2026-07-29 19:38:06 -07:00
test_completions.py feat(cli): strix cloud — managed platform CLI (login, scans, billing, and the rest of the API) (#1177) 2026-09-01 14:39:34 -04:00
test_config_loader.py feat(web_search): add Exa as a web search provider alongside Perplexity (#1270) 2026-09-04 10:34:28 -07:00
test_console_logging.py fix(cli): install the stderr log handler at startup and prepare the run state before the TUI launches 2026-10-04 05:44:06 +03:00
test_context_budget.py perf: take heavy imports off the startup path and pre-warm them in the background (#1141) 2026-08-20 20:24:08 -07:00
test_cost_tracking.py docs + unit test fix 2026-09-30 04:46:01 +03:00
test_coverage_tool.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_dedupe_model.py fix(config): drop the stored LLM connection when a linked env var changes 2026-09-02 16:10:52 +03:00
test_disable_streaming.py feat(llm): structured per-attempt provider request log with provider request ids (#1353) 2026-09-22 20:49:41 -07:00
test_docker_client_delete.py revert(runtime): drop the PTY teardown change in StrixDockerSandboxClient.delete() 2026-10-04 18:35:07 +03:00
test_docker_connection.py fix(docker): keep DOCKER_HOST on from_env so TLS settings apply; a missing current context is an error 2026-10-05 03:00:25 +03:00
test_e2e_budget_lifecycle.py refactor(tools): split wait_for_message into respond_to_user + wait_for_agents 2026-08-02 02:15:51 +03:00
test_execution.py feat(llm): scrub images from every request to a text-only model (#1466) 2026-10-07 23:59:19 +03:00
test_execution_request_log.py feat(llm): structured per-attempt provider request log with provider request ids (#1353) 2026-09-22 20:49:41 -07:00
test_execution_transient_retry.py fix(inputs): send reasoning_effort as configured; no route-specific handling 2026-10-02 18:17:27 +03:00
test_fenced_code.py refactor: move strix/viewer under strix/interface 2026-07-26 13:11:14 -07:00
test_finish_coverage_gate.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_go_tui_runtime.py fix(cli): install the stderr log handler at startup and prepare the run state before the TUI launches 2026-10-04 05:44:06 +03:00
test_hooks.py feat(runtime): graduated wrap-up warnings, budget reserve, and interactive budget pause/continue (#893) 2026-07-26 20:37:14 -07:00
test_import_warmup.py test(warmup): assert wait_for_import_warmup blocks until the thread finishes 2026-09-05 02:45:34 +03:00
test_inputs.py fix(inputs): send reasoning_effort as configured; no route-specific handling 2026-10-02 18:17:27 +03:00
test_install_script.py Add Linux ARM64 standalone release support (#886) 2026-07-26 16:27:02 -07:00
test_list_reports.py report: add update_vulnerability_report so an agent can revise a filed finding (#1210) 2026-09-01 13:07:17 -07:00
test_llm_extra_headers.py fix(llm): apply LLM_EXTRA_HEADERS on native OpenAI route even without a custom base 2026-07-30 04:13:25 +03:00
test_llm_request_log.py feat(llm): structured per-attempt provider request log with provider request ids (#1353) 2026-09-22 20:49:41 -07:00
test_local_sources.py runtime: read_only local sources become :ro bind mounts 2026-09-20 06:10:13 +03:00
test_main_launch.py fix(cli): let --fail-on through when no terminal is attached 2026-10-05 00:37:27 +03:00
test_mcp_client.py feat(llm): scrub images from every request to a text-only model (#1466) 2026-10-07 23:59:19 +03:00
test_mcp_resilience.py fix(mcp): keep the session on tool-call protocol errors and report quarantine truthfully (#1228) 2026-09-01 22:43:54 -04:00
test_model_retry.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_models.py feat(llm): detect whether a model accepts image input (#1460) 2026-10-07 22:58:43 +03:00
test_notes.py Treat literal 'null'/'none' strings as absent for optional tool args (#1164) 2026-08-25 13:21:12 -04:00
test_output_store.py feat(context): spill oversized tool output into the sandbox workspace (#882) 2026-07-26 14:42:22 -07:00
test_packaging.py feat(tui): replace Textual with a Go/Bubble Tea interface (#941) 2026-08-03 19:23:07 -07:00
test_preflight_header_values.py fix(preflight): check extra headers for subscription models and the dedupe key as sent 2026-10-04 01:48:35 +03:00
test_preflight_timeout.py test(preflight): exercise warm_up_llm with a dedicated dedupe model without touching process-wide SDK defaults 2026-10-04 23:12:23 +03:00
test_pricing.py test(pricing): accept any identically priced provider for bare grok-4.5 2026-10-04 20:38:45 +03:00
test_provider_dependencies.py fix(deps): declare boto3 directly and exercise Vertex AI and Bedrock auth from the frozen binary in CI 2026-10-04 22:05:21 +03:00
test_proxy_client.py feat(reporting): link HTTP exchange evidence (#1281) 2026-09-09 07:50:23 -07:00
test_pty_warning_filter.py fix(logging): suppress Agents SDK PTY warning that interferes with TUI (#1392) 2026-10-07 11:32:42 -07:00
test_report_coverage.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_report_pdf.py fix(viewer): harden PDF report rendering (#1192) 2026-08-31 20:22:33 -04:00
test_report_writer.py csv injection hardening (#1203) 2026-08-30 23:36:16 -04:00
test_reporting_fields.py test: reject a leading heading in any finish_scan example field 2026-10-03 23:07:56 +03:00
test_reporting_tool.py fix: calibrate vulnerability severity to demonstrated impact 2026-08-03 23:40:32 +03:00
test_resume_picker.py feat(cli): add -r for --resume and a title column to the resume picker 2026-10-06 07:51:00 +03:00
test_run_summaries.py feat(cli): add -r for --resume and a title column to the resume picker 2026-10-06 07:51:00 +03:00
test_runner_interrupt.py fix(runner): resume after a user interrupt instead of failing (#1023) 2026-08-08 16:44:12 -07:00
test_runner_mcp.py feat(mcp): initialize connections lazily (#1347) 2026-09-22 14:02:02 -04:00
test_runner_rate_limit.py fix(core): recover from hallucinated tool names instead of ending the scan 2026-08-04 06:14:54 +03:00
test_runner_root_prompt.py feat(llm): scrub images from every request to a text-only model (#1466) 2026-10-07 23:59:19 +03:00
test_runner_teardown.py Stop false "ended without calling finish_scan" error on PR reviews (#1449) 2026-10-05 19:58:20 -04:00
test_sarif.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_sarif_stride.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_secret_files.py Create credential files with owner-only permissions (#945) 2026-08-03 19:36:59 -07:00
test_session_entries.py runtime: read_only local sources become :ro bind mounts 2026-09-20 06:10:13 +03:00
test_session_fd.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_skill_dir_extension.py feat(llm): keep images away from text-only models (#1465) 2026-10-07 23:11:58 +03:00
test_state_coverage_artifact.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_state_repo_context.py fix(report): omit SARIF provenance for multiple repos (#726) 2026-07-10 09:41:18 -04:00
test_stream_idle_timeout.py fix(llm): abandon a model stream that stops producing events (#978) 2026-08-06 00:07:14 +03:00
test_telemetry_errors.py feat(telemetry): classify error beacons by phase and exception class 2026-09-05 04:08:09 +03:00
test_telemetry_resume.py handle resume tokens gracefully (#1097) 2026-08-17 16:55:27 -04:00
test_threat_model_tool.py Mirror the run's threat models into its state dir so resume keeps them 2026-08-27 01:36:47 +03:00
test_todo.py fix(todo): stop a todo plan failing on priority or duplicates (#1021) 2026-08-08 15:18:48 -07:00
test_tool_call_ids.py Fill in blank tool-call ids so strict providers accept the history (#1355) 2026-09-23 18:34:22 -07:00
test_tool_call_ids_providers.py ci: run ruff, mypy, bandit, pytest, Go TUI and viewer checks on every pull request 2026-10-04 21:07:06 +03:00
test_tool_call_limits.py fix(llm): cap the tool calls one assistant response may queue (#977) 2026-08-06 00:06:59 +03:00
test_tool_param_descriptions.py refactor(agents): replace respond_to_user with a text-free wait_for_user 2026-10-05 04:09:17 +03:00
test_tui_backend_controller.py chore(models): remove the model quality warning and its allowlists 2026-09-30 10:04:28 +03:00
test_tui_backend_server.py Fix user message retry lifecycle and TUI sync 2026-08-28 23:46:49 +03:00
test_tui_protocol_conformance.py feat(tui): replace Textual with a Go/Bubble Tea interface (#941) 2026-08-03 19:23:07 -07:00
test_tui_resume_history.py Fix user message retry lifecycle and TUI sync 2026-08-28 23:46:49 +03:00
test_unknown_tool_recovery.py fix(core): recover from hallucinated tool names instead of ending the scan 2026-08-04 06:14:54 +03:00
test_unraisable_hook.py refactor(telemetry): keep only the unraisable hook, no docstrings 2026-10-04 18:35:07 +03:00
test_update_check.py fix(update): re-exec runs the new binary after self-update (endless update-prompt loop) (#1168) 2026-08-25 15:13:35 -07:00
test_usage_subscription.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_utf8_streams.py style: drop docstrings from the UTF-8 stream helpers 2026-10-04 01:19:48 +03:00
test_viewer.py fix(viewer): show stopped run status 2026-09-04 01:10:10 +03:00
test_viewer_auth.py refactor: move strix/viewer under strix/interface 2026-07-26 13:11:14 -07:00
test_viewer_runs_gating.py refactor: move strix/viewer under strix/interface 2026-07-26 13:11:14 -07:00
test_wait_dedupe.py fix(agents): stop parents waiting on finished non-interactive children 2026-09-02 22:11:12 +03:00
test_wait_for_user.py refactor(agents): replace respond_to_user with a text-free wait_for_user 2026-10-05 04:09:17 +03:00
test_web_search.py fix(web_search): send only the agent's query to Exa search 2026-09-13 19:40:50 +03:00
test_workspace_files.py feat: place caller-provided files into the sandbox workspace (extra_files, --workspace-file) (#1085) 2026-08-14 16:43:08 -04:00