strix/tests
alex s ca20626f84
Some checks are pending
CI / python (push) Waiting to run
CI / viewer (push) Waiting to run
CI / package (push) Waiting to run
CI / tui (push) Waiting to run
CI / workflows (push) Waiting to run
CI / ci-passed (push) Blocked by required conditions
fix(models): replay malformed tool-call arguments as valid JSON (#1327)
* fix(models): replay malformed tool-call arguments as valid JSON

A model that emits a tool call with truncated or otherwise invalid JSON
arguments fails that one call, but the raw string is recorded in the
session. Strict OpenAI-compatible servers (vLLM, SGLang) validate every
assistant tool call in the request, so each later turn is rejected with
"Assistant tool call function.arguments must be valid JSON" and the agent
never recovers. The turn guard now rewrites such arguments on the way out
to a JSON object carrying the original text, alongside the existing
duplicate call-id repair.

* fix(agents): tell the model to re-issue tool calls whose arguments are not valid JSON

A streamed turn that ends mid-tool-call leaves the client with a JSON prefix
as the arguments. Instead of the SDK's generic parse-error result, the tool
wrapper now short-circuits with a clear 'not executed, re-issue with complete
arguments' message so the model recovers in one turn.

* fix(models): treat NaN/Infinity tool-call arguments as malformed

* test(cost): pin model resolution in the provider-prefixed cost test

* fix(models): treat over-nested tool-call arguments as malformed
2026-10-14 18:17:20 -04:00
..
__init__.py Add configurable token / cost usage limits (#576) 2026-06-22 11:17:08 -04:00
conftest.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_agent_factory_shell.py feat(llm): keep images away from text-only models (#1465) 2026-10-07 23:11:58 +03:00
test_agent_factory_tool_arguments.py fix(models): replay malformed tool-call arguments as valid JSON (#1327) 2026-10-14 18:17:20 -04:00
test_agent_graph_coordination.py fix(agents): stop parents waiting on finished non-interactive children 2026-09-02 22:11:12 +03:00
test_agent_tool_registration.py feat(agents): let callers choose the top-level agent's finish tool (#1496) 2026-10-08 19:11:50 -04:00
test_api_spec.py feat: support API specs and Postman collections as targets (#866) 2026-08-03 21:07:44 -07:00
test_api_spec_targets.py feat: support API specs and Postman collections as targets (#866) 2026-08-03 21:07:44 -07:00
test_auth_cli.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_budget_pause_policy.py fix(budget): parked agents count as active; park never overwrites a stop 2026-09-30 03:13:40 +03:00
test_caido_bootstrap.py perf: bootstrap Caido concurrently with the scan start (#1143) 2026-08-21 12:09:59 -07:00
test_caido_handle.py perf: bootstrap Caido concurrently with the scan start (#1143) 2026-08-21 12:09:59 -07:00
test_cli_fail_on.py fix(cli): let --fail-on through when no terminal is attached 2026-10-05 00:37:27 +03:00
test_cli_mcp_config.py Add MCP server support (#1137) 2026-08-24 14:00:16 -04:00
test_cli_resume_picker.py feat(cli): add -r for --resume and a title column to the resume picker 2026-10-06 07:51:00 +03:00
test_cli_target_list.py feat(cli): strix cloud — managed platform CLI (login, scans, billing, and the rest of the API) (#1177) 2026-09-01 14:39:34 -04:00
test_cloud_cli.py fix(cloud): print top-up instructions on 402 and guide oversize or archive --source (#1242) 2026-09-02 15:26:37 -04:00
test_cloud_cli_runtime.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_idempotency.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_payment_proxy.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_session.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_cloud_source_upload.py fix(cloud): print top-up instructions on 402 and guide oversize or archive --source (#1242) 2026-09-02 15:26:37 -04:00
test_cloud_wallet.py Link CLI wallet (#1222) 2026-09-01 16:00:32 -04:00
test_codex_auth.py feat(tui): replace Textual with a Go/Bubble Tea interface (#941) 2026-08-03 19:23:07 -07:00
test_codex_streaming.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_compaction.py fix(llm): pass LLM_EXTRA_HEADERS through ModelSettings so they reach the agent loop (#937) 2026-07-29 19:38:06 -07:00
test_completions.py feat(cli): strix cloud — managed platform CLI (login, scans, billing, and the rest of the API) (#1177) 2026-09-01 14:39:34 -04:00
test_config_loader.py feat(web_search): add Exa as a web search provider alongside Perplexity (#1270) 2026-09-04 10:34:28 -07:00
test_console_logging.py fix(cli): install the stderr log handler at startup and prepare the run state before the TUI launches 2026-10-04 05:44:06 +03:00
test_context_budget.py perf: take heavy imports off the startup path and pre-warm them in the background (#1141) 2026-08-20 20:24:08 -07:00
test_cost_tracking.py fix(models): replay malformed tool-call arguments as valid JSON (#1327) 2026-10-14 18:17:20 -04:00
test_coverage_tool.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_dedupe_model.py fix(config): drop the stored LLM connection when a linked env var changes 2026-09-02 16:10:52 +03:00
test_deferred_session.py feat(runtime): accept an awaitable extra_files in create_or_reuse so the sandbox create does not wait for it 2026-10-14 03:32:29 +03:00
test_disable_streaming.py feat(llm): structured per-attempt provider request log with provider request ids (#1353) 2026-09-22 20:49:41 -07:00
test_docker_client_delete.py revert(runtime): drop the PTY teardown change in StrixDockerSandboxClient.delete() 2026-10-04 18:35:07 +03:00
test_docker_connection.py fix(docker): keep DOCKER_HOST on from_env so TLS settings apply; a missing current context is an error 2026-10-05 03:00:25 +03:00
test_e2e_budget_lifecycle.py refactor(tools): split wait_for_message into respond_to_user + wait_for_agents 2026-08-02 02:15:51 +03:00
test_execution.py feat(agents): let callers choose the top-level agent's finish tool (#1496) 2026-10-08 19:11:50 -04:00
test_execution_request_log.py feat(llm): structured per-attempt provider request log with provider request ids (#1353) 2026-09-22 20:49:41 -07:00
test_execution_transient_retry.py feat(llm): retry 4xx model errors (#1486) 2026-10-07 18:35:33 -07:00
test_fenced_code.py refactor: move strix/viewer under strix/interface 2026-07-26 13:11:14 -07:00
test_finish_coverage_gate.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_go_tui_runtime.py fix(cli): install the stderr log handler at startup and prepare the run state before the TUI launches 2026-10-04 05:44:06 +03:00
test_hooks.py feat(agents): let callers choose the top-level agent's finish tool (#1496) 2026-10-08 19:11:50 -04:00
test_import_warmup.py test(warmup): assert wait_for_import_warmup blocks until the thread finishes 2026-09-05 02:45:34 +03:00
test_inputs.py fix(inputs): send reasoning_effort as configured; no route-specific handling 2026-10-02 18:17:27 +03:00
test_install_script.py Add Linux ARM64 standalone release support (#886) 2026-07-26 16:27:02 -07:00
test_list_reports.py report: add update_vulnerability_report so an agent can revise a filed finding (#1210) 2026-09-01 13:07:17 -07:00
test_llm_extra_headers.py fix(llm): apply LLM_EXTRA_HEADERS on native OpenAI route even without a custom base 2026-07-30 04:13:25 +03:00
test_llm_request_log.py feat(llm): log the upstream provider behind OpenRouter (#1459) 2026-10-07 18:35:12 -07:00
test_local_sources.py runtime: read_only local sources become :ro bind mounts 2026-09-20 06:10:13 +03:00
test_main_launch.py fix(cli): let --fail-on through when no terminal is attached 2026-10-05 00:37:27 +03:00
test_mcp_client.py fix(mcp): load catalogs before agents discover connections (#1492) 2026-10-08 12:42:39 -04:00
test_mcp_resilience.py fix(mcp): keep the session on tool-call protocol errors and report quarantine truthfully (#1228) 2026-09-01 22:43:54 -04:00
test_model_retry.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_models.py feat(llm): detect whether a model accepts image input (#1460) 2026-10-07 22:58:43 +03:00
test_notes.py Treat literal 'null'/'none' strings as absent for optional tool args (#1164) 2026-08-25 13:21:12 -04:00
test_output_store.py feat(context): spill oversized tool output into the sandbox workspace (#882) 2026-07-26 14:42:22 -07:00
test_packaging.py feat(tui): replace Textual with a Go/Bubble Tea interface (#941) 2026-08-03 19:23:07 -07:00
test_preflight_header_values.py fix(preflight): check extra headers for subscription models and the dedupe key as sent 2026-10-04 01:48:35 +03:00
test_preflight_timeout.py test(preflight): exercise warm_up_llm with a dedicated dedupe model without touching process-wide SDK defaults 2026-10-04 23:12:23 +03:00
test_pricing.py test(pricing): accept any identically priced provider for bare grok-4.5 2026-10-04 20:38:45 +03:00
test_provider_dependencies.py fix(deps): declare boto3 directly and exercise Vertex AI and Bedrock auth from the frozen binary in CI 2026-10-04 22:05:21 +03:00
test_proxy_client.py fix(proxy): log all Caido SDK errors as warnings 2026-10-12 22:18:42 +03:00
test_pty_warning_filter.py fix(logging): suppress Agents SDK PTY warning that interferes with TUI (#1392) 2026-10-07 11:32:42 -07:00
test_report_coverage.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_report_pdf.py fix(viewer): harden PDF report rendering (#1192) 2026-08-31 20:22:33 -04:00
test_report_writer.py csv injection hardening (#1203) 2026-08-30 23:36:16 -04:00
test_reporting_fields.py fix(reporting): drop the static-analysis disclaimer from dependency evidence (#1520) 2026-10-14 10:59:11 -04:00
test_reporting_tool.py fix: calibrate vulnerability severity to demonstrated impact 2026-08-03 23:40:32 +03:00
test_resume_picker.py test(resume-picker): put the pty in cbreak so the key-reader test can't hang on macOS (#1527) 2026-10-14 15:55:22 -04:00
test_run_summaries.py feat(cli): add -r for --resume and a title column to the resume picker 2026-10-06 07:51:00 +03:00
test_runner_interrupt.py fix(runner): resume after a user interrupt instead of failing (#1023) 2026-08-08 16:44:12 -07:00
test_runner_mcp.py fix(mcp): load catalogs before agents discover connections (#1492) 2026-10-08 12:42:39 -04:00
test_runner_rate_limit.py fix(core): recover from hallucinated tool names instead of ending the scan 2026-08-04 06:14:54 +03:00
test_runner_root_prompt.py fix(mcp): load catalogs before agents discover connections (#1492) 2026-10-08 12:42:39 -04:00
test_runner_teardown.py Stop false "ended without calling finish_scan" error on PR reviews (#1449) 2026-10-05 19:58:20 -04:00
test_sarif.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_sarif_stride.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_secret_files.py Create credential files with owner-only permissions (#945) 2026-08-03 19:36:59 -07:00
test_session_entries.py runtime: read_only local sources become :ro bind mounts 2026-09-20 06:10:13 +03:00
test_session_fd.py fix(ci): make the pre-commit mypy hook and the test suite pass on a fresh checkout 2026-09-02 15:15:51 +03:00
test_skill_dir_extension.py feat(llm): keep images away from text-only models (#1465) 2026-10-07 23:11:58 +03:00
test_state_coverage_artifact.py feat(agents): evidence discipline, and coverage as a first-class artifact (#961) 2026-08-24 03:34:09 -07:00
test_state_repo_context.py fix(report): omit SARIF provenance for multiple repos (#726) 2026-07-10 09:41:18 -04:00
test_stream_idle_timeout.py feat(llm): bound the first stream event and the whole stream (#1487) 2026-10-07 18:35:45 -07:00
test_telemetry_errors.py feat(telemetry): classify error beacons by phase and exception class 2026-09-05 04:08:09 +03:00
test_telemetry_resume.py handle resume tokens gracefully (#1097) 2026-08-17 16:55:27 -04:00
test_threat_model_tool.py Mirror the run's threat models into its state dir so resume keeps them 2026-08-27 01:36:47 +03:00
test_todo.py fix(todo): stop a todo plan failing on priority or duplicates (#1021) 2026-08-08 15:18:48 -07:00
test_tool_call_arguments.py fix(models): replay malformed tool-call arguments as valid JSON (#1327) 2026-10-14 18:17:20 -04:00
test_tool_call_ids.py Fill in blank tool-call ids so strict providers accept the history (#1355) 2026-09-23 18:34:22 -07:00
test_tool_call_ids_providers.py ci: run ruff, mypy, bandit, pytest, Go TUI and viewer checks on every pull request 2026-10-04 21:07:06 +03:00
test_tool_call_limits.py fix(llm): cap the tool calls one assistant response may queue (#977) 2026-08-06 00:06:59 +03:00
test_tool_param_descriptions.py refactor(agents): replace respond_to_user with a text-free wait_for_user 2026-10-05 04:09:17 +03:00
test_tui_backend_controller.py chore(models): remove the model quality warning and its allowlists 2026-09-30 10:04:28 +03:00
test_tui_backend_server.py Fix user message retry lifecycle and TUI sync 2026-08-28 23:46:49 +03:00
test_tui_protocol_conformance.py feat(tui): replace Textual with a Go/Bubble Tea interface (#941) 2026-08-03 19:23:07 -07:00
test_tui_resume_history.py Fix user message retry lifecycle and TUI sync 2026-08-28 23:46:49 +03:00
test_unknown_tool_recovery.py fix(core): recover from hallucinated tool names instead of ending the scan 2026-08-04 06:14:54 +03:00
test_unraisable_hook.py refactor(telemetry): keep only the unraisable hook, no docstrings 2026-10-04 18:35:07 +03:00
test_update_check.py fix(update): re-exec runs the new binary after self-update (endless update-prompt loop) (#1168) 2026-08-25 15:13:35 -07:00
test_usage_subscription.py Sign in with a ChatGPT subscription for inference (#854) 2026-07-24 15:41:19 -07:00
test_utf8_streams.py style: drop docstrings from the UTF-8 stream helpers 2026-10-04 01:19:48 +03:00
test_viewer.py fix(viewer): show stopped run status 2026-09-04 01:10:10 +03:00
test_viewer_auth.py refactor: move strix/viewer under strix/interface 2026-07-26 13:11:14 -07:00
test_viewer_runs_gating.py refactor: move strix/viewer under strix/interface 2026-07-26 13:11:14 -07:00
test_wait_dedupe.py fix(agents): stop parents waiting on finished non-interactive children 2026-09-02 22:11:12 +03:00
test_wait_for_user.py refactor(agents): replace respond_to_user with a text-free wait_for_user 2026-10-05 04:09:17 +03:00
test_web_search.py fix(web_search): send only the agent's query to Exa search 2026-09-13 19:40:50 +03:00
test_workspace_files.py feat: place caller-provided files into the sandbox workspace (extra_files, --workspace-file) (#1085) 2026-08-14 16:43:08 -04:00