mirror of https://github.com/iflytek/skillhub.git synced 2026-08-28 11:25:00 +00:00

XiaoSeS 3bc97ff1b8 feat(security): add security scanning system with multi-scanner support and frontend UI (#144 )

* feat(security): extend scanner config with full analyzer options

Integrate skill-scanner's 8 analysis engines and policy configuration
into SkillHub's config system. Operators can now control behavioral,
LLM, Meta, AI Defense, VirusTotal, and trigger analyzers via
application.yml or environment variables.

Changes:
- Add Analyzers and Policy nested classes to SkillScannerProperties
- Create ScanOptions record to encapsulate analyzer flags
- Update SkillScannerService to pass options in /scan body and /scan-upload query params
- Wire ScanOptions through SkillScannerConfig and SkillScannerAdapter
- Extend application.yml with full scanner config block and env var overrides
- Update all tests to verify new configuration flow

All tests pass.

* feat(security): add domain model and integrate scan into publish flow

Add SCANNING/SCAN_FAILED status to SkillVersionStatus. Introduce
SecurityScanService, SecurityScanner port, ScanTask, SecurityAudit
and related domain types. Wire scan trigger into SkillPublishService
so non-auto-publish versions enter scanning when scanner is enabled,
falling back to review task creation when disabled.

* feat(security): add infra layer for scanner HTTP client and adapters

Add WebClient-based HttpClient abstraction with WebClientHttpClient
implementation. Add SkillScannerApiResponse record, SecurityScanException,
and SecurityAuditJpaRepository. Add webflux and test dependencies to
infra module.

* feat(security): add Redis stream consumers, audit API, and DB migration

Add AbstractStreamConsumer base class, ScanTaskConsumer for processing
scan results from Redis stream, and RedisScanTaskProducer. Add
RedisStreamConfig for stream/group initialization. Add SecurityAudit
REST controller and DTO. Add V35 Flyway migration for security_audits
table.

* feat(security): add scanner config to application profiles

Add scanner enabled flag to application-local.yml and
application-test.yml. Enable behavioral analyzer by default
in application.yml.

* feat(deploy): add skill-scanner to docker-compose and k8s manifests

Add skill-scanner service to docker-compose.yml with health check.
Add scanner k8s deployment, service, and configmap entries. Wire
scanner env vars into Makefile dev-all flow. Add verify-scanner.sh
script for post-deploy validation.

* docs(security): add scanner documentation suite

Add scanner docs: configuration guide, failure impact analysis,
monitoring guide, improvement recommendations, custom rules guide,
and skill-vetter rules conversion example. Update deployment docs
with scanner section. Add security-scanning overview and PRD.

* feat(security): add skill-vetter custom rule examples

Add example Regex and YARA rules derived from skill-vetter RED FLAGS
in scanner/examples/vetter-rules/. Includes 7 Regex rules
(signatures-append.yaml) and 3 YARA rules (skillhub_vetter.yara)
covering agent memory theft, IP-based exfiltration, and browser
data theft detection.

* feat(security): add scanner Docker build context

Add Dockerfile for cisco-ai-skill-scanner container and
.env.example with LLM configuration placeholders.

* fix(security): align Finding mapping with scanner API response schema

SkillScannerApiResponse.Finding used incorrect field names (message,
location.file, location.line, code_snippet) that did not match the
scanner's actual JSON output (description, file_path, line_number,
snippet), causing all four fields to deserialize as null.

Flatten Finding to match scanner API: remove nested Location, rename
fields to description/file_path/line_number/snippet. Add skill_name
and timestamp to SkillScannerApiResponse. Extend SecurityFinding with
remediation, analyzer, and metadata fields to capture LLM analyzer
output. Retain 8-arg compact constructor for backward compatibility.

* chore(security): add debug logging to scanner response mapping

Log raw scanner API response and mapped SecurityFinding fields
side-by-side to help verify data consistency between scanner
output and database records.

* feat(security): add multi-scanner support and soft delete for security audits

- Add ScannerType enum for type-safe scanner identification
- Update V35 migration to support multiple scanners and soft delete
- Remove CASCADE delete, use code-level soft delete (deleted_at)
- Add repository methods for querying latest audit by scanner type
- Update SecurityScanService to handle scanner type parameter
- Integrate soft delete in SkillHardDeleteService
- Update all tests to use ScannerType enum

This enables multiple scanner integrations (skill-scanner, future LLM/compliance scanners)
and preserves complete audit history through soft deletion.

* feat(security): add security audit UI to review detail and skill detail pages

Display security scan results on the review detail page (full audit
section with collapsible findings) and the skill detail sidebar (compact
summary with dialog for details). Handles empty/404 gracefully by
returning null, avoids loading shimmer flicker, and separates lifecycle
action buttons with a visual divider.

* docs(security): add security audit UI PRD

* fix(security): replace LocalDateTime with Instant in security audit and align controller test with list API

SecurityAudit and SecurityScanService used LocalDateTime.now() which
violated the project time guardrail. Replaced with Instant and
Clock.systemUTC() to match existing conventions.

Also fixed SecurityAuditControllerTest to mock the correct repository
method (findLatestActiveByVersionId) and assert against the list
response shape.

* test(security): add useQuery mock for security audit components in frontend tests

The SecurityAuditSummary and SecurityAuditSection components use
useQuery via useSecurityAudits hook, which was missing from the
@tanstack/react-query mocks in skill-detail and review-detail tests.

2026-03-23 09:56:03 +08:00

3.6 KiB

Raw Blame History

Skill Scanner Backend Runtime Guide

Overview

SkillHub now supports a backend-only security scanning chain around skill-scanner. The publish flow changes are:

publish request enters SkillPublishService
if scanner is enabled, the version moves to SCANNING
the backend enqueues a ScanTask
ScanTaskConsumer calls skill-scanner
the scan result is stored in security_audit
the version moves to PENDING_REVIEW, or to SCAN_FAILED after final retry exhaustion
review still happens through the existing review workflow

Frontend is intentionally out of scope here. The frontend should fetch audit details through the dedicated backend API instead of expecting scanner data inside existing review detail payloads.

Runtime Modes

Two runtime modes are supported:

local Use POST /scan and pass a filesystem path. This only works when SkillHub and skill-scanner can see the same files.
upload Use POST /scan-upload and upload the package archive. This is the safer default for split deployments.

Recommended usage:

local development with shared filesystem: local
Kubernetes or any split-service deployment: upload

Backend Configuration

Application properties:

skillhub:
  security:
    scanner:
      enabled: false
      base-url: http://localhost:8000
      health-path: /health
      scan-path: /scan-upload
      mode: local
      connect-timeout-ms: 5000
      read-timeout-ms: 300000
      retry-max-attempts: 3
    stream:
      key: skillhub:scan:requests
      group: skillhub-scanners

Important environment variables:

SKILLHUB_SECURITY_SCANNER_ENABLED
SKILLHUB_SECURITY_SCANNER_URL
SKILLHUB_SECURITY_SCANNER_MODE
SKILLHUB_SCAN_STREAM_KEY
SKILLHUB_SCAN_STREAM_GROUP

Scanner-side optional environment variables:

SKILL_SCANNER_LLM_API_KEY
SKILL_SCANNER_LLM_MODEL

If the LLM variables are absent, the scanner should still run with non-LLM analyzers.

Kubernetes Notes

Current repository manifests assume separate skillhub-server and skillhub-scanner deployments. Because these deployments do not share a writable package directory, Kubernetes should use:

SKILLHUB_SECURITY_SCANNER_MODE=upload
SKILLHUB_SECURITY_SCANNER_URL=http://skillhub-scanner:8000

Relevant manifests:

deploy/k8s/scanner-deployment.yaml
deploy/k8s/services.yaml
deploy/k8s/backend-deployment.yaml
deploy/k8s/configmap.yaml

The scanner service is internal-only by default and is consumed by the backend through cluster DNS.

Verification

Verify the scanner service itself:

sh scripts/verify-scanner.sh http://localhost:8000
sh scripts/verify-scanner.sh http://localhost:8000 /path/to/skill.zip

Recommended backend checks after enabling the feature:

publish a test package
confirm the version status becomes SCANNING
confirm a security_audit row is created
confirm the version eventually moves to PENDING_REVIEW or SCAN_FAILED
call GET /api/v1/skills/{skillId}/versions/{versionId}/security-audit

Audit Query API

Backend audit data is available from:

GET /api/v1/skills/{skillId}/versions/{versionId}/security-audit

Response fields include:

scanId
scannerType
verdict
isSafe
maxSeverity
findingsCount
findings
scanDurationSeconds
scannedAt
createdAt

Failure Semantics

scan task retries are handled by AbstractStreamConsumer
final failure marks the version as SCAN_FAILED
even after scan failure, a review task is still created so the package does not get stuck forever

This keeps the existing human review path intact while making scanner failures visible.

3.6 KiB Raw Blame History