mirror of https://github.com/iflytek/skillhub.git synced 2026-08-28 11:25:00 +00:00

XiaoSeS 3bc97ff1b8 feat(security): add security scanning system with multi-scanner support and frontend UI (#144 )

* feat(security): extend scanner config with full analyzer options

Integrate skill-scanner's 8 analysis engines and policy configuration
into SkillHub's config system. Operators can now control behavioral,
LLM, Meta, AI Defense, VirusTotal, and trigger analyzers via
application.yml or environment variables.

Changes:
- Add Analyzers and Policy nested classes to SkillScannerProperties
- Create ScanOptions record to encapsulate analyzer flags
- Update SkillScannerService to pass options in /scan body and /scan-upload query params
- Wire ScanOptions through SkillScannerConfig and SkillScannerAdapter
- Extend application.yml with full scanner config block and env var overrides
- Update all tests to verify new configuration flow

All tests pass.

* feat(security): add domain model and integrate scan into publish flow

Add SCANNING/SCAN_FAILED status to SkillVersionStatus. Introduce
SecurityScanService, SecurityScanner port, ScanTask, SecurityAudit
and related domain types. Wire scan trigger into SkillPublishService
so non-auto-publish versions enter scanning when scanner is enabled,
falling back to review task creation when disabled.

* feat(security): add infra layer for scanner HTTP client and adapters

Add WebClient-based HttpClient abstraction with WebClientHttpClient
implementation. Add SkillScannerApiResponse record, SecurityScanException,
and SecurityAuditJpaRepository. Add webflux and test dependencies to
infra module.

* feat(security): add Redis stream consumers, audit API, and DB migration

Add AbstractStreamConsumer base class, ScanTaskConsumer for processing
scan results from Redis stream, and RedisScanTaskProducer. Add
RedisStreamConfig for stream/group initialization. Add SecurityAudit
REST controller and DTO. Add V35 Flyway migration for security_audits
table.

* feat(security): add scanner config to application profiles

Add scanner enabled flag to application-local.yml and
application-test.yml. Enable behavioral analyzer by default
in application.yml.

* feat(deploy): add skill-scanner to docker-compose and k8s manifests

Add skill-scanner service to docker-compose.yml with health check.
Add scanner k8s deployment, service, and configmap entries. Wire
scanner env vars into Makefile dev-all flow. Add verify-scanner.sh
script for post-deploy validation.

* docs(security): add scanner documentation suite

Add scanner docs: configuration guide, failure impact analysis,
monitoring guide, improvement recommendations, custom rules guide,
and skill-vetter rules conversion example. Update deployment docs
with scanner section. Add security-scanning overview and PRD.

* feat(security): add skill-vetter custom rule examples

Add example Regex and YARA rules derived from skill-vetter RED FLAGS
in scanner/examples/vetter-rules/. Includes 7 Regex rules
(signatures-append.yaml) and 3 YARA rules (skillhub_vetter.yara)
covering agent memory theft, IP-based exfiltration, and browser
data theft detection.

* feat(security): add scanner Docker build context

Add Dockerfile for cisco-ai-skill-scanner container and
.env.example with LLM configuration placeholders.

* fix(security): align Finding mapping with scanner API response schema

SkillScannerApiResponse.Finding used incorrect field names (message,
location.file, location.line, code_snippet) that did not match the
scanner's actual JSON output (description, file_path, line_number,
snippet), causing all four fields to deserialize as null.

Flatten Finding to match scanner API: remove nested Location, rename
fields to description/file_path/line_number/snippet. Add skill_name
and timestamp to SkillScannerApiResponse. Extend SecurityFinding with
remediation, analyzer, and metadata fields to capture LLM analyzer
output. Retain 8-arg compact constructor for backward compatibility.

* chore(security): add debug logging to scanner response mapping

Log raw scanner API response and mapped SecurityFinding fields
side-by-side to help verify data consistency between scanner
output and database records.

* feat(security): add multi-scanner support and soft delete for security audits

- Add ScannerType enum for type-safe scanner identification
- Update V35 migration to support multiple scanners and soft delete
- Remove CASCADE delete, use code-level soft delete (deleted_at)
- Add repository methods for querying latest audit by scanner type
- Update SecurityScanService to handle scanner type parameter
- Integrate soft delete in SkillHardDeleteService
- Update all tests to use ScannerType enum

This enables multiple scanner integrations (skill-scanner, future LLM/compliance scanners)
and preserves complete audit history through soft deletion.

* feat(security): add security audit UI to review detail and skill detail pages

Display security scan results on the review detail page (full audit
section with collapsible findings) and the skill detail sidebar (compact
summary with dialog for details). Handles empty/404 gracefully by
returning null, avoids loading shimmer flicker, and separates lifecycle
action buttons with a visual divider.

* docs(security): add security audit UI PRD

* fix(security): replace LocalDateTime with Instant in security audit and align controller test with list API

SecurityAudit and SecurityScanService used LocalDateTime.now() which
violated the project time guardrail. Replaced with Instant and
Clock.systemUTC() to match existing conventions.

Also fixed SecurityAuditControllerTest to mock the correct repository
method (findLatestActiveByVersionId) and assert against the list
response shape.

* test(security): add useQuery mock for security audit components in frontend tests

The SecurityAuditSummary and SecurityAuditSection components use
useQuery via useSecurityAudits hook, which was missing from the
@tanstack/react-query mocks in skill-detail and review-detail tests.

2026-03-23 09:56:03 +08:00

11 KiB

Raw Permalink Blame History

Scanner 配置说明

概述

本文档详细说明 SkillHub Scanner 的所有配置项，包括基础配置、分析器配置、策略配置和运维配置。

配置文件位置

开发环境：server/skillhub-app/src/main/resources/application-local.yml
测试环境：server/skillhub-app/src/main/resources/application-test.yml
生产环境：server/skillhub-app/src/main/resources/application.yml
Kubernetes：deploy/k8s/configmap.yaml 和 deploy/k8s/secret.yaml

完整配置示例

skillhub:
  security:
    scanner:
      # 基础配置
      enabled: ${SKILLHUB_SECURITY_SCANNER_ENABLED:false}
      base-url: ${SKILLHUB_SECURITY_SCANNER_URL:http://localhost:8000}
      mode: ${SKILLHUB_SECURITY_SCANNER_MODE:local}

      # 分析器配置
      analyzers:
        behavioral: ${SKILLHUB_SCANNER_USE_BEHAVIORAL:false}
        llm: ${SKILLHUB_SCANNER_USE_LLM:false}
        llm-provider: ${SKILLHUB_SCANNER_LLM_PROVIDER:anthropic}
        llm-consensus-runs: ${SKILLHUB_SCANNER_LLM_CONSENSUS_RUNS:3}
        meta: ${SKILLHUB_SCANNER_USE_META:true}
        ai-defense: ${SKILLHUB_SCANNER_USE_AI_DEFENSE:false}
        ai-defense-api-key: ${SKILLHUB_SCANNER_AI_DEFENSE_API_KEY:}
        virus-total: ${SKILLHUB_SCANNER_USE_VIRUS_TOTAL:false}
        trigger: ${SKILLHUB_SCANNER_USE_TRIGGER:false}

      # 策略配置
      policy:
        preset: ${SKILLHUB_SCANNER_POLICY_PRESET:balanced}
        custom-policy-path: ${SKILLHUB_SCANNER_CUSTOM_POLICY_PATH:}
        fail-on-severity: ${SKILLHUB_SCANNER_FAIL_ON_SEVERITY:high}

配置项详解

1. 基础配置

`enabled`

类型：Boolean
默认值：false
环境变量：SKILLHUB_SECURITY_SCANNER_ENABLED
说明：是否启用安全扫描功能
影响：
- true：技能包发布时会触发安全扫描
- false：跳过安全扫描，直接进入审核流程

示例：

# 开发环境：禁用扫描
enabled: false

# 生产环境：启用扫描
enabled: true

`base-url`

类型：String (URL)
默认值：http://localhost:8000
环境变量：SKILLHUB_SECURITY_SCANNER_URL
说明：Scanner 服务的基础 URL
格式：http(s)://host:port

示例：

# 本地开发
base-url: http://localhost:8000

# Kubernetes 内部服务
base-url: http://skill-scanner:8000

# 外部服务
base-url: https://scanner.example.com

`mode`

类型：String (Enum)
可选值：local | upload
默认值：local
环境变量：SKILLHUB_SECURITY_SCANNER_MODE
说明：扫描模式
- local：Scanner 直接访问本地文件系统（适用于 Scanner 和 SkillHub 在同一主机）
- upload：通过 HTTP 上传 ZIP 文件（适用于 Scanner 和 SkillHub 分离部署）

示例：

# Docker Compose 环境（共享卷）
mode: local

# Kubernetes 环境（独立 Pod）
mode: upload

2. 分析器配置

`analyzers.behavioral`

类型：Boolean
默认值：false
环境变量：SKILLHUB_SCANNER_USE_BEHAVIORAL
说明：是否启用行为分析引擎
功能：检测可疑的运行时行为（如文件系统访问、网络请求等）

`analyzers.llm`

类型：Boolean
默认值：false
环境变量：SKILLHUB_SCANNER_USE_LLM
说明：是否启用 LLM 分析引擎
功能：使用大语言模型进行代码语义分析
依赖：需要配置 llm-provider

`analyzers.llm-provider`

类型：String (Enum)
可选值：anthropic | openai | azure
默认值：anthropic
环境变量：SKILLHUB_SCANNER_LLM_PROVIDER
说明：LLM 提供商
依赖：需要在 Scanner 服务中配置对应的 API Key

示例：

# 使用 Anthropic Claude
llm-provider: anthropic

# 使用 OpenAI GPT
llm-provider: openai

# 使用 Azure OpenAI
llm-provider: azure

`analyzers.llm-consensus-runs`

类型：Integer
默认值：3
范围：1-10
环境变量：SKILLHUB_SCANNER_LLM_CONSENSUS_RUNS
说明：LLM 共识运行次数（多次运行取共识结果，提高准确性）
性能影响：值越大，扫描时间越长，但准确性越高

`analyzers.meta`

类型：Boolean
默认值：true
环境变量：SKILLHUB_SCANNER_USE_META
说明：是否启用元数据分析引擎
功能：检查 package.json、依赖版本、许可证等元数据

`analyzers.ai-defense`

类型：Boolean
默认值：false
环境变量：SKILLHUB_SCANNER_USE_AI_DEFENSE
说明：是否启用 AI Defense 引擎
功能：使用 AI Defense API 进行高级威胁检测
依赖：需要配置 ai-defense-api-key

`analyzers.ai-defense-api-key`

类型：String (Secret)
默认值：空字符串
环境变量：SKILLHUB_SCANNER_AI_DEFENSE_API_KEY
说明：AI Defense API Key
安全：应通过 Kubernetes Secret 或环境变量注入，不要硬编码

`analyzers.virus-total`

类型：Boolean
默认值：false
环境变量：SKILLHUB_SCANNER_USE_VIRUS_TOTAL
说明：是否启用 VirusTotal 引擎
功能：使用 VirusTotal API 检测已知恶意文件

`analyzers.trigger`

类型：Boolean
默认值：false
环境变量：SKILLHUB_SCANNER_USE_TRIGGER
说明：是否启用触发器分析引擎
功能：检测可疑的触发器模式（如定时任务、事件监听等）

3. 策略配置

`policy.preset`

类型：String (Enum)
可选值：strict | balanced | permissive
默认值：balanced
环境变量：SKILLHUB_SCANNER_POLICY_PRESET
说明：安全策略预设
- strict：严格模式，任何可疑行为都会标记为不安全
- balanced：平衡模式，只标记高风险行为
- permissive：宽松模式，只标记明确的恶意行为

示例：

# 生产环境：使用严格模式
preset: strict

# 开发环境：使用宽松模式
preset: permissive

`policy.custom-policy-path`

类型：String (File Path)
默认值：空字符串
环境变量：SKILLHUB_SCANNER_CUSTOM_POLICY_PATH
说明：自定义策略文件路径（覆盖 preset）
格式：YAML 文件

示例：

# 使用自定义策略
custom-policy-path: /etc/skillhub/scanner-policy.yaml

`policy.fail-on-severity`

类型：String (Enum)
可选值：critical | high | medium | low
默认值：high
环境变量：SKILLHUB_SCANNER_FAIL_ON_SEVERITY
说明：扫描失败的严重级别门槛
- critical：只有发现 critical 级别的问题才标记为不安全
- high：发现 high 或 critical 级别的问题标记为不安全
- medium：发现 medium、high 或 critical 级别的问题标记为不安全
- low：发现任何级别的问题都标记为不安全

示例：

# 生产环境：high 及以上标记为不安全
fail-on-severity: high

# 测试环境：只有 critical 标记为不安全
fail-on-severity: critical

环境变量配置

Docker Compose

# docker-compose.yml
services:
  skillhub-backend:
    environment:
      - SKILLHUB_SECURITY_SCANNER_ENABLED=true
      - SKILLHUB_SECURITY_SCANNER_URL=http://skill-scanner:8000
      - SKILLHUB_SECURITY_SCANNER_MODE=local
      - SKILLHUB_SCANNER_USE_BEHAVIORAL=false
      - SKILLHUB_SCANNER_USE_LLM=false
      - SKILLHUB_SCANNER_USE_META=true
      - SKILLHUB_SCANNER_POLICY_PRESET=balanced
      - SKILLHUB_SCANNER_FAIL_ON_SEVERITY=high

Kubernetes ConfigMap

# deploy/k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: skillhub-config
data:
  SKILLHUB_SECURITY_SCANNER_ENABLED: "true"
  SKILLHUB_SECURITY_SCANNER_URL: "http://skill-scanner:8000"
  SKILLHUB_SECURITY_SCANNER_MODE: "upload"
  SKILLHUB_SCANNER_USE_BEHAVIORAL: "false"
  SKILLHUB_SCANNER_USE_LLM: "false"
  SKILLHUB_SCANNER_USE_META: "true"
  SKILLHUB_SCANNER_POLICY_PRESET: "balanced"
  SKILLHUB_SCANNER_FAIL_ON_SEVERITY: "high"

Kubernetes Secret

# deploy/k8s/secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: skillhub-secrets
type: Opaque
stringData:
  SKILLHUB_SCANNER_AI_DEFENSE_API_KEY: "your-api-key-here"

配置最佳实践

1. 开发环境配置

skillhub:
  security:
    scanner:
      enabled: false  # 开发时禁用扫描，加快迭代速度
      base-url: http://localhost:8000
      mode: local
      analyzers:
        meta: true  # 只启用元数据分析
      policy:
        preset: permissive  # 使用宽松策略
        fail-on-severity: critical

2. 测试环境配置

skillhub:
  security:
    scanner:
      enabled: true  # 测试环境启用扫描
      base-url: http://skill-scanner:8000
      mode: local
      analyzers:
        behavioral: true
        meta: true
        llm: false  # LLM 分析较慢，测试环境可选
      policy:
        preset: balanced
        fail-on-severity: high

3. 生产环境配置

skillhub:
  security:
    scanner:
      enabled: true  # 生产环境必须启用扫描
      base-url: http://skill-scanner:8000
      mode: upload  # 使用上传模式，更安全
      analyzers:
        behavioral: true
        llm: true  # 启用 LLM 分析，提高准确性
        llm-provider: anthropic
        llm-consensus-runs: 3
        meta: true
        ai-defense: true  # 启用高级威胁检测
        virus-total: true
        trigger: true
      policy:
        preset: strict  # 使用严格策略
        fail-on-severity: high

性能调优

扫描速度 vs 准确性

配置	扫描时间	准确性	适用场景
只启用 meta	~5 秒	低	开发环境
meta + behavioral	~15 秒	中	测试环境
meta + behavioral + llm	~60 秒	高	生产环境
全部启用	~120 秒	最高	高安全要求

故障排查

问题 1：Scanner 连接失败

检查配置：

# 检查 base-url 是否正确
curl -f $SKILLHUB_SECURITY_SCANNER_URL/health

# 检查网络连通性
ping skill-scanner

问题 2：扫描超时

调整配置：

# 减少 LLM 共识运行次数
analyzers:
  llm-consensus-runs: 1  # 从 3 降到 1

# 或禁用 LLM 分析
analyzers:
  llm: false

问题 3：扫描失败率高

调整策略：

# 降低严重级别门槛
policy:
  fail-on-severity: critical  # 从 high 改为 critical

# 或使用宽松策略
policy:
  preset: permissive  # 从 strict 改为 permissive

11 KiB

Raw Permalink Blame History

Scanner 配置说明

概述

配置文件位置

完整配置示例

配置项详解

1. 基础配置

`enabled`

`base-url`

`mode`

2. 分析器配置

`analyzers.behavioral`

`analyzers.llm`

`analyzers.llm-provider`

`analyzers.llm-consensus-runs`

`analyzers.meta`

`analyzers.ai-defense`

`analyzers.ai-defense-api-key`

`analyzers.virus-total`

`analyzers.trigger`

3. 策略配置

`policy.preset`

`policy.custom-policy-path`

`policy.fail-on-severity`

环境变量配置

Docker Compose

Kubernetes ConfigMap

Kubernetes Secret

配置最佳实践

1. 开发环境配置

2. 测试环境配置

3. 生产环境配置

性能调优

扫描速度 vs 准确性

推荐配置

故障排查

问题 1：Scanner 连接失败

问题 2：扫描超时

问题 3：扫描失败率高

相关文档

11 KiB Raw Permalink Blame History Unescape Escape

Scanner 配置说明

概述

配置文件位置

完整配置示例

配置项详解

1. 基础配置

enabled

base-url

mode

2. 分析器配置

analyzers.behavioral

analyzers.llm

analyzers.llm-provider

analyzers.llm-consensus-runs

analyzers.meta

analyzers.ai-defense

analyzers.ai-defense-api-key

analyzers.virus-total

analyzers.trigger

3. 策略配置

policy.preset

policy.custom-policy-path

policy.fail-on-severity

环境变量配置

Docker Compose

Kubernetes ConfigMap

Kubernetes Secret

配置最佳实践

1. 开发环境配置

2. 测试环境配置

3. 生产环境配置

性能调优

扫描速度 vs 准确性

推荐配置

故障排查

问题 1：Scanner 连接失败

问题 2：扫描超时

问题 3：扫描失败率高

相关文档

11 KiB

Raw Permalink Blame History

`enabled`

`base-url`

`mode`

`analyzers.behavioral`

`analyzers.llm`

`analyzers.llm-provider`

`analyzers.llm-consensus-runs`

`analyzers.meta`

`analyzers.ai-defense`

`analyzers.ai-defense-api-key`

`analyzers.virus-total`

`analyzers.trigger`

`policy.preset`

`policy.custom-policy-path`

`policy.fail-on-severity`