mirror of
https://github.com/iflytek/skillhub.git
synced 2026-08-28 11:25:00 +00:00
3bc97ff1b8
* feat(security): extend scanner config with full analyzer options Integrate skill-scanner's 8 analysis engines and policy configuration into SkillHub's config system. Operators can now control behavioral, LLM, Meta, AI Defense, VirusTotal, and trigger analyzers via application.yml or environment variables. Changes: - Add Analyzers and Policy nested classes to SkillScannerProperties - Create ScanOptions record to encapsulate analyzer flags - Update SkillScannerService to pass options in /scan body and /scan-upload query params - Wire ScanOptions through SkillScannerConfig and SkillScannerAdapter - Extend application.yml with full scanner config block and env var overrides - Update all tests to verify new configuration flow All tests pass. * feat(security): add domain model and integrate scan into publish flow Add SCANNING/SCAN_FAILED status to SkillVersionStatus. Introduce SecurityScanService, SecurityScanner port, ScanTask, SecurityAudit and related domain types. Wire scan trigger into SkillPublishService so non-auto-publish versions enter scanning when scanner is enabled, falling back to review task creation when disabled. * feat(security): add infra layer for scanner HTTP client and adapters Add WebClient-based HttpClient abstraction with WebClientHttpClient implementation. Add SkillScannerApiResponse record, SecurityScanException, and SecurityAuditJpaRepository. Add webflux and test dependencies to infra module. * feat(security): add Redis stream consumers, audit API, and DB migration Add AbstractStreamConsumer base class, ScanTaskConsumer for processing scan results from Redis stream, and RedisScanTaskProducer. Add RedisStreamConfig for stream/group initialization. Add SecurityAudit REST controller and DTO. Add V35 Flyway migration for security_audits table. * feat(security): add scanner config to application profiles Add scanner enabled flag to application-local.yml and application-test.yml. Enable behavioral analyzer by default in application.yml. * feat(deploy): add skill-scanner to docker-compose and k8s manifests Add skill-scanner service to docker-compose.yml with health check. Add scanner k8s deployment, service, and configmap entries. Wire scanner env vars into Makefile dev-all flow. Add verify-scanner.sh script for post-deploy validation. * docs(security): add scanner documentation suite Add scanner docs: configuration guide, failure impact analysis, monitoring guide, improvement recommendations, custom rules guide, and skill-vetter rules conversion example. Update deployment docs with scanner section. Add security-scanning overview and PRD. * feat(security): add skill-vetter custom rule examples Add example Regex and YARA rules derived from skill-vetter RED FLAGS in scanner/examples/vetter-rules/. Includes 7 Regex rules (signatures-append.yaml) and 3 YARA rules (skillhub_vetter.yara) covering agent memory theft, IP-based exfiltration, and browser data theft detection. * feat(security): add scanner Docker build context Add Dockerfile for cisco-ai-skill-scanner container and .env.example with LLM configuration placeholders. * fix(security): align Finding mapping with scanner API response schema SkillScannerApiResponse.Finding used incorrect field names (message, location.file, location.line, code_snippet) that did not match the scanner's actual JSON output (description, file_path, line_number, snippet), causing all four fields to deserialize as null. Flatten Finding to match scanner API: remove nested Location, rename fields to description/file_path/line_number/snippet. Add skill_name and timestamp to SkillScannerApiResponse. Extend SecurityFinding with remediation, analyzer, and metadata fields to capture LLM analyzer output. Retain 8-arg compact constructor for backward compatibility. * chore(security): add debug logging to scanner response mapping Log raw scanner API response and mapped SecurityFinding fields side-by-side to help verify data consistency between scanner output and database records. * feat(security): add multi-scanner support and soft delete for security audits - Add ScannerType enum for type-safe scanner identification - Update V35 migration to support multiple scanners and soft delete - Remove CASCADE delete, use code-level soft delete (deleted_at) - Add repository methods for querying latest audit by scanner type - Update SecurityScanService to handle scanner type parameter - Integrate soft delete in SkillHardDeleteService - Update all tests to use ScannerType enum This enables multiple scanner integrations (skill-scanner, future LLM/compliance scanners) and preserves complete audit history through soft deletion. * feat(security): add security audit UI to review detail and skill detail pages Display security scan results on the review detail page (full audit section with collapsible findings) and the skill detail sidebar (compact summary with dialog for details). Handles empty/404 gracefully by returning null, avoids loading shimmer flicker, and separates lifecycle action buttons with a visual divider. * docs(security): add security audit UI PRD * fix(security): replace LocalDateTime with Instant in security audit and align controller test with list API SecurityAudit and SecurityScanService used LocalDateTime.now() which violated the project time guardrail. Replaced with Instant and Clock.systemUTC() to match existing conventions. Also fixed SecurityAuditControllerTest to mock the correct repository method (findLatestActiveByVersionId) and assert against the list response shape. * test(security): add useQuery mock for security audit components in frontend tests The SecurityAuditSummary and SecurityAuditSection components use useQuery via useSecurityAudits hook, which was missing from the @tanstack/react-query mocks in skill-detail and review-detail tests.
131 lines
3.6 KiB
Markdown
131 lines
3.6 KiB
Markdown
# Skill Scanner Backend Runtime Guide
|
|
|
|
## Overview
|
|
|
|
SkillHub now supports a backend-only security scanning chain around `skill-scanner`.
|
|
The publish flow changes are:
|
|
|
|
1. publish request enters `SkillPublishService`
|
|
2. if scanner is enabled, the version moves to `SCANNING`
|
|
3. the backend enqueues a `ScanTask`
|
|
4. `ScanTaskConsumer` calls `skill-scanner`
|
|
5. the scan result is stored in `security_audit`
|
|
6. the version moves to `PENDING_REVIEW`, or to `SCAN_FAILED` after final retry exhaustion
|
|
7. review still happens through the existing review workflow
|
|
|
|
Frontend is intentionally out of scope here. The frontend should fetch audit details through the dedicated backend API instead of expecting scanner data inside existing review detail payloads.
|
|
|
|
## Runtime Modes
|
|
|
|
Two runtime modes are supported:
|
|
|
|
- `local`
|
|
Use `POST /scan` and pass a filesystem path. This only works when SkillHub and `skill-scanner` can see the same files.
|
|
- `upload`
|
|
Use `POST /scan-upload` and upload the package archive. This is the safer default for split deployments.
|
|
|
|
Recommended usage:
|
|
|
|
- local development with shared filesystem: `local`
|
|
- Kubernetes or any split-service deployment: `upload`
|
|
|
|
## Backend Configuration
|
|
|
|
Application properties:
|
|
|
|
```yaml
|
|
skillhub:
|
|
security:
|
|
scanner:
|
|
enabled: false
|
|
base-url: http://localhost:8000
|
|
health-path: /health
|
|
scan-path: /scan-upload
|
|
mode: local
|
|
connect-timeout-ms: 5000
|
|
read-timeout-ms: 300000
|
|
retry-max-attempts: 3
|
|
stream:
|
|
key: skillhub:scan:requests
|
|
group: skillhub-scanners
|
|
```
|
|
|
|
Important environment variables:
|
|
|
|
- `SKILLHUB_SECURITY_SCANNER_ENABLED`
|
|
- `SKILLHUB_SECURITY_SCANNER_URL`
|
|
- `SKILLHUB_SECURITY_SCANNER_MODE`
|
|
- `SKILLHUB_SCAN_STREAM_KEY`
|
|
- `SKILLHUB_SCAN_STREAM_GROUP`
|
|
|
|
Scanner-side optional environment variables:
|
|
|
|
- `SKILL_SCANNER_LLM_API_KEY`
|
|
- `SKILL_SCANNER_LLM_MODEL`
|
|
|
|
If the LLM variables are absent, the scanner should still run with non-LLM analyzers.
|
|
|
|
## Kubernetes Notes
|
|
|
|
Current repository manifests assume **separate** `skillhub-server` and `skillhub-scanner` deployments.
|
|
Because these deployments do not share a writable package directory, Kubernetes should use:
|
|
|
|
```text
|
|
SKILLHUB_SECURITY_SCANNER_MODE=upload
|
|
SKILLHUB_SECURITY_SCANNER_URL=http://skillhub-scanner:8000
|
|
```
|
|
|
|
Relevant manifests:
|
|
|
|
- `deploy/k8s/scanner-deployment.yaml`
|
|
- `deploy/k8s/services.yaml`
|
|
- `deploy/k8s/backend-deployment.yaml`
|
|
- `deploy/k8s/configmap.yaml`
|
|
|
|
The scanner service is internal-only by default and is consumed by the backend through cluster DNS.
|
|
|
|
## Verification
|
|
|
|
Verify the scanner service itself:
|
|
|
|
```bash
|
|
sh scripts/verify-scanner.sh http://localhost:8000
|
|
sh scripts/verify-scanner.sh http://localhost:8000 /path/to/skill.zip
|
|
```
|
|
|
|
Recommended backend checks after enabling the feature:
|
|
|
|
1. publish a test package
|
|
2. confirm the version status becomes `SCANNING`
|
|
3. confirm a `security_audit` row is created
|
|
4. confirm the version eventually moves to `PENDING_REVIEW` or `SCAN_FAILED`
|
|
5. call `GET /api/v1/skills/{skillId}/versions/{versionId}/security-audit`
|
|
|
|
## Audit Query API
|
|
|
|
Backend audit data is available from:
|
|
|
|
```text
|
|
GET /api/v1/skills/{skillId}/versions/{versionId}/security-audit
|
|
```
|
|
|
|
Response fields include:
|
|
|
|
- `scanId`
|
|
- `scannerType`
|
|
- `verdict`
|
|
- `isSafe`
|
|
- `maxSeverity`
|
|
- `findingsCount`
|
|
- `findings`
|
|
- `scanDurationSeconds`
|
|
- `scannedAt`
|
|
- `createdAt`
|
|
|
|
## Failure Semantics
|
|
|
|
- scan task retries are handled by `AbstractStreamConsumer`
|
|
- final failure marks the version as `SCAN_FAILED`
|
|
- even after scan failure, a review task is still created so the package does not get stuck forever
|
|
|
|
This keeps the existing human review path intact while making scanner failures visible.
|