Commit graph

221 commits

Author SHA1 Message Date
dongmucat
f902c6f59b fix(search): restore clawhub namespace-only access 2026-04-21 17:28:05 +08:00
dongmucat
2c519c8efd fix(search): harden portal search query parsing 2026-04-20 16:57:47 +08:00
wrj97
7be6a36960
feat(auth): add GitLab OAuth2 provider support (#264)
* feat(auth): add GitLab OAuth2 provider support

Add GitLab as an additional OAuth2 authentication provider alongside
GitHub. This includes:

- GitLab OAuth2 client configuration with customizable base URL
- GitLabClaimsExtractor for handling GitLab-specific user claims
- Multi-provider login UI with provider-specific icons
- Updated localization to use OAuth-agnostic terminology
- JSON type annotation for IdentityBinding entity

* fix(auth): restore oauth redirect and gitlab email checks

* test(auth): align oauth login handler expectation

---------

Co-authored-by: wowo-zZ <zhenggui5228@126.com>
2026-04-17 19:56:32 +08:00
XiaoSeS
080be28eef
fix(search): align portal visibility for private skills (#323)
* feat(ci): add AI-powered release notes generation

- Add GitHub Models integration for automated release notes
- Support bilingual (EN) release notes with highlights extraction
- Fallback to conventional commit grouping when LLM unavailable
- Trigger on tag push or manual workflow dispatch
- Zero configuration: uses GitHub Models (gpt-4o-mini) by default

* chore: pin action versions and update gitignore

- Pin checkout and setup-deno to commit hashes matching project convention
- Add .playwright-mcp/ and .mcp.json to gitignore

* fix(search): keep private skills out of portal discovery

- remove super admin expanded visibility from portal search
- exclude all private skills from portal search results
- keep namespace-only skills visible only to namespace members
- reserve private skill access for my-skills and future admin surfaces

* test(search): update test to reflect portal visibility policy

- rename test from platformWideAccessShouldBypass... to platformWideAccessShouldNotBypass...
- verify that platformWideAccess flag does not grant extra visibility in portal search
- verify PRIVATE skills are excluded from search SQL
- aligns test with new requirement: portal search uses standard visibility rules

* test(search): update app service test for portal visibility policy

- rename test from search_shouldGrantPlatformWideAccessToSuperAdmin to search_shouldNotGrantPlatformWideAccessToSuperAdminInPortal
- verify super admin does not get platformWideAccess in portal search
- aligns with requirement: portal uses standard visibility, admin features reserved for dedicated interface
2026-04-17 17:46:05 +08:00
dongmucat
2cfd4a730b
Merge pull request #302 from iflytek/fix/rerelease-precheck-warnings
fix(rerelease): support precheck warning confirmation flow
2026-04-14 20:28:00 +08:00
dongmucat
5d87a0ccd3
Merge pull request #304 from iflytek/feat/oss-02-super-admin-visibility
feat(access): add SUPER_ADMIN platform role support
2026-04-14 20:27:39 +08:00
xiose
7c2f06d1b6 test(rerelease): add confirmWarnings coverage and sync generated schema
- Add domain tests for rerelease with precheck warnings (reject + confirm)
- Add controller test verifying confirmWarnings passthrough
- Sync SkillVersionRereleaseRequest generated type with backend DTO
2026-04-14 17:55:56 +08:00
dongmucat
1246cca930
Merge pull request #279 from iflytek/fix/issue-276-review-own-namespace
fix(review): allow namespace admins to review own submissions
2026-04-14 17:22:08 +08:00
xiose
edcc248244 fix(portal): keep skill detail on viewer permissions 2026-04-14 16:42:22 +08:00
xiose
a1e4904d97 fix(i18n): update version delete error message to include UPLOADED and SCAN_FAILED
The error message for unsupported version deletion still referenced only
DRAFT/REJECTED. Updated both EN and ZH messages to reflect the actual
deletable statuses: DRAFT, UPLOADED, REJECTED, SCAN_FAILED.

Also updated OSS-02 design doc to mark all blocking items as completed.
2026-04-14 15:49:25 +08:00
xiose
9801b549fc feat(access): add SUPER_ADMIN platform role support
- Add platformRoles parameter to VisibilityChecker.canAccess() for platform-level access control
- SUPER_ADMIN can access all skills regardless of visibility or publication status
- Add archived namespace check to SkillQueryService.getSkillDetail()
- Extract platformRoles from AuthContext in SkillController
- Replace VisibilityChecker mock with real instance in SkillQueryServiceTest
- Add 5 new tests for SUPER_ADMIN access scenarios
- Add version-status-badge.tsx component for frontend status display

Tests: 347 domain tests + 16 app tests passing
2026-04-14 14:18:43 +08:00
xiose
b597a6f511 merge: resolve conflicts after merging origin/main
- Remove duplicate useSubmitForReview/useConfirmPublish in hooks and client
- Restore resolveVersionStatusLabel (version-status-badge not in this branch)
- Fix missing Card closing tag in versions tab
2026-04-14 13:44:30 +08:00
xiose
b26fe6a364 fix(rerelease): support precheck warning confirmation flow
- Backend: Add confirmWarnings parameter to rerelease DTO, domain service, and app service
- Frontend: Add warning dialog with retry logic when precheck warnings are detected
- i18n: Add rerelease warning dialog translations (en/zh)

Fixes the issue where rereleasing a published version with secret detection warnings
always fails with 400 error. Now follows the same confirm-and-retry pattern as initial publish.
2026-04-14 11:58:15 +08:00
dongmucat
c419a119f1 merge: sync origin/main into fix/security-hardening-unauth 2026-04-14 10:28:25 +08:00
dongmucat
38ebb13133
feat(auth): 邮箱验证码重置密码与 SMTP 配置支持 (#273)
* feat(auth): add email-based password reset with SMTP config docs

* test(e2e): stabilize password reset flow

* test(e2e): isolate password reset rate limits

* test(ci): stabilize backend and register e2e

* docs(auth): sanitize smtp setup examples
2026-04-13 20:27:00 +08:00
dongmucat
1184e00a00
fix(compat): support namespace-aware clawhub publish (#291) 2026-04-13 20:26:42 +08:00
dongmucat
38757084ba fix(review): restore namespace admin review access 2026-04-13 17:03:41 +08:00
XiaoSeS
532d0450aa
feat(skill): add UPLOADED status for PRIVATE skill lifecycle (#290)
* feat(skill): add UPLOADED status for PRIVATE skill lifecycle

## Summary
- Add UPLOADED status for PRIVATE skills after security scan passes
- PRIVATE skill owners can test before confirming publish or submitting for review
- Rerelease now follows visibility rules (PRIVATE→UPLOADED, PUBLIC→PENDING_REVIEW)
- Auto-withdraw changes status to UPLOADED (not DRAFT) to keep versions visible

## Changes
- SkillVersionStatus: Add UPLOADED enum value
- SkillPublishService: PRIVATE skills go to UPLOADED after scan
- SecurityScanService: Visibility-based status transition after scan
- SkillGovernanceService: Withdraw→UPLOADED, delete allows UPLOADED
- SkillQueryService: Include UPLOADED in version list filters
- SkillReviewSubmitService: New service for submit-review and confirm-publish
- SkillLifecycleController: Add submit-review and confirm-publish endpoints
- Frontend: Add buttons, dialogs, and hooks for new operations

## Workflow
- PRIVATE: Publish → SCANNING → UPLOADED → confirm-publish → PUBLISHED
- PUBLIC: Publish → SCANNING → PENDING_REVIEW → PUBLISHED

* feat(review): add backward compatibility for DRAFT status

Support both DRAFT (legacy) and UPLOADED (new flow) status in:
- SkillReviewSubmitService.submitForReview
- SkillReviewSubmitService.confirmPublish
- ReviewService.submitReview (both overloads)

This ensures existing data with DRAFT status continues to work
with the new visibility-based workflow introduced in OSS-02.
2026-04-13 11:39:45 +08:00
xiose
f55c520ebe feat(skill): add UPLOADED status for PRIVATE skill lifecycle
## Summary
- Add UPLOADED status for PRIVATE skills after security scan passes
- PRIVATE skill owners can test before confirming publish or submitting for review
- Rerelease now follows visibility rules (PRIVATE→UPLOADED, PUBLIC→PENDING_REVIEW)
- Auto-withdraw changes status to UPLOADED (not DRAFT) to keep versions visible

## Changes
- SkillVersionStatus: Add UPLOADED enum value
- SkillPublishService: PRIVATE skills go to UPLOADED after scan
- SecurityScanService: Visibility-based status transition after scan
- SkillGovernanceService: Withdraw→UPLOADED, delete allows UPLOADED
- SkillQueryService: Include UPLOADED in version list filters
- SkillReviewSubmitService: New service for submit-review and confirm-publish
- SkillLifecycleController: Add submit-review and confirm-publish endpoints
- Frontend: Add buttons, dialogs, and hooks for new operations

## Workflow
- PRIVATE: Publish → SCANNING → UPLOADED → confirm-publish → PUBLISHED
- PUBLIC: Publish → SCANNING → PENDING_REVIEW → PUBLISHED
2026-04-13 09:26:01 +08:00
wowo
2def67b037
feat(publish): relax pre-publish checks into warning + confirm flow (#288)
* feat(publish): allow warning-confirmed pre-publish checks\n\nFixes #287

* fix(i18n): add missing register validation translation keys

The registration form uses i18n keys like register.usernameInvalid,
register.passwordTooShort etc. but they were never defined in the
locale files, causing E2E tests to fail because the raw key strings
were displayed instead of human-readable messages.
2026-04-12 19:15:57 +08:00
wowo
348eb4e717
fix(storage): defer S3 bucket verification until first access (#289)
* fix(storage): defer S3 bucket verification until first access

* test(storage): cover deferred S3 bucket verification

* fix(runtime): widen backend container healthcheck window

* fix(runtime): widen backend container healthcheck window

* fix(test): use ddl-auto=create to prevent cross-context table drops

Multiple @SpringBootTest classes with different @MockBean configs
cause separate Spring contexts sharing the same H2 in-memory database.
With create-drop, one context's shutdown drops tables needed by another,
causing "Table not found (this database is empty)" errors.

* fix(test): widen awaitIndexedDocument timeout to 15s

CI runners are resource-constrained and async search indexing may not
complete within the previous 5-second window, causing flaky failures.
2026-04-12 15:24:09 +08:00
Uğur Tafralı
02b3ac5b62
fix: add support for .cjs and .mjs JavaScript extensions (#285) 2026-04-11 11:15:36 +08:00
dongmucat
40807e7fa0 test(app): isolate H2 db per Spring test context 2026-04-10 10:12:27 +08:00
dongmucat
27b631a5d6 merge: sync origin/main into fix/security-hardening-unauth 2026-04-10 09:26:19 +08:00
dongmucat
cd7c1ba384
fix(review): correct review queue totals (#265) 2026-04-09 18:12:02 +08:00
dongmucat
b24cc58338
fix(search): include permitted skills in clawhub explore (#258) 2026-04-09 18:11:47 +08:00
dongmucat
44f556037b merge: sync origin/main into fix/security-hardening-unauth 2026-04-09 18:11:35 +08:00
tenten-shih
77e271f24c
Merge pull request #269 from iflytek/pr/exclude-playwright-report
fix register validation, review visibility, and search e2e coverage
2026-04-09 14:37:46 +05:30
wowo
8f694ddc7c
[codex] add issue triage automation mvp (#268)
* add issue triage automation mvp

* Document issue automation design in Chinese

* Fix legacy compat slug tests
2026-04-09 15:04:33 +08:00
huishi3
2167981392 fix(review): sync approval state before returning tasks 2026-04-09 09:56:58 +05:30
Seasoning
739e21e0fd
Feat/namespace member display username (#236)
* feat: display username and email in namespace member management

MemberResponse DTO now includes displayName and email fields.
NamespacePortalQueryAppService batch-loads UserAccount data via
findByIdIn to avoid N+1 queries. Frontend member table shows
username (with userId as fallback subtitle) and email columns
instead of raw user IDs.

* test(namespace): add displayName/email assertions and new test coverage

- Controller: add displayName and email assertions to addMember and updateMemberRole tests
- QueryAppService: add listMembers tests for batch user enrichment and null degradation
- CommandAppService: add addMember/updateMemberRole tests for displayName/email population and graceful degradation

Addresses PR #236 review feedback comments:
- Issue 2: Controller tests now assert displayName and email from mocked UserAccount
- Issue 3: Service layer tests now cover batch query and null user degradation
2026-04-08 20:02:20 +08:00
dongmucat
441dc9e0e7 fix(security): close unauthorized metrics and compat access paths 2026-04-07 16:24:15 +08:00
dongmucat
0b84e4eff3
fix(compat): support canonical query slug for resolve/download (#227)
Co-authored-by: huihan3 <huihan3@iflytek.com>
2026-04-03 17:27:02 +08:00
eryuan
5d7bee2c58 Pr/fix sse timeout global exception handler (#178)
* feat : 调整本地开发环境

* feat : 调整sse 超时 GlobalExceptionHandler 拦截异常

See :
rg.springframework.web.context.request.async.AsyncRequestTimeoutException: null
	at org.springframework.web.context.request.async.TimeoutDeferredResultProcessingInterceptor.handleTimeout(TimeoutDeferredResultProcessingInterceptor.java:42) ~[spring-web-6.1.4.jar:6.1.4]
	at org.springframework.web.context.request.async.DeferredResultInterceptorChain.triggerAfterTimeout(DeferredResultInterceptorChain.java:81) ~[spring-web-6.1.4.jar:6.1.4]
	at org.springframework.web.context.request.async.WebAsyncManager.lambda$startDeferredResultProcessing$5(WebAsyncManager.java:430) ~[spring-web-6.1.4.jar:6.1.4]
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
	at org.springframework.web.context.request.async.StandardServletAsyncWebRequest.onTimeout(StandardServletAsyncWebRequest.java:149) ~[spring-web-6.1.4.jar:6.1.4]
	at org.apache.catalina.core.AsyncListenerWrapper.fireOnTimeout(AsyncListenerWrapper.java:44) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.catalina.core.AsyncContextImpl.timeout(AsyncContextImpl.java:136) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:135) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:243) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:57) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
	at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]

* Revert "feat : 调整sse 超时 GlobalExceptionHandler 拦截异常"

This reverts commit 6f0cea85b5daac47025767efe1bd90565b5da469.

* Revert "feat : 调整本地开发环境"

This reverts commit 0695a3d7c823491ce5c83aabe3660ba9997f28c5.

* feat : 调整sse 超时 GlobalExceptionHandler 拦截异常

See :
rg.springframework.web.context.request.async.AsyncRequestTimeoutException: null
    at org.springframework.web.context.request.async.TimeoutDeferredResultProcessingInterceptor.handleTimeout(TimeoutDeferredResultProcessingInterceptor.java:42) ~[spring-web-6.1.4.jar:6.1.4]
    at org.springframework.web.context.request.async.DeferredResultInterceptorChain.triggerAfterTimeout(DeferredResultInterceptorChain.java:81) ~[spring-web-6.1.4.jar:6.1.4]
    at org.springframework.web.context.request.async.WebAsyncManager.lambda$startDeferredResultProcessing$5(WebAsyncManager.java:430) ~[spring-web-6.1.4.jar:6.1.4]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
    at org.springframework.web.context.request.async.StandardServletAsyncWebRequest.onTimeout(StandardServletAsyncWebRequest.java:149) ~[spring-web-6.1.4.jar:6.1.4]
    at org.apache.catalina.core.AsyncListenerWrapper.fireOnTimeout(AsyncListenerWrapper.java:44) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.catalina.core.AsyncContextImpl.timeout(AsyncContextImpl.java:136) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:135) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:243) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:57) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.19.jar:10.1.19]
    at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]

* fix(app): stabilize async timeout handling

---------

Co-authored-by: zhaieryuan <zhaieryuan@enn.cn>
Co-authored-by: wowo-zZ <zhenggui5228@126.com>
2026-03-31 11:20:42 +08:00
wowo
34de5bb53e feat(upload): allow xsd, xsl, dtd file types in skill packages (#185)
* feat(upload): allow xsd, xsl, dtd file types in skill packages

Add XML schema-related file extensions (.xsd, .xsl, .dtd) to the
upload allowlist and text content validation. Users uploading skills
with XML Schema files (e.g., Anthropic's docx skill) were getting
rejected because .xsd was not in the allowed extensions list.

Closes #165

* feat(upload): expand office file allowlist

* test(app): verify publish extension env override

* docs(readme): document upload allowlist override
2026-03-30 18:13:37 +08:00
XiaoSeS
1237d42119 Webclient fix (#173)
* fix(scanner): wire timeout config and create dedicated scanner HttpClient

- Create scanner-specific HttpClient bean with proper timeout configuration
- Wire SkillScannerProperties.connectTimeoutMs (5s) and readTimeoutMs (5min) into WebClient
- Add logging for timeout configuration verification
- Fix cloud environment 5-second timeout issue by ensuring responseTimeout is properly applied

The previous global WebClient.Builder configuration was not being used correctly,
causing scanner requests to timeout after 5 seconds in cloud environment.
This fix creates a dedicated HttpClient for scanner with explicit timeout settings.

* refactor(webclient): use WebClientCustomizer and add scanner connection pool

- Replace singleton WebClient.Builder bean with WebClientCustomizer
  (follows Spring Boot best practice for prototype-scoped builder)
- Add connection pool config to scanner HttpClient (maxConn=10,
  maxIdleTime=20s, evictInBackground=30s) to prevent stale connections
- Add connectTimeout to global WebClient config

* fix(skill): use system default timezone for auto-generated version numbers

- Change AUTO_VERSION_FORMATTER from UTC to ZoneId.systemDefault()
- Version format yyyyMMdd.HHmmss now uses server's local timezone
- Update test to validate format instead of exact value (timezone-independent)

This allows the service to adapt to deployment location:
- Deployed in China → uses Asia/Shanghai timezone
- Deployed in US → uses US timezone
- Follows server's system timezone configuration
2026-03-27 15:41:58 +08:00
XiaoSeS
90c1f3c805 fix(scanner): wire timeout config and create dedicated scanner HttpClient (#166)
- Create scanner-specific HttpClient bean with proper timeout configuration
- Wire SkillScannerProperties.connectTimeoutMs (5s) and readTimeoutMs (5min) into WebClient
- Add logging for timeout configuration verification
- Fix cloud environment 5-second timeout issue by ensuring responseTimeout is properly applied

The previous global WebClient.Builder configuration was not being used correctly,
causing scanner requests to timeout after 5 seconds in cloud environment.
This fix creates a dedicated HttpClient for scanner with explicit timeout settings.
2026-03-27 11:16:07 +08:00
XiaoSeS
175eb8e1ae fix(scanner): stage upload bundles from object storage (#164)
* fix(scanner): stage upload bundles from object storage

* fix(web): wrap long skill detail values
2026-03-26 17:08:23 +08:00
XiaoSeS
86b4d0508b fix(skill): defer version storage deletion until commit (#162)
* fix(skill): resolve duplicate result error when deleting skill with shared namespace+slug

V13 migration changed the unique constraint from (namespace_id, slug) to
(namespace_id, slug, owner_id), allowing multiple skills with the same
namespace+slug but different owners. The findByNamespaceSlugAndSlug query
returned Optional<Skill> which threw IncorrectResultSizeDataAccessException
when 2 rows matched.

Changed the query to return List<Skill> and added ownerId query param to
DELETE endpoints so the frontend can specify exactly which skill to delete.

* fix(review): keep failed scans reviewable and expose feedback

* fix(skill-delete): delete skills by id

* fix(skill): defer version storage deletion until commit
2026-03-26 13:54:30 +08:00
XiaoSeS
a78684ce44 Fix/skill delete duplicate result (#159)
* fix(skill): resolve duplicate result error when deleting skill with shared namespace+slug

V13 migration changed the unique constraint from (namespace_id, slug) to
(namespace_id, slug, owner_id), allowing multiple skills with the same
namespace+slug but different owners. The findByNamespaceSlugAndSlug query
returned Optional<Skill> which threw IncorrectResultSizeDataAccessException
when 2 rows matched.

Changed the query to return List<Skill> and added ownerId query param to
DELETE endpoints so the frontend can specify exactly which skill to delete.

* fix(review): keep failed scans reviewable and expose feedback
2026-03-25 16:15:50 +08:00
XiaoSeS
6020006a25 fix: scanner redis sentinel, publish flush, version delete, clipboard fallback (#157)
* fix(scanner): migrate scan stream to redisson

* fix(web): update security audit pending labels

* fix(scanner): support redis sentinel in redisson config

* fix(db): allow deleting skill versions with audit history

* fix(publish): flush replaced versions before recreation

* fix(i18n): localize agent setup command in Chinese locale

Translate the agent quick-start prompt from English to Chinese and add
a test to verify both locale strings.

* fix(clipboard): add fallback for insecure contexts

navigator.clipboard is undefined in HTTP/iframe/WebView contexts,
causing TypeError on copy. Extract copyToClipboard() utility with
execCommand fallback and apply across all 6 call sites.
2026-03-24 20:26:50 +08:00
XiaoSeS
0fcc40cd4c fix(scanner): support redis sentinel in redisson config (#154)
* fix(scanner): migrate scan stream to redisson

* fix(web): update security audit pending labels

* fix(scanner): support redis sentinel in redisson config
2026-03-24 15:54:39 +08:00
XiaoSeS
747f649fb7 fix: migrate scanner stream to redisson and refine audit status labels` (#153)
* fix(scanner): migrate scan stream to redisson

* fix(web): update security audit pending labels
2026-03-24 14:56:16 +08:00
vsxd
eda8764b30 fix: db migration script version 2026-03-23 20:23:50 +08:00
yun-zhi-ztl
a3d7a81d0c chore(notification): sync main and expand regression coverage 2026-03-23 15:04:47 +08:00
XiaoSeS
ee33505b7c feat: File Browser Sidebar with Preview, Security Audit UI, and Syntax Highlighting (#145)
* feat(security): extend scanner config with full analyzer options

Integrate skill-scanner's 8 analysis engines and policy configuration
into SkillHub's config system. Operators can now control behavioral,
LLM, Meta, AI Defense, VirusTotal, and trigger analyzers via
application.yml or environment variables.

Changes:
- Add Analyzers and Policy nested classes to SkillScannerProperties
- Create ScanOptions record to encapsulate analyzer flags
- Update SkillScannerService to pass options in /scan body and /scan-upload query params
- Wire ScanOptions through SkillScannerConfig and SkillScannerAdapter
- Extend application.yml with full scanner config block and env var overrides
- Update all tests to verify new configuration flow

All tests pass.

* feat(security): add domain model and integrate scan into publish flow

Add SCANNING/SCAN_FAILED status to SkillVersionStatus. Introduce
SecurityScanService, SecurityScanner port, ScanTask, SecurityAudit
and related domain types. Wire scan trigger into SkillPublishService
so non-auto-publish versions enter scanning when scanner is enabled,
falling back to review task creation when disabled.

* feat(security): add infra layer for scanner HTTP client and adapters

Add WebClient-based HttpClient abstraction with WebClientHttpClient
implementation. Add SkillScannerApiResponse record, SecurityScanException,
and SecurityAuditJpaRepository. Add webflux and test dependencies to
infra module.

* feat(security): add Redis stream consumers, audit API, and DB migration

Add AbstractStreamConsumer base class, ScanTaskConsumer for processing
scan results from Redis stream, and RedisScanTaskProducer. Add
RedisStreamConfig for stream/group initialization. Add SecurityAudit
REST controller and DTO. Add V35 Flyway migration for security_audits
table.

* feat(security): add scanner config to application profiles

Add scanner enabled flag to application-local.yml and
application-test.yml. Enable behavioral analyzer by default
in application.yml.

* feat(deploy): add skill-scanner to docker-compose and k8s manifests

Add skill-scanner service to docker-compose.yml with health check.
Add scanner k8s deployment, service, and configmap entries. Wire
scanner env vars into Makefile dev-all flow. Add verify-scanner.sh
script for post-deploy validation.

* docs(security): add scanner documentation suite

Add scanner docs: configuration guide, failure impact analysis,
monitoring guide, improvement recommendations, custom rules guide,
and skill-vetter rules conversion example. Update deployment docs
with scanner section. Add security-scanning overview and PRD.

* feat(security): add skill-vetter custom rule examples

Add example Regex and YARA rules derived from skill-vetter RED FLAGS
in scanner/examples/vetter-rules/. Includes 7 Regex rules
(signatures-append.yaml) and 3 YARA rules (skillhub_vetter.yara)
covering agent memory theft, IP-based exfiltration, and browser
data theft detection.

* feat(security): add scanner Docker build context

Add Dockerfile for cisco-ai-skill-scanner container and
.env.example with LLM configuration placeholders.

* fix(security): align Finding mapping with scanner API response schema

SkillScannerApiResponse.Finding used incorrect field names (message,
location.file, location.line, code_snippet) that did not match the
scanner's actual JSON output (description, file_path, line_number,
snippet), causing all four fields to deserialize as null.

Flatten Finding to match scanner API: remove nested Location, rename
fields to description/file_path/line_number/snippet. Add skill_name
and timestamp to SkillScannerApiResponse. Extend SecurityFinding with
remediation, analyzer, and metadata fields to capture LLM analyzer
output. Retain 8-arg compact constructor for backward compatibility.

* chore(security): add debug logging to scanner response mapping

Log raw scanner API response and mapped SecurityFinding fields
side-by-side to help verify data consistency between scanner
output and database records.

* feat(security): add multi-scanner support and soft delete for security audits

- Add ScannerType enum for type-safe scanner identification
- Update V35 migration to support multiple scanners and soft delete
- Remove CASCADE delete, use code-level soft delete (deleted_at)
- Add repository methods for querying latest audit by scanner type
- Update SecurityScanService to handle scanner type parameter
- Integrate soft delete in SkillHardDeleteService
- Update all tests to use ScannerType enum

This enables multiple scanner integrations (skill-scanner, future LLM/compliance scanners)
and preserves complete audit history through soft deletion.

* feat(security): add security audit UI to review detail and skill detail pages

Display security scan results on the review detail page (full audit
section with collapsible findings) and the skill detail sidebar (compact
summary with dialog for details).  Handles empty/404 gracefully by
returning null, avoids loading shimmer flicker, and separates lifecycle
action buttons with a visual divider.

* docs(security): add security audit UI PRD

* docs(prd): add related documents section to file browser sidebar PRD

* feat(skill): add file tree builder utility

Implement buildFileTree function to convert flat SkillFile[] into hierarchical tree structure.
- Nodes sorted alphabetically by path
- Support for nested directories
- Each node includes depth, type, and path information
- Includes comprehensive test coverage

* feat(skill): add file type detection utilities

Implement utilities for file type detection and preview capability checks:
- isPreviewable: checks if file can be previewed based on extension and size
- canPreviewFile: provides detailed reason when file cannot be previewed
- getFileTypeLabel: returns human-readable file type labels
- getFileIcon: maps file types to appropriate Lucide icons
- Supports 1MB max file size limit
- Comprehensive test coverage for all functions

* feat(skill): add file tree node component

Implement recursive FileTreeNodeComponent with:
- Expand/collapse functionality for directories
- File and directory icons from Lucide React
- Hover effects showing file sizes
- Proper indentation based on depth
- Support for nested directory structures

* refactor(skill): upgrade file tree to hierarchical structure

Replace flat file list with tree structure:
- Use buildFileTree to convert flat files into hierarchy
- Integrate FileTreeNodeComponent for rendering
- Update onFileClick to accept FileTreeNode instead of SkillFile
- Add file count badge in header
- Root-level directories expanded by default

* feat(skill): add file preview dialog component

Implement FilePreviewDialog with:
- Markdown rendering via existing MarkdownRenderer
- Plain text/code display with monospace font
- Non-previewable file message with download button
- Copy-to-clipboard and download actions in header
- File path display in footer
- Loading and error states

* feat(i18n): add file tree and preview translations

Add Chinese and English translations for:
- File browser title
- Preview error messages (load failure, too large, binary, unsupported)
- Download file button

* feat(skill): integrate file preview into skill detail page

- Add useSkillFile hook for fetching arbitrary file content
- Add file preview state, click handler, and download handler
- Pass onFileClick to FileTree for opening preview dialog
- Add FilePreviewDialog to skill detail page

* feat(review): add file reading API endpoint

Add GET /api/v1/reviews/{id}/file?path=... for reading single files
from the review-bound skill version:
- ReviewController: new endpoint with path traversal validation
- GovernanceWorkflowAppService: route method for review file access
- ReviewSkillDetailAppService: authorization and delegation
- SkillQueryService: getFileContentByVersionId for direct version access

* feat(review): integrate file preview into review detail

- Add useReviewFile hook for fetching review file content
- Add file preview state and handlers to ReviewSkillDetailSection
- Pass reviewId prop for API calls
- Add FilePreviewDialog to review detail expanded section
- Update review-detail.tsx to pass taskId as reviewId

* fix(test): update test mocks for file preview hooks

- Add useSkillFile mock to skill-detail.test.tsx
- Add useQuery mock to @tanstack/react-query mock
- Add useReviewFile mock to review-skill-detail-section.test.tsx
- Fix SkillFile test fixtures to include all required properties
- Remove unused imports in test and component files

* fix(skill): use DomainBadRequestException for version not found

Replace DomainNotFoundException with DomainBadRequestException in
getFileContentByVersionId to match existing patterns and imports.

* feat(skill): add file tree sidebar with file sizes

- Move file tree to right sidebar for persistent visibility
- Display file sizes always visible (not just on hover)
- Keep Files tab showing the same tree structure
- Add scrollable container with max-height for long file lists

* fix(skill): improve file tree sidebar UI and UX

- Remove redundant title from file tree sidebar
- Adjust scrollbar placement with proper flex layout
- Sort folders first, then files (both alphabetically)
- Collapse all folders by default for cleaner initial view
- Increase dialog width to max-w-5xl and height to 90vh
- Remove duplicate close button (X icon)
- Add hover effects to action buttons
- Use flexbox for proper content scrolling
- Allow .pyc files in backend configuration

Fixes:
1. File tree sidebar now has cleaner layout without title
2. Scrollbar properly contained within card
3. Folders appear before files in tree
4. All folders collapsed by default
5. Preview dialog wider and more usable
6. Single close method (ESC or click outside)
7. Action buttons have visual feedback on hover

* fix(skill): refine file tree sidebar and preview dialog

1. Default all folders to collapsed state
2. File tree sidebar now uses Card+bare pattern matching SecurityAuditSummary
   - Scrollbar inside the card content area instead of wrapping the tree
   - Consistent padding and header layout with other sidebar cards
3. Preview dialog close button fixed:
   - Hide DialogContent's built-in close button via [&>button]:hidden
   - Add X button in the header row, same level as copy/download
   - All action buttons share consistent hover animation (opacity transition)
4. Sort directories before files at every tree level
5. Update tests for new directory-first sort order

* feat(skill): collapsible file browser, wider dialog, button animations

1. File browser sidebar now supports expand/collapse toggle
   - Click the header row to toggle visibility
   - ChevronUp/Down icon indicates state
   - Default expanded on page load

2. File preview dialog widened to 72rem (was 5xl/64rem)
   - Uses w-[min(calc(100vw-2rem),72rem)] for responsive max

3. Header action buttons now have micro-interactions:
   - Copy/Download: scale up on hover, scale down on click
   - Close (X): rotates 90° on hover with destructive tint
   - All buttons: opacity 60→100 transition on hover

* feat(skill): chevron animation, button effects, download path hint, review sidebar

1. File browser chevron: single ChevronDown with rotate-180 CSS transition
   instead of swapping two icons — smooth 200ms rotation animation

2. Preview dialog action button animations:
   - Copy icon: active:scale-125 feedback on click
   - Download icon: hover:translate-y-0.5 subtle downward motion
   - Close icon: hover:rotate-90 with destructive tint (unchanged)

3. Download tooltip now shows full file path:
   "下载 src/prompts/system.md" instead of generic "下载文件"
   - New i18n keys: filePreview.downloadHint, filePreview.copy, filePreview.close

4. Review detail page now has a sidebar file browser:
   - Two-column layout (main content + lg:w-80 sidebar)
   - File tree sourced from reviewSkillDetail.files (active review version)
   - Collapsible with same chevron animation as skill detail page
   - Shows active version badge below file tree
   - File preview uses review file API (/reviews/{id}/file?path=...)
   - Test mocks updated for new useReviewFile dependency

* fix(skill): download shows filename, copy rotates with success toast

1. Download button tooltip/text now shows actual filename:
   "下载 README.md" instead of generic path
   - i18n key changed from {{path}} to {{name}}

2. Copy button animation reworked to rotation:
   - idle: hover:rotate-180 on the Copy icon
   - click: animate-spin during clipboard write
   - done: swap to green Check icon for 1.5s, then reset

3. Copy success toast notification:
   - Shows "已复制到剪贴板" / "Copied to clipboard" via toast.success
   - New i18n key: filePreview.copySuccess

* fix(skill): set download attribute so browser saves with original filename

The <a> element was missing the download attribute, causing the browser
to derive the filename from the URL path segment (/file?path=...) which
always resolved to "file". Now explicitly sets link.download = node.name
so the saved file uses the original name (e.g. README.md, config.json).

Applied to all three download handlers:
- skill-detail.tsx (skill version file)
- review-detail.tsx (review sidebar file)
- review-skill-detail-section.tsx (review expanded section file)

* feat(skill): add icons to all sidebar card headers

Add Lucide icons to each sidebar section for visual consistency
with the existing SecurityAuditSummary (Shield) and file browser (Folder):

- Terminal: Install command
- Clock: Pending review (amber-colored to match card theme)
- RefreshCw: Lifecycle management
- Tag: Label management (in SkillLabelPanel component)
- ArrowUpCircle: Promotion
- ShieldCheck: Governance

All icons use the same pattern: w-4 h-4 text-muted-foreground
placed inside a flex row with gap-2 before the section title.

* fix(skill): allow owners to preview all version statuses

When a skill has only a REJECTED version, the detail page crashed with
400 "版本未发布" because assertPreviewAccessible() only allowed PUBLISHED
and PENDING_REVIEW(owner). The frontend fallback to versions[0] would
pick the REJECTED version and fail all file/readme queries.

Backend changes:
- assertPreviewAccessible(): now allows owners and namespace admins to
  preview ANY version status (DRAFT, REJECTED, YANKED, SCANNING, etc.)
  via canManageRestrictedSkill() check. Also passes userNsRoles so
  namespace admin role is respected, not just owner.
- resolveOwnerPendingPreview(): expanded from PENDING_REVIEW-only to
  include all non-published, non-yanked statuses so headlineVersion
  resolves correctly for owners with REJECTED/DRAFT/SCANNING versions.

Frontend changes:
- isOwnerPreviewResolution(): relaxed from checking PENDING_REVIEW
  specifically to checking any non-PUBLISHED status, matching the
  backend's broader projection.

Fixes: owner viewing skill with only REJECTED version → 400 error
Also fixes: DRAFT-only, SCANNING-only, SCAN_FAILED-only scenarios

* docs(requirements): add comprehensive requirements for file preview syntax highlighting

Generated complete documentation suite including:
- Core PRD (clarity score: 95/100)
- Constraints specification (8 business rules)
- Impact analysis (8 risks identified)
- Acceptance cases (25 test cases: 10 positive, 5 error, 7 boundary, 3 security)
- Test plan (100% coverage)

Mode: Quick (3 clarification rounds)
Complexity: Simple (3.5 days estimated)

* feat(skill): add syntax highlighting for code file preview

Implement syntax highlighting for code files in the file preview dialog:
- Add CodeRenderer component using lowlight (highlight.js wrapper via rehype-highlight)
- Add getLanguageForHighlight() to map file extensions to highlight.js languages
- Support 20+ languages: Python, JS/TS, Java, Go, Rust, C/C++, Ruby, PHP, Shell, JSON, YAML, XML, etc.
- Apply 500KB threshold for syntax highlighting (larger files show plain text)
- Maintain visual consistency with Markdown code blocks
- Auto-adapt to light/dark theme using existing CSS variables
- Add lowlight as direct dependency to enable programmatic syntax highlighting

Implements solution 1 from PRD (reuse rehype-highlight infrastructure).

* fix(security): resolve security audit FK constraint and stale query issues

Backend:
- Add hardDeleteByVersionId to physically remove audit records before
  deleting skill versions (fixes FK constraint violation on hard delete)
- Add softDeleteByVersionId calls in SkillGovernanceService and
  SkillPublishService version deletion paths
- Add V36 migration to convert security_audit TIMESTAMP columns to
  TIMESTAMPTZ matching project convention
- Update tests to verify audit cleanup in all deletion flows

Frontend:
- Prevent stale API calls after skill deletion by gating all query
  hooks with a skillDeleted flag that disables them immediately
- Move cache cleanup from useDeleteSkill onSuccess to the handler
  to avoid refetching while the component is still mounted

* fix(skill): handle SCANNING and SCAN_FAILED statuses across all pages

Backend:
- Include SCANNING/SCAN_FAILED in version listing filters so owners
  can see versions being scanned
- Add explicit lifecycle sort priority for scanner statuses
- Allow deletion of SCAN_FAILED versions alongside DRAFT and REJECTED

Frontend:
- Add i18n labels and CSS classes for SCANNING/SCAN_FAILED in my-skills
- Add localized version status labels in skill-detail version list
- Hide archive button when no published version exists (my-skills and
  skill-detail)
- Allow deleting SCAN_FAILED versions from skill-detail

* gitignore
2026-03-23 14:38:47 +08:00
vsxd
fcfd9b203b fix(review): add pagination and time sorting to audit queues 2026-03-23 14:32:54 +08:00
vsxd
ddf9e6e1d2 refactor: address code review findings from 2026-03-22
Implements 6 high-priority improvements from code review:

Backend:
- Make label business limits configurable via application.yml (max-definitions, max-per-skill)

Frontend:
- Split use-skill-queries.ts into domain-specific modules (label/namespace/user/skill)
- Enable @typescript/no-explicit-any as warning and clean up 10 any types
- Unify API error handling by removing unwrap() in favor of fetchJson()

Testing:
- Add label system scenarios to smoke test script
- Add 6 new tests for label management UI (validation, rendering)

All tests passing: 301 backend, 129 frontend
2026-03-23 13:48:23 +08:00
yun-zhi-ztl
6e8b257abb feat(notification): add in-app notifications and harden delivery 2026-03-23 12:31:28 +08:00
XiaoSeS
3bc97ff1b8 feat(security): add security scanning system with multi-scanner support and frontend UI (#144)
* feat(security): extend scanner config with full analyzer options

Integrate skill-scanner's 8 analysis engines and policy configuration
into SkillHub's config system. Operators can now control behavioral,
LLM, Meta, AI Defense, VirusTotal, and trigger analyzers via
application.yml or environment variables.

Changes:
- Add Analyzers and Policy nested classes to SkillScannerProperties
- Create ScanOptions record to encapsulate analyzer flags
- Update SkillScannerService to pass options in /scan body and /scan-upload query params
- Wire ScanOptions through SkillScannerConfig and SkillScannerAdapter
- Extend application.yml with full scanner config block and env var overrides
- Update all tests to verify new configuration flow

All tests pass.

* feat(security): add domain model and integrate scan into publish flow

Add SCANNING/SCAN_FAILED status to SkillVersionStatus. Introduce
SecurityScanService, SecurityScanner port, ScanTask, SecurityAudit
and related domain types. Wire scan trigger into SkillPublishService
so non-auto-publish versions enter scanning when scanner is enabled,
falling back to review task creation when disabled.

* feat(security): add infra layer for scanner HTTP client and adapters

Add WebClient-based HttpClient abstraction with WebClientHttpClient
implementation. Add SkillScannerApiResponse record, SecurityScanException,
and SecurityAuditJpaRepository. Add webflux and test dependencies to
infra module.

* feat(security): add Redis stream consumers, audit API, and DB migration

Add AbstractStreamConsumer base class, ScanTaskConsumer for processing
scan results from Redis stream, and RedisScanTaskProducer. Add
RedisStreamConfig for stream/group initialization. Add SecurityAudit
REST controller and DTO. Add V35 Flyway migration for security_audits
table.

* feat(security): add scanner config to application profiles

Add scanner enabled flag to application-local.yml and
application-test.yml. Enable behavioral analyzer by default
in application.yml.

* feat(deploy): add skill-scanner to docker-compose and k8s manifests

Add skill-scanner service to docker-compose.yml with health check.
Add scanner k8s deployment, service, and configmap entries. Wire
scanner env vars into Makefile dev-all flow. Add verify-scanner.sh
script for post-deploy validation.

* docs(security): add scanner documentation suite

Add scanner docs: configuration guide, failure impact analysis,
monitoring guide, improvement recommendations, custom rules guide,
and skill-vetter rules conversion example. Update deployment docs
with scanner section. Add security-scanning overview and PRD.

* feat(security): add skill-vetter custom rule examples

Add example Regex and YARA rules derived from skill-vetter RED FLAGS
in scanner/examples/vetter-rules/. Includes 7 Regex rules
(signatures-append.yaml) and 3 YARA rules (skillhub_vetter.yara)
covering agent memory theft, IP-based exfiltration, and browser
data theft detection.

* feat(security): add scanner Docker build context

Add Dockerfile for cisco-ai-skill-scanner container and
.env.example with LLM configuration placeholders.

* fix(security): align Finding mapping with scanner API response schema

SkillScannerApiResponse.Finding used incorrect field names (message,
location.file, location.line, code_snippet) that did not match the
scanner's actual JSON output (description, file_path, line_number,
snippet), causing all four fields to deserialize as null.

Flatten Finding to match scanner API: remove nested Location, rename
fields to description/file_path/line_number/snippet. Add skill_name
and timestamp to SkillScannerApiResponse. Extend SecurityFinding with
remediation, analyzer, and metadata fields to capture LLM analyzer
output. Retain 8-arg compact constructor for backward compatibility.

* chore(security): add debug logging to scanner response mapping

Log raw scanner API response and mapped SecurityFinding fields
side-by-side to help verify data consistency between scanner
output and database records.

* feat(security): add multi-scanner support and soft delete for security audits

- Add ScannerType enum for type-safe scanner identification
- Update V35 migration to support multiple scanners and soft delete
- Remove CASCADE delete, use code-level soft delete (deleted_at)
- Add repository methods for querying latest audit by scanner type
- Update SecurityScanService to handle scanner type parameter
- Integrate soft delete in SkillHardDeleteService
- Update all tests to use ScannerType enum

This enables multiple scanner integrations (skill-scanner, future LLM/compliance scanners)
and preserves complete audit history through soft deletion.

* feat(security): add security audit UI to review detail and skill detail pages

Display security scan results on the review detail page (full audit
section with collapsible findings) and the skill detail sidebar (compact
summary with dialog for details).  Handles empty/404 gracefully by
returning null, avoids loading shimmer flicker, and separates lifecycle
action buttons with a visual divider.

* docs(security): add security audit UI PRD

* fix(security): replace LocalDateTime with Instant in security audit and align controller test with list API

SecurityAudit and SecurityScanService used LocalDateTime.now() which
violated the project time guardrail. Replaced with Instant and
Clock.systemUTC() to match existing conventions.

Also fixed SecurityAuditControllerTest to mock the correct repository
method (findLatestActiveByVersionId) and assert against the list
response shape.

* test(security): add useQuery mock for security audit components in frontend tests

The SecurityAuditSummary and SecurityAuditSection components use
useQuery via useSecurityAudits hook, which was missing from the
@tanstack/react-query mocks in skill-detail and review-detail tests.
2026-03-23 09:56:03 +08:00