From 4661d2e1aab4153c014dce59e501cdb9b5cca421 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=8D=83=E4=B9=98=E5=A6=8D=20=28Xiaoyaner=29?= <258399167+xiaoyaner0201@users.noreply.github.com> Date: Thu, 29 Oct 2026 23:19:46 +0800 Subject: [PATCH] docs(analytics): document configuration-only deployment and acceptance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace manual injection instructions with Compose, Docker and Helm configuration, explicit platform initialization, consent controls, privacy boundaries and verified task status. Refs #989 Signed-off-by: 千乘妍 (Xiaoyaner) <258399167+xiaoyaner0201@users.noreply.github.com> --- charts/skillhub/README.md | 7 + docs/09-deployment.md | 2 + docs/frontend-analytics-integration.md | 337 +++++++----- docs/frontend-analytics.md | 498 +++++------------- .../design.md | 8 + .../proposal.md | 2 +- .../tasks.md | 24 +- 7 files changed, 375 insertions(+), 503 deletions(-) diff --git a/charts/skillhub/README.md b/charts/skillhub/README.md index 640aa0795..3cb6a041e 100644 --- a/charts/skillhub/README.md +++ b/charts/skillhub/README.md @@ -105,6 +105,13 @@ PVC UID 与绑定 PV 在升级前后保持不变。四个场景分别覆盖默 Sentinel、实际 MinIO S3 连接，以及由 Kubernetes API 接受的 TLS Ingress 路由。 默认清理自己创建的 namespace；设置 `KEEP_HELM_SMOKE=true` 可保留现场用于排查。 +### 可选前端动作统计 + +默认关闭。`web.analytics.config` 接受 JSON 字符串，`web.analytics.upstream` 接受仅部署可见的 +HTTP(S) origin。支持 Umami，不需要编辑 JS 或 Nginx；先在平台创建网站并取得 UUID。 +配置变更触发 Web 滚动更新，旧标签页不会自动更新。字段、授权接口、初始化、隐私及验收见 +[前端动作统计接入指南](../../docs/frontend-analytics-integration.md)。 + ### 高可用模式 ```bash diff --git a/docs/09-deployment.md b/docs/09-deployment.md index af6c3462b..001f0bfbf 100644 --- a/docs/09-deployment.md +++ b/docs/09-deployment.md @@ -1,6 +1,8 @@ # skillhub 部署架构与运维 对象存储可选部署、凭证和迁移说明见[对象存储手册](object-storage.md)。 +可选前端动作统计默认关闭。Docker／Compose 和 Helm 只填配置即可接入 Umami， +详见[前端动作统计接入指南](frontend-analytics-integration.md)。 ### CLI 设备授权地址 diff --git a/docs/frontend-analytics-integration.md b/docs/frontend-analytics-integration.md index ad1c5e942..909a8648b 100644 --- a/docs/frontend-analytics-integration.md +++ b/docs/frontend-analytics-integration.md @@ -1,80 +1,145 @@ # 前端动作统计接入指南 -## 1. 先了解边界 +## 1. 默认关闭，按需开启 -SkillHub 页面只提供被动的 `data-analytics-id` 标签，默认不采集、不加载统计 SDK、不发送事件。可选 helper 把允许的原生交互转换成 `{ id, event }`，再交给部署方选择的 adapter。点击下载只表示下载意图，不表示下载、安装成功；提交发布也不表示发布成功。部署方负责授权、隐私合规、保留期限和平台安全。 +不配置时，不加载采集模块或 SDK，也不发送统计请求。正常业务不等待统计平台。 +现在 Docker、Compose、Helm 用户只需填写配置。不需要改 JS、复制文件或手写 Nginx。 +第一版支持 Umami 3.4.1。其他平台需要开发并发布新的注册 adapter，不是任意远程脚本插件。 -本文对应 PR-1（#1010）的手工接入示例，不是产品内置开关。仅通过 Docker／Compose 或 Helm 配置开启、默认关闭和按需加载的能力计划在 PR-2 交付，见 [OpenSpec 变更](../openspec/changes/add-configurable-frontend-analytics/proposal.md)。Issue #989 要等 PR-2 完成并验证后才关闭。精确接口、事件边界和隐私约束以[英文契约](frontend-analytics.md)为准。 +点击下载只是下载意图，不是下载或安装成功。提交发布也不是发布成功。 +部署方负责授权策略、平台安全、数据保留和合规。接口和事件边界见[英文契约](frontend-analytics.md)。 -## 2. 选择接入路径 +## 2. 复用已有 Umami -### 路径 A：复用已有统计平台 +前提：SkillHub 已正常部署；Web 镜像包含本功能；有可达的 Umami 3.4.1。 +先在 Umami 创建网站，拿到公开的网站 UUID。管理员密码、API token 不要放入前端配置。 +确认 Web 容器能够访问平台的 HTTP(S) origin。HTTPS 证书需由镜像的系统 CA 信任。 +自签名平台需先按组织规范配置受信任 CA，不要关闭证书验证。 -先确认平台接收地址、公开站点标识、授权策略和 CSP。不要把平台管理密钥放到浏览器。平台不受支持时，需要部署方编写自己的 adapter；当前版本不能只填平台名字就接入任意平台。 +在受保护的 `.env.release` 中填写下面两项。把示例 UUID 换成自己网站的 UUID。 +JSON 保持一行。单引号用于 Compose 的 dotenv 文件，不是 JSON 的一部分。 -1. 复用 [helper](examples/analytics/observe/v1/skillhub-intents.mjs)，不要修改业务页面或引入厂商 SDK。 -2. 参照 [Umami adapter](examples/analytics/observe/v1/umami-adapter.mjs)实现 `send(intent) -> Promise` 和 `stop()`。send 必须消化同步异常和异步拒绝，不向页面抛错；stop 后不能再派发请求，已经开始的 fetch 无法撤回。 -3. 由 adapter 验证平台参数、清洗最终发送字段、管理 SDK 加载和超时。不要只清洗 helper 回调后就信任 SDK 自动附加的 URL、标题或身份字段。 -4. 在部署方入口中连接 helper 与 adapter，只有策略允许时才初始化： - -```js -import { observeSkillhubIntents } from './skillhub-intents.mjs' -// adapter 由部署方创建；必须先确认当前授权。 -const stopObserver = observeSkillhubIntents({ - root: document, - onIntent: intent => adapter.send(intent), -}) -function stop() { stopObserver(); adapter.stop() } +```dotenv +SKILLHUB_WEB_ANALYTICS_CONFIG='{"enabled":true,"adapter":"umami","consent":"granted","adapterConfig":{"websiteId":"4398de8e-40df-4483-9eea-ed213643525f"}}' +SKILLHUB_WEB_ANALYTICS_UPSTREAM=https://stats.example.com ``` -入口交付、同源代理、授权回调和 CSP 由部署方维护。下面的 edge 仅演示 Umami 两个端点；使用其他平台时，需要自行审核并替换 adapter 和代理端点。不要把平台管理界面代理到公开采集前缀。 +`granted` 表示部署方确认当前策略允许采集，不表示每位用户都点击了同意。 +需要用户单独授权时，初始使用 `denied`，再接入第 5 节的公开接口。本功能不提供同意弹窗。 -### 路径 B：本地自托管 Umami 演示 - -前提：已有能正常使用的 SkillHub Web；已安装 Docker 和 Compose；能使用示例指定的 Umami 3.4.1、Postgres 16、Nginx 1.30.6 镜像；本机 8300 和 8081 端口空闲。所有 shell 命令从仓库根目录运行。示例会创建独立演示网络和临时数据库，不用于生产持久化。请在隔离环境运行。 - -先检查配置，再启动： +所有命令从仓库根目录运行。正常发布配置仍需包含数据库、存储和认证等参数。 ```sh -docker compose -f docs/examples/analytics/compose.yaml config -docker compose -f docs/examples/analytics/compose.yaml up -d +make validate-release-config +docker compose --env-file .env.release -f compose.release.yml config --quiet +docker compose --env-file .env.release -f compose.release.yml up -d --no-deps --force-recreate web +docker compose --env-file .env.release -f compose.release.yml logs --tail=100 web ``` -默认上游是 `host.docker.internal:80`，对应 release Compose 的 `WEB_PORT` 默认 80。这里是容器访问宿主机的 Web 入口，不是后端端口，也不是 edge 自己的 8081。若 Web 实际发布在其他端口（比如 18080），先在当前终端导出，再启动。后面所有 compose 命令都要在这个终端里执行，否则重建 edge 时会回到默认的 80： +不要公开分享完整的 Compose 渲染结果，其中可能包含业务凭证。 +Web 启动时自动生成同源代理，只开放 `/_skillhub-analytics/script.js` 和 `/_skillhub-analytics/api/send`。 +其他采集路径返回 404，不公开 Umami 管理界面。统计 DNS 在请求时解析，不是 Web 启动前提。 +统计配置错误只关闭统计。容器日志中的 `[analytics]` 提示字段，不回显原值。 +原有 API/auth 配置、100 MiB 上传上限、TLS、业务缓存及权限不变。不放宽 CSP。 + +直接运行 Docker 时，把同样两项用 `--env-file` 传给正常的 Web 容器即可。文件使用原始 +`KEY=value` 格式，JSON 外不要加 dotenv 单引号。保留原有 API、公开地址和端口配置。 +不要把 API 上游改成统计平台地址。 + +## 3. 可选自托管组合 + +需要 Docker Compose、可用的镜像网络及持久磁盘。这个组合增加 Umami 和它自己的 +Postgres，不增加 Nginx，也不修改 SkillHub 数据库。正式使用前配置备份和访问控制。 + +在 `.env.release` 追加独立随机凭证。先在本地生成，再把输出填入文件，不要提交文件。 ```sh -export SKILLHUB_WEB_UPSTREAM=host.docker.internal:18080 -docker compose -f docs/examples/analytics/compose.yaml up -d +openssl rand -hex 24 +openssl rand -hex 32 ``` -原生 Linux Docker 需要 `host.docker.internal` 映射，示例 edge 已包含 `extra_hosts: ["host.docker.internal:host-gateway"]`。Docker Desktop／OrbStack 已提供这个名字。宿主机 Web 必须监听容器可达的地址；如果防火墙或仅回环监听阻止访问，先调整可达性，或填写容器可以访问的其他 `host:port`。不要填写 `http://` 或 URL 路径。 +```dotenv +UMAMI_DB_PASSWORD=<第一条命令生成的值> +UMAMI_APP_SECRET=<第二条命令生成的值> +UMAMI_ADMIN_PORT=8300 +``` -打开 `http://localhost:8300`。首次演示登录是 `admin` / `umami`，立即改密码。示例公开的数据库口令和 APP_SECRET 也只能用于演示；生产需要独立密钥、持久化、备份和访问控制。 - -在 Umami 创建一个演示网站，记录网站 UUID。打开 `docs/examples/analytics/observe/v1/observer.mjs`： - -1. 将全零 `website` 改为刚创建的网站 UUID。 -2. 将 `enabled: false` 改为 `enabled: true`。 -3. 将 `hasConsent()` 接入自己的当前授权判断。仅在隔离演示、明确允许采集时可暂改为返回 true；正式接入不能把它当作用户同意。 -4. 按下一节配置 rules。示例额外的 `skill.example.open` 没有对应产品元素，不会凭空产生事件。 - -然后重建 edge，使单文件挂载更新可靠生效： +数据库口令使用字母和数字最简单。包含特殊字符时，另填 `UMAMI_DATABASE_URL`，对 URL +密码部分做百分号编码；Postgres 的 `UMAMI_DB_PASSWORD` 仍保留原值。 +先只启动平台，此时统计配置可以留空： ```sh -docker compose -f docs/examples/analytics/compose.yaml up -d --force-recreate edge -docker compose -f docs/examples/analytics/compose.yaml exec edge nginx -t -docker compose -f docs/examples/analytics/compose.yaml exec edge nginx -T +docker compose --env-file .env.release -f compose.release.yml -f compose.analytics-umami.yml config --quiet +docker compose --env-file .env.release -f compose.release.yml -f compose.analytics-umami.yml up -d analytics-postgres umami ``` -打开 `http://localhost:8081`，不要从原 Web 入口验收注入。 +管理端口只绑定宿主机回环地址。远程机器需使用自己的 SSH 转发或受控管理入口。 +本机打开 `http://localhost:8300`。Umami 初始登录是 `admin` / `umami`，立即修改密码。 +创建网站，复制 UUID。此步骤不能省略，启动容器不会自动创建网站。 +将第 2 节 JSON 的 websiteId 换成该 UUID，再填写： -注意缓存：模块按 immutable 长期缓存，浏览器会一直用第一次拿到的 `observer.mjs`。本地演示每次改完入口文件，都要在开发者工具 Network 勾选「Disable cache」后再刷新，或者用一个新的无痕窗口打开。正式交付中不能原地改文件，要发布到新的版本目录（如 `v2`）并同步注入路径，不能覆盖旧版本字节。 +```dotenv +SKILLHUB_WEB_ANALYTICS_UPSTREAM=http://umami:3000 +``` -## 3. 配置规则和增加动作 +这里是 Compose 内部服务名。然后重建正常 Web： -省略 `rules` 使用冻结的 `DEFAULT_RULES`，共 12 条： +```sh +docker compose --env-file .env.release -f compose.release.yml -f compose.analytics-umami.yml up -d --no-deps --force-recreate web +``` -| ID | 规则 | +同一 Compose 网络提供 DNS。Web 在独立容器或主机时需填写真正可达的内部 origin。 +不使用旧示例的 edge 或宿主机 Web 代理。统计平台初始化、改密、备份仍是运维步骤， +不能把首次部署称为无需准备的一条命令。 + +## 4. Helm + +保留已有生产 values 和 Secret。在 values 中增加以下内容： + +```yaml +web: + analytics: + config: '{"enabled":true,"adapter":"umami","consent":"granted","adapterConfig":{"websiteId":"4398de8e-40df-4483-9eea-ed213643525f"}}' + upstream: https://stats.example.com +``` + +```sh +helm lint --strict charts/skillhub -f values-production.yaml +helm upgrade --install skillhub ./charts/skillhub -n skillhub -f values-production.yaml +kubectl -n skillhub rollout status deployment/skillhub-web +kubectl -n skillhub logs deployment/skillhub-web --tail=100 +``` + +deployment 名称使用 Chart 默认 release 名 `skillhub`；自定义名称时替换命令中的名称。 +值进入 ConfigMap 和 Web 环境变量，checksum 改变会触发 Web 滚动更新。 +Chart 不自动部署 Umami。先在已有平台创建网站，并确认 Pod 出站网络、DNS 和证书可用。 +集群内 origin 使用完整服务名，例如 `http://umami.stats.svc.cluster.local:3000`， +Nginx 的运行时 DNS 不自动补 Pod 的搜索后缀。 + +## 5. 字段、规则和动态授权 + +`SKILLHUB_WEB_ANALYTICS_CONFIG` 是可选 JSON 对象，最大 64 KiB，嵌套最多 32 层。 +空值等同关闭。重复 JSON 键拒绝，避免部署和浏览器解释不同。 + +| 字段 | 含义 | +| --- | --- | +| enabled | 布尔值，默认 false。总开关，start() 不能绕过它。 | +| adapter | 静态注册名，目前只有 umami。 | +| consent | denied 或 granted，默认 denied。 | +| rules | 可选规则对象。省略用默认 12 条；显式提供会替换；空对象不采集。 | +| adapterConfig | 适配器公开参数对象。Umami 只接受 websiteId UUID。 | + +未知配置字段或不合法的平台参数关闭统计。rules 沿用纯 `resolveRules` 的 active/skipped +语义：非法容器没有 active，非法条目跳过，其他合法规则继续工作。日志不输出非法 ID +或原配置。开发者可本地检查 `resolveRules(rules).skipped`，不要发送诊断给平台。 + +`SKILLHUB_WEB_ANALYTICS_UPSTREAM` 只供部署代理使用，不进入浏览器配置。 +只接受 HTTP(S) origin，无凭证、路径、查询、fragment、空白或末尾斜杠。 +不要放入平台管理密钥。浏览器前缀由 Web 自动生成，不能配置任意脚本 URL。 + +默认规则保持冻结： + +| ID | 类型 | | --- | --- | | search.query.submit | submit | | skill.card.open | card | @@ -89,112 +154,116 @@ docker compose -f docs/examples/analytics/compose.yaml exec edge nginx -T | skill.delete.confirm | click | | skill.delete.cancel | click | -显式传入 rules 会替换默认值，空对象表示没有动作。要保留默认值再增加已存在的动作，先合并： +例如只统计搜索和首页导航，可在 JSON 中增加： -```js -import { DEFAULT_RULES, resolveRules } from './skillhub-intents.mjs' -const rules = { ...DEFAULT_RULES, 'navigation.home.open': 'click' } -const { active, skipped } = resolveRules(rules) -console.table(skipped) // 只在本地检查，不发送这些诊断。 +```json +{"rules":{"search.query.submit":"submit","navigation.home.open":"click"}} ``` -把 `active` 传给 helper 的 `rules`。也可在浏览器开发者工具 Console 中直接执行以下代码，无需编辑源码，就能查看无效项： +这是 rules 字段示意，不是完整的开启配置。要保留默认值，需显式把默认条目也写进去。 +ID 匹配 `^[a-z]+(?:\.[a-z]+){2,}$`，只用静态动作类别，不拼用户或资源信息。 +`click` 用于普通按钮/链接，排除表单、选择器、文本框和非按钮 INPUT；`submit` 只用于 +FORM；`card` 只用于已有键盘处理的 DIV role=link。选择、文件、拖放、裁剪不支持这些规则， +不要猜成 click。`skill.detail.render` 不是自动 pageview。 -```js -const { resolveRules } = await import('/_skillhub-observe/v1/skillhub-intents.mjs') -console.table(resolveRules({ 'skill.example.open': 'drop' }).skipped) -``` - -应看到 `id: skill.example.open`、`reason: unknown rule type`。非法 ID、非字符串规则也会跳过；非法容器返回 `id: null` 的诊断和空 active。该检查只验证配置，不能证明 DOM 元素支持某类事件。 - -仅支持三种规则： - -- `click`：普通按钮、链接；忽略 FORM、SELECT、TEXTAREA、非按钮 INPUT（包括文件选择器）。 -- `submit`：只用于 FORM。点击提交和按 Enter 都归一到原生 submit；原生校验失败可能没有事件。 -- `card`：已有 Enter／Space 处理的 DIV role=link，观察 click 和对应 keydown。helper 不会替应用增加键盘行为。 - -不支持的选择器、拖放、文件和裁剪行为不要硬填 click。未配置的动作不采集。`skill.detail.render` 是结构标签，不是自动页面访问量。 - -新增产品动作时，开发者在正确的现有元素上增加静态字面量 ID，再从仓库根目录运行： - -```sh -node web/scripts/analytics-manifest.mjs --write -node web/scripts/analytics-manifest.mjs -``` - -审核生成的 [manifest](frontend-analytics-manifest.json) 差异和原生交互测试。部署方再加一条规则即可。ID 必须匹配 `^[a-z]+(?:\.[a-z]+){2,}$`：至少三段小写字母，不含数字、下划线或资源信息。已有 ID 含义冻结，不能把用户、技能名、版本、路径拼进去。manifest 是版本清单，不是运行时封闭枚举。 - -## 4. 部署、子路径和 CSP - -edge 仅在 UI HTML 注入一个延迟执行的 module，不改变 API、登录、下载、静态资源或 runtime-config。server 级 `client_max_body_size 100M` 保留标准 Web 的 100 MiB 请求体上限；实际业务校验仍可能拒绝上传。不要在应用代理 location 覆盖成更小值。 - -正式启用前检查 Nginx 含 http_sub_module、`nginx -t` 成功，并审核 `nginx -T`。HTML 输入不能仍被上游 gzip／Brotli 压缩。示例禁用 HTML 缓存和旧校验器；启用或回退都应清理 CDN／浏览器旧 HTML。不要修改 API 或 immutable 资源的缓存规则。 - -根路径演示的 CSP 需要允许同源外部 module 和同源连接。保留原有 CSP；nonce-only 或 strict-dynamic 策略不能只靠同源满足。需要部署方协调每响应 nonce 或已批准的 hash／SRI；做不到就不要启用。不要加 unsafe-inline、unsafe-eval 或通配符。还要保留原安全头，注意 Nginx add_header 的 location 继承规则。 - -`/skillhub/` 等子路径不是只改一个 URL。先按正常部署方式配置 `SKILLHUB_WEB_BASE_PATH=/skillhub/` 和匹配的同源 API base，再同时调整 UI location、无尾斜线重定向、旁路正则、模块目录／注入 src、两个 Umami 精确端点和 adapter prefix。edge 到 Web 的 proxy_pass 不带尾斜线，保留原 URI，不能再次重写。逐项配置见[英文契约的子路径映射](frontend-analytics.md#supported-sub-path-mapping)。OAuth、公网 HTTPS 和资源地址仍遵守原部署要求。 - -## 5. 验收：照这个顺序检查 - -下面命令假定根路径和默认 edge 端口。子路径时为所有采集 URL 加相同前缀；自定义端口时统一替换地址。 - -1. 检查模块确实由 edge 提供，不是 SPA 返回的 HTML： - -```sh -curl -i http://localhost:8081/_skillhub-observe/v1/skillhub-intents.mjs -curl -i http://localhost:8081/_skillhub-observe/v1/missing.mjs -curl -I http://localhost:8081/skills -``` - -第一个应为 200、JavaScript MIME、nosniff 和 immutable。第二个应为非 HTML 的 404。第三个应为 200、Cache-Control: no-store，并保留原 CSP（若原部署配置了 CSP）。 - -2. 打开 edge 的 SkillHub 页面，在开发者工具 Network 勾选「Preserve log」和「Disable cache」，筛选 `/_skillhub-umami/`。刷新后应加载 script.js，但没有自动 pageview 或 send。 -3. 搜索框输入任意本地测试文字，点击搜索。应恰好有一条 `/_skillhub-umami/api/send` POST，响应 200。再按 Enter 搜索一次，应新增一条，不是两条。切换 Network 的 Payload 查看真正出站 JSON，不要只看回调。 -4. JSON 只能包含 `{ type: 'event', payload: { website, name, data: { event } } }`。website 是配置的 UUID；搜索 name 为 search.query.submit，event 为 submit。不得出现搜索词、完整 URL、标题、referrer、身份或额外 data 字段。 -5. 打开 Umami 网站事件页，选择当前时间范围，应看到 search.query.submit 和对应数量。平台处理可能稍有延迟；不能把 HTTP 200 单独当作入库证明。 -6. 执行下面的 Console 命令后再搜索，页面仍能使用，但 Network 不得新增 send： +当前页面挂载后提供接口，可接入已有同意组件；浏览器 Console 也可用于验收： ```js window.skillhubAnalytics.stop() +// 仅在重新获准后： +await window.skillhubAnalytics.start() ``` -stop 不能撤回已开始的 fetch。重新获准后才可调用 start()；同一个 root 重复初始化保留第一次规则和回调，修改规则需要先 stop 再初始化。 +start() 是当前页的显式授权，支持初始 consent=denied。重复 start 不重复安装。 +stop() 幂等；等待中的模块/adapter 不能迟到后重启采集。stop 返回后不新派发请求， +但已经开始的 fetch 无法撤回。规则变更使用 stop 再初始化；环境变量变化需重建或重启 Web。 +旧标签页不会自动收到新环境参数。修改 env 文件本身不会改变正在运行的容器。 -7. 将入口恢复 `enabled: false`，重新创建 edge。保持「Disable cache」勾选（或换一个新的无痕窗口），刷新页面后再搜索：不得加载 script.js、产生采集回调或 send。完全不注入时也应零采集。最后在隔离演示中停止 Umami，再打开新页面搜索和打开技能卡片，应仍正常，无统计依赖导致的页面错误： +## 6. 根路径、子路径与验收 + +根路径无需改路由。子路径只需保持现有三项一致： + +```dotenv +SKILLHUB_WEB_BASE_PATH=/skillhub/ +SKILLHUB_WEB_API_BASE_URL=/skillhub +SKILLHUB_PUBLIC_BASE_URL=https://skills.example.com/skillhub +``` + +浏览器采集路径自动变为 `/skillhub/_skillhub-analytics/`，正常入口只重写一次。 +`/_skillhub-analytics/` 是保留前缀，不能作为 Web base path。 +保留当前 CSP；需要允许已批准的同源外部脚本和连接。nonce-only/strict-dynamic 部署仍需 +按本身的安全策略授权，无法授权就不开启。不要增加 unsafe-inline、unsafe-eval 或通配符。 + +按顺序验收。下面用本机默认 Web 端口，实际环境替换 origin 和子路径： ```sh -docker compose -f docs/examples/analytics/compose.yaml stop umami +curl -i http://localhost/runtime-config.js +curl -i http://localhost/_skillhub-analytics/script.js +curl -i http://localhost/_skillhub-analytics/api/websites ``` -验收后如需继续演示，使用上面的 up 命令恢复。不要为了测试停止共享平台或生产服务。 +1. runtime-config 为 no-store；原 API/auth 键仍在；不得包含 upstream。 +2. 开启且 granted 时 script.js 应是 JS 200；未注册路径应为 404。 +3. 新开 SkillHub 页面，Network 保留日志。先看到应用挂载，再按需加载 controller、adapter + 和 script.js。不会自动发送 pageview。 +4. 搜索框输入本地测试文字，点击搜索，再按 Enter 搜索。每次应各有一条 api/send，页面正常跳转。 +5. 检查真正出站 JSON，只允许 `{type:'event',payload:{website,name,data:{event}}}`。 + 搜索 name 为 search.query.submit，event 为 submit。不能有查询文字、URL、标题、身份或额外 data。 +6. 在 Umami 网站事件页核对事件和数量。HTTP 200 不能单独证明入库。 +7. 调用 stop，再搜索。业务继续，不能新增 send。重新 start 后允许继续。 +8. 关闭配置或 denied 后重建 Web，打开新页面：不得加载 controller/adapter、script.js 或 send。 + denied 页面只有显式 start 才会加载。 +9. 在隔离测试环境停止 Umami，或用不可解析的内部域名启动 Web。Web 仍应就绪，搜索、 + 登录、卡片、下载继续遵守原业务行为。不要停止共享或生产平台来测试。 +10. 根/子路径都测试正常上传（大于 1 MiB 且业务有效）、登录与下载。统计不代表业务成功。 -## 6. 故障排查 +## 7. 排查、升级和关闭 -| 现象 | 常见原因 | 处理 | -| --- | --- | --- | -| module 404 | 未挂载模块，版本目录或子路径不一致 | 对照 nginx -T、挂载路径、注入 src 和相对 import，三份 mjs 都要可访问；不要回退到 index.html | -| MIME 报错或返回 HTML | 模块误入 SPA fallback，缺少 mjs MIME | 使用示例独立前缀、try_files 404 和 JavaScript MIME，保留 nosniff | -| CSP 拦截 | script-src／connect-src 不允许，或 nonce-only 策略 | 按原 CSP 的授权机制交付；不要删除 CSP 或放宽为通配符 | -| 上传 413 | 外层代理仍默认 1 MiB，或请求超过 100 MiB | 用 nginx -T 检查整条代理链；应用代理继承 100M，超过标准上限仍应拒绝 | -| 没有事件 | 走了原 Web 而非 edge，enabled／授权未开，UUID 错，规则未配置或宿主不支持 | 按第 5 节从模块、skipped、Network 到平台事件页逐步确认；检查时间范围；不要把结构标签当 pageview | -| edge 502 | Web 上游地址／端口错误，宿主不可达 | 核对 WEB_PORT、host-gateway、防火墙和监听地址；不要把后端或 edge 端口填成 Web 上游 | -| Umami 不可用 | SDK 404、超时、平台停止或网络故障 | 页面应继续正常；检查平台和代理日志，恢复后重新打开页面；没有重试补发保证 | +| 现象 | 检查 | +| --- | --- | +| 没有模块/事件 | 总开关、授权、容器是否重建、是否新页面；默认关闭是正常情况。 | +| 配置报错 | 看 `[analytics]` 字段诊断；JSON 用双引号，enabled 是布尔值，UUID 正确；不要发出原值。 | +| script.js 502 | Web 到平台的 DNS、网络、端口或证书；origin 不带路径；业务应仍可用。 | +| SDK 失败后无事件 | 加载失败/超时后不重试；恢复平台后新开页面，或 stop 再 start。 | +| 模块/资源返回 HTML | 检查部署版本、base path 和入口路由，不能改成任意远程脚本。 | +| CSP 拦截 | 按原策略授权，不删除 CSP 或放宽通配符。 | +| 有点击但没有统计 | 规则未配置、宿主不支持、超过 Umami 名称 50 字符，或 best-effort 丢失。 | +| 上传 413 | 检查整条外层代理链，正常 Web 仍是 100M，统计未降低它。 | -## 7. 升级、关闭和已知限制 +升级先比较[manifest](frontend-analytics-manifest.json)，已有 ID 含义冻结，新 ID 不自动加入默认规则。 +消失的配置 ID 不再触发。SDK 固定 Umami 3.4.1；升级需重测真实 payload、stop、并发和 +根/子路径业务。回滚 Web 镜像要同时使用该版本支持的配置；先关闭统计最稳妥。 -升级时先比较新旧 manifest。已有 ID 语义不变；含义变化需要新 ID，重命名／删除必须有发布说明。消失的 ID 不再触发，新 ID 默认不自动加入规则。模块／SDK 升级要重新验证真实出站字段、stop 派发边界、并发发送、根／子路径和业务行为，不能只看构建成功。 +持久关闭：把配置留空或 enabled 改 false，按第 2 节重新创建 Web。新页面应零模块/SDK/请求。 +老标签页不自动更新；需让用户关闭/刷新，或在现有同意组件调用 stop。 +紧急全局阻断可在平台或独立采集路由禁用接收，再处理旧页；不能撤回已发出的请求。 -临时关闭当前页用 stop()；持久关闭把入口改回 enabled:false，并按版本发布规则更新、清理 HTML 缓存和刷新。彻底关闭则撤掉注入／edge，让访问回到正常 Web 入口，确认新 HTML 不引用模块、Network 无采集。关闭平台网站或封锁采集路由是额外控制，不能代替授权撤回。 - -结束本地演示（会删除演示数据库，先确认无需要保留的数据）： +自托管只停止统计服务，不要对整个业务栈执行 down： ```sh -docker compose -f docs/examples/analytics/compose.yaml down -v +docker compose --env-file .env.release -f compose.release.yml -f compose.analytics-umami.yml stop umami analytics-postgres ``` -限制必须一起评估： +数据库卷保留。删除统计数据需另行备份、确认后只处理 `analytics_postgres_data` 对应的卷， +不要使用整个项目的 `down -v`。 -- 尽力发送，无重试、无持久队列，可能少计。SDK 加载期最多在内存暂存 20 条，默认最多等三秒；超额、失败、超时或 stop 都丢弃。已就绪发送彼此独立，不因一个请求挂起而阻塞后续动作。 -- 示例固定 Umami 3.4.1。stop 的派发保护依赖此版本在 fetch 前同步序列化的顺序，升级必须重跑适配器测试和真实浏览器验收；不要同时加载第二个 Umami tracker。 -- JSON 字段少不等于匿名传输。IP、User-Agent、Origin 仍到平台；SDK 还带网站、实际 hostname 和后续会话 cache 头。edge 去掉采集代理的 Cookie、Authorization、Referer，并使用 no-referrer。Umami 仍可能据传输元数据推导浏览器、系统、设备、地域和会话。 -- SDK 在清洗前仍会在页面内读取标题、URL、referrer 和自身 localStorage 开关；通用 helper 不读取这些内容。平台日志、数据保留和权限需另审计，详见[隐私契约](frontend-analytics.md#privacy-and-integration-modes)。 +## 8. 隐私和自定义开发边界 + +JSON 只保留静态 ID 和原生 event。helper 不读内容、输入值、URL、Cookie、存储或身份。 +SDK 在清洗前仍会读取页面标题、URL、referrer 和自己的 localStorage 开关。 +少字段不等于匿名：IP、User-Agent、Origin 仍到平台；Umami 还有 website、实际 hostname、 +会话 cache 请求头。Nginx 在统计上游前移除 Cookie、Authorization、Referer，并隐藏 Set-Cookie。 +浏览器到 Web 本身仍可能带这些 HTTP 元数据，需按原 Web 日志和权限策略管理。 +Umami 仍可能由传输元数据推导浏览器、系统、设备、地域和会话。数据库缺失 URL 的回退 +是空 path/query 和 localhost，不是用户当前路径。访问日志、保留和权限要单独审核。 + +尽力发送，无重试或持久队列。SDK 加载期最多内存暂存 20 条，默认等三秒；超额、失败、 +超时或 stop 会丢弃。就绪后的请求彼此独立。不要同时加载第二个 Umami tracker。 +stop 的派发保护依赖固定版本在 fetch 前同步序列化的顺序，升级必须重新验证。 + +自定义 adapter 属于开发流程，不是部署流程：在 `web/src/features/analytics/adapters/` +增加参数校验与 send/stop factory，注册到静态 registry，再增加部署端校验和精确代理端点。 +factory 检查取消信号后才创建 SDK。通用 controller 不认识厂商字段，业务组件只提供被动标签。 +复用唯一的 `skillhub-intents.mjs`，不复制 helper，也不开放远程 JS URL。 +新增标签时运行 `node web/scripts/analytics-manifest.mjs --write` 并审核语义和原生交互。 +测试命令为 `pnpm -C web test src/features/analytics` 和 `python3 scripts/tests/web-analytics-test.py`。 diff --git a/docs/frontend-analytics.md b/docs/frontend-analytics.md index 621b6d0d6..d7cba572f 100644 --- a/docs/frontend-analytics.md +++ b/docs/frontend-analytics.md @@ -1,392 +1,178 @@ -# Passive frontend analytics attributes +# Passive frontend analytics contract -For step-by-step setup, see the [Chinese integration guide](frontend-analytics-integration.md). -This document is the English contract reference. Configuration-only deployment is -planned in [PR-2](../openspec/changes/add-configurable-frontend-analytics/proposal.md); -the current examples remain deployment-owned and opt-in. - -SkillHub exposes semantic HTML attributes for separately maintained, optional -collectors. It does not collect or send analytics by default. There is no SDK, -vendor dependency, vendor token, network reporter, app script loader or -telemetry configuration switch. The application entrypoint and production Nginx -configuration are unchanged. Deployment owners control consent, script delivery, -retention, transport and compliance; this is not an official vendor integration -or a durable business-event feed. +For deployment, use the [Chinese configuration guide](frontend-analytics-integration.md). +Docker/Compose and Helm ship optional analytics with Web. No configuration means no +controller/adapter chunk, SDK or collection request. The small configuration gate is +part of bootstrap; the controller and SDK are not. Business components remain passive. ## Contract and scope -`data-analytics-id` names an action category (what kind of action), not a unique -event or instance ID, and describes intent, not its result. Many elements and -many events share the same category. Lowercase dotted names are independent of -language, visible text and layout. +`data-analytics-id` names an action category, not a unique event, resource or user. +It describes intent, not its result. IDs match `^[a-z]+(?:\.[a-z]+){2,}$`. +Lowercase dotted names are independent of language, visible text and layout. +Published meanings are frozen. A meaning change requires a new ID. Renames/removals +need release notes and a reviewed [manifest](frontend-analytics-manifest.json) diff. +The manifest is an AST source inventory, not a runtime closed enum or a count of +users, pageviews, successful operations or exhaustive business coverage. -The example callback payload is `{ id, event }`: `id` is the matched category; -`event` is the native DOM event type observed (`click`, `submit`, or `keydown`), -not a business result. Evolution within this module version is additive only: -new optional fields must be allowlisted static values, consumers ignore unknown -keys, and existing field meanings never change. Breaking changes use a new -versioned module path. +The inventory includes landing/search, skills, suites, publishing, review, +governance, namespaces, GitHub integration, notifications, preferences, account, +authentication, administration and navigation. Literal branches distinguish finite +actions. Never interpolate resource, slug, version, user, provider, role or backend +values into IDs. Fixed navigation metadata is source-owned. Forwarding sites do +not necessarily represent DOM elements; portals and repeated resources share IDs. -Published IDs are frozen. A meaning change needs a new ID; rename/removal must be -called out in the PR and release notes and visible in the generated manifest diff -(the CI drift test checks source against that manifest). Deployers should diff -the manifest between versions: configured IDs that disappear simply never fire; -new IDs remain inert until explicitly configured. The manifest is not a runtime -closed enum. -Literal conditional branches distinguish finite business actions (for example, -namespace freeze/unfreeze/archive/restore/delete and each confirmation/cancel). -No resource, slug, version, user, provider, role, label instance or backend string -is interpolated into attributes. Fixed navigation metadata is source-owned. -`data-analytics-event`, where present, describes a native observation boundary; -it is a hint for a deployment collector, not an application event dispatcher. - -The source inventory covers these domains: - -- Landing/catalog/search: queries, sorting, filters, results, pagination and links. -- Skills: detail structure, tabs/versions/comparison, documentation and files, - acquisition/copy, social actions, labels, visibility and lifecycle confirmations. -- Publishing: package/folder selection, preview/retry, submission/warnings and - avatar editing, staging, retries and cancellation. -- Suites: catalog/detail, members/versions, editor/import, publishing tasks, - lifecycle and destructive actions. -- Reviews, promotions, reports and governance: queues, detail inspection, - dispositions, layered confirmations, rejection editing, cancellation and retry. -- Namespaces: management, membership/import, invitations, ownership and lifecycle. -- GitHub: collection/settings, discovery, credentials, scheduling/admission, - repositories/sources/runs, conflicts and review inspection. -- Notifications/preferences, account/profile/security/token flows, login, - registration/reset/device/CLI authorization, administration and global navigation. - -[The generated manifest](frontend-analytics-manifest.json) is the authoritative -source index: registered routes and their implementations, literal JSX sites, -conditional values, static navigation definitions and forwarding sites. Counts -are recomputed from the TypeScript AST; they are not quotas. Unique IDs are -semantic categories, JSX sites are source declarations, and forwarding sites -may have no literal ID until a caller opts in. A wrapper prop is not itself a -DOM element. Repeated resources/pages, desktop/mobile navigation and portal -instances can share an ID; neither IDs nor source sites count runtime elements, -unique users, pageviews, successful operations or exhaustive business coverage. - -Development checks (from the repository root with web dependencies installed): +Run from the repository root with dependencies already installed: ```sh node web/scripts/analytics-manifest.mjs -# After reviewing intentional source changes: +# Only after reviewing intentional source changes: node web/scripts/analytics-manifest.mjs --write -pnpm -C web test src/app/analytics-manifest.test.ts +pnpm -C web test src/app/analytics-manifest.test.ts src/features/analytics ``` -Do not ship the manifest as a runtime closed enum. A deployment collector should -have its own small, explicit event/ID allowlist and update it deliberately. - ### Original eight compatibility meanings | ID | Meaning | | --- | --- | -| `search.query.submit` | Search form submit; button and Enter converge here. | -| `skill.card.open` | Existing card activation; click or its direct Enter/Space handler. | -| `skill.detail.render` | Structural detail root after loading/error/missing/deleted guards; not a unique pageview or success. | -| `skill.download.request` | Package download attempt; authentication/guards may reject it. | -| `skill.command.copy` | Command-copy attempt for the displayed installation method; not clipboard or installation success. | -| `skill.prompt.copy` | Agent-prompt-copy attempt; not execution or installation. | -| `skill.publish.request` | Publish/update/resubmit attempt, never avatar-only save. | -| `skill.publish.confirm` | Explicit publish warning continuation, never all generic dialogs or avatar-only save. | +| search.query.submit | Native search form submission, not typing or search success. | +| skill.card.open | Existing card click or direct Enter/Space activation. | +| skill.detail.render | Structural loaded detail root, not an automatic pageview. | +| skill.download.request | Download attempt; guards may reject it. | +| skill.command.copy | Command-copy attempt, not clipboard or installation success. | +| skill.prompt.copy | Prompt-copy attempt, not execution or installation success. | +| skill.publish.request | Publish/update/resubmit attempt, not avatar-only save. | +| skill.publish.confirm | Explicit publish warning continuation, not every dialog. | -Installation method tabs distinguish SkillHub and ClawHub; the original command -copy ID remains compatible. Public/namespace/private choices and business modes -use static categories, not data fetched from the selected object. +Installation-method tabs distinguish SkillHub and ClawHub without changing the +original command-copy meaning. Visibility choices and business modes use static +categories, not data fetched from the selected object. ## Native observation policy and limits -Attributes alone never create events. Choose the native event per host, and -count one semantic target per event rather than every marked ancestor: +Attributes alone do not create events. Keep one nearest semantic target per event. +Never change defaults, propagation, focus, history, permissions or app handlers. -- Forms: observe `submit`, not both submit-button click and submit. Enter and - native validation follow existing app behavior. Rejected validation may produce - no submit. Search typing/clear is not a search submission. -- Buttons/links: observe click, including browser-generated keyboard click. Ignore - disabled, aria-disabled and data-disabled controls. An enabled click can still - fail app guards; busy/authorization semantics remain the application's. -- Custom DIV cards with existing Enter/Space handlers need that keydown in addition - to click. Native buttons must not count both. Tabs generate click from navigation - keys; do not count Arrow/Home/End keydown again. Do not invent keyboard handlers - for existing mouse-only rows or directories. -- Native select: observe change. Where the static marker is on an option, read only - `selectedOptions[0].getAttribute('data-analytics-id')`, never option value/text. - A select marker describes its context, not the raw choice. Avoid click/change - double counting. Checkbox/radio change is an intent, not saved server state. -- Radix Select items are actual option DOM, sometimes in a portal. Pointerup or - Enter/Space on an item is selection intent, not proof that onValueChange committed - a new value. Opening the trigger and navigation/highlight are not committed - selections. A click-only collector is insufficient. Validate event order with - the deployed component/browser version rather than manufacturing app hooks. - Trigger `*.interact` markers cover observable activation/keydown intent, including - closed-trigger typeahead; never capture the typed key, infer the chosen value, - or call that interaction a committed selection. Typeahead can change selection - without opening an item. Keep trigger interaction and item selection-intent - metrics separate instead of counting both as the same selection. -- File inputs: change means a selection was offered, not upload success. Drop is - offered files, not accepted/validated files. Picker-opening click differs from - change; ignore programmatic input clicks as additional user intent. Never read - filenames, paths or file metadata. Picker cancellation is not observed. -- Crop surface pointer/keyboard interaction is adjustment intent, not proof the - crop changed or an image upload succeeded; no coordinates or image data may leave. -- Explicit cancel/close buttons, including X, preserve workflow-specific meaning. - Escape, overlay dismissal, route teardown, browser Back, automatic close and - native picker cancel are not reliably observable through these static attrs. - Closing an editor is not proof an in-flight request was aborted. +- FORM uses submit, not submit-button click plus submit. Enter converges here; + rejected native validation may produce no submit. Typing/clear is not submission. +- Ordinary buttons/links use click, including browser-generated keyboard click. + Ignore disabled, aria-disabled and data-disabled controls. App guards still apply. +- DIV role=link cards with existing Enter/Space handlers also use keydown. Do not + synthesize activation. Ignore repeat, IME and modifier keydown. Native buttons + must not count keydown and their generated click twice. +- Tabs generate click from navigation keys; do not count Arrow/Home/End again. + Do not invent keyboard support for existing mouse-only rows/directories. +- Native selects, checkboxes and radio changes have different boundaries. An + option marker can describe static selection intent, never its value/text. Do + not infer committed selection from opening a trigger or highlighting an item. +- Radix selection may use pointerup or Enter/Space in a portal. Trigger typeahead + can change selection without opening an item. Trigger interaction is not proof + of committed selection; never capture typed keys. Current finite rules do not + implement select/file/drop/crop; do not misconfigure them as click. +- File selection and dropping only offer files, not upload success. Do not read + filename/path/metadata. Programmatic picker clicks are not extra user actions; + cancellation is not observed. Crop interaction is not proof of changed pixels. +- Explicit cancel/close retains workflow meaning. Escape, overlay, route teardown, + browser Back, automatic close and native picker cancel are not reliably observed + by these markers. Closing is not proof a pending request was aborted. -Use document-level delegation for portals; React ancestry is not DOM ancestry. -Wrapper forwarding is caller-scoped, including confirmation/cancel and close X; -generic UI primitives do not guess business IDs. Keep the nearest semantic target -and reject nested unmarked interactive controls rather than attributing them to -an ancestor card/form. Synthetic clicks (including temporary download anchors) -are not new successful user actions. Do not block or change default behavior, -propagation, focus, route history, permissions or app handlers. +Use document delegation for portals; React ancestry is not DOM ancestry. Wrappers +forward caller-scoped IDs, not guessed business IDs. Reject nested unmarked +interactive controls rather than attributing them to an ancestor card/form. +Synthetic temporary download anchors do not prove new successful user actions. +SPA classification, consent transitions and view deduplication are separate +policies; never derive them from full URLs/query strings. -`skill.detail.render` is not an automatic pageview event. SPA route classification, -render retries, consent transitions, duplicate initialization and view deduplication -are separate collector policies. Never derive these from full URLs/query strings. +## Single product source and interfaces -### Privacy and integration modes +The sole helper is `web/src/features/analytics/skillhub-intents.mjs`: -The recommended output allowlist is only `{ id, event }`, each validated against -operator-owned static categories. Do not collect textContent/innerHTML, input or -option values, clipboard contents, href/full URLs/query/fragment, cookies, storage, -identity/account IDs, device codes, tokens, invitation links, private coordinates, -filenames/paths, review/report reasons or crop data. Do not scrape arbitrary DOM -attributes or reuse test IDs (some intentionally contain resource identifiers). +- `observeSkillhubIntents({ root, rules, onIntent }) -> stop`. +- Frozen `DEFAULT_RULES` has the original twelve rules listed in the guide. +- `resolveRules(rules) -> { active, skipped: [{ id, reason }] }` is pure. Invalid + containers produce id:null and no active rules. Unknown IDs are not rejected + merely for being absent from the manifest. Invalid grammar/types are skipped. +- Omitted rules use defaults. Explicit rules replace defaults; {} collects nothing. +- click excludes FORM/SELECT/TEXTAREA/non-button INPUT; submit is FORM-only; card + is DIV role=link only. Host validity is checked at observation, not resolution. +- Intent is exactly `{ id, event }`; event is click/submit/keydown. Same-version + evolution is additive optional allowlisted static fields; consumers ignore + unknown keys. Breaking changes need a new module contract. +- Importing does nothing; without callback no listeners are installed. Same root + returns the same stop and retains the FIRST rules/callback. Stop then initialize + to change configuration. Old disposers cannot remove a new installation. Stop + is idempotent; nested observers share event deduplication. +- Callback throws/rejections are swallowed without affecting business events. + The helper itself has no network, cookies, storage, identity or history hooks. -Disabled integration means no injected script and zero analytics collection. -Enabled delivery does not itself authorize collection: initialize only after -applicable consent/policy approval, stop on withdrawal, and separately review any -outbound adapter and CSP connect destinations. External JS has page privileges; -serve reviewed versioned code, restrict access and treat compromise as a security -incident. Vendor credentials and ingestion endpoints belong outside core. +The controller owns authorization, rule resolution, generations and lifecycle. +Static registry metadata validates public adapterConfig; factories lazily import +adapters. Factory cancellation prevents late imports from creating an SDK after +stop. Adapter exposes `send(intent) -> Promise` (never rejects) and stop(). +The loader runs separately after the first React DOM commit, never blocking mount. +Disabled, absent, invalid or denied configuration does not import the controller. +Public `window.skillhubAnalytics.start()` explicitly grants current-page consent; +stop() immediately disables observation. Enabled=false cannot be overridden by +start. Deployment changes require recreating/restarting Web; old tabs do not update. -## Configure rules and add tags +## Configuration and proxy -[The vendor-neutral helper](examples/analytics/observe/v1/skillhub-intents.mjs) -exports `observeSkillhubIntents({ root, rules, onIntent })`, frozen `DEFAULT_RULES` -(the original 12 categories), and pure `resolveRules(rules)`: +`SKILLHUB_WEB_ANALYTICS_CONFIG` is JSON with enabled (boolean/default false), +adapter (static registry name), consent (denied/granted/default denied), rules +(optional finite rule map), adapterConfig (adapter-owned public object). Umami +accepts only websiteId UUID. Invalid configuration disables only analytics with +field-only diagnostics, never raw values. Rule resolution keeps its skip behavior. +The startup parser limits JSON to 64 KiB/32 levels and rejects duplicate keys. -```js -import { DEFAULT_RULES, resolveRules, observeSkillhubIntents } from './skillhub-intents.mjs' -const rules = { ...DEFAULT_RULES, 'skill.example.open': 'click' } -const { active, skipped } = resolveRules(rules) // Inspect locally; do not send config diagnostics. -const stop = observeSkillhubIntents({ root: document, rules: active, onIntent: sendIntent }) -``` +`SKILLHUB_WEB_ANALYTICS_UPSTREAM` is a deploy-only HTTP(S) origin: no credentials, +path, query, fragment, whitespace or injection characters. It is never put in +browser configuration. Existing BusyBox tools validate deployment control fields; +base64 transports JSON safely as data into the existing no-store runtime-config.js. +No jq, Node runtime, extra package, CSP relaxation or HTML injection is required. -Omitting `rules` uses the defaults; passing it REPLACES them. Merge explicitly as -above. The example accepts the same grammar the product manifest enforces: -`^[a-z]+(?:\.[a-z]+){2,}$` (at least three dot-separated segments, lowercase letters -only; no digits or underscores). For example, `skill.command.skillhub.select` -can reuse `click`. Only finite rule types are supported: +The reserved same-origin prefix is `/_skillhub-analytics/`. BASE_PATH is applied in +the browser; existing Nginx rewrite strips it once. Enabled configurations generate +exact script.js and api/send routes, with method restrictions; all other paths are +404, never Umami admin. Consent-denied enabled configurations retain those routes +for explicit dynamic start, but do not load/request them automatically. +Variable proxy_pass uses the existing resolver at request time. HTTPS verifies +certificates using the system trust bundle and correct SNI. Collector proxying +strips Cookie/Authorization/Referer and hides Set-Cookie. API/auth routes, 100M +uploads, TLS, business caching and permissions are not changed. -- `click`: ordinary buttons/links. FORM, SELECT, TEXTAREA and non-button INPUT - hosts are ignored (including file inputs). -- `submit`: FORM only; button click and Enter converge on native submit. -- `card`: DIV role=link with its existing Enter/Space handler, plus click. The - observer does not implement keyboard activation or synthesize clicks. +## Privacy and integration modes -Select/file/drop/crop and other special interactions need a future rule type; -never configure them as `click`. Unknown/unconfigured actions, invalid IDs, -non-string rules and unsupported rule types/hosts are skipped without affecting -the page. `resolveRules(...).skipped` gives `{ id, reason }` for invalid entries; -an invalid rules container yields an `id: null` diagnostic and an empty active map. -It validates configuration, not the live DOM host. +Allowlist only static id/event, not text/HTML, input/option values, clipboard, +URLs/query/fragment, cookies, storage, identities, codes/tokens, private paths, +filenames, review reasons or crop data. Test IDs can contain resource identifiers; +do not scrape them. The manifest is not a payload. -Importing the helper does nothing; without a callback it installs no listeners. -The same root returns the existing stop function and keeps the FIRST config and -callback. To change config, call `stop()` and initialize again. Nested observers -from this module instance share event deduplication; only the nearest marked host -is considered, never an ancestor behind an unmarked nested interactive control. -Native-button keydown, repeated/IME/modifier card keydown are ignored. Sync throws -and returned rejected promises from the callback are swallowed, with no retry or -queue. No network, cookies, storage, identity or route patching occurs in the -helper, and it never prevents defaults or stops propagation. +Umami 3.4.1 uses manual-only script attributes and a final before-send sanitizer. +Only `{type:'event',payload:{website,name,data:{event}}}` survives; website must +match configuration, name must match product grammar and Umami's 50-character +limit. Pageview/identify/performance/context/extra data are dropped. Payload values +are read once. A non-enumerable toJSON guard rechecks stop synchronously at fetch +dispatch; upgrading the SDK must reverify its ordering with dispatch-boundary tests. +Stop cannot recall a fetch already started. -To add a normal action: product owners add a literal `data-analytics-id` on the -appropriate existing host, run `node web/scripts/analytics-manifest.mjs --write`, -review the manifest diff and test the host/native behavior. Deployers then add one -rule line. No product listener or SDK is needed. Download click is NOT download or -installation success; publish submission is NOT publish success. +At most 20 valid intents wait in memory for up to three seconds while the SDK +loads. Excess, failure, timeout or stop drops them. Once ready, requests are +independent and time-bounded, with no retry/durable queue/post-load backlog. +Do not load a second Umami tracker. Missing/blocked/offline collectors never reject +into the application. SDK loading is trusted code with page privileges: review and +pin its version, access and retention; it is not an arbitrary remote-JS plugin. -The repository-integration test lives at `web/analytics-example.test.ts`, alongside -the existing root-level web tests. Vitest discovers it with `pnpm -C web test`; -run it alone with `pnpm -C web test analytics-example.test.ts`. It stays outside -the production TypeScript project's `src`/`test` inputs because the docs-only -example is not present in the `web/` Docker build context. +This is NOT anonymous transport. IP/User-Agent/Origin still reach the platform. +The SDK sends website, actual hostname and session cache headers, reads +URL/title/referrer before sanitization, and reads its own localStorage switch. +Nginx strips sensitive headers before the collector, not before the browser's +request reaches Web. Web logs require their own privacy policy. Umami can derive +browser/OS/device/geo/session from transport metadata. With pinned 3.4.1 absent URL +stores empty path/query and fallback hostname localhost; title/referrer/screen/ +language/distinct identity remain absent. Removing JSON fields does not disable +transport-derived processing. Operators must review consent, logs, access and law. -## Connect a platform: local Umami v3.4.1 demo - -[The Compose example](examples/analytics/compose.yaml) runs Umami v3.4.1 (MIT), -Postgres 16 and an optional edge; SkillHub remains your existing web upstream. -From the repository root: - -```sh -docker compose -f docs/examples/analytics/compose.yaml config -docker compose -f docs/examples/analytics/compose.yaml up -d -``` - -The default Web upstream is `host.docker.internal:80`, matching the release -Compose `WEB_PORT` default of 80. Override `SKILLHUB_WEB_UPSTREAM` with the actual -reachable host:port if different. The edge's `extra_hosts` maps this hostname to -`host-gateway` on native Linux; Docker Desktop/OrbStack already provide it. The -host Web listener must be reachable from containers. The edge inherits the standard -server-level `client_max_body_size 100M` for application proxying (100 MiB); it does -not reduce uploads to Nginx's default 1 MiB. - -This creates a demo network and ephemeral database, not production storage. Use -an isolated local Docker environment with available network pools. Open -`http://localhost:8300` for local setup only. Umami's default `admin` / `umami` -credentials, the public DB password and app secret are DEMO ONLY: change them. -Create a synthetic website in Umami, copy its UUID into -[`observer.mjs`](examples/analytics/observe/v1/observer.mjs), then explicitly change -`enabled` and the consent hook for your local policy. Open SkillHub through -`http://localhost:8081` and exercise an enabled action; inspect the request to -`/_skillhub-umami/api/send` and Umami's event view. Tear down the local demo with -`docker compose -f docs/examples/analytics/compose.yaml down -v` (demo data is removed). - -[`umami-adapter.mjs`](examples/analytics/observe/v1/umami-adapter.mjs) is the only -vendor-specific code. It loads `/script.js` with `data-auto-track="false"`, so -v3.4.1 installs no pageview, history hook, click listener or performance observer. -`data-host-url` is the same-origin `/_skillhub-umami` prefix, and -`data-before-send` names its sanitizer. Do not load another Umami tracker on the -same page (the SDK/global hook are singletons). It calls `umami.track(id, { event })`; -at most 20 valid intents wait in memory for up to three seconds while the script -loads (excess intents are dropped). They are released when ready, or dropped on -load failure, timeout or `stop()`. Once ready, every valid intent calls the tracker -independently: concurrent or hung requests do not block later intents. Each call's -wait is time-bounded; missing, blocked, throwing or rejecting trackers resolve -silently. There is no retry, durable queue or post-load backlog; best-effort counts -can undercount. After `stop()` returns, no new request is dispatched; only a -request whose fetch already started cannot be recalled (nor cancelled by a timeout). -The before-send hook keeps its immediate stop check and adds a non-enumerable -`toJSON` guard that re-checks consent when the SDK serializes at dispatch. In pinned -v3.4.1 this runs synchronously in the fetch expression inside the SDK's try/catch: -throwing on stop prevents fetch and is swallowed by the SDK. This relies on that -ordering; a different Umami version must re-verify it with the dispatch-boundary -regression tests in `web/analytics-umami-example.test.ts`. - -The sanitizer allowlists the FINAL SDK payload, not just the callback. Only -`{ type: 'event', payload: { website, name, data: { event } } }` survives; website -must match configuration, name must match the ID shape (and Umami's 50-character -limit), event must be click/submit/keydown. It returns null for pageviews (including -unnamed `event` payloads), identify, performance and everything else. Hostname, URL, -title, referrer, screen, language, tag, identity and extra data keys are removed. -Values are read once, and only validated primitives are forwarded. -The SDK still reads title/URL/referrer in-page before sanitizing; it also reads -localStorage for its own `umami.disabled` switch. Those are SDK behaviors, not -behaviors of the vendor-neutral helper. - -This is NOT anonymous transport or zero server processing: IP, User-Agent and -Origin still leave the browser. Umami sends `x-umami-website-id`, -`x-umami-hostname` (the actual hostname) and, after a response, `x-umami-cache` -(a session cache token). They are NOT sanitized JSON. The edge tightens UI -Referrer-Policy to `no-referrer` and strips Referer, Cookie and Authorization on -collector proxying; otherwise same-origin Referer can expose paths/queries. -Umami v3.4.1 hashes website/IP/User-Agent/salt into a server-side session and derives -browser/OS/device/geo from transport metadata. Removing JSON screen/language does -not disable these derivations; review access logs, retention, consent and local -law. Its missing-URL fallback stores empty url_path and fallback hostname -`localhost`, not the current page; url_query is an empty string (not SQL NULL in -v3.4.1), while title/referrer/screen/language stay null. - -To replace the platform, replace the adapter and edge ingestion endpoints, keeping -the helper and static payload contract. No admin configuration page, SkillHub -analytics DB, persistent queue or multi-vendor SDK is introduced. - -## Turn collection off - -No injection means zero collection. `enabled: false` in the deployer's entry is -also inert: no helper initialization or tracker load. On consent withdrawal call -`window.skillhubAnalytics.stop()`; it removes listeners and denies SDK payloads, -including those awaiting dispatch. After it returns, no new request is dispatched; -only a request whose fetch already started cannot be recalled. The dispatch guard -relies on pinned v3.4.1 serializing at dispatch; re-run the adapter regression tests -to re-verify this ordering for a different Umami version. Resume only after approval -via `start()`. Platform-side disabling/deleting the website, Umami's local -`umami.disabled` switch, or blocking the two ingestion routes are additional -operator controls, not substitutes for consent. Removing injection and purging -HTML is the deployment rollback; default SkillHub never loads any of this. - -## Optional deployment-owned Nginx injection - -Prefer a separately maintained same-origin HTTPS module, optionally inserted by -Nginx `sub_filter`, not an app entrypoint modification. Start with -[the opt-in edge template](examples/analytics/nginx.conf). It is not loaded by the -product. Compose substitutes only the two upstream host:port variables. Publish -its HTTP listener on loopback behind your HTTPS terminator. Serve the supplied -`observer.mjs`, helper and adapter outside the app bundle; only enable the entry -after configuring website/rules/consent. - -The sample filters only the UI fallback's default `text/html` type, once per -response, inserting a module before ``. Module loading is deferred and no -application code depends on it; missing/failed JS must not stop SkillHub. The -versioned prefix location overrides SPA fallback, sets JavaScript MIME and nosniff, -and uses `try_files $uri =404` with directory listing off. The served directory -contains only the three JS modules, never `.d.mts` or config. Missing modules are -404, never index.html. Code is immutable: deploy changed bytes/config under a new -version directory and update the injection path (breaking contracts need a new -module version). The two exact Umami routes expose tracker and send only; all -other paths under that prefix return 404. Same-origin script-src/connect-src are -sufficient for this demo, subject to the existing policy below. - -Deployment checklist: - -1. Check `nginx -V` for http_sub_module (built with `--with-http_sub_module` or an - equivalent loaded module), then `nginx -t`. Module absence is a blocker, not an - invitation to weaken security. Review actual final config with `nginx -T`. -2. Upstream `Accept-Encoding ""` requests uncompressed HTML. Ensure upstream - gzip_static/Brotli/CDN compression cannot still send encoded bytes. Filtering - compressed input does not work; downstream compression may run after filtering. - Leave API, auth, download, asset and runtime-config locations unmodified. -3. Filtered HTML must not reuse unfiltered validators or cached 304 responses. The - example removes upstream ETag/Last-Modified/Content-MD5 and cache headers, - suppresses conditional requests and disables proxy cache, serving no-store HTML. - Purge old edge/CDN HTML on enable/rollback; test GET/HEAD, conditional headers, - existing browser caches and 200/304 behavior. Do not cache injected HTML under - the old unfiltered entity validator. API and immutable assets retain their policy. -4. Preserve existing CSP and all security headers. The script must satisfy the - current script-src policy (same-origin alone does not satisfy nonce-only or - strict-dynamic policies). A per-response nonce or approved external hash/SRI - policy needs deployment-specific coordination; otherwise do not enable injection. - Never add unsafe-inline/unsafe-eval or broad wildcards. Any reporter needs an - explicitly permitted connect-src endpoint; this example uses same-origin only. - Nginx add_header inheritance is location-sensitive: merge/repeat existing edge - security headers as necessary; do not drop or hide upstream CSP to make it work. -5. Test actual built SkillHub HTML, not just a synthetic HTML fixture: exactly one - script for UI routes, none for APIs/assets/downloads/auth endpoints; script MIME, - missing-script 404, root/deep links, compression, CSP, disabled delivery and - collector failure. Then browser-check native events, portals, guards and consent. - A fixture filter probe is not final-config, SkillHub or production acceptance. - -### Supported sub-path mapping - -The sample is root deployment. For a supported `/skillhub/` deployment, keep that -prefix on the edge-to-app proxy URI (proxy_pass without a trailing slash), and -configure the app normally with SKILLHUB_WEB_BASE_PATH=/skillhub/ and matching -same-origin API base. Its existing container entrypoint owns internal prefix -rewriting; do not rewrite it a second time at the edge. - -Adapt the sample's UI `location /` to `location /skillhub/`; add an exact -`/skillhub` redirect to `/skillhub/`; scope the bypass regex to -`^/skillhub/(api/|oauth2/|login/oauth2/|\.well-known/|assets/|registry/|runtime-config\.js$|nginx-health)`. -Change the versioned module prefix location and injected src to -`/skillhub/_skillhub-observe/v1/` and -`/skillhub/_skillhub-observe/v1/observer.mjs`, plus the matching deny prefix. -With `root /srv`, install the three JS files under -`/srv/skillhub/_skillhub-observe/v1/`; relative imports follow automatically. -Also scope BOTH exact Umami endpoints and the deny prefix to -`/skillhub/_skillhub-umami/`, and change the entry's adapter `prefix` accordingly; -upstream `/script.js` and `/api/send` stay unchanged. Keep other site locations -unchanged; do not put sub_filter at http/server scope. Use any supported validated -base path consistently, preserving original request URIs. HTTPS/public registry, -OAuth callbacks and generated asset URLs still need the normal deployment setup. - -Rollback: remove the sub_filter directives/injection location customization, test -configuration, reload through the normal deployment procedure, and purge cached -HTML. Existing cached versioned JS can remain temporarily but must not be referenced -or initialized. Verify new HTML has no script, no callback/reporting occurs and -SkillHub remains usable. This document does not authorize production changes. +Supported-platform deployment, initialization, verification, upgrade, shutdown and +custom adapter development are documented in the guide. Custom adapters require +reviewed code, registry validation, cancellation, final payload cleaning and exact +proxy routes; business components must remain vendor-neutral and passive. diff --git a/openspec/changes/add-configurable-frontend-analytics/design.md b/openspec/changes/add-configurable-frontend-analytics/design.md index 2592fd0ba..df3e60e8e 100644 --- a/openspec/changes/add-configurable-frontend-analytics/design.md +++ b/openspec/changes/add-configurable-frontend-analytics/design.md @@ -53,6 +53,14 @@ Nginx 通过独立 include 片段选择已注册适配器的精确端点：Umami 部署配置变更经重建/重启 Web 使新页面生效；旧标签页不自动接收新环境参数。动态停止通过当前页面接口生效；紧急全局停止可关闭独立采集路由。文档不得宣称仅修改 env 即刻撤销已在发送的请求。模块使用正常发布构建产物与缓存策略，不把可变配置放入 immutable JS。 +### 5. Serialization and loading implementation + +PR-2 不新增 jq 或 Node 运行时依赖。镜像已有 BusyBox awk 负责有界 JSON 语法和部署控制字段校验，base64 将 JSON 作为数据追加到现有 no-store runtime-config.js；浏览器用 JSON.parse 解码，不经 eval，也不把任意 JSON 插入 JS 字面量。配置最大 64 KiB、最多 32 层，重复键拒绝。非法 rules 仍按既有 resolveRules 跳过，不升级为整个配置失败。 + +bootstrap 仅静态包含小型 gate 和注册参数校验；controller/helper 与 adapter 保持独立懒加载 chunk。在首个 React DOM commit 后的独立任务中启动 gate，不把正常挂载与 SDK 绑定。enabled=true 且 consent=denied 时不加载模块，但代理保留精确端点，供当前页显式 start() 授权后使用。静态 adapter factory 在动态导入返回后检查取消信号，迟到的导入不能重新创建 SDK。 + +旧示例目录仅保留指南指针。helper/adapter 与全部断言迁入产品测试树；历史 edge/Compose 只保留为测试 fixture，不是部署路径，不维护第二份采集实现。 + ## Risks and verification 重点验证默认零 SDK/采集请求、错误配置降级、页面启动不等统计 SDK、DNS失败不阻断Web、stop/init竞争、适配器字段清洗、根/子路径、100M原业务上传、登录/下载及配置回滚。固定 SDK 升级须验证真实发送顺序；不能将静态配置或模拟API测试描述为真实业务E2E。正式测试矩阵在实现和独立Review收敛后生成。 diff --git a/openspec/changes/add-configurable-frontend-analytics/proposal.md b/openspec/changes/add-configurable-frontend-analytics/proposal.md index 922a57161..6f276a7db 100644 --- a/openspec/changes/add-configurable-frontend-analytics/proposal.md +++ b/openspec/changes/add-configurable-frontend-analytics/proposal.md @@ -31,6 +31,6 @@ Issue #989 / PR #1010 已提供被动动作标签、通用 helper 和独立平 ## Impact -影响按阶段区分：PR-1 仅涉及被动标签契约、示例、测试和接入文档；以下产品部署影响属于 PR-2。影响 Web 可选模块、运行时配置、容器启动、Compose/Helm、部署测试和文档。新增小型 jq 镜像依赖用于标准 JSON 校验与安全转义，避免用 envsubst 拼接任意配置。默认部署不启动统计服务、不加载采集模块、不外发事件。 +影响按阶段区分：PR-1 仅涉及被动标签契约、示例、测试和接入文档；以下产品部署影响属于 PR-2。影响 Web 可选模块、运行时配置、容器启动、Compose/Helm、部署测试和文档。复用镜像已有 BusyBox awk 校验有界 JSON，使用 base64 将 JSON 安全传入现有 runtime-config，避免 envsubst 拼接任意配置，不新增 jq 或其他镜像依赖。默认部署不启动统计服务、不加载采集模块、不外发事件。 范围外：后台采集配置页、SkillHub 分析数据库、通用远程代码插件平台、用户级同意弹窗、可靠消息/重试队列、服务端业务成功事件、生产环境部署和自动合并。统计平台自己的数据库属于可选外部服务，不修改 SkillHub schema。 diff --git a/openspec/changes/add-configurable-frontend-analytics/tasks.md b/openspec/changes/add-configurable-frontend-analytics/tasks.md index 79fcab128..e5aaea562 100644 --- a/openspec/changes/add-configurable-frontend-analytics/tasks.md +++ b/openspec/changes/add-configurable-frontend-analytics/tasks.md @@ -12,25 +12,25 @@ - [ ] 2.7 Independent parent review and publication; keep Issue #989 open. ## 3. PR-2 (follow-up): optional frontend capability -- [ ] 3.1 Move helper/adapter to a single product source under web/src/features/analytics and preserve all PR-1 assertions. -- [ ] 3.2 Add default-closed runtime loader and vendor-neutral controller/registry; register Umami as the first adapter. -- [ ] 3.3 Cover consent, validation, duplicate init, async stop races and adapter failures. +- [x] 3.1 Move helper/adapter to a single product source under web/src/features/analytics and preserve all PR-1 assertions. +- [x] 3.2 Add default-closed runtime loader and vendor-neutral controller/registry; register Umami as the first adapter. +- [x] 3.3 Cover consent, validation, duplicate init, async stop races and adapter failures. - [ ] 3.4 Validate this OpenSpec change with the CLI when available and bind the follow-up PR/task. ## 4. PR-2 (follow-up): deployment -- [ ] 4.1 Safely serialize and validate configuration without breaking existing runtime config. -- [ ] 4.2 Add isolated proxy snippets with request-time DNS, certificate validation and exact endpoints. -- [ ] 4.3 Wire Docker/Compose/Helm defaults and reserved sub-path validation. -- [ ] 4.4 Provide optional persistent self-hosted Compose combination without another Nginx. +- [x] 4.1 Safely serialize and validate configuration without breaking existing runtime config. +- [x] 4.2 Add isolated proxy snippets with request-time DNS, certificate validation and exact endpoints. +- [ ] 4.3 Wire Docker/Compose/Helm defaults and reserved sub-path validation. (Implemented; Compose/runtime checks passed. Helm CLI rendering remains unverified.) +- [x] 4.4 Provide optional persistent self-hosted Compose combination without another Nginx. ## 5. PR-2 (follow-up): documentation -- [ ] 5.1 Replace source-editing quickstart with configuration-only production paths for supported platforms; distinguish custom adapter development. -- [ ] 5.2 Document fields, platform prerequisites, initialization, verification, troubleshooting, upgrade and shutdown. -- [ ] 5.3 Link deployment entry points and document adapter extension and actual privacy boundaries. +- [x] 5.1 Replace source-editing quickstart with configuration-only production paths for supported platforms; distinguish custom adapter development. +- [x] 5.2 Document fields, platform prerequisites, initialization, verification, troubleshooting, upgrade and shutdown. +- [x] 5.3 Link deployment entry points and document adapter extension and actual privacy boundaries. ## 6. PR-2 (follow-up): quality gates - [ ] 6.1 Consolidated independent implementation review, fixes and final base checkpoint. -- [ ] 6.2 Reviewed requirement/scenario/test matrix and targeted source/deployment checks. -- [ ] 6.3 Exact-SHA release Compose S3/auth smoke and Chromium normal/changed/failure scenarios, including default zero module/SDK/request and root/sub-path upload/login/download/CSP/cache compatibility. +- [ ] 6.2 Reviewed requirement/scenario/test matrix and targeted source/deployment checks. (Worker matrix and local checks complete; independent review and Helm execution pending.) +- [ ] 6.3 Exact-SHA release Compose S3/auth smoke and Chromium normal/changed/failure scenarios, including default zero module/SDK/request and root/sub-path upload/login/download/CSP/cache compatibility. (Local browser/real-collector and HTTP proxy checks passed; full backend S3/auth smoke remains pending.) - [ ] 6.4 CI/privacy/readiness, one validation report and retained preview for human acceptance. - [ ] 6.5 Close Issue #989 only after configuration-only deployment is complete and verified, not when PR-1 merges.