diff --git a/.github/workflows/pr-scripts.yml b/.github/workflows/pr-scripts.yml index a813412b6..38da2c209 100644 --- a/.github/workflows/pr-scripts.yml +++ b/.github/workflows/pr-scripts.yml @@ -57,6 +57,8 @@ jobs: - run: python -m unittest discover -s scanner/tests -p 'test_*.py' - run: bash scripts/tests/publish-cli-test.sh - run: bash scripts/tests/runtime-secret-test.sh + - run: bash scripts/tests/runtime-mail-test.sh + - run: bash scripts/tests/smtp-compose-render-test.sh - run: bash scripts/tests/runtime-analytics-test.sh - run: bash scripts/tests/validate-release-config-test.sh - run: bash scripts/tests/storage-compose-test.sh diff --git a/docs/09-deployment.md b/docs/09-deployment.md index ce5d221d1..7df887824 100644 --- a/docs/09-deployment.md +++ b/docs/09-deployment.md @@ -68,6 +68,24 @@ docker compose --env-file .env.release -f compose.release.yml up -d --no-deps -- `make staging` 仍用于从源码构建的容器回归验证,见[开发工作流](dev-workflow.md),不等同于正式交付。正式单机部署使用 `compose.release.yml`,不使用 `docker-compose.prod.yml`。 +## 一键部署后的邮件配置 + +邮件是已有 Registry 的可选能力,不改变 `runtime.sh up` 的部署顺序,也不要求为邮件创建第二套 Compose 项目。业务邮件要求所用发行包含[对应改动](https://github.com/iflytek/skillhub/pull/1004);未启用邮件时无需配置 SMTP 凭据。 + +一键安装后记录 **Runtime dir** 和 **Lifecycle commands**。邮件配置只编辑该实例原有的 `.env.release`,权限保持 `600`,不重新复制示例覆盖原文件。默认未指定 `--home` 时,脚本使用 `${TMPDIR:-/tmp}/skillhub-runtime`;正式实例固定持久目录,后续保持同一 home、文件来源和镜像选项。运行目录没有源码 Make 校验工具,不能在该目录照抄 `make validate-release-config`。 + +| 配置 | 所属位置 | 生效方式 | +|---|---|---| +| SMTP、TLS、授权码、发件人 | 原运行目录 `.env.release` | 编辑后重跑输出中的同实例 up 命令,不能只 restart | +| 平台开关、摘要周期、邮件站点地址 | 系统配置 → 业务邮件通知 | 环境仅首次初始化缺失设置;已有策略以数据库为准 | +| 收件证明、即时/摘要类别 | 用户通知设置 | 每个账号自行验邮并订阅;站内通知不要求验邮 | + +已有环境先在后台关闭业务邮件,再配置 SMTP 并重跑原 up;核对后端健康和 SMTP 配置后,在协调窗口重新启用平台邮件,再用未验邮测试账号获取验证码,或用已验邮测试账号触发新业务事件,验证即时收件、窗口摘要和邮件链接。平台关闭时不能测试通知验证码或业务邮件;重新启用期间,既有合格订阅用户的新事件也可能发信,不能把受控启用当作测试账号隔离。测试失败则再次在后台关闭,按状态排查,不盲目重试 UNKNOWN;通过后恢复测试偏好并按部署安排正常开放。SMTP 更新也影响共用连接的密码重置邮件,两类发件地址分别配置。旧空 SSL_TRUST 由修复脚本清除,明确主机例外保留;不使用空值或通配信任代替 JVM 证书验证。 + +升级使用匹配的脚本/配置来源与镜像版本;`--ref` 不等于 `--version`,`--aliyun` 的运行文件来自 OSS,不能仅凭 Git ref 推定其版本。新版模板不会覆盖现存 `.env.release`,也不会覆盖入库平台策略。关闭邮件不会撤回已授权或已被 SMTP 接受的发送,不要删除队列或数据卷作为停用步骤。 + +完整命令、SMTP 参数、升级保留、密码轮换和故障矩阵见[一键部署邮件接入流程](notification-email.md);仅密码重置邮件见[SMTP 配置指南](19-smtp-password-reset-email-setup.md)。 + ## 2 单机交付拓扑 ``` @@ -591,7 +609,7 @@ env: Kubernetes 集群节点或出口网关的公网 IP 必须加入钉钉服务器出口 IP 白名单。Ingress 只负责浏览器 回调可达性,不会替代后端出站 IP 白名单。 -- 如果要启用密码重置验证码邮件,参见:`docs/19-smtp-password-reset-email-setup.md` +密码重置验证码使用独立发件人及安全流程,参见[SMTP 配置指南](19-smtp-password-reset-email-setup.md);业务邮件按[邮件通知接入流程](notification-email.md)配置平台策略及用户订阅。 ## 8 OIDC 登录配置 diff --git a/docs/19-smtp-password-reset-email-setup.md b/docs/19-smtp-password-reset-email-setup.md index 067bb4f0d..550645fd8 100644 --- a/docs/19-smtp-password-reset-email-setup.md +++ b/docs/19-smtp-password-reset-email-setup.md @@ -1,6 +1,6 @@ # SkillHub SMTP 邮箱配置指南(验证码邮件) -本文说明如何为 SkillHub 配置 SMTP,用于发送“密码重置验证码”邮件。 +本页说明如何为 SkillHub 配置 SMTP,用于发送“密码重置验证码”邮件。 审核、发布、订阅更新等业务邮件及通知收件验证,请另见[邮件通知接入与使用](notification-email.md)。两类邮件复用 SMTP 连接,但发件人配置、验证码用途和用户订阅相互独立。 @@ -10,7 +10,7 @@ 补充说明: - SMTP 本质是邮件传输协议,不是单一厂商产品。 -- 你可以使用企业邮箱、云邮箱或本地测试 SMTP 服务(例如 MailHog)作为 SMTP 服务端。 +- 可使用企业邮箱、云邮箱或本地测试 SMTP 服务(例如 MailHog)作为 SMTP 服务端。 当前密码重置页面入口说明: - 当前前端统一使用 `/reset-password` 页面。 @@ -37,7 +37,7 @@ 说明: - 当前文档统一按 `465 + SSL` 配置,不再展开 `587 + STARTTLS` 方案。 - 使用 `465` 时配置:`STARTTLS=false`、`SSL_ENABLE=true`。 -- 若出现 `PKIX path building failed` / `SSLHandshakeException`,可尝试增加 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST=`(本地联调常用)。 +- 若出现 `PKIX path building failed` / `SSLHandshakeException`,先检查 SMTP 证书链与 JVM 信任库;不要直接关闭 TLS 或绕过证书校验。 - 默认不得配置 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST`,以使用 JVM 证书信任库。旧 `.env.release` 若包含 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST=` 空行配置,必须删除该配置行后重建后端;空字符串会导致 `Server is not trusted`,不能用它表示“使用默认信任库”。只有经过部署方确认的证书信任例外才填写明确主机名,不得填写 `*`。 - `SKILLHUB_AUTH_PASSWORD_RESET_CODE_EXPIRY` 支持如 `PT5M`、`PT10M`、`PT30M`。 @@ -102,41 +102,32 @@ SKILLHUB_AUTH_PASSWORD_RESET_FROM_ADDRESS=mailer@example.com SKILLHUB_AUTH_PASSWORD_RESET_FROM_NAME=your-from-name ``` -## 2. 单机交付(Compose)配置步骤 +## 2. 单机交付配置步骤 -1. 复制环境模板(若尚未创建): +### 一键安装的已有实例 + +1. 从安装输出找到 **Runtime dir** 和 **Lifecycle commands**。只编辑该实例 Runtime dir 中已有的 `.env.release`;不要重新复制模板,不要改用新的 `--home`。未指定目录时脚本默认 `${TMPDIR:-/tmp}/skillhub-runtime`,不是 `~/.skillhub`。 +2. 文件权限保持 `600`,按第 1 节填写 SMTP、TLS、密码或授权码,并设置获 SMTP 许可的 `SKILLHUB_AUTH_PASSWORD_RESET_FROM_ADDRESS`。真实秘密不得放在命令参数、日志或 Git 中;不要执行 `source .env.release`。包含特殊字符的密码按 Compose `.env` 单引号字面值语法保存。 +3. 若该 SMTP 还用于业务通知,先在后台关闭业务邮件;修改现有环境的 seed 开关不会覆盖已保存的平台设置。业务邮件与通知验邮使用 `SKILLHUB_NOTIFICATION_EMAIL_FROM_ADDRESS`,不是密码重置发件人参数。 +4. 保存后重新执行该实例输出的 **up** 命令,保留原 `--home`、文件来源及镜像参数。没有单独的 SMTP 命令;不能只执行 restart 或 pull。脚本保留现存环境文件,更新配置后由 Compose 重建受影响容器;使用 ps/logs 命令核对后端状态。 +5. 在 `/reset-password` 为受控测试账号申请验证码,实际检查收件箱及垃圾邮件并验证重置流程。SMTP 已配置或容器健康不证明实际收件。密码重置不要求先开启业务邮件,也不要求用户先完成通知邮箱验邮。 +6. 同时使用业务邮件时,按[一键部署邮件接入流程](notification-email.md)先在协调窗口重新启用平台,再用未验邮测试账号验证通知验证码,或用已验邮账号触发新业务事件验证即时/摘要;不能在平台关闭时进行这些测试。既有合格订阅用户在启用期间的新事件也可能发信;失败则后台关闭,不盲目重试 UNKNOWN,测试通过后恢复测试偏好并正常开放。 + +包含本次修复的脚本会移除旧 `.env.release` 中空的 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST`,保留明确主机例外及其他 SMTP 行。旧脚本需先更新,或手动删除该空值行后重跑 up。不要配置 `*`;正常服务端证书使用 JVM 信任库。 + +运行目录只含部署文件,不包含 Makefile 和源码检查脚本,不在该目录运行 `make validate-release-config`。固定发行时匹配已发布的脚本、`--ref` 和镜像 `--version`;使用阿里云、镜像仓库或其他选项的原实例继续保留这些选项。详细来源与升级边界见[部署运维手册](09-deployment.md)。 + +### 已有源码仓库的 Compose 实例 + +首次准备时才复制 `.env.release.example`;已有配置文件不覆盖。编辑 SMTP 参数后,在仓库根目录执行: ```bash -cp .env.release.example .env.release +make validate-release-config +docker compose --env-file .env.release -f compose.release.yml up -d --no-deps server +docker compose --env-file .env.release -f compose.release.yml logs --tail=100 server ``` -2. 编辑 `.env.release`,填写 SMTP 变量: - -```dotenv -SPRING_MAIL_HOST=smtp.mail.example -SPRING_MAIL_PORT=465 -SPRING_MAIL_USERNAME=mailer@example.com -SPRING_MAIL_PASSWORD=your-smtp-app-password -SPRING_MAIL_SMTP_AUTH=true -SPRING_MAIL_SMTP_STARTTLS_ENABLE=false -SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_ENABLE=true - -SKILLHUB_AUTH_PASSWORD_RESET_CODE_EXPIRY=PT10M -SKILLHUB_AUTH_PASSWORD_RESET_FROM_ADDRESS=mailer@example.com -SKILLHUB_AUTH_PASSWORD_RESET_FROM_NAME=your-from-name -``` - -3. 重启后端容器使配置生效: - -```bash -docker compose --env-file .env.release -f compose.release.yml up -d server -``` - -4. 查看后端日志确认启动正常: - -```bash -docker compose --env-file .env.release -f compose.release.yml logs -f server -``` +以上适用于未自定义项目名或 override 的已有栈;原部署若使用 `-p`、`COMPOSE_PROJECT_NAME` 或其他 `-f` 文件,必须保留全部原参数,防止操作另一实例或丢失设置。首次部署使用完整栈启动,不照抄 `--no-deps`。`restart` 不更新容器环境。配置检查输出可能带配置值,不公开粘贴完整输出。 ## 3. 本地开发配置与验证 @@ -164,10 +155,16 @@ make dev-server - 必须 `source ~/.zshrc` 或重开终端后变量才会生效 - 需要在“同一个终端”启动 `make dev-server` -可先确认变量是否在当前 shell 中: +只确认所需变量是否存在,不输出变量值: ```bash -env | rg '^(SPRING_MAIL_|SKILLHUB_AUTH_PASSWORD_RESET_)' +for key in SPRING_MAIL_HOST SPRING_MAIL_USERNAME SPRING_MAIL_PASSWORD SKILLHUB_AUTH_PASSWORD_RESET_FROM_ADDRESS; do + if [ -n "$(printenv "$key")" ]; then + printf '%s: configured\n' "$key" + else + printf '%s: missing\n' "$key" + fi +done ``` ### 3.3 推荐联调方式(MailHog) @@ -269,7 +266,13 @@ curl -X POST http://localhost:8080/api/v1/auth/local/password-reset/request \ lsof -nP -iTCP:8080 -sTCP:LISTEN # 查看当前 shell 是否有 SMTP 环境变量 -env | rg '^(SPRING_MAIL_|SKILLHUB_AUTH_PASSWORD_RESET_)' +for key in SPRING_MAIL_HOST SPRING_MAIL_USERNAME SPRING_MAIL_PASSWORD SKILLHUB_AUTH_PASSWORD_RESET_FROM_ADDRESS; do + if [ -n "$(printenv "$key")" ]; then + printf '%s: configured\n' "$key" + else + printf '%s: missing\n' "$key" + fi +done ``` ### 5.4 发件人被拒绝 diff --git a/docs/notification-email.md b/docs/notification-email.md index 846d9ac2a..6fc2677ac 100644 --- a/docs/notification-email.md +++ b/docs/notification-email.md @@ -4,6 +4,112 @@ SkillHub 可以通过部署方的 SMTP 服务发送业务通知。邮件与站 SMTP 连接、TLS 和企业邮箱接入方式见 [SMTP 配置指南](19-smtp-password-reset-email-setup.md)。该指南中的密码重置验证码配置仍属于密码重置;业务通知与通知邮箱验证码使用本页说明的独立发件人配置,不改变登录或注册流程。 +## 一键部署:从安装到实际收件 + +本节适用于 `runtime.sh` 安装的单机实例。业务通知要求目标发行包含[业务邮件通知改动](https://github.com/iflytek/skillhub/pull/1004);仅存在 SMTP 参数不能证明所用镜像已提供该功能。不需要启用邮件的实例保持默认配置,不必申请 SMTP 账号,也不必完成用户验邮。 + +### 保持同一个实例与版本来源 + +首次安装仍使用原一键入口,邮件不增加第二套 Compose 项目或独立服务。例如: + +```bash +curl -fsSL https://imageless.oss-cn-beijing.aliyuncs.com/runtime.sh | sh -s -- up \ + --home /opt/skillhub \ + --public-url https://skills.example.com +``` + +`/opt/skillhub` 和域名均为部署示例;执行账号必须能写入该目录并访问 Docker。成功后记录输出中的 **Runtime dir** 和 **Lifecycle commands**,后续优先复制该实例输出的生命周期命令。已有实例不得改用示例目录:`--home` 改变后可能操作另一套实例。未指定目录时,脚本默认使用 `${TMPDIR:-/tmp}/skillhub-runtime`,不是当前仓库目录或 `~/.skillhub`;正式部署必须使用可持久保存的明确目录。 + +| 操作 | 保持的参数与配置 | 说明 | +|---|---|---| +| 重跑安装命令 | 原 `--home`、文件来源及镜像选项 | 脚本更新 Compose 和示例模板,已有 `.env.release` 保留 | +| 固定发行 | 匹配的 `--ref` 与 `--version` | `--ref` 选择 GitHub 配置文件来源,`--version` 选择镜像标签,不能互相替代 | +| 使用阿里云入口 | 原 `--aliyun` 或 `--mirror-registry` 等选项 | `--aliyun` 的运行文件来自 OSS,不能仅凭 `--ref` 推定 OSS 文件已固定为某个 Git tag | +| 使用自定义项目或 override | 原项目名、环境文件和全部 override | 保留原部署命令;不要改用另一组基础 Compose 命令 | + +默认 GitHub 文件来源是 `main`。需要固定发行时,必须从匹配的已发布 Git ref 获取脚本,并将同一 ref 传给 `--ref`、将匹配镜像标签传给 `--version`;目标取值查对应 Release,不使用未发布标签。只指定 `--version` 不会同时固定脚本及 Compose 文件。 + +### 编辑原运行目录中的私有配置 + +一键脚本只下载部署清单和环境模板,不下载 Makefile 或配置校验脚本。不要在 Runtime dir 运行 `make validate-release-config`,也不要重新复制模板覆盖已有配置。 + +先在 **系统配置 → 业务邮件通知** 关闭平台邮件;初次安装默认关闭。已有数据库中的平台开关不受环境初值覆盖,因此不能只把 `SKILLHUB_NOTIFICATION_EMAIL_ENABLED` 改为 `false` 就认为已停止发送。 + +进入实际 Runtime dir,编辑已有 `.env.release`。下列命令中的目录须替换为该实例记录的目录: + +```bash +cd /opt/skillhub +chmod 600 .env.release +vi .env.release +``` + +仅更新相应行,不替换数据库密码、存储凭证、匿名下载密钥或其他已有配置。真实密码只写入该私有文件,不放在命令参数、聊天记录或 Git 中;密码含 `$`、`#` 等字符时,使用 Compose 支持的单引号字面值写法,不执行 `source .env.release`。 + +```dotenv +SPRING_MAIL_HOST=smtp.example.com +SPRING_MAIL_PORT=465 +SPRING_MAIL_USERNAME=notifications@example.com +SPRING_MAIL_PASSWORD='replace-with-smtp-app-password' +SPRING_MAIL_SMTP_AUTH=true +SPRING_MAIL_SMTP_STARTTLS_ENABLE=false +SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_ENABLE=true +SKILLHUB_NOTIFICATION_EMAIL_FROM_ADDRESS=notifications@example.com +SKILLHUB_NOTIFICATION_EMAIL_FROM_NAME=SkillHub +SKILLHUB_NOTIFICATION_EMAIL_ENABLED=false +SKILLHUB_NOTIFICATION_EMAIL_DIGEST_WINDOW_MINUTES=60 +SKILLHUB_NOTIFICATION_EMAIL_PUBLIC_BASE_URL=https://skills.example.com +``` + +以上均为占位值,发件地址必须得到 SMTP 账号许可。`465 + SSL` 需要 `STARTTLS=false`、`SSL_ENABLE=true`;服务商要求 `587 + STARTTLS` 时改为端口 `587`、`STARTTLS=true`、`SSL_ENABLE=false`。不配置 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST`,默认使用 JVM 证书信任库;不能用空字符串或 `*` 代替正常证书校验。旧配置中该变量的空值由包含本次修复的脚本移除,明确主机例外保留;旧脚本实例需先更新脚本或手动删除空行。 + +| 配置层 | 修改入口 | 生效与边界 | +|---|---|---| +| SMTP 地址、密码、TLS、发件人 | 原 `.env.release` | 重跑同实例 `up` 后重建受影响容器;只 `restart` 不重新注入环境 | +| 平台开关、摘要窗口、站点地址的初值 | `SKILLHUB_NOTIFICATION_EMAIL_ENABLED` 等前三项 | 仅初始化缺失的数据库设置;不是覆盖后台的开关 | +| 平台邮件策略 | 超级管理员的系统配置 | 保存到数据库后生效,普通重启和升级不覆盖 | +| 用户邮箱证明与类别偏好 | 当前账号的通知设置 | 用户自行验邮并选择即时或摘要;管理员不能代订阅 | + +### 仍使用原一键命令加载配置 + +保存文件后,重新执行安装时记录的 **up** 命令,保持同一 `--home` 和文件/镜像来源。例如上面的实例仍执行相同命令: + +```bash +curl -fsSL https://imageless.oss-cn-beijing.aliyuncs.com/runtime.sh | sh -s -- up \ + --home /opt/skillhub \ + --public-url https://skills.example.com +``` + +一键脚本没有单独的 SMTP 开关命令。`up` 会按原顺序准备部署文件、检查数据库连接并启动应用;邮件参数由已有 Compose 透传,环境变化时更新容器。`pull` 只下载镜像,不启动更新后的应用;单独执行 `docker compose restart` 也不加载新环境。原部署若使用固定 `--ref`、`--version`、镜像仓库或其他选项,必须继续保留,不能直接照抄省略这些选项的示例。 + +通过记录的 **ps** 和 **logs** 命令查看服务及启动日志。Runtime dir 中也可使用该实例原有的 Compose 项目参数查看后端日志;不要运行完整 `config`、`env` 或 `docker inspect` 后把秘密值贴到问题单。 + +### 管理员启用,用户完成收件验证 + +1. 超级管理员打开 **系统配置 → 业务邮件通知**,核对“SMTP 配置存在”,设置真实 Web 站点根地址和摘要窗口后启用。子路径必须包含在站点地址中,不能使用仅服务器自身可访问的 `localhost` 或 `127.0.0.1`;该地址不会自动配置 DNS、证书或反向代理。 +2. 测试账号进入 **通知设置**,核对该账号的脱敏邮箱,获取并填写通知邮箱验证码。真实邮箱收到验证码且页面显示“已验证”后,才证明应用到收件邮箱的该条路径可用;后台显示“SMTP 配置存在”不能替代这个验证。 +3. 将一个测试类别设为即时,产生一次新的发布或审核事件;核对真实收件、投递记录及邮件链接。再将该类别设为摘要,在同一窗口内产生两条事件并等窗口结束,核对合并内容。 +4. 用收到邮件的同一 SkillHub 账号打开明细,核对跳转目标;换另一个账号应被拒绝。测试结束后恢复测试偏好,再由用户自主订阅。 + +### 升级、轮换与关闭 + +升级前备份 `.env.release`、数据库和对象存储,并记录原文件来源与镜像标签。在同一个 Runtime dir 更新到匹配的发行,重跑对应 **up**;脚本只为不存在的 `.env.release` 复制模板,不用新版模板覆盖已有 SMTP 参数。丢失私有文件必须恢复备份,不能把重新生成的默认文件当作原凭证。 + +轮换 SMTP 密码或发件人时,先在后台关闭业务邮件并处理在途状态,再编辑原 `.env.release`、重跑同实例 **up**。先核对后端健康和 SMTP 配置;这一步不证明实际收件。随后在协调好的验证窗口重新启用平台邮件:未验邮测试账号先获取验证码并验邮,已验邮测试账号直接触发新业务事件,分别验证即时和摘要。平台关闭时通知邮箱验证码及业务邮件测试均不可用,不能把实际收件测试放在重新启用之前。验证失败时在后台再次关闭,按投递状态排查,不盲目重试 UNKNOWN。 + +受控启用不是收件账号隔离开关;已经验邮并订阅的既有用户,在平台启用期间也可能收到其新事件。验证窗口须协调业务操作,测试完成后恢复测试账号偏好并按部署安排正常开放。SMTP 连接被密码重置邮件共用,修改会同时影响该通道;若启用密码重置,另显式配置获准的 `SKILLHUB_AUTH_PASSWORD_RESET_FROM_ADDRESS`。仅轮换 SMTP 基础设施不要求把已验证用户全部重新验邮,但用户账号邮箱发生变化后必须重新验证。 + +暂停业务邮件只需在后台关闭;不删除容器、数据库、队列或数据卷。关闭会取消尚未取得发送授权的项,已授权或 SMTP 已接受的邮件不能撤回,重新开启也不补发关闭期间事件。应用降级另按本页“可靠性与保留边界”处理,不能用退回镜像代替数据库及验邮证明的兼容检查。 + +| 现象 | 定位位置 | 检查与处理 | +|---|---|---| +| 不配置 SMTP 后服务无法启动 | 同实例后端启动日志 | 默认不启用邮件不应要求 SMTP 凭据;先排查数据库、存储等实际错误,不填虚构 SMTP 服务器绕过启动 | +| 配置已改,发送仍使用旧参数 | Runtime dir、原 up 命令、容器更新时间 | 确认编辑的是同一实例文件并重跑 up;不要仅 restart 或另建一个 home | +| SSL 报 `Server is not trusted` | 旧 `.env.release` 的信任覆盖项 | 移除空 SSL_TRUST,使用修复脚本重跑 up;不要关闭 TLS 或设置通配信任 | +| 验证码发送失败后提示频繁 | 通知设置的重发倒计时、后端安全错误码 | 等待服务端冷却结束后再试,先处理 SMTP/TLS/账号错误;不要连续点击或清空限流状态 | +| 验证码收到但没有业务邮件 | 平台开关、当前账号验证状态、对应邮件类别 | 开启必要条件后产生新事件;不要用历史事件或站内通知开关推断邮件已订阅 | +| 摘要尚未到达 | 管理员摘要窗口和投递记录 | 等该窗口到期,再按实际 SMTP 结果排查;不要将等待窗口当作失败反复重试 | +| 邮件能收到,链接打不开 | 后台站点地址、当前登录账号 | 核对真实域名/子路径并使用收件账号登录;不要放宽明细权限解决账号不匹配 | + ## 用户如何开启 1. 登录后进入 **通知设置**(`/settings/notifications`)。 @@ -82,7 +188,7 @@ SPRING_MAIL_SMTP_READ_TIMEOUT_MS=10000 SPRING_MAIL_SMTP_WRITE_TIMEOUT_MS=10000 ``` -保持关闭部署并验证连通后,由超级管理员在后台启用。SMTP 和发件人改变需要更新部署、重启后端;通知开关、周期和站点 URL 保存后由数据库生效。 +部署时保持关闭,先核对后端健康和 SMTP 配置,再由超级管理员在协调窗口受控启用,最后执行通知验邮及实际业务收件测试;平台关闭时不能执行这些测试。SMTP 和发件人改变需要更新部署并重建后端容器;直接运行的宿主机后端需重启进程。通知开关、周期和站点 URL 保存后由数据库生效。 ### Helm @@ -161,4 +267,4 @@ Chart 配置与安装方式见 [Helm 手册](../charts/skillhub/README.md)。渲 到期历史按有限批次逐步清理。恢复组开始清理后即视为过期,用户明细和管理员投递列表不再显示,也不能人工重试;物理记录可能在后续多轮才全部删除。清理中断后继续原进度,不会因为重启而恢复部分已删除的邮件内容。分页限制约束单轮写入规模,不代表任意数据量下都有固定清理耗时。 -回退应用先关闭邮件、等待 worker 结束并保留表。旧应用不维护邮箱修订号,因此回退前须由受控维护操作失效全部通知验邮证明/挑战、抑制未发项;重新升级保持关闭,用户重新验邮和订阅。不得仅回退镜像后恢复旧验邮证明,也不要删除业务或站内通知数据来“回滚”。具体维护命令应由部署者针对已验证的版本与数据库备份执行。 +回退应用先关闭邮件、等待 worker 结束并保留表。旧应用不维护邮箱修订号,因此回退前须由受控维护操作失效全部通知验邮证明/挑战、抑制未发项;重新升级先保持关闭;完成部署核对并在协调窗口受控启用后,用户重新验邮和订阅。不得仅回退镜像后恢复旧验邮证明,也不要删除业务或站内通知数据来“回滚”。具体维护命令应由部署者针对已验证的版本与数据库备份执行。 diff --git a/docs/skillhub/en/guide/deployment.md b/docs/skillhub/en/guide/deployment.md index a7201bac5..074c6b804 100644 --- a/docs/skillhub/en/guide/deployment.md +++ b/docs/skillhub/en/guide/deployment.md @@ -1,10 +1,43 @@ # Docker Compose Deployment and Upgrades -You do not need to deploy a server to install skills from an existing registry. Use this guide when hosting your own registry. `make dev-all` runs source development; `make staging` is for container regression testing; `compose.release.yml` is the single-server release deployment. For clusters, see [Kubernetes](kubernetes.md). +You do not need to deploy a server to install skills from an existing registry. Use this guide when hosting your own registry. `make dev-all` runs source development; `make staging` is for container regression testing; `compose.release.yml` is the single-server release deployment. For clusters, see [Kubernetes](kubernetes.html). + +## One-click script deployment + +The existing installer does not require SMTP configuration. Registry deployment and use remain unchanged when email is disabled. Use an explicit persistent directory for a long-lived instance: + +```bash +curl -fsSL https://imageless.oss-cn-beijing.aliyuncs.com/runtime.sh | sh -s -- up \ + --home /opt/skillhub \ + --public-url https://skills.example.com +``` + +The directory and domain are examples. The operator must have write access to the directory and access to Docker. A public URL does not configure DNS, TLS, or a reverse proxy. Record the printed **Runtime dir** and **Lifecycle commands**. Reuse those commands for the existing instance rather than changing its directory to the example. Without `--home`, the default is `${TMPDIR:-/tmp}/skillhub-runtime`, not `~/.skillhub`. + +The script downloads Compose and `.env.release.example`, and creates the private `.env.release` only when it does not exist. Existing SMTP credentials remain in that file. Runtime dir does not contain the repository's Makefile or validation tools; do not run the source-checkout-only `make validate-release-config` there. + +| Operation | Procedure | Explanation | +|---|---|---| +| Change SMTP or other environment settings | Edit the existing Runtime dir's `.env.release`, then repeat that instance's up command | Changed environment values require container recreation; restart alone does not reload them | +| Update images | Keep the same home and source options, and use the target release's up command | pull downloads images without activating new containers | +| Pin a release | Use matching published script files, `--ref`, and `--version` | ref selects GitHub deployment files; version selects image tags; neither pins both | +| Inspect or stop | Use the printed ps, logs, and down commands | Do not add `-v` to down or use clean as a troubleshooting step | + +Preserve `--aliyun`, mirror/image options, and any custom Compose configuration from the original deployment. With `--aliyun`, runtime files come from OSS; `--ref` alone does not prove those files match a Git tag. Check both deployment-file sources and image versions before upgrading. + +### Configure business email + +The target release must include the [business email changes](https://github.com/iflytek/skillhub/pull/1004). Disable business email in administration before changing SMTP. Edit the existing Runtime dir's `.env.release` with the SMTP host, account/app password, port, TLS mode, and an authorized sender address. Keep file permissions at `600`; do not commit or share it. Do not copy a fresh template over existing database, storage, or mail credentials. + +Repeat the original up command and check backend health and SMTP configuration. A super administrator then sets the real site URL and digest window and enables business email during a coordinated verification window in **System configuration → Business email notifications**. An unverified test account can then verify its address in **Notification settings**; an already verified test account can trigger a new event to test immediate and digest delivery. Notification verification codes and business emails cannot be tested while the platform is disabled. Enabling is not recipient isolation: new events for existing eligible subscribers can also send email during that window. Disable the platform again on failure and inspect delivery status without blindly retrying UNKNOWN; after acceptance, restore test preferences and resume normal operation. Rotating SMTP credentials alone does not require all users to verify again; changing an account's email does. SMTP credentials belong to deployment configuration; persisted platform policy is managed in administration; user verification and preferences are separate. Environment edits do not replace persisted settings. + +The updated installer removes empty legacy `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST` overrides while preserving explicit host exceptions and certificate validation. For an older installer, remove the empty override manually before recreating the backend. Leave the variable unset by default; do not use `*`. + +Acceptance includes receiving a real verification code, an immediate event, a combined digest for one window, and opening email details with the receiving SkillHub account. Configuration-present indicators and healthy containers do not prove inbox delivery. Upgrades preserve private configuration; deployment seed values do not overwrite saved platform policy. See the [email operations guide (Chinese)](https://github.com/iflytek/skillhub/blob/main/docs/notification-email.md) for parameters, initial setup, rotation, disabling, and failure diagnosis. ## Configure the instance -Install Docker and Docker Compose, and obtain repository files matching your target release. From the repository root: +The following procedure is for a source checkout; one-click installations use the Runtime dir procedure above. Install Docker and Docker Compose, and obtain repository files matching your target release. From the repository root: ```bash cp .env.release.example .env.release @@ -20,7 +53,7 @@ Edit `.env.release` before starting: | `SKILLHUB_DOWNLOAD_ANON_COOKIE_SECRET` | Generate a random value of at least 32 characters; the placeholder prevents startup | | `POSTGRES_PASSWORD` | Set your own database password | | `BOOTSTRAP_ADMIN_ENABLED` / `BOOTSTRAP_ADMIN_PASSWORD` | Decide whether to initialize an administrator; replace the example password when enabled | -| Storage | Follow the template and the [object storage guide](https://github.com/iflytek/skillhub/blob/main/docs/object-storage.md) for external storage | +| Storage | Follow the template and the [object storage guide (Chinese)](https://github.com/iflytek/skillhub/blob/main/docs/object-storage.md) for external storage | Do not commit or share `.env.release`. Changing an environment variable does not rotate the password of an existing database. Do not enable the `local` profile's mock authentication in a release deployment. @@ -36,7 +69,7 @@ docker compose --env-file .env.release -f compose.release.yml logs --tail=100 se Resolve validation errors before starting. Backend startup runs Flyway database migrations. Open your configured public URL, sign in with your configured administrator credentials, and verify search, download, and an authorized publishing/review flow. Healthy containers alone do not prove those workflows work. -Suite review writes are enabled by default in the single-server release Compose topology; Suite Bundle confirmation is disabled by default. See [Bundle import](suite-bundle.md) for enabling it. For rolling upgrades where old and new backends coexist, follow the [Suite deployment order](https://github.com/iflytek/skillhub/blob/main/docs/25-skill-suites.md). +Suite review writes are enabled by default in the single-server release Compose topology; Suite Bundle confirmation is disabled by default. See [Bundle import](suite-bundle.html) for enabling it. For rolling upgrades where old and new backends coexist, follow the [Suite deployment order (Chinese)](https://github.com/iflytek/skillhub/blob/main/docs/25-skill-suites.md). ## CLI device authorization URL @@ -59,7 +92,7 @@ docker compose --env-file .env.release -f compose.release.yml up -d --no-deps -- If the original deployment uses `-p`, `COMPOSE_PROJECT_NAME`, or additional `-f` files, retain its project name, environment file and every `-f` argument in the original order. Do not copy the base command unchanged: it could target another instance or omit override settings. This only recreates the existing backend; it does not download deployment files again. -Use the full `up -d` for an initial deployment. A plain `restart` does not reload container environment variables. If other shared settings changed, recreate their affected services too. To verify the device URL, retry the login command without `--token`, `SKILLHUB_TOKEN`, or saved credentials for that registry, and check the returned public URL. Successful login with an existing token does not verify the device URL. Do not share codes or tokens. See the [operations reference](https://github.com/iflytek/skillhub/blob/main/docs/09-deployment.md) for the full configuration contract. +Use the full `up -d` for an initial deployment. A plain `restart` does not reload container environment variables. If other shared settings changed, recreate their affected services too. To verify the device URL, retry the login command without `--token`, `SKILLHUB_TOKEN`, or saved credentials for that registry, and check the returned public URL. Successful login with an existing token does not verify the device URL. Do not share codes or tokens. See the [operations reference (Chinese)](https://github.com/iflytek/skillhub/blob/main/docs/09-deployment.md) for the full configuration contract. ## Upgrade, stop, and troubleshoot @@ -70,7 +103,7 @@ Before upgrading, record the current image version and configuration, and back u docker compose --env-file .env.release -f compose.release.yml down ``` -For startup failures, inspect `ps` and service logs, then check database, Redis, storage, and public URL configuration. Include image versions, redacted logs, and request IDs when reporting a problem, never credentials or `.env.release`. See the [operations reference](https://github.com/iflytek/skillhub/blob/main/docs/09-deployment.md) for Redis, reverse proxies, collection services, and migrations. +For startup failures, inspect `ps` and service logs, then check database, Redis, storage, and public URL configuration. Include image versions, redacted logs, and request IDs when reporting a problem, never credentials or `.env.release`. See the [operations reference (Chinese)](https://github.com/iflytek/skillhub/blob/main/docs/09-deployment.md) for Redis, reverse proxies, collection services, and migrations. ## Configuration and acceptance checks diff --git a/docs/skillhub/guide/deployment.md b/docs/skillhub/guide/deployment.md index ae8f69703..e10b9830c 100644 --- a/docs/skillhub/guide/deployment.md +++ b/docs/skillhub/guide/deployment.md @@ -1,10 +1,43 @@ # Docker Compose 部署与升级 -使用已有 Registry 安装技能不需要部署服务。只有需要自托管 Registry 时才执行本页流程。源码开发使用 `make dev-all`;容器回归使用 `make staging`;正式单机交付使用 `compose.release.yml`。集群部署见 [Kubernetes](kubernetes.md)。 +使用已有 Registry 安装技能不需要部署服务。只有需要自托管 Registry 时才执行本页流程。源码开发使用 `make dev-all`;容器回归使用 `make staging`;正式单机交付使用 `compose.release.yml`。集群部署见 [Kubernetes](kubernetes.html)。 + +## 一键脚本部署 + +一键安装仍使用原入口,不要求先配置 SMTP;不启用邮件时,Registry 的部署和使用流程保持不变。正式实例使用固定且可持久保存的目录: + +```bash +curl -fsSL https://imageless.oss-cn-beijing.aliyuncs.com/runtime.sh | sh -s -- up \ + --home /opt/skillhub \ + --public-url https://skills.example.com +``` + +目录与域名为示例,执行账号必须能写入目录并访问 Docker;公开地址不会自动配置 DNS、TLS 或反向代理。记录脚本输出的 **Runtime dir** 和 **Lifecycle commands**;已有实例后续使用输出的同一实例命令,不改用示例目录。不指定 `--home` 时默认目录是 `${TMPDIR:-/tmp}/skillhub-runtime`,不是 `~/.skillhub`。 + +脚本下载 Compose 清单和 `.env.release.example`,仅在私有 `.env.release` 不存在时创建它;已有文件及 SMTP 密钥保留。该目录不包含 Makefile 和源码校验工具,不要在 Runtime dir 执行下面源码部署专用的 `make validate-release-config`。 + +| 操作 | 执行方式 | 说明 | +|---|---|---| +| 修改 SMTP 或其他环境配置 | 编辑原 Runtime dir 的 `.env.release`,重跑该实例 up 命令 | 环境变化后更新容器;不能仅 restart | +| 更新镜像 | 保留原 home 和来源参数,使用目标发行对应的 up 命令 | pull 只下载镜像,不启用新容器 | +| 固定发行 | 使用匹配的已发布脚本、`--ref` 和 `--version` | ref 选择 GitHub 配置来源,version 选择镜像;只指定一个不构成完整固定 | +| 查看与停止 | 使用输出的 ps、logs、down 命令 | down 不附加 `-v`;不要用 clean 作为排障步骤 | + +原部署使用 `--aliyun`、镜像仓库或额外 Compose 配置时,必须保留这些选项。`--aliyun` 的运行文件来自 OSS,不能仅凭 `--ref` 推定 OSS 文件已固定到某 Git tag;升级前核对实际文件来源与镜像版本。 + +### 配置业务邮件 + +目标发行须包含[业务邮件通知改动](https://github.com/iflytek/skillhub/pull/1004)。在后台先关闭业务邮件,再编辑原运行目录的 `.env.release`,填入 SMTP 地址、账号/授权码、端口、TLS 模式及获准发件地址;文件权限保持 `600`,不提交或分享。不要重新复制环境模板覆盖数据库、存储或邮件凭据。 + +重跑原 up 命令后先核对健康和 SMTP 配置,再由超级管理员在 **系统配置 → 业务邮件通知** 设置真实站点地址、摘要周期,并在协调窗口受控启用;未验邮测试账号再进入 **通知设置** 验邮,已验邮测试账号直接触发新业务事件,验证即时或摘要。平台关闭时不能测试通知验证码或业务邮件;启用期间既有合格订阅用户的新事件也可能发信,并非测试账号隔离。失败则在后台关闭且不盲目重试 UNKNOWN,通过后恢复测试偏好并正常开放。仅轮换 SMTP 凭据不要求全部用户重新验邮,账号邮箱变化后才必须重新验证。SMTP 凭据属于部署配置;平台开关、周期、地址已入库后由后台管理;用户验邮及类别偏好独立管理,不能通过改环境变量替代。 + +旧配置中的空 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST` 会被包含本次修复的一键脚本移除,明确主机例外保留;不关闭证书校验。仍使用旧脚本时应手动删除空覆盖再重建。默认不设置该变量,不能填写 `*`。 + +邮件验收至少包括:真实收到验证码、即时事件收件、同窗口摘要合并、用收件账号打开邮件明细。后台显示配置存在或容器健康,均不等于实际收件成功。升级保留私有配置;已有平台设置不被部署初值覆盖。完整参数、首次配置、轮换、关闭及失败定位见[邮件运维手册](https://github.com/iflytek/skillhub/blob/main/docs/notification-email.md)。 ## 准备配置 -需要 Docker、Docker Compose,以及与目标版本匹配的仓库文件。在仓库根目录执行: +以下为已有源码仓库的部署方式;一键安装实例使用上一节的 Runtime dir 流程。需要 Docker、Docker Compose,以及与目标版本匹配的仓库文件。在仓库根目录执行: ```bash cp .env.release.example .env.release @@ -36,7 +69,7 @@ docker compose --env-file .env.release -f compose.release.yml logs --tail=100 se 校验通过后再启动。首次启动或升级时,后端会执行 Flyway 数据库迁移。访问配置的公开地址,使用自己配置的管理员凭证登录,并验证搜索、下载以及一次符合权限的发布/审核流程;容器健康不代表这些业务路径已通过。 -Suite 审核写入在单机 release Compose 中默认开启;Suite Bundle 确认默认关闭。需要批量导入时,按 [Bundle 导入](suite-bundle.md#部署开关)启用对应开关。滚动升级新旧后端共存时,另按[套装部署顺序](https://github.com/iflytek/skillhub/blob/main/docs/25-skill-suites.md#部署顺序)操作。 +Suite 审核写入在单机 release Compose 中默认开启;Suite Bundle 确认默认关闭。需要批量导入时,按 [Bundle 导入](suite-bundle.html#部署开关)启用对应开关。滚动升级新旧后端共存时,另按[套装部署顺序](https://github.com/iflytek/skillhub/blob/main/docs/25-skill-suites.md#部署顺序)操作。 ## 升级、停止与排障 diff --git a/openspec/changes/add-notification-email-delivery/specs/notification-email-delivery/spec.md b/openspec/changes/add-notification-email-delivery/specs/notification-email-delivery/spec.md index 031113fba..1aefe95ff 100644 --- a/openspec/changes/add-notification-email-delivery/specs/notification-email-delivery/spec.md +++ b/openspec/changes/add-notification-email-delivery/specs/notification-email-delivery/spec.md @@ -115,6 +115,17 @@ IMMEDIATE SHALL 表示不等待摘要窗口,按独立通知实例尽快异步 - **WHEN** 系统升级、SMTP 尚未完整配置,或普通用户直接调用全局设置/投递重试接口 - **THEN** 邮件默认关闭且不阻断站内功能,开启缺配置会明确拒绝,越权操作被拒绝且不修改配置或任务 +#### Scenario: 一键部署保持邮件可选和运行配置持久 +- **WHEN** 部署者首次使用 `runtime.sh up`,或在同一 `--home` 修改 SMTP 配置后再次执行 `up`、`pull`、`ps`、`logs`、`down` +- **THEN** 未配置 SMTP 不新增平台启动前置条件;首次仅在运行文件缺失时复制模板,私有配置权限为 `600` +- **AND** 重新下载 Compose 和模板不得覆盖已有 SMTP 凭据、发件人或用户自定义配置;修改后的 SMTP 在重新创建后端时传入,不能仅靠 restart 生效 +- **AND** 已保存的平台开关、摘要窗口和公开地址仍以数据库设置为准,升级不自动订阅用户,也不覆盖其验邮和类别偏好 + +#### Scenario: 旧模板空信任配置安全迁移 +- **WHEN** 已有一键部署环境包含空的 `SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST`,并运行更新后的部署脚本 +- **THEN** 脚本仅移除该历史空覆盖,保持 JVM 默认的证书验证,不关闭 TLS、不改成通配信任 +- **AND** 明确配置的主机信任例外、含特殊字符的 SMTP 密码和无关配置保持不变,迁移结果不输出秘密,重跑保持幂等 + #### Scenario: SCN-019 通道记录生命周期独立 - **WHEN** 用户删除站内消息、站内清理任务执行、邮件历史清理执行或应用回退 - **THEN** 未完成邮件任务不会被站内删除牵连,邮件历史按自身策略清理,回退关闭能力而不删除数据表或重放旧队列 diff --git a/scripts/runtime.sh b/scripts/runtime.sh index e2afdfad3..0454bee06 100755 --- a/scripts/runtime.sh +++ b/scripts/runtime.sh @@ -231,6 +231,57 @@ secure_env_file() { fi } +# Older examples supplied an empty JavaMail trust list, which rejects SSL SMTP +# hosts instead of selecting the JVM trust store. Preserve every nonempty value +# and all unrelated dotenv text; never evaluate credentials as shell code. +migrate_empty_smtp_trust_override() { + case "${SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST:-}" in + *[![:space:]]*) ;; + *) unset SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST ;; + esac + + migration_umask="$(umask)" + umask 077 + if migration_tmp="$(mktemp "$ENV_FILE.smtp.XXXXXX")"; then + : + else + migration_status=$? + umask "$migration_umask" + return "$migration_status" + fi + if awk ' + function empty_trust(line, value, quote, tail, closing, content, rest) { + if (line !~ /^[[:space:]]*(export[[:space:]]+)?SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST[[:space:]]*=/) + return 0 + value = line + sub(/^[^=]*=/, "", value) + sub(/^[[:space:]]*/, "", value) + if (value ~ /^($|#)/) return 1 + quote = substr(value, 1, 1) + if (quote != "\"" && quote != sprintf("%c", 39)) return 0 + tail = substr(value, 2) + closing = index(tail, quote) + if (!closing) return 0 + content = substr(tail, 1, closing - 1) + rest = substr(tail, closing + 1) + return content ~ /^[[:space:]]*$/ && rest ~ /^[[:space:]]*(#.*)?$/ + } + empty_trust($0) { removed = 1; next } + { print } + END { if (!removed) exit 3 } + ' "$ENV_FILE" >"$migration_tmp"; then + umask "$migration_umask" + mv "$migration_tmp" "$ENV_FILE" + chmod 600 "$ENV_FILE" + echo "Removed empty SMTP SSL trust override; JVM certificate verification remains enabled." + else + migration_status=$? + umask "$migration_umask" + rm -f "$migration_tmp" + [ "$migration_status" -eq 3 ] || return "$migration_status" + fi +} + get_env_value() { key="$1" default_value="${2:-}" @@ -362,6 +413,7 @@ prepare_runtime_files() { umask "$old_umask" fi secure_env_file + migrate_empty_smtp_trust_override if [ -n "$SKILLHUB_MIRROR_REGISTRY_VALUE" ]; then mirror_registry="${SKILLHUB_MIRROR_REGISTRY_VALUE%/}" diff --git a/scripts/tests/runtime-mail-test.sh b/scripts/tests/runtime-mail-test.sh new file mode 100755 index 000000000..097e65b83 --- /dev/null +++ b/scripts/tests/runtime-mail-test.sh @@ -0,0 +1,228 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +python3 - "$ROOT" <<'PY' +import hashlib +import json +import os +import pathlib +import shutil +import stat +import subprocess +import sys +import tempfile + +root = pathlib.Path(sys.argv[1]) +script = pathlib.Path(os.environ.get('RUNTIME_SCRIPT', str(root / 'scripts/runtime.sh'))) +real_docker = shutil.which('docker') +assert real_docker, 'Docker Compose is required for render-only verification' +checks = [] +key = 'SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST' +password = 'fixture$pass # "quoted" value' + +with tempfile.TemporaryDirectory(prefix='skillhub-runtime-mail-') as work: + work = pathlib.Path(work) + bin_dir = work / 'bin' + bin_dir.mkdir() + docker = bin_dir / 'docker' + docker.write_text('''#!/usr/bin/env python3 +import hashlib,json,os,pathlib,subprocess,sys +args=sys.argv[1:] +if args==['compose','version']:raise SystemExit(0) +assert args[0]=='compose' +flags=args[1:] +config=[] +while flags and flags[0] in ['--env-file','-f']: + config.extend(flags[:2]);flags=flags[2:] +operation=flags[0] +render=json.loads(subprocess.check_output([os.environ['REAL_DOCKER'],'compose',*config,'config','--format','json'],stderr=subprocess.PIPE)) +server=render['services']['server']['environment'] +# Compose config serializes dollars as $$ so its output can be reused as input. +# Decode that serialization once; the dotenv source itself is checked byte-for-byte. +password=(server.get('SPRING_MAIL_PASSWORD') or '').replace('$$','$') +row={'operation':operation,'smtpHost':server.get('SPRING_MAIL_HOST'),'smtpPort':server.get('SPRING_MAIL_PORT'),'smtpSsl':server.get('SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_ENABLE'),'smtpTrust':server.get('SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_TRUST'),'fromAddress':server.get('SKILLHUB_NOTIFICATION_EMAIL_FROM_ADDRESS'),'mailEnabled':server.get('SKILLHUB_NOTIFICATION_EMAIL_ENABLED'),'passwordHash':hashlib.sha256(password.encode()).hexdigest()} +with open(os.environ['DOCKER_LOG'],'a') as f:f.write(json.dumps(row)+'\\n') +if operation==os.environ.get('DOCKER_FAIL_ON'):raise SystemExit(55) +''') + docker.chmod(0o755) + curl = bin_dir / 'curl' + curl.write_text('''#!/usr/bin/env python3 +import os,pathlib,shutil,sys,urllib.parse +args=sys.argv[1:];src=args[args.index('-o')-1];dest=args[args.index('-o')+1] +source=pathlib.Path(urllib.parse.unquote(urllib.parse.urlparse(src).path)) +if source.name==os.environ.get('DOWNLOAD_FAIL_BASENAME'): + print('fixture download failure',file=sys.stderr);raise SystemExit(22) +shutil.copyfile(source,dest) +''') + curl.chmod(0o755) + awk = bin_dir / 'awk' + awk.write_text('''#!/usr/bin/env python3 +import os,stat,sys +assert stat.S_IMODE(os.fstat(sys.stdout.fileno()).st_mode)==0o600, 'migration temporary file must be exclusive and private' +os.execv(os.environ['REAL_AWK'], [os.environ['REAL_AWK'], *sys.argv[1:]]) +''') + awk.chmod(0o755) + mktemp = bin_dir / 'mktemp' + mktemp.write_text('''#!/usr/bin/env python3 +import os,sys +if os.environ.get('FAIL_MKTEMP'):raise SystemExit(73) +os.execv(os.environ['REAL_MKTEMP'], [os.environ['REAL_MKTEMP'], *sys.argv[1:]]) +''') + mktemp.chmod(0o755) + env = {**os.environ, 'PATH': str(bin_dir) + os.pathsep + os.environ['PATH'], + 'SKILLHUB_RAW_BASE': root.as_uri(), 'REAL_DOCKER': real_docker, + 'REAL_AWK': shutil.which('awk'), 'REAL_MKTEMP': shutil.which('mktemp')} + for inherited in list(env): + if inherited.startswith(('SPRING_MAIL_', 'SKILLHUB_NOTIFICATION_EMAIL_', 'POSTGRES_')) or inherited in ('SKILLHUB_HOME', 'SKILLHUB_VERSION', 'SKILLHUB_PUBLIC_BASE_URL'): + env.pop(inherited, None) + # The test never invokes a real lifecycle operation: only real Compose config. + def run(home, command='up', extra=None, success=True): + home.mkdir(exist_ok=True) + log = home / 'calls.jsonl' + current = {**env, 'DOCKER_LOG': str(log), **(extra or {})} + result = subprocess.run(['sh', str(script), command, '--home', str(home)], env=current, + text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + assert (result.returncode == 0) is success, f'{command} unexpected exit {result.returncode}' + assert password not in result.stdout + result.stderr, 'runtime output exposed fixture password' + return result, [json.loads(line) for line in log.read_text().splitlines()] if log.exists() else [] + def fixture(home, trust=None): + home.mkdir(exist_ok=True) + text = '\n'.join([ + 'SKILLHUB_DOWNLOAD_ANON_COOKIE_SECRET=fixture-existing-cookie-secret-at-least-32-bytes', + 'POSTGRES_PASSWORD=fixture-database-secret', + 'SPRING_MAIL_HOST=smtp.example.test', 'SPRING_MAIL_PORT=465', + "SPRING_MAIL_PASSWORD='" + password + "'", 'SPRING_MAIL_USERNAME=sender@example.test', + 'SPRING_MAIL_SMTP_AUTH=true', 'SPRING_MAIL_SMTP_STARTTLS_ENABLE=false', + 'SPRING_MAIL_PROPERTIES_MAIL_SMTP_SSL_ENABLE=true', + 'SKILLHUB_NOTIFICATION_EMAIL_FROM_ADDRESS=sender@example.test', + 'SKILLHUB_NOTIFICATION_EMAIL_ENABLED=false', + 'SKILLHUB_NOTIFICATION_EMAIL_DIGEST_WINDOW_MINUTES=15', + 'SKILLHUB_NOTIFICATION_EMAIL_PUBLIC_BASE_URL=https://skills.example.test/skillhub', + 'SKILLHUB_PUBLIC_BASE_URL=https://skills.example.test/skillhub', + 'SKILLHUB_VERSION=fixture-pinned-version', + ]) + '\n' + if trust is not None: + text += key + '=' + trust + '\n' + (home / '.env.release').write_text(text) + return text + def configured(rows): + assert rows, 'runtime never reached the Compose contract' + row = rows[-1] + assert row['smtpHost'] == 'smtp.example.test' + assert str(row['smtpPort']) == '465' + assert str(row['smtpSsl']).lower() == 'true' + assert row['fromAddress'] == 'sender@example.test' + assert row['passwordHash'] == hashlib.sha256(password.encode()).hexdigest(), 'fixture SMTP password changed during Compose rendering' + assert str(row['mailEnabled']).lower() == 'false' + first = work / 'first install' + _, calls = run(first) + assert stat.S_IMODE((first / '.env.release').stat().st_mode) == 0o600 + assert not calls[-1]['smtpHost'] + assert str(calls[-1]['mailEnabled']).lower() == 'false' + assert calls[-1]['smtpTrust'] is None + checks.append('fresh install without SMTP remains optional and private') + + home = work / 'configured' + expected = fixture(home) + for command in ['up', 'pull', 'down', 'ps', 'logs']: + _, calls = run(home, command) + assert (home / '.env.release').read_text() == expected + assert stat.S_IMODE((home / '.env.release').stat().st_mode) == 0o600 + configured(calls) + checks.append('up/pull/down/ps/logs preserve SMTP secrets, sender, bootstrap policy, public URL and pinned version') + + for index, value in enumerate(['', ' ', '""', "''", '" "', "' '", ' # old comment', '"" # old comment']): + legacy = work / f'legacy-{index}' + expected = fixture(legacy) + with (legacy / '.env.release').open('a') as f:f.write(key + '=' + value + '\n') + _, calls = run(legacy) + assert (legacy / '.env.release').read_text() == expected + assert calls[-1]['smtpTrust'] is None + configured(calls) + assert not (legacy / '.env.release.smtp-tmp').exists() + checks.append('legacy empty/whitespace/quoted/commented trust overrides migrate without changing other values') + + for index, value in enumerate(['smtp.example.test', '"smtp.example.test" # approved exception', "'smtp.example.test'"]): + explicit = work / f'explicit-{index}' + expected = fixture(explicit, value) + _, calls = run(explicit) + assert (explicit / '.env.release').read_text() == expected + assert calls[-1]['smtpTrust'] == 'smtp.example.test' + checks.append('unquoted and quoted explicit trust hosts are preserved byte-for-byte') + + duplicate = work / 'duplicate-trust' + fixture(duplicate, 'smtp.example.test') + with (duplicate / '.env.release').open('a') as f:f.write(' export ' + key + ' = "" # legacy\n') + _, calls = run(duplicate) + assert calls[-1]['smtpTrust'] == 'smtp.example.test' + assert key + '=smtp.example.test' in (duplicate / '.env.release').read_text() + checks.append('explicit nonempty trust host remains intact when an empty duplicate is removed') + + for target_type in ['file', 'symlink']: + protected = work / f'preexisting-temp-{target_type}' + expected = fixture(protected) + with (protected / '.env.release').open('a') as f:f.write(key + '=\n') + stale = protected / '.env.release.smtp-tmp' + victim = protected / 'unrelated-file' + victim.write_text('unrelated marker\n') + victim.chmod(0o644) + if target_type == 'file': + stale.write_text('existing temporary marker\n') + stale.chmod(0o644) + else: + stale.symlink_to(victim) + _, calls = run(protected) + assert (protected / '.env.release').read_text() == expected + assert stat.S_IMODE((protected / '.env.release').stat().st_mode) == 0o600 + assert victim.read_text() == 'unrelated marker\n' + assert stat.S_IMODE(victim.stat().st_mode) == 0o644 + if target_type == 'file': + assert stale.read_text() == 'existing temporary marker\n' + assert stat.S_IMODE(stale.stat().st_mode) == 0o644 + else: + assert stale.is_symlink() + assert not list(protected.glob('.env.release.smtp.*')), 'private temporary file was not removed' + configured(calls) + checks.append('exclusive private temporary files do not overwrite existing broad-permission files or symlinks') + + shell_empty = work / 'shell-empty' + fixture(shell_empty) + _, calls = run(shell_empty, extra={key: ' '}) + assert calls[-1]['smtpTrust'] is None + _, calls = run(shell_empty, extra={key: 'smtp.shell.example.test'}) + assert calls[-1]['smtpTrust'] == 'smtp.shell.example.test' + checks.append('blank inherited shell trust is omitted and explicit inherited host is preserved') + + failed_temp = work / 'failed-temp' + original = fixture(failed_temp, '') + _, calls = run(failed_temp, extra={'FAIL_MKTEMP': '1'}, success=False) + assert not calls, 'temporary-file failure reached Docker' + assert (failed_temp / '.env.release').read_text() == original + assert not list(failed_temp.glob('.env.release.smtp.*')) + run(failed_temp) + checks.append('temporary-file allocation failure leaves original configuration intact and supports retry') + + failed_download = work / 'failed-download' + original = fixture(failed_download, '') + for fail in ['compose.release.yml', '.env.release.example']: + log = failed_download / 'calls.jsonl' + if log.exists():log.unlink() + _, calls = run(failed_download, 'up', extra={'DOWNLOAD_FAIL_BASENAME': fail}, success=False) + assert not calls, 'download failure reached Docker' + assert (failed_download / '.env.release').read_text() == original + run(failed_download, 'up') + assert key + '=' not in (failed_download / '.env.release').read_text() + checks.append('download failure leaves SMTP config untouched and a successful retry completes migration') + + failed_docker = work / 'failed-docker' + expected = fixture(failed_docker) + for command, operation in [('up', 'up'), ('pull', 'pull'), ('down', 'down')]: + run(failed_docker, command, extra={'DOCKER_FAIL_ON': operation}, success=False) + assert (failed_docker / '.env.release').read_text() == expected + run(failed_docker, command) + assert (failed_docker / '.env.release').read_text() == expected + checks.append('Docker up/pull/down failures propagate and retry preserves the same SMTP configuration') + +print(json.dumps({'status': 'passed', 'checks': checks, 'real_compose_render': True, + 'real_services_started': False, 'real_smtp_used': False}, indent=2)) +PY