docs(operations): explain that reindex cannot refresh stale chunks (#585)

* docs(operations): explain that reindex cannot refresh stale chunks

#524 reports that persisted chunks survive a chunker change and are never
re-chunked. Maintainers closed #528 with the position that automatic
invalidation stays out of scope and the recovery path is to clear derived
state and let the watchers rescan, but that path was only described in the
issue thread. The operations guide already warned that reindex does not
rechunk; add the missing half: reindex reuses stored chunks as-is, so a
chunker change requires the re-chunk recovery instead.

* docs(operations): narrow recovery scope and stop the service before cleanup

Review on #585: moving the whole metadata/ tree makes the resource
watcher treat existing resources as newly added, which can update daily
notes - so 'source files are never modified' was too broad, and the
cleanup should target only the affected store's derived state while
preserving unrelated catalogs. The recovery sequence now also stops the
service before moving anything, so shutdown persistence cannot write the
old in-memory state back over the cleanup.

The reindex paragraph drops the repeated 'does not rechunk' explanation
and focuses on when a rebuild from source is needed. English and
Chinese pages stay symmetric.

* docs(operations): simplify rechunk recovery guidance

---------

Co-authored-by: jinli.yl <jinli.yl@alibaba-inc.com>
This commit is contained in:
Ziyang Guo 2026-10-15 22:17:01 +08:00 • committed by GitHub
parent 32e1f7faad
commit b616944265
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 12 additions and 6 deletions

View file

@ -49,6 +49,8 @@ reme reindex scope=embedding
`reindex` rebuilds BM25 and/or embedding indexes from the current `file_chunks`. It does not scan the workspace, rechunk files, or rebuild the wikilink graph. Diagnose the watcher first when ingestion is the problem.
When a ReMe upgrade or chunker configuration change alters chunking behavior, existing chunks do not refresh automatically. To apply the new behavior, follow [Recover derived state](#recover-derived-state).
Rebuild a daily index page separately:
```bash
@ -90,9 +92,10 @@ Do not remove anything until a backup exists. Then:
1. preserve `session/`, `resource/`, `daily/`, and `digest/`;
2. record the effective configuration and component backends;
3. verify that the failure is limited to `metadata/`;
4. move suspect derived state to an isolated backup location;
5. restart with the same configuration and let watchers rebuild;
6. validate search, graph traversal, and daily indexes.
4. stop the service, so shutdown persistence cannot write the old in-memory state back over the cleanup;
5. move only the affected store's derived state to an isolated backup location, preserving unrelated catalogs;
6. restart with the same configuration and let watchers rebuild the files in their configured watch scope;
7. validate search, graph traversal, and daily indexes.
The internal layout of metadata files is not a public automation contract.

View file

@ -52,6 +52,8 @@ reme reindex scope=embedding
如果问题发生在文件摄取阶段,应先确认后台 watcher 正常运行;不能把 `reindex` 当成通用“重新扫描”命令。
升级 ReMe 或修改 chunker 配置导致分块规则变化后,已有分块不会自动刷新。如需应用新规则,请按[从派生状态故障恢复](#从派生状态故障恢复)流程重建。
Daily 索引页可单独重建:
```bash
@ -93,9 +95,10 @@ reme start workspace_dir=/new/location/reme-memory
1. 保留 `session/`、`resource/`、`daily/` 和 `digest/`;
2. 记录当前配置和组件 backend;
3. 确认故障只发生在 `metadata/`;
4. 将有问题的派生状态移到隔离备份位置;
5. 用同一配置启动 ReMe,让 watcher 从源文件重建;
6. 验证搜索、图谱和 Daily 索引。
4. 先停止服务,避免关闭时的持久化把旧的内存状态写回、覆盖刚做的清理;
5. 只把受影响存储的派生状态移到隔离备份位置,保留无关 catalog 状态;
6. 用同一配置启动 ReMe,让 watcher 重建其配置覆盖的文件;
7. 验证搜索、图谱和 Daily 索引。
具体 metadata 文件属于实现细节,不应在自动化脚本里依赖其内部格式。