mirror of
https://github.com/HKUDS/OpenSpace.git
synced 2026-08-28 05:15:00 +00:00
Document recommended local embedding routing
This commit is contained in:
parent
e815622872
commit
8834a75b08
2 changed files with 32 additions and 0 deletions
16
README.md
16
README.md
|
|
@ -157,6 +157,22 @@ pip install -e .
|
|||
openspace-mcp --help # verify installation
|
||||
```
|
||||
|
||||
> [!TIP]
|
||||
> **Recommended split routing for OpenAI-compatible gateways**
|
||||
>
|
||||
> If your main model runs through an OpenAI-compatible provider or local relay (for example `gpt-5.4` via `http://127.0.0.1:8080/v1`), the recommended default is:
|
||||
> - keep the main LLM on that provider via `OPENSPACE_LLM_*`
|
||||
> - keep skill-router embeddings local via `OPENSPACE_SKILL_EMBEDDING_BACKEND=local`
|
||||
> - use `OPENSPACE_SKILL_EMBEDDING_MODEL=BAAI/bge-small-en-v1.5`
|
||||
>
|
||||
> Why this is the default recommendation:
|
||||
> - lower latency for routing and prefilter
|
||||
> - no dependence on a remote `/v1/embeddings` endpoint
|
||||
> - no extra token spend for embedding generation
|
||||
> - stronger main LLM still handles final reasoning and selection
|
||||
>
|
||||
> Architecture notes and flow diagram: [`docs/current-routing-flow.md`](docs/current-routing-flow.md)
|
||||
|
||||
> [!TIP]
|
||||
> **Slow clone?** The `assets/` folder (~50 MB of images) makes the default clone large. Use this lightweight alternative to skip it:
|
||||
> ```bash
|
||||
|
|
|
|||
16
README_CN.md
16
README_CN.md
|
|
@ -157,6 +157,22 @@ pip install -e .
|
|||
openspace-mcp --help # 验证安装
|
||||
```
|
||||
|
||||
> [!TIP]
|
||||
> **OpenAI 兼容网关下,默认推荐双路由方案 A**
|
||||
>
|
||||
> 如果你的主模型走的是 OpenAI 兼容 provider 或本地 relay(例如 `gpt-5.4` 走 `http://127.0.0.1:8080/v1`),当前最推荐的默认配置是:
|
||||
> - 主 LLM 继续走 `OPENSPACE_LLM_*`
|
||||
> - skill router 的 embedding 走本地:`OPENSPACE_SKILL_EMBEDDING_BACKEND=local`
|
||||
> - 本地 embedding 模型使用:`OPENSPACE_SKILL_EMBEDDING_MODEL=BAAI/bge-small-en-v1.5`
|
||||
>
|
||||
> 这样选的原因:
|
||||
> - 路由和预筛选延迟更低
|
||||
> - 不依赖远程 `/v1/embeddings`
|
||||
> - embedding 不额外消耗 provider token
|
||||
> - 更强的主 LLM 仍然负责最终推理和选择
|
||||
>
|
||||
> 架构说明和流程图见:[`docs/current-routing-flow.md`](docs/current-routing-flow.md)
|
||||
|
||||
> [!TIP]
|
||||
> **Clone 太慢?** `assets/` 目录包含约 50 MB 的图片文件,导致仓库较大。使用以下轻量方式跳过它:
|
||||
> ```bash
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue