mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-11-14 09:01:22 +00:00
fix(dsh): keep the saved backends and the probe in step with ReMe
Two ways the plugin could write or read a backend the service cannot use:
The embedding backend was left out of the default rule, because this
repository's own configuration has its embedding components commented out.
That only holds for the version in this repository: ReMe 0.4.0.x - the
oldest the plugin supports - declares as_embedding with
"backend: ${EMBEDDING_BACKEND:-openai}" and instantiates it, so an
explicitly empty value names no backend and the service refuses to start,
exactly like the LLM backend did. EMBEDDING_BACKEND now writes its default
out as well.
A known backend written in the wrong case was folded to the default before
the probe ran, so the probe tested a configuration ReMe's case-sensitive
registry refuses ("Unregistered backend 'OpenAI'"). The probe now reports
the spelling to use and sends no request. A name the plugin has never heard
of is still probed tolerantly: its list is a snapshot, and ReMe may know
more than the plugin does.
This commit is contained in:
parent
b16914f91a
commit
e9e572cddd
6 changed files with 113 additions and 10 deletions
|
|
@ -104,9 +104,10 @@ DSH settings.
|
|||
cleared: a credential cannot be deleted by emptying its control, so the blank field always means “keep what is
|
||||
stored”. Emptying a **non-secret** field and saving **clears** it. Only fields you edited are written, so the rest
|
||||
of the file keeps its comments, unknown keys and order.
|
||||
- The backend selection is the one non-secret field with a default: clearing it writes the default
|
||||
(`LLM_BACKEND=openai`) instead of an empty value. ReMe applies `${LLM_BACKEND:-openai}` only when the variable is
|
||||
absent, so an empty value would name no backend at all and the service would refuse to start.
|
||||
- The two backend selections are the non-secret fields with a default: clearing one writes the default
|
||||
(`LLM_BACKEND=openai`, `EMBEDDING_BACKEND=openai`) instead of an empty value. ReMe applies
|
||||
`${LLM_BACKEND:-openai}` and `${EMBEDDING_BACKEND:-openai}` only when the variable is absent, so an empty
|
||||
value would name no backend at all and the service would refuse to start.
|
||||
- A save writes the edits that were submitted and leaves newer ones alone: typing another field while the save is in
|
||||
flight keeps that text, and the card says that unsaved edits are still pending.
|
||||
- Saving validates the whole merged file, not just the values you typed: ReMe 0.4.0.x instantiates the embedding
|
||||
|
|
@ -133,6 +134,9 @@ DSH settings.
|
|||
(`LLM config incomplete (LLM_BACKEND)`) and no request is sent: ReMe would start with no backend at all, so the
|
||||
probe must not test a configuration the service cannot use. A `.env` without the key is probed with the default
|
||||
backend, exactly as ReMe would start it.
|
||||
- A known backend written in the wrong case (`LLM_BACKEND=OpenAI`) is reported as invalid, with the spelling to
|
||||
use: ReMe's backend registry is case-sensitive and refuses that name. A backend name the plugin has never heard
|
||||
of is still probed tolerantly — the list here is a snapshot, and ReMe may know more than the plugin does.
|
||||
- The draft, the save message and the LLM-test result belong to the workspace they were produced for: switching
|
||||
the workspace disposes the section's state, so neither a key typed for one workspace nor a success reported for
|
||||
it can be carried into another.
|
||||
|
|
|
|||
|
|
@ -84,8 +84,9 @@ ReMe 在服务启动时读取 workspace 目录下的 `.env`;标签页从不把
|
|||
- 清空**密钥**字段并保存会保留已存的密钥——即便先输入过替代值再清空也是如此:凭据无法通过清空输入框来删除,
|
||||
因此留空始终表示“保留已存的值”。清空**非密钥**字段并保存即表示**删除**该值。只有编辑过的字段会被写入,
|
||||
文件其余部分的注释、未知键与顺序保持不变。
|
||||
- 后端选择是唯一带默认值的非密钥字段:清空它会写入默认值(`LLM_BACKEND=openai`),而不是空值。ReMe 只在变量
|
||||
**不存在**时才应用 `${LLM_BACKEND:-openai}`,空值等于没有指定任何后端,服务将无法启动。
|
||||
- 两个后端选择是带默认值的非密钥字段:清空后会写入默认值(`LLM_BACKEND=openai`、`EMBEDDING_BACKEND=openai`),
|
||||
而不是空值。ReMe 只在变量**不存在**时才应用 `${LLM_BACKEND:-openai}` 与 `${EMBEDDING_BACKEND:-openai}`,
|
||||
空值等于没有指定任何后端,服务将无法启动。
|
||||
- 保存只清除本次提交的那部分编辑,更晚的编辑会被保留:保存进行中再改动其它字段时,那份文本不会丢失,页面也会
|
||||
提示仍有未保存的改动。
|
||||
- 保存时会校验合并后的整份文件,而不只是本次输入:ReMe 0.4.0.x 会无条件实例化 embedding 组件,因此即使不使用向量检索
|
||||
|
|
@ -106,6 +107,8 @@ ReMe 在服务启动时读取 workspace 目录下的 `.env`;标签页从不把
|
|||
- 若 `.env` 把 `LLM_BACKEND` 设为空值,探测会报告配置不完整(`LLM config incomplete (LLM_BACKEND)`)并且不发送
|
||||
任何请求:此时 ReMe 启动时并没有任何后端,探测不能去测试服务根本无法使用的配置。若 `.env` 里没有这个键,
|
||||
探测会与 ReMe 一样使用默认后端。
|
||||
- 已知后端名写错了大小写(如 `LLM_BACKEND=OpenAI`)会被报告为无效,并给出正确写法:ReMe 的后端注册表区分大小写,
|
||||
会拒绝这个名字。而插件从未听说过的后端名仍被宽容地探测——这里的列表只是一份快照,ReMe 可能认识得更多。
|
||||
- 草稿、保存提示与 LLM 测试结果都属于产生它们的 workspace:切换 workspace 会随旧区块一并丢弃,因此为某个 workspace
|
||||
输入的密钥、或属于它的一次成功提示,都不会被带到另一个。
|
||||
|
||||
|
|
|
|||
|
|
@ -62,7 +62,15 @@ export const ENV_FIELDS: readonly EnvFieldMeta[] = [
|
|||
{ key: "LLM_MODEL_NAME", secret: false },
|
||||
{ key: "LLM_API_KEY", secret: true },
|
||||
{ key: "LLM_BASE_URL", secret: false },
|
||||
{ key: "EMBEDDING_BACKEND", secret: false },
|
||||
{
|
||||
key: "EMBEDDING_BACKEND",
|
||||
secret: false,
|
||||
// ReMe 0.4.0.x declares `as_embedding` with
|
||||
// `backend: ${EMBEDDING_BACKEND:-openai}` and instantiates it, so the
|
||||
// same rule holds here: the default is written out, because an empty
|
||||
// value names no backend and the service then refuses to start.
|
||||
defaultValue: "openai",
|
||||
},
|
||||
{ key: "EMBEDDING_MODEL_NAME", secret: false },
|
||||
{
|
||||
key: "EMBEDDING_API_KEY",
|
||||
|
|
|
|||
|
|
@ -290,6 +290,21 @@ export function llmProtocolFor(backend: string): LlmProtocol {
|
|||
return LLM_PROTOCOLS[llmBackendOf(backend)] ?? OPENAI_PROTOCOL;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the file names a backend in a spelling ReMe will not have.
|
||||
*
|
||||
* ReMe's backend registry is case-sensitive, so a name that differs from a
|
||||
* known backend only in case is a misspelling the service refuses
|
||||
* ("Unregistered backend"). A name the plugin has never heard of is left
|
||||
* alone: its list is a snapshot, and ReMe may know more than the plugin does.
|
||||
*/
|
||||
export function backendSpellingIsWrong(name: string): boolean {
|
||||
return (
|
||||
LLM_PROTOCOLS[name] === undefined &&
|
||||
LLM_PROTOCOLS[name.toLowerCase()] !== undefined
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The probe program for one provider, run by the selected interpreter inside
|
||||
* the workspace so it reads the same `.env` the service would discover. The
|
||||
|
|
@ -423,6 +438,16 @@ export async function testLlm(
|
|||
latencyMs: 0,
|
||||
error: "LLM config incomplete (LLM_BACKEND)",
|
||||
};
|
||||
// A known backend written in the wrong case is refused by ReMe's
|
||||
// case-sensitive registry, so probing it would test a configuration the
|
||||
// service cannot use.
|
||||
if (backend !== undefined && backendSpellingIsWrong(backend))
|
||||
return {
|
||||
ok: false,
|
||||
status: 0,
|
||||
latencyMs: 0,
|
||||
error: `LLM config invalid (LLM_BACKEND must be written in lower case: ${backend.toLowerCase()})`,
|
||||
};
|
||||
const protocol = llmProtocolFor(backend ?? DEFAULT_LLM_BACKEND);
|
||||
try {
|
||||
const result = await io.run(
|
||||
|
|
|
|||
|
|
@ -169,10 +169,19 @@ test("the default rule does not touch the other keys", () => {
|
|||
assert.deepEqual(cleared, { LLM_BASE_URL: "" });
|
||||
});
|
||||
|
||||
test("only the backend names a default", () => {
|
||||
// The embedding backend has no default: ReMe's embedding components are
|
||||
// commented out, so the plugin must not invent a rule for it.
|
||||
test("both backends name a default", () => {
|
||||
// ReMe 0.4.0.x declares `as_embedding` with
|
||||
// `backend: ${EMBEDDING_BACKEND:-openai}` and instantiates it, so the
|
||||
// embedding backend follows the same rule as the LLM one.
|
||||
assert.equal(envFieldMeta("LLM_BACKEND")?.defaultValue, "openai");
|
||||
assert.equal(envFieldMeta("EMBEDDING_BACKEND")?.defaultValue, undefined);
|
||||
assert.equal(envFieldMeta("EMBEDDING_BACKEND")?.defaultValue, "openai");
|
||||
assert.equal(envFieldMeta("LLM_MODEL_NAME")?.defaultValue, undefined);
|
||||
assert.equal(envFieldMeta("LLM_BASE_URL")?.defaultValue, undefined);
|
||||
});
|
||||
|
||||
test("a cleared embedding backend is written as the default", () => {
|
||||
// The same failure as the LLM backend: an empty value names no backend, and
|
||||
// ReMe refuses to start with one.
|
||||
const updates = envUpdates({ EMBEDDING_BACKEND: "" }, STORED);
|
||||
assert.deepEqual(updates, { EMBEDDING_BACKEND: "openai" });
|
||||
});
|
||||
|
|
|
|||
|
|
@ -434,6 +434,25 @@ test("a cleared backend is saved as the default, never as an empty value", (t) =
|
|||
assert.match(text, /^EMBEDDING_API_KEY=EMPTY$/m);
|
||||
});
|
||||
|
||||
test("a cleared embedding backend is saved as the default", (t) => {
|
||||
// ReMe 0.4.0.x declares `as_embedding` with
|
||||
// `backend: ${EMBEDDING_BACKEND:-openai}` and instantiates it, so an empty
|
||||
// value names no backend there either.
|
||||
const { manager, directory } = managerIn(t);
|
||||
const envPath = join(directory, ".env");
|
||||
writeFileSync(
|
||||
envPath,
|
||||
"EMBEDDING_BACKEND=gemini\nLLM_API_KEY=sk-keep\nEMBEDDING_API_KEY=EMPTY\n",
|
||||
);
|
||||
|
||||
manager.writeEnv(directory, { EMBEDDING_BACKEND: "" });
|
||||
|
||||
const text = readFileSync(envPath, "utf8");
|
||||
assert.match(text, /^EMBEDDING_BACKEND=openai$/m);
|
||||
assert.doesNotMatch(text, /^EMBEDDING_BACKEND=$/m);
|
||||
assert.match(text, /^LLM_API_KEY=sk-keep$/m);
|
||||
});
|
||||
|
||||
test("the probe refuses a .env whose backend is empty", async (t) => {
|
||||
// An explicitly empty backend names no backend, and ReMe would start with
|
||||
// none at all. The honest answer is an incomplete configuration, and no
|
||||
|
|
@ -472,6 +491,41 @@ test("the probe refuses a .env whose backend is empty", async (t) => {
|
|||
assert.equal(calls.length, 0, "the snippet is not run");
|
||||
});
|
||||
|
||||
test("the probe refuses a known backend written in the wrong case", async (t) => {
|
||||
// ReMe's backend registry is case-sensitive, so "OpenAI" is refused as an
|
||||
// unregistered backend. Probing it would test a configuration the service
|
||||
// cannot use.
|
||||
const directory = mkdtempSync(join(tmpdir(), "reme-llm-case-"));
|
||||
t.after(() => rmSync(directory, { recursive: true, force: true }));
|
||||
writeFileSync(
|
||||
join(directory, ".env"),
|
||||
[
|
||||
"LLM_BACKEND=OpenAI",
|
||||
"LLM_MODEL_NAME=test-model",
|
||||
"LLM_API_KEY=sk-test",
|
||||
"LLM_BASE_URL=https://provider.invalid",
|
||||
].join("\n"),
|
||||
);
|
||||
const calls = [];
|
||||
const result = await testLlm(
|
||||
{
|
||||
run: async (exe, args, options) => {
|
||||
calls.push({ exe, args, options });
|
||||
return {
|
||||
code: 0,
|
||||
stdout: '{"ok": true, "status": 200, "latencyMs": 12, "error": ""}\n',
|
||||
stderr: "",
|
||||
};
|
||||
},
|
||||
},
|
||||
{ workspaceDir: directory, pythonPath: "C:\\env\\python.exe" },
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.error, /LLM_BACKEND/);
|
||||
assert.match(result.error, /openai/, "the message names the spelling to use");
|
||||
assert.equal(calls.length, 0, "the snippet is not run");
|
||||
});
|
||||
|
||||
test("a .env without the backend key is probed with the default protocol", async (t) => {
|
||||
// The key is absent, so ReMe applies its default; the probe does the same.
|
||||
const directory = mkdtempSync(join(tmpdir(), "reme-llm-absent-"));
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue