Added script

This commit is contained in:
IanBarrow 2026-08-25 16:18:07 -04:00
parent 913efb1895
commit ab18e4c55d
5 changed files with 414 additions and 4 deletions

View file

@ -6,7 +6,7 @@ services:
image: open-webui-claude-cli:local
depends_on: !reset []
environment:
OLLAMA_BASE_URL: http://host.docker.internal:11434
OLLAMA_BASE_URL: http://ollama:11434
# Fixed, not '' (auto-generate-on-first-boot): the auto-generated key
# lives in the container's own writable layer
# (/app/backend/.webui_secret_key), *outside* the persistent

View file

@ -91,11 +91,14 @@ docs/ichirouganaim-integration/bootstrap_admin_api_key.py
docs/ichirouganaim-integration/bootstrap.sh
```
The five scripts referenced later, in "Long-term operation," are optional
but worth having too — not required for a working setup, only for ongoing
maintenance once it's been running a while:
The scripts referenced later, in step 8/8b and "Long-term operation," are
optional but worth having too — not required for a working setup, only
for wiring up MCP access and ongoing maintenance once it's been running a
while:
```
docs/ichirouganaim-integration/configure_mcp.sh
docs/ichirouganaim-integration/register_mcp_tool_server.sh
docs/ichirouganaim-integration/check_claude_auth.sh
docs/ichirouganaim-integration/backup_data_volume.sh
docs/ichirouganaim-integration/check_claude_code_version.sh
@ -323,6 +326,65 @@ curl -s -X POST $OPEN_WEBUI_BASE_URL/api/v1/functions/id/claude_cli/valves/updat
-d '{"MCP_SERVER_URL":"<your MCP URL>"}'
```
**Or, packaged as a script** (does the same reachability check plus a
readback confirming the write actually took, rather than trusting "the
API call didn't error"):
```bash
docs/ichirouganaim-integration/configure_mcp.sh <your MCP URL>
docs/ichirouganaim-integration/configure_mcp.sh "" # clears it, disables MCP for claude_cli
```
## 8b. (Optional) Register the MCP server for *every* model, not just `claude_cli`
Step 8 above only wires the MCP server into `claude_cli` specifically —
the `claude` CLI subprocess connects to it directly, bypassing this
fork's own MCP client entirely. If you also want native (non-`claude_cli`)
models in this instance to be able to use the same MCP server, that's a
genuinely different, independent mechanism: this fork's built-in **Tool
Server** support
(`backend/open_webui/routers/configs.py`'s `/api/v1/configs/tool_servers`
endpoints). The two don't conflict — the MCP server ends up with two
separate client connections into it, which any real MCP server is
designed to handle.
```bash
docs/ichirouganaim-integration/register_mcp_tool_server.sh \
--id ichirouganaim_mcp \
--url <your MCP URL> \
--name "Ichirouganaim MCP" \
[--public]
```
Verifies the MCP server actually responds (via a real handshake, no
Claude usage spent) before saving, and re-running with the same `--id`
updates the existing entry in place rather than duplicating it. Without
`--public`, the registered connection defaults to **admin-only** access
(confirmed by reading `has_connection_access` directly — no
`access_grants` configured means only admins can use it); `--public`
grants read access to every user via the exact grant shape `has_access`'s
own docstring documents for that.
**Registering it makes it *available*, not automatically used by every
model.** A chat still needs `tool_ids` containing
`"server:mcp:<your-id>"` for that specific request to actually connect
and call it. **Confirmed live (frontend source, not assumed): there's
currently no way to make a model auto-use a Tool Server by default at
all** — a model's own edit page (`Workspace → Models → edit`
`ToolsSelector.svelte`) only lets you pick from the internal Tools
registry (`$lib/apis/tools`, a separate, older mechanism — individually
registered Python function tools, not Tool Server connections), and
doesn't reference `tool_server.connections` at all. The *only* way to
enable a registered Tool Server for a conversation is per-chat: the "+"
tools icon in the message input toolbar, which opens
`ToolServersModal.svelte` listing both internal Tools and Tool Servers —
select it there, every time, for every chat that needs it.
**Where to check it's registered**: Admin Settings (gear icon) →
**Integrations** tab (labeled "External Tool Servers" in the UI,
`Integrations.svelte`) — the connection this script created should be
listed there, editable/removable the same as one added by hand.
## 9. Verify end to end
Plain chat:

View file

@ -0,0 +1,88 @@
#!/usr/bin/env bash
# Reliably wires (or re-wires) the claude_cli Pipe's MCP_SERVER_URL Valve
# -- the one config change that activates MCP tool access (SETUP.md step
# 8). Packages that step as a script instead of hand-typed curl commands,
# so it's the same repeatable operation whether it's the first time this
# is set up, or re-confirming it after a container recreation, a Valve
# accidentally getting cleared, or moving to a new machine.
#
# Verifies reachability *from inside the container* before setting
# anything -- SETUP.md's own step 8 warns not to assume that based on it
# working from the host, since they're genuinely different network paths
# (confirmed live earlier this session: a URL reachable from the host can
# still be unreachable from inside the container's own network namespace).
# Then sets the Valve, reads it back to confirm the write actually took,
# and reports the final state -- never trusts "the API call didn't error"
# alone as proof it worked.
#
# Usage:
# export OPEN_WEBUI_API_KEY=sk-...
# ./configure_mcp.sh http://host.docker.internal:8931/mcp
# ./configure_mcp.sh "" # clear it -- disables MCP tool access
# OPEN_WEBUI_BASE_URL=http://localhost:8080 ./configure_mcp.sh <url>
#
# Does NOT spend any Claude usage by itself -- this only touches the
# Valve config, no `claude` CLI invocation happens here. Pair with a real
# chat afterward (SETUP.md step 9, or concurrency_test.sh) to confirm
# tool-calling actually works end to end, which does spend usage.
set -euo pipefail
MCP_URL="${1-}"
if [ $# -eq 0 ]; then
echo "Usage: configure_mcp.sh <mcp-server-url> (pass \"\" to clear/disable MCP)" >&2
exit 1
fi
BASE_URL="${OPEN_WEBUI_BASE_URL:-http://localhost:3000}"
API_KEY="${OPEN_WEBUI_API_KEY:?Set OPEN_WEBUI_API_KEY}"
CONTAINER="${OPEN_WEBUI_CONTAINER:-open-webui}"
if [ -n "$MCP_URL" ]; then
echo "==> Checking '$CONTAINER' can reach $MCP_URL (from inside the container, not just the host)..."
if ! docker exec "$CONTAINER" true 2>/dev/null; then
echo "Error: container '$CONTAINER' isn't running or isn't reachable." >&2
exit 1
fi
# Checked separately from the command substitution, not via `cmd || echo
# FAIL` inside it -- confirmed live that pattern is broken here: curl
# prints "000" to stdout on a connection failure *and* exits non-zero,
# so `$(cmd || echo FAIL)` concatenates both into "000FAIL", which then
# matched neither exact-string check below and silently let a bad URL
# through. Capturing the exit status via $? right after, instead of
# inside the substitution, avoids that.
set +e
HTTP_CODE="$(docker exec "$CONTAINER" curl -sS -o /dev/null -w '%{http_code}' --max-time 5 "$MCP_URL" 2>/dev/null)"
CURL_STATUS=$?
set -e
if [ "$CURL_STATUS" -ne 0 ] || [ "$HTTP_CODE" = "000" ] || [ -z "$HTTP_CODE" ]; then
echo "Error: '$CONTAINER' cannot reach $MCP_URL at all (connection failure, not just a non-2xx status)." >&2
echo "If the MCP server runs on the same host machine outside Docker, use http://host.docker.internal:<port>/... instead of localhost -- localhost inside the container means the container itself, not the host." >&2
exit 1
fi
echo " Reachable (HTTP $HTTP_CODE — any real status code, even a 4xx, confirms the network path works)."
else
echo "==> Clearing MCP_SERVER_URL (disables MCP tool access, plain chat keeps working)..."
fi
echo "==> Setting the Valve..."
curl -sS -X POST "$BASE_URL/api/v1/functions/id/claude_cli/valves/update" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d "{\"MCP_SERVER_URL\":\"$MCP_URL\"}" \
-o /dev/null -w " HTTP %{http_code}\n"
echo "==> Reading the Valve back to confirm the write actually took..."
ACTUAL="$(curl -sS "$BASE_URL/api/v1/functions/id/claude_cli/valves" -H "Authorization: Bearer $API_KEY")"
echo " $ACTUAL"
READBACK_URL="$(echo "$ACTUAL" | python3 -c "import json,sys; print(json.load(sys.stdin).get('MCP_SERVER_URL',''))" 2>/dev/null || echo "PARSE_FAILED")"
if [ "$READBACK_URL" != "$MCP_URL" ]; then
echo "Error: Valve readback ('$READBACK_URL') doesn't match what was set ('$MCP_URL') -- the write may not have taken." >&2
exit 1
fi
echo "==> Done. MCP_SERVER_URL is now: '${MCP_URL:-<empty, disabled>}'"
echo " This does NOT confirm tool-calling actually works end to end -- send a"
echo " real chat that needs a tool call (SETUP.md step 9, or concurrency_test.sh)"
echo " to verify that, which spends real Claude usage unlike this script."

View file

@ -962,3 +962,121 @@ running on the target hardware. `concurrency_test.sh` would need a real variant
an actual deed-entry-style task instead of an echoed token, and verification would need to check for
genuine tool-call success/record creation rather than a simple string match. Not built this session; noted
here so it isn't lost track of.
## 2026-08-23 — Real deed-entry concurrency harness: built, but left with an unresolved bug -- paused mid-debugging, not finished
Started building `docs/ichirouganaim-integration/deed_concurrency_test.py` per the follow-up above: a
real multi-turn, human-approval-gated deed-entry conversation simulator (using the actual
`docs/prompts/example.standard-deed-prompt.md` from `ichirouganaim_mcp`, with a dummy
`CONCURRENCY-TEST-VOL-<timestamp>` volume swapped in for the real archival one, per user's choice of
"full multi-turn simulation" over a single-prompt shortcut). Two real bugs found and fixed along the way,
both live-verified:
1. **`chat_id` provided on turn 1 breaks new-chat provisioning.** Supplying a self-generated `chat_id`
up front (so later turns could reference it) makes `main.py`'s own `is_new_chat` check `False` (it
requires *no* `chat_id` present) -- turn 1 silently routed into the "existing chat" branch for a chat
that didn't exist yet, returning HTTP 200 with an empty body and never actually creating the chat
(confirmed live: the chat_id didn't resolve via `GET /api/v1/chats/<id>` afterward). Fixed by
pre-creating the chat via `POST /api/v1/chats/new` first, so every turn -- including the first --
consistently uses the "existing chat" branch for a chat that's genuinely there. Also found and used the
previously-missing piece from the earlier "null response" mystery in this same conversation: `main.py`
expects a structured `user_message` object (`id`, `parentId`, `role`, `content`) as its own top-level
field, separate from the `messages` array and from the top-level `id`/`parent_id` (which are for the
*assistant* placeholder) -- omitting it is what caused chat history to never build correctly.
2. **`post_completion`'s line-by-line SSE iteration silently returned zero content for a real, long,
tool-calling response**, even though the chat *did* persist with real, substantial model output
(confirmed by reading it back via the API -- the model's actual response, including it correctly
pushing back on a "blanket pre-approval" override in the test prompt as a suspicious instruction
conflict, was fully present in the chat's stored `output`). `ok=True` was returned every time, not an
error -- ruled out a caught exception. Switched `post_completion` to `resp.read()` (read the whole body,
confirmed working via a separate manual debug call) instead of `for raw_line in resp:`, since a fast,
no-tool-call request worked fine under either approach but a real multi-minute tool-calling response
consistently didn't under line iteration.
**Not resolved**: after the `resp.read()` fix, a follow-up isolated debug call (a single, cheap
`list_workflows`-only prompt, not the full expensive standing-instructions turn) still came back as a bare
4-byte `"null"` response body -- the same failure signature seen much earlier in this same session before
the `user_message` field was discovered, even though this later call *did* include `user_message`
correctly. No server-side log line for that specific request could be found anywhere in `docker logs`,
which is itself suspicious (every other request in this session, successful or not, produced a uvicorn
access-log line) -- not yet explained. Session got interrupted here by an unrelated, higher-priority ask
(native MCP Tool Server registration, next entry) before this could be chased further.
**Status, explicitly**: `deed_concurrency_test.py` exists in the repo, has the two fixes above applied and
live-verified individually, but has **not** been confirmed to work end-to-end for even a single full
conversation since the `resp.read()` fix -- the last full run (before that fix) technically "completed"
with real chat content persisted server-side but zero content captured by the harness itself, and the
smaller follow-up debug probe (after the fix) hit the unexplained bare-`null` response. Do not treat this
script as validated. Next session picking this up should start by resolving the bare-`null`/no-log-line
mystery on a cheap single-tool-call request before trusting any concurrency numbers this script produces.
## 2026-08-23 — Native MCP Tool Server registration: a genuinely different mechanism from claude_cli's own MCP Valve
User asked to "hook the MCP back into open-webui" -- initially read as "make sure `claude_cli`'s own
`MCP_SERVER_URL` Valve still works" (checked live: it did, valve intact, server reachable both from host
and container), then clarified: they meant registering the MCP server so **any model** in this instance
can use it, not just `claude_cli`. That's a different, independent mechanism -- `claude_cli.py`'s own
Valve is consumed directly by the `claude` CLI subprocess, bypassing this fork's own MCP client
entirely; this fork's built-in **Tool Server** support
(`backend/open_webui/routers/configs.py`'s `/api/v1/configs/tool_servers` endpoints,
`backend/open_webui/utils/middleware.py`'s `connect_mcp_server`) is what native models actually reach
through. The two connections to the same MCP server don't conflict -- confirmed live, running both
simultaneously.
**A real bug found in `configure_mcp.sh` (the Valve-reliability script from earlier) while testing it
more thoroughly**: its reachability check used `HTTP_CODE="$(docker exec ... curl ... || echo
'CONN_FAIL')"`. curl prints `"000"` to stdout on a connection failure *and* exits non-zero -- so on a
genuine failure, the command substitution captured **both**: curl's own `"000"` output, then (since the
overall pipeline still exited non-zero) the `|| echo 'CONN_FAIL'` fallback text appended after it, giving
`"000CONN_FAIL"` -- a string that matched *neither* of the script's exact-match failure checks (`=
"CONN_FAIL"` or `= "000"`), so a genuinely unreachable URL was silently accepted and written to the Valve.
Caught by deliberately testing the failure path, not just the success path (this project's own
discipline, again paying off) -- reproduced live, fixed by checking `$?` separately after the command
substitution instead of combining `||` inside it, re-tested both the failure path (now correctly rejected,
Valve left untouched) and the success path (still works) to confirm.
**Built `register_mcp_tool_server.sh`.** Key implementation details, each verified against the actual
code rather than assumed:
- `ToolServerConnection`'s shape (`backend/open_webui/routers/configs.py:217-227`): for `type: "mcp"`,
`path` is unused entirely -- confirmed by reading `/tool_servers/verify`'s own branching, which calls
`MCPClient().connect(form_data.url, ...)` directly for MCP-type connections (only the `openapi` branch
combines `url`+`path`). Left `path` as an empty string.
- **Access control defaults to admin-only.** `has_connection_access` (`utils/access_control/__init__.py`):
no `config.access_grants` configured means only admins can use the connection, even with
`BYPASS_ADMIN_ACCESS_CONTROL` off for non-admin-bypass scenarios -- confirmed by reading the function
directly, not assumed from the field being optional. The script's `--public` flag needed the *exact*
grant shape `has_access`'s own docstring documents for public read
(`{"principal_type": "user", "principal_id": "*", "permission": "read"}`) -- an earlier draft guessed a
different, made-up shape (`{"type": "public"}`) before this was checked against the real docstring and
corrected prior to any live test.
- **Registering a connection makes it *selectable*, not automatically used.** Traced
`connect_mcp_server`'s only call site in `middleware.py`: it fires when a chat request's `tool_ids`
contains `"server:mcp:<server_id>"`, which comes from the chat's own request metadata (the UI's "+"
tools picker, or a model's own default-enabled tools) -- not something registering the connection alone
turns on for every model. Documented as an explicit, separate follow-up step in `SETUP.md`, not silently
glossed over.
**Live-verified, idempotent, no Claude usage spent** (this only talks to the MCP server directly, never
invokes a model): registered `ichirouganaim_mcp` at `http://host.docker.internal:8931/mcp` --
`/tool_servers/verify` reported **137 tools discovered** (matches the tool count implied by the earlier
`system init` capture's own tools list from step 4's live testing). Re-ran with `--public` added: still
exactly 1 connection afterward (confirmed idempotent, not duplicated), `access_grants` correctly updated
to the public-read shape.
**Not yet tried**: an actual chat where a native (non-`claude_cli`) model calls a tool through this
registration. Everything verified above is the backend talking to the MCP server directly
(`/tool_servers/verify`'s own handshake) -- no model was ever involved, so this doesn't yet prove a real
model can successfully use it mid-conversation, only that the connection itself is live and discoverable.
**Correction to an assumption in the first version of this entry, caught via a real frontend-source
search rather than left standing**: guessed a Tool Server could be set as a given model's own
default-enabled tool through the model editor, as a "further step this script doesn't do." Searched the
actual frontend source instead of trusting that guess: `Workspace → Models → edit`'s `ToolsSelector.svelte`
only reads from the internal Tools registry (`$lib/apis/tools`, individually registered Python function
tools -- a different, older mechanism), never references `tool_server.connections` at all. **There is
currently no way to make a model auto-use a registered Tool Server by default** -- the only mechanism is
the per-chat "+" tools picker (`ToolServersModal.svelte`), every single time, for every chat that needs
it. `SETUP.md` corrected to state this as confirmed rather than the earlier speculative phrasing.
`Integrations.svelte` (Admin Settings → Integrations, labeled "External Tool Servers" in the UI) is where
the registered connection is visible/editable in the browser.

View file

@ -0,0 +1,142 @@
#!/usr/bin/env bash
# Registers an MCP server as a native open-webui Tool Server -- the
# built-in mechanism (backend/open_webui/routers/configs.py's
# /api/v1/configs/tool_servers endpoints) that makes an MCP server usable
# by *any* model in this instance, not just the claude_cli Pipe.
#
# This is a genuinely different mechanism from configure_mcp.sh:
# configure_mcp.sh sets claude_cli's own MCP_SERVER_URL Valve, which only
# that one Pipe function can use (the claude CLI subprocess connects
# directly, bypassing this fork's own MCP client entirely). This script
# instead registers the server in this fork's global tool_server.connections
# config, which native (non-claude-cli) models reach through
# backend/open_webui/utils/middleware.py's own MCPClient. The two are
# independent -- running both means the same MCP server is reachable two
# separate ways, which is fine, they don't conflict.
#
# What registering does NOT do by itself: make every model use it
# automatically. A chat still needs tool_ids containing
# "server:mcp:<server-id>" for a given request to actually connect and
# call it -- either via the model's own default-enabled tools (a separate,
# per-model config step, not done by this script) or the user picking it
# from the chat UI's "+" tools menu. This script only makes the connection
# exist and be selectable; verify_mcp_tool_server.sh (or SETUP.md's own
# guidance) confirms it actually connects.
#
# Idempotent: re-running with the same --id updates the existing entry in
# place (matched by info.id) rather than duplicating it.
#
# Usage:
# export OPEN_WEBUI_API_KEY=sk-...
# ./register_mcp_tool_server.sh --id ichirouganaim_mcp --url http://host.docker.internal:8931/mcp
# ./register_mcp_tool_server.sh --id ichirouganaim_mcp --url <url> --name "Ichirouganaim MCP" --public
#
# --public grants read access to every user, via the exact shape
# has_access's own docstring documents for "public read"
# ({"principal_type": "user", "principal_id": "*", "permission": "read"}
# -- backend/open_webui/utils/access_control/__init__.py). Without it, a
# freshly registered connection with no access_grants defaults to
# admin-only (confirmed by reading has_connection_access directly, not
# assumed).
#
# Does not spend Claude usage -- registers and verifies the MCP server's
# own tool listing directly, no model invocation involved.
set -euo pipefail
BASE_URL="${OPEN_WEBUI_BASE_URL:-http://localhost:3000}"
API_KEY="${OPEN_WEBUI_API_KEY:?Set OPEN_WEBUI_API_KEY}"
SERVER_ID=""
MCP_URL=""
NAME=""
PUBLIC=false
while [ $# -gt 0 ]; do
case "$1" in
--id) SERVER_ID="$2"; shift 2 ;;
--url) MCP_URL="$2"; shift 2 ;;
--name) NAME="$2"; shift 2 ;;
--public) PUBLIC=true; shift ;;
*) echo "Unknown argument: $1" >&2; exit 1 ;;
esac
done
if [ -z "$SERVER_ID" ] || [ -z "$MCP_URL" ]; then
echo "Usage: register_mcp_tool_server.sh --id <server-id> --url <mcp-url> [--name <display name>] [--public]" >&2
exit 1
fi
NAME="${NAME:-$SERVER_ID}"
echo "==> Fetching existing tool server connections..."
EXISTING="$(curl -sS "$BASE_URL/api/v1/configs/tool_servers" -H "Authorization: Bearer $API_KEY")"
echo "==> Building updated connection list (id=$SERVER_ID)..."
UPDATED="$(python3 -c "
import json, sys
existing = json.loads(sys.argv[1])
connections = existing.get('TOOL_SERVER_CONNECTIONS', [])
server_id = sys.argv[2]
url = sys.argv[3]
name = sys.argv[4]
public = sys.argv[5] == 'true'
new_conn = {
'url': url,
'path': '',
'type': 'mcp',
'auth_type': 'none',
'headers': None,
'key': None,
'config': {'access_grants': [{'principal_type': 'user', 'principal_id': '*', 'permission': 'read'}] if public else []},
'info': {'id': server_id, 'name': name},
}
connections = [c for c in connections if (c.get('info') or {}).get('id') != server_id]
connections.append(new_conn)
print(json.dumps({'TOOL_SERVER_CONNECTIONS': connections}))
" "$EXISTING" "$SERVER_ID" "$MCP_URL" "$NAME" "$PUBLIC")"
echo "==> Verifying the MCP server actually connects before saving (no Claude usage, direct MCP handshake)..."
VERIFY_PAYLOAD="$(python3 -c "
import json
print(json.dumps({
'url': '$MCP_URL', 'path': '', 'type': 'mcp', 'auth_type': 'none',
'headers': None, 'key': None, 'config': {}, 'info': {'id': '$SERVER_ID', 'name': '$NAME'},
}))
")"
VERIFY_RESULT="$(curl -sS -w '\n%{http_code}' -X POST "$BASE_URL/api/v1/configs/tool_servers/verify" \
-H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
-d "$VERIFY_PAYLOAD")"
VERIFY_STATUS="$(echo "$VERIFY_RESULT" | tail -1)"
VERIFY_BODY="$(echo "$VERIFY_RESULT" | sed '$d')"
if [ "$VERIFY_STATUS" != "200" ]; then
echo "Error: verification failed (HTTP $VERIFY_STATUS): $VERIFY_BODY" >&2
echo "Not saving the connection -- fix reachability first (this endpoint runs from inside the backend process itself, same network as the running container)." >&2
exit 1
fi
TOOL_COUNT="$(echo "$VERIFY_BODY" | python3 -c "import json,sys; print(len(json.load(sys.stdin).get('specs',[])))" 2>/dev/null || echo "?")"
echo " Verified -- MCP server responded, $TOOL_COUNT tool(s) discovered."
echo "==> Saving the connection..."
SAVE_RESULT="$(curl -sS -w '\n%{http_code}' -X POST "$BASE_URL/api/v1/configs/tool_servers" \
-H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
-d "$UPDATED")"
SAVE_STATUS="$(echo "$SAVE_RESULT" | tail -1)"
if [ "$SAVE_STATUS" != "200" ]; then
echo "Error: saving the connection failed (HTTP $SAVE_STATUS): $(echo "$SAVE_RESULT" | sed '$d')" >&2
exit 1
fi
echo "==> Done. Registered as tool_ids entry: server:mcp:$SERVER_ID"
if [ "$PUBLIC" = false ]; then
echo " Access is admin-only (no --public flag) -- only admin users can select or use this tool server."
fi
echo " This makes it *available*, not automatically used by every model --"
echo " a chat still needs tool_ids: [\"server:mcp:$SERVER_ID\"] to actually call it"
echo " (the UI's '+' tools menu, or configuring it as a model's default tool)."