openpetswithchatandmcp/PULL_REQUEST.md

Pull Request: Floating Chat, Memory, MCP Toolkit & Pet Sizing

Title: feat(desktop): add floating chat, local memory, vanilla MCP tools, expanded pet sizing, and multi-screen support

Author: VectorShell working fork Base: upstream main at cba81b7 Status: ready for review / adopt selectively

---

Summary

This PR adds a complete in-app chat layer, a durable local memory system, interactive MCP tool activation, and several companion-quality improvements to the desktop pet. Every change is additive and preserves existing MCP/agent integration behavior.

Motivation

• Users wanted to chat with the pet without a terminal or external agent.
• Long-running companions benefit from remembering facts, preferences, and identity.
• A curated but practical MCP surface helps users adopt tools without unsafe one-click installs.
• Pet sizing was limited to three presets with no continuous or interactive adjustment.

What changed

New surfaces

• Floating Prompt Window (apps/desktop/src/prompt-window.ts)

  • Frameless, always-on-top chat UI opened by double-clicking the pet.
  • Compact mode at 96px height; expanded editor/history mode at 280px.
  • Built as inline HTML/CSS/JS with a dedicated preload bridge.

• Chat History List UI

  • Replaced plain-text history textarea with styled per-entry cards.
  • Role badges (You / Pet / System / Error), color coding, timestamps.
  • Persists to disk (openpets-chat-history.json).

• Memory Viewer

  • New Settings section to search, edit, and delete OpenPets memories.

• Vanilla Chat MCP Tool Activation

  • New Integrations UI to activate MCP tools for use inside floating chat.
  • Tiered chip selector (Starter / Terminal & Systems / Advanced).

New core modules

• apps/desktop/src/openapi-chat.ts — OpenAPI BYOK chat with responses + chat/completions support.
• apps/desktop/src/mcp-chat-client.ts — Internal stdio MCP client for floating chat tools.
• apps/desktop/src/openpets-memory.ts — Local memory engine with JSON store and markdown mirror.
• apps/desktop/src/mcp-toolkit-installer.ts — Toolkit install orchestration.
• apps/desktop/src/renderer/src/mcp-toolkit-catalog.ts — Catalog data for the toolkit UI.
• apps/desktop/prompt-window-preload.cjs — Preload for the floating chat renderer.

Pet improvements

• Continuous pet scale slider from 0.16x to 10x in Settings.
• Dynamic window sizing so larger scales are never clipped.
• Multi-screen positioning (clampToNearestWorkArea).
• Removed the on-pet drag-to-resize handle in favor of the Settings slider.
• Virtual-pet status (openpets.virtual-pet) moved from an on-pet triple-click HUD overlay to the right-click pet context menu, so it never blocks chat clicks. Single-clicking the pet triggers the pet() care action.

Settings additions

• OpenAPI chat: endpoint preset, model, credential, theme, base-instructions toggle.
• Moonshot/Kimi endpoint preset.
• Pet character / system prompt field.
• Memory viewer with search/edit/delete.

Bug fixes

• Memory capture/retrieval is now more robust in floating chat: capture is logged and errors are caught instead of breaking the send path.
• Speech bubble text no longer clips for long messages; bubble bodies scroll when content exceeds the bubble max-height.

Updated files

• README.md — added floating chat, memory, and vanilla chat tool bullets.
• FEATURES.md — comprehensive inventory of new capabilities.
• docs/pr-chat-memory-mcp-toolkit.md — implementation explainer.
• docs/phases/phase-14-pet-scale-setting.md — expanded with new sizes and drag resize.
• apps/desktop/src/pet-window.ts — removed scale handle; IPC handlers; dynamic window sizing.
• apps/desktop/pet-preload.cjs — single/double-click detection; single-click emits pet:clicked for virtual-pet petting, double-click opens floating chat.
• apps/desktop/src/app-state-core.ts — expanded scale range.
• apps/desktop/src/renderer/src/main.tsx — Settings/Integrations UI additions.
• apps/desktop/src/windows.ts — memory IPC, toolkit IPC, preference handling.
• apps/desktop/tests/onboarding-state.test.ts — updated scale assertions.

New files

FEATURES.md
README_OUR_CHANGES.md
FEATURES_OUR_CHANGES.md
PULL_REQUEST.md
apps/desktop/prompt-window-preload.cjs
apps/desktop/src/mcp-chat-client.ts
apps/desktop/src/mcp-toolkit-installer.ts
apps/desktop/src/openapi-chat.ts
apps/desktop/src/openpets-memory.ts
apps/desktop/src/prompt-window.ts
apps/desktop/src/renderer/src/mcp-toolkit-catalog.ts

How to test

pnpm install
pnpm --filter @open-pets/desktop build
pnpm --filter @open-pets/desktop test

All existing tests pass. Additional manual verification:

1. Launch the desktop app and double-click the pet — floating chat should open.
2. Add an OpenAI-compatible API key in Settings > OpenAPI Chat.
3. Send a message; assistant reply should appear in the pet bubble and in the history list.
4. Say "remember that my favorite color is blue" — memory should be captured.
5. Open Settings > Memory — the memory should be searchable, editable, and deletable.
6. Open Integrations > MCP Toolkit — activate Filesystem + Fetch/Web, then chat "list my home directory files" — tools should run.
7. Right-click the pet → choose Virtual Pet ▸ Feed/Play/Pet/Nap; stats update and the context menu reflects the new values.
8. Change display — pet should stay on the display it is currently near.

Backwards compatibility

• Existing MCP server behavior is unchanged.
• Existing agent integrations (Claude, OpenCode, Cursor, Pi) are unchanged.
• Existing plugin system is unchanged.
• Preferences default safely: users without API keys see a clear "Add credential" state.

Notes for upstream

• Two documentation styles are provided:
  • Merged: README.md and FEATURES.md are updated in-place for easy adoption.
  • Isolated: README_OUR_CHANGES.md, FEATURES_OUR_CHANGES.md, and PULL_REQUEST.md describe only our additions so you can pick and choose.
• If you prefer smaller PRs, this can be split along these boundaries:
  1. Floating chat + OpenAPI BYOK
  2. Memory system + memory viewer
  3. Vanilla chat MCP tools
  4. Pet sizing expansion + drag handle
  5. Multi-screen positioning