4d99cd32fe
- Rename UI panel to "MCP Toolkit Guide" with a plainer description. - Apply taglines to README.md and package.json descriptions. - Gate pinned-bubble backdrop-filter on Windows to avoid gray rectangles. - Add packaging-contract assertions for pinned bubble and new description. - Add custom-familiar-name.test.ts covering normalizeFamiliarName. - Sync docs (FEATURES, README_OUR_CHANGES, FEATURES_OUR_CHANGES, PULL_REQUEST, FEATURE_REGISTRY, bug-squashing, rebrand plan, pr-chat-memory-mcp-toolkit).
157 lines
8.7 KiB
Markdown
157 lines
8.7 KiB
Markdown
# FamiliarOS — Our Changes (Isolated Feature List)
|
||
|
||
This document lists only the features, UI surfaces, and capabilities added by our working fork. It is intentionally separate from the upstream `README.md` and `FEATURES.md` so the original repo maintainer can choose whether to adopt these changes wholesale, partially, or not at all.
|
||
|
||
## Floating Chat Surface
|
||
|
||
- Double-click the familiar to open a frameless, always-on-top floating prompt window.
|
||
- Compact prompt-first UX at a minimal 96px height; expanded editor/history mode at 280px.
|
||
- Prompt bar sized for 2–3 lines of text.
|
||
- Resizable with a corner grip; size is remembered per mode (compact/expanded).
|
||
- Theme-aware (system / light / dark) shared with Control Center.
|
||
- Inline error presentation and keyboard shortcuts (Enter sends, Ctrl+Enter newline, Escape closes).
|
||
|
||
## Chat History List UI
|
||
|
||
- Conversation history renders as a styled list of individual entries, not a raw textarea.
|
||
- Per-entry role badges: You, Familiar, System, Error.
|
||
- Color-coded entry backgrounds per role.
|
||
- Timestamps shown per entry.
|
||
- Scrollable with automatic scroll-to-bottom on new messages.
|
||
- Persists to disk across app restarts (`familiaros-chat-history.json`).
|
||
|
||
## OpenAPI BYOK Chat
|
||
|
||
- Main-process OpenAPI-compatible chat service.
|
||
- Supports both `responses` and `chat/completions` transports with automatic route selection.
|
||
- Secure credential storage via Electron `safeStorage` with plain fallback.
|
||
- Model selection and endpoint override in Settings.
|
||
- Built-in presets: OpenAI, OpenRouter, Azure template, LiteLLM, vLLM, Moonshot/Kimi, custom/local templates.
|
||
- Localhost-only allowance for plain `http`.
|
||
|
||
## Familiar Character & Base Instructions
|
||
|
||
- User-editable system-prompt style familiar character field in General Settings.
|
||
- Character prompt persists in app state and applies to future assistant replies.
|
||
- Base instructions toggle to include/exclude the default FamiliarOS behavior instructions in every chat.
|
||
|
||
## Custom Familiar Name
|
||
|
||
- Settings → General includes a user-editable **Familiar name** field for the default Familiar.
|
||
- The name persists in app state (`familiaros-state.json`) and is applied through `validatePreferencePatch`.
|
||
- The chosen name appears in the tray tooltip and tray context menu.
|
||
- Falls back to the Familiar's display name or "Built-in Familiar" when no custom name is set.
|
||
|
||
## Familiar Text-to-Speech (Phase 2)
|
||
|
||
- Settings > **Text-to-Speech** panel for the familiar and floating chat.
|
||
- **System voice** via the OS speech engine with voice matching and 0.5×–2.0× speed control.
|
||
- **Cloud providers**: OpenAI TTS, ElevenLabs, OpenAI-compatible (OpenRouter / LiteLLM / WaveSpeedAI / custom).
|
||
- **Local provider**: Piper for offline speech.
|
||
- Credentials stored with Electron `safeStorage` (plain fallback).
|
||
- Per-provider model, voice, speed, endpoint preset, and custom endpoint.
|
||
- Dynamic voice list fetching for ElevenLabs; static lists for OpenAI.
|
||
- Test voice + Stop buttons in Settings.
|
||
- Speech skips quiet hours, validates returned audio, and enforces HTTPS / localhost rules for endpoints.
|
||
- Plugin SDK `voice:speak` permission lets plugins speak through the same pipeline.
|
||
|
||
## Memory System
|
||
|
||
- Local-first durable FamiliarOS memory store.
|
||
- Automatic memory capture from prompts (`remember that...`, `/remember`, `/memorize`, `/note`, `don't forget...`, `my name is...`, `my favorite...`, `I prefer...`, `I like/love/dislike/hate...`).
|
||
- Specific memory kinds (identity, preference) take precedence over generic notes when the prompt matches both.
|
||
- Relevance-based retrieval injected into future chat system prompts.
|
||
- Memory kinds: preference, identity, fact, note.
|
||
- Importance weighting and tags.
|
||
- In-app memory viewer in Settings with search, inline edit, and delete with confirmation.
|
||
- Human-inspectable markdown mirror files on disk.
|
||
- **Chat history memory**: cross-conversation message search that injects relevant past conversation excerpts into the system prompt.
|
||
- Separate databases for explicit memories (`familiaros-memory.json`) and chat history (`familiaros-chat-history.json`).
|
||
|
||
## MCP Tool Servers (Built-in Chat)
|
||
|
||
- Internal stdio MCP client in the desktop main process (`mcp-chat-client.ts`).
|
||
- Top-level **MCP Tool Servers** panel in Control Center → Integrations.
|
||
- Multi-select tool activation for the built-in floating chat, independent of external agents.
|
||
- Tiered tool browser: Starter, Terminal & Systems, Advanced.
|
||
- Save & Activate / Deactivate All controls.
|
||
- ReAct-style tool-calling loop (max 5 iterations) with responses/chat-completions integration.
|
||
- Supported tools: filesystem, terminal/shell, memory, fetch-web, sequential-thinking, playwright, git, github, docker, sqlite.
|
||
|
||
## FamiliarOS MCP Server (External-Agent Bridge)
|
||
|
||
- Central **FamiliarOS MCP Server** panel in Control Center → Integrations.
|
||
- Command mode selector (published / bundled / local) in one place.
|
||
- Node path override in the central panel.
|
||
- Familiar routing selector in the central panel.
|
||
- One-click **Test Server** health check.
|
||
- One-click **Copy MCP JSON** for external hosts.
|
||
|
||
## MCP Toolkit Guide Integration
|
||
|
||
- MCP Toolkit Guide panel inside Control Center → Integrations.
|
||
- Hand-picked reference surface, not an automatic installer.
|
||
- Each entry explains what the tool does, what permissions it needs, and how to set it up manually or as a persistent bundle in supported hosts.
|
||
- Multi-select chips by tier (Starter, Terminal & Systems, Advanced).
|
||
- Copy-paste snippets and docs links for deliberate adoption.
|
||
- Removed redundant Curated Stack section in favor of direct tiered selection.
|
||
|
||
## Knowledge Store
|
||
|
||
- Local-first file + memory knowledge repository added to the desktop app.
|
||
- Store files from Settings > Knowledge Store or directly from floating-chat attachments.
|
||
- Text extraction from UTF-8 text-like files; MIME inference from extension.
|
||
- Indexed metadata with file count cap (100) and per-file size cap (5 MB).
|
||
- Relevance search across file names and extracted text; results injected into OpenAPI chat instructions.
|
||
- Combined search across stored files and FamiliarOS memories.
|
||
- Dedicated Settings tab for upload, search, list, and delete.
|
||
- Toolbar button in the floating prompt window stores attachments into the Knowledge Store.
|
||
- Electron-free core (`knowledge-store-core.ts`) + unit tests (`knowledge-store.test.ts`).
|
||
|
||
## Familiar Scale & Sizing
|
||
|
||
- Settings slider for familiar scale from 0.16x up to 10x, large enough to fill the screen.
|
||
- Familiar window dynamically resizes to fit larger scales without clipping.
|
||
- Speech bubbles grow with the familiar and larger window sizes; long messages scroll instead of cutting off.
|
||
|
||
## Multi-Screen Support
|
||
|
||
- Familiar positioning uses `screen.getDisplayNearestPoint()` instead of always clamping to the primary display.
|
||
- `clampToNearestWorkArea()` keeps the familiar within the work area of the display it is currently on.
|
||
|
||
## Multi-Conversation Chat
|
||
|
||
- Per-conversation message history instead of a single global transcript.
|
||
- Conversation list UI in the prompt window History button.
|
||
- Auto-generated conversation titles from the first user message.
|
||
- Create, switch, and delete conversations from the floating chat window.
|
||
- Editor button shows the current conversation's message list (unchanged).
|
||
- New Chat button creates a fresh conversation (old conversations are preserved).
|
||
- Conversation metadata stored in `familiaros-chat-conversations.json`.
|
||
|
||
## Settings & Control Center
|
||
|
||
- Memory viewer section in Settings.
|
||
- MCP Tool Servers panel in Integrations for built-in chat tool activation.
|
||
- FamiliarOS MCP Server panel in Integrations for external-agent bridge configuration.
|
||
- OpenAPI chat settings: endpoint, model, credential, theme, base instructions toggle.
|
||
- Moonshot/Kimi API endpoint preset.
|
||
|
||
## Virtual Familiar Status HUD
|
||
|
||
- Bundled `familiaros.virtual-familiar` plugin tracks hunger, energy, happiness, and affection.
|
||
- Status values are shown as read-only labels inside the right-click familiar context menu (Virtual Familiar ▸ Mood, Food, Energy, Play, Bond, Level), so they never block familiar clicks or double-click chat.
|
||
- Single-click the familiar to familiar it (increases bond and happiness).
|
||
- Feed / play / familiar / nap commands live in the familiar context menu.
|
||
- Low-stat nudges continue to work in the background.
|
||
|
||
## Build & Packaging
|
||
|
||
- Sharp Windows native binary packaging resolved via `asarUnpack` + post-build copy.
|
||
- `prompt-window-preload.cjs` added for the floating chat renderer bridge.
|
||
- Additional preload scripts and assets packaged for the new surfaces.
|
||
|
||
## Bug Fixes
|
||
|
||
- Floating-chat memory capture/retrieval is now more resilient: capture is logged and errors are caught instead of breaking the send path.
|
||
- Long speech-bubble text no longer clips; bubble bodies scroll when content exceeds the bubble max-height.
|