# FamiliarOS — Our Changes (Isolated Feature List)

This document lists only the features, UI surfaces, and capabilities added by our working fork. It is intentionally separate from the upstream `README.md` and `FEATURES.md` so the original repo maintainer can choose whether to adopt these changes wholesale, partially, or not at all.

## Floating Chat Surface

- Double-click the familiar to open a frameless, always-on-top floating prompt window.
- Compact prompt-first UX at a minimal 96px height; expanded editor/history mode at 280px.
- Prompt bar sized for 2–3 lines of text.
- Resizable with a corner grip; size is remembered per mode (compact/expanded).
- Theme-aware (system / light / dark) shared with Control Center.
- Inline error presentation and keyboard shortcuts (Enter sends, Ctrl+Enter newline, Escape closes).

## Chat History List UI

- Conversation history renders as a styled list of individual entries, not a raw textarea.
- Per-entry role badges: You, Familiar, System, Error.
- Color-coded entry backgrounds per role.
- Timestamps shown per entry.
- Scrollable with automatic scroll-to-bottom on new messages.
- Persists to disk across app restarts (`familiaros-chat-history.json`).

## OpenAPI BYOK Chat

- Main-process OpenAPI-compatible chat service.
- Supports both `responses` and `chat/completions` transports with automatic route selection.
- Secure credential storage via Electron `safeStorage` with plain fallback.
- Model selection and endpoint override in Settings.
- Built-in presets: OpenAI, OpenRouter, Azure template, LiteLLM, vLLM, Moonshot/Kimi, custom/local templates.
- Localhost-only allowance for plain `http`.

## Familiar Character & Base Instructions

- User-editable system-prompt style familiar character field in General Settings.
- Character prompt persists in app state and applies to future assistant replies.
- Base instructions toggle to include/exclude the default FamiliarOS behavior instructions in every chat.

## Custom Familiar Name

- Settings → General includes a user-editable **Familiar name** field for the default Familiar.
- The name persists in app state (`familiaros-state.json`) and is applied through `validatePreferencePatch`.
- The chosen name appears in the tray tooltip and tray context menu.
- Falls back to the Familiar's display name or "Built-in Familiar" when no custom name is set.

## Familiar Text-to-Speech (Phase 2)

- Settings > **Text-to-Speech** panel for the familiar and floating chat.
- **System voice** via the OS speech engine with voice matching and 0.5×–2.0× speed control.
- **Cloud providers**: OpenAI TTS, ElevenLabs, OpenAI-compatible (OpenRouter / LiteLLM / WaveSpeedAI / custom).
- **Local provider**: Piper for offline speech.
- Credentials stored with Electron `safeStorage` (plain fallback).
- Per-provider model, voice, speed, endpoint preset, and custom endpoint.
- Dynamic voice list fetching for ElevenLabs; static lists for OpenAI.
- Test voice + Stop buttons in Settings.
- Speech skips quiet hours, validates returned audio, and enforces HTTPS / localhost rules for endpoints.
- Plugin SDK `voice:speak` permission lets plugins speak through the same pipeline.

## Memory System

- Local-first durable FamiliarOS memory store.
- Automatic memory capture from prompts (`remember that...`, `/remember`, `/memorize`, `/note`, `don't forget...`, `my name is...`, `my favorite...`, `I prefer...`, `I like/love/dislike/hate...`).
- Specific memory kinds (identity, preference) take precedence over generic notes when the prompt matches both.
- Relevance-based retrieval injected into future chat system prompts.
- Memory kinds: preference, identity, fact, note.
- Importance weighting and tags.
- In-app memory viewer in Settings with search, inline edit, and delete with confirmation.
- Human-inspectable markdown mirror files on disk.
- **Chat history memory**: cross-conversation message search that injects relevant past conversation excerpts into the system prompt.
- Separate databases for explicit memories (`familiaros-memory.json`) and chat history (`familiaros-chat-history.json`).

## MCP Tool Servers (Built-in Chat)

- Internal stdio MCP client in the desktop main process (`mcp-chat-client.ts`).
- Top-level **MCP Tool Servers** panel in Control Center → Integrations.
- Multi-select tool activation for the built-in floating chat, independent of external agents.
- Tiered tool browser: Starter, Terminal & Systems, Advanced.
- Save & Activate / Deactivate All controls.
- ReAct-style tool-calling loop (max 5 iterations) with responses/chat-completions integration.
- Supported tools: filesystem, terminal/shell, memory, fetch-web, sequential-thinking, playwright, git, github, docker, sqlite.

## FamiliarOS MCP Server (External-Agent Bridge)

- Central **FamiliarOS MCP Server** panel in Control Center → Integrations.
- Command mode selector (published / bundled / local) in one place.
- Node path override in the central panel.
- Familiar routing selector in the central panel.
- One-click **Test Server** health check.
- One-click **Copy MCP JSON** for external hosts.

## MCP Toolkit Guide Integration

- MCP Toolkit Guide panel inside Control Center → Integrations.
- Hand-picked reference surface, not an automatic installer.
- Each entry explains what the tool does, what permissions it needs, and how to set it up manually or as a persistent bundle in supported hosts.
- Multi-select chips by tier (Starter, Terminal & Systems, Advanced).
- Copy-paste snippets and docs links for deliberate adoption.
- Removed redundant Curated Stack section in favor of direct tiered selection.

## Knowledge Store

- Local-first file + memory knowledge repository added to the desktop app.
- Store files from Settings > Knowledge Store or directly from floating-chat attachments.
- Text extraction from UTF-8 text-like files; MIME inference from extension.
- Indexed metadata with file count cap (100) and per-file size cap (5 MB).
- Relevance search across file names and extracted text; results injected into OpenAPI chat instructions.
- Combined search across stored files and FamiliarOS memories.
- Dedicated Settings tab for upload, search, list, and delete.
- Toolbar button in the floating prompt window stores attachments into the Knowledge Store.
- Electron-free core (`knowledge-store-core.ts`) + unit tests (`knowledge-store.test.ts`).

## Familiar Scale & Sizing

- Settings slider for familiar scale from 0.16x up to 10x, large enough to fill the screen.
- Familiar window dynamically resizes to fit larger scales without clipping.
- Speech bubbles grow with the familiar and larger window sizes; long messages scroll instead of cutting off.

## Multi-Screen Support

- Familiar positioning uses `screen.getDisplayNearestPoint()` instead of always clamping to the primary display.
- `clampToNearestWorkArea()` keeps the familiar within the work area of the display it is currently on.

## Multi-Conversation Chat

- Per-conversation message history instead of a single global transcript.
- Conversation list UI in the prompt window History button.
- Auto-generated conversation titles from the first user message.
- Create, switch, and delete conversations from the floating chat window.
- Editor button shows the current conversation's message list (unchanged).
- New Chat button creates a fresh conversation (old conversations are preserved).
- Conversation metadata stored in `familiaros-chat-conversations.json`.

## Settings & Control Center

- Memory viewer section in Settings.
- MCP Tool Servers panel in Integrations for built-in chat tool activation.
- FamiliarOS MCP Server panel in Integrations for external-agent bridge configuration.
- OpenAPI chat settings: endpoint, model, credential, theme, base instructions toggle.
- Moonshot/Kimi API endpoint preset.

## Virtual Familiar Status HUD

- Bundled `familiaros.virtual-familiar` plugin tracks hunger, energy, happiness, and affection.
- Status values are shown as read-only labels inside the right-click familiar context menu (Virtual Familiar ▸ Mood, Food, Energy, Play, Bond, Level), so they never block familiar clicks or double-click chat.
- Single-click the familiar to familiar it (increases bond and happiness).
- Feed / play / familiar / nap commands live in the familiar context menu.
- Low-stat nudges continue to work in the background.

## Build & Packaging

- Sharp Windows native binary packaging resolved via `asarUnpack` + post-build copy.
- `prompt-window-preload.cjs` added for the floating chat renderer bridge.
- Additional preload scripts and assets packaged for the new surfaces.

## Bug Fixes

- Floating-chat memory capture/retrieval is now more resilient: capture is logged and errors are caught instead of breaking the send path.
- Long speech-bubble text no longer clips; bubble bodies scroll when content exceeds the bubble max-height.