From bb758e9d0f9c2678bc8b237e7734f5b581cdebd3 Mon Sep 17 00:00:00 2001 From: OpenPets Dev Date: Tue, 16 Jun 2026 18:40:42 +0000 Subject: [PATCH] feat(desktop): add Text-to-Speech Phase 2 and update docs/registry - Add configurable TTS providers: system OS voice, OpenAI TTS, ElevenLabs, Piper (local), and OpenAI-compatible (OpenRouter/LiteLLM/WaveSpeedAI/custom). - Add secure TTS credential storage, endpoint presets, voice/model/speed UI, and test/stop controls in Settings. - Wire TTS to pet/floating-chat renderer via openpets:tts-speak/audio/stop IPC. - Respect quiet hours; validate cloud endpoints and returned audio buffers. - Add plugin voice:speak bridge and tts-engine unit tests. - Update FEATURES.md, FEATURES_OUR_CHANGES.md, README.md, README_OUR_CHANGES.md, PULL_REQUEST.md, FEATURE_REGISTRY.md, and add docs/pull-requests/PR_TTS_PHASE_2.md. --- FEATURES.md | 38 +- FEATURES_OUR_CHANGES.md | 32 +- FEATURE_REGISTRY.md | 37 +- PULL_REQUEST.md | 46 +- README.md | 18 +- README_OUR_CHANGES.md | 31 +- apps/desktop/control-center-preload.cjs | 8 + apps/desktop/pet-preload.cjs | 34 +- apps/desktop/prompt-window-preload.cjs | 45 + apps/desktop/scripts/run-tests.mjs | 1 + apps/desktop/src/agent-setup.ts | 52 +- apps/desktop/src/app-state.ts | 18 + apps/desktop/src/i18n/locales/en.ts | 60 +- apps/desktop/src/i18n/locales/es-419.ts | 29 +- apps/desktop/src/i18n/locales/ja.ts | 29 +- apps/desktop/src/i18n/locales/ko.ts | 29 +- apps/desktop/src/i18n/locales/pt-BR.ts | 29 +- apps/desktop/src/i18n/locales/zh-Hans.ts | 29 +- apps/desktop/src/i18n/locales/zh-Hant.ts | 29 +- apps/desktop/src/logger.ts | 2 +- apps/desktop/src/openapi-chat.ts | 9 + apps/desktop/src/plugin-voice.ts | 12 +- apps/desktop/src/prompt-window.ts | 4 + apps/desktop/src/renderer/src/main.tsx | 1019 ++++++++++++----- apps/desktop/src/tray.ts | 2 +- apps/desktop/src/tts-catalog.ts | 56 + apps/desktop/src/tts-credentials.ts | 109 ++ apps/desktop/src/tts-engine.ts | 256 +++++ apps/desktop/src/tts-service.ts | 111 ++ apps/desktop/src/windows.ts | 116 +- apps/desktop/tests/tts-engine.test.ts | 32 + .../OPENPETS_3_1_0_MOD_NICHE_PULL_REQUESTS.md | 8 +- 32 files changed, 1944 insertions(+), 386 deletions(-) create mode 100644 apps/desktop/src/tts-catalog.ts create mode 100644 apps/desktop/src/tts-credentials.ts create mode 100644 apps/desktop/src/tts-engine.ts create mode 100644 apps/desktop/src/tts-service.ts create mode 100644 apps/desktop/tests/tts-engine.test.ts diff --git a/FEATURES.md b/FEATURES.md index 8f0ad96b..b8761d0a 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -51,6 +51,23 @@ It combines: - Safer speech rules for agent-driven bubble content - Bubble behavior that avoids showing code, logs, URLs, paths, or secrets in normal integration speech +## Pet Text-to-Speech (Phase 2) + +- Settings > **Text-to-Speech** panel controls voice output for the pet and floating chat. +- **System voice** uses the OS speech engine through the renderer (`window.speechSynthesis`) with optional voice-name matching and a 0.5×–2.0× speed multiplier. +- **Cloud TTS providers**: OpenAI TTS, ElevenLabs, and an **OpenAI-compatible** preset for OpenRouter, LiteLLM, WaveSpeedAI, or a custom endpoint. +- **Local TTS provider**: Piper (spawned as a local process) for fully offline speech. +- Provider credentials are stored with Electron `safeStorage` and fall back to plain local storage when encryption is unavailable (`apps/desktop/src/tts-credentials.ts`). +- Per-provider model, voice, speed, endpoint preset, and custom endpoint controls. +- Dynamic voice list fetching for ElevenLabs; static voice lists for OpenAI / OpenAI-compatible. +- In-app **Test voice** and **Stop** buttons to preview the configured voice. +- Assistant replies and other speech are spoken through the pet window renderer via a shared TTS service (`apps/desktop/src/tts-service.ts`). +- Respects quiet-hours: speech is skipped while quiet hours are active. +- Audio returned by cloud providers is validated (MP3/WAV magic bytes) and played through a renderer `