From b39d104bbc14d676ca8c8556566bec4d408d2674 Mon Sep 17 00:00:00 2001 From: axiomlogicnexus Date: Mon, 22 Jun 2026 06:13:21 +0000 Subject: [PATCH] Add public website crawl surfaces --- ...LING_AND_DISTRIBUTION_PACKET_2026-06-22.md | 1 + .../HyperTwist/FEATURE_REGISTRY.md | 2 +- .../HyperTwist/ROADMAP.md | 4 +- website/README.md | 1 + website/public/robots.txt | 7 ++++ website/public/sitemap.xml | 39 +++++++++++++++++++ 6 files changed, 52 insertions(+), 2 deletions(-) create mode 100644 website/public/robots.txt create mode 100644 website/public/sitemap.xml diff --git a/docs/ops/HYPERTWIST_PUBLIC_WEBSITE_AUTH_BILLING_AND_DISTRIBUTION_PACKET_2026-06-22.md b/docs/ops/HYPERTWIST_PUBLIC_WEBSITE_AUTH_BILLING_AND_DISTRIBUTION_PACKET_2026-06-22.md index 8857238..19eaffa 100644 --- a/docs/ops/HYPERTWIST_PUBLIC_WEBSITE_AUTH_BILLING_AND_DISTRIBUTION_PACKET_2026-06-22.md +++ b/docs/ops/HYPERTWIST_PUBLIC_WEBSITE_AUTH_BILLING_AND_DISTRIBUTION_PACKET_2026-06-22.md @@ -179,6 +179,7 @@ for `hypertwist.app` when a built `website/dist/index.html` is present: - `WEBSITE_DIST_PATH` and `SERVE_STATIC_WEBSITE` now expose explicit deployment control for that lane - the runtime-readiness verifier now warns when same-origin public posture leaves that static-serving mode implicit - request-level server coverage now also proves the live route behavior for public/app routes versus `/api/*`, `/auth*`, `/health`, and missing asset paths +- the frontend public deployment surface now also includes first-party `robots.txt` plus `sitemap.xml` assets for the current public route set, while keeping `/app`, `/login`, and `/register` out of crawler posture This is browser-based user access for the operator/account surface. diff --git a/docs/v6_5_deep_manual_pack/HyperTwist/FEATURE_REGISTRY.md b/docs/v6_5_deep_manual_pack/HyperTwist/FEATURE_REGISTRY.md index d9ed8ed..1721e8c 100644 --- a/docs/v6_5_deep_manual_pack/HyperTwist/FEATURE_REGISTRY.md +++ b/docs/v6_5_deep_manual_pack/HyperTwist/FEATURE_REGISTRY.md @@ -263,7 +263,7 @@ repo. | Feature | Status | Primary authority | Notes | |---|---|---|---| -| Public `hypertwist.app` marketing shell | Implemented now | first-party `website/` app + feature registry/roadmap authority | HyperTwist now has a dedicated first-party public web surface for homepage, about, resources, pricing, download, support, and legal routes. This lane is separate from the embedded Unreal browser runtime under `Content/Browser/` and does not claim browser-simulator parity. The same package now also carries a first-party external runtime-readiness verifier so deploy-time env and live health posture can be checked outside the dashboard, plus separated local-versus-production env templates whose placeholder values are intentionally rejected until real launch config is in place, bootstrap CI now validates both the frontend and auth-server website commands directly, and the auth server can now auto-serve the built `website/dist` bundle with bounded SPA fallback for same-origin public deployment. Request-level server coverage now also proves that public/app shell delivery does not shadow `/api/*`, `/auth*`, `/health`, or missing asset paths, while the pricing/download/notices routes now surface first-party preview-versus-launch posture from the same bounded launch checklist instead of relying on hidden operator-only status. The live website lane now also owns route-aware title/description/canonical/Open-Graph/Twitter metadata for the real `hypertwist.app` marketing surface so deployed public pages no longer remain on a single generic SPA title/description. The real `check-runtime-readiness` CLI is now also exercised against the checked-in production example env files, and a spawned `website/server` bootstrap proof now verifies the live same-origin process path from production-shaped env into `/health`, `/api/auth/health`, and built-shell serving. | +| Public `hypertwist.app` marketing shell | Implemented now | first-party `website/` app + feature registry/roadmap authority | HyperTwist now has a dedicated first-party public web surface for homepage, about, resources, pricing, download, support, and legal routes. This lane is separate from the embedded Unreal browser runtime under `Content/Browser/` and does not claim browser-simulator parity. The same package now also carries a first-party external runtime-readiness verifier so deploy-time env and live health posture can be checked outside the dashboard, plus separated local-versus-production env templates whose placeholder values are intentionally rejected until real launch config is in place, bootstrap CI now validates both the frontend and auth-server website commands directly, and the auth server can now auto-serve the built `website/dist` bundle with bounded SPA fallback for same-origin public deployment. Request-level server coverage now also proves that public/app shell delivery does not shadow `/api/*`, `/auth*`, `/health`, or missing asset paths, while the pricing/download/notices routes now surface first-party preview-versus-launch posture from the same bounded launch checklist instead of relying on hidden operator-only status. The live website lane now also owns route-aware title/description/canonical/Open-Graph/Twitter metadata for the real `hypertwist.app` marketing surface so deployed public pages no longer remain on a single generic SPA title/description, plus first-party `robots.txt` and `sitemap.xml` assets for the public route set while keeping `/app`, `/login`, and `/register` out of crawler posture. The real `check-runtime-readiness` CLI is now also exercised against the checked-in production example env files, and a spawned `website/server` bootstrap proof now verifies the live same-origin process path from production-shaped env into `/health`, `/api/auth/health`, and built-shell serving. | | Browser-based operator/account dashboard | Implemented now | first-party `website/` app + shared auth/dashboard packet | A protected browser dashboard is now live for operator access, account state, download posture, browser-access boundary explanation, notices review, and bounded billing/entitlement status. It reuses the shared SuperTokens auth posture proven in FamiliarOS and ScriptoriumAI while remaining HyperTwist-specific in product content and boundary claims, the current auth-health surface now truthfully distinguishes configured versus reachable or ready shared-core posture while exposing fallback-active reason instead of hardcoding readiness, and the same dashboard now also surfaces launch-readiness truth for download URLs, checkout links, source/notices URLs, billing-secret/map configuration, and local-versus-public runtime deployment posture. Focused frontend coverage now also protects deep-link login redirect preservation, safe `next`-path normalization across auth entry points, fallback/email auth-bootstrap normalization, login/register continuation behavior, public download-gating behavior, protected-route/shell behavior, real lazy-route tree behavior for key public and protected paths, top-level app-bootstrap and SuperTokens-wrapper posture, login/register unhappy-path and OAuth-button behavior, support-topic fallback routing when live checkout is not configured, desktop-link verify-url/dashboard readiness behavior, and explicit `noindex,nofollow` posture on protected/auth browser surfaces. The validation lane now also has a bounded signed test-session harness under `TEST_MODE=testing` that proves `/api/auth/me` and `/api/auth/desktop-link` behavior through the live spawned auth-server process without widening production auth posture. | | Desktop download posture and browser-to-desktop pairing | Implemented now | first-party `website/` app + `website/server` desktop-link endpoints | Public download targets, dashboard-side release posture, and short-lived desktop-link token generation/verification are now first-party owned. The current server posture now enforces exact website-origin matching, bounded per-user issuance, one-time token consumption, and billing-backed plan/download entitlement resolution with focused `website/server` tests green on `2026-06-22`, and the verify handshake now returns the same resolved download-entitlement posture the dashboard sees instead of only identity plus plan/role. The public `/download` page now keeps raw download URLs behind the protected dashboard instead of exposing them directly, preserves requested platform continuity through `/app/downloads?platform=...`, and surfaces that requested target again after auth handoff inside the protected release lane. Both the public and protected download surfaces now also carry first-party rollout steps plus release/notices/source references so the desktop setup lane is more than a generic link bucket. Actual release URLs remain deployment configuration rather than hardcoded product truth. | | Paddle-ready pricing and billing webhook seam | Implemented now | first-party `website/` app + `website/server` billing endpoint | The public pricing surface now exists with plan structure, checkout-link configuration seams, and the same `/api/billing/paddle/webhook` endpoint family used by the broader product website lane. The current server now verifies `Paddle-Signature` against `PADDLE_WEBHOOK_SECRET` using the documented raw-body HMAC flow, persists a bounded first-party billing state file, and applies verified Paddle events into account/download entitlement state that the browser dashboard consumes, with focused `website/server` tests green on `2026-06-22`. A spawned live-process proof now also verifies that a real signed webhook updates processed-event health and persisted billing state through the actual auth-server runtime, not only helper-level store tests, and transaction events no longer leak their id into stored `subscriptionId` state. Production checkout URLs, secret management, and broader operator/admin billing workflows remain deployment/application tasks, not shipped-code omissions. | diff --git a/docs/v6_5_deep_manual_pack/HyperTwist/ROADMAP.md b/docs/v6_5_deep_manual_pack/HyperTwist/ROADMAP.md index 62eea2f..111ecc9 100644 --- a/docs/v6_5_deep_manual_pack/HyperTwist/ROADMAP.md +++ b/docs/v6_5_deep_manual_pack/HyperTwist/ROADMAP.md @@ -252,7 +252,9 @@ Current consolidated milestone snapshot: `noindex,nofollow`, plus richer first-party release guidance on both the public `/download` and protected `/app/downloads` surfaces so rollout steps, release references, and source/notices posture stay attached to the desktop - distribution lane itself, + distribution lane itself, plus first-party `robots.txt` and `sitemap.xml` + assets for the public route set while keeping `/app`, `/login`, and + `/register` out of crawler posture, persists a bounded first-party billing-state file, applies verified Paddle events into account/download entitlement state, and surfaces that resolved billing/download posture back through `/api/auth/me`, the protected browser diff --git a/website/README.md b/website/README.md index bc273fa..f0066ca 100644 --- a/website/README.md +++ b/website/README.md @@ -10,6 +10,7 @@ First-party `hypertwist.app` surface for HyperTwist: - dashboard-side launch-readiness surface for download, checkout, auth, and notice configuration - public launch-status callouts across pricing, download, and notices surfaces - route-aware metadata, canonical, Open Graph, Twitter-card, and protected-route noindex posture for the live `hypertwist.app` surface +- first-party `robots.txt` plus `sitemap.xml` for the public crawlable route set while keeping `/app`, `/login`, and `/register` out of crawler posture - runtime diagnostics that distinguish local, mixed, and public auth deployment posture - public open-source notices surface required by HyperTwist's MPL distribution doctrine diff --git a/website/public/robots.txt b/website/public/robots.txt new file mode 100644 index 0000000..48e7bbb --- /dev/null +++ b/website/public/robots.txt @@ -0,0 +1,7 @@ +User-agent: * +Allow: / +Disallow: /app +Disallow: /login +Disallow: /register + +Sitemap: https://hypertwist.app/sitemap.xml diff --git a/website/public/sitemap.xml b/website/public/sitemap.xml new file mode 100644 index 0000000..67a89b1 --- /dev/null +++ b/website/public/sitemap.xml @@ -0,0 +1,39 @@ + + + + https://hypertwist.app/ + + + https://hypertwist.app/about + + + https://hypertwist.app/resources + + + https://hypertwist.app/docs + + + https://hypertwist.app/support + + + https://hypertwist.app/changelog + + + https://hypertwist.app/pricing + + + https://hypertwist.app/download + + + https://hypertwist.app/open-source-notices + + + https://hypertwist.app/privacy + + + https://hypertwist.app/terms + + + https://hypertwist.app/shipping-payment + +