mirror of
https://github.com/supermemoryai/supermemory.git
synced 2026-08-28 05:25:33 +00:00
New Version of Supermemory Consumer App
This commit is contained in:
parent
5b35e3ea78
commit
39003aff23
634 changed files with 22332 additions and 64226 deletions
|
|
@ -1 +0,0 @@
|
|||
i am building an app called "supermemory"
|
||||
41
.gitignore
vendored
41
.gitignore
vendored
|
|
@ -1,12 +1,8 @@
|
|||
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
|
||||
bun.lockb
|
||||
tests/
|
||||
new-api/
|
||||
|
||||
packages/scripts/*
|
||||
# Dependencies
|
||||
node_modules/
|
||||
/.pnp
|
||||
node_modules
|
||||
.pnp
|
||||
.pnp.js
|
||||
|
||||
# Local env files
|
||||
|
|
@ -15,10 +11,9 @@ node_modules/
|
|||
.env.development.local
|
||||
.env.test.local
|
||||
.env.production.local
|
||||
.env*.local
|
||||
|
||||
# Testing
|
||||
/coverage
|
||||
coverage
|
||||
|
||||
# Turbo
|
||||
.turbo
|
||||
|
|
@ -27,10 +22,11 @@ node_modules/
|
|||
.vercel
|
||||
|
||||
# Build Outputs
|
||||
/.next
|
||||
.next/
|
||||
out/
|
||||
build
|
||||
/dist
|
||||
dist
|
||||
|
||||
|
||||
# Debug
|
||||
npm-debug.log*
|
||||
|
|
@ -40,28 +36,3 @@ yarn-error.log*
|
|||
# Misc
|
||||
.DS_Store
|
||||
*.pem
|
||||
|
||||
# IDE specific files
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# Sensitive data
|
||||
*.key
|
||||
**/credentials.*
|
||||
**/secrets.*
|
||||
config.private.*
|
||||
|
||||
# Cache
|
||||
.cache/
|
||||
.npm
|
||||
.eslintcache
|
||||
|
||||
# Local database files
|
||||
*.sqlite
|
||||
*.db
|
||||
|
||||
# Personal notes/todos
|
||||
TODO.md
|
||||
NOTES.md
|
||||
|
|
|
|||
0
.gitmodules
vendored
0
.gitmodules
vendored
2
.npmrc
2
.npmrc
|
|
@ -1,2 +0,0 @@
|
|||
save-exact=true
|
||||
engine-strict=true
|
||||
17
.vscode/settings.json
vendored
17
.vscode/settings.json
vendored
|
|
@ -1,17 +0,0 @@
|
|||
{
|
||||
"editor.formatOnSave": true,
|
||||
"editor.defaultFormatter": "esbenp.prettier-vscode",
|
||||
"editor.codeActionsOnSave": {
|
||||
"source.fixAll.eslint": "always"
|
||||
},
|
||||
"sqltools.connections": [
|
||||
{
|
||||
"previewLimit": 50,
|
||||
"server": "localhost",
|
||||
"driver": "PostgreSQL",
|
||||
"name": "SupermemoryDB",
|
||||
"connectString": "postgresql://dhravya:DPOSTGRES99%40c22%23hLab27postgres@168.119.111.189/supermemory"
|
||||
}
|
||||
],
|
||||
"millionLint.disableOutdatedVersionMessage": true
|
||||
}
|
||||
119
CLAUDE.md
Normal file
119
CLAUDE.md
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Repository Structure
|
||||
|
||||
This is a **Turbo monorepo** containing multiple applications and shared packages:
|
||||
|
||||
### Applications (`apps/`)
|
||||
- **`web/`** - Next.js web application
|
||||
|
||||
## Development Commands
|
||||
|
||||
### Root Level (Monorepo)
|
||||
- `bun run dev` - Start all applications in development mode
|
||||
- `bun run build` - Build all applications
|
||||
- `bun run check-types` - Run TypeScript checks across all apps
|
||||
- `bun run format-lint` - Format and lint code using Biome
|
||||
|
||||
### Web Application (`apps/web/`)
|
||||
- `bun run dev` - Start Next.js development server
|
||||
- `bun run build` - Build Next.js application
|
||||
- `bun run lint` - Run Next.js linting
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Core Technology Stack
|
||||
- **Runtime**: Next.js (web)
|
||||
- **Framework**: Next.js (web)
|
||||
- **Language**: TypeScript throughout
|
||||
- **Package Manager**: Bun
|
||||
- **Monorepo**: Turbo
|
||||
- **Authentication**: Better Auth
|
||||
- **Monitoring**: Sentry
|
||||
|
||||
### API Application (Primary Backend)
|
||||
The API serves as the core backend with these key features:
|
||||
|
||||
**Key API Routes**
|
||||
- `/v3/memories` - CRUD operations for documents/memories
|
||||
- `/v3/search` - Semantic search across indexed content
|
||||
- `/v3/connections` - External service integrations (Google Drive, Notion, OneDrive)
|
||||
- `/v3/settings` - Organization and user settings
|
||||
- `/v3/analytics` - Usage analytics and reporting
|
||||
- `/api/auth/*` - Authentication endpoints
|
||||
|
||||
### Web Application
|
||||
Next.js application providing user interface for:
|
||||
|
||||
## Key Libraries & Dependencies
|
||||
|
||||
### Shared Dependencies
|
||||
- `better-auth` - Authentication system with organization support
|
||||
- `drizzle-orm` - Database ORM
|
||||
- `zod` - Schema validation
|
||||
- `hono` - Web framework (API & MCP)
|
||||
- `@sentry/*` - Error monitoring
|
||||
- `turbo` - Monorepo build system
|
||||
|
||||
### Web-Specific
|
||||
- `next` - React framework
|
||||
- `@radix-ui/*` - UI components
|
||||
- `@tanstack/react-query` - Data fetching
|
||||
- `recharts` - Analytics visualization
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Content Processing Pipeline
|
||||
All content goes through the `IngestContentWorkflow` which handles:
|
||||
- Content type detection and extraction
|
||||
- AI-powered summarization and automatic tagging
|
||||
- Vector embedding generation using Cloudflare AI
|
||||
- Chunking for semantic search optimization
|
||||
- Space relationship management
|
||||
|
||||
### Environment Configuration
|
||||
- Uses `wrangler.jsonc` for Cloudflare Workers configuration
|
||||
- Supports staging and production environments
|
||||
- Requires Cloudflare bindings: Hyperdrive (DB), AI, KV storage, Workflows
|
||||
- Cron triggers every 4 hours for connection imports
|
||||
|
||||
### Error Handling & Monitoring
|
||||
- HTTPException for consistent API error responses
|
||||
- Sentry integration with user and organization context
|
||||
- Custom logging that filters analytics noise
|
||||
|
||||
## Code Quality & Standards
|
||||
|
||||
### Linting & Formatting
|
||||
- **Biome** used for linting and formatting across the monorepo
|
||||
- Run `bun run format-lint` to format and lint all code
|
||||
- Configuration in `biome.json` at repository root
|
||||
|
||||
### TypeScript
|
||||
- Strict TypeScript configuration with `@total-typescript/tsconfig`
|
||||
- Type checking with `bun run check-types`
|
||||
- Cloudflare Workers type generation with `cf-typegen`
|
||||
|
||||
### Database Management
|
||||
- Drizzle ORM with schema located in shared packages
|
||||
- Database migrations handled through Drizzle Kit
|
||||
- Schema types automatically generated and shared
|
||||
|
||||
## Security & Best Practices
|
||||
|
||||
### Authentication
|
||||
- Better Auth handles user authentication and organization management
|
||||
- API key authentication for external access
|
||||
- Role-based access control within organizations
|
||||
|
||||
### Data Handling
|
||||
- Content hashing to prevent duplicate processing
|
||||
- Secure handling of external service credentials
|
||||
- Automatic content type detection and validation
|
||||
|
||||
### Deployment
|
||||
- Cloudflare Workers for scalable serverless deployment
|
||||
- Source map uploads to Sentry for production debugging
|
||||
- Environment-specific configuration management
|
||||
37
Dockerfile
37
Dockerfile
|
|
@ -1,37 +0,0 @@
|
|||
FROM postgres:17
|
||||
|
||||
# Install build dependencies
|
||||
RUN apt-get update && apt-get install -y \
|
||||
build-essential \
|
||||
git \
|
||||
curl \
|
||||
pkg-config \
|
||||
libssl-dev \
|
||||
libclang-dev \
|
||||
llvm-dev \
|
||||
postgresql-server-dev-all \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install pgvector
|
||||
RUN git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git \
|
||||
&& cd pgvector \
|
||||
&& make \
|
||||
&& make install
|
||||
|
||||
# Install Rust
|
||||
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
|
||||
ENV PATH="/root/.cargo/bin:${PATH}"
|
||||
|
||||
# Install and initialize pgrx
|
||||
RUN cargo install cargo-pgrx --version 0.12.5 --locked && \
|
||||
cargo pgrx init --pg17 pg_config
|
||||
|
||||
# Clone and install pgvectorscale
|
||||
RUN cd /tmp && \
|
||||
git clone --branch 0.5.0 https://github.com/timescale/pgvectorscale && \
|
||||
cd pgvectorscale/pgvectorscale && \
|
||||
cargo pgrx install --release
|
||||
|
||||
# Create initialization script to enable both extensions
|
||||
RUN echo 'CREATE EXTENSION IF NOT EXISTS vector;' > /docker-entrypoint-initdb.d/01-init-vector.sql && \
|
||||
echo 'CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;' > /docker-entrypoint-initdb.d/02-init-vectorscale.sql
|
||||
132
README.md
132
README.md
|
|
@ -1,89 +1,99 @@
|
|||
<div align="center">
|
||||
<div align="center" style="padding-bottom:10px;padding-top:10px">
|
||||
<img src="logo.svg" alt="supermemory Logo" width="400" />
|
||||
<p><strong>The Memory API for the AI era</strong></p>
|
||||
</div>
|
||||
|
||||
> [!WARNING]
|
||||
> This repo contains archived code for supermemory v1 and no longer receives updates or support.
|
||||
<div align="center" style="padding-bottom:10px;padding-top:10px">
|
||||
<img src="apps/web/public/landing-page.jpeg" alt="supermemory" width="100%" />
|
||||
</div>
|
||||
|
||||
## 🧠 What is supermemory?
|
||||
## ✨ Features
|
||||
|
||||
supermemory is a powerful, developer-friendly API that seamlessly integrates external knowledge into your AI applications. It serves as the perfect memory layer for your AI stack, providing semantic search and retrieval capabilities that enhance your models with relevant context.
|
||||
### Core Functionality
|
||||
- **Add Memories from Any Content**: Easily add memories from URLs, PDFs, and plain text—just paste, upload, or link.
|
||||
- **Chat with Your Memories**: Converse with your stored content using natural language chat.
|
||||
- **Supermemory MCP Integration**: Seamlessly connect with all major AI tools (Claude, Cursor, etc.) via Supermemory MCP.
|
||||
- **Graph View for All Memories**: Visualize and explore your memories and their connections in an interactive graph mode.
|
||||
|
||||
With supermemory, you can:
|
||||
## 🏗️ Architecture
|
||||
|
||||
- **Store and organize knowledge** in a searchable database that understands meaning, not just keywords
|
||||
- **Enhance AI responses** with accurate, up-to-date information from your data
|
||||
- **Eliminate hallucinations** by grounding AI outputs in your trusted content
|
||||
- **Connect to any source** with pre-built integrations for websites, PDFs, images, and more
|
||||
This is a **Turborepo monorepo**
|
||||
|
||||
## ✨ Key Features
|
||||
### Technology Stack
|
||||
- **Frontend**: Next.js 15 with React 19
|
||||
- **Backend**: Hono API framework on Cloudflare Workers
|
||||
- **Database**: PostgreSQL with Drizzle ORM
|
||||
- **Authentication**: Better Auth with organization support
|
||||
- **Package Manager**: Bun
|
||||
- **Monorepo**: Turbo for build optimization
|
||||
- **Styling**: Tailwind CSS with Radix UI components
|
||||
- **Monitoring**: Sentry for error tracking and performance monitoring
|
||||
|
||||
- **Universal Content Handling**: Automatically process and index content from URLs, PDFs, text, and more
|
||||
- **Semantic Search**: Find information based on meaning, not just keyword matching
|
||||
- **Advanced Filtering**: Organize and retrieve information using metadata, categories, and user partitioning
|
||||
- **Query Enhancement**: Rewriting and reranking for more relevant results
|
||||
- **Simple Integration**: Clean, consistent API with SDKs for TypeScript and Python
|
||||
### Project Structure
|
||||
```
|
||||
├── apps/
|
||||
│ └── web/ # Next.js web application
|
||||
├── packages/ # Shared packages and utilities
|
||||
├── CLAUDE.md # Development guidelines for AI assistants
|
||||
├── turbo.json # Turborepo configuration
|
||||
└── package.json # Root package configuration
|
||||
```
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
Getting started with supermemory takes just minutes:
|
||||
### Prerequisites
|
||||
- **Bun** package manager
|
||||
|
||||
1. Sign up at [console.supermemory.ai](https://console.supermemory.ai)
|
||||
2. Create your API key
|
||||
3. Start adding and querying content
|
||||
### Installation
|
||||
|
||||
```javascript
|
||||
// Install: npm install supermemory
|
||||
import { supermemory } from 'supermemory';
|
||||
1. **Clone the repository**
|
||||
```bash
|
||||
git clone https://github.com/supermemoryai/supermemory-app.git
|
||||
cd supermemory
|
||||
```
|
||||
|
||||
const client = new supermemory({
|
||||
apiKey: 'YOUR_API_KEY',
|
||||
});
|
||||
2. **Install dependencies**
|
||||
```bash
|
||||
bun install
|
||||
```
|
||||
|
||||
// Add content to your knowledge base
|
||||
await client.memory.create({
|
||||
content: "https://en.wikipedia.org/wiki/Artificial_intelligence",
|
||||
metadata: {
|
||||
source: "wikipedia",
|
||||
category: "AI"
|
||||
}
|
||||
});
|
||||
3. **Environment Setup**
|
||||
|
||||
Create environment files for each app:
|
||||
```bash
|
||||
# Copy environment templates
|
||||
cp apps/web/.env.example apps/web/.env.local
|
||||
```
|
||||
|
||||
// Query your knowledge base
|
||||
const results = await client.search.create({
|
||||
q: "What are the ethical considerations in AI development?",
|
||||
limit: 5
|
||||
});
|
||||
### Development
|
||||
|
||||
#### Start all applications in development mode:
|
||||
```bash
|
||||
bun run dev
|
||||
```
|
||||
|
||||
## 📚 Documentation
|
||||
This will start:
|
||||
- Web app at `http://localhost:3000`
|
||||
- API endpoints available through the web app
|
||||
|
||||
We've created comprehensive documentation to help you get the most out of supermemory:
|
||||
|
||||
- [Quick Start Guide](https://docs.supermemory.ai/quickstart/overview)
|
||||
- [API Reference](https://docs.supermemory.ai/api-reference)
|
||||
- [SDK Documentation](https://docs.supermemory.ai/sdks)
|
||||
- [Use Cases & Examples](https://docs.supermemory.ai/overview/use-cases)
|
||||
## 🧪 Development Workflow
|
||||
|
||||
## 🌟 Use Cases
|
||||
### Code Quality
|
||||
- **Linting & Formatting**: Uses Biome for consistent code style
|
||||
- **Type Safety**: Strict TypeScript configuration across all packages
|
||||
|
||||
supermemory powers a wide range of AI-enhanced applications:
|
||||
## 🤝 Contributing
|
||||
|
||||
- **RAG (Retrieval Augmented Generation)**: Enhance LLM outputs with accurate data
|
||||
- **Knowledge Bases & Documentation**: Create intelligent, searchable repositories
|
||||
- **Customer Support**: Build chatbots with access to your support documentation
|
||||
- **Research Assistants**: Query across papers, notes, and references
|
||||
- **Content Management**: Organize and retrieve multimedia content semantically
|
||||
### Development Guidelines
|
||||
- Follow the code style enforced by Biome
|
||||
- Write tests for new features
|
||||
- Update documentation when adding new functionality
|
||||
- Ensure all checks pass before submitting PRs
|
||||
|
||||
## 💬 Support
|
||||
|
||||
Have questions or feedback? We're here to help:
|
||||
- Email: [dhravya@supermemory.com](mailto:dhravya@supermemory.com)
|
||||
- Documentation: [docs.supermemory.ai](https://docs.supermemory.ai)
|
||||
## 💬 Support & Community
|
||||
|
||||
## 🔄 Updates & Roadmap
|
||||
- **Issues**: [GitHub Issues](https://github.com/supermemoryai/supermemory-app/issues)
|
||||
- **Email**: [dhravya@supermemory.com](mailto:dhravya@supermemory.com)
|
||||
- **Twitter**: [@supermemoryai](https://x.com/supermemoryai)
|
||||
|
||||
Stay up to date with the latest improvements:
|
||||
- [Changelog](https://docs.supermemory.ai/changelog/overview)
|
||||
- [X](https://x.com/supermemoryai)
|
||||
|
|
|
|||
|
|
@ -1,90 +0,0 @@
|
|||
# Supermemory self-hosting guide
|
||||
|
||||
## Local Setup
|
||||
|
||||
### 1. Database Setup
|
||||
|
||||
To spin up the database locally, use Docker Compose:
|
||||
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
This will start a PostgreSQL database with pgvector extension at `localhost:5432`.
|
||||
|
||||
### 2. Database Migrations
|
||||
|
||||
To generate a migration:
|
||||
|
||||
```bash
|
||||
bun run generate-migration
|
||||
```
|
||||
|
||||
To apply migrations:
|
||||
|
||||
```bash
|
||||
bun run migrate:local
|
||||
```
|
||||
|
||||
> Note: You MUST use the drizzle-orm functions exported from `packages/db` for interacting with the database. Not using them will cause type errors that are hard to debug.
|
||||
|
||||
### 3. Environment Variables
|
||||
|
||||
#### Backend (`apps/backend/.env` and `apps/backend/.dev.vars`):
|
||||
|
||||
```env
|
||||
WORKOS_API_KEY=your_workos_api_key
|
||||
WORKOS_CLIENT_ID=your_workos_client_id
|
||||
WORKOS_COOKIE_PASSWORD=your_cookie_password
|
||||
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/supermemory"
|
||||
CONTENT_WORKFLOW=your_content_workflow
|
||||
GEMINI_API_KEY=your_gemini_api_key
|
||||
NODE_ENV=development
|
||||
OPEN_AI_API_KEY=your_openai_api_key
|
||||
BRAINTRUST_API_KEY=your_braintrust_api_key
|
||||
RESEND_API_KEY=your_resend_api_key
|
||||
TURNSTILE_SECRET_KEY=your_turnstile_secret_key
|
||||
```
|
||||
|
||||
#### Web (`apps/web/.env` and `apps/web/.dev.vars`):
|
||||
|
||||
```env
|
||||
WORKOS_CLIENT_ID=your_workos_client_id
|
||||
WORKOS_API_KEY=your_workos_api_key
|
||||
WORKOS_REDIRECT_URI="http://localhost:3000/callback"
|
||||
WORKOS_COOKIE_PASSWORD=your_cookie_password
|
||||
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/supermemorydhravya"
|
||||
|
||||
CLOUDFLARE_ACCOUNT_ID=your_cloudflare_account_id
|
||||
R2_ACCESS_KEY_ID=your_r2_access_key_id
|
||||
R2_SECRET_ACCESS_KEY=your_r2_secret_access_key
|
||||
|
||||
BACKEND_URL=http://localhost:8787
|
||||
OPENAI_API_KEY=your_openai_api_key
|
||||
NOTION_CLIENT_ID=your_notion_client_id
|
||||
NOTION_CLIENT_SECRET=your_notion_client_secret
|
||||
|
||||
NODE_ENV=development
|
||||
STRIPE_CHECKOUT_KEY=your_stripe_checkout_key
|
||||
STRIPE_WEBHOOK_SECRET=your_stripe_webhook_secret
|
||||
```
|
||||
|
||||
You also need to update the Wrangler config for the web app and backend to your own account's resources on Cloudflare.
|
||||
|
||||
### 4. Schema Changes
|
||||
|
||||
To edit the database schema, modify the files in `packages/db/schema.ts`, and then repeat the steps in the [Database Migrations](#2-database-migrations) section.
|
||||
|
||||
### 5. Running the Application
|
||||
|
||||
1. Install dependencies:
|
||||
|
||||
```bash
|
||||
bun install
|
||||
```
|
||||
|
||||
2. Start the development servers:
|
||||
|
||||
```bash
|
||||
bun run dev
|
||||
```
|
||||
35
apps/backend/.gitignore
vendored
35
apps/backend/.gitignore
vendored
|
|
@ -1,35 +0,0 @@
|
|||
# prod
|
||||
dist/
|
||||
.dev.vars
|
||||
*.vars
|
||||
|
||||
# dev
|
||||
.yarn/
|
||||
!.yarn/releases
|
||||
.vscode/*
|
||||
!.vscode/launch.json
|
||||
!.vscode/*.code-snippets
|
||||
.idea/workspace.xml
|
||||
.idea/usage.statistics.xml
|
||||
.idea/shelf
|
||||
|
||||
# deps
|
||||
node_modules/
|
||||
.wrangler
|
||||
|
||||
# env
|
||||
.env
|
||||
.env.production
|
||||
.dev.vars
|
||||
|
||||
# logs
|
||||
logs/
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
lerna-debug.log*
|
||||
|
||||
# misc
|
||||
.DS_Store
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
```
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
```
|
||||
npm run deploy
|
||||
```
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
import { defineConfig } from "drizzle-kit";
|
||||
|
||||
export default defineConfig({
|
||||
dialect: "postgresql",
|
||||
schema: "../../packages/db",
|
||||
out: "./drizzle",
|
||||
dbCredentials: {
|
||||
url: process.env.PROD_DATABASE_URL!,
|
||||
},
|
||||
});
|
||||
|
|
@ -1,18 +0,0 @@
|
|||
import { config } from "dotenv";
|
||||
import { defineConfig } from "drizzle-kit";
|
||||
import process from "process";
|
||||
|
||||
config();
|
||||
|
||||
if (process.env.NODE_ENV !== "production" && !process.env.DATABASE_URL) {
|
||||
throw new Error("DATABASE_URL is not set");
|
||||
}
|
||||
|
||||
export default defineConfig({
|
||||
dialect: "postgresql",
|
||||
schema: "../../packages/db",
|
||||
out: "./drizzle",
|
||||
dbCredentials: {
|
||||
url: process.env.DATABASE_URL!,
|
||||
},
|
||||
});
|
||||
|
|
@ -1,189 +0,0 @@
|
|||
CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS "chat_threads" (
|
||||
"id" bigserial PRIMARY KEY NOT NULL,
|
||||
"uuid" varchar(36) NOT NULL,
|
||||
"firstMessage" text NOT NULL,
|
||||
"user_id" integer NOT NULL,
|
||||
"messages" jsonb NOT NULL,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
CONSTRAINT "chat_threads_uuid_unique" UNIQUE("uuid")
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "chunks" (
|
||||
"id" serial PRIMARY KEY NOT NULL,
|
||||
"document_id" integer NOT NULL,
|
||||
"text_content" text,
|
||||
"order_in_document" integer NOT NULL,
|
||||
"embeddings" vector(1536),
|
||||
"metadata" jsonb,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "content_to_space" (
|
||||
"content_id" integer NOT NULL,
|
||||
"space_id" integer NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "document_type" (
|
||||
"type" text PRIMARY KEY NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "documents" (
|
||||
"id" bigserial PRIMARY KEY NOT NULL,
|
||||
"uuid" varchar(36) NOT NULL,
|
||||
"url" text,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone,
|
||||
"type" text NOT NULL,
|
||||
"title" text,
|
||||
"description" text,
|
||||
"og_image" text,
|
||||
"raw" text,
|
||||
"user_id" integer NOT NULL,
|
||||
"content" text,
|
||||
CONSTRAINT "documents_uuid_unique" UNIQUE("uuid")
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "job" (
|
||||
"id" serial PRIMARY KEY NOT NULL,
|
||||
"user_id" integer NOT NULL,
|
||||
"url" text NOT NULL,
|
||||
"status" text NOT NULL,
|
||||
"attempts" integer DEFAULT 0 NOT NULL,
|
||||
"lastAttemptAt" timestamp with time zone,
|
||||
"error" text,
|
||||
"created_at" timestamp with time zone,
|
||||
"updated_at" timestamp with time zone
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "space_access" (
|
||||
"space_id" integer,
|
||||
"user_email" varchar(512),
|
||||
"status" text
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "space_access_status" (
|
||||
"status" text PRIMARY KEY NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "space_members" (
|
||||
"spaceId" integer NOT NULL,
|
||||
"user_id" integer NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "spaces" (
|
||||
"id" bigserial PRIMARY KEY NOT NULL,
|
||||
"uuid" varchar(36) NOT NULL,
|
||||
"name" text NOT NULL,
|
||||
"created_at" timestamp DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp DEFAULT now() NOT NULL,
|
||||
"ownerId" integer NOT NULL,
|
||||
"is_public" boolean DEFAULT false NOT NULL,
|
||||
CONSTRAINT "spaces_uuid_unique" UNIQUE("uuid")
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE IF NOT EXISTS "users" (
|
||||
"id" serial PRIMARY KEY NOT NULL,
|
||||
"uuid" varchar(36) NOT NULL,
|
||||
"email" text NOT NULL,
|
||||
"first_name" text,
|
||||
"last_name" text,
|
||||
"email_verified" boolean DEFAULT false NOT NULL,
|
||||
"profile_picture_url" text,
|
||||
"telegram_id" varchar(255),
|
||||
"has_onboarded" integer DEFAULT 0 NOT NULL,
|
||||
"created_at" timestamp DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp DEFAULT now() NOT NULL,
|
||||
CONSTRAINT "users_uuid_unique" UNIQUE("uuid"),
|
||||
CONSTRAINT "users_email_unique" UNIQUE("email")
|
||||
);
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "chat_threads" ADD CONSTRAINT "chat_threads_user_id_users_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "chunks" ADD CONSTRAINT "chunks_document_id_documents_id_fk" FOREIGN KEY ("document_id") REFERENCES "public"."documents"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "content_to_space" ADD CONSTRAINT "content_to_space_content_id_documents_id_fk" FOREIGN KEY ("content_id") REFERENCES "public"."documents"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "content_to_space" ADD CONSTRAINT "content_to_space_space_id_spaces_id_fk" FOREIGN KEY ("space_id") REFERENCES "public"."spaces"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "documents" ADD CONSTRAINT "documents_type_document_type_type_fk" FOREIGN KEY ("type") REFERENCES "public"."document_type"("type") ON DELETE no action ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "documents" ADD CONSTRAINT "documents_user_id_users_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE restrict ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "job" ADD CONSTRAINT "job_user_id_users_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "space_access" ADD CONSTRAINT "space_access_space_id_spaces_id_fk" FOREIGN KEY ("space_id") REFERENCES "public"."spaces"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "space_access" ADD CONSTRAINT "space_access_status_space_access_status_status_fk" FOREIGN KEY ("status") REFERENCES "public"."space_access_status"("status") ON DELETE no action ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "space_members" ADD CONSTRAINT "space_members_spaceId_users_id_fk" FOREIGN KEY ("spaceId") REFERENCES "public"."users"("id") ON DELETE restrict ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "space_members" ADD CONSTRAINT "space_members_user_id_users_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE restrict ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "chat_threads_user_idx" ON "chat_threads" USING btree ("user_id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "chunk_id_idx" ON "chunks" USING btree ("id");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "chunk_document_id_idx" ON "chunks" USING btree ("document_id");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "embeddingIndex" ON "chunks" USING diskann ("embeddings" vector_cosine_ops);--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "content_id_space_id_unique" ON "content_to_space" USING btree ("content_id","space_id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "document_id_idx" ON "documents" USING btree ("id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "document_uuid_idx" ON "documents" USING btree ("uuid");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "document_type_idx" ON "documents" USING btree ("type");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "document_url_user_id_idx" ON "documents" USING btree ("url","user_id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "user_id_url_idx" ON "job" USING btree ("user_id","url");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "space_id_user_email_idx" ON "space_access" USING btree ("space_id","user_email");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "space_members_space_user_idx" ON "space_members" USING btree ("spaceId","user_id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "spaces_id_idx" ON "spaces" USING btree ("id");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "spaces_owner_id_idx" ON "spaces" USING btree ("ownerId");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "spaces_name_idx" ON "spaces" USING btree ("name");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "users_id_idx" ON "users" USING btree ("id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "users_uuid_idx" ON "users" USING btree ("uuid");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "users_email_idx" ON "users" USING btree ("email");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "users_name_idx" ON "users" USING btree ("first_name","last_name");--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "users_created_at_idx" ON "users" USING btree ("created_at");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "users_telegram_id_idx" ON "users" USING btree ("telegram_id");
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
-- Custom SQL migration file, put you code below! --
|
||||
INSERT INTO "document_type" (type) VALUES ('tweet'), ('page'), ('note') ON CONFLICT DO NOTHING;
|
||||
INSERT INTO "document_type" (type) VALUES ('document') ON CONFLICT DO NOTHING;
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
-- Active: 1732249624784@@_@5432@supermemorymain
|
||||
ALTER TABLE "documents" ADD COLUMN "is_successfully_processed" boolean DEFAULT false;
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
ALTER TABLE "documents" ALTER COLUMN "is_successfully_processed" DROP NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "space_access" ADD COLUMN "access_type" text DEFAULT 'read' NOT NULL;--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "document_raw_user_idx" ON "documents" USING btree ("raw","user_id");--> statement-breakpoint
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
CREATE TABLE IF NOT EXISTS "saved_spaces" (
|
||||
"user_id" integer NOT NULL,
|
||||
"space_id" integer NOT NULL,
|
||||
"saved_at" timestamp DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "saved_spaces" ADD CONSTRAINT "saved_spaces_user_id_users_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "saved_spaces" ADD CONSTRAINT "saved_spaces_space_id_spaces_id_fk" FOREIGN KEY ("space_id") REFERENCES "public"."spaces"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS "saved_spaces_user_space_idx" ON "saved_spaces" USING btree ("user_id","space_id");
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
-- Custom SQL migration file, put you code below! --
|
||||
INSERT INTO "space_access_status" (status) VALUES ('pending'), ('accepted'), ('rejected') ON CONFLICT DO NOTHING;
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
CREATE TABLE IF NOT EXISTS "waitlist" (
|
||||
"email" varchar(512) PRIMARY KEY NOT NULL
|
||||
);
|
||||
|
|
@ -1 +0,0 @@
|
|||
DROP INDEX IF EXISTS "document_url_user_id_idx";
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
-- Custom SQL migration file, put you code below! --
|
||||
INSERT INTO "document_type" (type) VALUES ('notion') ON CONFLICT DO NOTHING;
|
||||
|
|
@ -1 +0,0 @@
|
|||
ALTER TABLE "waitlist" ADD COLUMN "created_at" timestamp with time zone DEFAULT now() NOT NULL;
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
ALTER TABLE "documents" DROP CONSTRAINT "documents_user_id_users_id_fk";
|
||||
--> statement-breakpoint
|
||||
DO $$ BEGIN
|
||||
ALTER TABLE "documents" ADD CONSTRAINT "documents_user_id_users_id_fk" FOREIGN KEY ("user_id") REFERENCES "public"."users"("id") ON DELETE cascade ON UPDATE no action;
|
||||
EXCEPTION
|
||||
WHEN duplicate_object THEN null;
|
||||
END $$;
|
||||
|
|
@ -1 +0,0 @@
|
|||
ALTER TABLE "documents" ADD COLUMN "error_message" text;
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
-- Active: 1732308352274@@127.0.0.1@5432@supermemorydhravya
|
||||
ALTER TABLE "users" ADD COLUMN "last_api_key_generated_at" timestamp DEFAULT now();
|
||||
|
|
@ -1 +0,0 @@
|
|||
ALTER TABLE "documents" ADD COLUMN "content_hash" text;
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
ALTER TABLE "users" ADD COLUMN "stripe_customer_id" text;--> statement-breakpoint
|
||||
ALTER TABLE "users" ADD COLUMN "tier" text DEFAULT 'free' NOT NULL;
|
||||
|
|
@ -1 +0,0 @@
|
|||
ALTER TABLE "chunks" ALTER COLUMN "embeddings" SET DATA TYPE vector(768);
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
ALTER TABLE "chunks" ALTER COLUMN "embeddings" SET DATA TYPE vector(768);--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "documents_search_idx" ON "documents" USING gin ((
|
||||
setweight(to_tsvector('english', coalesce("content", '')),'A') ||
|
||||
setweight(to_tsvector('english', coalesce("title", '')),'B') ||
|
||||
setweight(to_tsvector('english', coalesce("description", '')),'C') ||
|
||||
setweight(to_tsvector('english', coalesce("url", '')),'D')
|
||||
));
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
DROP INDEX IF EXISTS "documents_search_idx";--> statement-breakpoint
|
||||
CREATE INDEX IF NOT EXISTS "documents_search_idx" ON "documents" USING gin ((
|
||||
setweight(to_tsvector('english', coalesce("content", '')),'A') ||
|
||||
setweight(to_tsvector('english', coalesce("title", '')),'B') ||
|
||||
setweight(to_tsvector('english', coalesce("description", '')),'C') ||
|
||||
setweight(to_tsvector('english', coalesce("url", '')),'D')
|
||||
));
|
||||
|
|
@ -1 +0,0 @@
|
|||
ALTER TABLE "documents" ADD COLUMN "metadata" jsonb;
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
-- Active: 1732308352274@@127.0.0.1@5432@supermemorydhravya
|
||||
DROP TABLE "job";
|
||||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -1,146 +0,0 @@
|
|||
{
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"entries": [
|
||||
{
|
||||
"idx": 0,
|
||||
"version": "7",
|
||||
"when": 1731880515136,
|
||||
"tag": "0000_odd_impossible_man",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 1,
|
||||
"version": "7",
|
||||
"when": 1731880538346,
|
||||
"tag": "0001_seed-types",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 2,
|
||||
"version": "7",
|
||||
"when": 1731970096987,
|
||||
"tag": "0002_skinny_princess_powerful",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 3,
|
||||
"version": "7",
|
||||
"when": 1732290500234,
|
||||
"tag": "0003_luxuriant_annihilus",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 4,
|
||||
"version": "7",
|
||||
"when": 1732291518161,
|
||||
"tag": "0004_early_rick_jones",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 5,
|
||||
"version": "7",
|
||||
"when": 1732292492979,
|
||||
"tag": "0005_create-access-types",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 6,
|
||||
"version": "7",
|
||||
"when": 1732308308233,
|
||||
"tag": "0006_wandering_grandmaster",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 7,
|
||||
"version": "7",
|
||||
"when": 1732476625236,
|
||||
"tag": "0007_fantastic_serpent_society",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 8,
|
||||
"version": "7",
|
||||
"when": 1732575691923,
|
||||
"tag": "0008_add-notion",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 9,
|
||||
"version": "7",
|
||||
"when": 1732693411536,
|
||||
"tag": "0009_milky_sleepwalker",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 10,
|
||||
"version": "7",
|
||||
"when": 1733037679877,
|
||||
"tag": "0010_heavy_preak",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 11,
|
||||
"version": "7",
|
||||
"when": 1733177542412,
|
||||
"tag": "0011_new_liz_osborn",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 12,
|
||||
"version": "7",
|
||||
"when": 1735033070115,
|
||||
"tag": "0012_small_mystique",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 13,
|
||||
"version": "7",
|
||||
"when": 1736500817155,
|
||||
"tag": "0013_sharp_hemingway",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 14,
|
||||
"version": "7",
|
||||
"when": 1736852938881,
|
||||
"tag": "0014_mighty_the_captain",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 15,
|
||||
"version": "7",
|
||||
"when": 1737920848112,
|
||||
"tag": "0015_perpetual_mauler",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 16,
|
||||
"version": "7",
|
||||
"when": 1739937938319,
|
||||
"tag": "0016_good_deathbird",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 18,
|
||||
"version": "7",
|
||||
"when": 1739939254444,
|
||||
"tag": "0018_past_inertia",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 19,
|
||||
"version": "7",
|
||||
"when": 1741025619581,
|
||||
"tag": "0019_vengeful_marten_broadcloak",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 20,
|
||||
"version": "7",
|
||||
"when": 1741026944027,
|
||||
"tag": "0020_opposite_steel_serpent",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,36 +0,0 @@
|
|||
{
|
||||
"name": "supermemory-backend",
|
||||
"scripts": {
|
||||
"dev": "bunx wrangler -v && wrangler dev",
|
||||
"deploy": "bunx wrangler deploy --minify",
|
||||
"generate-migration": "dotenv -- npx drizzle-kit generate",
|
||||
"migrate:local": "bun run ./scripts/migrate.ts",
|
||||
"migrate:prod": "NODE_ENV=production bun run ./scripts/migrate.ts",
|
||||
"tail": "bunx wrangler tail"
|
||||
},
|
||||
"dependencies": {
|
||||
"@ai-sdk/google": "^0.0.51",
|
||||
"@ai-sdk/openai": "^0.0.70",
|
||||
"@hono/swagger-ui": "^0.5.0",
|
||||
"@hono/zod-openapi": "^0.18.3",
|
||||
"@hono/zod-validator": "^0.4.1",
|
||||
"@supermemory/db": "workspace:*",
|
||||
"ai": "4.0.16",
|
||||
"compromise": "^14.14.2",
|
||||
"dotenv": "^16.4.5",
|
||||
"drizzle-kit": "^0.25.0",
|
||||
"drizzle-orm": "^0.34.1",
|
||||
"hono": "^4.6.4",
|
||||
"openai": "^4.68.4",
|
||||
"postgres": "^3.4.4",
|
||||
"uuid": "^11.0.1",
|
||||
"wrangler": "^3.111.0",
|
||||
"zod": "^3.23.8"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cloudflare/workers-types": "^4.20250124.3"
|
||||
},
|
||||
"overrides": {
|
||||
"iron-webcrypto": "^1.2.1"
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 15 KiB |
|
|
@ -1,853 +0,0 @@
|
|||
*, ::before, ::after {
|
||||
--tw-border-spacing-x: 0;
|
||||
--tw-border-spacing-y: 0;
|
||||
--tw-translate-x: 0;
|
||||
--tw-translate-y: 0;
|
||||
--tw-rotate: 0;
|
||||
--tw-skew-x: 0;
|
||||
--tw-skew-y: 0;
|
||||
--tw-scale-x: 1;
|
||||
--tw-scale-y: 1;
|
||||
--tw-pan-x: ;
|
||||
--tw-pan-y: ;
|
||||
--tw-pinch-zoom: ;
|
||||
--tw-scroll-snap-strictness: proximity;
|
||||
--tw-gradient-from-position: ;
|
||||
--tw-gradient-via-position: ;
|
||||
--tw-gradient-to-position: ;
|
||||
--tw-ordinal: ;
|
||||
--tw-slashed-zero: ;
|
||||
--tw-numeric-figure: ;
|
||||
--tw-numeric-spacing: ;
|
||||
--tw-numeric-fraction: ;
|
||||
--tw-ring-inset: ;
|
||||
--tw-ring-offset-width: 0px;
|
||||
--tw-ring-offset-color: #fff;
|
||||
--tw-ring-color: rgb(59 130 246 / 0.5);
|
||||
--tw-ring-offset-shadow: 0 0 #0000;
|
||||
--tw-ring-shadow: 0 0 #0000;
|
||||
--tw-shadow: 0 0 #0000;
|
||||
--tw-shadow-colored: 0 0 #0000;
|
||||
--tw-blur: ;
|
||||
--tw-brightness: ;
|
||||
--tw-contrast: ;
|
||||
--tw-grayscale: ;
|
||||
--tw-hue-rotate: ;
|
||||
--tw-invert: ;
|
||||
--tw-saturate: ;
|
||||
--tw-sepia: ;
|
||||
--tw-drop-shadow: ;
|
||||
--tw-backdrop-blur: ;
|
||||
--tw-backdrop-brightness: ;
|
||||
--tw-backdrop-contrast: ;
|
||||
--tw-backdrop-grayscale: ;
|
||||
--tw-backdrop-hue-rotate: ;
|
||||
--tw-backdrop-invert: ;
|
||||
--tw-backdrop-opacity: ;
|
||||
--tw-backdrop-saturate: ;
|
||||
--tw-backdrop-sepia: ;
|
||||
--tw-contain-size: ;
|
||||
--tw-contain-layout: ;
|
||||
--tw-contain-paint: ;
|
||||
--tw-contain-style: ;
|
||||
}
|
||||
|
||||
::backdrop {
|
||||
--tw-border-spacing-x: 0;
|
||||
--tw-border-spacing-y: 0;
|
||||
--tw-translate-x: 0;
|
||||
--tw-translate-y: 0;
|
||||
--tw-rotate: 0;
|
||||
--tw-skew-x: 0;
|
||||
--tw-skew-y: 0;
|
||||
--tw-scale-x: 1;
|
||||
--tw-scale-y: 1;
|
||||
--tw-pan-x: ;
|
||||
--tw-pan-y: ;
|
||||
--tw-pinch-zoom: ;
|
||||
--tw-scroll-snap-strictness: proximity;
|
||||
--tw-gradient-from-position: ;
|
||||
--tw-gradient-via-position: ;
|
||||
--tw-gradient-to-position: ;
|
||||
--tw-ordinal: ;
|
||||
--tw-slashed-zero: ;
|
||||
--tw-numeric-figure: ;
|
||||
--tw-numeric-spacing: ;
|
||||
--tw-numeric-fraction: ;
|
||||
--tw-ring-inset: ;
|
||||
--tw-ring-offset-width: 0px;
|
||||
--tw-ring-offset-color: #fff;
|
||||
--tw-ring-color: rgb(59 130 246 / 0.5);
|
||||
--tw-ring-offset-shadow: 0 0 #0000;
|
||||
--tw-ring-shadow: 0 0 #0000;
|
||||
--tw-shadow: 0 0 #0000;
|
||||
--tw-shadow-colored: 0 0 #0000;
|
||||
--tw-blur: ;
|
||||
--tw-brightness: ;
|
||||
--tw-contrast: ;
|
||||
--tw-grayscale: ;
|
||||
--tw-hue-rotate: ;
|
||||
--tw-invert: ;
|
||||
--tw-saturate: ;
|
||||
--tw-sepia: ;
|
||||
--tw-drop-shadow: ;
|
||||
--tw-backdrop-blur: ;
|
||||
--tw-backdrop-brightness: ;
|
||||
--tw-backdrop-contrast: ;
|
||||
--tw-backdrop-grayscale: ;
|
||||
--tw-backdrop-hue-rotate: ;
|
||||
--tw-backdrop-invert: ;
|
||||
--tw-backdrop-opacity: ;
|
||||
--tw-backdrop-saturate: ;
|
||||
--tw-backdrop-sepia: ;
|
||||
--tw-contain-size: ;
|
||||
--tw-contain-layout: ;
|
||||
--tw-contain-paint: ;
|
||||
--tw-contain-style: ;
|
||||
}
|
||||
|
||||
/*
|
||||
! tailwindcss v3.4.15 | MIT License | https://tailwindcss.com
|
||||
*/
|
||||
|
||||
/*
|
||||
1. Prevent padding and border from affecting element width. (https://github.com/mozdevs/cssremedy/issues/4)
|
||||
2. Allow adding a border to an element by just adding a border-width. (https://github.com/tailwindcss/tailwindcss/pull/116)
|
||||
*/
|
||||
|
||||
*,
|
||||
::before,
|
||||
::after {
|
||||
box-sizing: border-box;
|
||||
/* 1 */
|
||||
border-width: 0;
|
||||
/* 2 */
|
||||
border-style: solid;
|
||||
/* 2 */
|
||||
border-color: #e5e7eb;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
::before,
|
||||
::after {
|
||||
--tw-content: '';
|
||||
}
|
||||
|
||||
/*
|
||||
1. Use a consistent sensible line-height in all browsers.
|
||||
2. Prevent adjustments of font size after orientation changes in iOS.
|
||||
3. Use a more readable tab size.
|
||||
4. Use the user's configured `sans` font-family by default.
|
||||
5. Use the user's configured `sans` font-feature-settings by default.
|
||||
6. Use the user's configured `sans` font-variation-settings by default.
|
||||
7. Disable tap highlights on iOS
|
||||
*/
|
||||
|
||||
html,
|
||||
:host {
|
||||
line-height: 1.5;
|
||||
/* 1 */
|
||||
-webkit-text-size-adjust: 100%;
|
||||
/* 2 */
|
||||
-moz-tab-size: 4;
|
||||
/* 3 */
|
||||
-o-tab-size: 4;
|
||||
tab-size: 4;
|
||||
/* 3 */
|
||||
font-family: ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
|
||||
/* 4 */
|
||||
font-feature-settings: normal;
|
||||
/* 5 */
|
||||
font-variation-settings: normal;
|
||||
/* 6 */
|
||||
-webkit-tap-highlight-color: transparent;
|
||||
/* 7 */
|
||||
}
|
||||
|
||||
/*
|
||||
1. Remove the margin in all browsers.
|
||||
2. Inherit line-height from `html` so users can set them as a class directly on the `html` element.
|
||||
*/
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
/* 1 */
|
||||
line-height: inherit;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
/*
|
||||
1. Add the correct height in Firefox.
|
||||
2. Correct the inheritance of border color in Firefox. (https://bugzilla.mozilla.org/show_bug.cgi?id=190655)
|
||||
3. Ensure horizontal rules are visible by default.
|
||||
*/
|
||||
|
||||
hr {
|
||||
height: 0;
|
||||
/* 1 */
|
||||
color: inherit;
|
||||
/* 2 */
|
||||
border-top-width: 1px;
|
||||
/* 3 */
|
||||
}
|
||||
|
||||
/*
|
||||
Add the correct text decoration in Chrome, Edge, and Safari.
|
||||
*/
|
||||
|
||||
abbr:where([title]) {
|
||||
-webkit-text-decoration: underline dotted;
|
||||
text-decoration: underline dotted;
|
||||
}
|
||||
|
||||
/*
|
||||
Remove the default font size and weight for headings.
|
||||
*/
|
||||
|
||||
h1,
|
||||
h2,
|
||||
h3,
|
||||
h4,
|
||||
h5,
|
||||
h6 {
|
||||
font-size: inherit;
|
||||
font-weight: inherit;
|
||||
}
|
||||
|
||||
/*
|
||||
Reset links to optimize for opt-in styling instead of opt-out.
|
||||
*/
|
||||
|
||||
a {
|
||||
color: inherit;
|
||||
text-decoration: inherit;
|
||||
}
|
||||
|
||||
/*
|
||||
Add the correct font weight in Edge and Safari.
|
||||
*/
|
||||
|
||||
b,
|
||||
strong {
|
||||
font-weight: bolder;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Use the user's configured `mono` font-family by default.
|
||||
2. Use the user's configured `mono` font-feature-settings by default.
|
||||
3. Use the user's configured `mono` font-variation-settings by default.
|
||||
4. Correct the odd `em` font sizing in all browsers.
|
||||
*/
|
||||
|
||||
code,
|
||||
kbd,
|
||||
samp,
|
||||
pre {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;
|
||||
/* 1 */
|
||||
font-feature-settings: normal;
|
||||
/* 2 */
|
||||
font-variation-settings: normal;
|
||||
/* 3 */
|
||||
font-size: 1em;
|
||||
/* 4 */
|
||||
}
|
||||
|
||||
/*
|
||||
Add the correct font size in all browsers.
|
||||
*/
|
||||
|
||||
small {
|
||||
font-size: 80%;
|
||||
}
|
||||
|
||||
/*
|
||||
Prevent `sub` and `sup` elements from affecting the line height in all browsers.
|
||||
*/
|
||||
|
||||
sub,
|
||||
sup {
|
||||
font-size: 75%;
|
||||
line-height: 0;
|
||||
position: relative;
|
||||
vertical-align: baseline;
|
||||
}
|
||||
|
||||
sub {
|
||||
bottom: -0.25em;
|
||||
}
|
||||
|
||||
sup {
|
||||
top: -0.5em;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Remove text indentation from table contents in Chrome and Safari. (https://bugs.chromium.org/p/chromium/issues/detail?id=999088, https://bugs.webkit.org/show_bug.cgi?id=201297)
|
||||
2. Correct table border color inheritance in all Chrome and Safari. (https://bugs.chromium.org/p/chromium/issues/detail?id=935729, https://bugs.webkit.org/show_bug.cgi?id=195016)
|
||||
3. Remove gaps between table borders by default.
|
||||
*/
|
||||
|
||||
table {
|
||||
text-indent: 0;
|
||||
/* 1 */
|
||||
border-color: inherit;
|
||||
/* 2 */
|
||||
border-collapse: collapse;
|
||||
/* 3 */
|
||||
}
|
||||
|
||||
/*
|
||||
1. Change the font styles in all browsers.
|
||||
2. Remove the margin in Firefox and Safari.
|
||||
3. Remove default padding in all browsers.
|
||||
*/
|
||||
|
||||
button,
|
||||
input,
|
||||
optgroup,
|
||||
select,
|
||||
textarea {
|
||||
font-family: inherit;
|
||||
/* 1 */
|
||||
font-feature-settings: inherit;
|
||||
/* 1 */
|
||||
font-variation-settings: inherit;
|
||||
/* 1 */
|
||||
font-size: 100%;
|
||||
/* 1 */
|
||||
font-weight: inherit;
|
||||
/* 1 */
|
||||
line-height: inherit;
|
||||
/* 1 */
|
||||
letter-spacing: inherit;
|
||||
/* 1 */
|
||||
color: inherit;
|
||||
/* 1 */
|
||||
margin: 0;
|
||||
/* 2 */
|
||||
padding: 0;
|
||||
/* 3 */
|
||||
}
|
||||
|
||||
/*
|
||||
Remove the inheritance of text transform in Edge and Firefox.
|
||||
*/
|
||||
|
||||
button,
|
||||
select {
|
||||
text-transform: none;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Correct the inability to style clickable types in iOS and Safari.
|
||||
2. Remove default button styles.
|
||||
*/
|
||||
|
||||
button,
|
||||
input:where([type='button']),
|
||||
input:where([type='reset']),
|
||||
input:where([type='submit']) {
|
||||
-webkit-appearance: button;
|
||||
/* 1 */
|
||||
background-color: transparent;
|
||||
/* 2 */
|
||||
background-image: none;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
/*
|
||||
Use the modern Firefox focus style for all focusable elements.
|
||||
*/
|
||||
|
||||
:-moz-focusring {
|
||||
outline: auto;
|
||||
}
|
||||
|
||||
/*
|
||||
Remove the additional `:invalid` styles in Firefox. (https://github.com/mozilla/gecko-dev/blob/2f9eacd9d3d995c937b4251a5557d95d494c9be1/layout/style/res/forms.css#L728-L737)
|
||||
*/
|
||||
|
||||
:-moz-ui-invalid {
|
||||
box-shadow: none;
|
||||
}
|
||||
|
||||
/*
|
||||
Add the correct vertical alignment in Chrome and Firefox.
|
||||
*/
|
||||
|
||||
progress {
|
||||
vertical-align: baseline;
|
||||
}
|
||||
|
||||
/*
|
||||
Correct the cursor style of increment and decrement buttons in Safari.
|
||||
*/
|
||||
|
||||
::-webkit-inner-spin-button,
|
||||
::-webkit-outer-spin-button {
|
||||
height: auto;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Correct the odd appearance in Chrome and Safari.
|
||||
2. Correct the outline style in Safari.
|
||||
*/
|
||||
|
||||
[type='search'] {
|
||||
-webkit-appearance: textfield;
|
||||
/* 1 */
|
||||
outline-offset: -2px;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
/*
|
||||
Remove the inner padding in Chrome and Safari on macOS.
|
||||
*/
|
||||
|
||||
::-webkit-search-decoration {
|
||||
-webkit-appearance: none;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Correct the inability to style clickable types in iOS and Safari.
|
||||
2. Change font properties to `inherit` in Safari.
|
||||
*/
|
||||
|
||||
::-webkit-file-upload-button {
|
||||
-webkit-appearance: button;
|
||||
/* 1 */
|
||||
font: inherit;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
/*
|
||||
Add the correct display in Chrome and Safari.
|
||||
*/
|
||||
|
||||
summary {
|
||||
display: list-item;
|
||||
}
|
||||
|
||||
/*
|
||||
Removes the default spacing and border for appropriate elements.
|
||||
*/
|
||||
|
||||
blockquote,
|
||||
dl,
|
||||
dd,
|
||||
h1,
|
||||
h2,
|
||||
h3,
|
||||
h4,
|
||||
h5,
|
||||
h6,
|
||||
hr,
|
||||
figure,
|
||||
p,
|
||||
pre {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
fieldset {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
legend {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
ol,
|
||||
ul,
|
||||
menu {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/*
|
||||
Reset default styling for dialogs.
|
||||
*/
|
||||
|
||||
dialog {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/*
|
||||
Prevent resizing textareas horizontally by default.
|
||||
*/
|
||||
|
||||
textarea {
|
||||
resize: vertical;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Reset the default placeholder opacity in Firefox. (https://github.com/tailwindlabs/tailwindcss/issues/3300)
|
||||
2. Set the default placeholder color to the user's configured gray 400 color.
|
||||
*/
|
||||
|
||||
input::-moz-placeholder, textarea::-moz-placeholder {
|
||||
opacity: 1;
|
||||
/* 1 */
|
||||
color: #9ca3af;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
input::placeholder,
|
||||
textarea::placeholder {
|
||||
opacity: 1;
|
||||
/* 1 */
|
||||
color: #9ca3af;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
/*
|
||||
Set the default cursor for buttons.
|
||||
*/
|
||||
|
||||
button,
|
||||
[role="button"] {
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
/*
|
||||
Make sure disabled buttons don't get the pointer cursor.
|
||||
*/
|
||||
|
||||
:disabled {
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
/*
|
||||
1. Make replaced elements `display: block` by default. (https://github.com/mozdevs/cssremedy/issues/14)
|
||||
2. Add `vertical-align: middle` to align replaced elements more sensibly by default. (https://github.com/jensimmons/cssremedy/issues/14#issuecomment-634934210)
|
||||
This can trigger a poorly considered lint error in some tools but is included by design.
|
||||
*/
|
||||
|
||||
img,
|
||||
svg,
|
||||
video,
|
||||
canvas,
|
||||
audio,
|
||||
iframe,
|
||||
embed,
|
||||
object {
|
||||
display: block;
|
||||
/* 1 */
|
||||
vertical-align: middle;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
/*
|
||||
Constrain images and videos to the parent width and preserve their intrinsic aspect ratio. (https://github.com/mozdevs/cssremedy/issues/14)
|
||||
*/
|
||||
|
||||
img,
|
||||
video {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
}
|
||||
|
||||
/* Make elements with the HTML hidden attribute stay hidden by default */
|
||||
|
||||
[hidden]:where(:not([hidden="until-found"])) {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.container {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
@media (min-width: 640px) {
|
||||
.container {
|
||||
max-width: 640px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (min-width: 768px) {
|
||||
.container {
|
||||
max-width: 768px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (min-width: 1024px) {
|
||||
.container {
|
||||
max-width: 1024px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (min-width: 1280px) {
|
||||
.container {
|
||||
max-width: 1280px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (min-width: 1536px) {
|
||||
.container {
|
||||
max-width: 1536px;
|
||||
}
|
||||
}
|
||||
|
||||
.mx-auto {
|
||||
margin-left: auto;
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
.mb-12 {
|
||||
margin-bottom: 3rem;
|
||||
}
|
||||
|
||||
.mb-4 {
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.mb-8 {
|
||||
margin-bottom: 2rem;
|
||||
}
|
||||
|
||||
.mt-8 {
|
||||
margin-top: 2rem;
|
||||
}
|
||||
|
||||
.block {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.flex {
|
||||
display: flex;
|
||||
}
|
||||
|
||||
.table {
|
||||
display: table;
|
||||
}
|
||||
|
||||
.grid {
|
||||
display: grid;
|
||||
}
|
||||
|
||||
.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.transform {
|
||||
transform: translate(var(--tw-translate-x), var(--tw-translate-y)) rotate(var(--tw-rotate)) skewX(var(--tw-skew-x)) skewY(var(--tw-skew-y)) scaleX(var(--tw-scale-x)) scaleY(var(--tw-scale-y));
|
||||
}
|
||||
|
||||
.items-center {
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.justify-center {
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
.justify-between {
|
||||
justify-content: space-between;
|
||||
}
|
||||
|
||||
.gap-8 {
|
||||
gap: 2rem;
|
||||
}
|
||||
|
||||
.space-x-4 > :not([hidden]) ~ :not([hidden]) {
|
||||
--tw-space-x-reverse: 0;
|
||||
margin-right: calc(1rem * var(--tw-space-x-reverse));
|
||||
margin-left: calc(1rem * calc(1 - var(--tw-space-x-reverse)));
|
||||
}
|
||||
|
||||
.space-x-8 > :not([hidden]) ~ :not([hidden]) {
|
||||
--tw-space-x-reverse: 0;
|
||||
margin-right: calc(2rem * var(--tw-space-x-reverse));
|
||||
margin-left: calc(2rem * calc(1 - var(--tw-space-x-reverse)));
|
||||
}
|
||||
|
||||
.space-y-2 > :not([hidden]) ~ :not([hidden]) {
|
||||
--tw-space-y-reverse: 0;
|
||||
margin-top: calc(0.5rem * calc(1 - var(--tw-space-y-reverse)));
|
||||
margin-bottom: calc(0.5rem * var(--tw-space-y-reverse));
|
||||
}
|
||||
|
||||
.rounded-lg {
|
||||
border-radius: 0.5rem;
|
||||
}
|
||||
|
||||
.border {
|
||||
border-width: 1px;
|
||||
}
|
||||
|
||||
.border-t {
|
||||
border-top-width: 1px;
|
||||
}
|
||||
|
||||
.border-gray-600 {
|
||||
--tw-border-opacity: 1;
|
||||
border-color: rgb(75 85 99 / var(--tw-border-opacity, 1));
|
||||
}
|
||||
|
||||
.border-gray-800 {
|
||||
--tw-border-opacity: 1;
|
||||
border-color: rgb(31 41 55 / var(--tw-border-opacity, 1));
|
||||
}
|
||||
|
||||
.bg-black\/50 {
|
||||
background-color: rgb(0 0 0 / 0.5);
|
||||
}
|
||||
|
||||
.bg-blue-600 {
|
||||
--tw-bg-opacity: 1;
|
||||
background-color: rgb(37 99 235 / var(--tw-bg-opacity, 1));
|
||||
}
|
||||
|
||||
.bg-gray-900 {
|
||||
--tw-bg-opacity: 1;
|
||||
background-color: rgb(17 24 39 / var(--tw-bg-opacity, 1));
|
||||
}
|
||||
|
||||
.bg-gray-900\/50 {
|
||||
background-color: rgb(17 24 39 / 0.5);
|
||||
}
|
||||
|
||||
.p-6 {
|
||||
padding: 1.5rem;
|
||||
}
|
||||
|
||||
.px-6 {
|
||||
padding-left: 1.5rem;
|
||||
padding-right: 1.5rem;
|
||||
}
|
||||
|
||||
.px-8 {
|
||||
padding-left: 2rem;
|
||||
padding-right: 2rem;
|
||||
}
|
||||
|
||||
.py-16 {
|
||||
padding-top: 4rem;
|
||||
padding-bottom: 4rem;
|
||||
}
|
||||
|
||||
.py-2 {
|
||||
padding-top: 0.5rem;
|
||||
padding-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
.py-20 {
|
||||
padding-top: 5rem;
|
||||
padding-bottom: 5rem;
|
||||
}
|
||||
|
||||
.py-3 {
|
||||
padding-top: 0.75rem;
|
||||
padding-bottom: 0.75rem;
|
||||
}
|
||||
|
||||
.py-4 {
|
||||
padding-top: 1rem;
|
||||
padding-bottom: 1rem;
|
||||
}
|
||||
|
||||
.pt-8 {
|
||||
padding-top: 2rem;
|
||||
}
|
||||
|
||||
.text-center {
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.text-2xl {
|
||||
font-size: 1.5rem;
|
||||
line-height: 2rem;
|
||||
}
|
||||
|
||||
.text-3xl {
|
||||
font-size: 1.875rem;
|
||||
line-height: 2.25rem;
|
||||
}
|
||||
|
||||
.text-4xl {
|
||||
font-size: 2.25rem;
|
||||
line-height: 2.5rem;
|
||||
}
|
||||
|
||||
.text-lg {
|
||||
font-size: 1.125rem;
|
||||
line-height: 1.75rem;
|
||||
}
|
||||
|
||||
.text-xl {
|
||||
font-size: 1.25rem;
|
||||
line-height: 1.75rem;
|
||||
}
|
||||
|
||||
.font-bold {
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.font-semibold {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.text-gray-300 {
|
||||
--tw-text-opacity: 1;
|
||||
color: rgb(209 213 219 / var(--tw-text-opacity, 1));
|
||||
}
|
||||
|
||||
.text-gray-400 {
|
||||
--tw-text-opacity: 1;
|
||||
color: rgb(156 163 175 / var(--tw-text-opacity, 1));
|
||||
}
|
||||
|
||||
.text-white {
|
||||
--tw-text-opacity: 1;
|
||||
color: rgb(255 255 255 / var(--tw-text-opacity, 1));
|
||||
}
|
||||
|
||||
.filter {
|
||||
filter: var(--tw-blur) var(--tw-brightness) var(--tw-contrast) var(--tw-grayscale) var(--tw-hue-rotate) var(--tw-invert) var(--tw-saturate) var(--tw-sepia) var(--tw-drop-shadow);
|
||||
}
|
||||
|
||||
.transition {
|
||||
transition-property: color, background-color, border-color, text-decoration-color, fill, stroke, opacity, box-shadow, transform, filter, -webkit-backdrop-filter;
|
||||
transition-property: color, background-color, border-color, text-decoration-color, fill, stroke, opacity, box-shadow, transform, filter, backdrop-filter;
|
||||
transition-property: color, background-color, border-color, text-decoration-color, fill, stroke, opacity, box-shadow, transform, filter, backdrop-filter, -webkit-backdrop-filter;
|
||||
transition-timing-function: cubic-bezier(0.4, 0, 0.2, 1);
|
||||
transition-duration: 150ms;
|
||||
}
|
||||
|
||||
.duration-300 {
|
||||
transition-duration: 300ms;
|
||||
}
|
||||
|
||||
.hover\:bg-blue-700:hover {
|
||||
--tw-bg-opacity: 1;
|
||||
background-color: rgb(29 78 216 / var(--tw-bg-opacity, 1));
|
||||
}
|
||||
|
||||
.hover\:bg-gray-700:hover {
|
||||
--tw-bg-opacity: 1;
|
||||
background-color: rgb(55 65 81 / var(--tw-bg-opacity, 1));
|
||||
}
|
||||
|
||||
.hover\:bg-gray-800:hover {
|
||||
--tw-bg-opacity: 1;
|
||||
background-color: rgb(31 41 55 / var(--tw-bg-opacity, 1));
|
||||
}
|
||||
|
||||
.hover\:text-white:hover {
|
||||
--tw-text-opacity: 1;
|
||||
color: rgb(255 255 255 / var(--tw-text-opacity, 1));
|
||||
}
|
||||
|
||||
@media (min-width: 768px) {
|
||||
.md\:flex {
|
||||
display: flex;
|
||||
}
|
||||
|
||||
.md\:grid-cols-3 {
|
||||
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||||
}
|
||||
|
||||
.md\:grid-cols-4 {
|
||||
grid-template-columns: repeat(4, minmax(0, 1fr));
|
||||
}
|
||||
}
|
||||
|
|
@ -1,37 +0,0 @@
|
|||
import { config } from "dotenv";
|
||||
import { drizzle } from "drizzle-orm/postgres-js";
|
||||
import { migrate } from "drizzle-orm/postgres-js/migrator";
|
||||
import process from "node:process";
|
||||
import postgres from "postgres";
|
||||
|
||||
config();
|
||||
|
||||
const isProd = process.env.NODE_ENV === "production";
|
||||
const connectionString = isProd ? process.env.PROD_DATABASE_URL : process.env.DATABASE_URL;
|
||||
|
||||
if (!connectionString) {
|
||||
throw new Error(`${isProd ? "PROD_DATABASE_URL" : "DATABASE_URL"} is not set`);
|
||||
}
|
||||
|
||||
console.log("Connecting to:", connectionString.replace(/:[^:@]+@/, ":****@")); // Log sanitized connection string
|
||||
|
||||
const migrationClient = postgres(connectionString, { max: 1 });
|
||||
|
||||
async function main() {
|
||||
console.log("Running migrations...");
|
||||
|
||||
try {
|
||||
const db = drizzle(migrationClient);
|
||||
await migrate(db, { migrationsFolder: "./drizzle" });
|
||||
console.log("Migrations completed!");
|
||||
} catch (error) {
|
||||
console.error("Migration failed:", error);
|
||||
} finally {
|
||||
await migrationClient.end();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error("Unexpected error:", err);
|
||||
process.exit(1);
|
||||
});
|
||||
|
|
@ -1,151 +0,0 @@
|
|||
import { Context, Next } from "hono";
|
||||
import { getSessionFromRequest } from "@supermemory/authkit-remix-cloudflare/src/session";
|
||||
import { and, database, eq, sql } from "@supermemory/db";
|
||||
import { User, users } from "@supermemory/db/schema";
|
||||
import { Env, Variables } from "./types";
|
||||
import { encrypt, decrypt } from "./utils/cipher";
|
||||
|
||||
interface EncryptedData {
|
||||
userId: string;
|
||||
lastApiKeyGeneratedAt: string;
|
||||
}
|
||||
|
||||
export const getApiKey = async (
|
||||
userId: string,
|
||||
lastApiKeyGeneratedAt: string,
|
||||
c: Context<{ Variables: Variables; Bindings: Env }>
|
||||
) => {
|
||||
const data = `${userId}-${lastApiKeyGeneratedAt}`;
|
||||
return "sm_" + (await encrypt(data, c.env.WORKOS_COOKIE_PASSWORD));
|
||||
};
|
||||
|
||||
export const decryptApiKey = async (
|
||||
encryptedKey: string,
|
||||
c: Context<{ Variables: Variables; Bindings: Env }>
|
||||
): Promise<EncryptedData> => {
|
||||
const ourKey = encryptedKey.slice(3);
|
||||
const decrypted = await decrypt(ourKey, c.env.WORKOS_COOKIE_PASSWORD);
|
||||
const [userId, lastApiKeyGeneratedAt] = decrypted.split("-");
|
||||
|
||||
return {
|
||||
userId,
|
||||
lastApiKeyGeneratedAt,
|
||||
};
|
||||
};
|
||||
|
||||
export const auth = async (
|
||||
c: Context<{ Variables: Variables; Bindings: Env }>,
|
||||
next: Next
|
||||
) => {
|
||||
// Handle CORS preflight requests
|
||||
if (c.req.method === "OPTIONS") {
|
||||
return next()
|
||||
}
|
||||
|
||||
// Set cache control headers
|
||||
c.header("Cache-Control", "private, no-cache, no-store, must-revalidate");
|
||||
c.header("Pragma", "no-cache");
|
||||
c.header("Expires", "0");
|
||||
|
||||
let user: User | User[] | undefined;
|
||||
|
||||
// Check for API key authentication first
|
||||
const authHeader = c.req.raw.headers.get("Authorization");
|
||||
if (authHeader?.startsWith("Bearer ")) {
|
||||
const apiKey = authHeader.slice(7);
|
||||
try {
|
||||
const { userId, lastApiKeyGeneratedAt } = await decryptApiKey(apiKey, c);
|
||||
|
||||
// Look up user with matching id and lastApiKeyGeneratedAt
|
||||
user = await database(c.env.HYPERDRIVE.connectionString)
|
||||
.select()
|
||||
.from(users)
|
||||
.where(
|
||||
and(
|
||||
eq(users.uuid, userId)
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (user && Array.isArray(user)) {
|
||||
user = user[0];
|
||||
if (user && user.lastApiKeyGeneratedAt?.getTime() === Number(lastApiKeyGeneratedAt)) {
|
||||
c.set("user", user);
|
||||
} else {
|
||||
return c.json({ error: "Invalid API key - user not found" }, 401);
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("API key authentication failed:", err);
|
||||
return c.json({ error: "Invalid API key format" }, 401);
|
||||
}
|
||||
}
|
||||
|
||||
// If no user found via API key, try cookie authentication
|
||||
if (!user) {
|
||||
const cookies = c.req.raw.headers.get("Cookie");
|
||||
if (cookies) {
|
||||
// Fake remix context object. this just works.
|
||||
const context = {
|
||||
cloudflare: {
|
||||
env: c.env,
|
||||
},
|
||||
};
|
||||
|
||||
const session = await getSessionFromRequest(c.req.raw, context);
|
||||
console.log("Session", session);
|
||||
c.set("session", session);
|
||||
|
||||
if (session?.user?.id) {
|
||||
user = await database(c.env.HYPERDRIVE.connectionString)
|
||||
.select()
|
||||
.from(users)
|
||||
.where(eq(users.uuid, session.user.id))
|
||||
.limit(1);
|
||||
|
||||
if ((!user || user.length === 0) && session?.user?.id) {
|
||||
const newUser = await database(c.env.HYPERDRIVE.connectionString)
|
||||
.insert(users)
|
||||
.values({
|
||||
uuid: session.user?.id,
|
||||
email: session.user?.email,
|
||||
firstName: session.user?.firstName,
|
||||
lastName: session.user?.lastName,
|
||||
createdAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
emailVerified: false,
|
||||
profilePictureUrl: session.user?.profilePictureUrl ?? "",
|
||||
})
|
||||
.returning()
|
||||
.onConflictDoUpdate({
|
||||
target: [users.email],
|
||||
set: {
|
||||
uuid: session.user.id,
|
||||
},
|
||||
});
|
||||
|
||||
user = newUser[0];
|
||||
}
|
||||
|
||||
user = Array.isArray(user) ? user[0] : user;
|
||||
c.set("user", user);
|
||||
console.log("User", user);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check if request requires authentication
|
||||
const isPublicSpaceRequest =
|
||||
c.req.url.includes("/v1/spaces/") || c.req.url.includes("/v1/memories");
|
||||
|
||||
if (!isPublicSpaceRequest && !c.get("user")) {
|
||||
console.log("Unauthorized access to", c.req.url);
|
||||
if (authHeader) {
|
||||
return c.json({ error: "Invalid authentication credentials" }, 401);
|
||||
} else {
|
||||
return c.json({ error: "Authentication required" }, 401);
|
||||
}
|
||||
}
|
||||
|
||||
return next();
|
||||
};
|
||||
|
|
@ -1,234 +0,0 @@
|
|||
import { html } from "hono/html";
|
||||
|
||||
export function LandingPage() {
|
||||
return (
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<link href="/output.css" rel="stylesheet" />
|
||||
<title>Supermemory API</title>
|
||||
</head>
|
||||
<body>
|
||||
<div className="gradient-dark">
|
||||
<header className="bg-gray-900/50 dot-pattern">
|
||||
<nav className="container mx-auto px-6 py-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="text-2xl font-bold text-white">
|
||||
Supermemory API
|
||||
</div>
|
||||
<div className="hidden md:flex space-x-8">
|
||||
<a
|
||||
href="#features"
|
||||
className="text-gray-300 hover:text-white"
|
||||
>
|
||||
Features
|
||||
</a>
|
||||
<a
|
||||
href="https://docs.supermemory.ai/"
|
||||
target="_blank"
|
||||
className="text-gray-300 hover:text-white"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Documentation
|
||||
</a>
|
||||
</div>
|
||||
<a
|
||||
href="https://docs.supermemory.ai/"
|
||||
target="_blank"
|
||||
className="bg-blue-600 text-white px-6 py-2 rounded-lg hover:bg-blue-700"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Get Started
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div className="container mx-auto px-6 py-16 text-center">
|
||||
<h1 className="text-4xl font-bold text-white mb-4">
|
||||
The Modern API for Knowledge Management
|
||||
</h1>
|
||||
<p className="text-xl text-gray-300 mb-8">
|
||||
Build powerful search and AI applications with our flexible,
|
||||
production-ready API
|
||||
</p>
|
||||
<div className="flex justify-center space-x-4">
|
||||
<a
|
||||
href="https://docs.supermemory.ai/"
|
||||
target="_blank"
|
||||
className="bg-blue-600 text-white px-8 py-3 rounded-lg hover:bg-blue-700"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Get Started Free
|
||||
</a>
|
||||
<a
|
||||
href="https://docs.supermemory.ai/"
|
||||
target="_blank"
|
||||
className="border border-gray-600 text-gray-300 px-8 py-3 rounded-lg hover:bg-gray-700"
|
||||
rel="noreferrer"
|
||||
>
|
||||
View Docs
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<section id="features" className="py-20 bg-gray-900/50 dot-pattern">
|
||||
<div className="container mx-auto px-6">
|
||||
<h2 className="text-3xl font-bold text-center text-white mb-12">
|
||||
Key Features
|
||||
</h2>
|
||||
<div className="grid md:grid-cols-3 gap-8">
|
||||
<div className="p-6 border border-gray-600 rounded-lg bg-gray-900 hover:bg-gray-800 transition duration-300">
|
||||
<h3 className="text-xl font-semibold mb-4 text-white">
|
||||
Battle-Tested RAG Stack
|
||||
</h3>
|
||||
<p className="text-gray-300">
|
||||
Production-ready retrieval augmented generation architecture
|
||||
for reliable and scalable information retrieval.
|
||||
</p>
|
||||
</div>
|
||||
<div className="p-6 border border-gray-600 rounded-lg bg-gray-900 hover:bg-gray-800 transition duration-300">
|
||||
<h3 className="text-xl font-semibold mb-4 text-white">
|
||||
Flexible LLM Integration
|
||||
</h3>
|
||||
<p className="text-gray-300">
|
||||
Use any LLM of your choice or operate in search-only mode
|
||||
for maximum flexibility and control.
|
||||
</p>
|
||||
</div>
|
||||
<div className="p-6 border border-gray-600 rounded-lg bg-gray-900 hover:bg-gray-800 transition duration-300">
|
||||
<h3 className="text-xl font-semibold mb-4 text-white">
|
||||
Advanced Access Control
|
||||
</h3>
|
||||
<p className="text-gray-300">
|
||||
Comprehensive collection filtering and permission management
|
||||
for secure data access.
|
||||
</p>
|
||||
</div>
|
||||
<div className="p-6 border border-gray-600 rounded-lg bg-gray-900 hover:bg-gray-800 transition duration-300">
|
||||
<h3 className="text-xl font-semibold mb-4 text-white">
|
||||
Seamless Data Import
|
||||
</h3>
|
||||
<p className="text-gray-300">
|
||||
Magic link import and platform synchronization for
|
||||
effortless data integration.
|
||||
</p>
|
||||
</div>
|
||||
<div className="p-6 border border-gray-600 rounded-lg bg-gray-900 hover:bg-gray-800 transition duration-300">
|
||||
<h3 className="text-xl font-semibold mb-4 text-white">
|
||||
Real-time Monitoring
|
||||
</h3>
|
||||
<p className="text-gray-300">
|
||||
Track and analyze memory usage patterns in real-time with
|
||||
detailed metrics.
|
||||
</p>
|
||||
</div>
|
||||
<div className="p-6 border border-gray-600 rounded-lg bg-gray-900 hover:bg-gray-800 transition duration-300">
|
||||
<h3 className="text-xl font-semibold mb-4 text-white">
|
||||
Easy Integration
|
||||
</h3>
|
||||
<p className="text-gray-300">
|
||||
Simple API endpoints that integrate seamlessly with your
|
||||
existing infrastructure.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<footer className="bg-black/50 dot-pattern">
|
||||
<div className="container mx-auto px-6">
|
||||
<div className="grid md:grid-cols-4 gap-8">
|
||||
<div>
|
||||
<h4 className="text-lg font-semibold mb-4">
|
||||
Supermemory API
|
||||
</h4>
|
||||
<p className="text-gray-400">
|
||||
Making memory management simple and efficient for developers
|
||||
worldwide.
|
||||
</p>
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-lg font-semibold mb-4">Product</h4>
|
||||
<ul className="space-y-2 text-gray-400">
|
||||
<li>
|
||||
<a href="#features" className="hover:text-white">
|
||||
Features
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a
|
||||
href="https://docs.supermemory.ai/"
|
||||
target="_blank"
|
||||
className="hover:text-white"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Documentation
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-lg font-semibold mb-4">Connect</h4>
|
||||
<ul className="space-y-2 text-gray-400">
|
||||
<li>
|
||||
<a
|
||||
href="https://x.com/supermemoryai"
|
||||
target="_blank"
|
||||
className="hover:text-white"
|
||||
rel="noreferrer"
|
||||
>
|
||||
X (formerly Twitter)
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a
|
||||
href="https://github.com/supermemoryai"
|
||||
target="_blank"
|
||||
className="hover:text-white"
|
||||
rel="noreferrer"
|
||||
>
|
||||
GitHub
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a
|
||||
href="https://discord.gg/b3BgKWpbtR"
|
||||
target="_blank"
|
||||
className="hover:text-white"
|
||||
rel="noreferrer"
|
||||
>
|
||||
Discord
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<div className="border-t border-gray-800 mt-8 pt-8 text-center text-gray-400">
|
||||
<p>© 2024 Supermemory API. All rights reserved.</p>
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
<style
|
||||
dangerouslySetInnerHTML={{
|
||||
__html: `
|
||||
.dot-pattern {
|
||||
background-image: radial-gradient(
|
||||
rgba(255, 255, 255, 0.1) 1px,
|
||||
transparent 1px
|
||||
);
|
||||
background-size: 24px 24px;
|
||||
}
|
||||
.gradient-dark {
|
||||
background: linear-gradient(to bottom right, rgb(17 24 39), rgb(0 0 0));
|
||||
}
|
||||
`,
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
export class BaseHttpError extends Error {
|
||||
public status: number;
|
||||
public message: string;
|
||||
|
||||
constructor(status: number, message: string) {
|
||||
super(message);
|
||||
this.status = status;
|
||||
this.message = message;
|
||||
Object.setPrototypeOf(this, new.target.prototype); // Restore prototype chain
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
export class BaseError extends Error {
|
||||
type: string;
|
||||
message: string;
|
||||
source: string;
|
||||
ignoreLog: boolean;
|
||||
|
||||
constructor(
|
||||
type: string,
|
||||
message?: string,
|
||||
source?: string,
|
||||
ignoreLog = false
|
||||
) {
|
||||
super();
|
||||
|
||||
Object.setPrototypeOf(this, new.target.prototype);
|
||||
|
||||
this.type = type;
|
||||
this.message =
|
||||
message ??
|
||||
"An unknown error occurred. If this persists, please contact us.";
|
||||
this.source = source ?? "unspecified";
|
||||
this.ignoreLog = ignoreLog;
|
||||
}
|
||||
|
||||
toJSON(): Record<PropertyKey, string> {
|
||||
return {
|
||||
type: this.type,
|
||||
message: this.message,
|
||||
source: this.source,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -1,31 +0,0 @@
|
|||
import { BaseError } from "./baseError";
|
||||
|
||||
export type Result<T, E extends Error> =
|
||||
| { ok: true; value: T }
|
||||
| { ok: false; error: E };
|
||||
|
||||
export const Ok = <T>(data: T): Result<T, never> => {
|
||||
return { ok: true, value: data };
|
||||
};
|
||||
|
||||
export const Err = <E extends BaseError>(error: E): Result<never, E> => {
|
||||
return { ok: false, error };
|
||||
};
|
||||
|
||||
export async function wrap<T, E extends BaseError>(
|
||||
p: Promise<T>,
|
||||
errorFactory: (err: Error, source: string) => E,
|
||||
source: string = "unspecified"
|
||||
): Promise<Result<T, E>> {
|
||||
try {
|
||||
return Ok(await p);
|
||||
} catch (e) {
|
||||
return Err(errorFactory(e as Error, source));
|
||||
}
|
||||
}
|
||||
|
||||
export function isErr<T, E extends Error>(
|
||||
result: Result<T, E>,
|
||||
): result is { ok: false; error: E } {
|
||||
return !result.ok;
|
||||
}
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
@tailwind base;
|
||||
@tailwind components;
|
||||
@tailwind utilities;
|
||||
|
|
@ -1,255 +0,0 @@
|
|||
import { z } from "zod";
|
||||
import { type Context, Hono } from "hono";
|
||||
import { auth } from "./auth";
|
||||
import { logger } from "hono/logger";
|
||||
import { timing } from "hono/timing";
|
||||
import type { Env, Variables } from "./types";
|
||||
import { zValidator } from "@hono/zod-validator";
|
||||
import { database } from "@supermemory/db";
|
||||
import { waitlist } from "@supermemory/db/schema";
|
||||
import { cors } from "hono/cors";
|
||||
import { ContentWorkflow } from "./workflow";
|
||||
import { Resend } from "resend";
|
||||
import { LandingPage } from "./components/landing";
|
||||
import user from "./routes/user";
|
||||
import spacesRoute from "./routes/spaces";
|
||||
import actions from "./routes/actions";
|
||||
import memories from "./routes/memories";
|
||||
import integrations from "./routes/integrations";
|
||||
import { fromHono } from "chanfana";
|
||||
import {
|
||||
DurableObjectRateLimiter,
|
||||
DurableObjectStore,
|
||||
} from "@hono-rate-limiter/cloudflare";
|
||||
import type { ConfigType, GeneralConfigType, rateLimiter } from "hono-rate-limiter";
|
||||
|
||||
// Create base Hono app first
|
||||
const honoApp = new Hono<{ Variables: Variables; Bindings: Env }>();
|
||||
|
||||
const app = fromHono(honoApp);
|
||||
|
||||
// Add all middleware and routes
|
||||
app.use("*", timing());
|
||||
app.use("*", logger());
|
||||
app.use(
|
||||
"*",
|
||||
cors({
|
||||
origin: [
|
||||
"http://localhost:3000",
|
||||
"https://supermemory.ai",
|
||||
"https://*.supermemory.ai",
|
||||
"https://*.supermemory.com",
|
||||
"https://supermemory.com",
|
||||
"chrome-extension://*",
|
||||
],
|
||||
allowHeaders: ["*"],
|
||||
allowMethods: ["*"],
|
||||
credentials: true,
|
||||
exposeHeaders: ["*"],
|
||||
})
|
||||
);
|
||||
|
||||
app.use("/v1/*", auth);
|
||||
app.use("/v1/*", (c, next) => {
|
||||
const user = c.get("user");
|
||||
|
||||
if (c.env.NODE_ENV === "development") {
|
||||
return next();
|
||||
}
|
||||
|
||||
// RATELIMITS
|
||||
const rateLimitConfig = {
|
||||
// Endpoints that bypass rate limiting
|
||||
excludedPaths: [
|
||||
"/v1/add",
|
||||
"/v1/chat",
|
||||
"/v1/suggested-learnings",
|
||||
"/v1/recommended-questions",
|
||||
] as (string | RegExp)[],
|
||||
|
||||
// Custom rate limits for specific endpoints
|
||||
customLimits: {
|
||||
notionImport: {
|
||||
paths: ["/v1/integrations/notion/import", "/v1/integrations/notion"],
|
||||
windowMs: 10 * 60 * 1000, // 10 minutes
|
||||
limit: 5, // 5 requests per 10 minutes
|
||||
},
|
||||
inviteSpace: {
|
||||
paths: [/^\v1\/spaces\/[^/]+\/invite$/],
|
||||
windowMs: 60 * 1000, // 1 minute
|
||||
limit: 5, // 5 requests per minute
|
||||
},
|
||||
} as Record<
|
||||
string,
|
||||
{ paths: (string | RegExp)[]; windowMs: number; limit: number }
|
||||
>,
|
||||
|
||||
default: {
|
||||
windowMs: 60 * 1000, // 1 minute
|
||||
limit: 100, // 100 requests per minute
|
||||
},
|
||||
|
||||
common: {
|
||||
standardHeaders: "draft-6",
|
||||
keyGenerator: (c: Context) =>
|
||||
`${user?.uuid ?? c.req.header("cf-connecting-ip")}-${new Date().getDate()}`, // day so that limit gets reset every day
|
||||
store: new DurableObjectStore({ namespace: c.env.RATE_LIMITER }),
|
||||
} as GeneralConfigType<ConfigType>,
|
||||
};
|
||||
|
||||
if (
|
||||
c.req.path &&
|
||||
rateLimitConfig.excludedPaths.some((path) =>
|
||||
typeof path === "string" ? c.req.path === path : path.test(c.req.path)
|
||||
)
|
||||
) {
|
||||
return next();
|
||||
}
|
||||
|
||||
// Check for custom rate limits
|
||||
for (const [_, config] of Object.entries(rateLimitConfig.customLimits)) {
|
||||
if (
|
||||
config.paths.some((path) =>
|
||||
typeof path === "string" ? c.req.path === path : path.test(c.req.path)
|
||||
)
|
||||
) {
|
||||
return rateLimiter({
|
||||
windowMs: config.windowMs,
|
||||
limit: config.limit,
|
||||
...rateLimitConfig.common,
|
||||
})(c as any, next);
|
||||
}
|
||||
}
|
||||
|
||||
// Apply default rate limit
|
||||
return rateLimiter({
|
||||
windowMs: rateLimitConfig.default.windowMs,
|
||||
limit: rateLimitConfig.default.limit,
|
||||
...rateLimitConfig.common,
|
||||
})(c as any, next);
|
||||
});
|
||||
|
||||
app.get("/", (c) => {
|
||||
return c.html(<LandingPage />);
|
||||
});
|
||||
|
||||
// TEMPORARY REDIRECT
|
||||
app.all("/api/*", async (c) => {
|
||||
// Get the full URL and path
|
||||
const url = new URL(c.req.url);
|
||||
const path = url.pathname;
|
||||
const newPath = path.replace("/api", "/v1");
|
||||
|
||||
// Preserve query parameters and build target URL
|
||||
const redirectUrl = `https://api.supermemory.ai${newPath}${url.search}`;
|
||||
|
||||
// Use c.redirect() for a proper redirect
|
||||
return c.redirect(redirectUrl);
|
||||
});
|
||||
|
||||
app.route("/v1/user", user);
|
||||
app.route("/v1/spaces", spacesRoute);
|
||||
app.route("/v1", actions);
|
||||
app.route("/v1/integrations", integrations);
|
||||
app.route("/v1/memories", memories);
|
||||
|
||||
app.get("/v1/session", (c) => {
|
||||
const user = c.get("user");
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
return c.json({
|
||||
user,
|
||||
});
|
||||
});
|
||||
|
||||
app.post(
|
||||
"/waitlist",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({ email: z.string().email(), token: z.string() })
|
||||
),
|
||||
async (c) => {
|
||||
const { email, token } = c.req.valid("json");
|
||||
|
||||
const address = c.req.raw.headers.get("CF-Connecting-IP");
|
||||
|
||||
const idempotencyKey = crypto.randomUUID();
|
||||
const url = "https://challenges.cloudflare.com/turnstile/v0/siteverify";
|
||||
const firstResult = await fetch(url, {
|
||||
body: JSON.stringify({
|
||||
secret: c.env.TURNSTILE_SECRET_KEY,
|
||||
response: token,
|
||||
remoteip: address,
|
||||
idempotency_key: idempotencyKey,
|
||||
}),
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
});
|
||||
|
||||
const firstOutcome = (await firstResult.json()) as { success: boolean };
|
||||
|
||||
if (!firstOutcome.success) {
|
||||
console.info("Turnstile verification failed", firstOutcome);
|
||||
return c.json(
|
||||
{ error: "Turnstile verification failed" },
|
||||
439 as StatusCode
|
||||
);
|
||||
}
|
||||
|
||||
const resend = new Resend(c.env.RESEND_API_KEY);
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const ip =
|
||||
c.req.header("cf-connecting-ip") ||
|
||||
`${c.req.raw.cf?.asn}-${c.req.raw.cf?.country}-${c.req.raw.cf?.city}-${c.req.raw.cf?.region}-${c.req.raw.cf?.postalCode}`;
|
||||
|
||||
const { success } = await c.env.EMAIL_LIMITER.limit({ key: ip });
|
||||
|
||||
if (!success) {
|
||||
return c.json({ error: "Rate limit exceeded" }, 429);
|
||||
}
|
||||
|
||||
const message = `Supermemory started as a side project a few months ago when I built it as a hackathon project.
|
||||
<br></br>
|
||||
you guys loved it too much. like wayy too much. it was embarrassing, because this was not it - it was nothing but a hackathon project.
|
||||
<br></br>
|
||||
I launched on github too. <a href="https://git.new/memory">https://github.com/supermemoryai/supermemory</a>, and we were somehow one of the fastest growing open source repositories in Q3 2024.
|
||||
<br></br><br></br>
|
||||
So, it's time to make this good. My vision is to make supermemory the best memory tool on the internet.
|
||||
`;
|
||||
|
||||
try {
|
||||
await db.insert(waitlist).values({ email });
|
||||
await resend.emails.send({
|
||||
from: "Dhravya From Supermemory <waitlist@m.supermemory.com>",
|
||||
to: email,
|
||||
subject: "You're in the waitlist - A personal note from Dhravya",
|
||||
html: `<p>Hi. I'm Dhravya. I'm building Supermemory to help people remember everything.<br></br> ${message} <br></br><br></br>I'll be in touch when we launch! Till then, just reply to this email if you wanna talk :)<br></br>If you want to follow me on X, here's my handle: <a href='https://x.com/dhravyashah'>@dhravyashah</a><br></br><br></br>- Dhravya</p>`,
|
||||
});
|
||||
} catch (e) {
|
||||
console.error(e);
|
||||
return c.json({ error: "Failed to add to waitlist" }, 400);
|
||||
}
|
||||
|
||||
return c.json({ success: true });
|
||||
}
|
||||
);
|
||||
|
||||
app.onError((err, c) => {
|
||||
console.error(err);
|
||||
return c.json({ error: "Internal server error" }, 500);
|
||||
});
|
||||
|
||||
export default {
|
||||
fetch: app.fetch,
|
||||
};
|
||||
|
||||
export { ContentWorkflow, DurableObjectRateLimiter };
|
||||
|
||||
export type AppType = typeof app;
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
import { createOpenAI, OpenAIProvider } from "@ai-sdk/openai";
|
||||
import { createGoogleGenerativeAI } from "@ai-sdk/google";
|
||||
import { Env } from "./types";
|
||||
|
||||
export function openai(
|
||||
env: Env,
|
||||
apiKey?: string
|
||||
): ReturnType<typeof createOpenAI> {
|
||||
return createOpenAI({
|
||||
apiKey: apiKey || env.OPEN_AI_API_KEY,
|
||||
baseURL: "https://gateway.ai.cloudflare.com/v1/47c2b4d598af9d423c06fc9f936226d5/supermemory/openai"
|
||||
});
|
||||
}
|
||||
|
||||
export function google(securityKey: string) {
|
||||
return createGoogleGenerativeAI({
|
||||
apiKey: securityKey,
|
||||
});
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,180 +0,0 @@
|
|||
import { Hono } from "hono";
|
||||
import type { Env, Variables } from "../types";
|
||||
import { getDecryptedKV } from "encrypt-workers-kv";
|
||||
import { getAllNotionPageContents } from "../utils/notion";
|
||||
import { and, eq, or } from "@supermemory/db";
|
||||
import { documents } from "@supermemory/db/schema";
|
||||
import { database } from "@supermemory/db";
|
||||
import { fromHono } from "chanfana";
|
||||
|
||||
const integrations = fromHono(
|
||||
new Hono<{ Variables: Variables; Bindings: Env }>()).get("/notion/import", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
// Create SSE stream
|
||||
const stream = new TransformStream();
|
||||
const writer = stream.writable.getWriter();
|
||||
const encoder = new TextEncoder();
|
||||
|
||||
// Create response first so client gets headers immediately
|
||||
const response = new Response(stream.readable, {
|
||||
headers: {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache",
|
||||
Connection: "keep-alive",
|
||||
// Required CORS headers for SSE
|
||||
"Access-Control-Allow-Origin": "*",
|
||||
"Access-Control-Allow-Credentials": "true",
|
||||
},
|
||||
});
|
||||
|
||||
const sendMessage = async (data: Record<string, any>) => {
|
||||
// Proper SSE format requires "data: " prefix and double newline
|
||||
const formattedData = `data: ${JSON.stringify(data)}\n\n`;
|
||||
await writer.write(encoder.encode(formattedData));
|
||||
};
|
||||
|
||||
// Start processing in background
|
||||
c.executionCtx.waitUntil(
|
||||
(async () => {
|
||||
try {
|
||||
// Send initial heartbeat
|
||||
await sendMessage({ type: "connected" });
|
||||
|
||||
const token = await getDecryptedKV(
|
||||
c.env.ENCRYPTED_TOKENS,
|
||||
`${user.uuid}-notion`,
|
||||
`${c.env.WORKOS_COOKIE_PASSWORD}-${user.uuid}`
|
||||
);
|
||||
|
||||
const stringToken = new TextDecoder().decode(token);
|
||||
if (!stringToken) {
|
||||
await sendMessage({ type: "error", error: "No token found" });
|
||||
await writer.close();
|
||||
return;
|
||||
}
|
||||
|
||||
await sendMessage({ type: "progress", progress: 5 });
|
||||
|
||||
// Fetch pages with progress updates
|
||||
const pages = await getAllNotionPageContents(
|
||||
stringToken,
|
||||
async (progress) => {
|
||||
// Map progress from 0-100 to 5-40 range
|
||||
const scaledProgress = Math.floor(5 + (progress * 35) / 100);
|
||||
await sendMessage({ type: "progress", progress: scaledProgress });
|
||||
}
|
||||
);
|
||||
|
||||
await sendMessage({ type: "progress", progress: 40 });
|
||||
|
||||
let processed = 0;
|
||||
const totalPages = pages.length;
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
for (const page of pages) {
|
||||
// Calculate document hash for duplicate detection
|
||||
const encoder = new TextEncoder();
|
||||
const data = encoder.encode(page.content);
|
||||
const hashBuffer = await crypto.subtle.digest("SHA-256", data);
|
||||
const hashArray = Array.from(new Uint8Array(hashBuffer));
|
||||
const documentHash = hashArray
|
||||
.map((b) => b.toString(16).padStart(2, "0"))
|
||||
.join("");
|
||||
|
||||
// Check for duplicates using hash
|
||||
const existingDocs = await db
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(
|
||||
and(
|
||||
eq(documents.userId, user.id),
|
||||
or(
|
||||
eq(documents.contentHash, documentHash),
|
||||
and(
|
||||
eq(documents.type, "notion"),
|
||||
or(
|
||||
eq(documents.url, page.url),
|
||||
eq(documents.raw, page.content)
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
if (existingDocs.length > 0) {
|
||||
await sendMessage({
|
||||
type: "warning",
|
||||
message: `Skipping duplicate page: ${page.title}`,
|
||||
});
|
||||
processed++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Insert into documents table first
|
||||
try {
|
||||
await db.insert(documents).values({
|
||||
uuid: page.id,
|
||||
userId: user.id,
|
||||
type: "notion",
|
||||
url: page.url,
|
||||
title: page.title,
|
||||
contentHash: documentHash,
|
||||
raw: page.content,
|
||||
});
|
||||
|
||||
await c.env.CONTENT_WORKFLOW.create({
|
||||
params: {
|
||||
userId: user.id,
|
||||
content: page.url,
|
||||
spaces: [],
|
||||
type: "notion",
|
||||
uuid: page.id,
|
||||
url: page.url,
|
||||
prefetched: {
|
||||
contentToVectorize: page.content,
|
||||
contentToSave: page.content,
|
||||
title: page.title,
|
||||
type: "notion",
|
||||
},
|
||||
createdAt: page.createdAt,
|
||||
},
|
||||
id: `${user.id}-${page.id}-${new Date().getTime()}`,
|
||||
});
|
||||
|
||||
processed++;
|
||||
const progress = 50 + Math.floor((processed / totalPages) * 50);
|
||||
await sendMessage({ type: "progress", progress, page: page.title });
|
||||
} catch (error) {
|
||||
console.error(`Failed to process page ${page.title}:`, error);
|
||||
await sendMessage({
|
||||
type: "warning",
|
||||
message: `Failed to process page: ${page.title}`,
|
||||
error: error instanceof Error ? error.message : "Unknown error",
|
||||
});
|
||||
processed++;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
await sendMessage({ type: "complete", progress: 100 });
|
||||
await writer.close();
|
||||
} catch (error) {
|
||||
console.error("Import error:", error);
|
||||
await sendMessage({
|
||||
type: "error",
|
||||
error: error instanceof Error ? error.message : "Import failed",
|
||||
});
|
||||
await writer.close();
|
||||
}
|
||||
})()
|
||||
);
|
||||
|
||||
return response;
|
||||
});
|
||||
|
||||
export default integrations;
|
||||
|
|
@ -1,293 +0,0 @@
|
|||
import { Hono } from "hono";
|
||||
import { Variables, Env } from "../types";
|
||||
import { zValidator } from "@hono/zod-validator";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
documents,
|
||||
spaces,
|
||||
spaceAccess,
|
||||
contentToSpace,
|
||||
} from "@supermemory/db/schema";
|
||||
import { and, database, desc, eq, or, sql, isNull } from "@supermemory/db";
|
||||
import { fromHono } from "chanfana";
|
||||
|
||||
const memories = fromHono(new Hono<{ Variables: Variables; Bindings: Env }>())
|
||||
.get(
|
||||
"/",
|
||||
zValidator(
|
||||
"query",
|
||||
z.object({
|
||||
start: z.string().default("0").transform(Number),
|
||||
count: z.string().default("10").transform(Number),
|
||||
spaceId: z.string().optional(),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const { start, count, spaceId } = c.req.valid("query");
|
||||
const user = c.get("user");
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
console.log("Fetching memories with spaceId", spaceId);
|
||||
console.log(c.req.url);
|
||||
// If spaceId provided, verify access
|
||||
if (spaceId) {
|
||||
console.log("SpaceID provided", spaceId);
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId.split("---")[0]))
|
||||
.limit(1);
|
||||
|
||||
if (!space[0]) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
// Check access - allow if public, user owns the space, or has access through spaceAccess
|
||||
if (!space[0].isPublic && !user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
if (!space[0].isPublic && space[0].ownerId !== user?.id) {
|
||||
const access = await db
|
||||
.select()
|
||||
.from(spaceAccess)
|
||||
.where(
|
||||
and(
|
||||
eq(spaceAccess.spaceId, space[0].id),
|
||||
eq(spaceAccess.userEmail, user?.email ?? ""),
|
||||
eq(spaceAccess.status, "accepted")
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (access.length === 0) {
|
||||
console.log("Unauthorized access to", c.req.url);
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
}
|
||||
|
||||
// Get documents for space
|
||||
const [items, totalResult] = await Promise.all([
|
||||
db
|
||||
.select({
|
||||
documents,
|
||||
})
|
||||
.from(documents)
|
||||
.innerJoin(
|
||||
contentToSpace,
|
||||
eq(documents.id, contentToSpace.contentId)
|
||||
)
|
||||
.where(eq(contentToSpace.spaceId, space[0].id))
|
||||
.orderBy(desc(documents.createdAt))
|
||||
.limit(count)
|
||||
.offset(start),
|
||||
db
|
||||
.select({
|
||||
total: sql<number>`count(*)`.as("total"),
|
||||
})
|
||||
.from(documents)
|
||||
.innerJoin(
|
||||
contentToSpace,
|
||||
eq(documents.id, contentToSpace.contentId)
|
||||
)
|
||||
.where(eq(contentToSpace.spaceId, space[0].id)),
|
||||
]);
|
||||
|
||||
const total = totalResult[0]?.total ?? 0;
|
||||
|
||||
return c.json({
|
||||
items: items.map((item) => ({
|
||||
...item.documents,
|
||||
id: item.documents.uuid,
|
||||
})),
|
||||
total,
|
||||
});
|
||||
}
|
||||
|
||||
// Regular user memories endpoint
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
// Set cache control headers for 5 minutes
|
||||
c.header("Cache-Control", "private, max-age=300");
|
||||
c.header("Vary", "Cookie"); // Vary on Cookie since response depends on user
|
||||
|
||||
// Generate ETag based on user ID, start, count
|
||||
const etag = `"${user.id}-${start}-${count}"`;
|
||||
c.header("ETag", etag);
|
||||
|
||||
// Check if client has matching ETag
|
||||
const ifNoneMatch = c.req.header("If-None-Match");
|
||||
if (ifNoneMatch === etag) {
|
||||
return new Response(null, { status: 304 });
|
||||
}
|
||||
|
||||
const [items, [{ total }]] = await Promise.all([
|
||||
db
|
||||
.select({
|
||||
documents: documents,
|
||||
})
|
||||
.from(documents)
|
||||
.leftJoin(contentToSpace, eq(documents.id, contentToSpace.contentId))
|
||||
.where(
|
||||
and(eq(documents.userId, user.id), isNull(contentToSpace.contentId))
|
||||
)
|
||||
.orderBy(desc(documents.createdAt))
|
||||
.limit(count)
|
||||
.offset(start),
|
||||
db
|
||||
.select({
|
||||
total: sql<number>`count(*)`.as("total"),
|
||||
})
|
||||
.from(documents)
|
||||
.leftJoin(contentToSpace, eq(documents.id, contentToSpace.contentId))
|
||||
.where(
|
||||
and(eq(documents.userId, user.id), isNull(contentToSpace.contentId))
|
||||
),
|
||||
]);
|
||||
|
||||
return c.json({
|
||||
items: items.map((item) => ({
|
||||
...item.documents,
|
||||
id: item.documents.uuid,
|
||||
})),
|
||||
total,
|
||||
});
|
||||
}
|
||||
)
|
||||
.get("/:id", zValidator("param", z.object({ id: z.string() })), async (c) => {
|
||||
const { id } = c.req.valid("param");
|
||||
const user = c.get("user");
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const memory = await database(c.env.HYPERDRIVE.connectionString)
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(and(eq(documents.uuid, id), eq(documents.userId, user.id)))
|
||||
.limit(1);
|
||||
|
||||
return c.json(memory[0]);
|
||||
})
|
||||
.delete(
|
||||
"/:id",
|
||||
zValidator("param", z.object({ id: z.string() })),
|
||||
async (c) => {
|
||||
const { id } = c.req.valid("param");
|
||||
const user = c.get("user");
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
let documentIdNum;
|
||||
|
||||
try {
|
||||
documentIdNum = Number(id);
|
||||
} catch (e) {
|
||||
documentIdNum = null;
|
||||
}
|
||||
|
||||
const doc = await db
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(
|
||||
and(
|
||||
documentIdNum
|
||||
? or(eq(documents.uuid, id), eq(documents.id, documentIdNum))
|
||||
: eq(documents.uuid, id),
|
||||
eq(documents.userId, user.id)
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (!doc[0]) {
|
||||
return c.json({ error: "Document not found" }, 404);
|
||||
}
|
||||
|
||||
const [document, contentToSpacei] = await Promise.all([
|
||||
db
|
||||
.delete(documents)
|
||||
.where(and(eq(documents.uuid, id), eq(documents.userId, user.id))),
|
||||
db
|
||||
.delete(contentToSpace)
|
||||
.where(eq(contentToSpace.contentId, doc[0].id)),
|
||||
]);
|
||||
|
||||
return c.json({ success: true });
|
||||
}
|
||||
)
|
||||
.post(
|
||||
"/batch-delete",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
ids: z.array(z.string()),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const { ids } = c.req.valid("json");
|
||||
const user = c.get("user");
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
try {
|
||||
// First get all valid documents that belong to the user
|
||||
const docs = await db
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(
|
||||
and(
|
||||
eq(documents.userId, user.id),
|
||||
sql`${documents.uuid} = ANY(ARRAY[${ids}]::text[])`
|
||||
)
|
||||
);
|
||||
|
||||
if (docs.length === 0) {
|
||||
return c.json({ error: "No valid documents found" }, 404);
|
||||
}
|
||||
|
||||
const docIds = docs.map((doc) => doc.id);
|
||||
|
||||
// Delete in transaction to ensure consistency
|
||||
await db.transaction(async (tx) => {
|
||||
await Promise.all([
|
||||
// Delete document entries
|
||||
tx
|
||||
.delete(documents)
|
||||
.where(
|
||||
and(
|
||||
eq(documents.userId, user.id),
|
||||
sql`${documents.uuid} = ANY(ARRAY[${ids}]::text[])`
|
||||
)
|
||||
),
|
||||
// Delete space connections
|
||||
tx
|
||||
.delete(contentToSpace)
|
||||
.where(
|
||||
sql`${contentToSpace.contentId} = ANY(ARRAY[${docIds}]::int[])`
|
||||
),
|
||||
]);
|
||||
});
|
||||
|
||||
return c.json({
|
||||
success: true,
|
||||
deletedCount: docs.length,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("Batch delete error:", error);
|
||||
return c.json({ error: "Failed to delete documents" }, 500);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export default memories;
|
||||
|
|
@ -1,709 +0,0 @@
|
|||
import { Hono } from "hono";
|
||||
import { Env, Variables } from "../types";
|
||||
import { and, database, desc, eq, isNotNull, or, sql } from "@supermemory/db";
|
||||
import {
|
||||
contentToSpace,
|
||||
documents,
|
||||
savedSpaces,
|
||||
spaceAccess,
|
||||
spaces,
|
||||
users,
|
||||
} from "@supermemory/db/schema";
|
||||
import { zValidator } from "@hono/zod-validator";
|
||||
import { z } from "zod";
|
||||
import { randomId } from "@supermemory/shared";
|
||||
import { fromHono } from "chanfana";
|
||||
|
||||
const spacesRoute = fromHono(
|
||||
new Hono<{ Variables: Variables; Bindings: Env }>())
|
||||
.get("/", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const [allSpaces, savedSpacesList, spaceOwners] = await Promise.all([
|
||||
db
|
||||
.select({
|
||||
id: spaces.id,
|
||||
uuid: spaces.uuid,
|
||||
name: sql<string>`REGEXP_REPLACE(${spaces.name}, E'[\\n\\r]+', ' ', 'g')`.as(
|
||||
"name"
|
||||
),
|
||||
ownerId: spaces.ownerId,
|
||||
isPublic: spaces.isPublic,
|
||||
createdAt: spaces.createdAt,
|
||||
accessType: spaceAccess.accessType,
|
||||
})
|
||||
.from(spaces)
|
||||
.leftJoin(
|
||||
spaceAccess,
|
||||
and(
|
||||
eq(spaces.id, spaceAccess.spaceId),
|
||||
eq(spaceAccess.userEmail, user.email),
|
||||
eq(spaceAccess.status, "accepted")
|
||||
)
|
||||
)
|
||||
.where(or(eq(spaces.ownerId, user.id), isNotNull(spaceAccess.spaceId)))
|
||||
.orderBy(desc(spaces.createdAt)),
|
||||
|
||||
db
|
||||
.select({
|
||||
spaceId: savedSpaces.spaceId,
|
||||
})
|
||||
.from(savedSpaces)
|
||||
.where(eq(savedSpaces.userId, user.id)),
|
||||
|
||||
db
|
||||
.select({
|
||||
id: users.id,
|
||||
uuid: users.uuid,
|
||||
name: users.firstName,
|
||||
email: users.email,
|
||||
profileImage: users.profilePictureUrl,
|
||||
})
|
||||
.from(users)
|
||||
.innerJoin(spaces, eq(spaces.ownerId, users.id)),
|
||||
]);
|
||||
|
||||
const savedSpaceIds = new Set(savedSpacesList.map((s) => s.spaceId));
|
||||
const ownerMap = new Map(spaceOwners.map((owner) => [owner.id, owner]));
|
||||
|
||||
const spacesWithDetails = allSpaces.map((space) => {
|
||||
const isOwner = space.ownerId === user.id;
|
||||
const owner = ownerMap.get(space.ownerId);
|
||||
|
||||
return {
|
||||
...space,
|
||||
favorited: savedSpaceIds.has(space.id),
|
||||
permissions: {
|
||||
canRead: space.isPublic || isOwner || space.accessType != null,
|
||||
canEdit: isOwner || space.accessType === "edit",
|
||||
isOwner,
|
||||
},
|
||||
owner: isOwner
|
||||
? null
|
||||
: {
|
||||
id: owner?.uuid,
|
||||
name: owner?.name,
|
||||
email: owner?.email,
|
||||
profileImage: owner?.profileImage,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
return c.json({ spaces: spacesWithDetails });
|
||||
})
|
||||
.get("/:spaceId", async (c) => {
|
||||
const user = c.get("user");
|
||||
const spaceId = c.req.param("spaceId");
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (!space[0]) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
// For public spaces, anyone can read but only owner can edit
|
||||
if (space[0].isPublic) {
|
||||
const canEdit = user?.id === space[0].ownerId;
|
||||
return c.json({
|
||||
...space[0],
|
||||
permissions: {
|
||||
canRead: true,
|
||||
canEdit,
|
||||
isOwner: space[0].ownerId === user?.id,
|
||||
isPublic: space[0].isPublic,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// For private spaces, require authentication
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
// Check if user is owner or has access via spaceAccess
|
||||
const isOwner = space[0].ownerId === user.id;
|
||||
let canEdit = isOwner;
|
||||
if (!isOwner) {
|
||||
const spaceAccessCheck = await db
|
||||
.select()
|
||||
.from(spaceAccess)
|
||||
.where(
|
||||
and(
|
||||
eq(spaceAccess.spaceId, space[0].id),
|
||||
eq(spaceAccess.userEmail, user.email),
|
||||
eq(spaceAccess.status, "accepted")
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (spaceAccessCheck.length === 0) {
|
||||
return c.json({ error: "Access denied" }, 403);
|
||||
}
|
||||
|
||||
canEdit = spaceAccessCheck[0].accessType === "edit";
|
||||
}
|
||||
|
||||
return c.json({
|
||||
...space[0],
|
||||
permissions: {
|
||||
canRead: true,
|
||||
canEdit,
|
||||
isOwner: space[0].ownerId === user.id,
|
||||
isPublic: space[0].isPublic,
|
||||
},
|
||||
});
|
||||
})
|
||||
.post(
|
||||
"/create",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
spaceName: z.string().min(1, "Space name cannot be empty").max(100),
|
||||
isPublic: z.boolean(), // keep this explicit please
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const body = c.req.valid("json");
|
||||
const user = c.get("user");
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
if (body.spaceName.trim() === "<HOME>") {
|
||||
return c.json({ error: "Cannot create space with name <HOME>" }, 400);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
const uuid = randomId();
|
||||
|
||||
try {
|
||||
const space = await db
|
||||
.insert(spaces)
|
||||
.values({
|
||||
name: body.spaceName.trim(),
|
||||
ownerId: user.id,
|
||||
uuid,
|
||||
isPublic: body.isPublic,
|
||||
createdAt: new Date(),
|
||||
})
|
||||
.returning();
|
||||
|
||||
return c.json({
|
||||
message: "Space created successfully",
|
||||
space: {
|
||||
uuid: space[0].uuid,
|
||||
name: space[0].name,
|
||||
ownerId: space[0].ownerId,
|
||||
isPublic: space[0].isPublic,
|
||||
createdAt: space[0].createdAt,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("[Space Creation Error]", error);
|
||||
return c.json({ error: "Failed to create space" }, 500);
|
||||
}
|
||||
}
|
||||
)
|
||||
.post(
|
||||
"/:spaceId/favorite",
|
||||
zValidator(
|
||||
"param",
|
||||
z.object({
|
||||
spaceId: z.string(),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const user = c.get("user");
|
||||
const { spaceId } = c.req.valid("param");
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
// Get space details
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (!space[0]) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
// Check if it's user's own space
|
||||
if (space[0].ownerId === user.id) {
|
||||
return c.json({ error: "Cannot favorite your own space" }, 400);
|
||||
}
|
||||
|
||||
try {
|
||||
await db.insert(savedSpaces).values({
|
||||
userId: user.id,
|
||||
spaceId: space[0].id,
|
||||
savedAt: new Date(),
|
||||
});
|
||||
|
||||
return c.json({ message: "Space favorited successfully" });
|
||||
} catch (error) {
|
||||
if (
|
||||
error instanceof Error &&
|
||||
error.message.includes("saved_spaces_user_space_idx")
|
||||
) {
|
||||
// Space is already favorited
|
||||
return c.json({ message: "Space already favorited" });
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
)
|
||||
.post(
|
||||
"/moveContent",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
spaceId: z.string(),
|
||||
documentId: z.string(),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const body = c.req.valid("json");
|
||||
const user = c.get("user");
|
||||
const { spaceId, documentId } = body;
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
try {
|
||||
await db.transaction(async (tx) => {
|
||||
// If moving to <HOME>, just remove all space connections
|
||||
if (spaceId === "<HOME>") {
|
||||
const doc = await tx
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(eq(documents.uuid, documentId))
|
||||
.limit(1);
|
||||
|
||||
if (!doc[0]) {
|
||||
return c.json({ error: "Document not found" }, 404);
|
||||
}
|
||||
|
||||
await tx
|
||||
.delete(contentToSpace)
|
||||
.where(eq(contentToSpace.contentId, doc[0].id));
|
||||
return;
|
||||
}
|
||||
|
||||
// Get space and document, verify space ownership
|
||||
const results = (
|
||||
await tx
|
||||
.select({
|
||||
spaceId: spaces.id,
|
||||
documentId: documents.id,
|
||||
ownerId: spaces.ownerId,
|
||||
spaceName: spaces.name,
|
||||
})
|
||||
.from(spaces)
|
||||
.innerJoin(
|
||||
documents,
|
||||
and(eq(spaces.uuid, spaceId), eq(documents.uuid, documentId))
|
||||
)
|
||||
.limit(1)
|
||||
)[0];
|
||||
|
||||
if (!results) {
|
||||
return c.json({ error: "Space or document not found" }, 404);
|
||||
}
|
||||
|
||||
if (results.ownerId !== user.id) {
|
||||
return c.json(
|
||||
{ error: "Not authorized to modify this space" },
|
||||
403
|
||||
);
|
||||
}
|
||||
|
||||
// Delete existing space relations for this document
|
||||
await tx
|
||||
.delete(contentToSpace)
|
||||
.where(eq(contentToSpace.contentId, results.documentId));
|
||||
|
||||
// Add new space relation
|
||||
await tx.insert(contentToSpace).values({
|
||||
contentId: results.documentId,
|
||||
spaceId: results.spaceId,
|
||||
});
|
||||
});
|
||||
|
||||
return c.json({ success: true, spaceId });
|
||||
} catch (e) {
|
||||
console.error("Failed to move content to space:", e);
|
||||
return c.json(
|
||||
{
|
||||
error: "Failed to move content to space",
|
||||
details: e instanceof Error ? e.message : "Unknown error",
|
||||
},
|
||||
500
|
||||
);
|
||||
}
|
||||
}
|
||||
)
|
||||
.post(
|
||||
"/addContent",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
spaceId: z.string(),
|
||||
documentId: z.string(),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const body = c.req.valid("json");
|
||||
const user = c.get("user");
|
||||
const { spaceId, documentId } = body;
|
||||
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
try {
|
||||
await db.transaction(async (tx) => {
|
||||
// If adding to <HOME>, just remove all space connections
|
||||
if (spaceId === "<HOME>") {
|
||||
const doc = await tx
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(eq(documents.uuid, documentId))
|
||||
.limit(1);
|
||||
|
||||
if (!doc[0]) {
|
||||
return c.json({ error: "Document not found" }, 404);
|
||||
}
|
||||
|
||||
await tx
|
||||
.delete(contentToSpace)
|
||||
.where(eq(contentToSpace.contentId, doc[0].id));
|
||||
return;
|
||||
}
|
||||
|
||||
// Get space and document, verify space ownership
|
||||
const results = (
|
||||
await tx
|
||||
.select({
|
||||
spaceId: spaces.id,
|
||||
documentId: documents.id,
|
||||
ownerId: spaces.ownerId,
|
||||
})
|
||||
.from(spaces)
|
||||
.innerJoin(
|
||||
documents,
|
||||
and(eq(spaces.uuid, spaceId), eq(documents.uuid, documentId))
|
||||
)
|
||||
.limit(1)
|
||||
)[0];
|
||||
|
||||
if (!results) {
|
||||
return c.json({ error: "Space or document not found" }, 404);
|
||||
}
|
||||
|
||||
if (results.ownerId !== user.id) {
|
||||
return c.json(
|
||||
{ error: "Not authorized to modify this space" },
|
||||
403
|
||||
);
|
||||
}
|
||||
|
||||
// Check if mapping already exists to avoid duplicates
|
||||
const existing = await tx
|
||||
.select()
|
||||
.from(contentToSpace)
|
||||
.where(
|
||||
and(
|
||||
eq(contentToSpace.contentId, results.documentId),
|
||||
eq(contentToSpace.spaceId, results.spaceId)
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (existing.length > 0) {
|
||||
return c.json({ error: "Content already exists in space" }, 409);
|
||||
}
|
||||
|
||||
await tx.insert(contentToSpace).values({
|
||||
contentId: results.documentId,
|
||||
spaceId: results.spaceId,
|
||||
});
|
||||
});
|
||||
|
||||
return c.json({ success: true });
|
||||
} catch (e) {
|
||||
console.error("Failed to add content to space:", e);
|
||||
return c.json(
|
||||
{
|
||||
error: "Failed to add content to space",
|
||||
details: e instanceof Error ? e.message : "Unknown error",
|
||||
},
|
||||
500
|
||||
);
|
||||
}
|
||||
}
|
||||
)
|
||||
.post(
|
||||
"/:spaceId/invite",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
email: z.string().email("Invalid email address"),
|
||||
accessType: z.enum(["read", "edit"], {
|
||||
errorMap: () => ({
|
||||
message: "Access type must be either 'read' or 'edit'",
|
||||
}),
|
||||
}),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const { spaceId } = c.req.param();
|
||||
const { email, accessType } = c.req.valid("json");
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
// Check if space exists and user has permission to invite
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (space.length === 0) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
// Only space owner can invite others
|
||||
if (space[0].ownerId !== user.id) {
|
||||
return c.json({ error: "Only space owner can invite users" }, 403);
|
||||
}
|
||||
|
||||
// Check if invite already exists
|
||||
const existingInvite = await db
|
||||
.select()
|
||||
.from(spaceAccess)
|
||||
.where(
|
||||
and(
|
||||
eq(spaceAccess.spaceId, space[0].id),
|
||||
eq(spaceAccess.userEmail, email)
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (existingInvite.length > 0) {
|
||||
return c.json(
|
||||
{ error: "User already has access or pending invite" },
|
||||
400
|
||||
);
|
||||
}
|
||||
|
||||
// Create invite
|
||||
await db.insert(spaceAccess).values({
|
||||
spaceId: space[0].id,
|
||||
userEmail: email,
|
||||
accessType,
|
||||
status: "pending",
|
||||
});
|
||||
|
||||
// TODO: send email to the user
|
||||
|
||||
return c.json({ success: true });
|
||||
}
|
||||
)
|
||||
.get(
|
||||
"/:spaceId/invitation",
|
||||
zValidator("param", z.object({ spaceId: z.string() })),
|
||||
async (c) => {
|
||||
const { spaceId } = c.req.valid("param");
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (space.length === 0) {
|
||||
console.log("Space not found", spaceId);
|
||||
return c.json({ error: "Space not found" }, 401);
|
||||
}
|
||||
|
||||
// Get pending invitation with access type
|
||||
const invitation = await db
|
||||
.select()
|
||||
.from(spaceAccess)
|
||||
.where(
|
||||
and(
|
||||
eq(spaceAccess.spaceId, space[0].id),
|
||||
eq(spaceAccess.userEmail, user.email),
|
||||
eq(spaceAccess.status, "pending")
|
||||
)
|
||||
)
|
||||
.limit(1);
|
||||
|
||||
if (invitation.length === 0) {
|
||||
return c.json({ error: "No pending invitation found" }, 403);
|
||||
}
|
||||
|
||||
return c.json({
|
||||
space: space[0],
|
||||
accessType: invitation[0].accessType,
|
||||
});
|
||||
}
|
||||
)
|
||||
.post(
|
||||
"/invites/:action",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
spaceId: z.string().min(5, "Invalid space ID format"),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const { action } = c.req.param();
|
||||
if (action !== "accept" && action !== "reject") {
|
||||
return c.json({ error: "Invalid action" }, 400);
|
||||
}
|
||||
|
||||
const { spaceId } = c.req.valid("json");
|
||||
console.log("space ID", spaceId);
|
||||
|
||||
// Get space
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (space.length === 0) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
// Update invite status
|
||||
const updateResult = await db
|
||||
.update(spaceAccess)
|
||||
.set({ status: action === "accept" ? "accepted" : "rejected" })
|
||||
.where(
|
||||
and(
|
||||
eq(spaceAccess.spaceId, space[0].id),
|
||||
eq(spaceAccess.userEmail, user.email),
|
||||
eq(spaceAccess.status, "pending")
|
||||
)
|
||||
);
|
||||
|
||||
if (updateResult.length === 0) {
|
||||
return c.json({ error: "No pending invite found" }, 404);
|
||||
}
|
||||
|
||||
return c.json({ success: true });
|
||||
}
|
||||
)
|
||||
.patch(
|
||||
"/:spaceId",
|
||||
zValidator(
|
||||
"json",
|
||||
z.object({
|
||||
name: z.string().min(1, "Space name cannot be empty").max(100),
|
||||
})
|
||||
),
|
||||
async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const { spaceId } = c.req.param();
|
||||
const { name } = c.req.valid("json");
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
// Get space and verify ownership
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (space.length === 0) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
if (space[0].ownerId !== user.id) {
|
||||
return c.json({ error: "Only space owner can edit space name" }, 403);
|
||||
}
|
||||
|
||||
if (name.trim() === "<HOME>") {
|
||||
return c.json({ error: "Cannot use reserved name <HOME>" }, 400);
|
||||
}
|
||||
|
||||
// Update space name
|
||||
await db
|
||||
.update(spaces)
|
||||
.set({ name: name.trim() })
|
||||
.where(eq(spaces.uuid, spaceId));
|
||||
|
||||
return c.json({ success: true, name: name.trim() });
|
||||
}
|
||||
)
|
||||
.delete("/:spaceId", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const { spaceId } = c.req.param();
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const space = await db
|
||||
.select()
|
||||
.from(spaces)
|
||||
.where(eq(spaces.uuid, spaceId))
|
||||
.limit(1);
|
||||
|
||||
if (space.length === 0) {
|
||||
return c.json({ error: "Space not found" }, 404);
|
||||
}
|
||||
|
||||
await db.delete(spaces).where(eq(spaces.uuid, spaceId));
|
||||
|
||||
return c.json({ success: true });
|
||||
});
|
||||
|
||||
export default spacesRoute;
|
||||
|
|
@ -1,193 +0,0 @@
|
|||
import { Hono } from "hono";
|
||||
import { Env, Variables } from "../types";
|
||||
import { and, database, desc, eq, isNotNull, or, sql } from "@supermemory/db";
|
||||
import {
|
||||
chatThreads,
|
||||
savedSpaces,
|
||||
spaceAccess,
|
||||
spaces,
|
||||
users,
|
||||
} from "@supermemory/db/schema";
|
||||
import { decryptApiKey, getApiKey } from "../auth";
|
||||
import { DurableObjectStore } from "@hono-rate-limiter/cloudflare";
|
||||
import { rateLimiter } from "hono-rate-limiter";
|
||||
import { fromHono } from "chanfana";
|
||||
|
||||
const user = fromHono(new Hono<{ Variables: Variables; Bindings: Env }>())
|
||||
.get("/", (c) => {
|
||||
return c.json(c.get("user"));
|
||||
})
|
||||
.get("/spaces", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const [allSpaces, savedSpacesList, spaceOwners] = await Promise.all([
|
||||
db
|
||||
.select({
|
||||
id: spaces.id,
|
||||
uuid: spaces.uuid,
|
||||
name: sql<string>`REGEXP_REPLACE(${spaces.name}, E'[\\n\\r]+', ' ', 'g')`.as(
|
||||
"name"
|
||||
),
|
||||
ownerId: spaces.ownerId,
|
||||
isPublic: spaces.isPublic,
|
||||
createdAt: spaces.createdAt,
|
||||
accessType: spaceAccess.accessType,
|
||||
})
|
||||
.from(spaces)
|
||||
.leftJoin(
|
||||
spaceAccess,
|
||||
and(
|
||||
eq(spaces.id, spaceAccess.spaceId),
|
||||
eq(spaceAccess.userEmail, user.email),
|
||||
eq(spaceAccess.status, "accepted")
|
||||
)
|
||||
)
|
||||
.where(or(eq(spaces.ownerId, user.id), isNotNull(spaceAccess.spaceId)))
|
||||
.orderBy(desc(spaces.createdAt)),
|
||||
|
||||
db
|
||||
.select({
|
||||
spaceId: savedSpaces.spaceId,
|
||||
})
|
||||
.from(savedSpaces)
|
||||
.where(eq(savedSpaces.userId, user.id)),
|
||||
|
||||
db
|
||||
.select({
|
||||
id: users.id,
|
||||
uuid: users.uuid,
|
||||
name: users.firstName,
|
||||
email: users.email,
|
||||
profileImage: users.profilePictureUrl,
|
||||
})
|
||||
.from(users)
|
||||
.innerJoin(spaces, eq(spaces.ownerId, users.id)),
|
||||
]);
|
||||
|
||||
const savedSpaceIds = new Set(savedSpacesList.map((s) => s.spaceId));
|
||||
const ownerMap = new Map(spaceOwners.map((owner) => [owner.id, owner]));
|
||||
|
||||
const spacesWithDetails = allSpaces.map((space) => {
|
||||
const isOwner = space.ownerId === user.id;
|
||||
const owner = ownerMap.get(space.ownerId);
|
||||
|
||||
return {
|
||||
...space,
|
||||
favorited: savedSpaceIds.has(space.id),
|
||||
permissions: {
|
||||
canRead: space.isPublic || isOwner || space.accessType != null,
|
||||
canEdit: isOwner || space.accessType === "edit",
|
||||
isOwner,
|
||||
},
|
||||
owner: isOwner
|
||||
? null
|
||||
: {
|
||||
id: owner?.uuid,
|
||||
name: owner?.name,
|
||||
email: owner?.email,
|
||||
profileImage: owner?.profileImage,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
return c.json({ spaces: spacesWithDetails });
|
||||
})
|
||||
.get("/history", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const history = await database(c.env.HYPERDRIVE.connectionString)
|
||||
.select()
|
||||
.from(chatThreads)
|
||||
.where(eq(chatThreads.userId, user.id))
|
||||
.orderBy(desc(chatThreads.createdAt))
|
||||
.limit(10);
|
||||
|
||||
return c.json({ history });
|
||||
})
|
||||
.get("/invitations", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
const invitations = await db
|
||||
.select({
|
||||
spaceAccess: spaceAccess,
|
||||
spaceUuid: spaces.uuid,
|
||||
spaceName: spaces.name,
|
||||
})
|
||||
.from(spaceAccess)
|
||||
.innerJoin(spaces, eq(spaceAccess.spaceId, spaces.id))
|
||||
.where(eq(spaceAccess.userEmail, user.email))
|
||||
.limit(100);
|
||||
|
||||
return c.json({ invitations });
|
||||
})
|
||||
.get("/key", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
// we need user.id and user.lastApiKeyGeneratedAt
|
||||
const lastApiKeyGeneratedAt = user.lastApiKeyGeneratedAt?.getTime();
|
||||
if (!lastApiKeyGeneratedAt) {
|
||||
return c.json({ error: "No API key generated" }, 400);
|
||||
}
|
||||
|
||||
const key = await getApiKey(user.uuid, lastApiKeyGeneratedAt.toString(), c);
|
||||
|
||||
const decrypted = await decryptApiKey(key, c);
|
||||
return c.json({ key, decrypted });
|
||||
})
|
||||
.post("/update", async (c) => {
|
||||
const user = c.get("user");
|
||||
if (!user) {
|
||||
return c.json({ error: "Unauthorized" }, 401);
|
||||
}
|
||||
|
||||
const body = await c.req.json();
|
||||
|
||||
// Only allow updating specific safe fields
|
||||
const allowedFields = {
|
||||
firstName: true,
|
||||
lastName: true,
|
||||
profilePictureUrl: true,
|
||||
hasOnboarded: true,
|
||||
};
|
||||
|
||||
const updateData: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(body)) {
|
||||
if (allowedFields[key as keyof typeof allowedFields]) {
|
||||
updateData[key] = value;
|
||||
}
|
||||
}
|
||||
|
||||
if (Object.keys(updateData).length === 0) {
|
||||
return c.json({ error: "No valid fields to update" }, 400);
|
||||
}
|
||||
|
||||
const db = database(c.env.HYPERDRIVE.connectionString);
|
||||
|
||||
await db
|
||||
.update(users)
|
||||
.set({
|
||||
...updateData,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(users.id, user.id));
|
||||
|
||||
return c.json({ success: true });
|
||||
});
|
||||
|
||||
export default user;
|
||||
|
|
@ -1,79 +0,0 @@
|
|||
import { DurableObjectRateLimiter } from "@hono-rate-limiter/cloudflare";
|
||||
import { Session } from "@supermemory/authkit-remix-cloudflare/src/interfaces";
|
||||
import { User } from "@supermemory/db/schema";
|
||||
import { z } from "zod";
|
||||
|
||||
export type Variables = {
|
||||
user: User | null;
|
||||
session: Session | null;
|
||||
};
|
||||
|
||||
export type WorkflowParams = {
|
||||
userId: number;
|
||||
content: string;
|
||||
spaces?: string[];
|
||||
type: string;
|
||||
uuid: string;
|
||||
url?: string;
|
||||
prefetched?: {
|
||||
contentToVectorize: string;
|
||||
contentToSave: string;
|
||||
title: string;
|
||||
type: string;
|
||||
description: string;
|
||||
ogImage: string;
|
||||
};
|
||||
createdAt: string;
|
||||
};
|
||||
|
||||
export type Env = {
|
||||
WORKOS_API_KEY: string;
|
||||
WORKOS_CLIENT_ID: string;
|
||||
WORKOS_COOKIE_PASSWORD: string;
|
||||
DATABASE_URL: string;
|
||||
CONTENT_WORKFLOW: Workflow;
|
||||
GEMINI_API_KEY: string;
|
||||
NODE_ENV: string;
|
||||
OPEN_AI_API_KEY: string;
|
||||
BRAINTRUST_API_KEY: string;
|
||||
RESEND_API_KEY: string;
|
||||
TURNSTILE_SECRET_KEY: string;
|
||||
|
||||
MD_CACHE: KVNamespace;
|
||||
HYPERDRIVE: Hyperdrive;
|
||||
EMAIL_LIMITER: {
|
||||
limit: (params: { key: string }) => Promise<{ success: boolean }>;
|
||||
};
|
||||
ENCRYPTED_TOKENS: KVNamespace;
|
||||
RATE_LIMITER: DurableObjectNamespace<DurableObjectRateLimiter>;
|
||||
AI: Ai
|
||||
};
|
||||
|
||||
export type JobData = {
|
||||
content: string;
|
||||
spaces?: Array<string>;
|
||||
user: number;
|
||||
type: string;
|
||||
};
|
||||
|
||||
type BaseChunks = {
|
||||
type: "tweet" | "page" | "note" | "image";
|
||||
};
|
||||
|
||||
export type PageOrNoteChunks = BaseChunks & {
|
||||
type: "page" | "note";
|
||||
chunks: string[];
|
||||
};
|
||||
|
||||
export type Metadata = {
|
||||
media?: Array<string>;
|
||||
links?: Array<string>; // idk how ideal this is will figure out after plate js thing
|
||||
};
|
||||
|
||||
export type SpaceStatus = {
|
||||
type: "inviting" | "invited" | "pending" | "accepted";
|
||||
};
|
||||
|
||||
export const recommendedQuestionsSchema = z
|
||||
.array(z.string().max(200))
|
||||
.length(10);
|
||||
|
|
@ -1,116 +0,0 @@
|
|||
import nlp from "compromise";
|
||||
|
||||
export default function chunkText(
|
||||
text: string,
|
||||
maxChunkSize: number,
|
||||
overlap: number = 0.2
|
||||
): string[] {
|
||||
// Pre-process text to remove excessive whitespace
|
||||
text = text.replace(/\s+/g, " ").trim();
|
||||
|
||||
const sentences = nlp(text).sentences().out("array");
|
||||
const chunks: {
|
||||
text: string;
|
||||
start: number;
|
||||
end: number;
|
||||
metadata?: {
|
||||
position: string;
|
||||
context?: string;
|
||||
};
|
||||
}[] = [];
|
||||
|
||||
let currentChunk: string[] = [];
|
||||
let currentSize = 0;
|
||||
|
||||
for (let i = 0; i < sentences.length; i++) {
|
||||
const sentence = sentences[i].trim();
|
||||
|
||||
// Skip empty sentences
|
||||
if (!sentence) continue;
|
||||
|
||||
// If a single sentence is longer than maxChunkSize, split it
|
||||
if (sentence.length > maxChunkSize) {
|
||||
if (currentChunk.length > 0) {
|
||||
chunks.push({
|
||||
text: currentChunk.join(" "),
|
||||
start: i - currentChunk.length,
|
||||
end: i - 1,
|
||||
metadata: {
|
||||
position: `${i - currentChunk.length}-${i - 1}`,
|
||||
context: currentChunk[0].substring(0, 100), // First 100 chars for context
|
||||
},
|
||||
});
|
||||
currentChunk = [];
|
||||
currentSize = 0;
|
||||
}
|
||||
|
||||
// Split long sentence into smaller chunks
|
||||
const words = sentence.split(" ");
|
||||
let tempChunk: string[] = [];
|
||||
|
||||
for (const word of words) {
|
||||
if (tempChunk.join(" ").length + word.length > maxChunkSize) {
|
||||
chunks.push({
|
||||
text: tempChunk.join(" "),
|
||||
start: i,
|
||||
end: i,
|
||||
metadata: {
|
||||
position: `${i}`,
|
||||
context: "Split sentence",
|
||||
},
|
||||
});
|
||||
tempChunk = [];
|
||||
}
|
||||
tempChunk.push(word);
|
||||
}
|
||||
|
||||
if (tempChunk.length > 0) {
|
||||
chunks.push({
|
||||
text: tempChunk.join(" "),
|
||||
start: i,
|
||||
end: i,
|
||||
metadata: {
|
||||
position: `${i}`,
|
||||
context: "Split sentence remainder",
|
||||
},
|
||||
});
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
currentChunk.push(sentence);
|
||||
currentSize += sentence.length;
|
||||
|
||||
if (currentSize >= maxChunkSize) {
|
||||
const overlapSize = Math.floor(currentChunk.length * overlap);
|
||||
chunks.push({
|
||||
text: currentChunk.join(" "),
|
||||
start: i - currentChunk.length + 1,
|
||||
end: i,
|
||||
metadata: {
|
||||
position: `${i - currentChunk.length + 1}-${i}`,
|
||||
context: currentChunk[0].substring(0, 100),
|
||||
},
|
||||
});
|
||||
|
||||
// Keep overlap sentences for next chunk
|
||||
currentChunk = currentChunk.slice(-overlapSize);
|
||||
currentSize = currentChunk.reduce((sum, s) => sum + s.length, 0);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle remaining sentences
|
||||
if (currentChunk.length > 0) {
|
||||
chunks.push({
|
||||
text: currentChunk.join(" "),
|
||||
start: sentences.length - currentChunk.length,
|
||||
end: sentences.length - 1,
|
||||
metadata: {
|
||||
position: `${sentences.length - currentChunk.length}-${sentences.length - 1}`,
|
||||
context: currentChunk[0].substring(0, 100),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
return chunks.map((chunk) => chunk.text);
|
||||
}
|
||||
|
|
@ -1,79 +0,0 @@
|
|||
async function encrypt(data: string, key: string): Promise<string> {
|
||||
try {
|
||||
const encoder = new TextEncoder();
|
||||
const encodedData = encoder.encode(data);
|
||||
|
||||
const baseForIv = encoder.encode(data + key);
|
||||
const ivHash = await crypto.subtle.digest('SHA-256', baseForIv);
|
||||
const iv = new Uint8Array(ivHash).slice(0, 12);
|
||||
|
||||
const cryptoKey = await crypto.subtle.importKey(
|
||||
"raw",
|
||||
encoder.encode(key),
|
||||
{ name: "AES-GCM", length: 256 },
|
||||
false,
|
||||
["encrypt", "decrypt"]
|
||||
);
|
||||
|
||||
const encrypted = await crypto.subtle.encrypt(
|
||||
{ name: "AES-GCM", iv: new Uint8Array(iv).buffer as ArrayBuffer },
|
||||
cryptoKey,
|
||||
encodedData
|
||||
);
|
||||
|
||||
const combined = new Uint8Array([...iv, ...new Uint8Array(encrypted)]);
|
||||
|
||||
// Convert to base64 safely
|
||||
const base64 = Buffer.from(combined).toString("base64");
|
||||
|
||||
// Make URL-safe
|
||||
return base64.replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
|
||||
} catch (err) {
|
||||
console.error("Encryption error:", err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
async function decrypt(encryptedData: string, key: string): Promise<string> {
|
||||
try {
|
||||
// Restore base64 padding and convert URL-safe chars
|
||||
const base64 = encryptedData
|
||||
.replace(/-/g, "+")
|
||||
.replace(/_/g, "/")
|
||||
.padEnd(
|
||||
encryptedData.length + ((4 - (encryptedData.length % 4)) % 4),
|
||||
"="
|
||||
);
|
||||
|
||||
// Use Buffer for safer base64 decoding
|
||||
const combined = Buffer.from(base64, "base64");
|
||||
const combinedArray = new Uint8Array(combined);
|
||||
|
||||
// Extract the IV that was used for encryption
|
||||
const iv = combinedArray.slice(0, 12);
|
||||
const encrypted = combinedArray.slice(12);
|
||||
|
||||
// Import the same key used for encryption
|
||||
const cryptoKey = await crypto.subtle.importKey(
|
||||
"raw",
|
||||
new TextEncoder().encode(key),
|
||||
{ name: "AES-GCM", length: 256 },
|
||||
false,
|
||||
["encrypt", "decrypt"]
|
||||
);
|
||||
|
||||
// Use the extracted IV and key to decrypt
|
||||
const decrypted = await crypto.subtle.decrypt(
|
||||
{ name: "AES-GCM", iv: new Uint8Array(iv).buffer as ArrayBuffer },
|
||||
cryptoKey,
|
||||
encrypted.buffer as ArrayBuffer
|
||||
);
|
||||
|
||||
return new TextDecoder().decode(decrypted);
|
||||
} catch (err) {
|
||||
console.error("Decryption error:", err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
export { encrypt, decrypt };
|
||||
|
|
@ -1,87 +0,0 @@
|
|||
import * as mammoth from "mammoth";
|
||||
import { NonRetryableError } from "cloudflare:workflows";
|
||||
import { resolvePDFJS } from 'pdfjs-serverless';
|
||||
|
||||
interface DocumentContent {
|
||||
content: string;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export const extractDocumentContent = async (
|
||||
url: string
|
||||
): Promise<DocumentContent> => {
|
||||
try {
|
||||
const fileExtension = url.split(".").pop()?.toLowerCase();
|
||||
|
||||
if (!fileExtension) {
|
||||
throw new Error("Invalid file URL");
|
||||
}
|
||||
|
||||
console.log("file", fileExtension);
|
||||
|
||||
switch (fileExtension) {
|
||||
case "pdf":
|
||||
return await extractPdfContent(url);
|
||||
case "md":
|
||||
case "txt":
|
||||
return await extractTextContent(url);
|
||||
case "doc":
|
||||
case "docx":
|
||||
return await extractWordContent(url);
|
||||
default:
|
||||
throw new NonRetryableError(`Unsupported file type: ${fileExtension}`);
|
||||
}
|
||||
} catch (error) {
|
||||
return {
|
||||
content: "",
|
||||
error: error instanceof Error ? error.message : "Unknown error occurred",
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
async function extractPdfContent(url: string): Promise<DocumentContent> {
|
||||
try {
|
||||
const response = await fetch(url);
|
||||
const arrayBuffer = await response.arrayBuffer();
|
||||
|
||||
// Initialize PDF.js with serverless compatibility
|
||||
const { getDocument } = await resolvePDFJS();
|
||||
|
||||
// Load the PDF document
|
||||
const pdf = await getDocument({
|
||||
data: arrayBuffer,
|
||||
useSystemFonts: true,
|
||||
}).promise;
|
||||
|
||||
let fullText = "";
|
||||
|
||||
// Extract text from each page
|
||||
for (let i = 1; i <= pdf.numPages; i++) {
|
||||
const page = await pdf.getPage(i);
|
||||
const textContent = await page.getTextContent();
|
||||
const pageText = textContent.items.map((item: any) => item.str).join(" ");
|
||||
fullText += pageText + "\n";
|
||||
}
|
||||
|
||||
return { content: fullText };
|
||||
} catch (error) {
|
||||
console.error("Error extracting PDF content:", error);
|
||||
return {
|
||||
content: "",
|
||||
error: error instanceof Error ? error.message : "Failed to extract PDF content",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
async function extractTextContent(url: string): Promise<DocumentContent> {
|
||||
const response = await fetch(url);
|
||||
const text = await response.text();
|
||||
return { content: text };
|
||||
}
|
||||
|
||||
async function extractWordContent(url: string): Promise<DocumentContent> {
|
||||
const response = await fetch(url);
|
||||
const arrayBuffer = await response.arrayBuffer();
|
||||
const result = await mammoth.extractRawText({ arrayBuffer });
|
||||
return { content: result.value };
|
||||
}
|
||||
|
|
@ -1,50 +0,0 @@
|
|||
import { Env } from "../types";
|
||||
|
||||
export const extractPageContent = async (content: string, env: Env) => {
|
||||
const resp = await fetch(`https://r.jina.ai/${content}`);
|
||||
|
||||
if (!resp.ok) {
|
||||
throw new Error(
|
||||
`Failed to fetch ${content}: ${resp.statusText}` + (await resp.text())
|
||||
);
|
||||
}
|
||||
|
||||
const metadataResp = await fetch(`https://md.dhr.wtf/metadata?url=${content}`);
|
||||
|
||||
if (!metadataResp.ok) {
|
||||
throw new Error(
|
||||
`Failed to fetch metadata for ${content}: ${metadataResp.statusText}` +
|
||||
(await metadataResp.text())
|
||||
);
|
||||
}
|
||||
|
||||
const metadata = await metadataResp.json() as {
|
||||
title?: string;
|
||||
description?: string;
|
||||
image?: string;
|
||||
favicon?: string;
|
||||
};
|
||||
|
||||
const responseText = await resp.text();
|
||||
|
||||
try {
|
||||
const json: {
|
||||
contentToVectorize: string;
|
||||
contentToSave: string;
|
||||
title?: string;
|
||||
description?: string;
|
||||
image?: string;
|
||||
favicon?: string;
|
||||
} = {
|
||||
contentToSave: responseText,
|
||||
contentToVectorize: responseText,
|
||||
title: metadata.title,
|
||||
description: metadata.description,
|
||||
image: metadata.image,
|
||||
favicon: metadata.favicon,
|
||||
};
|
||||
return json;
|
||||
} catch (e) {
|
||||
throw new Error(`Failed to parse JSON from ${content}: ${e}`);
|
||||
}
|
||||
};
|
||||
|
|
@ -1,143 +0,0 @@
|
|||
import { WorkflowStep } from "cloudflare:workers";
|
||||
import { isErr, Ok } from "../errors/results";
|
||||
import { typeDecider } from "./typeDecider";
|
||||
import { Env, WorkflowParams } from "../types";
|
||||
import { unrollTweets } from "./tweetsToThreads";
|
||||
import { Tweet } from "react-tweet/api";
|
||||
import { NonRetryableError } from "cloudflare:workflows";
|
||||
import { extractPageContent } from "./extractor";
|
||||
import { extractDocumentContent } from "./extractDocumentContent";
|
||||
|
||||
export const fetchContent = async (
|
||||
params: WorkflowParams,
|
||||
env: Env,
|
||||
step: WorkflowStep
|
||||
) => {
|
||||
const type = typeDecider(params.content);
|
||||
|
||||
if (isErr(type)) {
|
||||
throw type.error;
|
||||
}
|
||||
|
||||
switch (type.value) {
|
||||
case "page":
|
||||
const pageContent = await step?.do(
|
||||
"extract page content",
|
||||
async () => await extractPageContent(params.content, env)
|
||||
);
|
||||
return {
|
||||
...pageContent,
|
||||
type: "page",
|
||||
};
|
||||
|
||||
case "tweet":
|
||||
const tweetUrl = new URL(params.content);
|
||||
tweetUrl.search = ""; // Remove all search params
|
||||
const tweetId = tweetUrl.pathname.split("/").pop();
|
||||
|
||||
const rawBaseTweetContent = await step.do(
|
||||
"extract tweet content",
|
||||
async () => {
|
||||
const url = `https://cdn.syndication.twimg.com/tweet-result?id=${tweetId}&lang=en&features=tfw_timeline_list%3A%3Btfw_follower_count_sunset%3Atrue%3Btfw_tweet_edit_backend%3Aon%3Btfw_refsrc_session%3Aon%3Btfw_fosnr_soft_interventions_enabled%3Aon%3Btfw_show_birdwatch_pivots_enabled%3Aon%3Btfw_show_business_verified_badge%3Aon%3Btfw_duplicate_scribes_to_settings%3Aon%3Btfw_use_profile_image_shape_enabled%3Aon%3Btfw_show_blue_verified_badge%3Aon%3Btfw_legacy_timeline_sunset%3Atrue%3Btfw_show_gov_verified_badge%3Aon%3Btfw_show_business_affiliate_badge%3Aon%3Btfw_tweet_edit_frontend%3Aon&token=4c2mmul6mnh`;
|
||||
|
||||
const resp = await fetch(url, {
|
||||
headers: {
|
||||
"User-Agent":
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
|
||||
Accept: "application/json",
|
||||
"Accept-Language": "en-US,en;q=0.5",
|
||||
"Accept-Encoding": "gzip, deflate, br",
|
||||
Connection: "keep-alive",
|
||||
"Upgrade-Insecure-Requests": "1",
|
||||
"Cache-Control": "max-age=0",
|
||||
TE: "Trailers",
|
||||
},
|
||||
});
|
||||
|
||||
const data = (await resp.json()) as Tweet;
|
||||
return data;
|
||||
}
|
||||
);
|
||||
|
||||
let tweetContent: {
|
||||
text: string;
|
||||
metadata: {
|
||||
media?: string[] | undefined;
|
||||
links?: string[] | undefined;
|
||||
};
|
||||
raw: string;
|
||||
};
|
||||
const unrolledTweetContent = {
|
||||
value: [rawBaseTweetContent],
|
||||
};
|
||||
if (true) {
|
||||
console.error("Can't get thread, reverting back to single tweet");
|
||||
tweetContent = {
|
||||
text: rawBaseTweetContent.text,
|
||||
metadata: {
|
||||
media: [
|
||||
...(rawBaseTweetContent.photos?.map((url) => url.expandedUrl) ??
|
||||
[]),
|
||||
...(rawBaseTweetContent.video?.variants[0].src ?? []),
|
||||
],
|
||||
},
|
||||
raw: `<raw>${JSON.stringify(rawBaseTweetContent)}</raw>`,
|
||||
};
|
||||
} else {
|
||||
tweetContent = {
|
||||
text: unrolledTweetContent.value
|
||||
.map((tweet) => tweet.text)
|
||||
.join("\n"),
|
||||
metadata: {
|
||||
media: unrolledTweetContent.value.flatMap((tweet) => [
|
||||
...tweet.videos,
|
||||
...tweet.images,
|
||||
]),
|
||||
links: unrolledTweetContent.value.flatMap((tweet) => tweet.links),
|
||||
},
|
||||
raw: `<raw>${JSON.stringify(rawBaseTweetContent)}</raw>`,
|
||||
};
|
||||
}
|
||||
|
||||
// make it the same type as the page content
|
||||
const pageContentType: Awaited<ReturnType<typeof extractPageContent>> & {
|
||||
type: string;
|
||||
} = {
|
||||
contentToVectorize:
|
||||
tweetContent.text +
|
||||
"\n\nMetadata for this tweet:\n" +
|
||||
JSON.stringify(tweetContent.metadata) +
|
||||
"\n\nRaw tweet data:\n" +
|
||||
tweetContent.raw,
|
||||
contentToSave: tweetContent.raw,
|
||||
title: "",
|
||||
description: JSON.stringify(tweetContent.metadata),
|
||||
image: "",
|
||||
favicon: "",
|
||||
type: "tweet",
|
||||
};
|
||||
return pageContentType;
|
||||
case "note":
|
||||
const noteContent = {
|
||||
contentToVectorize: params.content,
|
||||
// TODO: different when using platejs
|
||||
contentToSave: params.content,
|
||||
// title is the first 30 characters of the first line
|
||||
title: params.content.split("\n")[0].slice(0, 30),
|
||||
type: "note",
|
||||
};
|
||||
return noteContent;
|
||||
case "document":
|
||||
const documentContent = await step.do(
|
||||
"extract document content",
|
||||
async () => await extractDocumentContent(params.content)
|
||||
);
|
||||
return {
|
||||
contentToVectorize: documentContent.content,
|
||||
contentToSave: documentContent.content,
|
||||
type: "document",
|
||||
};
|
||||
default:
|
||||
throw new NonRetryableError("Unknown content type");
|
||||
}
|
||||
};
|
||||
|
|
@ -1,239 +0,0 @@
|
|||
interface PageContent {
|
||||
content: string;
|
||||
url: string;
|
||||
title: string;
|
||||
id: string;
|
||||
createdAt: string;
|
||||
}
|
||||
|
||||
interface NotionBlock {
|
||||
type: string;
|
||||
[key: string]: any;
|
||||
}
|
||||
|
||||
interface SearchResponse {
|
||||
results: {
|
||||
id: string;
|
||||
object: string;
|
||||
url: string;
|
||||
created_time: string;
|
||||
properties: {
|
||||
title?: {
|
||||
title: Array<{
|
||||
plain_text: string;
|
||||
}>;
|
||||
};
|
||||
Name?: {
|
||||
title: Array<{
|
||||
plain_text: string;
|
||||
}>;
|
||||
};
|
||||
};
|
||||
}[];
|
||||
next_cursor: string | undefined;
|
||||
has_more: boolean;
|
||||
}
|
||||
|
||||
interface BlockResponse {
|
||||
results: NotionBlock[];
|
||||
next_cursor: string | undefined;
|
||||
has_more: boolean;
|
||||
}
|
||||
|
||||
export const getAllNotionPageContents = async (
|
||||
token: string,
|
||||
onProgress: (progress: number) => Promise<void>
|
||||
): Promise<PageContent[]> => {
|
||||
const pages: PageContent[] = [];
|
||||
const NOTION_API_VERSION = "2022-06-28";
|
||||
const BASE_URL = "https://api.notion.com/v1";
|
||||
const MAX_RETRIES = 3;
|
||||
const BATCH_SIZE = 10; // Number of concurrent requests
|
||||
const PAGE_SIZE = 100; // Number of pages to fetch per search request
|
||||
|
||||
const delay = (ms: number) =>
|
||||
new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
const notionFetch = async (
|
||||
endpoint: string,
|
||||
options: RequestInit = {},
|
||||
retries = 0
|
||||
): Promise<any> => {
|
||||
try {
|
||||
const response = await fetch(`${BASE_URL}${endpoint}`, {
|
||||
...options,
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Notion-Version": NOTION_API_VERSION,
|
||||
"Content-Type": "application/json",
|
||||
...((options.headers || {}) as Record<string, string>),
|
||||
},
|
||||
});
|
||||
|
||||
if (response.status === 429) {
|
||||
// Rate limit error
|
||||
const retryAfter = parseInt(response.headers.get("Retry-After") || "5");
|
||||
if (retries < MAX_RETRIES) {
|
||||
await delay(retryAfter * 1000);
|
||||
return notionFetch(endpoint, options, retries + 1);
|
||||
}
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
throw new Error(
|
||||
`Notion API error: ${response.statusText}\n${errorText}`
|
||||
);
|
||||
}
|
||||
|
||||
return response.json();
|
||||
} catch (error) {
|
||||
if (retries < MAX_RETRIES) {
|
||||
await delay(2000 * (retries + 1)); // Exponential backoff
|
||||
return notionFetch(endpoint, options, retries + 1);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
|
||||
const convertBlockToMarkdown = (block: NotionBlock): string => {
|
||||
switch (block.type) {
|
||||
case "paragraph":
|
||||
return (
|
||||
block.paragraph?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("") || ""
|
||||
);
|
||||
case "heading_1":
|
||||
return `# ${block.heading_1?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
case "heading_2":
|
||||
return `## ${block.heading_2?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
case "heading_3":
|
||||
return `### ${block.heading_3?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
case "bulleted_list_item":
|
||||
return `* ${block.bulleted_list_item?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
case "numbered_list_item":
|
||||
return `1. ${block.numbered_list_item?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
case "to_do":
|
||||
const checked = block.to_do?.checked ? "x" : " ";
|
||||
return `- [${checked}] ${block.to_do?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
case "code":
|
||||
return `\`\`\`${block.code?.language || ""}\n${block.code?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n\`\`\`\n`;
|
||||
case "quote":
|
||||
return `> ${block.quote?.rich_text
|
||||
?.map((text: any) => text.plain_text)
|
||||
.join("")}\n`;
|
||||
default:
|
||||
return "";
|
||||
}
|
||||
};
|
||||
|
||||
const getAllBlocks = async (pageId: string): Promise<NotionBlock[]> => {
|
||||
const blocks: NotionBlock[] = [];
|
||||
let cursor: string | undefined = undefined;
|
||||
|
||||
do {
|
||||
const endpoint = `/blocks/${pageId}/children${
|
||||
cursor ? `?start_cursor=${cursor}` : ""
|
||||
}`;
|
||||
const response = (await notionFetch(endpoint)) as BlockResponse;
|
||||
blocks.push(...response.results);
|
||||
cursor = response.next_cursor;
|
||||
} while (cursor);
|
||||
|
||||
return blocks;
|
||||
};
|
||||
|
||||
try {
|
||||
let hasMore = true;
|
||||
let cursor: string | undefined = undefined;
|
||||
let allPages: SearchResponse["results"] = [];
|
||||
|
||||
// First, collect all pages
|
||||
while (hasMore) {
|
||||
const searchResponse = (await notionFetch("/search", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
filter: {
|
||||
value: "page",
|
||||
property: "object",
|
||||
},
|
||||
sort: {
|
||||
direction: "ascending",
|
||||
timestamp: "last_edited_time",
|
||||
},
|
||||
start_cursor: cursor,
|
||||
page_size: PAGE_SIZE,
|
||||
}),
|
||||
})) as SearchResponse;
|
||||
|
||||
allPages = [...allPages, ...searchResponse.results];
|
||||
cursor = searchResponse.next_cursor;
|
||||
hasMore = searchResponse.has_more;
|
||||
|
||||
// Report progress for page collection (0-30%)
|
||||
const progressPercent = (allPages.length / (allPages.length + searchResponse.results.length)) * 30;
|
||||
await onProgress(progressPercent);
|
||||
}
|
||||
|
||||
// Process pages in parallel batches
|
||||
for (let i = 0; i < allPages.length; i += BATCH_SIZE) {
|
||||
const batch = allPages.slice(i, i + BATCH_SIZE);
|
||||
const batchResults = await Promise.all(
|
||||
batch.map(async (page) => {
|
||||
try {
|
||||
const blocks = await getAllBlocks(page.id);
|
||||
const pageContent = {
|
||||
content: blocks.map(convertBlockToMarkdown).join("\n"),
|
||||
url: page.url || `https://notion.so/${page.id.replace(/-/g, "")}`,
|
||||
title:
|
||||
page.properties?.Name?.title?.[0]?.plain_text ||
|
||||
page.properties?.title?.title?.[0]?.plain_text ||
|
||||
"Untitled",
|
||||
id: page.id,
|
||||
createdAt: page.created_time,
|
||||
};
|
||||
return pageContent.content.length > 10 ? pageContent : null;
|
||||
} catch (error) {
|
||||
console.error(`Error processing page ${page.id}:`, error);
|
||||
return null;
|
||||
}
|
||||
})
|
||||
);
|
||||
|
||||
pages.push(
|
||||
...batchResults.filter(
|
||||
(result): result is PageContent => result !== null
|
||||
)
|
||||
);
|
||||
|
||||
// Report progress for page processing (30-100%)
|
||||
const progressPercent = 30 + ((i + BATCH_SIZE) / allPages.length) * 70;
|
||||
await onProgress(Math.min(progressPercent, 100));
|
||||
|
||||
// Add a small delay between batches to respect rate limits
|
||||
if (i + BATCH_SIZE < allPages.length) {
|
||||
await delay(1000);
|
||||
}
|
||||
}
|
||||
|
||||
return pages.filter((page) => page.content.length > 10);
|
||||
} catch (error) {
|
||||
console.error("Error fetching Notion pages:", error);
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
|
|
@ -1,108 +0,0 @@
|
|||
import * as cheerio from "cheerio";
|
||||
import { BaseError } from "../errors/baseError";
|
||||
import { Ok, Result } from "../errors/results";
|
||||
|
||||
interface Tweet {
|
||||
id: string;
|
||||
text: string;
|
||||
links: Array<string>;
|
||||
images: Array<string>;
|
||||
videos: Array<string>;
|
||||
}
|
||||
|
||||
class ProcessTweetsError extends BaseError {
|
||||
constructor(message?: string, source?: string) {
|
||||
super("[Thread Proceessing Error]", message, source);
|
||||
}
|
||||
}
|
||||
|
||||
type TweetProcessResult = Array<Tweet>;
|
||||
|
||||
// there won't be a need for url caching right?
|
||||
export async function unrollTweets(
|
||||
url: string
|
||||
): Promise<Result<TweetProcessResult, ProcessTweetsError>> {
|
||||
const tweetId = url.split("/").pop();
|
||||
const response = await fetch(`https://unrollnow.com/status/${tweetId}`, {
|
||||
headers: {
|
||||
"User-Agent":
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36",
|
||||
"Cache-Control": "max-age=3600",
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.text();
|
||||
console.error(error);
|
||||
throw new Error(`HTTP error! status: ${response.status} - ${error}`);
|
||||
}
|
||||
|
||||
const html = await response.text();
|
||||
const $ = cheerio.load(html);
|
||||
const tweets: Array<Tweet> = [];
|
||||
|
||||
const urlRegex = /(https?:\/\/\S+)/g;
|
||||
const paragraphs = $(".mainarticle p").toArray();
|
||||
|
||||
const processedTweets = await Promise.all(
|
||||
paragraphs.map(async (element, i) => {
|
||||
const $tweet = $(element);
|
||||
let tweetText = $tweet.text().trim();
|
||||
if (tweetText.length < 1) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (i === paragraphs.length - 1 && tweetText.toLowerCase() === "yes") {
|
||||
return null;
|
||||
}
|
||||
|
||||
const shortUrls = tweetText.match(urlRegex) || [];
|
||||
console.log("SHORT_URLS_LEN", shortUrls.length);
|
||||
console.log("SHORT_URLS", shortUrls);
|
||||
|
||||
const expandedUrls = await Promise.all(shortUrls.map(expandShortUrl));
|
||||
|
||||
tweetText = tweetText.replace(urlRegex, "").trim().replace(/\s+/g, " ");
|
||||
|
||||
const images = $tweet
|
||||
.nextUntil("p")
|
||||
.find("img.tweetimg")
|
||||
.map((i, img) => $(img).attr("src"))
|
||||
.get();
|
||||
|
||||
const videos = $tweet
|
||||
.nextUntil("p")
|
||||
.find("video > source")
|
||||
.map((i, vid) => $(vid).attr("src"))
|
||||
.get();
|
||||
|
||||
return {
|
||||
id: `${tweetId}_${i}`,
|
||||
text: tweetText,
|
||||
links: expandedUrls,
|
||||
images: images,
|
||||
videos: videos,
|
||||
};
|
||||
})
|
||||
);
|
||||
|
||||
tweets.push(
|
||||
...processedTweets.filter((tweet): tweet is Tweet => tweet !== null)
|
||||
);
|
||||
|
||||
return Ok(tweets);
|
||||
}
|
||||
|
||||
async function expandShortUrl(shortUrl: string): Promise<string> {
|
||||
try {
|
||||
const response = await fetch(shortUrl, {
|
||||
method: "HEAD",
|
||||
redirect: "follow",
|
||||
});
|
||||
const expandedUrl = response.url;
|
||||
return expandedUrl;
|
||||
} catch (error) {
|
||||
console.error(`Failed to expand URL: ${shortUrl}`, error);
|
||||
return shortUrl;
|
||||
}
|
||||
}
|
||||
|
|
@ -1,41 +0,0 @@
|
|||
import { Result, Ok, Err } from "../errors/results";
|
||||
import { BaseError } from "../errors/baseError";
|
||||
|
||||
export type contentType = "page" | "tweet" | "note" | "document" | "notion";
|
||||
|
||||
class GetTypeError extends BaseError {
|
||||
constructor(message?: string, source?: string) {
|
||||
super("[Decide Type Error]", message, source);
|
||||
}
|
||||
}
|
||||
export const typeDecider = (
|
||||
content: string
|
||||
): Result<contentType, GetTypeError> => {
|
||||
try {
|
||||
// if the content is a URL, then it's a page. if its a URL with https://x.com/user/status/123, then it's a tweet.
|
||||
// if it ends with .pdf etc then it's a document. else, it's a note.
|
||||
// do strict checking with regex
|
||||
if (
|
||||
content.match(/https?:\/\/(x\.com|twitter\.com)\/[\w]+\/[\w]+\/[\d]+/)
|
||||
) {
|
||||
return Ok("tweet");
|
||||
} else if (content.match(/\.(pdf|doc|docx|txt|rtf|odt|md)/i)) {
|
||||
return Ok("document");
|
||||
} else if (
|
||||
content.match(/https?:\/\/(www\.)?notion\.so\/.*/)
|
||||
) {
|
||||
return Ok("notion");
|
||||
} else if (
|
||||
content.match(
|
||||
/^(https?:\/\/)?(www\.)?[a-z0-9]+([-.]{1}[a-z0-9]+)*\.[a-z]{2,5}(\/.*)?$/i
|
||||
)
|
||||
) {
|
||||
return Ok("page");
|
||||
} else {
|
||||
return Ok("note");
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("[Decide Type Error]", e);
|
||||
return Err(new GetTypeError((e as Error).message, "typeDecider"));
|
||||
}
|
||||
};
|
||||
|
|
@ -1,217 +0,0 @@
|
|||
import {
|
||||
WorkflowEntrypoint,
|
||||
WorkflowStep,
|
||||
WorkflowEvent,
|
||||
} from "cloudflare:workers";
|
||||
import { Env, WorkflowParams } from "../types";
|
||||
import { fetchContent } from "../utils/fetchers";
|
||||
import chunkText from "../utils/chunkers";
|
||||
import { database, eq, inArray } from "@supermemory/db";
|
||||
import {
|
||||
ChunkInsert,
|
||||
contentToSpace,
|
||||
documents,
|
||||
spaces,
|
||||
} from "@supermemory/db/schema";
|
||||
import { embedMany } from "ai";
|
||||
import { openai } from "../providers";
|
||||
import { chunk } from "@supermemory/db/schema";
|
||||
import { NonRetryableError } from "cloudflare:workflows";
|
||||
|
||||
// TODO: handle errors properly here.
|
||||
|
||||
export class ContentWorkflow extends WorkflowEntrypoint<Env, WorkflowParams> {
|
||||
async run(event: WorkflowEvent<WorkflowParams>, step: WorkflowStep) {
|
||||
// Step 0: Check if user has reached memory limit
|
||||
await step.do("check memory limit", async () => {
|
||||
const existingMemories = await database(
|
||||
this.env.HYPERDRIVE.connectionString
|
||||
)
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(eq(documents.userId, event.payload.userId));
|
||||
|
||||
if (existingMemories.length >= 2000) {
|
||||
await database(this.env.HYPERDRIVE.connectionString)
|
||||
.delete(documents)
|
||||
.where(eq(documents.uuid, event.payload.uuid));
|
||||
throw new NonRetryableError(
|
||||
"You have reached the maximum limit of 2000 memories"
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// Step 1: Get and format the content.
|
||||
const rawContent =
|
||||
event.payload.prefetched ??
|
||||
(await step.do(
|
||||
"fetch content",
|
||||
async () => await fetchContent(event.payload, this.env, step)
|
||||
));
|
||||
|
||||
// check that the rawcontent is not too big
|
||||
if (rawContent.contentToVectorize.length > 100000) {
|
||||
await database(this.env.HYPERDRIVE.connectionString)
|
||||
.delete(documents)
|
||||
.where(eq(documents.uuid, event.payload.uuid));
|
||||
throw new NonRetryableError("The content is too big (maximum 20 pages)");
|
||||
}
|
||||
|
||||
const chunked = await step.do("chunk content", async () =>
|
||||
chunkText(rawContent.contentToVectorize, 768)
|
||||
);
|
||||
|
||||
// Step 2: Create the document in the database.
|
||||
const document = await step.do("create document", async () => {
|
||||
try {
|
||||
// First check if document exists
|
||||
const existingDoc = await database(this.env.HYPERDRIVE.connectionString)
|
||||
.select()
|
||||
.from(documents)
|
||||
.where(eq(documents.uuid, event.payload.uuid))
|
||||
.limit(1);
|
||||
|
||||
return await database(this.env.HYPERDRIVE.connectionString)
|
||||
.insert(documents)
|
||||
.values({
|
||||
userId: event.payload.userId,
|
||||
type: event.payload.type,
|
||||
uuid: event.payload.uuid,
|
||||
...(event.payload.url && { url: event.payload.url }),
|
||||
title: rawContent.title,
|
||||
content: rawContent.contentToSave,
|
||||
description:
|
||||
"description" in rawContent
|
||||
? (rawContent.description ?? "")
|
||||
: (event.payload.prefetched?.description ?? undefined),
|
||||
ogImage:
|
||||
"image" in rawContent
|
||||
? (rawContent.image ?? "")
|
||||
: (event.payload.prefetched?.ogImage ?? undefined),
|
||||
raw: rawContent.contentToVectorize,
|
||||
isSuccessfullyProcessed: false,
|
||||
updatedAt: new Date(),
|
||||
...(event.payload.createdAt && {
|
||||
createdAt: new Date(event.payload.createdAt),
|
||||
}),
|
||||
})
|
||||
.onConflictDoUpdate({
|
||||
target: documents.uuid,
|
||||
set: {
|
||||
title: rawContent.title,
|
||||
content: rawContent.contentToSave,
|
||||
description:
|
||||
"description" in rawContent
|
||||
? (rawContent.description ?? "")
|
||||
: (event.payload.prefetched?.description ?? undefined),
|
||||
ogImage:
|
||||
"image" in rawContent
|
||||
? (rawContent.image ?? "")
|
||||
: (event.payload.prefetched?.ogImage ?? undefined),
|
||||
raw: rawContent.contentToVectorize,
|
||||
isSuccessfullyProcessed: false,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
})
|
||||
.returning();
|
||||
} catch (error) {
|
||||
console.log("here's the error", error);
|
||||
// Check if error is a unique constraint violation
|
||||
if (
|
||||
error instanceof Error &&
|
||||
error.message.includes("document_url_user_id_idx")
|
||||
) {
|
||||
// Document already exists for this user, stop workflow
|
||||
await database(this.env.HYPERDRIVE.connectionString)
|
||||
.delete(documents)
|
||||
.where(eq(documents.uuid, event.payload.uuid));
|
||||
throw new NonRetryableError("Document already exists for this user");
|
||||
}
|
||||
if (
|
||||
error instanceof Error &&
|
||||
error.message.includes("document_raw_user_idx")
|
||||
) {
|
||||
await database(this.env.HYPERDRIVE.connectionString)
|
||||
.delete(documents)
|
||||
.where(eq(documents.uuid, event.payload.uuid));
|
||||
throw new NonRetryableError("The exact same document already exists");
|
||||
}
|
||||
throw error; // Re-throw other errors
|
||||
}
|
||||
});
|
||||
|
||||
if (!document || document.length === 0) {
|
||||
throw new Error(
|
||||
"Failed to create/update document - no document returned"
|
||||
);
|
||||
}
|
||||
|
||||
// Step 3: Generate embeddings
|
||||
const { data: embeddings } = await this.env.AI.run(
|
||||
"@cf/baai/bge-base-en-v1.5",
|
||||
{
|
||||
text: chunked,
|
||||
}
|
||||
);
|
||||
|
||||
// Step 4: Prepare chunk data
|
||||
const chunkInsertData: ChunkInsert[] = await step.do(
|
||||
"prepare chunk data",
|
||||
async () =>
|
||||
chunked.map((chunk, index) => ({
|
||||
documentId: document[0].id,
|
||||
textContent: chunk,
|
||||
orderInDocument: index,
|
||||
embeddings: embeddings[index],
|
||||
}))
|
||||
);
|
||||
|
||||
// Step 5: Insert chunks
|
||||
if (chunkInsertData.length > 0) {
|
||||
await step.do("insert chunks", async () =>
|
||||
database(this.env.HYPERDRIVE.connectionString).transaction(
|
||||
async (trx) => {
|
||||
await trx.insert(chunk).values(chunkInsertData);
|
||||
}
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// step 6: add content to spaces
|
||||
if (event.payload.spaces) {
|
||||
await step.do("add content to spaces", async () => {
|
||||
await database(this.env.HYPERDRIVE.connectionString).transaction(
|
||||
async (trx) => {
|
||||
// First get the space IDs from the UUIDs
|
||||
const spaceIds = await trx
|
||||
.select({ id: spaces.id })
|
||||
.from(spaces)
|
||||
.where(inArray(spaces.uuid, event.payload.spaces ?? []));
|
||||
|
||||
if (spaceIds.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Then insert the content-space mappings using the actual space IDs
|
||||
await trx.insert(contentToSpace).values(
|
||||
spaceIds.map((space) => ({
|
||||
contentId: document[0].id,
|
||||
spaceId: space.id,
|
||||
}))
|
||||
);
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
// Step 7: Mark the document as successfully processed
|
||||
await step.do("mark document as successfully processed", async () => {
|
||||
await database(this.env.HYPERDRIVE.connectionString)
|
||||
.update(documents)
|
||||
.set({
|
||||
isSuccessfullyProcessed: true,
|
||||
})
|
||||
.where(eq(documents.id, document[0].id));
|
||||
});
|
||||
}
|
||||
}
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
/** @type {import('tailwindcss').Config} */
|
||||
module.exports = {
|
||||
content: ["./src/**/*.{js,ts,jsx,tsx}"],
|
||||
theme: {
|
||||
extend: {},
|
||||
},
|
||||
plugins: [],
|
||||
}
|
||||
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
{
|
||||
"compilerOptions": {
|
||||
"target": "ESNext",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"strict": true,
|
||||
"skipLibCheck": true,
|
||||
"lib": ["ESNext"],
|
||||
"types": [
|
||||
"@cloudflare/workers-types/experimental",
|
||||
"@cloudflare/workers-types"
|
||||
],
|
||||
"jsx": "react-jsx",
|
||||
"jsxImportSource": "hono/jsx"
|
||||
}
|
||||
}
|
||||
3
apps/backend/types.d.ts
vendored
3
apps/backend/types.d.ts
vendored
|
|
@ -1,3 +0,0 @@
|
|||
declare module "@mixmark-io/domino" {
|
||||
export function createDocument(html: string): Document;
|
||||
}
|
||||
|
|
@ -1,55 +0,0 @@
|
|||
name = "supermemory-backend"
|
||||
main = "src/index.tsx"
|
||||
compatibility_date = "2024-10-11"
|
||||
compatibility_flags = [ "nodejs_compat" ]
|
||||
|
||||
[assets]
|
||||
directory = "./public/"
|
||||
binding = "ASSETS"
|
||||
|
||||
|
||||
[observability]
|
||||
enabled = true
|
||||
|
||||
[placement]
|
||||
mode = "smart"
|
||||
|
||||
[ai]
|
||||
binding = "AI"
|
||||
|
||||
[[workflows]]
|
||||
name = "content-workflow-supermemory"
|
||||
binding = "CONTENT_WORKFLOW"
|
||||
class_name = "ContentWorkflow"
|
||||
|
||||
|
||||
[[kv_namespaces]]
|
||||
binding= "MD_CACHE"
|
||||
id = "3186489f943d409a9b772d876a58a73e"
|
||||
preview_id = "3186489f943d409a9b772d876a58a73e"
|
||||
|
||||
[[kv_namespaces]]
|
||||
binding = "ENCRYPTED_TOKENS"
|
||||
id = "a1f048ee14644468ad63b817b5648a31"
|
||||
preview_id = "a1f048ee14644468ad63b817b5648a31"
|
||||
|
||||
[[hyperdrive]]
|
||||
binding = "HYPERDRIVE"
|
||||
id = "3a377d1b9c084e698ee201f10dfa8131"
|
||||
localConnectionString = "postgres://postgres:postgres@localhost:5432/supermemorylocal?sslmode=require"
|
||||
|
||||
[[unsafe.bindings]]
|
||||
name = "EMAIL_LIMITER"
|
||||
type = "ratelimit"
|
||||
namespace_id = "2114284"
|
||||
simple = { limit = 1, period = 60 }
|
||||
|
||||
tail_consumers = [{service = "supermemory-backend-tail"}]
|
||||
|
||||
[[durable_objects.bindings]]
|
||||
name = "RATE_LIMITER"
|
||||
class_name = "DurableObjectRateLimiter"
|
||||
|
||||
[[migrations]]
|
||||
tag = "v1"
|
||||
new_classes = ["DurableObjectRateLimiter"]
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
# Mintlify Starter Kit
|
||||
|
||||
Click on `Use this template` to copy the Mintlify starter kit. The starter kit contains examples including
|
||||
|
||||
- Guide pages
|
||||
- Navigation
|
||||
- Customizations
|
||||
- API Reference pages
|
||||
- Use of popular components
|
||||
|
||||
### Development
|
||||
|
||||
Install the [Mintlify CLI](https://www.npmjs.com/package/mintlify) to preview the documentation changes locally. To install, use the following command
|
||||
|
||||
```
|
||||
npm i -g mintlify
|
||||
```
|
||||
|
||||
Run the following command at the root of your documentation (where mint.json is)
|
||||
|
||||
```
|
||||
mintlify dev
|
||||
```
|
||||
|
||||
### Publishing Changes
|
||||
|
||||
Install our Github App to auto propagate changes from your repo to your deployment. Changes will be deployed to production automatically after pushing to the default branch. Find the link to install on your dashboard.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
- Mintlify dev isn't running - Run `mintlify install` it'll re-install dependencies.
|
||||
- Page loads as a 404 - Make sure you are running in a folder with `mint.json`
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
---
|
||||
openapi: get /connect/{app}
|
||||
---
|
||||
|
||||
You may connect supermemory to other apps.
|
||||
when you send a GET request to the \/connect:APP?id= endpoint, you will get a redirectURL. This is a safe URL that your users can click and select the appropriate files with. Once this is done, supermemory will periodically re-fetch and make sure that the data is always fresh.
|
||||
|
||||
As of right now, these apps are supported:
|
||||
- Notion
|
||||
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
---
|
||||
openapi: get /connections/{connectionId}
|
||||
---
|
||||
|
||||
Get the connection details using this endpoint.
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
---
|
||||
openapi: delete /delete/{id}
|
||||
---
|
||||
|
|
@ -1,59 +0,0 @@
|
|||
---
|
||||
openapi: post /add
|
||||
---
|
||||
|
||||
Add a new memory with content and metadata.
|
||||
|
||||
Fields:
|
||||
|
||||
`content`: string
|
||||
|
||||
`id`: string
|
||||
|
||||
`metadata`: Record
|
||||
|
||||
The `content` can be of the following types:
|
||||
|
||||
- note \/ Markdown
|
||||
|
||||
- If it is a markdown, all the images inside `![]` image tags will automatically be parsed.
|
||||
|
||||
- pdf
|
||||
|
||||
- tweet
|
||||
|
||||
- google_doc
|
||||
|
||||
- notion_doc
|
||||
|
||||
- webpage URL
|
||||
|
||||
- Images and other content is also intelligently parsed in case of a webpage.
|
||||
|
||||
|
||||
The metadata provided is a JSON object.
|
||||
|
||||
for eg.
|
||||
|
||||
``` json
|
||||
{
|
||||
"classId": "21412",
|
||||
"year": "fifth"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
If you wish to do exact searches, please use strings. But if you want to search in a range (time, numbers, prices), you can use numbers too.
|
||||
|
||||
``` json
|
||||
{
|
||||
"price": 1250
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
More about \[metadata filtering here\]([https://docs.supermemory.ai/essentials/metadata-filtering](https://docs.supermemory.ai/essentials/metadata-filtering))
|
||||
|
||||
The `id` is optional. If provided, supermemory will store the same ID as your internal database. This can help for retrieval purposes.
|
||||
|
||||
If the `id` already exists, supermemory will update it instead.
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
---
|
||||
openapi: put /update/{id}
|
||||
---
|
||||
|
||||
Update an existing memory.
|
||||
Please note that all existing metadata will be replaced with the new ones.
|
||||
You can also use the \/add endpoint along with the ID specified.
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
---
|
||||
openapi: get /fastsearch
|
||||
---
|
||||
|
||||
Fast, lossy search using quantized embeddings. This can be used in case your app has text completions, or when searching fast is absolutely necessary.
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
---
|
||||
openapi: post /search
|
||||
---
|
||||
|
||||
Search through documents with metadata filtering.
|
||||
|
||||
Body:
|
||||
`q`: Your search query
|
||||
|
||||
`limit`: Number of documents you want to get
|
||||
|
||||
`filters`: Filters can be applied as `AND, OR, negate, numeric` types. You can read more about it here - \[metadata filtering here\]([https://docs.supermemory.ai/essentials/metadata-filtering](https://docs.supermemory.ai/essentials/metadata-filtering))
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
---
|
||||
openapi: put /settings
|
||||
---
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
---
|
||||
title: "Product Updates"
|
||||
description: "New updates and improvements"
|
||||
mode: "center"
|
||||
---
|
||||
|
||||
<Update label="2025-02-01" description="v0.1.1">
|
||||
- You can now search for memories in multiple spaces at once.
|
||||
- All endpoints have been updated to `/v1` for better versioning
|
||||
- Improved documentation and examples
|
||||
- Interactive [API Playground](https://docs.supermemory.ai/api-reference)
|
||||
</Update>
|
||||
|
|
@ -1,78 +0,0 @@
|
|||
---
|
||||
title: "Managing Multi-User Search Results"
|
||||
description: "Learn how to handle search results for different users in Supermemory"
|
||||
icon: "users"
|
||||
---
|
||||
|
||||
When building multi-user applications with Supermemory, you'll often need to manage data for different users accessing the same account.
|
||||
|
||||
You might also want filters, like memories from **_multiple users_**, or in a certain **_time range_**, or products within a certain price category.
|
||||
|
||||
You can do all this filtering using Supermemory's api.
|
||||
|
||||
Here's a quick example
|
||||
|
||||
```json
|
||||
{
|
||||
"AND": [
|
||||
{
|
||||
"filterType": "numeric",
|
||||
"key": "timestamp",
|
||||
"value": "1742745777",
|
||||
"negate": false,
|
||||
"numericOperator": ">"
|
||||
},
|
||||
{
|
||||
"key": "group",
|
||||
"value": "jira_users",
|
||||
"negate": false
|
||||
},
|
||||
{
|
||||
"OR": [
|
||||
{
|
||||
"key": "team_name",
|
||||
"value": "engineering",
|
||||
"negate": false
|
||||
},
|
||||
{
|
||||
"key": "org_name",
|
||||
"value": "supermemory",
|
||||
"negate": false
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
You can compose these conditions together to add filtering:
|
||||
|
||||
- `AND`
|
||||
- `OR`
|
||||
- `numeric` (greater than / less than)
|
||||
|
||||
Here's an example call:
|
||||
|
||||
```bash
|
||||
curl --location 'https://v2.api.supermemory.ai/search' \
|
||||
--header 'x-api-key: supermemory_RXPx' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"q": "How to use teamcity to set up a project?",
|
||||
"limit": 10,
|
||||
"filters": {
|
||||
"AND": [
|
||||
{
|
||||
"key": "book",
|
||||
"value": "maths",
|
||||
"negate": false
|
||||
},
|
||||
{
|
||||
"key": "author",
|
||||
"value": "r.d. sharma",
|
||||
"negate": false
|
||||
}
|
||||
]
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
|
@ -1,47 +0,0 @@
|
|||
---
|
||||
title: "Pricing"
|
||||
description: "Our pricing plans"
|
||||
icon: "dollar-sign"
|
||||
---
|
||||
|
||||
### Free!
|
||||
|
||||
Yes, everything is free & open source.
|
||||
Supermemory is built by [me](https://dhravya.dev), a college student. My life situations make it very difficult and almost impossible to monetise the product.
|
||||
|
||||
Any kind of sponsorships / support would mean a lot to me, and help me keep supermemory alive.
|
||||
|
||||
You can sponsor on my Github sponsors page - https://github.com/sponsors/dhravya
|
||||
|
||||
### How can I trust you?
|
||||
|
||||
Making the product free makes it hard to trust. "How will you manage the infra?", "How will you keep the product running?" are very valid questions.
|
||||
|
||||
I've got you covered.
|
||||
|
||||
Supermemory has the support of [Cloudflare startups program](https://www.cloudflare.com/forstartups/), [Google Cloud startup program](https://cloud.google.com/startups) and grants by [Vercel](https://vercel.com) and other amazing companies.
|
||||
|
||||
We're fully committed to keeping the product running, and happy to sign any agreements you'd like.
|
||||
|
||||
### Ask
|
||||
|
||||
Please email me at dhravya@supermemory.com if you're interested in:
|
||||
|
||||
- Sponsoring the product
|
||||
- Funding the product
|
||||
- Signing a letter of intent for potential funding rounds
|
||||
- Partnering with us
|
||||
|
||||
### Self hosting guidelines
|
||||
|
||||
As of right now, Supermemory is licensed under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https://github.com/supermemoryai/supermemory/blob/main/LICENSE)
|
||||
|
||||
This means,
|
||||
|
||||
- You can use the code for personal projects, given appropriate attribution.
|
||||
- For non-commercial use, the code must be open source.
|
||||
- Please reach out to me if you want to use the code for commercial projects.
|
||||
|
||||
If you're an enterprise, please reach out to me at dhravya@supermemory.com.
|
||||
|
||||
You can still use the API as a hosted service.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 78 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 157 KiB |
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue