--- title: "S3 Connector" sidebarTitle: "S3" description: "Connect Amazon S3 or S3-compatible storage to sync files into your Supermemory knowledge base" icon: "/icons/hugeicons/database-01.svg" --- Connect Amazon S3 buckets or S3-compatible storage services (MinIO, DigitalOcean Spaces, Cloudflare R2, Tigris) to sync files into your Supermemory knowledge base. The S3 connector requires a **Scale Plan** or higher. You can also create S3 connectors directly from the [Supermemory Console](https://console.supermemory.ai). ## Quick setup ```typescript import { Supermemory } from "supermemory" const supermemory = new Supermemory({ apiKey: process.env.SUPERMEMORY_API_KEY }) const connector = await supermemory.connectors.create("org-123", { provider: "s3", config: { bucket: "my-documents-bucket", region: "us-east-1", accessKeyId: process.env.AWS_ACCESS_KEY_ID!, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!, }, }) // S3 doesn't require OAuth; authorization is null and the first sync starts now console.log("Connector ID:", connector.id) ``` ```python from supermemory import Supermemory import os client = Supermemory(api_key=os.environ["SUPERMEMORY_API_KEY"]) connector = client.connectors.create( "org-123", request={ "provider": "s3", "config": { "bucket": "my-documents-bucket", "region": "us-east-1", "accessKeyId": os.environ["AWS_ACCESS_KEY_ID"], "secretAccessKey": os.environ["AWS_SECRET_ACCESS_KEY"], }, }, ) # S3 doesn't require OAuth; authorization is None and the first sync starts now print(f"Connector ID: {connector.id}") ``` ```bash curl -X POST "https://api.supermemory.ai/ns/org-123/connectors" \ -H "Authorization: Bearer $SUPERMEMORY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "provider": "s3", "config": { "bucket": "my-documents-bucket", "region": "us-east-1", "accessKeyId": "AKIAIOSFODNN7EXAMPLE", "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" } }' # Response: {"id": "PTzGiUYei7pgzg5buzZHgA", "authorization": null} ``` ## Configuration options For S3, provider-specific fields go inside the `config` object. The namespace is in the URL, and `documentLimit` stays top-level in the body. | Parameter | Location | Required | Description | |-----------|----------|----------|-------------| | `bucket` | `config.bucket` | Yes | S3 bucket name | | `region` | `config.region` | Yes | AWS region (e.g., `us-east-1`). Use `auto` for S3-compatible providers that don't expose AWS-style regions (MinIO, R2, Tigris). | | `accessKeyId` | `config.accessKeyId` | Yes | AWS access key ID or S3-compatible service key | | `secretAccessKey` | `config.secretAccessKey` | Yes | AWS secret access key | | `endpoint` | `config.endpoint` | No | Custom endpoint for S3-compatible services | | `prefix` | `config.prefix` | No | Key prefix filter (e.g., `documents/`) | | `documentLimit` | top-level | No | Maximum documents to sync (default: 10,000) | Credentials are stored encrypted and are never returned by `connectors.get` or `connectors.list`. The `config` in responses contains only non-secret fields such as `bucket`, `region`, `endpoint` and `prefix`. ## S3-compatible services Use `config.endpoint` to connect to S3-compatible storage. These services don't use AWS-style regions, so set `config.region` to `auto`. The value is still required for request signing but the service ignores it. ```typescript // MinIO const connector = await supermemory.connectors.create("minio-sync", { provider: "s3", config: { bucket: "my-bucket", region: "auto", accessKeyId: "minio-key", secretAccessKey: "minio-secret", endpoint: "https://minio.example.com", }, }) ``` Common S3-compatible endpoint values: | Service | `config.endpoint` | `config.region` | |---------|-------------------|-----------------| | DigitalOcean Spaces | `https://nyc3.digitaloceanspaces.com` | `nyc3` | | Cloudflare R2 | `https://.r2.cloudflarestorage.com` | `auto` | | Tigris | `https://t3.storage.dev` | `auto` | Cloudflare R2 example: ```typescript const connector = await supermemory.connectors.create("r2-sync", { provider: "s3", config: { bucket: "my-bucket", region: "auto", accessKeyId: process.env.R2_ACCESS_KEY_ID!, secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!, endpoint: "https://.r2.cloudflarestorage.com", }, }) ``` For S3-compatible services, `config.endpoint` is the base S3 endpoint. Do not include the bucket name in the endpoint URL; pass the bucket separately as `config.bucket`. ## Prefix filtering Sync only files within a specific path: ```typescript const connector = await supermemory.connectors.create("engineering-docs", { provider: "s3", config: { bucket: "company-data", region: "us-east-1", accessKeyId: process.env.AWS_ACCESS_KEY_ID!, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!, prefix: "documents/engineering/", // Only syncs files under this path }, }) ``` ## Multi-Tenant Buckets A connector syncs into exactly one namespace. For a bucket shared by many tenants, create one connector per tenant namespace and point each at that tenant's path with `config.prefix`: ```typescript await supermemory.connectors.create("user-123", { provider: "s3", config: { bucket: "user-files", region: "us-east-1", accessKeyId: process.env.AWS_ACCESS_KEY_ID!, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!, prefix: "users/user-123/", }, }) ``` ## Connector Management ### Check Sync Status ```typescript const connector = await supermemory.connectors.get("org-123", "PTzGiUYei7pgzg5buzZHgA", { include: "syncs", }) console.log("Bucket:", connector.config?.bucket) console.log("Last sync:", connector.latestRun?.system.status, connector.latestRun?.error) console.log("Files synced:", connector.documentCount) ``` ```bash curl "https://api.supermemory.ai/ns/org-123/connectors/PTzGiUYei7pgzg5buzZHgA?include=syncs" \ -H "Authorization: Bearer $SUPERMEMORY_API_KEY" ``` ### Delete Connector ```typescript await supermemory.connectors.delete("org-123", "PTzGiUYei7pgzg5buzZHgA") ``` ```bash curl -X DELETE "https://api.supermemory.ai/ns/org-123/connectors/PTzGiUYei7pgzg5buzZHgA" \ -H "Authorization: Bearer $SUPERMEMORY_API_KEY" ``` By default, deleting a connector removes all synced documents from Supermemory. To keep documents, pass `deleteDocuments: false` (`DELETE /ns/{namespace}/connectors/{id}?deleteDocuments=false`). ### Manual sync The call returns `409` while a sync for that connector is already running. ```typescript await supermemory.connectors.sync("org-123", "PTzGiUYei7pgzg5buzZHgA") ``` ```bash curl -X POST "https://api.supermemory.ai/ns/org-123/connectors/PTzGiUYei7pgzg5buzZHgA/sync" \ -H "Authorization: Bearer $SUPERMEMORY_API_KEY" # Response: {"id": "PTzGiUYei7pgzg5buzZHgA", "status": "queued"} ``` ## Sync behavior | Feature | Behavior | |---------|----------| | **Initial sync** | Fetches all files matching prefix filter | | **Incremental sync** | Only files modified since last sync | | **Sync schedule** | Every 4 hours + manual triggers | | **Document limit** | 10,000 files per connector (default) | ## IAM permissions Minimum required permissions: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:GetObject", "s3:ListBucket"], "Resource": [ "arn:aws:s3:::your-bucket-name", "arn:aws:s3:::your-bucket-name/*" ] } ] } ``` ## Error codes | Code | Message | Solution | |------|---------|----------| | 401 | Authentication failed | Verify access key and secret | | 403 | Access denied | Check IAM permissions and bucket policy | | 404 | Bucket not found | Verify bucket name and region |