mirror of
https://github.com/supermemoryai/supermemory.git
synced 2026-10-27 06:15:15 +00:00
Fixes connector docs. - **`authorization.url` examples.** They now show the URL the API actually returns (`/v3/connections/auth/redirect?state=…`), not provider URLs. - **Broken examples:** - TS `connectors.create` examples were missing commas. - Python dicts now use the SDK's snake_case keys. - `connector.createdAt` → `connector.system.createdAt`. - The list default `limit` is 10. - **Pending connectors.** They are visible through get (`pending_authorization`), just not in list. - **Gmail:** - It needs the Max plan. - Nothing syncs until labels are picked. - The watch covers the selected labels, not just INBOX. - `from` filters use `contains`. - **Smaller fixes:** - The OAuth redirect-URI troubleshooting. - Notion property metadata keys. - Picker error codes. Checked against the current API and the published SDKs (TS 5.0.1, Python 5.0.0). Legacy v3/v4 docs are unchanged apart from factual fixes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
279 lines
8.6 KiB
Text
279 lines
8.6 KiB
Text
---
|
|
title: "S3 Connector"
|
|
sidebarTitle: "S3"
|
|
description: "Connect Amazon S3 or S3-compatible storage to sync files into your Supermemory knowledge base"
|
|
icon: "/icons/hugeicons/database-01.svg"
|
|
---
|
|
|
|
Connect Amazon S3 buckets or S3-compatible storage services (MinIO, DigitalOcean Spaces, Cloudflare R2, Tigris) to sync files into your Supermemory knowledge base.
|
|
|
|
<Note>
|
|
The S3 connector requires a **Scale Plan** or higher. You can also create S3 connectors directly from the [Supermemory Console](https://console.supermemory.ai).
|
|
</Note>
|
|
|
|
## Quick setup
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
import { Supermemory } from "supermemory"
|
|
|
|
const supermemory = new Supermemory({ apiKey: process.env.SUPERMEMORY_API_KEY })
|
|
|
|
const connector = await supermemory.connectors.create("org-123", {
|
|
provider: "s3",
|
|
config: {
|
|
bucket: "my-documents-bucket",
|
|
region: "us-east-1",
|
|
accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
|
|
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
|
|
},
|
|
})
|
|
|
|
// S3 doesn't require OAuth; authorization is null and the first sync starts now
|
|
console.log("Connector ID:", connector.id)
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
from supermemory import Supermemory
|
|
import os
|
|
|
|
client = Supermemory(api_key=os.environ["SUPERMEMORY_API_KEY"])
|
|
|
|
connector = client.connectors.create(
|
|
"org-123",
|
|
request={
|
|
"provider": "s3",
|
|
"config": {
|
|
"bucket": "my-documents-bucket",
|
|
"region": "us-east-1",
|
|
"access_key_id": os.environ["AWS_ACCESS_KEY_ID"],
|
|
"secret_access_key": os.environ["AWS_SECRET_ACCESS_KEY"],
|
|
},
|
|
},
|
|
)
|
|
|
|
# S3 doesn't require OAuth; authorization is None and the first sync starts now
|
|
print(f"Connector ID: {connector.id}")
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
curl -X POST "https://api.supermemory.ai/ns/org-123/connectors" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"provider": "s3",
|
|
"config": {
|
|
"bucket": "my-documents-bucket",
|
|
"region": "us-east-1",
|
|
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
|
|
"secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
|
|
}
|
|
}'
|
|
|
|
# Response: {"id": "PTzGiUYei7pgzg5buzZHgA", "authorization": null}
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Configuration options
|
|
|
|
For S3, provider-specific fields go inside the `config` object. The namespace is in the URL, and `documentLimit` stays top-level in the body.
|
|
|
|
| Parameter | Location | Required | Description |
|
|
|-----------|----------|----------|-------------|
|
|
| `bucket` | `config.bucket` | Yes | S3 bucket name |
|
|
| `region` | `config.region` | Yes | AWS region (e.g., `us-east-1`). Use `auto` for S3-compatible providers that don't expose AWS-style regions (MinIO, R2, Tigris). |
|
|
| `accessKeyId` | `config.accessKeyId` | Yes | AWS access key ID or S3-compatible service key |
|
|
| `secretAccessKey` | `config.secretAccessKey` | Yes | AWS secret access key |
|
|
| `endpoint` | `config.endpoint` | No | Custom endpoint for S3-compatible services |
|
|
| `prefix` | `config.prefix` | No | Key prefix filter (e.g., `documents/`) |
|
|
| `documentLimit` | top-level | No | Maximum documents to sync (default: 10,000) |
|
|
|
|
<Note>
|
|
Credentials are stored encrypted and are never returned by `connectors.get` or `connectors.list`. The `config` in responses contains only non-secret fields such as `bucket`, `region`, `endpoint` and `prefix`.
|
|
</Note>
|
|
|
|
## S3-compatible services
|
|
|
|
Use `config.endpoint` to connect to S3-compatible storage. These services don't use AWS-style regions, so set `config.region` to `auto`. The value is still required for request signing but the service ignores it.
|
|
|
|
```typescript
|
|
// MinIO
|
|
const connector = await supermemory.connectors.create("minio-sync", {
|
|
provider: "s3",
|
|
config: {
|
|
bucket: "my-bucket",
|
|
region: "auto",
|
|
accessKeyId: "minio-key",
|
|
secretAccessKey: "minio-secret",
|
|
endpoint: "https://minio.example.com",
|
|
},
|
|
})
|
|
```
|
|
|
|
Common S3-compatible endpoint values:
|
|
|
|
| Service | `config.endpoint` | `config.region` |
|
|
|---------|-------------------|-----------------|
|
|
| DigitalOcean Spaces | `https://nyc3.digitaloceanspaces.com` | `nyc3` |
|
|
| Cloudflare R2 | `https://<account-id>.r2.cloudflarestorage.com` | `auto` |
|
|
| Tigris | `https://t3.storage.dev` | `auto` |
|
|
|
|
Cloudflare R2 example:
|
|
|
|
```typescript
|
|
const connector = await supermemory.connectors.create("r2-sync", {
|
|
provider: "s3",
|
|
config: {
|
|
bucket: "my-bucket",
|
|
region: "auto",
|
|
accessKeyId: process.env.R2_ACCESS_KEY_ID!,
|
|
secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!,
|
|
endpoint: "https://<account-id>.r2.cloudflarestorage.com",
|
|
},
|
|
})
|
|
```
|
|
|
|
<Note>
|
|
For S3-compatible services, `config.endpoint` is the base S3 endpoint. Do not include the bucket name in the endpoint URL; pass the bucket separately as `config.bucket`.
|
|
</Note>
|
|
|
|
## Prefix filtering
|
|
|
|
Sync only files within a specific path:
|
|
|
|
```typescript
|
|
const connector = await supermemory.connectors.create("engineering-docs", {
|
|
provider: "s3",
|
|
config: {
|
|
bucket: "company-data",
|
|
region: "us-east-1",
|
|
accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
|
|
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
|
|
prefix: "documents/engineering/", // Only syncs files under this path
|
|
},
|
|
})
|
|
```
|
|
|
|
## Multi-Tenant Buckets
|
|
|
|
A connector syncs into exactly one namespace. For a bucket shared by many tenants, create one connector per tenant namespace and point each at that tenant's path with `config.prefix`:
|
|
|
|
```typescript
|
|
await supermemory.connectors.create("user-123", {
|
|
provider: "s3",
|
|
config: {
|
|
bucket: "user-files",
|
|
region: "us-east-1",
|
|
accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
|
|
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
|
|
prefix: "users/user-123/",
|
|
},
|
|
})
|
|
```
|
|
|
|
## Connector Management
|
|
|
|
### Check Sync Status
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
const connector = await supermemory.connectors.get("org-123", "PTzGiUYei7pgzg5buzZHgA", {
|
|
include: "syncs",
|
|
})
|
|
|
|
console.log("Bucket:", connector.config?.bucket)
|
|
console.log("Last sync:", connector.latestRun?.system.status, connector.latestRun?.error)
|
|
console.log("Files synced:", connector.documentCount)
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
curl "https://api.supermemory.ai/ns/org-123/connectors/PTzGiUYei7pgzg5buzZHgA?include=syncs" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
### Delete Connector
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
await supermemory.connectors.delete("org-123", "PTzGiUYei7pgzg5buzZHgA")
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
curl -X DELETE "https://api.supermemory.ai/ns/org-123/connectors/PTzGiUYei7pgzg5buzZHgA" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Warning>
|
|
By default, deleting a connector removes all synced documents from Supermemory. To keep documents, pass `deleteDocuments: false` (`DELETE /ns/{namespace}/connectors/{id}?deleteDocuments=false`).
|
|
</Warning>
|
|
|
|
### Manual sync
|
|
|
|
The call returns `409` while a sync for that connector is already running.
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
await supermemory.connectors.sync("org-123", "PTzGiUYei7pgzg5buzZHgA")
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
curl -X POST "https://api.supermemory.ai/ns/org-123/connectors/PTzGiUYei7pgzg5buzZHgA/sync" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
|
|
# Response: {"id": "PTzGiUYei7pgzg5buzZHgA", "status": "queued"}
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Sync behavior
|
|
|
|
| Feature | Behavior |
|
|
|---------|----------|
|
|
| **Initial sync** | Fetches all files matching prefix filter |
|
|
| **Incremental sync** | Only files modified since last sync |
|
|
| **Sync schedule** | Every 4 hours + manual triggers |
|
|
| **Document limit** | 10,000 files per connector (default) |
|
|
|
|
## IAM permissions
|
|
|
|
Minimum required permissions:
|
|
|
|
```json
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": ["s3:GetObject", "s3:ListBucket"],
|
|
"Resource": [
|
|
"arn:aws:s3:::your-bucket-name",
|
|
"arn:aws:s3:::your-bucket-name/*"
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Error codes
|
|
|
|
Supermemory checks the credentials against the bucket before it creates the connector. If that check fails, `POST /ns/{namespace}/connectors` returns `400` with one of these messages. The code column is the status S3 returned.
|
|
|
|
| S3 status | Message | Solution |
|
|
|------|---------|----------|
|
|
| 401 | Authentication failed | Verify access key and secret |
|
|
| 403 | Access denied | Check IAM permissions and bucket policy |
|
|
| 404 | Bucket not found | Verify bucket name and region |
|