From Amazon S3, R2, or S3-compatible storage
This guide moves an existing object bucket to Comwit Storage. It applies to Amazon S3, Cloudflare R2, MinIO, Vultr Object Storage, and other providers with an S3-compatible download path.
The migration copies objects. Provider policies, IAM users, CORS rules, lifecycle rules, event notifications, incomplete multipart uploads, and custom domains do not transfer automatically.
1. Inventory the source
Section titled “1. Inventory the source”Before copying, record:
- object count and total bytes;
- key prefixes and case sensitivity assumptions;
- metadata and
Content-Typevalues your application depends on; - versioning, retention, lifecycle, CORS, and public-access behavior;
- the current public URL and every place it is stored or generated;
- the writes that can occur during migration.
Do not use ETag as a universal checksum: multipart ETags are not necessarily an MD5 digest. Preserve a provider checksum or application digest when available.
2. Create the destination
Section titled “2. Create the destination”comwit storage create \ --project <project-id> \ --name <globally-unique-destination-bucket>Use a cwt_ token with storage:read and storage:write for the migration.
Set the destination S3 values as described in Use the S3 API.
3. Make the initial copy
Section titled “3. Make the initial copy”For a bounded migration, the most portable approach is a two-stage AWS CLI sync through an encrypted temporary directory. Keep source and destination credentials in separate shells or profiles so they cannot be confused.
Amazon S3 source:
aws s3 sync s3://<source-bucket> ./storage-migration \ --profile <source-aws-profile>Cloudflare R2 or another S3-compatible source:
AWS_ACCESS_KEY_ID="$SOURCE_ACCESS_KEY" \AWS_SECRET_ACCESS_KEY="$SOURCE_SECRET_KEY" \aws s3 sync s3://<source-bucket> ./storage-migration \ --endpoint-url "$SOURCE_S3_ENDPOINT"Upload to Comwit Storage:
AWS_ACCESS_KEY_ID=comwit \AWS_SECRET_ACCESS_KEY=comwit \AWS_SESSION_TOKEN="$COMWIT_STORAGE_TOKEN" \AWS_DEFAULT_REGION=auto \aws s3 sync ./storage-migration s3://<destination-bucket> \ --endpoint-url https://storage.cloud.comwit.ioThe temporary directory must have enough capacity for the bucket and should be encrypted and deleted after verification. For very large buckets, use a bounded copy worker with independent source and destination S3 clients instead of local staging; keep the same inventory, checksum, retry, and rollback gates.
4. Verify before cutover
Section titled “4. Verify before cutover”Compare source and destination object counts and total bytes by prefix. Sample small and multipart objects, binary files, Unicode keys, metadata, range reads, and content types. Run the application against the destination using a staging token before changing production traffic.
5. Freeze or dual-write, then copy the delta
Section titled “5. Freeze or dual-write, then copy the delta”Choose one:
- briefly stop source writes, repeat the copy, then switch;
- dual-write to both buckets until a final comparison passes.
Do not use sync --delete until you have reviewed exactly which destination
keys it would remove. Comwit Storage never requires destructive mirroring for
cutover.
6. Switch the application
Section titled “6. Switch the application”Change only these connection values:
endpoint -> https://storage.cloud.comwit.ioregion -> autobucket -> destination bucketcredential -> cwp_ project token in the S3 session-token fieldpath style -> enabledIf the source was public, enable public access, wait for public_base_url, and
replace stored/generated asset bases. Do not assume the old provider hostname
continues to work.
7. Observe and retain rollback
Section titled “7. Observe and retain rollback”Watch object errors, missing keys, latency, gateway 4xx/5xx, and public TLS/cache behavior. Keep the source bucket read-only for an agreed rollback window. Delete source data and temporary migration files only after the application and object inventory remain clean through that window.