Skip to content

From Amazon S3, R2, or S3-compatible storage

This guide moves an existing object bucket to Comwit Storage. It applies to Amazon S3, Cloudflare R2, MinIO, Vultr Object Storage, and other providers with an S3-compatible download path.

The migration copies objects. Provider policies, IAM users, CORS rules, lifecycle rules, event notifications, incomplete multipart uploads, and custom domains do not transfer automatically.

Before copying, record:

  • object count and total bytes;
  • key prefixes and case sensitivity assumptions;
  • metadata and Content-Type values your application depends on;
  • versioning, retention, lifecycle, CORS, and public-access behavior;
  • the current public URL and every place it is stored or generated;
  • the writes that can occur during migration.

Do not use ETag as a universal checksum: multipart ETags are not necessarily an MD5 digest. Preserve a provider checksum or application digest when available.

Terminal window
comwit storage create \
--project <project-id> \
--name <globally-unique-destination-bucket>

Use a cwt_ token with storage:read and storage:write for the migration. Set the destination S3 values as described in Use the S3 API.

For a bounded migration, the most portable approach is a two-stage AWS CLI sync through an encrypted temporary directory. Keep source and destination credentials in separate shells or profiles so they cannot be confused.

Amazon S3 source:

Terminal window
aws s3 sync s3://<source-bucket> ./storage-migration \
--profile <source-aws-profile>

Cloudflare R2 or another S3-compatible source:

Terminal window
AWS_ACCESS_KEY_ID="$SOURCE_ACCESS_KEY" \
AWS_SECRET_ACCESS_KEY="$SOURCE_SECRET_KEY" \
aws s3 sync s3://<source-bucket> ./storage-migration \
--endpoint-url "$SOURCE_S3_ENDPOINT"

Upload to Comwit Storage:

Terminal window
AWS_ACCESS_KEY_ID=comwit \
AWS_SECRET_ACCESS_KEY=comwit \
AWS_SESSION_TOKEN="$COMWIT_STORAGE_TOKEN" \
AWS_DEFAULT_REGION=auto \
aws s3 sync ./storage-migration s3://<destination-bucket> \
--endpoint-url https://storage.cloud.comwit.io

The temporary directory must have enough capacity for the bucket and should be encrypted and deleted after verification. For very large buckets, use a bounded copy worker with independent source and destination S3 clients instead of local staging; keep the same inventory, checksum, retry, and rollback gates.

Compare source and destination object counts and total bytes by prefix. Sample small and multipart objects, binary files, Unicode keys, metadata, range reads, and content types. Run the application against the destination using a staging token before changing production traffic.

5. Freeze or dual-write, then copy the delta

Section titled “5. Freeze or dual-write, then copy the delta”

Choose one:

  • briefly stop source writes, repeat the copy, then switch;
  • dual-write to both buckets until a final comparison passes.

Do not use sync --delete until you have reviewed exactly which destination keys it would remove. Comwit Storage never requires destructive mirroring for cutover.

Change only these connection values:

endpoint -> https://storage.cloud.comwit.io
region -> auto
bucket -> destination bucket
credential -> cwp_ project token in the S3 session-token field
path style -> enabled

If the source was public, enable public access, wait for public_base_url, and replace stored/generated asset bases. Do not assume the old provider hostname continues to work.

Watch object errors, missing keys, latency, gateway 4xx/5xx, and public TLS/cache behavior. Keep the source bucket read-only for an agreed rollback window. Delete source data and temporary migration files only after the application and object inventory remain clean through that window.