Clarify the Reference on How to Configure an S3 Compatible Replica

[ky-bd]

Quote from the reference doc:

The easiest way to configure an S3 replica is to use the url field:

dbs:
  - path: /var/lib/db
    replicas:
      - url: s3://mybkt.litestream.io/db

However, you can break this out into separate fields as well:

dbs:
  - path: /var/lib/db
    replicas:
      - type:   s3
        bucket: mybkt.litestream.io
        path:   db

From my understanding, this indicates that setting the s3://... URL is equivalent to specifying each field or vice versa. However it is not true, since the URL is only parsed for limited providers, and fallback to AWS:

litestream/s3/replica_client.go

Lines 696 to 738 in 749bc0d

	func ParseHost(s string) (bucket, region, endpoint string, forcePathStyle bool) {
	// Extract port if one is specified.
	host, port, err := net.SplitHostPort(s)
	if err != nil {
	host = s
	}
	// Default to path-based URLs, except for with AWS S3 itself.
	forcePathStyle = true
	// Extract fields from provider-specific host formats.
	scheme := "https"
	if a := localhostRegex.FindStringSubmatch(host); a != nil {
	bucket, region = a[1], "us-east-1"
	scheme, endpoint = "http", "localhost"
	} else if a := backblazeRegex.FindStringSubmatch(host); a != nil {
	bucket, region = a[1], a[2]
	endpoint = fmt.Sprintf("s3.%s.backblazeb2.com", region)
	} else if a := filebaseRegex.FindStringSubmatch(host); a != nil {
	bucket, endpoint = a[1], "s3.filebase.com"
	} else if a := digitalOceanRegex.FindStringSubmatch(host); a != nil {
	bucket, region = a[1], a[2]
	endpoint = fmt.Sprintf("%s.digitaloceanspaces.com", region)
	} else if a := linodeRegex.FindStringSubmatch(host); a != nil {
	bucket, region = a[1], a[2]
	endpoint = fmt.Sprintf("%s.linodeobjects.com", region)
	} else {
	bucket = host
	forcePathStyle = false
	}
	// Add port back to endpoint, if available.
	if endpoint != "" && port != "" {
	endpoint = net.JoinHostPort(endpoint, port)
	}
	// Prepend scheme to endpoint.
	if endpoint != "" {
	endpoint = scheme + "://" + endpoint
	}
	return bucket, region, endpoint, forcePathStyle
	}

In case the user misconfigured the URL (e.g. putting a Cloudflare R2 endpoint in it), litestream will not only failed to replicate the database, but also wronly send the access_key_id to the AWS. The reference document should clarify that the URL is only available to some listed providers, and specifying each field has a better compatibility.

Related issues:

• Default S3 minio config shown in the Docs page does not work on remote servers #591
• Limited support for S3-compatible storage #491 (The forcePathStyle is not overwritten actually, but the author did find the compatibility issue of the URL)
• Cloudflare R2 support #372 (Where I found how to configure Cloudflare R2 with litestream properly)