Description
Tell us more about this new feature.
As described in Maximum Object Size: writing files larger than 78.1 GiB will fail by default:
In its default configuration, there is no maximum on the size of objects Mountpoint can read. However, Mountpoint uses multipart upload when writing new objects, and multipart upload allows a maximum of 10,000 parts for an object. This means Mountpoint can only upload objects up to 80,000 MiB (78.1 GiB) in size. If your application tries to write objects larger than this limit, writes will fail with an out of space error.
To increase the maximum object size for writes, use the --write-part-size command-line argument to specify a maximum number of bytes per part, which defaults to 8 MiB. The maximum object size will be 10,000 multiplied by the value you provide for this argument. Even with multipart upload, S3 allows a maximum object size of 5 TiB, and so setting this argument higher than 524.3 MiB will not further increase the object size limit.
Setting this to a size that can accomodate 2TB files has the downside of disabling multiple part uploads for files less than 2GB. For cases where files in the 2TB range are outliers, it would be better if the default 8MiB write-part-size could still be used and have the mountpoint adjust the write-part-size to accommodate splitting large file outliers into 10,000 parts (assuming the file size can be detected)