Multipart Upload Fails at final step. #431
Description
Multipart upload fails, see error below, when setting multi_part to false, it works, but gives a warning due to large file size...
I'm using MinIO behind Cloudflare. Upload via mc copy command works fine, so its prob. a problem with this plugin. Thx
## load package
library("aws.s3")
## code goes here
put_object(
file = local_file_name,
object = file_name,
bucket = data_bucket,
multipart = TRUE,
show_progress = TRUE
)
## session info for your system
> sessionInfo()
R version 4.2.3 (2023-03-15)
Platform: x86_64-apple-darwin22.3.0 (64-bit)
Running under: macOS Ventura 13.3.1
Matrix products: default
BLAS: /usr/local/Cellar/openblas/0.3.23/lib/libopenblasp-r0.3.23.dylib
LAPACK: /usr/local/Cellar/r/4.2.3/lib/R/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] imputeTS_3.3 rvest_1.0.3 rtweet_1.1.0 ggpubr_0.6.0
[5] feasts_0.3.1 fable_0.3.3 fabletools_0.3.3 clock_0.6.1
[9] scales_1.2.1 quantmod_0.4.22 TTR_0.24.3 xts_0.12.2
[13] zoo_1.8-11 aws.s3_0.3.21 tsibble_1.1.3 httr_1.4.5
[17] jsonlite_1.8.4 readxl_1.4.2 lubridate_1.9.2 forcats_1.0.0
[21] stringr_1.5.0 dplyr_1.1.1 purrr_1.0.1 readr_2.1.4
[25] tidyr_1.3.0 tibble_3.2.1 ggplot2_3.4.2 tidyverse_2.0.0
loaded via a namespace (and not attached):
[1] Rcpp_1.0.10 lattice_0.20-45
[3] digest_0.6.31 lmtest_0.9-40
[5] utf8_1.2.3 mime_0.12
[7] aws.signature_0.6.0 R6_2.5.1
[9] cellranger_1.1.0 backports_1.4.1
[11] pillar_1.8.1 rlang_1.1.0
[13] curl_4.3.3 car_3.1-1
[15] fracdiff_1.5-2 gridtext_0.1.5
[17] munsell_0.5.0 broom_1.0.4
[19] anytime_0.3.9 compiler_4.2.3
[21] pkgconfig_2.0.3 base64enc_0.1-3
[23] forecast_8.20 urca_1.3-3
[25] ggtext_0.1.2 nnet_7.3-18
[27] tidyselect_1.2.0 quadprog_1.5-8
[29] fansi_1.0.4 tzdb_0.3.0
[31] withr_2.5.0 grid_4.2.3
[33] distributional_0.3.1.9000 nlme_3.1-162
[35] gtable_0.3.1 lifecycle_1.0.3
[37] magrittr_2.0.3 cli_3.6.0
[39] stringi_1.7.12 carData_3.0-5
[41] farver_2.1.1 ggsignif_0.6.4
[43] tseries_0.10-52 timeDate_4022.108
[45] xml2_1.3.3 ellipsis_0.3.2
[47] generics_0.1.3 vctrs_0.6.1
[49] tools_4.2.3 glue_1.6.2
[51] hms_1.1.2 abind_1.4-5
[53] parallel_4.2.3 timechange_0.2.0
[55] colorspace_2.1-0 stinepack_1.4
[57] rstatix_0.7.2 Log:
> put_object(
+ file = local_file_name,
+ object = file_name,
+ bucket = data_bucket,
+ multipart = TRUE,
+ show_progress = TRUE
+ )
Uploading part 1 of 2-part upload
|======================================================================| 100%
Uploading part 2 of 2-part upload
|======================================================================| 100%
Completing multi-part upload
List of 10
$ url : chr "https://s3.mortality.watch/data/mortality/world_weekly_104w_sma.csv"
$ status_code: int 400
$ headers :List of 11
..$ date : chr "Tue, 18 Apr 2023 03:43:32 GMT"
..$ content-type : chr "text/plain"
..$ content-length : chr "341"
..$ accept-ranges : chr "bytes"
..$ vary : chr "Origin"
..$ cf-cache-status: chr "DYNAMIC"
..$ report-to : chr "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=hlWyUJpFuQAk0kDwNS%2BurQ0Qm3Iwe1Msj"| __truncated__
..$ nel : chr "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"
..$ server : chr "cloudflare"
..$ cf-ray : chr "7b99e7763e0d102d-LAX"
..$ alt-svc : chr "h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"
..- attr(*, "class")= chr [1:2] "insensitive" "list"
$ all_headers:List of 1
..$ :List of 3
.. ..$ status : int 400
.. ..$ version: chr "HTTP/2"
.. ..$ headers:List of 11
.. .. ..$ date : chr "Tue, 18 Apr 2023 03:43:32 GMT"
.. .. ..$ content-type : chr "text/plain"
.. .. ..$ content-length : chr "341"
.. .. ..$ accept-ranges : chr "bytes"
.. .. ..$ vary : chr "Origin"
.. .. ..$ cf-cache-status: chr "DYNAMIC"
.. .. ..$ report-to : chr "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=hlWyUJpFuQAk0kDwNS%2BurQ0Qm3Iwe1Msj"| __truncated__
.. .. ..$ nel : chr "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"
.. .. ..$ server : chr "cloudflare"
.. .. ..$ cf-ray : chr "7b99e7763e0d102d-LAX"
.. .. ..$ alt-svc : chr "h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"
.. .. ..- attr(*, "class")= chr [1:2] "insensitive" "list"
$ cookies :'data.frame': 0 obs. of 7 variables:
..$ domain : logi(0)
..$ flag : logi(0)
..$ path : logi(0)
..$ secure : logi(0)
..$ expiration: 'POSIXct' num(0)
..$ name : logi(0)
..$ value : logi(0)
$ content : raw [1:341] 3c 3f 78 6d ...
$ date : POSIXct[1:1], format: "2023-04-18 03:43:32"
$ times : Named num [1:6] 0 0.000056 0 0.000218 0.000222 ...
..- attr(*, "names")= chr [1:6] "redirect" "namelookup" "connect" "pretransfer" ...
$ request :List of 7
..$ method : chr "POST"
..$ url : chr "https://s3.mortality.watch/data/mortality/world_weekly_104w_sma.csv"
..$ headers : Named chr [1:6] "application/json, text/xml, application/xml, */*" "text/plain" "224" "20230418T034332Z" ...
.. ..- attr(*, "names")= chr [1:6] "Accept" "Content-Type" "Content-Length" "x-amz-date" ...
..$ fields : NULL
..$ options :List of 4
.. ..$ useragent : chr "libcurl/7.87.0 r-curl/4.3.3 httr/1.4.5"
.. ..$ post : logi TRUE
.. ..$ readfunction :function (nbytes, ...)
.. ..$ postfieldsize_large: num 224
..$ auth_token: NULL
..$ output : list()
.. ..- attr(*, "class")= chr [1:2] "write_memory" "write_function"
..- attr(*, "class")= chr "request"
$ handle :Class 'curl_handle' <externalptr>
- attr(*, "class")= chr "aws_error"
- attr(*, "headers")=List of 11
..$ date : chr "Tue, 18 Apr 2023 03:43:32 GMT"
..$ content-type : chr "text/plain"
..$ content-length : chr "341"
..$ accept-ranges : chr "bytes"
..$ vary : chr "Origin"
..$ cf-cache-status: chr "DYNAMIC"
..$ report-to : chr "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=hlWyUJpFuQAk0kDwNS%2BurQ0Qm3Iwe1Msj"| __truncated__
..$ nel : chr "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"
..$ server : chr "cloudflare"
..$ cf-ray : chr "7b99e7763e0d102d-LAX"
..$ alt-svc : chr "h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"
..- attr(*, "class")= chr [1:2] "insensitive" "list"
- attr(*, "request_canonical")= chr "POST\n/data/mortality/world_weekly_104w_sma.csv\n\ncontent-length:224\nhost:s3.mortality.watch\nx-amz-date:2023"| __truncated__
- attr(*, "request_string_to_sign")= chr "AWS4-HMAC-SHA256\n20230418T034332Z\n20230418/s3/s3/aws4_request\n73802dd93b74ca21116b9115649e353d454512adddd4f6"| __truncated__
- attr(*, "request_signature")= chr "AWS4-HMAC-SHA256 Credential=minio/20230418/s3/s3/aws4_request,SignedHeaders=content-length;host;x-amz-date,Sign"| __truncated__
NULL
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Bad Request (HTTP 400).