gcs_upload sometimes hangs for a very long time

[wpdavis]

We are using the gcs_upload function to upload CSVs to a GCS bucket. The sizes vary, but they are rarely more than 20MB. They are sometimes overwriting each other.

We have noticed the call hangs for a very long time — say, 5-10 minutes — before completing. Sometimes it just fails entirely. We aren't able to discern any rhyme or reason to it — it does not seem correlated with file size, for example, or whether it is overwriting a pre-existing file. We have tried both upload_type = simple and upload_type = resumable, with no success.

Because it happens infrequently, it's been impossible to create repro steps. But are there things we could be logging that could help?

Below is our current call to gcs_upload.

googleCloudStorageR::gcs_upload(
    data,
    object_function = function(input, output) readr::write_csv(x = input, file = output), 
     type = "text/csv",
    name = path,
    bucket = bucket,
    predefinedAcl = "bucketLevel"
)

[mmoisse]

I experience similar issues. For me it was mainly linked with smaller files and I resolved it by forcing googleCloudStorageR to use the resumable upload for all files options(googleCloudStorageR.upload_limit = 0L)