more doc fixes

Author: hongooi73

.Rbuildignore

@@ -12,3 +12,4 @@ drat.sh
 ^__blobstorage__
 ^__queuestorage__
 ^\.github$
+^SECURITY\.md$

DESCRIPTION

@@ -31,4 +31,4 @@ Suggests:
     processx,
     uuid
 Roxygen: list(markdown=TRUE, r6=FALSE, old_usage=TRUE)
-RoxygenNote: 7.3.1
+RoxygenNote: 7.3.2

NAMESPACE

@@ -2,6 +2,12 @@
 
 S3method(adls_filesystem,adls_endpoint)
 S3method(adls_filesystem,character)
+S3method(azcopy_download_opts,adls_filesystem)
+S3method(azcopy_download_opts,blob_container)
+S3method(azcopy_download_opts,file_share)
+S3method(azcopy_upload_opts,adls_filesystem)
+S3method(azcopy_upload_opts,blob_container)
+S3method(azcopy_upload_opts,file_share)
 S3method(blob_container,blob_endpoint)
 S3method(blob_container,character)
 S3method(copy_url_to_storage,blob_container)
@@ -45,6 +51,9 @@ S3method(delete_storage_file,blob_container)
 S3method(delete_storage_file,file_share)
 S3method(delete_storage_snapshot,blob_container)
 S3method(delete_storage_version,blob_container)
+S3method(dispose_download_dest,conn_dest)
+S3method(dispose_download_dest,file_dest)
+S3method(dispose_download_dest,null_dest)
 S3method(file_share,character)
 S3method(file_share,file_endpoint)
 S3method(get_account_sas,az_storage)
@@ -64,6 +73,9 @@ S3method(get_user_delegation_key,blob_endpoint)
 S3method(get_user_delegation_sas,az_storage)
 S3method(get_user_delegation_sas,blob_endpoint)
 S3method(get_user_delegation_sas,default)
+S3method(init_download_dest,NULL)
+S3method(init_download_dest,character)
+S3method(init_download_dest,rawConnection)
 S3method(list_adls_filesystems,adls_endpoint)
 S3method(list_adls_filesystems,character)
 S3method(list_blob_containers,blob_endpoint)
@@ -80,15 +92,21 @@ S3method(list_storage_files,file_share)
 S3method(list_storage_snapshots,blob_container)
 S3method(list_storage_versions,blob_container)
 S3method(multicopy_url_to_storage,blob_container)
+S3method(normalize_src,character)
+S3method(normalize_src,rawConnection)
+S3method(normalize_src,textConnection)
 S3method(print,adls_endpoint)
 S3method(print,adls_filesystem)
 S3method(print,blob_container)
 S3method(print,file_share)
 S3method(print,storage_endpoint)
+S3method(retry_transfer,error)
+S3method(retry_transfer,response)
 S3method(revoke_user_delegation_keys,az_storage)
 S3method(set_storage_metadata,adls_filesystem)
 S3method(set_storage_metadata,blob_container)
 S3method(set_storage_metadata,file_share)
+S3method(sign_request,default)
 S3method(storage_container,adls_endpoint)
 S3method(storage_container,blob_endpoint)
 S3method(storage_container,character)

NEWS.md

@@ -66,8 +66,8 @@
 
 - ADLS, file and block blob uploads gain the option to compute and store the MD5 hash of the uploaded file, via the `put_md5` argument to `upload_adls_file`, `upload_azure_file` and `upload_blob`.
 - Similarly, downloads gain the option to verify the integrity of the downloaded file using the MD5 hash, via the `check_md5` argument to `download_adls_file`, `download_azure_file` and `download_blob`. This requires that the file's `Content-MD5` property is set.
-- Add support for uploading to [append blobs](https://docs.microsoft.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs), which are a type of blob optimized for append operations. They are useful for data that is constantly growing, but should not be modified once written, such as server logs. See `?upload_blob` for more details.
-- Add support for the [Azurite](https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azurite) and [Azure SDK](https://docs.microsoft.com/en-us/azure/storage/common/storage-use-emulator) storage emulators. To connect to the endpoint, use the service-specific functions `blob_endpoint` and `queue_endpoint` (the latter from the AzureQstor package), passing the full URL including the account name: `blob_endpoint("http://127.0.0.1:10000/myaccount", key="mykey")`. The warning about an unrecognised endpoint can be ignored. See the linked pages for full details on how to authenticate to the emulator.<br/>
+- Add support for uploading to [append blobs](https://learn.microsoft.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs), which are a type of blob optimized for append operations. They are useful for data that is constantly growing, but should not be modified once written, such as server logs. See `?upload_blob` for more details.
+- Add support for the [Azurite](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite) and [Azure SDK](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-emulator) storage emulators. To connect to the endpoint, use the service-specific functions `blob_endpoint` and `queue_endpoint` (the latter from the AzureQstor package), passing the full URL including the account name: `blob_endpoint("http://127.0.0.1:10000/myaccount", key="mykey")`. The warning about an unrecognised endpoint can be ignored. See the linked pages for full details on how to authenticate to the emulator.<br/>
   Note that the Azure SDK emulator is no longer being actively developed; it's recommended to use Azurite.
 - Add a 10-second fuzz factor to the default starting datetime for a generated SAS, to allow for differences in clocks.
 - More fixes to the directory handling of `list_blobs()`.

R/azcopy.R

@@ -81,6 +81,7 @@ azcopy_upload_opts <- function(container, ...)
     UseMethod("azcopy_upload_opts")
 }
 
+#' @exportS3Method
 azcopy_upload_opts.blob_container <- function(container, type="BlockBlob", blocksize=2^24, recursive=FALSE,
                                               lease=NULL, put_md5=FALSE, ...)
 {
@@ -90,12 +91,14 @@ azcopy_upload_opts.blob_container <- function(container, type="BlockBlob", block
       if(put_md5) "--put-md5")
 }
 
+#' @exportS3Method
 azcopy_upload_opts.file_share <- function(container, blocksize=2^22, recursive=FALSE, put_md5=FALSE, ...)
 {
     c("--block-size-mb", sprintf("%.0f", blocksize/1048576), if(recursive) "--recursive",
       if(put_md5) "--put-md5")
 }
 
+#' @exportS3Method
 azcopy_upload_opts.adls_filesystem <- function(container, blocksize=2^24, recursive=FALSE, lease=NULL,
                                                put_md5=FALSE, ...)
 {
@@ -123,18 +126,21 @@ azcopy_download_opts <- function(container, ...)
 }
 
 # currently all azcopy_download_opts methods are the same
+#' @exportS3Method
 azcopy_download_opts.blob_container <- function(container, overwrite=FALSE, recursive=FALSE, check_md5=FALSE, ...)
 {
     c(paste0("--overwrite=", tolower(as.character(overwrite))), if(recursive) "--recursive",
       if(check_md5) c("--check-md5", "FailIfDifferent"))
 }
 
+#' @exportS3Method
 azcopy_download_opts.file_share  <- function(container, overwrite=FALSE, recursive=FALSE, check_md5=FALSE, ...)
 {
     c(paste0("--overwrite=", tolower(as.character(overwrite))), if(recursive) "--recursive",
       if(check_md5) c("--check-md5", "FailIfDifferent"))
 }
 
+#' @exportS3Method
 azcopy_download_opts.adls_filesystem <- function(container, overwrite=FALSE, recursive=FALSE, check_md5=FALSE, ...)
 {
     c(paste0("--overwrite=", tolower(as.character(overwrite))), if(recursive) "--recursive",

R/download_dest.R

@@ -4,6 +4,7 @@ init_download_dest <- function(dest, overwrite)
 }
 
 
+#' @exportS3Method
 init_download_dest.character <- function(dest, overwrite)
 {
     if(!overwrite && file.exists(dest))
@@ -15,12 +16,14 @@ init_download_dest.character <- function(dest, overwrite)
 }
 
 
+#' @exportS3Method
 init_download_dest.rawConnection <- function(dest, overwrite)
 {
     structure(dest, class=c("conn_dest", class(dest)))
 }
 
 
+#' @exportS3Method
 init_download_dest.NULL <- function(dest, overwrite)
 {
     con <- rawConnection(raw(0), "w+b")
@@ -34,18 +37,21 @@ dispose_download_dest <- function(dest)
 }
 
 
+#' @exportS3Method
 dispose_download_dest.file_dest <- function(dest)
 {
     close(dest)
 }
 
 
+#' @exportS3Method
 dispose_download_dest.conn_dest <- function(dest)
 {
     seek(dest, 0)
 }
 
 
+#' @exportS3Method
 dispose_download_dest.null_dest <- function(dest)
 {
     close(dest)

R/sign_request.R

@@ -12,6 +12,7 @@ sign_request <- function(endpoint, ...)
 }
 
 
+#' @exportS3Method
 sign_request.default <- function(endpoint, verb, url, headers, api, ...)
 {
     host_url <- httr::parse_url(endpoint$url)

R/storage_utils.R

@@ -219,12 +219,14 @@ retry_transfer <- function(res)
     UseMethod("retry_transfer")
 }
 
+#' @exportS3Method
 retry_transfer.error <- function(res)
 {
     grepl("curl", deparse(res$call[[1]]), fixed=TRUE) &&
         !grepl("Could not resolve host", res$message, fixed=TRUE)
 }
 
+#' @exportS3Method
 retry_transfer.response <- function(res)
 {
     httr::status_code(res) == 400L &&

R/transfer_utils.R

@@ -4,6 +4,7 @@ normalize_src <- function(src, put_md5=FALSE)
 }
 
 
+#' @exportS3Method
 normalize_src.character <- function(src, put_md5=FALSE)
 {
     content_type <- mime::guess_type(src)
@@ -19,6 +20,7 @@ normalize_src.character <- function(src, put_md5=FALSE)
 }
 
 
+#' @exportS3Method
 normalize_src.textConnection <- function(src, put_md5=FALSE)
 {
     # convert to raw connection
@@ -33,6 +35,7 @@ normalize_src.textConnection <- function(src, put_md5=FALSE)
 }
 
 
+#' @exportS3Method
 normalize_src.rawConnection <- function(src, put_md5=FALSE)
 {
     content_type <- "application/octet-stream"

README.md

@@ -4,7 +4,7 @@
 ![Downloads](https://cranlogs.r-pkg.org/badges/AzureStor)
 ![R-CMD-check](https://github.com/Azure/AzureStor/workflows/R-CMD-check/badge.svg)
 
-This package implements both an admin- and client-side interface to [Azure Storage Services](https://docs.microsoft.com/en-us/rest/api/storageservices/). The admin interface uses R6 classes and extends the framework provided by [AzureRMR](https://github.com/Azure/AzureRMR). The client interface provides several S3 methods for efficiently managing storage and performing file transfers.
+This package implements both an admin- and client-side interface to [Azure Storage Services](https://learn.microsoft.com/en-us/rest/api/storageservices/). The admin interface uses R6 classes and extends the framework provided by [AzureRMR](https://github.com/Azure/AzureRMR). The client interface provides several S3 methods for efficiently managing storage and performing file transfers.
 
 The primary repo for this package is at https://github.com/Azure/AzureStor; please submit issues and PRs there. It is also mirrored at the Cloudyr org at https://github.com/cloudyr/AzureStor. You can install the development version of the package with `devtools::install_github("Azure/AzureStor")`.
 
@@ -162,7 +162,7 @@ storage_upload(cont, src="logfile2.csv", dest="logfile.csv", type="AppendBlob",
 
 ### Interface to AzCopy
 
-AzureStor includes an interface to [AzCopy](https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10), Microsoft's high-performance commandline utility for copying files to and from storage. To take advantage of this, simply include the argument `use_azcopy=TRUE` on any upload or download function. AzureStor will then call AzCopy to perform the file transfer, rather than using its own internal code. In addition, a `call_azcopy` function is provided to let you use AzCopy for any task.
+AzureStor includes an interface to [AzCopy](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10), Microsoft's high-performance commandline utility for copying files to and from storage. To take advantage of this, simply include the argument `use_azcopy=TRUE` on any upload or download function. AzureStor will then call AzCopy to perform the file transfer, rather than using its own internal code. In addition, a `call_azcopy` function is provided to let you use AzCopy for any task.
 
 ```r
 # use azcopy to download