{read.abares}: Download and Import Agricultural Data from the Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES) and Australian Bureau of Statistics (ABS)

An R package for automated downloading and ingestion of data from the Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES).

About ABARES

ABARES is the research arm of the Australian federal Government Department of Agriculture, Fisheries and Forestry (DAFF). ABARES' main role is "to provide professionally independent data, research, analysis and advice that informs public and private decisions affecting Australian agriculture, fisheries and forestry"¹.

The data provided by ABARES is extensive and varied, and includes data on agricultural production, trade, and forecasts, as well as spatial data on farm locations and topsoil thickness. ABARES data are not available from other sources but are mostly available under Creative Commons Licences for reuse.

About {read.abares}: why this package?

ABARES makes several data sets freely available as spreadsheets in Microsoft Excel or CSV file formats and zip archives of geospatial data as NetCDF, GeoTIFF or shape files. {read.abares} facilitates downloading and importing these files in your R session.

Data serviced includes but is not limited to:

• the ABARES Estimates;
• the Australian Gridded Farm Data (AGFD) set;
• the Australian Agricultural and Grazing Industries Survey (AAGIS) region mapping files;
• the Historical Agricultural Forecast Database;
• the Soil Thickness for Australian areas of intensive agriculture of Layer 1 (A Horizon - top-soil) (derived from soil mapping) map;
• the ABARES Trade Data including:
  • the trade data;
  • the trade region data; and
• the Land Use Data including:
  • National scale land use data;
  • Catchment scale land use data.

Quick Start

{read.abares} is not available through CRAN (yet). The preferred method is to use {remotes} like so. Note that for Linux users, you will need to install system libraries to support geospatial packages in R, e.g., {sf} and {terra} as well as some packages for downloading data via curl. Linux users, please see².

if (!require("remotes")) {
  install.packages("remotes")
}

remotes::install_github("adamhsparks/read.abares",
  dependencies = TRUE,
  build_vignettes = TRUE
)

Or, if you prefer using {pak}, it does have some advantages. If you are using Linux and will need to install system libraries to support geospatial packages in R, e.g., {sf} and {terra}, you can install {read.abares} like so using {pak}, this method will try to handle system dependencies for you. It may also be a faster installation method as well for any operating system.

o <- options() # save default options
options(pkg.build_vignettes = TRUE)

if (!require("pak")) {
  install.packages("pak",
    repos = sprintf(
      "https://r-lib.github.io/p/pak/stable/%s/%s/%s",
      .Platform$pkgType, R.Version()$os, R.Version()$arch
    )
  )
}

pak::pak("adamhsparks/read.abares", dependencies = TRUE)
options(o) # restore default options

Features

Standardised Column Names and Orders

ABARES spreadsheet data are not always consistent in their column names or orders. {read.abares} standardises the column names and orders and uses snake_case for all colnames with the first letter capitalised of every column to help you do your work more efficiently. Columns are formatted correctly for the data type, e.g., dates are converted to Date class, and numbers are converted to numeric class where necessary, etc.

Automated Repairing of Geospatial Data

The Australian Agricultural and Grazing Industries Survey (AAGIS) region mapping files report geometry errors that can be repaired using the sf::st_make_valid() function; {read.abares} automatically repairs these geometries for you when you import the data. Or the Soil Thickness for Australian areas of intensive agriculture of Layer 1 (A Horizon - top-soil) (derived from soil mapping) NetCDF file has a classification that makes the resulting maps have an incongruous legend; {read.abares} cleans this up on import so that you only have nice continuous data in your resulting map.

Multiple Geospatial Data Classes Supported

{read.abares} supports multiple classes of objects to support your workflow with the NetCDF data. Select from spatial classes for the Australian Gridded Farm Data (AGFD) NetCDF files:

• {stars},
• {terra},
• {tidync}, or if you prefer,
• a {data.table} data.frame object of the whole data set.

Or for the Soil Thickness for Australian areas of intensive agriculture of Layer 1 (A Horizon - top-soil) (derived from soil mapping) and land use change data sets, select from:

• {stars}, or
• {terra}.

Just Shutup and do Your Work!

{read.abares} offers users a way to opt out of verbosity at the package level. There are three levels that are offered.

• quiet - no feedback except for on failure,
• minimal - feedback on failure and warnings is provided, and
• verbose - verbose feedback is provided on failure, warning and for processes like download time or data import.

About Data Serviced

You might note that not all ABARES data are serviced by this package. The list is hand-picked to be reasonably useful and maintainable, i.e., frequently updated values are not included in this, e.g., Australian crop reports. However, if there is a data set that you feel would be useful to be serviced by {read.abares}, please feel free to open an issue with details about the data set or better yet, open a pull request!

Metadata

Please report any issues or bugs.

License: MIT

Citations

Citing the data: Please refer to the ABARES website, https://www.agriculture.gov.au/abares/products/citations, on how to cite these data when you use them.

Citing {read.abares}: When citing the use of this package, please use,

library(read.abares)
citation("read.abares")
#> To cite package 'read.abares' in publications use:
#> 
#>   Sparks A (????). _read.abares: Download and Import Agricultural Data
#>   from the Australian Bureau of Agricultural and Resource Economics and
#>   Sciences (ABARES) and Australian Bureau of Statistics (ABS)_. R
#>   package version 2.0.0, <https://adamhsparks.github.io/read.abares/>.
#> 
#> A BibTeX entry for LaTeX users is
#> 
#>   @Manual{,
#>     title = {{read.abares}: Download and Import Agricultural Data from the
#>     Australian Bureau of Agricultural and Resource Economics and Sciences
#>     (ABARES) and Australian Bureau of Statistics (ABS)},
#>     author = {Adam H. Sparks},
#>     note = {R package version 2.0.0},
#>     url = {https://adamhsparks.github.io/read.abares/},
#>   }

Contributing

Code of Conduct

Please note that the {read.abares} project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

Footnotes

1. https://www.agriculture.gov.au/abares/about ↩

2. If you are using Linux, you will likely need to install several system-level libraries, {pak} will do it's best to install most of them but some may not be installable this way. For Nectar with a fresh Ubuntu image, you can use the following command to install system libraries to install {pak} and then {read.abares}. In your Linux terminal (not your R console, the "terminal" tab in RStudio should do here in most cases) type: sudo apt update && sudo apt install libcurl4-openssl-dev libgdal-dev gdal-bin libgeos-dev libproj-dev libsqlite3-dev libudunits2-dev libxml2-dev. ↩