Is R rerddap package installed/available in the local and server settings?

[emiliom]

I did a quick look in what seemed like the package R installation and environment files. I don't know if I looked in all the right places, but it looks like the rerddap package is not installed. If that's true, I think it should be added (given the importance of ERDDAP in this workshop).

• https://github.com/ropensci/rerddap
• https://cran.r-project.org/web/packages/rerddap/vignettes/Using_rerddap.html
• https://ropensci.org/tutorials/rerddap_tutorial/

I don't know what combination of @lsetiawan, @bbest and @ocefpaf this would entail.

cc @jbosch-noaa @albenson-usgs

[ocefpaf]

Indeed. I just added it to the conda env here but that is supposed to be a plan B.

@lsetiawan I guess we should list that in the install script for R too.

[emiliom]

Thanks @ocefpaf !

Roy Mendelsohn's xtractomatic may be another nice, complementary, ERDDAP-focused package.

[ocefpaf]

Roy Mendelsohn's xtractomatic may be another nice, complementary, ERDDAP-focused package.

Done.

On a related note I am planning to show this in my 15 min presentation:

http://nbviewer.jupyter.org/gist/ocefpaf/c7eb9a23c097efcc4962b369d84dd30e

[lsetiawan]

@bbest, Are you updating your RScript to include this installation? I will use that in install the packages. Thanks!

[bbest]

Good call @emiliom on adding rerddap. Yes please @lsetiawan, thanks for using install-R-packages.R which I just updated to include rerddap:

BioData-Training-Workshop/install-R-packages.R

Line 12 in 10aec7b

"RNetCDF", "hdf5r", "ncdf4","rerddap",

[lsetiawan]

Perfect! Thanks @bbest.

[emiliom]

Thanks @bbest and @lsetiawan!

Sounds like we can close this issue now? Or should xtractomatic be added to install-R-packages.R too? @ocefpaf already added it to the conda env.

[bbest]

I have the impression that rerddap is newer and preferred over the original xtractomatic, but perhaps creators @rmendels or @sckott could chime in if they serve different purposes?

• ropensci/rerddap: R client for working with ERDDAP servers
• rmendels/xtractomatic: xtractomatic R package for accessing environmental data

Then there's:

• rmendels/rerddapXtracto: xtractomactic using rerddap.

Per the rerddapXtracto vignette:

rerddapXtracto is an R package developed to subset and extract satellite and other oceanographic related data from any ERDDAP server using the R package rerddap developed by Scott Chamberlain and the wonderful people at rOpenSci. ERDDAP is a simple to use yet powerful web data service developed by Bob Simons. rerddapXtracto extends the rerddap package by being able to extract data for a moving point in time along a user-supplied set of longitude, latitude and time points; and also extracting data within a polygon (through time). rerddapXtracto extends the functionality of the R package xtractomatic by being able to work with (hopefully) most gridded datasets available from any ERDDAP server. The disadavantage compared to xtractomatic is that the user has to do more work to obtain information about the dataset to be accessed, while in xtractomatic that information is built in.

[rmendels]

xtractomatic and rerddapXtracto can do two things that rerddap can not do without further programming: 1. make an extract along a track 2. make an extract in a polygon Clearly there is no trick to this. If you look at the rerddapXtracto code, all it does is form the URLs (after a lot of tests that the input makes sense) and make appropriate rerddap calls, or in the polygon case find the smallest enclosing box, make the calls, and then mask. Clearly if you can program this sort of thing your own code will be faster, because you do not need to check that the user is passing things that don't make sense. The difference between xtractomatic and rerddapXtracto is the former only accesses certain datasets on the ERD ERDDAP server, with information built in and making its own calls to httr, while the latter in theory can access any gridded dataset on any ERDDAP server using rerddap. xtractomatic is more extensively tested and probably more reliable, since there are many ERDDAPs with many weird grids, and to test that all cases are handled correctly in rerddapXtracto is daunting. That is why there are so many warnings on the rerddapXtracto github page. Neither xtractomatic nor rerddapXtracto handle what ERDDAP calls "table" data, rerddap can (see below for vignette with examples) A full set of Vignettes can be found at: https://rmendels.github.io including one we wrote for rerddap that wasn't in the original and gives many more actual examples. BTW - many kudos to Scott and ROpenSci for producing rerddap. I would also point out in the Python realm the in-development ERDDAPy: https://github.com/pyoceans/erddapy Sort of rerddap for Python. Let me know if you have more questions.

-Roy

On Feb 5, 2018, at 2:36 PM, Ben Best ***@***.***> wrote: I have the impression that rerddap is newer and preferred over the original xtractomatic, but perhaps creators @rmendels or @sckott could chime in if they serve different purposes? • ropensci/rerddap: R client for working with ERDDAP servers • rmendels/xtractomatic: xtractomatic R package for accessing environmental data Then there's: • rmendels/rerddapXtracto: xtractomactic using rerddap. Per the rerddapXtracto vignette: rerddapXtracto is an R package developed to subset and extract satellite and other oceanographic related data from any ERDDAP server using the R package rerddap developed by Scott Chamberlain and the wonderful people at rOpenSci. ERDDAP is a simple to use yet powerful web data service developed by Bob Simons. rerddapXtracto extends the rerddap package by being able to extract data for a moving point in time along a user-supplied set of longitude, latitude and time points; and also extracting data within a polygon (through time). rerddapXtracto extends the functionality of the R package xtractomatic by being able to work with (hopefully) most gridded datasets available from any ERDDAP server. The disadavantage compared to xtractomatic is that the user has to do more work to obtain information about the dataset to be accessed, while in xtractomatic that information is built in. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn@noaa.gov www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[sckott]

@bbest I'd go with whatever @rmendels says on this - I just try to make sure rerddap works, but I don't know much about which pkgs are best in which scenarios

[rmendels]

I would add one further note - the present version of xtractomatic (3.4.1 on CRAN) is likely the last, except for bug fixes and/or maintaining compatibility with newer versions of R or R package used in the code. My hope is to do more cleanup of code and more testing or rerddapXtracto, and have that be the go-to code in the future, and it is much more general.

-Roy

On Feb 5, 2018, at 2:52 PM, Roy Mendelssohn ***@***.***> wrote: xtractomatic and rerddapXtracto can do two things that rerddap can not do without further programming: 1. make an extract along a track 2. make an extract in a polygon Clearly there is no trick to this. If you look at the rerddapXtracto code, all it does is form the URLs (after a lot of tests that the input makes sense) and make appropriate rerddap calls, or in the polygon case find the smallest enclosing box, make the calls, and then mask. Clearly if you can program this sort of thing your own code will be faster, because you do not need to check that the user is passing things that don't make sense. The difference between xtractomatic and rerddapXtracto is the former only accesses certain datasets on the ERD ERDDAP server, with information built in and making its own calls to httr, while the latter in theory can access any gridded dataset on any ERDDAP server using rerddap. xtractomatic is more extensively tested and probably more reliable, since there are many ERDDAPs with many weird grids, and to test that all cases are handled correctly in rerddapXtracto is daunting. That is why there are so many warnings on the rerddapXtracto github page. Neither xtractomatic nor rerddapXtracto handle what ERDDAP calls "table" data, rerddap can (see below for vignette with examples) A full set of Vignettes can be found at: https://rmendels.github.io including one we wrote for rerddap that wasn't in the original and gives many more actual examples. BTW - many kudos to Scott and ROpenSci for producing rerddap. I would also point out in the Python realm the in-development ERDDAPy: https://github.com/pyoceans/erddapy Sort of rerddap for Python. Let me know if you have more questions. -Roy > On Feb 5, 2018, at 2:36 PM, Ben Best ***@***.***> wrote: > > I have the impression that rerddap is newer and preferred over the original xtractomatic, but perhaps creators @rmendels or @sckott could chime in if they serve different purposes? > > • ropensci/rerddap: R client for working with ERDDAP servers > • rmendels/xtractomatic: xtractomatic R package for accessing environmental data > Then there's: > > • rmendels/rerddapXtracto: xtractomactic using rerddap. > Per the rerddapXtracto vignette: > > rerddapXtracto is an R package developed to subset and extract satellite and other oceanographic related data from any ERDDAP server using the R package rerddap developed by Scott Chamberlain and the wonderful people at rOpenSci. ERDDAP is a simple to use yet powerful web data service developed by Bob Simons. rerddapXtracto extends the rerddap package by being able to extract data for a moving point in time along a user-supplied set of longitude, latitude and time points; and also extracting data within a polygon (through time). rerddapXtracto extends the functionality of the R package xtractomatic by being able to work with (hopefully) most gridded datasets available from any ERDDAP server. The disadavantage compared to xtractomatic is that the user has to do more work to obtain information about the dataset to be accessed, while in xtractomatic that information is built in. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. > ********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: ***@***.*** www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn@noaa.gov www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[rmendels]

@bbest Since I responded from my email and not here, just checking that people were notified that I replied.

-Roy

[bbest]

Awesome @rmendels! Thank you so much for the thorough excellent comparison of the three packages. We greatly appreciate it and all the super useful tools for science that you and @sckott / rOpenSci are producing.