Skip to content

Commit

Permalink
ruODK v0.9.1: patch geotrace/geoshape coords
Browse files Browse the repository at this point in the history
* Closes #88
* Include test data snapshot with "bad" coordinates
* Test that bad coords are fixed, good coords kept
* Update news, citation, codemeta
* Add vignette "Spatial"
  • Loading branch information
Florian Mayer authored and Florian Mayer committed Aug 10, 2020
1 parent 41b0527 commit 8bb9f8d
Show file tree
Hide file tree
Showing 76 changed files with 579 additions and 79 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Type: Package
Package: ruODK
Title: An R Client for the ODK Central API
Version: 0.9.0.9000
Version: 0.9.1
Authors@R:
c(person(given = c("Florian", "W."),
family = "Mayer",
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ export(attachment_get)
export(attachment_link)
export(attachment_list)
export(audit_get)
export(drop_null_coords)
export(enexpr)
export(enquo)
export(ensym)
Expand Down
11 changes: 11 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# `ruODK` (development version)
* Development continues in the default branch `main`.

# `ruODK` 0.9.1
## Major fixes
ODK Central versions 0.7 to 0.9 export geotraces and geoshapes via OData with
a trailing empty coordinate. `ruODK` removes any trailing empty coordinates from
both GeoJSON and WKT formats. (#88, HT Timon Weitkamp for the bug report)

## Documentation
A new vignette "Spatial" demonstrates how to parse spatial data into native
formats, such as `sf`, and gives pointers on what to do next with them.


# `ruODK` 0.9.0
This is the release on passing
Expand Down
34 changes: 34 additions & 0 deletions R/data.R
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,23 @@
#' @encoding UTF-8
"geo_gj"

#' The parsed submissions of a form containing geofields in GeoJSON
#' with trailing empty coordinates present.
#'
#' \lifecycle{stable}
#'
#' This issue was fixed in #88.
#' ODK Central versions 0.7 - 0.9 export geotraces and geoshapes with trailing
#' empty coordinates. ruODK has a patch to drop trailing empty coordinates.
#' This dataset is used to test the patch in ruODK.
#'
#' @source \code{\link{odata_submission_get}(wkt=FALSE, parse=TRUE)}
#' run on the test form
#' `system.file("extdata", "Locations.xml", package = "ruODK")`.
#' @family included
#' @encoding UTF-8
"geo_gj88"

#' The unparsed submissions of a form containing geofields in WKT.
#'
#' \lifecycle{stable}
Expand All @@ -413,6 +430,23 @@
#' @encoding UTF-8
"geo_wkt"

#' The parsed submissions of a form containing geofields in WKT
#' with trailing empty coordinates present.
#'
#' \lifecycle{stable}
#'
#' This issue was fixed in #88.
#' ODK Central versions 0.7 - 0.9 export geotraces and geoshapes with trailing
#' empty coordinates. ruODK has a patch to drop trailing empty coordinates.
#' This dataset is used to test the patch in ruODK.
#'
#' @source \code{\link{odata_submission_get}(wkt=TRUE, parse=TRUE)}
#' run on the test form
#' `system.file("extdata", "Locations.xml", package = "ruODK")`.
#' @family included
#' @encoding UTF-8
"geo_wkt88"

#' The unparsed XML form_schema of a form from ODK Central v0.6 as nested list.
#'
#' \lifecycle{stable}
Expand Down
60 changes: 60 additions & 0 deletions R/drop_null_coords.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#' Drop any NULL coordinates from a GeoJSON geometry.
#'
#' This helper patches a bug/feature in ODK Central (versions 0.7-0.9), where
#' geotrace / geoshape GeoJSON contains a last coordinate pair with NULL
#' lat/lon (no alt/acc), and WKT ends in `, undefined NaN`.
#'
#' While \code{\link{split_geotrace}} and \code{\link{split_geoshape}} modify
#' the WKT inline, it is more maintainable to separate the GeoJSON cleaner
#' into this function.
#'
#' This helper drops the last element of a GeoJSON coordinate list if it is
#' `list(NULL, NULL)`.
#'
#' @param x A GeoJSON geometry parsed as nested list.
#' E.g. `geo_gj$path_location_path_gps`.
#' @return The nested list minus the last element (if NULL).
#' @family utilities
#' @export
#' @examples
#' # A snapshot of geo data with trailing empty coordinates.
#' data("geo_gj88")
#'
#' len_coords <- length(geo_gj88$path_location_path_gps[[1]]$coordinates)
#'
#' length(geo_gj88$path_location_path_gps[[1]]$coordinates[[len_coords]]) %>%
#' testthat::expect_equal(2)
#'
#' geo_gj88$path_location_path_gps[[1]]$coordinates[[len_coords]][[1]] %>%
#' testthat::expect_null()
#'
#' geo_gj88$path_location_path_gps[[1]]$coordinates[[len_coords]][[2]] %>%
#' testthat::expect_null()
#'
#' # The last coordinate pair is a list(NULL, NULL).
#' # Invalid coordinates like these are a choking hazard for geospatial
#' # packages. We should remove them before we can convert ODK data into native
#' # spatial formats, such as sf.
#' str(geo_gj88$path_location_path_gps[[1]]$coordinates[[len_coords]])
#'
#' geo_gj_repaired <- geo_gj88 %>%
#' dplyr::mutate(
#' path_location_path_gps = path_location_path_gps %>%
#' purrr::map(drop_null_coords)
#' )
#'
#' len_coords_repaired <- length(
#' geo_gj_repaired$path_location_path_gps[[1]]$coordinates
#' )
#' testthat::expect_equal(len_coords_repaired + 1, len_coords)
drop_null_coords <- function(x) {
# Extract and simplify last coords. list(NULL, NULL) becomes list().
xx <- purrr::compact(x$coordinates[[length(x$coordinates)]])
# Remove empty coords by setting the entire list(NULL, NULL) to NULL.
if (is.list(xx) && length(xx) == 0) {
x$coordinates[[length(x$coordinates)]] <- NULL
}
x
}

# usethis::use_test("drop_null_coords") # nolint
20 changes: 13 additions & 7 deletions R/split_geoshape.R
Original file line number Diff line number Diff line change
Expand Up @@ -69,17 +69,16 @@
#' )
#' }
split_geoshape <- function(
data,
colname,
wkt = FALSE,
odkc_version = odkc_version
) {
data,
colname,
wkt = FALSE,
odkc_version = odkc_version) {
if (nrow(data) == 0) {
# Option 1: Early exit - nothing to do
return(data)
} else if (odkc_version < 0.8) {
# Option 2: ODK linestring
# ODK Central <=0.7 ignores the WKT argument for geotrace and geoshape
# ODK Central <=0.7 ignores the WKT argument for geotrace and geoshape.
# nolint start
# ruODK::odata_submission_get(wkt = TRUE, parse = TRUE)
# ruODK::odata_submission_get(wkt = FALSE, parse = FALSE) %>%
Expand Down Expand Up @@ -116,8 +115,13 @@ split_geoshape <- function(
dplyr::rename_at(
dplyr::vars(dplyr::starts_with("XXX")),
list(~ stringr::str_replace(., "XXX", colname))
) %>%
# Drop last empty coordinate from colname, a list(NULL, NULL).
# Affects ODK Central Version 0.7-0.9.
dplyr::mutate_at(
dplyr::vars(colname),
list(~ purrr::map(., drop_null_coords))
)
# TODO #88 drop last empty coord from colname
} else {
# WKT
data %>%
Expand All @@ -132,6 +136,8 @@ split_geoshape <- function(
remove = FALSE,
convert = TRUE
) %>%
# Drop last empty coordinate from colname, a trailing ",undefined NaN".
# Affects ODK Central Version 0.7-0.9.
dplyr::mutate_at(
dplyr::vars(colname),
list(~ stringr::str_replace_all(., ",undefined NaN", ""))
Expand Down
7 changes: 6 additions & 1 deletion R/split_geotrace.R
Original file line number Diff line number Diff line change
Expand Up @@ -130,8 +130,13 @@ split_geotrace <- function(data,
dplyr::rename_at(
dplyr::vars(dplyr::starts_with("XXX")),
list(~ stringr::str_replace(., "XXX", colname))
) %>%
# Drop last empty coordinate from colname, a list(NULL, NULL).
# Affects ODK Central Version 0.7-0.9.
dplyr::mutate_at(
dplyr::vars(colname),
list(~ purrr::map(., drop_null_coords))
)
# TODO #88 drop last empty coord from colname
} else {
# Option 4: ODKC v0.8 WKT
data %>%
Expand Down
58 changes: 36 additions & 22 deletions codemeta.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,10 @@
],
"issueTracker": "https://github.com/ropensci/ruODK/issues",
"license": "https://spdx.org/licenses/GPL-3.0",
"version": "0.9.0",
"version": "0.9.1",
"programmingLanguage": {
"@type": "ComputerLanguage",
"name": "R",
"version": "4.0.0",
"url": "https://r-project.org"
},
"runtimePlatform": "R version 4.0.0 (2020-04-24)",
Expand Down Expand Up @@ -136,6 +135,19 @@
},
"sameAs": "https://CRAN.R-project.org/package=listviewer"
},
{
"@type": "SoftwareApplication",
"identifier": "mapview",
"name": "mapview",
"version": ">= 2.7.8",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=mapview"
},
{
"@type": "SoftwareApplication",
"identifier": "rmarkdown",
Expand All @@ -162,6 +174,19 @@
},
"sameAs": "https://CRAN.R-project.org/package=roxygen2"
},
{
"@type": "SoftwareApplication",
"identifier": "sf",
"name": "sf",
"version": ">= 0.9-5",
"provider": {
"@id": "https://cran.r-project.org",
"@type": "Organization",
"name": "Comprehensive R Archive Network (CRAN)",
"url": "https://cran.r-project.org"
},
"sameAs": "https://CRAN.R-project.org/package=sf"
},
{
"@type": "SoftwareApplication",
"identifier": "testthat",
Expand Down Expand Up @@ -472,28 +497,17 @@
],
"name": "ruODK: Client for the ODK Central API",
"url": "https://github.com/ropensci/ruODK",
"description": "R package version 0.9.0"
"description": "R package version 0.9.1"
}
],
"releaseNotes": "https://github.com/ropensci/ruODK/blob/master/NEWS.md",
"readme": "https://github.com/ropensci/ruODK/blob/main/README.md",
"fileSize": "1010.359KB",
"contIntegration": [
"https://travis-ci.org/ropensci/ruODK",
"https://ci.appveyor.com/project/florianm/ruodk/branch/master",
"https://codecov.io/github/ropensci/ruODK?branch=master",
"https://travis-ci.org/ropensci/ruODK",
"https://codecov.io/github/ropensci/ruODK?branch=master",
"https://ci.appveyor.com/project/florianm/ruodk/branch/master",
"https://codecov.io/gh/validmeasures/odkr",
"https://travis-ci.org/unhcr/koboloadeR",
"https://ci.appveyor.com/project/unhcr/koboloadeR",
"https://codecov.io/gh/unhcr/koboloadeR",
"https://travis-ci.org/ropensci/ruODK",
"https://ci.appveyor.com/project/florianm/ruodk/branch/master",
"https://codecov.io/github/ropensci/ruODK?branch=master",
"https://travis-ci.org/edwindj/cbsodataR",
"https://ci.appveyor.com/project/edwindj/cbsodatar"
],
"developmentStatus": "https://www.repostatus.org/#active"
"fileSize": "3126.119KB",
"contIntegration": ["https://travis-ci.org/ropensci/ruODK", "https://ci.appveyor.com/project/florianm/ruodk/branch/main", "https://codecov.io/gh/ropensci/ruODK", "https://travis-ci.org/ropensci/ruODK", "https://codecov.io/gh/ropensci/ruODK", "https://ci.appveyor.com/project/florianm/ruodk/branch/main", "https://codecov.io/gh/validmeasures/odkr", "https://travis-ci.org/unhcr/koboloadeR", "https://ci.appveyor.com/project/unhcr/koboloadeR", "https://codecov.io/gh/unhcr/koboloadeR", "https://travis-ci.org/ropensci/ruODK", "https://ci.appveyor.com/project/florianm/ruodk/branch/main", "https://codecov.io/gh/ropensci/ruODK", "https://travis-ci.org/edwindj/cbsodataR", "https://ci.appveyor.com/project/edwindj/cbsodatar"],
"developmentStatus": "https://www.repostatus.org/#active",
"review": {
"@type": "Review",
"url": "https://github.com/ropensci/software-review/issues/335",
"provider": "https://ropensci.org"
}
}
2 changes: 1 addition & 1 deletion data-raw/make_release.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ source(here::here("data-raw/make_data.R"))
devtools::test()
#
# Docs
lintr:::addin_lint_package()
styler::style_pkg()
lintr:::addin_lint_package()
devtools::document(roclets = c("rd", "collate", "namespace"))
spelling::spell_check_package()
spelling::spell_check_files("README.Rmd", lang = "en_AU") # TODO update wordlist
Expand Down
Binary file modified data/geo_gj.rda
Binary file not shown.
Binary file added data/geo_gj88.rda
Binary file not shown.
Binary file added data/geo_wkt88.rda
Binary file not shown.
4 changes: 2 additions & 2 deletions inst/CITATION
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ citEntry(
entry = "Misc",
title = "ruODK: Client for the ODK Central API",
author = "Florian W. Mayer",
note = "R package version 0.9.0",
note = "R package version 0.9.1",
year = 2020,
url = "https://github.com/ropensci/ruODK",
textVersion = paste(
"Mayer, Florian Wendelin. (2020, July 21). ",
"ruODK: An R Client for the ODK Central API (Version 0.9.0). ",
"ruODK: An R Client for the ODK Central API (Version 0.9.1). ",
"Zenodo. http://doi.org/10.5281/zenodo.3953159"
)
)
Loading

0 comments on commit 8bb9f8d

Please sign in to comment.