Add CI/CD tests with gctorture2(1, inhibit_release = TRUE)

[krlmlr]

Running the full test suite with gctorture() is prohibitive. I think it might be possible to devise a minimal set of tests, and in these tests to run only the tibblify() call in gctorture() . The following set of tests (to be stored in covr.R) already gives 50% of coverage:

tibblify(list())

spec <- spec_object(
  a = tib_dbl("a")
)
try(tibblify(list(a = 1:2), spec = spec))
try(tibblify(list(), spec = spec))
try(tibblify(list(a = 1, a = 1), spec = spec))
try(tibblify(list(1), spec = spec))
try(tibblify(list(1, b = 1), spec = spec))

spec <- spec_object(
  a = tib_dbl("a", transform = as.numeric)
)
tibblify(list(a = "1"), spec = spec)

spec <- spec_object(
  a = tib_scalar("a", hms::hms(), transform = hms::hms)
)
tibblify(list(a = 1), spec = spec)

I am using the following script (e.g. in run.R) to find the next uncovered item:

library(tidyverse)

r_files <- fs::dir_ls("R", glob = "*.R")
# already has 100% coverage
c_files <- fs::dir_ls("src", glob = "*.c")
files <- c(r_files, c_files)

text <- map(rlang::set_names(files), readLines)
line_exclusions <- map(text, seq_along)

covr <- covr::package_coverage(type = "none", line_exclusions = line_exclusions, code = paste(readLines("covr.R"), collapse = "\n"))
covr::report(covr)

This script runs in just a few seconds. Run this script, find the next uncovered line, devise code that triggers that line, rinse, repeat.

The goal would be to:

• expand the set of minimal tests to get ~100% coverage for the C++ code
• further minimize the set of tests keeping full coverage
• implement and use gctorture_tibblify() in those tests
• if needed, split the tests into multiple files so that each runs in just a few seconds even with gctorture
• perhaps skip_if_ci() those tests, or run them in a special step on GHA

Follow-up to #113.

[krlmlr]

Current iteration, with still 50% coverage, and giving an error with gctorture():

tibblify(list())

spec <- spec_object(
  tib_dbl("a")
)
try(tibblify(list(a = 1:2), spec = spec))
try(tibblify(list(), spec = spec))
try(tibblify(list(a = 1, a = 1), spec = spec))
try(tibblify(list(1), spec = spec))
try(tibblify(list(1, b = 1), spec = spec))

# transform
spec <- spec_object(
  tib_dbl("a", transform = as.numeric)
)
tibblify(list(a = "1"), spec = spec)

# types
spec <- spec_object(
  a = tib_scalar("a", hms::hms(), required = FALSE, transform = hms::hms)
)
tibblify(list(a = 1), spec = spec)
tibblify(list(), spec = spec)
try(tibblify(list(a = 1:2), spec = spec))

spec <- spec_object(
  tib_lgl("a", required = FALSE)
)
tibblify(list(a = TRUE), spec = spec)
tibblify(list(), spec = spec)

spec <- spec_object(
  tib_int("a", required = FALSE)
)
tibblify(list(a = 1L), spec = spec)
tibblify(list(), spec = spec)

spec <- spec_object(
  tib_dbl("a", required = FALSE)
)
tibblify(list(a = 1), spec = spec)
tibblify(list(), spec = spec)

spec <- spec_object(
  tib_chr("a", required = FALSE)
)
tibblify(list(a = "x"), spec = spec)
tibblify(list(), spec = spec)

spec <- spec_object(
  tib_int_vec("a", required = FALSE)
)
tibblify(list(a = 1:2), spec = spec)

spec <- spec_df(
  tib_int("a")
)
tibblify(list(list(a = 1L), list(a = 2L)), spec = spec)

spec <- spec_df(
  tib_int("a")
)
tibblify(list(list(a = 1L), list(a = 2L)), spec = spec)

spec <- spec_df(
  .names_to = "x",
  tib_int("a")
)
tibblify(list(x = list(a = 1L), y = list(a = 2L)), spec = spec)

spec <- spec_df(
  .names_to = "x",
  tib_variant("a")
)
tibblify(list(x = list(a = 1L), y = list(a = "z")), spec = spec)

[krlmlr]

The following gives an error after four minutes (!) with gctorture2(1, inhibit_release = TRUE)

# types
spec <- spec_object(
  a = tib_scalar("a", hms::hms(), required = FALSE, transform = hms::hms)
)
tibblify(list(a = 1), spec = spec)

[krlmlr]

It seems that the most time is spent constructing and throwing the error message:

Error in `abort_lossy_cast()`:
! Lossy cast from <character> to <hms> at position(s) 1
Backtrace:
     ▆
  1. ├─base::source("covr2.R", echo = TRUE)
  2. │ ├─base::withVisible(eval(ei, envir))
  3. │ └─base::eval(ei, envir)
  4. │   └─base::eval(ei, envir)
  5. └─tibblify::tibblify(list(a = 1), spec = spec) at covr2.R:5:0
  6.   └─tibblify::tibblify_impl(x, spec) at tibblify/R/tibblify.R:53:2
  7.     └─tibblify `<fn>`()
  8.       └─hms:::vec_cast.hms.character(...)
  9.         └─hms:::abort_lossy_cast(x, to, ..., lossy = lossy)
 10.           └─rlang::abort(...)
Execution halted

[krlmlr]

Would you be open to using plogr for logging the C++ code?

[krlmlr]

...and I can confirm that #113 (comment) is indeed the cause for this particular failure.

[mgirlich]

The alternative - maybe easier - would be to test out a cpp11 implementation and see whether I encounter the same performance issues and if so whether we can fix them. Might be the easier route. What do you think?

[krlmlr]

How would a cpp11 implementation help? Are you suggesting to replace all PROTECT() and UNPROTECT() calls with cpp11 objects?

I believe a minimal test suite would also be useful in that case.