Speed up group mutate operations

[ppaxisa]

I rewrote the group mutate to match previous remarks discussed in #109 , notably avoid breaking changes when S4 columns are present in metadata. I've written a new version that uses plyranges methods when S4 columns are present and prints a message to inform user of potential speed up if coercion to S3:

mutate.GroupedGenomicRanges <- function(.data, ...) {
  
  dots <- set_dots_named(...)
  check_colnames(names(dots))
  core_cols <- names(dots) %in% c("start", "end", "width", "seqnames", "strand")
  
  # if any S4 columns in mcols use plyranges group mutate
  if(any(sapply(mcols(.data), isS4))) {
    message("metadata contains S4 columns, using plyranges group operations.\nYou can speed up group operations by coercing metadata columns to S3")
    return(mutate_grp(.data, dots))
  }
  # PPA grouped mutate speedup, 2025
  # if mutating core columns, call plyranges group for those
  if(any(core_cols)) {
    .data <- mutate_grp(.data, dots[core_cols])
  }
  # if only core columns in call, return object
  if(all(core_cols)) {
    return(.data)
  } else {
    # use vanilla dplyr group operations on metadata
    mutate_mcols_grp(.data, dots[!core_cols])
  }
}

For more details, the changes are done in a new branch here

I also added some tests to check that S4 columns are preserved if present. This version passes checks:

==> devtools::check(document = FALSE)

══ Building ════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Setting env vars:
• CFLAGS    : -Wall -pedantic -fdiagnostics-color=always
• CXXFLAGS  : -Wall -pedantic -fdiagnostics-color=always
• CXX11FLAGS: -Wall -pedantic -fdiagnostics-color=always
• CXX14FLAGS: -Wall -pedantic -fdiagnostics-color=always
• CXX17FLAGS: -Wall -pedantic -fdiagnostics-color=always
• CXX20FLAGS: -Wall -pedantic -fdiagnostics-color=always
── R CMD build ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/Volumes/workSD1/Pierre-Paul/CRCT/PPA_work/project/plyranges/DESCRIPTION’ ...
─  preparing ‘plyranges’: (421ms)
✔  checking DESCRIPTION meta-information ...
─  installing the package to build vignettes
✔  creating vignettes (56.4s)
─  checking for LF line-endings in source and make files and shell scripts (886ms)
─  checking for empty or unneeded directories
   Removed empty directory ‘plyranges/pkgdown’
─  building ‘plyranges_1.23.2.tar.gz’
   
══ Checking ════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Setting env vars:
• _R_CHECK_CRAN_INCOMING_REMOTE_               : FALSE
• _R_CHECK_CRAN_INCOMING_                      : FALSE
• _R_CHECK_FORCE_SUGGESTS_                     : FALSE
• _R_CHECK_PACKAGES_USED_IGNORE_UNUSED_IMPORTS_: FALSE
• NOT_CRAN                                     : true
── R CMD check ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
─  using log directory ‘/Volumes/workSD1/Pierre-Paul/CRCT/PPA_work/project/plyranges.Rcheck’ (360ms)
─  using R version 4.4.1 (2024-06-14)
─  using platform: x86_64-apple-darwin20
─  R was compiled by
       Apple clang version 14.0.0 (clang-1400.0.29.202)
       GNU Fortran (GCC) 12.2.0
─  running under: macOS Ventura 13.6.9
─  using session charset: UTF-8
─  using options ‘--no-manual --as-cran’ (621ms)
✔  checking for file ‘plyranges/DESCRIPTION’
─  checking extension type ... Package
─  this is package ‘plyranges’ version ‘1.23.2’
─  package encoding: UTF-8
✔  checking package namespace information ...
✔  checking package dependencies (2.2s)
✔  checking if this is a source package
✔  checking if there is a namespace
✔  checking for executable files (2.4s)
✔  checking for hidden files and directories
✔  checking for portable file names ...
✔  checking for sufficient/correct file permissions
─  checking whether package ‘plyranges’ can be installed ... [36s/37s] OK (36.6s)
✔  checking installed package size ...
✔  checking package directory ...
N  checking for future file timestamps ...
   unable to verify current time
✔  checking ‘build’ directory
✔  checking DESCRIPTION meta-information ...
✔  checking top-level files
✔  checking for left-over files ...
✔  checking index information (360ms)
✔  checking package subdirectories (1.2s)
✔  checking code files for non-ASCII characters ...
✔  checking R files for syntax errors ...
✔  checking whether the package can be loaded (10s)
✔  checking whether the package can be loaded with stated dependencies (10s)
✔  checking whether the package can be unloaded cleanly (10s)
✔  checking whether the namespace can be loaded with stated dependencies (9.7s)
✔  checking whether the namespace can be unloaded cleanly (9.8s)
─  checking loading without being on the library search path ... [11s/11s] OK (11.3s)
✔  checking dependencies in R code (19.5s)
✔  checking S3 generic/method consistency (10s)
✔  checking replacement functions (10.4s)
✔  checking foreign function calls (10.6s)
─  checking R code for possible problems ... [48s/49s] OK (48.8s)
N  checking Rd files (606ms)
   checkRd: (-1) group_by-ranges.Rd:57: Lost braces in \itemize; meant \describe ?
   checkRd: (-1) group_by-ranges.Rd:58: Lost braces in \itemize; meant \describe ?
   checkRd: (-1) ranges-anchor.Rd:58: Lost braces in \itemize; meant \describe ?
   checkRd: (-1) ranges-anchor.Rd:59: Lost braces in \itemize; meant \describe ?
   checkRd: (-1) ranges-anchor.Rd:60: Lost braces in \itemize; meant \describe ?
   checkRd: (-1) ranges-anchor.Rd:61-62: Lost braces in \itemize; meant \describe ?
   checkRd: (-1) ranges-anchor.Rd:63-64: Lost braces in \itemize; meant \describe ?
✔  checking Rd metadata ...
✔  checking Rd line widths ...
✔  checking Rd cross-references (392ms)
✔  checking for missing documentation entries (10.1s)
✔  checking for code/documentation mismatches (28.8s)
✔  checking Rd \usage sections (9.8s)
✔  checking Rd contents ...
✔  checking for unstated dependencies in examples (391ms)
✔  checking installed files from ‘inst/doc’ (368ms)
✔  checking files in ‘vignettes’ (370ms)
─  checking examples ... [47s/48s] OK (48.2s)
✔  checking for unstated dependencies in ‘tests’ ...
─  checking tests (508ms)
    [42s/42s] OKthat.R’
   * checking for unstated dependencies in vignettes ... OK
   * checking package vignettes ... OK
   * checking re-building of vignette outputs ... [18s/19s] OK
   * checking for non-standard things in the check directory ... OK
   * checking for detritus in the temp directory ... OK
   * DONE
   
   Status: 2 NOTEs
   See
     ‘/Volumes/workSD1/Pierre-Paul/CRCT/PPA_work/project/plyranges.Rcheck/00check.log’
   for details.
   
── R CMD check results ─────────────────────────────────────────────────────────────────────────────── plyranges 1.23.2 ────
Duration: 6m 7.5s

❯ checking for future file timestamps ... NOTE
  unable to verify current time

❯ checking Rd files ... NOTE
  checkRd: (-1) group_by-ranges.Rd:57: Lost braces in \itemize; meant \describe ?
  checkRd: (-1) group_by-ranges.Rd:58: Lost braces in \itemize; meant \describe ?
  checkRd: (-1) ranges-anchor.Rd:58: Lost braces in \itemize; meant \describe ?
  checkRd: (-1) ranges-anchor.Rd:59: Lost braces in \itemize; meant \describe ?
  checkRd: (-1) ranges-anchor.Rd:60: Lost braces in \itemize; meant \describe ?
  checkRd: (-1) ranges-anchor.Rd:61-62: Lost braces in \itemize; meant \describe ?
  checkRd: (-1) ranges-anchor.Rd:63-64: Lost braces in \itemize; meant \describe ?

0 errors ✔ | 0 warnings ✔ | 2 notes ✖

R CMD check succeeded

Let me know what you think!

[sa-lee]

Looks good, and very compact @ppaxisa. Happy for this to be merged if you are @mikelove.

[mikelove]

Thanks -- let me check it out this week. I'll aim to merge by Friday at latest

[mikelove]

Thanks @ppaxisa, a big speedup!

I'm thinking of modifying the message from

metadata contains S4 columns, using plyranges group operations.
You can speed up group operations by coercing metadata columns to S3

to something like:

metadata contains S4 columns, using lapply group operations.
removing S4 columns will speed up group operations.

Thoughts?

I'm not sure S3 is accurate here.

[ppaxisa]

Yes S3 is not really accurate actually, I did not think too much about the message content tbh. The edited version looks good to me. Super happy that we get to merge :)