Zarr support

[keller-mark]

Fixes #91

These changes are from both me and @Artur-man

The main public-facing changes here are:

• The ZarrAnnData class
• read_zarr and write_zarr top-level functions
• Support for from_Seurat(output_class="ZarrAnnData")
• Support for from_SingleCellExperiment(output_class="ZarrAnnData")

Internally:

• read_zarr_helpers.R is the zarr analog of read_h5ad_helpers.R
• write_zarr_helpers.R is the zarr analog of write_h5ad_helpers.R
• Test fixtures within inst/extdata/example.zarr (this makes the diff noisy, apologies)
• Lots of tests:
  • test-Zarr-read.R (35 new tests)
  • test-Zarr-write.R (70)
  • test-ZarrAnnData.R (26)
  • test-h5ad-zarr.R (17)

A number of these functions generate warnings in the R console that are intended to be followed up on to improve the code (and should probably be resolved as end users may not appreciate them), but the tests still pass despite these warnings.

Known things that are not implemented here:

• support for recarrays
• usage of mode = c("r", "r+", "a", "w", "w-", "x") parameter value

[rcannood]

Fantastic work @keller-mark and @Artur-man !

I went through the PR for a first time and left some minor comments. I will review the code by running it a couple of times next :)

[rcannood]

where are a lot of linting issues in this file -- could you run lintr::lint_package() and fix any issues that pop up?

[Artur-man]

done ... did a full lint_package check and corrected some R check issues too!

[rcannood]

This file should probably be removed

[Artur-man]

done

[Artur-man]

@rcannood @keller-mark should we add pizzarr only under Suggests in DESCRIPTION and no need to add a Remotes list here ?

Suggests:
    ...
    ...
    pizzarr
Remotes:
    keller-mark/pizzarr

[keller-mark]

Good point, at minimum we should add a note to the Github README here https://github.com/scverse/anndataR/tree/main?tab=readme-ov-file#installation

[Artur-man]

Ah, this is already added here, nevermind then only Suggests should work.
https://github.com/keller-mark/anndataR/blob/d192e68ed312f212476a5490c973619f08e7c8de/R/ZarrAnnData.R#L240-L246.

but will add this to README as well!!

[lazappi]

Not a proper review of the code but I wanted to raise that we plan to submit {anndataR} to Bioconductor (or maybe CRAN) at some point (hopefully soonish). This means we won't be able to depend on GitHub only packages. Do you have plans to submit {pizzarr} somewhere or could you rewrite this to use one of the public Zarr packages?

[Artur-man]

@lazappi thanks for the notification, pizzarr could be submitted to CRAN soon. keller-mark/pizzarr#93

[Artur-man]

@keller-mark any opinions on this ? perhaps we can make Rarr/pizzarr optional with options() depending on the speed.

Regardless it would be good to have zarr support for AnnDataR. We are currently using reticulate/basilisk env in SpatialData and causes problems already (too many dependancies, multiple env etc.). https://github.com/HelenaLC/SpatialData

That being said, any dependancy should either be in CRAN or BioC. If there are missing utilities in Rarr we can also contact the maintainer and/or send PRs there (I did some, which I can continue too).

[Artur-man]

I have quickly checked if pizzarr utilities could be replaced with https://github.com/grimbough/Rarr, unfortunately there exists a set of limitations to the BioC native package, which

• for some basic utilities such as opening zarr stores and creating groups, had to come up with custom functions
• can't read/write boolean arrays yet
• and throw errors when reading from some character arrays
  • negative length vectors are not allowed error for read_zarr_array Huber-group-EMBL/Rarr#20

Will be in touch to see if these are resolved in the future, otherwise no zarr R package is currently both in CRAN/BioC and functionally complete yet.