Skip to content

Updates for light imagery #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 23 commits into from
Sep 5, 2025
Merged

Updates for light imagery #15

merged 23 commits into from
Sep 5, 2025

Conversation

@krokicki
Copy link
Contributor

@krokicki krokicki commented Aug 27, 2025

This PR consolidates several feature branches that support conversion of light imagery:

  • Builds on top of the tif_volume branch
  • Move to Pixi build
  • Allow LSF directives on the command line
  • Create .zattrs before writing data
  • Use Z axis for slabs
  • Optionally turn off read optimizations because they don't work in all cases
  • Don't write empty units (breaks Neuroglancer)
  • Additional logging with print
  • Flush to see prints while running on LSF
  • Add multiscale code from github.com/janelia-cellmap/zarr-multiscale (still needs to be properly integrated)

yuriyzubov and others added 23 commits August 8, 2025 20:28
@yuriyzubov
🐛 Fixing loading of tiff tiles
0a5340d
@yuriyzubov
Add support for (c, z, y, x) 4D tiffs
23c5df8
@yuriyzubov
add tests for 4D (c, z, y, x) TIFFs
f9880b2
@yuriyzubov
Add option for logging LSF dask workers output in a directory
02a4297
@krokicki
Merge pull request #1 from krokicki/tif_volume
4bb9221 
TIFF support
@krokicki
move to pixi build
649f858
@krokicki
added zarrify task
d2487f2
@krokicki
allow extra LSF directives on the command line
d2a6474
@krokicki
clean up
36f52cc
@krokicki
remove compile dependency
bbbe89f
@krokicki
refactor volumes so that they don't create output zarrs
6583b6a
@krokicki
rename zarr_factory to zarr_utils
887028e
@krokicki
determine full_scale_group_name from array name
36a4744
@krokicki
use z axis for slabs
d7f4921
@krokicki
don't output empty units
556f3bc
@krokicki
use prints consistently
27550d1
@krokicki
made read chunk optimization into an option, because it doesn't gener…
8d868a0 
…alize yet
@krokicki
flush prints
5710086
@krokicki
added multiscale code from janelia-cellmap/zarr-multiscale (downsampl…
e1ec799 
…ing factors branch)
@krokicki
ignore claude files
db22398
@krokicki
flush prints
5eea24f
@krokicki
updated README
80b3bbe
@krokicki
Merge pull request #2 from krokicki/updated-docs
e58056f 
Refactor and add multiscale
@yuriyzubov yuriyzubov mentioned this pull request Sep 5, 2025
Merged
yuriyzubov added a commit that referenced this pull request Sep 5, 2025
@yuriyzubov
Merge pull request #16 from janelia-cellmap/multiscale
5511156 
extension of #15 
This PR consolidates several feature branches that support conversion of light imagery:

    Builds on top of the tif_volume branch
    Move to Pixi build
    Allow LSF directives on the command line
    Create .zattrs before writing data
    Use Z axis for slabs
    Optionally turn off read optimizations because they don't work in all cases
    Don't write empty units (breaks Neuroglancer)
Added:

    Integrated multiscale pyramid generation
    Added config as click option to specify all the necessary parameters in a yaml file
    Added pydantic model to validate input parameters
    Rewrite optimize reads for tiff file conversion. Read the smallest tiff segment but big enough to write into chunks only once (align with the chunk grid)
@yuriyzubov yuriyzubov merged commit e58056f into janelia-cellmap:main Sep 5, 2025
0 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@krokicki @yuriyzubov