Skip to content

Multiscale #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 34 commits into from
Sep 5, 2025
Merged

Multiscale #16

merged 34 commits into from
Sep 5, 2025

Conversation

@yuriyzubov
Copy link
Member

@yuriyzubov yuriyzubov commented Sep 5, 2025

extension of #15
This PR consolidates several feature branches that support conversion of light imagery:

Builds on top of the tif_volume branch
Move to Pixi build
Allow LSF directives on the command line
Create .zattrs before writing data
Use Z axis for slabs
Optionally turn off read optimizations because they don't work in all cases
Don't write empty units (breaks Neuroglancer)

Added:

Integrated multiscale pyramid generation
Added config as click option to specify all the necessary parameters in a yaml file
Added pydantic model to validate input parameters
Rewrite optimize reads for tiff file conversion. Read the smallest tiff segment but big enough to write into chunks only once (align with the chunk grid)

yuriyzubov and others added 30 commits August 8, 2025 20:28
@yuriyzubov
🐛 Fixing loading of tiff tiles
0a5340d
@yuriyzubov
Add support for (c, z, y, x) 4D tiffs
23c5df8
@yuriyzubov
add tests for 4D (c, z, y, x) TIFFs
f9880b2
@yuriyzubov
Add option for logging LSF dask workers output in a directory
02a4297
@krokicki
Merge pull request #1 from krokicki/tif_volume
4bb9221 
TIFF support
@krokicki
move to pixi build
649f858
@krokicki
added zarrify task
d2487f2
@krokicki
allow extra LSF directives on the command line
d2a6474
@krokicki
clean up
36f52cc
@krokicki
remove compile dependency
bbbe89f
@krokicki
refactor volumes so that they don't create output zarrs
6583b6a
@krokicki
rename zarr_factory to zarr_utils
887028e
@krokicki
determine full_scale_group_name from array name
36a4744
@krokicki
use z axis for slabs
d7f4921
@krokicki
don't output empty units
556f3bc
@krokicki
use prints consistently
27550d1
@krokicki
made read chunk optimization into an option, because it doesn't gener…
8d868a0 
…alize yet
@krokicki
flush prints
5710086
@krokicki
added multiscale code from janelia-cellmap/zarr-multiscale (downsampl…
e1ec799 
…ing factors branch)
@krokicki
ignore claude files
db22398
@krokicki
flush prints
5eea24f
@krokicki
updated README
80b3bbe
@krokicki
Merge pull request #2 from krokicki/updated-docs
e58056f 
Refactor and add multiscale
@yuriyzubov
Switch to python 3.12, use dask-jobqueue==0.9.0
f49d159
@yuriyzubov
use logging; fix slab axis bug, in case z is missing
6542653
@yuriyzubov
Add config example
e66235f
@yuriyzubov
Add pydantic models for CLI/config input validation
3cb4ab0
@yuriyzubov
Add pydantic models for CLI/config input validation
484753a
@yuriyzubov
Add multiscale pyramid generation methods to Volume class
6fa0090
@yuriyzubov
Rewrite optimize reads for tiff file conversion. Read the smallest ti…
c7bb46b 
…ff segment but big enough to write into chunks only once (align with the chunk grid)
@yuriyzubov yuriyzubov merged commit 5511156 into main Sep 5, 2025
1 check passed
@yuriyzubov yuriyzubov deleted the multiscale branch September 19, 2025 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yuriyzubov @krokicki