Skip to content

Commit ce50f40

Browse files
ilan-goldLDeakin
ilan-gold
and
LDeakin
authored
(chore): update benchmarks (#6)
* (chore): update benchmarks * (fix): dask perf + block indexing fix * (chore): upload more results * fix: use shard shape in non-dask benchmarks * chore: bump all deps * chore: add updated benchmarks * chore: add updated plots --------- Co-authored-by: Lachlan Deakin <[email protected]>
1 parent 6172bcf commit ce50f40

28 files changed

+2160
-2361
lines changed

‎makefile

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
11
setup:
22
uv venv --python 3.12
33
uv pip install -r requirements.txt
4-
uv pip install git+https://github.com/ilan-gold/zarrs-python.git@ld/chunk_codec_concurrency
5-
# uv pip install --upgrade --force-reinstall ../zarrs-python
6-
cargo binstall [email protected]
4+
cargo binstall [email protected]
75

86
binstall:
97
curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash

‎measurements/benchmark_read_all.csv

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
Metric,Time (s),Time (s),Time (s),Time (s),Time (s),Time (s),Memory (GB),Memory (GB),Memory (GB),Memory (GB),Memory (GB),Memory (GB)
22
Implementation,zarrs_rust,tensorstore_python,zarr_python,zarrs_python,zarr_dask_python,zarrs_dask_python,zarrs_rust,tensorstore_python,zarr_python,zarrs_python,zarr_dask_python,zarrs_dask_python
33
Image,,,,,,,,,,,,
4-
data/benchmark.zarr,2.37,2.98,3.42,3.09,5.03,4.54,9.078368,11.447836,8.76694,9.22934,16.839548,16.847508
5-
data/benchmark_compress.zarr,2.15,2.94,3.06,2.51,5.19,4.0,9.3663,9.688068,8.988624,9.42066,17.131092,17.061112
6-
data/benchmark_compress_shard.zarr,1.65,2.02,29.08,2.28,40.73,4.24,8.458552,8.63794,9.01874,8.508508,17.01946,16.937372
4+
data/benchmark.zarr,2.39,2.88,3.1,2.99,5.21,4.92,9.1216,11.357908,8.751452,9.225376,16.881816,16.886612
5+
data/benchmark_compress.zarr,2.13,2.51,3.12,2.57,5.25,4.55,9.366084,9.5691,8.987,9.405832,17.160644,16.998852
6+
data/benchmark_compress_shard.zarr,1.61,2.05,29.54,2.26,44.98,4.33,8.457372,8.621676,9.01544,8.510408,17.066332,16.984744

‎measurements/benchmark_read_all.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
| Image | Time (s)<br>zarrs<br>rust | <br>tensorstore<br>python | <br>zarr<br>python | <br>zarrs<br>python | <br>zarr<br>dask<br>python | <br>zarrs<br>dask<br>python | Memory (GB)<br>zarrs<br>rust | <br>tensorstore<br>python | <br>zarr<br>python | <br>zarrs<br>python | <br>zarr<br>dask<br>python | <br>zarrs<br>dask<br>python |
22
|:-----------------------------------|----------------------------:|----------------------------:|---------------------:|----------------------:|-----------------------------:|------------------------------:|-------------------------------:|----------------------------:|---------------------:|----------------------:|-----------------------------:|------------------------------:|
3-
| data/benchmark.zarr | 2.37 | 2.98 | 3.42 | 3.09 | 5.03 | 4.54 | 9.08 | 11.45 | 8.77 | 9.23 | 16.84 | 16.85 |
4-
| data/benchmark_compress.zarr | 2.15 | 2.94 | 3.06 | 2.51 | 5.19 | 4.00 | 9.37 | 9.69 | 8.99 | 9.42 | 17.13 | 17.06 |
5-
| data/benchmark_compress_shard.zarr | 1.65 | 2.02 | 29.08 | 2.28 | 40.73 | 4.24 | 8.46 | 8.64 | 9.02 | 8.51 | 17.02 | 16.94 |
3+
| data/benchmark.zarr | 2.39 | 2.88 | 3.10 | 2.99 | 5.21 | 4.92 | 9.12 | 11.36 | 8.75 | 9.23 | 16.88 | 16.89 |
4+
| data/benchmark_compress.zarr | 2.13 | 2.51 | 3.12 | 2.57 | 5.25 | 4.55 | 9.37 | 9.57 | 8.99 | 9.41 | 17.16 | 17.00 |
5+
| data/benchmark_compress_shard.zarr | 1.61 | 2.05 | 29.54 | 2.26 | 44.98 | 4.33 | 8.46 | 8.62 | 9.02 | 8.51 | 17.07 | 16.98 |

‎measurements/benchmark_read_chunks.csv

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,21 @@
11
Metric,,Time (s),Time (s),Time (s),Time (s),Time (s),Time (s),Memory (GB),Memory (GB),Memory (GB),Memory (GB),Memory (GB),Memory (GB)
22
Implementation,,zarrs_rust,tensorstore_python,zarr_python,zarrs_python,zarr_dask_python,zarrs_dask_python,zarrs_rust,tensorstore_python,zarr_python,zarrs_python,zarr_dask_python,zarrs_dask_python
33
Image,Concurrency,,,,,,,,,,,,
4-
data/benchmark.zarr,1,4.1,5.46,5.0,6.45,5.52,6.74,0.038192,0.123992,0.115932,0.120476,0.124684,0.131216
5-
data/benchmark.zarr,2,2.43,3.26,3.25,3.89,3.42,3.76,0.073104,0.190044,0.183576,0.186528,0.19014,0.193088
6-
data/benchmark.zarr,4,1.62,2.46,2.71,2.55,2.82,2.71,0.136564,0.312456,0.312724,0.313584,0.321028,0.302072
7-
data/benchmark.zarr,8,1.54,2.2,2.68,2.61,3.18,2.63,0.258544,0.414088,0.574772,0.475676,0.57898,0.410292
8-
data/benchmark.zarr,16,1.5,2.18,2.85,2.48,3.17,2.65,0.522484,0.713756,1.05534,0.726668,1.036324,0.668352
9-
data/benchmark.zarr,32,1.54,2.36,2.92,2.5,3.2,2.68,0.771308,1.564,2.06194,1.253872,1.93922,1.20242
10-
data/benchmark_compress.zarr,1,4.6,6.49,6.06,5.98,6.16,6.33,0.048276,0.15028,0.131444,0.374032,0.135604,0.348036
11-
data/benchmark_compress.zarr,2,2.2,3.76,3.65,3.46,3.78,3.57,0.09458,0.219012,0.213024,0.423576,0.259176,0.412632
12-
data/benchmark_compress.zarr,4,1.13,2.4,2.56,2.21,2.5,2.27,0.177688,0.364212,0.46168,0.530944,0.407076,0.507436
13-
data/benchmark_compress.zarr,8,0.69,1.95,2.22,1.93,2.32,2.09,0.327952,0.582932,0.741816,0.696572,0.644508,0.62458
14-
data/benchmark_compress.zarr,16,0.71,1.94,2.22,1.92,2.42,2.06,0.512548,0.887996,1.176644,0.93344,1.207852,0.879792
15-
data/benchmark_compress.zarr,32,0.76,1.93,2.42,1.97,2.73,2.23,0.723472,1.533288,2.17394,1.475916,2.337784,1.352212
16-
data/benchmark_compress_shard.zarr,1,2.28,2.81,35.19,3.64,26.36,3.59,0.082008,0.136236,0.33542,0.137716,0.314544,0.14142
17-
data/benchmark_compress_shard.zarr,2,1.53,2.17,33.19,2.2,24.7,2.39,0.12454,0.178248,0.441248,0.209328,0.428516,0.19052
18-
data/benchmark_compress_shard.zarr,4,1.32,1.84,32.03,1.85,27.12,2.05,0.203408,0.265768,0.586076,0.305272,0.638644,0.305424
19-
data/benchmark_compress_shard.zarr,8,1.16,1.72,31.47,1.92,29.55,2.01,0.360468,0.41328,0.834048,0.466712,0.958224,0.42636
20-
data/benchmark_compress_shard.zarr,16,1.06,1.67,31.72,1.88,33.62,2.05,0.619252,0.522008,1.372836,0.70762,1.784544,0.674872
21-
data/benchmark_compress_shard.zarr,32,1.13,1.73,33.18,1.97,36.8,2.21,0.884464,0.912768,2.448768,1.228768,3.489496,1.234684
4+
data/benchmark.zarr,1,4.76,5.74,4.93,6.62,5.64,6.9,0.038416,0.119924,0.113,0.1173,0.16642,0.168356
5+
data/benchmark.zarr,2,2.28,3.49,3.01,3.68,3.62,4.01,0.071064,0.186696,0.178636,0.183236,0.231764,0.235456
6+
data/benchmark.zarr,4,1.57,2.43,2.75,2.54,3.14,3.27,0.130256,0.318244,0.313612,0.31204,0.364584,0.349972
7+
data/benchmark.zarr,8,1.53,2.15,2.74,2.63,3.11,2.76,0.249144,0.407156,0.5737,0.454324,0.598492,0.483656
8+
data/benchmark.zarr,16,1.56,2.03,2.8,2.34,3.35,2.94,0.514944,0.739828,1.029564,0.7405,1.030476,0.721012
9+
data/benchmark.zarr,32,1.6,2.43,2.88,2.39,3.28,3.09,0.78834,1.627972,2.048828,1.280544,2.043356,1.241544
10+
data/benchmark_compress.zarr,1,4.11,5.64,6.25,5.66,6.77,6.35,0.047556,0.145592,0.128384,0.14066,0.17968,0.195464
11+
data/benchmark_compress.zarr,2,2.05,3.27,3.63,3.14,4.03,3.69,0.095644,0.229376,0.211856,0.220604,0.288052,0.26788
12+
data/benchmark_compress.zarr,4,1.1,1.98,2.52,2.14,2.8,2.55,0.164932,0.367956,0.430372,0.379696,0.44482,0.380744
13+
data/benchmark_compress.zarr,8,0.69,1.51,2.21,1.93,2.64,2.37,0.301296,0.516724,0.732408,0.591456,0.792676,0.565356
14+
data/benchmark_compress.zarr,16,0.74,1.59,2.19,1.92,2.64,2.36,0.5247,0.827748,1.260192,0.867852,1.250276,0.818392
15+
data/benchmark_compress.zarr,32,0.79,1.63,2.33,1.94,2.78,2.37,0.688444,1.424012,2.18272,1.33024,2.164008,1.313892
16+
data/benchmark_compress_shard.zarr,1,2.42,2.81,36.58,3.57,28.63,3.86,0.081292,0.137776,0.309596,0.136036,0.354168,0.18636
17+
data/benchmark_compress_shard.zarr,2,1.65,2.23,31.37,2.17,27.13,2.76,0.120428,0.17838,0.427832,0.208684,0.470932,0.224104
18+
data/benchmark_compress_shard.zarr,4,1.31,1.92,30.28,1.91,29.61,2.32,0.204636,0.26516,0.583808,0.313416,0.648056,0.312048
19+
data/benchmark_compress_shard.zarr,8,1.16,1.79,29.38,1.78,33.13,2.21,0.362204,0.387804,0.840772,0.456744,0.995296,0.460824
20+
data/benchmark_compress_shard.zarr,16,1.12,1.73,30.11,1.82,36.78,2.22,0.609532,0.545428,1.395968,0.737816,1.817132,0.720144
21+
data/benchmark_compress_shard.zarr,32,1.17,1.72,31.79,1.76,39.55,2.27,0.883988,0.903796,2.491672,1.246516,3.559228,1.233076

0 commit comments

Comments
 (0)