Does rioxarray.open_rasterio work better with threads or workers?

[andhuang-CLGX]

From:
https://stackoverflow.com/questions/49406987/how-do-we-choose-nthreads-and-nprocs-per-worker-in-dask-distributed

Basically, does rioxarray.open_rasterio release the GIL through GDAL?

What about to_raster()

[snowman2]

Basically, does rioxarray.open_rasterio release the GIL through GDAL?
What about to_raster()

Both of them work by opening up a new file handle for each thread/process in both reading and writing, so they should be thread/process safe. The GIL can only be released the GDAL python wraper rasterio where applicable,

[andhuang-CLGX]

Reading https://corteva.github.io/rioxarray/stable/examples/dask_read_write.html,
so far, these are the options I found best for reading one large file and exporting that large file:

import rioxarray
from dask.utils import SerializableLock
ds = rioxarray.open_rasterio(fp, chunks=True, lock=False)
ds.rio.to_raster("test.tif", driver="COG", tiled=True, lock=SerializableLock())  # note the example threading.Lock() crashes with dask

[andhuang-CLGX]

When I use multiple workers:

~/anaconda3/lib/python3.7/site-packages/rioxarray/raster_writer.py in __setitem__()
    194 
    195         with rasterio.open(self.raster_path, "r+") as rds:
--> 196             rds.write(item, window=Window(chx_off, chy_off, chx, chy), indexes=indexes)
    197 
    198     def to_raster(self, xarray_dataarray, tags, windowed, lock, compute, **kwargs):

rasterio/_io.pyx in rasterio._io.DatasetWriterBase.write()

RasterioIOError: Read or write failed. test.tif, band 1: IReadBlock failed at X offset 11, Y offset 43: TIFFReadEncodedTile() failed.

[snowman2]

Code snippet?

[andhuang-CLGX]

Nevermind, I realized I need to use not SerializableLock, but Lock as specified in docs
ds.rio.to_raster("test.tif", driver="COG", tiled=True, lock=Lock("rio", client=client))