-
Notifications
You must be signed in to change notification settings - Fork 21
Open
Description
I am writing a Zarr file to Google Drive using gdrivefs and Xarray, and the writes are extremely slow. The file is local to my notebook and is about 1.5 GB in size. It has a lot of variables (89), and it is chunked by day into 365 days. It has been working for about an hour and is approximately 10% finished. Here are the steps I am using:
ds = xr.open_zarr('ICE.2000.01-12.c41.zarr', consolidated=True, decode_times=False, chunks=False,
decode_cf=False, mask_and_scale=True)
gdfs = gdrivefs.GoogleDriveFileSystem(root_file_id='1PCBDhk5f3v5PoPCY3Rdcqgy4S_Yj2kCC', token='cache')
mapper = gdfs.get_mapper('ICE.2000.01-12.c41.zarr')
ds.to_zarr(mapper, compute=True, consolidated=True, encoding={})
Anything obvious that I am doing wrong here? Thanks.
cc: @raf-antwerpen
Metadata
Metadata
Assignees
Labels
No labels