Skip to content

Harden materialize to unexpected files #282

@robmarkcole

Description

@robmarkcole

I had opened a geotif in QGIS, later my materialize failed with

Traceback (most recent call last):
  File "/Users/robin.cole/gitlab/eda-rslearn-soil-moisture-dataset/venv/bin/rslearn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/robin.cole/gitlab/eda-rslearn-soil-moisture-dataset/rslearn/rslearn/main.py", line 887, in main
    handler()
  File "/Users/robin.cole/gitlab/eda-rslearn-soil-moisture-dataset/rslearn/rslearn/main.py", line 749, in dataset_materialize
    apply_on_windows_args(fn, args)
  File "/Users/robin.cole/gitlab/eda-rslearn-soil-moisture-dataset/rslearn/rslearn/main.py", line 372, in apply_on_windows_args
    apply_on_windows(
  File "/Users/robin.cole/gitlab/eda-rslearn-soil-moisture-dataset/rslearn/rslearn/main.py", line 364, in apply_on_windows
    for _ in tqdm.tqdm(outputs, total=num_batches):
  File "/Users/robin.cole/gitlab/eda-rslearn-soil-moisture-dataset/venv/lib/python3.11/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 873, in next
    raise value
rasterio.errors.RasterioIOError: 'dataset/tiles/era5/era5land_monthlyaveraged_2023_11/total-precipitation/geotiff.tif.aux.xml' not recognized as being in a supported file format.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions