version conflicts using gen_forcing_era5

[biglimp]

Can you something obvious that is causing this error?

SuPy version: 2024.5.24.dev
INPUT PARAMETERS:
lat = 58.58667099540737
lon = 16.180797329367802
Start = 2000-01-01T00:00:00
End = 2000-01-02T00:00:00
Diagnostic height = 10.0
Output folder = C:\Users\xlinfr\Desktop
Traceback (most recent call last):
File "C:\Users/xlinfr/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\xlinfr\AppData\Roaming\Python\Python312\site-packages\supy\util\_era5.py", line 747, in gen_forcing_era5
ds_sfc = xr.open_mfdataset(
^^^^^^^^^^^^^^^^^^
File "C:\OSGeo4W\apps\Python312\Lib\site-packages\xarray\backends\api.py", line 1635, in open_mfdataset
datasets = [open_(p, **open_kwargs) for p in paths1d]
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OSGeo4W\apps\Python312\Lib\site-packages\xarray\backends\api.py", line 668, in open_dataset
engine = plugins.guess_engine(filename_or_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OSGeo4W\apps\Python312\Lib\site-packages\xarray\backends\plugins.py", line 194, in guess_engine
raise ValueError(error_msg)
ValueError: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'scipy', 'rasterio']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see:
https://docs.xarray.dev/en/stable/getting-started-guide/installing.html
https://docs.xarray.dev/en/stable/user-guide/io.html

Execution failed after 55.02 seconds

Loading resulting layers
Algorithm 'Meteorological Data: Download data (ERA5)' finished

[sunt05]

It appears to be an upstream issue in xarray—check whether the necessary NetCDF libraries are installed.

[biglimp]

When you say upstream, you mean on my machine? I have xarray 2025.3.0. I also tried to do pip install "xarray[io]" but it did not help. I wonder if it is my computer that is polluted or if it is a general error. We are having a lot of issues with this downloader...

@gusbacos , could you try to download a single day on your new laptop and see if you get any errors? Dont forget to set up your cdsapi key on their website.

[sunt05]

I meant the complaint is from xarray, but in supy code.

I'll probably disable that downloader—I don't have the energy to maintain it. Better to leave it as is for now, or remove it in a future version.

[gusbacos]

I get another error

QGIS version: 3.42.2-Münster
QGIS code revision: 402273f8c2
Qt version: 5.15.13
Python version: 3.12.10
GDAL version: 3.10.3
GEOS version: 3.13.1-CAPI-1.19.2
PROJ version: Rel. 9.6.0, March 15th, 2025
PDAL version: 2.8.3 (git-version: 56e405)
Algorithm started at: 2025-05-23T09:53:30
Algorithm 'Meteorological Data: Download data (ERA5)' starting…
Input parameters:
{ 'CRS' : QgsCoordinateReferenceSystem('EPSG:4326'), 'DATEINIEND' : '2000-01-02T00:00:00', 'DATEINISTART' : '2000-01-01T00:00:00', 'DIAG_HEIGHT' : 10, 'INPUT_POINT' : '1351508.887069,7918252.067771 [EPSG:3857]', 'OUTPUT_DIR' : 'C:\temp' }

SuPy version: 2024.5.24.dev
INPUT PARAMETERS:
lat = 1351508.887069
lon = 7918252.067771
Start = 2000-01-01T00:00:00
End = 2000-01-02T00:00:00
Diagnostic height = 10.0
Output folder = C:\temp
Traceback (most recent call last):
File "C:\Users/xbacos/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
^^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\legacy_client.py", line 167, in retrieve
submitted = self.client.submit_and_wait_on_results(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\client.py", line 397, in submit_and_wait_on_results
return self._retrieve_api.submit(collection_id, request).get_results()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 544, in get_results
return self._make_results(wait=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 519, in _make_results
self._wait_on_results()
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 497, in _wait_on_results
while not self.results_ready:
^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 511, in results_ready
results = self._make_results(wait=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 525, in _make_results
results = Results.from_request("get", results_url, **self._request_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 176, in from_request
cads_raise_for_status(response)
File "C:\Users\xbacos\AppData\Roaming\Python\Python312\site-packages\ecmwf\datastores\processing.py", line 99, in cads_raise_for_status
raise requests.HTTPError(message, response=response)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://cds.climate.copernicus.eu/api/retrieve/v1/jobs/e1e5f907-0a7e-4bed-bc1c-a53bd8e845e0/results
The job has failed.
The job failed with: FileNotFoundError

Execution failed after 78.90 seconds (1 minute 19 seconds)

Loading resulting layers
Algorithm 'Meteorological Data: Download data (ERA5)' finished

from the Python console I read

2025-05-23 09:53:31,817 INFO [2024-09-26T00:00:00] Watch our Forum for Announcements, news and other discussed topics.
2025-05-23 09:53:31,818 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
2025-05-23 09:53:33,310 INFO Request ID is e1e5f907-0a7e-4bed-bc1c-a53bd8e845e0
2025-05-23 09:53:33,402 INFO status has been updated to accepted
2025-05-23 09:54:23,901 INFO status has been updated to running
2025-05-23 09:54:49,623 INFO status has been updated to failed

So perhaps the API-request needs to be changed

I also tried to download directly using the cdsapi in python console, which worked fine

Oskar

[paolodep36]

The problem Is that the files are fake netcdf. Instead they are zip. So you have to unzip It and you Will find 2 real netcdf files One for instant and one for accumulation variables. So xarray Will fail reading a zip.I sent you the era5.py in another discussion