Skip to content

r.out.gdal: Skip nodata and range checks if export is forced #6170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

ninsbl
Copy link
Member

@ninsbl ninsbl commented Aug 4, 2025

This PR follows up a post on Discourse:
https://discourse.osgeo.org/t/r-out-gdal-performance-with-known-data-type-and-range/148619

The code changes make r.out.gdal skip data range and nodata checks if the f-flag (force export) as well as the nodata, and data type option are given by the user. In that case the assumption is the user has made a deliberate choice about this parameters. Skipping checks alomst cuts export time by half...

In a worflow I have to regularly export data to multi-band GeoTiff (COG actually) where I know the data range and the suitable GDAL data type across all bands to export in advance. Even though I use the -f flag and set the nodata option, a full data type and nodata check is performed on every band. To me that feels like unnecessary processing and seems inconsistent with the documentation of the -f flag.

Any feedback is welcome...

@ninsbl ninsbl added this to the 8.5.0 milestone Aug 4, 2025
@ninsbl ninsbl added raster Related to raster data processing C Related code is in C labels Aug 4, 2025
@github-actions github-actions bot added the module label Aug 4, 2025
ninsbl and others added 2 commits August 5, 2025 08:20
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@ninsbl
Copy link
Member Author

ninsbl commented Aug 5, 2025

I see that r.out.gdal lacks a test. Will add one. [Done.]

createopt="COMPRESS=LZW",
)
assert output_file.exists(), f"{mapname} not exportd to {output_file}"
# Here I would like to assert "Checking GDAL data type and nodata value" not in std_err
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot figure out how to capture the std_err with Tools... Setting the capture_stderr parameter in Tools to True does not seem to be enough... Any hints are moste welcome...



@pytest.mark.parametrize("file_format", list(FORMAT_DICT.keys()))
def test_basic_cog_export(simple_raster_map, file_format):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test fails for COG, although the export is successful as r.out.gdal gives returncode -11. I assume the warning about the MEM driver is the reason? How can I circumvent that false negative testresult?

@ninsbl ninsbl requested review from wenzeslaus, pesekon2 and metzm August 6, 2025 21:10
@github-actions github-actions bot added Python Related code is in Python tests Related to Test Suite labels Aug 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C Related code is in C module Python Related code is in Python raster Related to raster data processing tests Related to Test Suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant