Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAS-1934: Most basic implementation to allow umm-grid definitions #14

Merged
merged 8 commits into from
Sep 11, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,20 @@
## v1.0.1
### 2024-04-05
# Changelog

## [v1.1.0] - 2024-08-29
### Changed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this update to the change log format, but you'll need to make a quick change to the script that extracts information from here to make the release notes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never remember this. Thanks ed0ff0e


- [[DAS-1934](https://bugs.earthdata.nasa.gov/browse/DAS-1934)]
Input parameters that include both both resolutions (`xres` and `yres`) and
dimenions (`height` and `width`) no longer always raise an exception. An
exception is raised only when the parameters describe a grid that is not
internally consistent. [#14](https://github.com/nasa/harmony-swath-projector/pull/14)

## [v1.0.1] - 2024-04-05

This version of the Swath Projector implements black code formatting across the
entire repository. There should be no functional changes to the service.

## v1.0.0
### 2023-11-16
## [v1.0.0] - 2023-11-16

This version of the Harmony Swath Projector contains all functionality
previously released internally to EOSDIS as `sds/swot-reproject:0.0.4`.
Expand All @@ -23,3 +32,7 @@ Repository structure changes include:

For more information on internal releases prior to NASA open-source approval,
see legacy-CHANGELOG.md.

[v1.1.0]:(https://github.com/nasa/harmony-swath-projector/releases/tag/1.0.1)
[v1.0.1]:(https://github.com/nasa/harmony-swath-projector/releases/tag/1.0.1)
[v1.0.0]:(https://github.com/nasa/harmony-swath-projector/releases/tag/1.0.0)
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,14 @@ The Swath Projector can specify several options for reprojection in the
CRS. If the `scaleExtent` is not specified, it is derived from the walking the
perimeter of the input grid, reprojected those points to the target CRS, and
finding the extreme values in each reprojected dimension.
* `scaleSize`: The resolution of each output pixel in the reprojected CRS. This
should not be specified if the `height` and `width` are also supplied. The
default values are derived from finding the total area of the swath via Gauss'
Area formula, and assuming the pixels are square.
* `scaleSize`: The resolution of each output pixel in the reprojected CRS. The
default values are derived from finding the total area of the swath via
Gauss' Area formula, and assuming the pixels are square. This should not
normally be specified if the `height` and `width` are also supplied, in this
case the grid definition must be internally consistent with itself. Where
consistency is determined by the equation `scaleSize = (scaleExtent.max -
scaleExtent.min) / dimension`


All the attributes in the `format` property are optional, and have defaults as
described.
Expand Down
25 changes: 16 additions & 9 deletions bin/extract-release-notes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,24 @@
# 2023-06-16: Created.
# 2023-10-10: Copied from earthdata-varinfo repository to HOSS.
# 2024-01-03: Copied from HOSS repository to the Swath Projector.
# 2024-09-01: Copied and modified from HyBIG repository to handle new format.
#
###############################################################################

CHANGELOG_FILE="CHANGELOG.md"
VERSION_PATTERN="^## v"
# Count number of versions in version file:
number_of_versions=$(grep -c "${VERSION_PATTERN}" ${CHANGELOG_FILE})

if [ ${number_of_versions} -gt 1 ]
then
grep -B 9999 -m 2 "${VERSION_PATTERN}" ${CHANGELOG_FILE} | sed '$d' | sed '$d'
else
cat ${CHANGELOG_FILE}
fi
## captures versions
## >## v1.0.0
## >## [v1.0.0]
VERSION_PATTERN="^## [\[]v"

## captures url links
#[v1.0.0]:(https://github.com/nasa/harmony-swath-projector/releases/tag/1.0.0)
LINK_PATTERN="^\[.*\].*/tag/.*"

# Read the file and extract text between the first two occurrences of the
# VERSION_PATTERN
result=$(awk "/$VERSION_PATTERN/{c++; if(c==2) exit;} c==1" "$CHANGELOG_FILE")

# Print the result
echo "$result" | grep -v "$VERSION_PATTERN" | grep -v "$LINK_PATTERN"
2 changes: 1 addition & 1 deletion docker/service_version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.0.1
1.1.0
8 changes: 4 additions & 4 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# $ jupyter notebook.
# 5) Click on the Swath Projector User Guide notebook.
#
harmony-py ~= 0.4.9
matplotlib ~= 3.7.2
netCDF4 ~= 1.6.4
notebook ~= 7.0.0
harmony-py==0.4.15
matplotlib==3.9.2
netCDF4==1.7.1.post2
notebook==7.2.2
9 changes: 9 additions & 0 deletions swath_projector/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,12 @@ def __init__(self, missing_coordinate):
'MissingCoordinatesError',
f'Could not find coordinate {missing_coordinate}.',
)


class InvalidTargetGrid(CustomError):
"""Raised when a request specifies an incomplete or invalid grid."""

def __init__(self):
super().__init__(
'InvalidTargetGrid', 'Insufficient or invalid target grid parameters.'
)
11 changes: 6 additions & 5 deletions swath_projector/reproject.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@
from typing import Dict

from harmony.message import Message
from harmony.message_utility import has_self_consistent_grid
from pyproj import Proj
from varinfo import VarInfoFromNetCDF4

from swath_projector import nc_merge
from swath_projector.exceptions import InvalidTargetGrid
from swath_projector.interpolation import resample_all_variables

RADIUS_EARTH_METRES = (
Expand Down Expand Up @@ -122,14 +124,13 @@ def get_parameters_from_message(
if parameters['interpolation'] in [None, '', 'None']:
parameters['interpolation'] = INTERPOLATION_DEFAULT

# ERROR 5: -tr and -ts options cannot be used at the same time.
# when a user requests both a resolution and dimensions, then ensure the
# extents are consistent.
if (parameters['xres'] is not None or parameters['yres'] is not None) and (
parameters['height'] is not None or parameters['width'] is not None
):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the behaviour when only one of the dimensions (x or y) has both the resolution and the height or width defined?

I think there's maybe something to think about here, here and here, because I think now it is possible to have a self consistent grid that has (for example) scaleExtent, scaleSize and then only one of height and/or width.

I think, with these changes, it technically makes no difference - what would happen is that the code would calculate the missing value (say width) and then also calculate height. And because the grid is self-consistent, the value of height would come out the same, but it feels a bit clunky to recalculate something we already have.

I guess the conclusion is: the code isn't doing anything wrong, or getting a bad value, but there's a coupling between the x and y grid parameters that means the code doesn't quite make as much sense in the case outlined. Feel free to tell me this doesn't matter too much, because it ends up all the same, but it feels like maybe it's better to decouple x and y to make things easier to grok.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure I understand you fully.

From what I see this function is called at the beginning of reproject() inside adaper's process_item

What is the behaviour when only one of the dimensions (x or y) has both the resolution and the height or width defined?

Then if the user only gave a single value height or width, it would wipe out whichever one was input here and recalculate both.
Same thing would happen if a single resolutions was input with valid dimensions and extents.

It feels a bit clunky to recalculate something we already have.

True, but I'm a big meh about it.

there's a coupling between the x and y grid parameters that means the code doesn't quite make as much sense in the case outlined.

I feel like you're trying to get at something with this, but I don't know what it is. The only difference I can see is that we don't automatically fail when you over describe your input paramters. Adding an single extra dimension that matches the input is kinda dump but it doesn't break anything, and if you provide both we don't do any recomputations (beyond the ones we did to verify the grid)

So I need more information but leaning towards a big shrug.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking more closely here, I guess that decoupling x and y is perhaps more tricky than I'd first thought. Perhaps, I'm backing down a little bit here on my initial thoughts: yes it feels clunky/unintuitive to couple x and y the way they are now, but maybe there isn't a clean way to separate them in some cases.

Maybe I'm beginning to also say 🤷

raise Exception(
'"scaleSize", "width" or/and "height" cannot '
'be used at the same time in the message.'
)
if not has_self_consistent_grid(message):
raise InvalidTargetGrid()

if not os.path.isfile(parameters['input_file']):
raise Exception('Input file does not exist')
Expand Down
8 changes: 5 additions & 3 deletions tests/unit/test_reproject.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ def test_get_parameters_error_5(self):
dimensions, an exception is raised.

"""
exception_snippet = 'cannot be used at the same time in the message.'
exception_snippet = 'Insufficient or invalid target grid parameters.'

test_args = [
['height and scaleSize', True, False, True, True],
Expand Down Expand Up @@ -121,8 +121,10 @@ def test_get_parameters_error_5(self):

with self.assertRaises(Exception) as context:
get_parameters_from_message(message, self.granule_url, self.granule)

self.assertTrue(str(context.exception).endswith(exception_snippet))
self.assertTrue(
str(context.exception).endswith(exception_snippet),
f'Test Failed: {description}',
)

def test_get_parameters_missing_extents_or_dimensions(self):
"""Ensure that an exception is raised if there is only one of either
Expand Down
Loading