Skip to content
This repository was archived by the owner on Dec 20, 2024. It is now read-only.

Commit 6cfcade

Browse files
fprillMeraXFlorian Prill
authored
feature: support icon graphs (#53)
* feat: topology-based encoder/processor/decoder graphs derived from an ICON grid file. * docs: attempt to set a correct license header. * doc: changed the docstring style to the Numpy-style instead of Google-style. * refactor: changed variable name `verts` into `vertices`. * fix: changed float division to double slash operator. * refactor: removed unnecessary `else` branch. * refactor: more descriptive variable name in for loop. * refactor: renamed variable (ignoring the minus sign of `phi`). * refactor: changed name of temporary variable. * refactor: remove unnecessary `else´branch. * refactor: added type annotation for completeness. * refactor: remove default in function argument. * refactor: change argument name to a more understandable name. * refactor: remove redundant code. * refactor: more Pythonic if-else statement. * refactor: more Pythonic if-else statement. * refactor: use more appropriate `LOGGER.debug` instead of verbosity flag. * refactor: more appropriate variable name. * refactor: more appropriate variable name. * refactor: unified the three ICON Edgebuilders and the two Nodebuilders. * refactor: more verbose but also clearer names for variables in mesh construction algorithm. * remove: removed obsolete function `set_constant_edge_id`. * refactor: replaced the sequential ID counter by a UUID. * revert change of copyright notice * [fix] add encoder & processor edges to the __all__ variable * [refactor] move auxiliary functions to utils.py in graphs/generate/ * [doc] added empty torso for class documentation. * [fix] fixed interfaces (masks). * [refactor] remove edge attribute calculation from this PR. * [chore] adjust copyright notice. * [doc] elaborate on icon mesh classes in rst file. * Add Icon tests * update Change log * Add __future__ annotations * fix change log --------- Co-authored-by: Marek Jacob <[email protected]> Co-authored-by: Florian Prill <[email protected]>
1 parent 30f039f commit 6cfcade

File tree

10 files changed

+968
-4
lines changed

10 files changed

+968
-4
lines changed

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Please add your functional changes to the appropriate section in the PR.
99
Keep it human-readable, your future self will thank you!
1010

1111
## [Unreleased](https://github.com/ecmwf/anemoi-graphs/compare/0.4.0...HEAD)
12+
- feat: Define node sets and edges based on an ICON icosahedral mesh (#53)
1213

1314
## [0.4.0 - LAM and stretched graphs](https://github.com/ecmwf/anemoi-graphs/compare/0.3.0...0.4.0) - 2024-11-08
1415

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
####################################
2+
Triangular Mesh with ICON Topology
3+
####################################
4+
5+
The classes `ICONMultimeshNodes` and `ICONCellGridNodes` define node
6+
sets based on an ICON icosahedral mesh:
7+
8+
- class `ICONCellGridNodes`: data grid, representing cell circumcenters
9+
- class `ICONMultimeshNodes`: hidden mesh, representing the vertices of
10+
a grid hierarchy
11+
12+
Both classes, together with the corresponding edge builders
13+
14+
- class `ICONTopologicalProcessorEdges`
15+
- class `ICONTopologicalEncoderEdges`
16+
- class `ICONTopologicalDecoderEdges`
17+
18+
are based on the mesh hierarchy that is reconstructed from an ICON mesh
19+
file in NetCDF format, making use of the `refinement_level_v` and
20+
`refinement_level_c` property contained therein.
21+
22+
- `refinement_level_v[vertex] = 0,1,2, ...`,
23+
where 0 denotes the vertices of the base grid, ie. the icosahedron
24+
including the step of root subdivision RXXB00.
25+
26+
- `refinement_level_c[cell]`: cell refinement level index such that
27+
value 0 denotes the cells of the base grid, ie. the icosahedron
28+
including the step of root subdivision RXXB00.
29+
30+
To avoid multiple runs of the reconstruction algorithm, a separate
31+
`ICONNodes` instance is created and used by the builders, see the
32+
following YAML example:
33+
34+
.. code:: yaml
35+
36+
nodes:
37+
# ICON mesh
38+
icon_mesh:
39+
node_builder:
40+
_target_: anemoi.graphs.nodes.ICONNodes
41+
name: "icon_grid_0026_R03B07_G"
42+
grid_filename: "icon_grid_0026_R03B07_G.nc"
43+
max_level_multimesh: 3
44+
max_level_dataset: 3
45+
# Data nodes
46+
data:
47+
node_builder:
48+
_target_: anemoi.graphs.nodes.ICONCellGridNodes
49+
icon_mesh: "icon_mesh"
50+
attributes: ${graph.attributes.nodes}
51+
# Hidden nodes
52+
hidden:
53+
node_builder:
54+
_target_: anemoi.graphs.nodes.ICONMultimeshNodes
55+
icon_mesh: "icon_mesh"
56+
57+
edges:
58+
# Processor configuration
59+
- source_name: ${graph.hidden}
60+
target_name: ${graph.hidden}
61+
edge_builder:
62+
_target_: anemoi.graphs.edges.ICONTopologicalProcessorEdges
63+
icon_mesh: "icon_mesh"
64+
attributes: ${graph.attributes.edges}

pyproject.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,13 @@ dependencies = [
4545
"healpy>=1.17",
4646
"hydra-core>=1.3",
4747
"matplotlib>=3.4",
48+
"netcdf4",
4849
"networkx>=3.1",
4950
"plotly>=5.19",
5051
"torch>=2.2",
5152
"torch-geometric>=2.3.1,<2.5",
5253
"trimesh>=4.1",
54+
"typeguard",
5355
]
5456

5557
optional-dependencies.all = [ ]

src/anemoi/graphs/edges/__init__.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,17 @@
88
# nor does it submit to any jurisdiction.
99

1010
from .builder import CutOffEdges
11+
from .builder import ICONTopologicalDecoderEdges
12+
from .builder import ICONTopologicalEncoderEdges
13+
from .builder import ICONTopologicalProcessorEdges
1114
from .builder import KNNEdges
1215
from .builder import MultiScaleEdges
1316

14-
__all__ = ["KNNEdges", "CutOffEdges", "MultiScaleEdges"]
17+
__all__ = [
18+
"KNNEdges",
19+
"CutOffEdges",
20+
"MultiScaleEdges",
21+
"ICONTopologicalProcessorEdges",
22+
"ICONTopologicalEncoderEdges",
23+
"ICONTopologicalDecoderEdges",
24+
]

src/anemoi/graphs/edges/builder.py

Lines changed: 134 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515

1616
import networkx as nx
1717
import numpy as np
18+
import scipy
1819
import torch
1920
from anemoi.utils.config import DotDict
2021
from hydra.utils import instantiate
@@ -80,10 +81,8 @@ def get_edge_index(self, graph: HeteroData) -> torch.Tensor:
8081
source_nodes, target_nodes = self.prepare_node_data(graph)
8182

8283
adjmat = self.get_adjacency_matrix(source_nodes, target_nodes)
83-
8484
# Get source & target indices of the edges
8585
edge_index = np.stack([adjmat.col, adjmat.row], axis=0)
86-
8786
return torch.from_numpy(edge_index).to(torch.int32)
8887

8988
def register_edges(self, graph: HeteroData) -> HeteroData:
@@ -381,7 +380,13 @@ class MultiScaleEdges(BaseEdgeBuilder):
381380
Update the graph with the edges.
382381
"""
383382

384-
VALID_NODES = [TriNodes, HexNodes, LimitedAreaTriNodes, LimitedAreaHexNodes, StretchedTriNodes]
383+
VALID_NODES = [
384+
TriNodes,
385+
HexNodes,
386+
LimitedAreaTriNodes,
387+
LimitedAreaHexNodes,
388+
StretchedTriNodes,
389+
]
385390

386391
def __init__(self, source_name: str, target_name: str, x_hops: int, **kwargs):
387392
super().__init__(source_name, target_name)
@@ -444,3 +449,129 @@ def update_graph(self, graph: HeteroData, attrs_config: DotDict | None = None) -
444449
), f"{self.__class__.__name__} requires {','.join(valid_node_names)} nodes."
445450

446451
return super().update_graph(graph, attrs_config)
452+
453+
454+
class ICONTopologicalBaseEdgeBuilder(BaseEdgeBuilder):
455+
"""Base class for computing edges based on ICON grid topology.
456+
457+
Attributes
458+
----------
459+
source_name : str
460+
The name of the source nodes.
461+
target_name : str
462+
The name of the target nodes.
463+
icon_mesh : str
464+
The name of the ICON mesh (defines both the processor mesh and the data)
465+
"""
466+
467+
def __init__(
468+
self,
469+
source_name: str,
470+
target_name: str,
471+
icon_mesh: str,
472+
source_mask_attr_name: str | None = None,
473+
target_mask_attr_name: str | None = None,
474+
):
475+
self.icon_mesh = icon_mesh
476+
super().__init__(source_name, target_name, source_mask_attr_name, target_mask_attr_name)
477+
478+
def update_graph(self, graph: HeteroData, attrs_config: DotDict = None) -> HeteroData:
479+
"""Update the graph with the edges."""
480+
assert self.icon_mesh is not None, f"{self.__class__.__name__} requires initialized icon_mesh."
481+
self.icon_sub_graph = graph[self.icon_mesh][self.sub_graph_address]
482+
return super().update_graph(graph, attrs_config)
483+
484+
def get_adjacency_matrix(self, source_nodes: NodeStorage, target_nodes: NodeStorage):
485+
"""Parameters
486+
----------
487+
source_nodes : NodeStorage
488+
The source nodes.
489+
target_nodes : NodeStorage
490+
The target nodes.
491+
"""
492+
LOGGER.info(f"Using ICON topology {self.source_name}>{self.target_name}")
493+
nrows = self.icon_sub_graph.num_edges
494+
adj_matrix = scipy.sparse.coo_matrix(
495+
(
496+
np.ones(nrows),
497+
(
498+
self.icon_sub_graph.edge_vertices[:, self.vertex_index[0]],
499+
self.icon_sub_graph.edge_vertices[:, self.vertex_index[1]],
500+
),
501+
)
502+
)
503+
return adj_matrix
504+
505+
506+
class ICONTopologicalProcessorEdges(ICONTopologicalBaseEdgeBuilder):
507+
"""Computes edges based on ICON grid topology: processor grid built
508+
from ICON grid vertices.
509+
"""
510+
511+
def __init__(
512+
self,
513+
source_name: str,
514+
target_name: str,
515+
icon_mesh: str,
516+
source_mask_attr_name: str | None = None,
517+
target_mask_attr_name: str | None = None,
518+
):
519+
self.sub_graph_address = "_multi_mesh"
520+
self.vertex_index = (1, 0)
521+
super().__init__(
522+
source_name,
523+
target_name,
524+
icon_mesh,
525+
source_mask_attr_name,
526+
target_mask_attr_name,
527+
)
528+
529+
530+
class ICONTopologicalEncoderEdges(ICONTopologicalBaseEdgeBuilder):
531+
"""Computes encoder edges based on ICON grid topology: ICON cell
532+
circumcenters for mapped onto processor grid built from ICON grid
533+
vertices.
534+
"""
535+
536+
def __init__(
537+
self,
538+
source_name: str,
539+
target_name: str,
540+
icon_mesh: str,
541+
source_mask_attr_name: str | None = None,
542+
target_mask_attr_name: str | None = None,
543+
):
544+
self.sub_graph_address = "_cell_grid"
545+
self.vertex_index = (1, 0)
546+
super().__init__(
547+
source_name,
548+
target_name,
549+
icon_mesh,
550+
source_mask_attr_name,
551+
target_mask_attr_name,
552+
)
553+
554+
555+
class ICONTopologicalDecoderEdges(ICONTopologicalBaseEdgeBuilder):
556+
"""Computes encoder edges based on ICON grid topology: mapping from
557+
processor grid built from ICON grid vertices onto ICON cell
558+
circumcenters.
559+
"""
560+
561+
def __init__(
562+
self,
563+
source_name: str,
564+
target_name: str,
565+
icon_mesh: str,
566+
source_mask_attr_name: str | None = None,
567+
target_mask_attr_name: str | None = None,
568+
):
569+
self.sub_graph_address = "_cell_grid"
570+
self.vertex_index = (0, 1)
571+
super().__init__(
572+
source_name,
573+
target_name,
574+
icon_mesh,
575+
source_mask_attr_name,
576+
target_mask_attr_name,
577+
)

0 commit comments

Comments
 (0)