Add VolumeMesher and VOLUME_MESH task type

[momchil-flex]

Currently this depends on #2569 and has been rebased on it.

Introduces the VolumeMesher, VolumeMeshMonitor, and associated data types to be able to compute and inspect a HeatChargeSimulation mesh before running the solver. From the user perspective the workflow is:

mesher = td.VolumeMesher(
    simulation=HeatChargeSimulation,
    monitors=tuple[VolumeMeshmonitor, ...],
)
mesh_job = web.Job(simulation=mesher, folder_name="VOLUME_MESH", task_name="mesher")
mesh_data = mesh_job.run()

# Inspect mesh e.g. of monitor named "2D"
mesh_data["2D"].mesh.plot()

heat_sim = mesher.simulation
web.run(
    heat_sim,
    task_name="heat_after_mesh",
    parent_tasks=[mesh_job.task_id],
)

Greptile Summary

Introduces VolumeMesher functionality to compute and inspect HeatChargeSimulation meshes before running the solver, enabling better mesh quality verification and debugging.

• Added new VolumeMesher class with associated VolumeMeshMonitor and data types to preview simulation meshes
• Extended VTK file support in TetrahedralGridDataset and TriangularGridDataset with ignore_invalid_cells parameter for improved mesh handling
• Implemented new VOLUME_MESH task type in web API with appropriate integrations
• Created AbstractHeatChargeSimulationData base class to unify HeatChargeSimulation and VolumeMesher results
• Added support for legacy VTK format (.vtk) files alongside existing .vtu support

[github-actions]

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

• tidy3d/init.py (100%)
• tidy3d/components/base_sim/data/sim_data.py (75.0%): Missing lines 134
• tidy3d/components/data/unstructured/base.py (100%)
• tidy3d/components/data/unstructured/tetrahedral.py (100%)
• tidy3d/components/data/unstructured/triangular.py (100%)
• tidy3d/components/tcad/data/monitor_data/mesh.py (94.7%): Missing lines 63
• tidy3d/components/tcad/data/sim_data.py (82.4%): Missing lines 101,569-570,572-575,579,583-586
• tidy3d/components/tcad/mesher.py (90.0%): Missing lines 24
• tidy3d/components/tcad/monitors/mesh.py (100%)
• tidy3d/components/tcad/simulation/heat_charge.py (100%)
• tidy3d/web/api/tidy3d_stub.py (33.3%): Missing lines 101-102,163-164,219-220
• tidy3d/web/api/webapi.py (100%)
• tidy3d/web/core/file_util.py (28.6%): Missing lines 93,96,98-99,102
• tidy3d/web/core/types.py (100%)

Summary

• Total: 163 lines
• Missing: 26 lines
• Coverage: 84%

tidy3d/components/base_sim/data/sim_data.py

Lines 130-135

  130         return field_value
  131 
  132     def get_monitor_by_name(self, name: str) -> AbstractMonitor:
  133         """Return monitor named 'name'."""
! 134         return self.simulation.get_monitor_by_name(name)

---

tidy3d/components/tcad/data/monitor_data/mesh.py

Lines 59-67

  59         return {"mesh": self.mesh}
  60 
  61     def field_name(self, val: str) -> str:
  62         """Gets the name of the fields to be plot."""
! 63         return "Mesh"
  64 
  65     @property
  66     def symmetry_expanded_copy(self) -> VolumeMeshData:
  67         """Return copy of self with symmetry applied."""

---

tidy3d/components/tcad/data/sim_data.py

Lines 97-105

   97     def _get_field_by_name(monitor_data: TCADMonitorDataType, field_name: Optional[str] = None):
   98         """Return a field data based on a monitor dataset and a specified field name."""
   99         if field_name is None:
  100             if len(monitor_data.field_components) > 1:
! 101                 raise DataError(
  102                     "'field_name' must be specified for datasets that store more than one field."
  103                 )
  104             field_name = next(iter(monitor_data.field_components))

---

Lines 565-587

  565     def data_monitors_match_sim(cls, val, values):
  566         """Ensure each :class:`AbstractMonitorData` in ``.data`` corresponds to a monitor in
  567         ``.simulation``.
  568         """
! 569         monitors = values.get("monitors")
! 570         mnt_names = {mnt.name for mnt in monitors}
  571 
! 572         for mnt_data in val:
! 573             monitor_name = mnt_data.monitor.name
! 574             if monitor_name not in mnt_names:
! 575                 raise DataError(
  576                     f"Data with monitor name '{monitor_name}' supplied "
  577                     f"but not found in the list of monitors."
  578                 )
! 579         return val
  580 
  581     def get_monitor_by_name(self, name: str) -> VolumeMeshMonitor:
  582         """Return monitor named 'name'."""
! 583         for monitor in self.monitors:
! 584             if monitor.name == name:
! 585                 return monitor
! 586         raise Tidy3dKeyError(f"No monitor named '{name}'")

---

tidy3d/components/tcad/mesher.py

Lines 20-25

  20         description="List of monitors to be used for the mesher.",
  21     )
  22 
  23     def _get_simulation_types(self) -> list[TCADAnalysisTypes]:
! 24         return [TCADAnalysisTypes.MESH]

---

tidy3d/web/api/tidy3d_stub.py

Lines 97-106

   97         elif type_ == "EMESimulation":
   98             sim = EMESimulation.from_file(file_path)
   99         elif type_ == "ModeSimulation":
  100             sim = ModeSimulation.from_file(file_path)
! 101         elif type_ == "VolumeMesher":
! 102             sim = VolumeMesher.from_file(file_path)
  103 
  104         return sim
  105 
  106     def to_file(

---

Lines 159-168

  159         if isinstance(self.simulation, EMESimulation):
  160             return TaskType.EME.name
  161         if isinstance(self.simulation, ModeSimulation):
  162             return TaskType.MODE.name
! 163         elif isinstance(self.simulation, VolumeMesher):
! 164             return TaskType.VOLUME_MESH.name
  165 
  166     def validate_pre_upload(
  167         self, source_required, parent_tasks: Optional[list[str]] = None
  168     ) -> None:

---

Lines 215-224

  215         elif type_ == "EMESimulationData":
  216             sim_data = EMESimulationData.from_file(file_path)
  217         elif type_ == "ModeSimulationData":
  218             sim_data = ModeSimulationData.from_file(file_path)
! 219         elif type_ == "VolumeMesherData":
! 220             sim_data = VolumeMesherData.from_file(file_path)
  221 
  222         return sim_data
  223 
  224     def to_file(self, file_path: str):

---

tidy3d/web/core/file_util.py

Lines 89-103

   89 
   90     Args:
   91         file_name: The path to the file.
   92     """
!  93     hasher = hashlib.md5()
   94 
   95     # Open the file in binary read mode ('rb'). This is crucial.
!  96     with open(file_name, "rb") as f:
   97         # Read the file in chunks to efficiently handle large files
!  98         for chunk in iter(lambda: f.read(4096), b""):
!  99             hasher.update(chunk)
  100 
  101     # Return the hexadecimal representation of the hash
! 102     return hasher.hexdigest()

---

[marc-flex]

Thanks Momchil for going through this.

I guess my main concern is that VolumeMesher doesn't add much. In fact, I think we don't need that class. Instead, we could create a new analysis type along the lines of CreateMesh() that can be passed to analysis_spec. Or even, it VolumeMesher could be an analysis type:

sim = td.HeatChargeSimulation(
     ...
     analysis_spec=td.VolumeMeshser(),
)

Even the parent task id and everything else could be used in the same way you were intended, right?

[dbochkov-flexcompute]

we could probably close #2569 since it's included in here. As for missing tests from there, I think we can just get a 2d vtk and a 3d vtk from actual backend runs and add frontend unit tests that loads them (successfully with ignore_invalid_cells=True, and failing with ignore_invalid_cells=False)

[greptile-apps]

_{14 files reviewed, 8 comments
Edit PR Review Bot Settings | Greptile}

[momchil-flex]

Added a commit to add a AbstractHeatChargeSimulationData.plot_mesh method. It can be used both for VolumeMesherData monitors and for HeatChargeSimulationData monitors, essentially wrapping the call of HeatChargeMonitorData.plot(..., field=False). Instead of overlaying material properties (since it's a bit hard to know which ones for a VolumeMeshMonitor), I made it plot the structures either filled or just an overlay, like

mesher_data.plot_mesh("mesh_ugrid_wg", ax=ax[0], structures_fill=True)
mesher_data.plot_mesh("mesh_ugrid_wg", ax=ax[1], structures_fill=False)

[momchil-flex]

we could probably close #2569 since it's included in here. As for missing tests from there, I think we can just get a 2d vtk and a 3d vtk from actual backend runs and add frontend unit tests that loads them (successfully with ignore_invalid_cells=True, and failing with ignore_invalid_cells=False)

Closing the PR and in another commit I added more tests including those for from_vtk as suggested here.