Final 3d fixes - Compiling and solving.... #1297

lsawade · 2025-10-16T01:38:08Z

Description

... now we just need to figure out whether it's garbage

core/assembly/boundaries/dim3 — NEEDS FIXING
core/assembly/boundary_values/dim3 — Probably NEEDS FIXING
core/assembly/jacobian_matrix/dim3 — Probably NEEDS FIXING
core/Point/stress::operator() — New
core/periodic_tasks/ — dimension templated, but 3d not implemented yet
include/algorithm/interpolate — NEEDS FIXING
include/parameter_parser/setup — dimension independent
include/parameter_parser/solver — dimension independent
src/specfem3d — Updated

Uses: #1292 so that will need updating down the line

Issue Number

Not a specific number...

Checklist

Please make sure to check developer documentation on specfem docs.

I ran the code through pre-commit to check style
THE DOCUMENTATION BUILDS WITHOUT WARNINGS/ERRORS
I have added labels to the PR (see right hand side of the PR page)
My code passes all the integration tests
I have added sufficient unittests to test my changes
I have added/updated documentation for the changes I am proposing
I have updated CMakeLists to ensure my code builds
My code builds across all platforms

…art2-generate_wavefield

…ng garbage!

Co-authored-by: Copilot <[email protected]>

Rohit-Kakodkar · 2025-10-17T12:32:12Z

core/specfem/assembly/assembly/dim3/compute_wavefield/helper.hpp

This helper function looks like its dimension independent. Should it be a single implementaiton for 2D/3D?

Rohit-Kakodkar · 2025-10-17T12:47:58Z

core/specfem/assembly/boundaries.cpp

 #include "boundaries/dim2/boundaries.tpp"
 #include "boundaries/dim2/impl/acoustic_free_surface.tpp"
 #include "boundaries/dim2/impl/stacey.tpp"
+// TODO: Implement dim3 boundaries - requires adapting to different mesh


Suggested change

// TODO: Implement dim3 boundaries - requires adapting to different mesh

// TODO(Lucas : DIM3_BOUNDARIES_UPDATE): Implement dim3 boundaries - requires adapting to different mesh

Just to make it easier to grep

core/specfem/data_access/check_compatibility.hpp

Rohit-Kakodkar · 2025-10-17T12:57:28Z

include/medium/impl/data_container.hpp

+template <typename MappingType, typename IndexType>
+KOKKOS_FORCEINLINE_FUNCTION
+    std::enable_if_t<IndexType::dimension_tag == specfem::dimension::type::dim2,
+                     std::size_t>
+    get_flat_index(const MappingType &mapping, const IndexType &index) {
+  return mapping(index.ispec, index.iz, index.ix);
+}
+
+// Helper function to get flat index from mapping for dim3
+template <typename MappingType, typename IndexType>
+KOKKOS_FORCEINLINE_FUNCTION
+    std::enable_if_t<IndexType::dimension_tag == specfem::dimension::type::dim3,
+                     std::size_t>
+    get_flat_index(const MappingType &mapping, const IndexType &index) {
+  return mapping(index.ispec, index.iz, index.iy, index.ix);
+}


These should be inside an impl namespace.

…an into assembly jacobian

Issue 1295 - Implements the seismogram writer for 3D seismograms

… into issue-1289

…ent. Updated locate point to match the specfem3d algo. Updated t0 update in specfem3d

- fixes jacobian reading and population - fixes wrong use of index in the gradient calculation - Updates source stations and specfem_config to fully match the fortran simulation - updates plot_seismograms and includes reference seismograms

also: - fixes issue related to local running of snakemakes that was introduced with the nightly benchmarks - UPDATES UV.LOCK which should have been updated in Kentaro's PR for gmeshlayer builder

Updated the receivers and the force

…he body.

Issue 1289 - Implements 3D VTK writing.

…riables

…EMPP into final-3d-fixes

Final bug fixes for 3D simulation

…EMPP into final-3d-fixes

commit ba84272ec61d28be7eca5eefb41494a655f0c989 Author: Lucas Sawade <[email protected]> Date: Thu Oct 30 14:34:17 2025 -0400 Updated the Mesh generation to be <100MB commit 7cd4035191ffdb82ce770c812d3d36b7cc31ddd5 Author: Lucas Sawade <[email protected]> Date: Wed Oct 29 17:16:24 2025 -0400 This moves channel file generator from utilities to io/seismogram/impl note that in newmark_tests.cpp I'm explicitly creating one since writer output directory and trace reader directory are not necessarily the same. The alternativ is static_casting the writer to seismogram_writer and making the writer output directory the same as the reference trace directory. Either way writer now inherits from the channelgenerator making file naming in the writer easier. commit 2abbdeb8e66a516cc8533f50501d19bede30bff5 Author: Lucas Sawade <[email protected]> Date: Wed Oct 29 14:42:33 2025 -0400 This commit does a bunch of things to make the first 3D displacement test pass - [x] Enables reading of seismograms - [x] Creates a new set of functions - get_band_code to use the timestep interval to figure out the FDSN seismometer band code - get_channel_code defines the channel to from component and timestep (0.011, 'X') -> 'BXX' - get_file_extension -> uses wavefield::type to return, e.g. `semd` for displacement - get_seismogram_file_names which combines output_directory, network, and station with the above three to return a vector of appropriate filenames. - [x] Update the mesh/source-station geometry and hence traces to minimize the simulation time yet still have an ok simulation without reflections

Rohit-Kakodkar

Added a couple of TODO tags, that can be addressed soonish

Rohit-Kakodkar · 2025-10-31T16:54:10Z

include/kokkos_kernels/impl/compute_material_derivatives.tpp

 #if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)
  constexpr bool using_simd = false;
 #else
-  constexpr bool using_simd = true;
+  constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;
 #endif


Suggested change

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

constexpr bool using_simd = true;

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

// TODO(Rohit : DIM3_SIMD) Enable simd implementation for dim3 solver

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

Rohit-Kakodkar · 2025-10-31T16:55:15Z

include/kokkos_kernels/impl/compute_stiffness_interaction.tpp

 #if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)
  constexpr bool using_simd = false;
 #else
-  constexpr bool using_simd = true;
+  constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;
 #endif


Suggested change

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

constexpr bool using_simd = true;

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

// TODO(Rohit : DIM3_SIMD) Enable simd execution for dim3 solver

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

Rohit-Kakodkar · 2025-10-31T16:55:44Z

include/kokkos_kernels/impl/divide_mass_matrix.tpp

 #if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)
  constexpr bool using_simd = false;
 #else
-  constexpr bool using_simd = true;
+  constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;
 #endif


Suggested change

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

constexpr bool using_simd = true;

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

// TODO(Rohit : DIM3_SIMD)

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

Rohit-Kakodkar · 2025-10-31T16:56:13Z

include/kokkos_kernels/impl/invert_mass_matrix.tpp

 #if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)
  constexpr bool using_simd = false;
 #else
-  constexpr bool using_simd = true;
+  constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;
 #endif


Suggested change

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

constexpr bool using_simd = true;

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP)

constexpr bool using_simd = false;

#else

// TODO (Rohit : DIM3_SIMD)

constexpr bool using_simd = (DimensionTag == specfem::dimension::type::dim2) ? true : false;

#endif

Rohit-Kakodkar · 2025-10-31T16:57:03Z

include/medium/impl/data_container.hpp

 #include "enumerations/medium.hpp"
 #include <boost/preprocessor.hpp>

+namespace specfem::medium::impl {


Have a look at this merge conflict coming from PR #1329

Issue 1321 - Displacement Test for 3D

lsawade added 10 commits October 1, 2025 12:05

Intermediate commit

30e3b66

Merge branch 'issue-1216-part1-move-periodic-tasks' into issue-1216-p…

0dc0984

…art2-generate_wavefield

Fixed things as far as I was able to

3d65de1

WIP: Need element quadrature load on device

2e79561

Added boundary containers etc

d704d31

Merge branch 'issue-1264' into final-3d-fixes

52dfc01

WIP: ComputeWavefield update

4dee23c

Merge branch 'issue-1216-part2-generate_wavefield' into final-3d-fixes

2c8aae9

this compiles. crazy.

99c44ab

Timeloop is looping. Now we just need to find out whether its computi…

4810d77

…ng garbage!

lsawade requested a review from Rohit-Kakodkar October 16, 2025 01:38

lsawade added the enhancement New feature or request label Oct 16, 2025

lsawade requested a review from icui October 16, 2025 01:38

lsawade changed the base branch from devel to issue-1275 October 16, 2025 01:38

Updated the writer. Seismograms are written, but they are all nans

74ab7cc

lsawade mentioned this pull request Oct 16, 2025

Issue 1264 - Update quadrature #1292

Merged

8 tasks

lsawade added 2 commits October 16, 2025 09:22

Removed simd execution for 3d

3e2381a

Merge branch 'final-3d-fixes' into issue-1295

0b5fa01

lsawade mentioned this pull request Oct 16, 2025

Dimension template or Assembly template periodic_tasks #1289

Closed

1 task

lsawade linked an issue Oct 16, 2025 that may be closed by this pull request

Dimension template or Assembly template periodic_tasks #1289

Closed

1 task

lsawade and others added 6 commits October 16, 2025 10:18

Merge branch 'issue-1275' into final-3d-fixes

d685147

Moved the plot_wavefield structs to dim2 space

2fd5537

Updated periodic tasks

ae231e3

Added 3D VTK writing, but it shows the nans.

4872a1f

Update setup.cpp

98b617c

Co-authored-by: Copilot <[email protected]>

Update wavefield.cpp

3b16e81

Co-authored-by: Copilot <[email protected]>

Rohit-Kakodkar requested changes Oct 17, 2025

View reviewed changes

lsawade added 3 commits October 17, 2025 11:50

Fix: Bug in reading the jacobian and bug in assigning the read jacobi…

0ebd712

…an into assembly jacobian

Merge pull request #1298 from PrincetonUniversity/issue-1295

e6347e6

Issue 1295 - Implements the seismogram writer for 3D seismograms

Only apply boundary conditions in 2D and fix mass matrix

da55188

lsawade added 4 commits October 17, 2025 13:37

Merge branch 'final-3d-fixes' into issue-1289

b3013bf

Updated the cmake lists to include seismogram plotting tool

569d472

Merge branch 'issue-1289' of github.com:PrincetonUniversity/SPECFEMPP…

a3c94ac

… into issue-1289

Some fixes

78db7a9

Base automatically changed from issue-1275 to devel October 20, 2025 13:07

lsawade and others added 20 commits October 20, 2025 09:55

Removed debugging loop from check_jacobian

4be909a

stacey index flip

45b1932

Updated the receivers and the force

655554c

Updated the force vector for a better comparison from within the elem…

61f607d

…ent. Updated locate point to match the specfem3d algo. Updated t0 update in specfem3d

Final 3D bug fixes and reference plotting

da7a1e8

- fixes jacobian reading and population - fixes wrong use of index in the gradient calculation - Updates source stations and specfem_config to fully match the fortran simulation - updates plot_seismograms and includes reference seismograms

Adds Snakemake workflow for the 3D example

af82223

also: - fixes issue related to local running of snakemakes that was introduced with the nightly benchmarks - UPDATES UV.LOCK which should have been updated in Kentaro's PR for gmeshlayer builder

Merge pull request #1311 from PrincetonUniversity/receiver-fix

91e206b

Updated the receivers and the force

Updated the plot_wavefield member functions to use dimension_tag in t…

2118a69

…he body.

Removed the dim2 wrapper around apply boundary condition

0e42016

Merge branch 'issue-1289' into field-mapping-fix

598d61a

Merge pull request #1303 from PrincetonUniversity/issue-1289

4a0ddb0

Issue 1289 - Implements 3D VTK writing.

Updated the snakefile to use rules.<other rules>. to reuse the set va…

056f85d

…riables

Commented out the display section

5766b1c

Updated the flat index mapping namespace, and the TODO in boundaries

3386bca

Accidentally deleted test linking

423d687

Merge branch 'final-3d-fixes' of github.com:PrincetonUniversity/SPECF…

7c0468a

…EMPP into final-3d-fixes

Merge pull request #1324 from PrincetonUniversity/field-mapping-fix

f2a0d3f

Final bug fixes for 3D simulation

Merge branch 'devel' into final-3d-fixes

3efa517

Merge branch 'final-3d-fixes' of github.com:PrincetonUniversity/SPECF…

5dcee24

…EMPP into final-3d-fixes

Rohit-Kakodkar self-requested a review October 31, 2025 13:04

Rohit-Kakodkar approved these changes Oct 31, 2025

View reviewed changes

lsawade added 3 commits November 5, 2025 10:18

Merged devel

9d42959

Merge branch 'final-3d-fixes' into issue-1321

48c5c77

Merge pull request #1332 from PrincetonUniversity/issue-1321

4f74081

Issue 1321 - Displacement Test for 3D

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Final 3d fixes - Compiling and solving.... #1297

Final 3d fixes - Compiling and solving.... #1297

Uh oh!

lsawade commented Oct 16, 2025 •

edited

Loading

Uh oh!

Rohit-Kakodkar Oct 17, 2025

Uh oh!

Rohit-Kakodkar Oct 17, 2025

Uh oh!

Uh oh!

Rohit-Kakodkar Oct 17, 2025

Uh oh!

Rohit-Kakodkar left a comment

Uh oh!

Rohit-Kakodkar Oct 31, 2025

Uh oh!

Rohit-Kakodkar Oct 31, 2025

Uh oh!

Rohit-Kakodkar Oct 31, 2025

Uh oh!

Rohit-Kakodkar Oct 31, 2025

Uh oh!

Rohit-Kakodkar Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	// TODO: Implement dim3 boundaries - requires adapting to different mesh
	// TODO(Lucas : DIM3_BOUNDARIES_UPDATE): Implement dim3 boundaries - requires adapting to different mesh

Final 3d fixes - Compiling and solving.... #1297

Are you sure you want to change the base?

Final 3d fixes - Compiling and solving.... #1297

Uh oh!

Conversation

lsawade commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue Number

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rohit-Kakodkar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lsawade commented Oct 16, 2025 •

edited

Loading