SPECFEM++ 0.5.0
🚀 TL;DR
New Features
- Support for non-conforming 2D meshes through a discontinuous Galerkin approach
- 3D elastic isotropic simulation support
Improved Documentation
- Overhaul of API documentation
- Improved structure
- Includes the implemented equations
- Two new cookbooks
- Discontinuous Galerkin for fluid-solid wave propagation with bathymetry
- Wave propagation in the Marmousi model (adapted from SPECFEM2D)
CI
- Nightly benchmarks for homogeneous elastic and fluid-solid simulations are run on both CPU and GPU
- Review dashboard visualizing the nightly benchmarks and other CI information
Visualization
VTKHDF— Plotting in 3D is rather difficult since there are too many parameters to play with. Instead, we opted to support the writing ofVTKHDFfiles: a single HDF5 file that stores all timesteps. Both 2D and 3D simulations support this file format.- Automatically cropped 4K snapshots
- Blue-red color scheme for the output wavefields
📋 What’s New
Support for Non-Conforming Meshes
SPECFEM++ now supports non-conforming boundaries using a discontinuous Galerkin approach to the SEM. This work was contributed by Kentaro Hanson (@int-ptr-ptr).
The graphs below show results from two simulations of an acoustic-elastic layered domain meshed using:
- A conforming mesh (top left)
- A non-conforming mesh (top right)
The bottom graph compares traces from both simulations at a receiver within the acoustic domain. For more details, refer to the non-conforming cookbook.
Improved Documentation
This release simplifies the API documentation by organizing it into distinct sections, with each namespace clearly separated. We have also included technical implementation details wherever possible. For instance, the API documentation for stress computation now clearly explains how each component of the stress tensor is computed. Below is a snippet of the documentation for the stress computation call for an elastic isotropic medium.
Support for 3D Elastic Isotropic Simulation
In this release, we added support for 3D elastic isotropic simulations. We include the compilation of the Fortran internal mesher, and generate the GLL databases on the fly in SPECFEM++ for an improved data layout compared to loading the Fortran databases. For a tutorial, please refer to our 3D homogeneous elastic isotropic cookbook: 3D Elastic Isotropic Cookbook. Through integration testing, we make sure that the simulation matches SPECFEM3D Cartesian sample-by-sample. To visualize the 3D wavefield in Paraview, we also included the writing of VTKHDF files.
Marmousi model simulation
We included a cookbook that walks the user through simulating wave propagation in the Marmousi model.
SPECFEM2D implemented a high-resolution Marmousi example. The original velocity model for this example was provided by Yann Capdeville, and Hom Nath Gharti created a very high-resolution CUBIT mesh from it. We included the static CUBIT mesh to create a cookbook for wave propagation in the Marmousi model. To run this cookbook, a GPU compilation is strongly recommended, as the mesh is quite fine and the code doesn't yet support MPI.
Blue-red color scheme
Until now, we only supported magnitude plots for vector fields and absolute value for scalar fields. In particular, for pressure and rotation (Cosserat media), this felt limiting. For any scalar field, the default is now red-blue, while for vector fields, unless a component is specified, we still use magnitude by default. To pick a component, simply update the display section to include the component parameter.
display:
format: PNG
directory: path/to/output_dir/
field: displacement
component: x # y, z, or magnitude are other possible values
simulation-field: forward
time-interval: 5Nightly Benchmarks
We run nightly benchmarks for both serial CPU and GPU simulations. At this time, the CPU simulations are run on Intel Gold 6548Y+ chips and GPU simulations on Nvidia H100s. The benchmarks are published on a static review dashboard that is updated every morning. An example of the CPU runs — a snapshot of all runs up until Jan 14 — is shown below.
📊 Statistics
New Contributors
🪵 Changelog
👨🔬 New physics
- Implements stacey boundaries for Cosserat media by @maxlchien in #1004
- Adjacency2d features by @int-ptr-ptr in #1044
- Issue 1035 - 3D specialization for specfem/Jacobian by @icui in #1037
- Issue 946- assembly kernels and properties by @icui in #1101
- Issue 1120 - Added 3d elastic isotropic stress and associated unit-tests by @lsawade in #1143
- Issue 1121 - Adds 3D mass matrix computation and associated tests by @lsawade in #1144
- Issue 1122 - implements 3D elastic isotropic source by @lsawade in #1157
- Mortar Transfer Function by @int-ptr-ptr in #1227
- Nonconforming interfaces struct by @int-ptr-ptr in #1241
- edge->intersection compute_coupling by @int-ptr-ptr in #1325
🚀 Performance enhancement
- Reorders Kokkos views by @Rohit-Kakodkar in #1023
- Use references for 'const std::string' function arguments. by @bangerth in #1081
- gmshlayerbuilder NCI finding to use rtree by @int-ptr-ptr in #1181
- Optimize macros. by @icui in #1267
✚ Enhancement
- Issue 892 - Adding Intrinsic spin seismograms by @lsawade in #941
- Updated fortran mesher to write control nodes by @Rohit-Kakodkar in #994
- Issue 987 - Implements full compatibility with unity building. [Already reviewed!] by @lsawade in #1006
- Issue 966 - Template jacobian_matrix and element_types with dimension. by @icui in #1007
- Add 3D material definitions. by @icui in #1005
- 946 - Add dim3 value_containers. by @icui in #1002
- Add routines to compute 3D shape functions by @Rohit-Kakodkar in #1012
- Adds 3D mesh struct to assembly by @Rohit-Kakodkar in #999
- 3D implementation for Jacobian Matrix by @Rohit-Kakodkar in #1021
- Issue 985 - Add tags to 3D mesh by @icui in #1014
- Issue 993 - Add 3D specification for element_types by @icui in #1028
- Assembly fields implementation for 3D by @Rohit-Kakodkar in #1033
- Update Docker with HDF5 by @Rohit-Kakodkar in #1061
- Issue 990 Part 7- Implemented 3D force, moment tensor, readers and associated tests by @lsawade in #1041
- Issue 1057 - Implements locate_point for 3D by @lsawade in #1098
- Issue 981 - Implement 3D receivers by @lsawade in #1139
- Interpolate function for 3D by @Rohit-Kakodkar in #1119
- gmsh: topography file to external mesh files by @int-ptr-ptr in #1148
- Update locate point using adjacency map by @Rohit-Kakodkar in #1137
- Adjacency map implementation by @Rohit-Kakodkar in #1116
- 1092 npy - Add npy support as the first step of supporting npz by @icui in #1125
- Edge-based locate point by @int-ptr-ptr in #1156
- Updated Deprecate and TODO macros by @Rohit-Kakodkar in #1176
- 1092 - avoid creating empty folders in wavefield io for medium that do not exist by @icui in #1161
- Refactor
FOR_EACHmacro to incorporate interface tags by @Rohit-Kakodkar in #1166 - 1092 npz implementation by @icui in #1182
- Issue 1187 - Implements specfem::mesh_entity::element by @lsawade in #1191
- Issue 1124 - Implements a 3D ChunkDomainIterator by @lsawade in #1192
- Issue 1092 - Add npy / npz support by @icui in #1207
- Issue 1211 - Implements 3D receiver IO and IO tests for receivers by @lsawade in #1215
- Iterators and Access functions for edges by @Rohit-Kakodkar in #1151
- Issue 984 - Implements basic compilable 3D assembly struct. by @lsawade in #1203
gmshlayerbuilderboundary conditions by @int-ptr-ptr in #1236- Issue 1169 - non-dimensionalize all kernels by @icui in #1235
- 1170 - refactor gradient / divergence and add 3D implementation by @icui in #1237
- Issue 1170 - Implement 3D algorithms by @icui in #1262
- Issue 1213 - Timescheme Update for dim3 by @lsawade in #1249
- MESHFEM3D mesh reader for control nodes and materials by @Rohit-Kakodkar in #1238
- Issue 1230/1231 - 3D specialization for domain_kernels and time_marching. by @icui in #1243
- Issue 1285 - Implements 2D VTKHDF writing. by @lsawade in #1286
- Issue 1216 - Towards vtk by @lsawade in #1296
- Issue 1275 - Implements 2D plotting in form of lagrange quads by @lsawade in #1288
- Issue 1299 - Implements GLL mappings by @Rohit-Kakodkar in #1306
- Accessor Draft for Nonconforming Interfaces by @int-ptr-ptr in #1301
- Jacbian matrix construction using MESHFEM3D Mesh by @Rohit-Kakodkar in #1317
- Issue 1289 - Implements 3D VTK writing. by @lsawade in #1303
- 1178 Part I - Remove write_adjacency_map flag by @icui in #1318
- Assembly mesh construction using MESHFEM3D mesh by @Rohit-Kakodkar in #1312
- Suppress
-Wreturn-local-addrin coupled interface by @int-ptr-ptr in #1327 - Issue 1334 - Construct assembly using MESHFEM3D mesh by @Rohit-Kakodkar in #1335
- Issue 1264 - Update quadrature by @icui in #1292
- Generalized CTAD construction guideline for ThreadMDIterator by @Rohit-Kakodkar in #1344
- 1178 - Generate specfempp database from snakemake and make Snakefile static by @icui in #1346
- Add
KOKKOS_PATHflag to CMake by @Rohit-Kakodkar in #1352 - Issue 1321 - Displacement Test for 3D by @lsawade in #1332
- Add force install options for ADIOS2 and Boost dependencies by @Copilot in #1341
- Add cmake flag to nc grid by @Rohit-Kakodkar in #1360
- Issue 1281 - Compute material properties using MESHFFEM3D mesh by @Rohit-Kakodkar in #1329
- NCI accessors: chunk_edge (normal, intersection_factor) and point (transfer_function) by @int-ptr-ptr in #1339
- Nonconforming Kernel by @int-ptr-ptr in #1343
- Issue 1219 - Implemented compute_source_array for tensor, as well as associated test. by @lsawade in #1354
- Make Meshfem3D database the default database for 3D by @Rohit-Kakodkar in #1364
- Updated the nightly benchmarks to include git info by @lsawade in #1367
- 1178 spin - Add seispie workflow for finite difference traces by @icui in #1365
- Final 3d Fixes Generate Databases by @lsawade in #1369
- Issue 1178 - Snakemake workflow for generating test data / traces by @icui in #1378
- Minor change to nc container constructor by @Rohit-Kakodkar in #1414
- Upgrades MESHFEM3D mesher input by @Rohit-Kakodkar in #1403
- Issue 1379 - Deprecate Fortran generate database by @Rohit-Kakodkar in #1402
- gmshlayerbuilder: allow solids to skip AFS mark by @int-ptr-ptr in #1426
- 1178 - check consistency between mesh coupled_interface and adjacency graph by @icui in #1430
- Issue 1307 - displacement test for moment tensor source by @icui in #1308
- Final 3d fixes - Compiling and solving.... by @lsawade in #1297
- Issue 1462 - Plotting now allows for +/- colormap by @lsawade in #1490
- Issue 1218 - implement Heaviside function & Update the documentation by @lsawade in #1498
- Nightly benchmarks update by @Rohit-Kakodkar in #1497
- Nightly Benchmarks - Adds fluid-solid interface by @Rohit-Kakodkar in #1516
- Added
flux_scheme_tagenum by @int-ptr-ptr in #1510 - NCI AccessorPack fixture draft by @int-ptr-ptr in #1528
📚 Documentation
- Updated the capabilities and the compiler table by @lsawade in #901
- Issue 899 - Add a fluid-solid-bathymetry example by @lsawade in #912
- Added parameter option for seismograms in the parameter doc, and fixe… by @lsawade in #974
- Issue 1068 - Adds migration cookbook by @lsawade in #1072
- fix example directory in docs by @tjhei in #1079
- fix links in readme by @tjhei in #1075
- Issue 1168 - Moving assembly documentation to api/specfem/assembly by @lsawade in #1180
- Issue 1131 - Implements better documentation for receivers and sources. by @lsawade in #1164
- Issue 1168 - Assembly sources receivers by @lsawade in #1185
- 1092 update documentation on npy / npz by @icui in #1196
- Updated the links in the readme and the docs by @lsawade in #1200
- Adds a documentation mode for Github Copilot Chat by @Rohit-Kakodkar in #1225
- Updated README.md badges by @lsawade in #1348
- Documentation update - getting started & presets by @Rohit-Kakodkar in #1404
- Documentation Update - Assembly, boundaries structs by @Rohit-Kakodkar in #1409
- Documentation update - MESHFEM by @Rohit-Kakodkar in #1405
- Documentation Update - Assembly sources by @Rohit-Kakodkar in #1412
- Documentation Update - Documentation Assembly jacobian matrix by @Rohit-Kakodkar in #1413
- Nonconforming Cookbook by @int-ptr-ptr in #1370
- Documentation Update - Assembly properties documentation by @Rohit-Kakodkar in #1433
- Documentation Update - Edge types and element types by @Rohit-Kakodkar in #1437
- Documentaiton update - Kernels by @Rohit-Kakodkar in #1447
- Documentation update - point fields. by @icui in #1449
- Nonconforming Cookbook Appendix by @int-ptr-ptr in #1445
- Remove namespace description from specfem::point documentation by @Copilot in #1485
- Documentation update - point boundary by @icui in #1456
- Documentation Update - Assembly mesh by @Rohit-Kakodkar in #1464
- Documentation Update - enumerations element by @Rohit-Kakodkar in #1465
- Documentation update - index by @icui in #1455
- Documentation update - conforming and nonconforming interfaces. by @icui in #1457
- Documentation Update - Multiple Enum classes by @Rohit-Kakodkar in #1466
- Documentation Update - parallel configuration by @Rohit-Kakodkar in #1477
- Documentation update - point field_derivative, source and stress by @icui in #1484
- Documentation update - point jacobian. by @icui in #1489
- Documentation Update - Final enumeration implementation by @Rohit-Kakodkar in #1467
- Documentation update - periodic tasks. by @icui in #1491
- Issue 1382 - Implement a 3D cookbook by @lsawade in #1492
- Documentation update for execution by @Rohit-Kakodkar in #1473
- Documentation Update - Kokkos kernels by @Rohit-Kakodkar in #1475
- Issue 1376 -Marmousi benchmark by @lsawade in #1515
- Issue 1388 - Documentation update - point properties by @icui in #1506
- Documentation Update - Chunk Element/Edge by @Rohit-Kakodkar in #1519
- Fixes STF titles in the documentation. by @lsawade in #1520
- Documentation update - Data access by @Rohit-Kakodkar in #1508
- Documentation Update - Parameter parser by @Rohit-Kakodkar in #1509
- Issue 1385 - Updated CI documentation by @lsawade in #1507
- Documentation update - SPECFEM++ mesh by @Rohit-Kakodkar in #1514
- Issue 1376 - Adds marmousi cookbook by @lsawade in #1518
- Issue 1513 - Updates the IO documentation by @lsawade in #1522
- Documentation Update - SPECFEM++ Datatypes by @Rohit-Kakodkar in #1523
- Documentation update - Algorithm by @icui in #1524
- Issue 1526 - Adds library link by @lsawade in #1530
- Issue 1480 - Deprecating specfem_mpi and adding documentation for core/specfem/mpi by @lsawade in #1531
- Documentation Update - Medium Materials by @Rohit-Kakodkar in #1534
- Issue 1529 - Add specfem/program documentation by @lsawade in #1533
- Documentaiton Update - Medium compute stress by @Rohit-Kakodkar in #1540
- Documentation update - jacobian by @icui in #1544
- Documentation Fix | Removes simd index by @Rohit-Kakodkar in #1551
- Removed supported media comments by @Rohit-Kakodkar in #1549
- Documentation Update - source, damping, cosserat by @Rohit-Kakodkar in #1543
- Documentation Update - Mass matrix & wavefield by @Rohit-Kakodkar in #1541
- Documentation Update - Properties/Kernels containers by @Rohit-Kakodkar in #1539
- Issue 1391 - Updates the documentation for timescheme by @lsawade in #1535
- Issue 1536 - Updates solver documentation and moves solver to core/ by @lsawade in #1538
- Repo File Fix by @lsawade in #1553
- Documentation update - shape function by @icui in #1545
- Issue 1384 - Small Meshfem Parameter Doc fixes by @lsawade in #1561
- Issue 1438 - Fixed parameter naming by @lsawade in #1546
- Issue 1390 - Implements Compute Source Array documentation by @lsawade in #1552
- Documentation update - Frechet derivatives by @Rohit-Kakodkar in #1559
- Documentation update - boundary_conditions by @icui in #1565
Fortran Meshfem2D/3D updates
- Nonconforming External Mesher and Database (2D) by @int-ptr-ptr in #1136
Refactor
- Issue 925 + 926: rename partial derivatives, ScalarPointViewType, VectorPointViewType by @icui in #934
- Issue 935 - Move compute to core/specfem/compute by @icui in #939
- Issue 940 - rename compute namespace to assembly by @icui in #942
- Adds template specialization for mesh by @Rohit-Kakodkar in #959
- Implements 2D specialization for fields by @Rohit-Kakodkar in #970
- Issue 961 - Templating sources and receivers by @lsawade in #964
- Adds dim2 template specialization for boundaries by @Rohit-Kakodkar in #972
- Issue 952 - Template assembly::kernels and assembly::properties with dimension by @icui in #962
- Adds 2D specialization for boundary values - take 2 by @Rohit-Kakodkar in #991
- Issue 975 Adds template specialization for coupled interfaces. by @lsawade in #977
- Refactor shape functions by @Rohit-Kakodkar in #1008
- Implements 2D assembly by @Rohit-Kakodkar in #992
- Refactor data access class by @Rohit-Kakodkar in #1019
- Issue 981 - Dimension templates the receiver class and moved it to core by @lsawade in #1010
- Issue 990 Part 2 - Update the way we compute the source vector and tensor by @lsawade in #1025
- Issue 990 Part 3 - Moved include,src/source to core/specfem/source by @lsawade in #1026
- Issue 990 Part 4 - Consolidating vector and tensor forces. by @lsawade in #1029
- Issue 990 Part 5 - Implementation of simplified tests for the vector and tensor sources by @lsawade in #1032
- Issue 1027 - rename free_surface to acoustic_free_surface by @icui in #1102
- Issue - 982 Added newer simpler versions of access functions for fields by @Rohit-Kakodkar in #1034
- Issue 990 -- Moves compute source array to assembly by @lsawade in #1003
- Create edge types to sort all edges with tags by @Rohit-Kakodkar in #1179
- Reimplement coupled interfaces using macros by @Rohit-Kakodkar in #1189
- Implements new kokkos kernel to compute coupling by @Rohit-Kakodkar in #1194
- Update compute coupling calls by @Rohit-Kakodkar in #1201
- Issue 1216 - Part 1 Moving the periodic tasks to core by @lsawade in #1252
- Issue 1245 - Moves simulation field data access functions by @lsawade in #1284
- Chunk -> ChunkElement and added ChunkEdge Accessing by @int-ptr-ptr in #1273
- 1264 Part II - Remove hprime_wgll by @icui in #1293
- 1264 Part II - Move element::quadrature to quadrature::lagrange_derivative by @icui in #1300
- Issue 1302 - Implement point weights by @icui in #1305
- Refactors all macros into single library by @Rohit-Kakodkar in #1357
- 1178 - Remove assemble_legacy by @icui in #1374
- 1178 - Construct assembly::coupled_interfaces from adjacency_graph by @icui in #1422
- Refactor non-conforming interfaces by @Rohit-Kakodkar in #1439
- specfem::Context struct and ContextGuard for initialization of kokkos, mpi, and handling initialization from python side etc. by @lsawade in #1149
- 1392 - Documentation Update - Macros by @icui in #1511
🐛 Bug Fixes
- Assembly 2D build - vtk build, fix by @lsawade in #1015
- fix bug in gmsh2meshfem for conforming, + new mesh by @int-ptr-ptr in #1260
- Adds Configure tests flag to cmake by @Rohit-Kakodkar in #1345
- Fix snakemake params in benchmarks folder by @icui in #1349
- Updated the Jenkins workflow to install in separate directories. by @lsawade in #1355
- Adds a Unity build batch size flag by @Rohit-Kakodkar in #1356
- Issue 1062 - Fixes coupled interfaces implementation by @Rohit-Kakodkar in #1377
- Address feedback on moving point kernel implementation to medium by @Copilot in #1487
- Issue 1503 - Fixes a couple of issues related to logger output by @lsawade in #1504
- Bug fix for transfer function tests by @Rohit-Kakodkar in #1583
Configuration/CMake
- Issue 861 - Updating the tests to be able to run the executable from the test directory by @lsawade in #956
- Issue 924 - Updated Jenkins tests to run in parallel and only produce output on failure by @lsawade in #960
- Feature unity build by @lsawade in #988
- Specfem prefix by @lsawade in #1085
- Putting the finding and/or installation of dependencies into a separate folder "cmake" for a cleaner Cmakelists.txt by @lsawade in #1083
- Fixes cmake issues with boost etc. by @lsawade in #1190
🔧 Maintenance
- Remove test.f90 by @Rohit-Kakodkar in #1040
- Deprecate Devel DG branch by @Rohit-Kakodkar in #1188
- Issue 824 - Nightly benchmarks snakemake workflows by @Rohit-Kakodkar in #1217
- Pin Python version to 3.12 by @icui in #1461
- Remove deprecated implementations by @Rohit-Kakodkar in #1478
🧾 New tests
- Added unit tests for stress computation by @Rohit-Kakodkar in #944
- Issue 1099 - Implements Tests for compute_jacobian/compute_locations by @lsawade in #1100
- Updated the 2D tests for locate point by @lsawade in #1113
- Issue 981 - Assembly receivers tests by @lsawade in #1142
- Update Jenkins runner to SPECFEMPP runner by @Rohit-Kakodkar in #1152
- Assembly-Agnostic
locate_point_on_edge-- algorithms test by @int-ptr-ptr in #1204 - Issue 1214 - Implements 3D Assembly fixture and stand-in displacement tests by @lsawade in #1234
- Helpers for tests: field load_on_host + coupled_interface to_string() by @int-ptr-ptr in #1255
- New Nonconforming Test Mesh + migration by @int-ptr-ptr in #1326
- Issue 1319 - Add tests for 2D/3D gradient computation by @Rohit-Kakodkar in #1320
- 1178 - Update snakefile for all database in test data by @icui in #1358
- 1178 final - finalize displacement tests by @icui in #1359
- Add provenance to AcousticElasticNonconformingCoupledDomain by @int-ptr-ptr in #1406
- Issue 1373 - Adds 3D Snakemake workflow to recreate the traces. by @lsawade in #1452
- Nonconforming transfer algorithm test by @Rohit-Kakodkar in #1468
- migrated fixtures, added power tests by @int-ptr-ptr in #1481
- Test Fixtures: Split Description -> Description + Name by @int-ptr-ptr in #1542
- Issue 1500 - Adds displacement tests for a moment tensor + heaviside combination by @lsawade in #1547
Other Changes
- Fixes allocation of memory for boundary values to when needed by @Rohit-Kakodkar in #1030
- Update wavefield reader / writer by @Rohit-Kakodkar in #1038
- Fix ASCII Reader/Writer by @Rohit-Kakodkar in #1059
- Fixed simulation field compilation by @Rohit-Kakodkar in #1055
- Bug Fix in simulation field by @Rohit-Kakodkar in #1060
- Update docker to be no SIMD build by @Rohit-Kakodkar in #1073
- Fix the couple stress computation. Error in Jacobian inverse. by @lsawade in #1024
- Issue 1159 - Fixes docker building by @lsawade in #1160
- Issue 945 - Updates the STF and force implementations to match SPECFEM2D Fortran by @lsawade in #1146
- [Fix] Performance issues with adjacency matrix implementation by @Rohit-Kakodkar in #1254
- Update compute coupling instantiation by @Rohit-Kakodkar in #1271
- minor bugfixes in devel from dg2d-kernelimpl by @int-ptr-ptr in #1290
- Final bug fixes for 3D simulation by @lsawade in #1324
- Issue 1336 - Bug fix for 3D iteration policies by @Rohit-Kakodkar in #1338
- Fix bug in coupled interfaces by @Rohit-Kakodkar in #1351
- [Fix] Fixes meshfem2D for fluid solid coupling by @Rohit-Kakodkar in #1423
- Dimension-Unify Wavefield Helper by @lsawade in #1451
- Issue 1503 - followup: fixing benchmarks for specfempp update. by @lsawade in #1505
- Regex Fix by @lsawade in #1582
- Updated the docker badges by @lsawade in #1586
- Release Candidate 0.5.0 by @lsawade in #1585
Full Changelog: v0.4.0...v0.5.0


