-
Notifications
You must be signed in to change notification settings - Fork 106
Description
Summary
Castro::add_sdc_source_to_states() applies SDC reaction source terms to face states on nodal boxes and, for the left state, explicitly reads from the adjacent cell (i-1, j-1, or k-1 depending on direction). This is an indexing pattern that relies on ghost values of sdc_src being valid at every grid/tile boundary.
In construct_ctu_hydro_source(), this neighbor-read path is used, but there is no local FillBoundary/FillPatch immediately before these accesses. The correctness therefore depends on upstream code having already filled and synchronized Simplified_SDC_React_Type ghost zones.
Evidence
In Source/hydro/Castro_ctu.cpp:
if (idir == 0) {
qleft(...,QPRES) += 0.5 * dt * sdc_src(i-1,j,k,QPRES);
qleft(...,QREINT) += 0.5 * dt * sdc_src(i-1,j,k,QREINT);
} else if (idir == 1) {
qleft(...,QPRES) += 0.5 * dt * sdc_src(i,j-1,k,QPRES);
...
} else {
qleft(...,QPRES) += 0.5 * dt * sdc_src(i,j,k-1,QPRES);
...
}In Source/hydro/Castro_ctu_hydro.cpp, this function is called on nodal face boxes (xbx/ybx/zbx) with sdc_src_arr = SDC_react_source.array(mfi), but there is no immediate local ghost fill of SDC_react_source in this routine.
Simplified_SDC_React_Type is defined with only one ghost cell in setup (addDescriptor(..., 1, NQ, ...)), so correctness is especially sensitive to whether that one layer is current and valid before hydro uses it.
Why this is risky
This can create boundary/grid-layout-dependent behavior:
- On interior faces near a Fab boundary,
i-1/j-1/k-1may land in ghost cells. - If ghosts are stale/not filled for the current time, advection states get inconsistent source contributions.
- Problems may appear only with certain decomposition/tiling layouts, making debugging difficult.
Suggested fix
Before calling add_sdc_source_to_states(), enforce a local ghost synchronization for SDC_react_source at the current time (e.g., FillBoundary/FillPatch as appropriate), or explicitly document/invariant-check that this has already been done.
Optionally, add debug assertions (in non-optimized builds) that the required neighbor region is contained in the valid+ghost box for sdc_src.