Skip to content

Potential indexing/ghost-validity issue: add_sdc_source_to_states() reads neighbor cells (i-1/j-1/k-1) without local ghost-fill #3301

@zingale

Description

@zingale

Summary

Castro::add_sdc_source_to_states() applies SDC reaction source terms to face states on nodal boxes and, for the left state, explicitly reads from the adjacent cell (i-1, j-1, or k-1 depending on direction). This is an indexing pattern that relies on ghost values of sdc_src being valid at every grid/tile boundary.

In construct_ctu_hydro_source(), this neighbor-read path is used, but there is no local FillBoundary/FillPatch immediately before these accesses. The correctness therefore depends on upstream code having already filled and synchronized Simplified_SDC_React_Type ghost zones.

Evidence

In Source/hydro/Castro_ctu.cpp:

if (idir == 0) {
    qleft(...,QPRES) += 0.5 * dt * sdc_src(i-1,j,k,QPRES);
    qleft(...,QREINT) += 0.5 * dt * sdc_src(i-1,j,k,QREINT);
} else if (idir == 1) {
    qleft(...,QPRES) += 0.5 * dt * sdc_src(i,j-1,k,QPRES);
    ...
} else {
    qleft(...,QPRES) += 0.5 * dt * sdc_src(i,j,k-1,QPRES);
    ...
}

In Source/hydro/Castro_ctu_hydro.cpp, this function is called on nodal face boxes (xbx/ybx/zbx) with sdc_src_arr = SDC_react_source.array(mfi), but there is no immediate local ghost fill of SDC_react_source in this routine.

Simplified_SDC_React_Type is defined with only one ghost cell in setup (addDescriptor(..., 1, NQ, ...)), so correctness is especially sensitive to whether that one layer is current and valid before hydro uses it.

Why this is risky

This can create boundary/grid-layout-dependent behavior:

  • On interior faces near a Fab boundary, i-1/j-1/k-1 may land in ghost cells.
  • If ghosts are stale/not filled for the current time, advection states get inconsistent source contributions.
  • Problems may appear only with certain decomposition/tiling layouts, making debugging difficult.

Suggested fix

Before calling add_sdc_source_to_states(), enforce a local ghost synchronization for SDC_react_source at the current time (e.g., FillBoundary/FillPatch as appropriate), or explicitly document/invariant-check that this has already been done.

Optionally, add debug assertions (in non-optimized builds) that the required neighbor region is contained in the valid+ghost box for sdc_src.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions