SUNDIALS minor release v5.1.0

Changes to SUNDIALS in release 5.1.0

Fixed a build system bug related to finding LAPACK/BLAS.

Fixed a build system bug related to checking if the KLU library works.

Fixed a build system bug related to finding PETSc when using the CMake
variables PETSC_INCLUDES and PETSC_LIBRARIES instead of PETSC_DIR.

Added a new build system option, CUDA_ARCH, to specify the CUDA architecture to compile for.

Fixed a bug in the Fortran 2003 interfaces to the ARKode Butcher table routines and structure.
This includes changing the ARKodeButcherTable type to be a type(c_ptr) in Fortran.

Added two utility functions, SUNDIALSFileOpen and SUNDIALSFileClose for creating/destroying
file pointers. These are useful when using the Fortran 2003 interfaces.

Added support for a user-supplied function to update the prediction for each
implicit stage solution in ARKStep. If supplied, this routine will be called
after any existing ARKStep predictor algorithm completes, so that the
predictor may be modified by the user as desired. The new user-supplied routine
has type ARKStepStagePredictFn, and may be set by calling ARKStepSetStagePredictFn.

The MRIStep module has been updated to support attaching different user data
pointers to the inner and outer integrators. If applicable, user codes will
need to add a call to ARKStepSetUserData to attach their user data
pointer to the inner integrator memory as MRIStepSetUserData will
not set the pointer for both the inner and outer integrators. The MRIStep
examples have been updated to reflect this change.

Added support for damping when using Anderson acceleration in KINSOL. See the
mathematical considerations section of the user guide and the description of the
KINSetDampingAA function for more details.

Added support for damping to the SUNNonlinearSolver_FixedPoint module when
using Anderson acceleration. See the SUNNonlinearSolver_FixedPoint section in
the user guides and the description of the SUNNonlinSolSetDamping_FixedPoint
function for more details.

SUNDIALS major release v5.0.0

Changes to SUNDIALS in release 5.0.0

Build System

Increased the minimum required CMake version to 3.5 for most SUNDIALS
configurations, and 3.10 when CUDA or OpenMP with device offloading are enabled.

The CMake option BLAS_ENABLE and the variable BLAS_LIBRARIES have been
removed to simplify builds as SUNDIALS packages do not use BLAS directly. For
third party libraries that require linking to BLAS, the path to the BLAS
library should be included in the _LIBRARIES variable for the third party
library e.g., SUPERLUDIST_LIBRARIES when enabling SuperLU_DIST.

Fixed a bug in the build system that prevented the PThreads NVECTOR module from
being built.

NVector

Two new functions were added to aid in creating custom NVECTOR objects. The
constructor N_VNewEmpty() allocates an ``empty'' generic NVECTOR with the
object's content pointer and the function pointers in the operations structure
initialized to NULL. When used in the constructor for custom objects this function
will ease the introduction of any new optional operations to the NVECTOR API by
ensuring only required operations need to be set. Additionally, the function
N_VCopyOps(w, v) has been added to copy the operation function pointers between
vector objects. When used in clone routines for custom vector objects these
functions also will ease the introduction of any new optional operations to the
NVECTOR API by ensuring all operations are copied when cloning objects.

Two new N_Vector implementations, NVECTOR_MANYVECTOR and NVECTOR_MPIMANYVECTOR,
have been created to support flexible partitioning of solution data among
different processing elements (e.g., CPU + GPU) or for multi-physics problems
that couple distinct MPI-based simulations together (see the NVECTOR_MANYVECTOR
and NVECTOR_MPIMANYVECTOR sections in the user guides for more details). This
implementation is accompanied by additions to user documentation and SUNDIALS
examples.

An additional NVECTOR implementation, NVECTOR_MPIPLUSX, has been created to
support the MPI+X paradigm where X is a type of on-node parallelism (e.g.,
OpenMP, CUDA). The implementation is accompanied by additions to user
documentation and SUNDIALS examples.

One new required vector operation and ten new optional vector operations have
been added to the NVECTOR API. The new required operation, N_VGetLength, returns
the global length of an N_Vector. The optional operations have been added to
support the new NVECTOR_MPIMANYVECTOR implementation. The operation
N_VGetCommunicator must be implemented by subvectors that are combined to create
an NVECTOR_MPIMANYVECTOR, but is not used outside of this context. The
remaining nine operations are optional local reduction operations intended to
eliminate unnecessary latency when performing vector reduction operations
(norms, etc.) on distributed memory systems. The optional local reduction vector
operations are N_VDotProdLocal, N_VMaxNormLocal, N_VMinLocal, N_VL1NormLocal,
N_VWSqrSumLocal, N_VWSqrSumMaskLocal, N_VInvTestLocal, N_VConstrMaskLocal, and
N_VMinQuotientLocal. If an NVECTOR implementation defines any of the local
operations as NULL, then the NVECTOR_MPIMANYVECTOR will call standard NVECTOR
operations to complete the computation.

The *_MPICuda and *_MPIRaja functions have been removed from the NVECTOR_CUDA
and NVECTOR_RAJA implementations respectively. Accordingly, the
nvector_mpicuda.h, nvector_mpiraja.h, libsundials_nvecmpicuda.lib, and
libsundials_nvecmpicudaraja.lib files have been removed. Users should use the
NVECTOR_MPIPLUSX module in conjunction with the NVECTOR_CUDA or NVECTOR_RAJA
modules to replace the functionality. The necessary changes are minimal and
should require few code modifications.

Fixed a memory leak in the NVECTOR_PETSC clone function.

Made performance improvements to the CUDA NVECTOR. Users who utilize a non
-default stream should no longer see default stream synchronizations after
memory transfers.

Added a new constructor to the CUDA NVECTOR that allows a user to provide
custom allocate and free functions for the vector data array and internal
reduction buffer.

Added new Fortran 2003 interfaces for most NVECTOR modules. See NEVTOR section
in the user guides for more details on how to use the interfaces.

Added three new NVECTOR utility functions, FN_VGetVecAtIndexVectorArray,
FN_VSetVecAtIndexVectorArray, and FN\_VNewVectorArray, for working with
N_Vector arrays when using the Fortran 2003 interfaces.

SUNMatrix

Two new functions were added to aid in creating custom SUNMATRIX objects. The
constructor SUNMatNewEmpty() allocates an ``empty'' generic SUNMATRIX with the
object's content pointer and the function pointers in the operations structure
initialized to NULL. When used in the constructor for custom objects this function
will ease the introduction of any new optional operations to the SUNMATRIX API by
ensuring only required operations need to be set. Additionally, the function
SUNMatCopyOps(A, B) has been added to copy the operation function pointers between
matrix objects. When used in clone routines for custom matrix objects these
functions also will ease the introduction of any new optional operations to the
SUNMATRIX API by ensuring all operations are copied when cloning objects.

A new operation, SUNMatMatvecSetup, was added to the SUNMatrix API. Users
who have implemented custom SUNMatrix modules will need to at least update
their code to set the corresponding ops structure member, matvecsetup, to NULL.

The generic SUNMatrix API now defines error codes to be returned by SUNMatrix operations.
Operations which return an integer flag indiciating success/failure may return different
values than previously.

A new SUNMatrix (and SUNLinearSolver) implementation was added to facilitate
the use of the SuperLU_DIST library with SUNDIALS.

Added new Fortran 2003 interfaces for most SUNMATRIX modules. See SUNMATRIX
section in the user guides for more details on how to use the interfaces.

SUNLinearSolver

A new function was added to aid in creating custom SUNLINEARSOLVER objects. The
constructor SUNLinSolNewEmpty() allocates an ``empty'' generic SUNLINEARSOLVER
with the object's content pointer and the function pointers in the operations
structure initialized to NULL. When used in the constructor for custom objects
this function will ease the introduction of any new optional operations to the
SUNLINEARSOLVER API by ensuring only required operations need to be set.

The return type of the SUNLinearSolver API function SUNLinSolLastFlag has
changed from long int to sunindextype to be consistent with the type
used to store row indices in dense and banded linear solver modules.

Added a new optional operation to the SUNLINEARSOLVER API, SUNLinSolGetID,
that returns a SUNLinearSolver_ID for identifying the linear solver module.

The SUNLinearSolver API has been updated to make the initialize and setup
functions optional.

A new SUNLinearSolver (and SUNMatrix) implementation was added to facilitate
the use of the SuperLU_DIST library with SUNDIALS.

Added a new SUNLinearSolver implementation, SUNLinearSolver_cuSolverSp_batchQR,
which leverages the NVIDIA cuSOLVER sparse batched QR method for efficiently
solving block diagonal linear systems on NVIDIA GPUs.

Added three new accessor functions to the SUNLinSol_KLU module,
SUNLinSol_KLUGetSymbolic(), SUNLinSol_KLUGetNumeric(), and
SUNLinSol_KLUGetCommon(), to provide user access to the underlying
KLU solver structures.

Added new Fortran 2003 interfaces for most SUNLINEARSOLVER modules. See
SUNLINEARSOLVER section in the user guides for more details on how to use
the interfaces.

SUNNonlinearSolver

A new function was added to aid in creating custom SUNNONLINEARSOLVER objects.
The constructor SUNNonlinSolNewEmpty() allocates an ``empty'' generic
SUNNONLINEARSOLVER with the object's content pointer and the function pointers
in the operations structure initialized to NULL. When used in the constructor
for custom objects this function will ease the introduction of any new optional
operations to the SUNNONLINEARSOLVER API by ensuring only required operations
need to be set.

To facilitate the use of user supplied nonlinear solver convergence test
functions the SUNNonlinSolSetConvTestFn function in the SUNNonlinearSolver API
has been updated to take a void* data pointer as input. The supplied data
pointer will be passed to the nonlinear solver convergence test function on each
call.

The inputs values passed to the first two inputs of the SUNNonlinSolSolve
function in the SUNNONLINEARSOLVER have been changed to be the predicted
state and the initial guess for the correction to that state. Additionally,
the definitions of SUNNonlinSolLSetupFn and SUNNonlinSolLSolveFn in the
SUNNonlinearSolver API have been updated to remove unused input parameters.
For more information on the nonlinear system formulation and the API functions
see the SUNNONLINEARSOLVER chapter in the user guides.

Added a new SUNNonlinearSolver implementation, SUNNonlinsol_PetscSNES,
which interfaces to the PETSc SNES nonlinear solver API.

Added new Fortran 2003 interfaces for most SUNNONLINEARSOLVER modules. See
SUNNONLINEARSOLVER section in the user guides for more details on how to use
the interfaces.

CVODE and CVODES

Fixed a bug in the CVODE and CVODES constraint handling where the step size
could be set below the minimum step size.

Fixed a bug in the CVODE and CVODES nonlinear solver interfaces where the norm
of the accumulated correction was not updated when using a non-default
convergence test function.

Fixed a bug in the CVODES cvRescale function where the loops to compute the
array of scalars for the fused vector scale operation stopped one iteration
early.

Fixed a...

SUNDIALS development release v5.0.0-dev.2

Changes to SUNDIALS in release 5.0.0-dev.2

Added a new optional operation to the SUNLINEARSOLVER API, SUNLinSolGetID,
that returns a SUNLinearSolver_ID for identifying the linear solver module.

Made performance improvements to the CUDA NVECTOR. Users who utilize a non
-default stream should no longer see default stream synchronizations after
memory transfers.

Added a new constructor to the CUDA NVECTOR that allows a user to provide
custom allocate and free functions for the vector data array and internal
reduction buffer.

Increased the minimum required CMake version to 3.5 for most SUNDIALS configurations,
and 3.10 when CUDA or OpenMP with device offloading are enabled.

The CMake option BLAS_ENABLE and the variable BLAS_LIBRARIES have been
removed to simplify builds as SUNDIALS packages do not use BLAS directly. For
third party libraries that require linking to BLAS, the path to the BLAS
library should be included in the _LIBRARIES variable for the third party
library e.g., SUPERLUDIST_LIBRARIES when enabling SuperLU_DIST.

Fixed a bug in the ARKStep time-stepping module in ARKode that would result in an
infinite loop if the nonlinear solver failed to converge more than the maximum
allowed times during a single step.

Added functions to get the current state and gamma value to CVODE(S) and ARKode.
These functions may be useful to users who chose to provide their own nonlinear
solver implementation.

Add two new 'Set' functions to MRIStep, MRIStepSetPreInnerFn() and
MRIStepSetPostInnerFn() for performing communication or memory
transfers needed before or after the inner integration.

The reinitialization functions ERKStepReInit(), ARKStepReInit(), and
MRIStepReInit() have been updated to retain the minimum and maxiumum step
size values from before reinitialization rather than resetting them to the
default values.

Added a new SUNLinearSolver implementation, SUNLinearSolver_cuSolverSp_batchQR,
which leverages the NVIDIA cuSOLVER sparse batched QR method for efficiently
solving block diagonal linear systems on NVIDIA GPUs.

Added three new accessor functions to the SUNLinSol_KLU module,
SUNLinSol_KLUGetSymbolic(), SUNLinSol_KLUGetNumeric(), and
SUNLinSol_KLUGetCommon(), to provide user access to the underlying
KLU solver structures.

A bug was fixed in the IDA(S) linear solver interface where an incorrect
Jacobian-vector product increment was used with iterative solvers other than
SPGMR and SPFGMR.

Fixed a bug in the CVODE(S) nonlinear solver interface where the norm of the
accumulated correction was not updated when using a non-default convergence
test function.

SUNDIALS v5.0.0-dev.1

Summary

Our latest release, v5.0.0-dev.1, is a beta release of the next major SUNDIALS version and offers a preview of what is to come in SUNDIALS. The release includes new Fortran 2003 interface modules that provide Fortran users access to most of the SUNDIALS C API including the CVODE, ARKode, IDA, and KINSOL packages. The release also includes new linear solver interface functions, ARKLsLinSysFn and CVLsLinSysFn, which can be used as alternative methods for evaluating matrices for the linear systems M - gamma J and I - gamma J respectively, a new N_Vector implementation, NVECTOR_MPIPLUSX, which supports the MPI+X paradigm where X is a type of on-node parallelism such as CUDA, and several other improvements and bug fixes. We do not foresee any major API changes occurring between this release and v5.0.0, but various minor improvements are planned.

List of Changes

Several new functions were added to aid in creating custom NVECTOR,
SUNMATRIX, SUNLINEARSOLVER, and SUNNONLINEARSOLVER objects. The constructors
N_VNewEmpty(), SUNMatNewEmpty(), SUNLinSolNewEmpty(), and SUNNonlinSolNewEmpty()
allocate ``empty'' generic NVECTOR, SUNMATRIX, SUNLINEARSOLVER, and
SUNNONLINEARSOLVER objects respectively with the object's content pointer and
the function pointers in the operations structure initialized to NULL. When used
in the constructor for custom objects these functions will ease the introduction
of any new optional operations to the NVECTOR, SUNMATRIX, SUNLINEARSOLVER, or
SUNNONLINEARSOLVER APIs by ensuring only required operations need to be set.
Additionally, the functions N_VCopyOps(w, v) and SUNMatCopyOps(A, B) have been
added to copy the operation function pointers between vector and matrix objects
respectively. When used in clone routines for custom vector and matrix objects
these functions also will ease the introduction of any new optional operations
to the NVECTOR or SUNMATRIX APIs by ensuring all operations are copied when
cloning objects.

New linear solver interface functions ARKLsLinSysFn and CVLsLinSysFn were added
as alternative methods for evaluating the linear systems M - gamma J and
I - gamma J respectively.

Fixed a bug in the build system that prevented the PThreads NVECTOR module from
being built.

Fixed a bug in ARKStep where the mass matrix linear solver setup function was
not called in the Matrix-free case.

Fixed a minor bug in ARKStep where an incorrect flag is reported when an
error occurs in the mass matrix setup or Jacobian-vector product setup
functions.

Fixed a memory leak in the NVECTOR_PETSC clone function.

Fixed a memeory leaks in FARKODE, FCVODE, and FIDA when not using the default
nonlinear solver.

Fixed a bug in IDAQuadReInitB where an incorrect memory structure was passed to
IDAQuadReInit.

Added two new embedded ARK methods of orders 4 and 5 to ARKode (from
Kennedy & Carpenter, Appl. Numer. Math., 136:183--205, 2019).

The ManyVector NVECTOR module has been split into two versions: one
that requires MPI (MPIManyVector) and another that does not use
MPI at all (ManyVector). The associated example problems have
been similarly updated to reflect this new structure.

An additional NVECTOR implementation, NVECTOR_MPIPLUSX, was created to support
the MPI+X paradigm where X is a type of on-node parallelism (e.g. OpenMP, CUDA).
The implementation is accompanied by additions to user documentation and SUNDIALS
examples.

The *_MPICuda and *_MPIRaja functions were removed from the NVECTOR_CUDA and
NVECTOR_RAJA implementations respectively. Accordingly, the nvector_mpicuda.h,
nvector_mpiraja.h, libsundials_nvecmpicuda.lib, and libsundials_nvecmpicudaraja.lib
files have been removed. Users should use the NVECTOR_MPIPLUSX module coupled with
the NVECTOR_CUDA or NVECTOR_RAJA to replace the functionality. The necessary changes
are minimal and should require few code modifications.

New Fortran 2003 interfaces to ARKODE, IDA, KINSOL, all generic SUNDIALS types
(i.e. NVECTOR, SUNMATRIX, SUNLINEARSOLVER, SUNNONLINEARSOLVER), and the
NVECTOR_PARALLEL were added. These new interfaces were generated with SWIG-Fortran
and provide a user an idiomatic Fortran 2003 interface to most of the SUNDIALS C API.
The CVODE interface, and all module implementations with existing Fortran 2003 interfaces
were updated accordingly. See the section "Using for Fortran Applications" in
the appropriate user guide for more details on how to use the interfaces.

Removed extraneous calls to N_VMin() for simulations where
the scalar valued absolute tolerance, or all entries of the
vector-valued absolute tolerance array, are strictly positive. In
this scenario, CVODE(S), IDA(S) and ARKode will remove at least one
global reduction per time step.

v5.0.0-dev.0

Changes to SUNDIALS in release 5.0.0-dev.0

An additional N_Vector implementation, NVECTOR_MANYVECTOR, was
created to support flexible partitioning of solution data among
different processing elements (e.g., CPU + GPU) or for multi-physics
problems that couple distinct MPI-based simulations together (see
the NVECTOR_MANYVECTOR section in the user guides for more details).
This implementation is accompanied by additions to user documentation
and SUNDIALS examples.

Eleven new optional vector operations have been added to the N_Vector API to
support the new NVECTOR_MANYVECTOR implementation (see N_Vector chapter is the
user guides for more details). Two of the operations, N_VGetCommunicator and
N_VGetLength, must be implemented by subvectors that are combined to create an
NVECTOR_MANYVECTOR, but are not used outside of this context. The remaining nine
operations are optional local reduction operations intended to eliminate
unnecessary latency when performing vector reduction operations (norms, etc.) on
distributed memory systems. The optional local reduction vector operations are
N_VDotProdLocal, N_VMaxNormLocal, N_VMinLocal, N_VL1NormLocal, N_VWSqrSumLocal,
N_VWSqrSumMaskLocal, N_VInvTestLocal, N_VConstrMaskLocal, and
N_VMinQuotientLocal. If an N_Vector implementation defines any of the local
operations as NULL, then the NVECTOR_MANYVECTOR will call standard N_Vector
operations to complete the computation.

Additionally this release includes the following changes:

• A new SUNMatrix and SUNLinearSolver implementation was added to facilitate
  the use of the SuperLU_DIST library with SUNDIALS.

• A new operation, SUNMatMatvecSetup, was added to the SUNMatrix API. Users
  who have implemented custom SUNMatrix modules will need to at least update
  their code to set the corresponding ops structure member, matvecsetup, to NULL.

• The generic SUNMatrix API now defines error codes to be returned by SUNMatrix operations.
  Operations which return an integer flag indiciating success/failure may return different
  values than previously.

• The MRIStep module has been updated to support explicit, implicit, or IMEX
  methods as the fast integrator using the ARKStep module. As a result some
  function signatures have been changed including MRIStepCreate which now
  takes an ARKStep memory structure for the fast integration as an input.

v4.1.0

Changes in minor release SUNDIALS v.4.1.0

An additional N_Vector implementation was added for Tpetra vector from Trilinos library to facilitate interoperability between SUNDIALS and Trilinos. This implementation is accompanied by additions to user documentation and SUNDIALS examples.

A bug was fixed where a nonlinear solver object could be freed twice in some use cases.

The EXAMPLES_ENABLE_RAJA CMake option has been removed. The optionEXAMPLES_ENABLE_CUDA enables all examples that use CUDA including the RAJA examples with a CUDA back end (if the RAJA NVECTOR is enabled).

The implementation header files (e.g. arkode_impl.h) are no longer installed. This means users who are directly manipulating package memory structures will need to update their code to use the package’s public API.

Python is no longer required to run make test and make test_install.

Fixed a bug in ARKodeButcherTable_Write when printing a Butcher table without an embedding.

v4.0.2

Changes in patch release SUNDIALS v.4.0.2

Added information about how to contribute to SUNDIALS as well as a contributor agreement.

Moved definitions of DLS and SPILS backwards compatibility functions to a source file. The symbols are now included in the appropriate package library, e.g. libsundials_cvode.lib.

v4.0.1

Changes in patch release SUNDIALS v.4.0.1

A bug in ARKode where single precision builds would fail to compile has been fixed.

v4.0.0

Changes in major release SUNDIALS v.4.0.0

The direct and iterative linear solver interfaces in all SUNDIALS packages have been merged into unified linear solver interfaces for each package that support any valid SUNLINSOL module. This includes the previous DIRECT and ITERATIVE types and the new MATRIX_ITERATIVE type. Details regarding how SUNDIALS packages utilize linear solvers of each type as well as discussion regarding intended use cases for user-supplied SUNLINSOL implementations are included in the SUNLINSOL chapter of the user guides.

The unified interfaces are very similar to the previous DLS and SPILS interfaces. To minimize challenges in user migration to the unified linear solver interfaces, the previous DLS and SPILS routines for all packages may still be used; these will be deprecated in future releases, so we recommend that users migrate to the new names soon. Additionally, we note that Fortran users will need to enlarge their iout array of optional integer outputs, and update the indices that they query for certain linear-solver-related statistics.

The names of all constructor routines for SUNDIALS-provided SUNLinSol implementations have been updated to follow the naming convention SUNLinSol_* where * is the name of the linear solver e.g., Dense, KLU, SPGMR, PCG, etc. Solver-specific “set” routine names have been similarly standardized. To minimize challenges in user migration to the new names, the previous routine names may still be used; these will be deprecated in future releases, so we recommend that users migrate to the new names soon.

SUNDIALS integrators (ARKode, CVODE, CVODES, IDA, and IDAS) have been updated to utilize generic nonlinear solver modules through the SUNNONLINSOL API. This API will ease the addition of new nonlinear solver options and allow for external or user-supplied nonlinear solvers. The SUNNONLINSOL API and provided SUNNONLINSOL modules are described in a new user guide chapter and follow the same object oriented design and implementation used by the NVECTOR, SUNMATRIX, and SUNLINSOL modules.

With the introduction of SUNNonlinSol modules, the input parameter iter to CVodeCreate has been removed along with the function CVodeSetIterType and the constants CV_NEWTON and CV_FUNCTIONAL. Similarly, the ITMETH parameter has been removed from the Fortran interface function FCVMALLOC. Instead of specifying the nonlinear iteration type when creating the CVODE(S) memory structure, CVODE(S) uses the SUNNONLINSOL_NEWTON module implementation of a Newton iteration by default.For details on using a non-default or user-supplied nonlinear solver see the “Using CVODE[S] for …” chapters in the user guides. CVODE(S) functions for setting the nonlinear solver options (e.g., CVodeSetMaxNonlinIters) or getting nonlinear solver statistics (e.g., CVodeGetNumNonlinSolvIters) remain unchanged and internally call generic SUNNonlinSol functions as needed.

By default IDA(S) uses the SUNNONLINSOL_NEWTON module. Since IDA(S) previously only used an internal implementation of a Newton iteration no changes are required to user programs and functions for setting the nonlinear solver options (e.g., IDASetMaxNonlinIters) or getting nonlinear solver statistics (e.g., IDAGetNumNonlinSolvIters) remain unchanged and internally call generic SUNNonlinSol functions as needed. While SUNDIALS includes a fixed-point nonlinear solver module, it is not currently supported in IDA(S).For details on using a non-default or user-supplied nonlinear solver see the “Using IDA[S] for …” chapters in the user guides. Additionally, the example program idaRoberts_dns.c explicitly creates an attaches a SUNNONLINSOL_NEWTON object to demonstrate the process of creating and attaching a nonlinear solver module (note this is not necessary in general as IDA uses the SUNNONLINSOL_NEWTON module by default).

Three fused vector operations and seven vector array operations have been added to the NVECTOR API. These optional operations are disabled by default and may be activated by calling vector specific routines after creating an NVECTOR. See the NVECTOR chapter in the user guides for more information on the new operations.

Added a new NVECTOR (NVECTOR_OPENMPDEV) which leverages OpenMP 4.5+ device offloading.

Multiple updates to the CUDA NVECTOR were made:

• Changed the N_VMake_Cuda function to take a host data pointer and a device data pointer instead of an N_VectorContent_Cuda object.
• Changed N_VGetLength_Cuda to return the global vector length instead of the local vector length.
• Added N_VGetLocalLength_Cuda to return the local vector length.
• Added N_VGetMPIComm_Cuda to return the MPI communicator used.
• Removed the accessor functions in the namespace suncudavec.
• Added the ability to set the cudaStream_t used for execution of the CUDA NVECTOR kernels. See the function N_VSetCudaStreams_Cuda.
• Added N_VNewManaged_Cuda, N_VMakeManaged_Cuda, and N_VIsManagedMemory_Cuda functions to accommodate using managed memory with the CUDA NVECTOR.

Multiple updates to the RAJA NVECTOR were made:

• Changed N_VGetLength_Raja to return the global vector length instead of the local vector length.
• Added N_VGetLocalLength_Raja to return the local vector length.
• Added N_VGetMPIComm_Raja to return the MPI communicator used.
• Removed the accessor functions in the namespace sunrajavec.

The SUNBandMatrix constructor has been simplified to remove the storage upper bandwidth argument.

Two changes were made in the CVODE/CVODES/ARKODE initial step size algorithm:

• Fixed an efficiency bug where an extra call to the RHS function was made.
• Changed the behavior of the algorithm if the max-iterations case is hit. Before the algorithm would exit with the step size calculated on the penultimate iteration. Now it will exit with the step size calculated on the final iteration.

Fortran 2003 interfaces to CVODE, the fixed-point and Newton nonlinear solvers, the dense, band, KLU, PCG, SPBCGS, SPFGMR, SPGMR, and SPTFQMR linear solvers, and the serial, PThreads, and OpenMP NVECTORs have been added.

The ARKode library has been entirely rewritten to support a modular approach to one-step methods, which should allow for rapid research and development of novel integration methods without affecting existing solver functionality.

A new ARKode stepper, MRIStep, has been added for two rate explicit-explicit multirate infinitesimal step methods.

ARKode’s dense output infrastructure has been improved to support higher-degree Hermite polynomial interpolants (up to degree 5) over the last successful time step.

v3.2.1

SUNDIALS Release v3.2.1