1D example not running with PETSC

[wearysebas]

After installing Hermes-3 and trying to run the first 1D-Threshold example, an error arises stating

"[0]PETSC ERROR: Argument out of range
[0]PETSC ERROR: New nonzero at (0,0) caused a malloc"

Output files just ended with message "Found value 0 for Jacobian".

If using CVODE the error doesn't occur.

[ZedThree]

Hi @wearysebas, please could you open this over at the Hermes-3 repo? This one is for the post-processing tools.

Please include: the version or commit of Hermes-3 that you're using, along with the version of PETSc, and the full output log. Thanks!

[bendudson]

Thanks @wearysebas for trying this and your report.
I rebuilt hermes-3 (latest master, commit bb1dfc1) with BOUT++ by running:

git clone git@github.com:boutproject/hermes-3.git
cmake . -B build -DBOUT_USE_PETSC=ON
cd build; make -j

Then running the example:

./hermes-3 -d examples/tokamak-1D/1D-threshold

produces this output

4.789e+03       3869       7.13e+00    94.6    0.0    0.1    0.4    5.0
9.579e+03        113       2.39e-01    82.7    0.0    0.1    8.6    8.7
1.437e+04        160       3.27e-01    85.4    0.0    0.1    6.0    8.6
1.916e+04        398       8.55e-01    91.4    0.0    0.1    2.3    6.2
...
2.347e+05         66       1.56e-01    73.3    0.0    0.1   16.3   10.3
2.395e+05         45       1.21e-01    66.8    0.0    0.1   21.2   11.9
-  Step 50 of 50. Elapsed 0:00:52.5 ETA 0:00:00.0
Run finished at  : Tue Oct 21 07:24:10 2025
Run time : 52 s

I'm using PETSc version 3.23.0. (Release, March 28, 2025)

[wearysebas]

Hi,

I built it using the last commit from https://github.com/mikekryjak/bout-build-scripts/tree/hermes3-viking2# which I believe are @mikekryjak 's scripts on branch Hermes3-Viking2.

From the BOUT.log file: "
Solver running for 50 outputs with output timestep of 4.789400e+03
Initialising solver

    SNES steady state solver
    3d fields = 7, 2d fields = 0 neq=2800, local_N=700

Creating vector
Create SNES
Setting Jacobian matrix sizes
Option solver:stencil:square = 0 (default)
Option solver:stencil:taxi = 0 (default)
Option solver:stencil:cross = 0 (default)
Setting solver:stencil:taxi = 2
Pre-allocating Jacobian
Marking non-zero Jacobian entries"

From the output:

Setting solver:stencil:taxi = 2
Pre-allocating Jacobian
Marking non-zero Jacobian entries
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Argument out of range
[0]PETSC ERROR: New nonzero at (0,0) caused a malloc
Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn off this check
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.17.4, Aug 01, 2022
[0]PETSC ERROR: hermes-3 on a named node104.viking2.yor.alces.network by sr1937 Tue Oct 21 16:02:46 2025
[0]PETSC ERROR: Configure options COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 --download-hypre --with-debugging=0 --prefix=../petsc-build
[0]PETSC ERROR: #1 MatSetValues_MPIAIJ() at /mnt/scratch/users/sr1937/BOUT-c4c149a/dependencies/petsc-3.17.4/src/mat/impls/aij/mpi/mpiaij.c:470
[0]PETSC ERROR: #2 MatSetValues() at /mnt/scratch/users/sr1937/BOUT-c4c149a/dependencies/petsc-3.17.4/src/mat/interface/matrix.c:1343
terminate called after throwing an instance of 'fmt::v10::format_error'
what(): argument not found
[node104:1254685] *** Process received signal ***
[node104:1254685] Signal: Aborted (6)
[node104:1254685] Signal code: (-6)
[node104:1254685] [ 0] /lib64/libpthread.so.0(+0x12990)[0x7ffff5862990]
[node104:1254685] [ 1] /lib64/libc.so.6(gsignal+0x10f)[0x7ffff3d3852f]
[node104:1254685] [ 2] /lib64/libc.so.6(abort+0x127)[0x7ffff3d0be65]
[node104:1254685] [ 3] /opt/apps/eb/software/GCCcore/10.3.0/lib64/libstdc++.so.6(+0xadecc)[0x7ffff545decc]
[node104:1254685] [ 4] /opt/apps/eb/software/GCCcore/10.3.0/lib64/libstdc++.so.6(+0xb8f86)[0x7ffff5468f86]
[node104:1254685] [ 5] /opt/apps/eb/software/GCCcore/10.3.0/lib64/libstdc++.so.6(+0xb8ff1)[0x7ffff5468ff1]
[node104:1254685] [ 6] /opt/apps/eb/software/GCCcore/10.3.0/lib64/libstdc++.so.6(+0xb9285)[0x7ffff5469285]
[node104:1254685] [ 7] /users/sr1937/scratch/BOUT-c4c149a/BOUT-dev/build/lib/libfmt.so.10(_ZN3fmt3v106detail18throw_format_errorEPKc+0x44)[0x7ffff7e9a194]
[node104:1254685] [ 8] /users/sr1937/scratch/BOUT-c4c149a/BOUT-dev/build/lib/libfmt.so.10(ZN3fmt3v106detail23parse_replacement_fieldIcRZNS1_10vformat_toIcEEvRNS1_6bufferIT_EENS0_17basic_string_viewIS5_EENS1_12vformat_argsIS5_E4typeENS1_10locale_refEE14format_handlerEEPKS5_SH_SH_OT0+0x20c)[0x7ffff7eb039c]
[node104:1254685] [ 9] /users/sr1937/scratch/BOUT-c4c149a/BOUT-dev/build/lib/libfmt.so.10(_ZN3fmt3v106detail10vformat_toIcEEvRNS1_6bufferIT_EENS0_17basic_string_viewIS4_EENS1_12vformat_argsIS4_E4typeENS1_10locale_refE+0x12f)[0x7ffff7eb1a2f]
[node104:1254685] [10] /users/sr1937/scratch/BOUT-c4c149a/BOUT-dev/build/lib/libfmt.so.10(_ZN3fmt3v107vformatB5cxx11ENS0_17basic_string_viewIcEENS0_17basic_format_argsINS0_20basic_format_contextINS0_8appenderEcEEEE+0x4f)[0x7ffff7e9f8bf]
[node104:1254685] [11] /users/sr1937/scratch/BOUT-c4c149a/BOUT-dev/build/lib/libbout++.so.5.1.0(_ZN10SNESSolver4initEv+0x209f)[0x7ffff7c956af]
[node104:1254685] [12] /users/sr1937/scratch/BOUT-c4c149a/BOUT-dev/build/lib/libbout++.so.5.1.0(_ZN6Solver5solveEid+0xa5)[0x7ffff7ca7d65]
[node104:1254685] [13] hermes-3[0x43ce0d]
[node104:1254685] [14] /lib64/libc.so.6(__libc_start_main+0xe5)[0x7ffff3d247e5]
[node104:1254685] [15] hermes-3[0x44188e]
[node104:1254685] *** End of error message ***

Running it using the cvode solver instead runs smoothly. I hope this helps!

[bendudson]

Thanks @wearysebas ! I suspect this line:
https://github.com/boutproject/BOUT-dev/blob/master/src/solver/impls/snes/snes.cxx#L501

MatSetOption(Jfd, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_TRUE);

That line tells PETSc to error when a new Jacobian element is allocated. I don't know why a new element is being allocated, that is really the underlying error, but I think if you change PETSC_TRUE to PETSC_FALSE then PETSc should allocate memory and continue without error.

Perhaps this flag should depend on the CHECK level.