About "nekrs" or "nrspre" compilation issues for large models - "Result of SIZE overflows"

[DiscoverXV]

Hello,
I am currently using NekRSv24.0, and I am trying to simulate a mesh with 1 million elements, and I will use 32 GPUs. However, an "overflow" occurs during the compilation process "nekrs", and even if I refer to other discussions and use "nrspre" for preprocessing, I still face this problem.

Note: Since nrspre is not found in nekrs V24.0/bin, nrspre is copied from nekrs V23.0.

At the same time, due to HPC requirements, I also modified the nrspre file as follows

I hope to discuss the problem of large model overflow.

Thank you.
Best regards.

[stgeke]

Please share the output of

cat .cache/nek5000/SIZE

[DiscoverXV]

Thanks for your reply.

Here is my "SIZE" file:

[stgeke]

It seems that you previously compiled a 4M element mesh using 4 MPI ranks. Now, you're attempting to compile a smaller mesh on fewer MPI ranks, which does not require a SIZE update.

To enforce a SIZE file update, please delete .cache/nek5000.

[DiscoverXV]

You are right. After I deleted it, nekrs can already proceed to "building udf...". Since my udf file has many configuration parameters, it may take some time to complete the preprocessing.

I would like to ask if the function of nekrs's "nrspre" is to compile all configuration files before the simulation runs. Later, do I only need to "srun -N 1 -n 32 nekrs " to start the simulation?

[stgeke]

I suggest reviewing the various job submission scripts available in the NEKRS_INSTALL_DIR/bin directory.

[DiscoverXV]

Thank you so much!

Best regards,