Kdcount building fail during cython build on NERSC

[ClaudioNahmad]

Hello,

I'm trying to install nbodykit on NERSC (Perlmutter) (on a login node) by following Jmsul's last comment on this issue, i.e.

module load python
module load conda

then

module load PrgEnv-gnu
module load gsl
conda create --name nbodykit_env python=3.7 pip
env MPICC=cc python -m pip install --no-cache-dir mpi4py
pip install numpy cython
pip install nbodykit[extras]

It all runs fine until the last command: pip install nbodykit[extras], specifically during the building of kdcount, the error seems to have to do with cython, and is as follows:

Collecting kdcount (from nbodykit)
  Using cached kdcount-0.3.29.tar.gz (204 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [42 lines of output]
      /global/homes/n/nahmad/.conda/envs/nbodykit_env/lib/python3.7/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-frjjy3qo/kdcount_83a3d6a6720848528e08d39385a8bd85/kdcount/pykdcount.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          for i in range(ndims):
              f[i] = 1
      
      cdef class KDForceCount(KDForceKernel):
          def __init__(self):
              self.func = kd_force_func_count
                          ^
      ------------------------------------------------------------
      
      kdcount/pykdcount.pyx:121:20: Cannot assign type 'void (double, double *, double *, int, void *) except *' to 'kd_force_func'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (double, double *, double *, int, void *) except *'.
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          for i in range(ndims):
              f[i] = dx[i] * ir3
      
      cdef class KDForcePlummer(KDForceKernel):
          def __init__(self, a):
              self.func = kd_force_func_plummer
                          ^
      ------------------------------------------------------------
      
      kdcount/pykdcount.pyx:138:20: Cannot assign type 'void (double, double *, double *, int, void *) except *' to 'kd_force_func'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (double, double *, double *, int, void *) except *'.
      Traceback (most recent call last):
        File "<string>", line 36, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-frjjy3qo/kdcount_83a3d6a6720848528e08d39385a8bd85/setup.py", line 45, in <module>
          ext_modules = cythonize(extensions),
        File "/global/homes/n/nahmad/.conda/envs/nbodykit_env/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
          cythonize_one(*args)
        File "/global/homes/n/nahmad/.conda/envs/nbodykit_env/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: kdcount/pykdcount.pyx
      Compiling kdcount/pykdcount.pyx because it changed.
      [1/1] Cythonizing kdcount/pykdcount.pyx
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Any help solving this?

I also tried to install using conda, by creating the environment and then installing via conda, i.e. conda install -c bccp nbodykit but it gets stuck for hours at

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment:

and never achieves installation.

[ClaudioNahmad]

Hello again, i think i solved the installation problems by doing the following:

module load python
module load conda
module load PrgEnv-gnu
module load gsl
conda create --name nbodykit-env python=3.6 pip
conda activate nbodykit-env
pip install mpi4py
pip install numpy
pip install cython
pip install nbodykit
pip install nbodykit[extras]

which returns a succesful installation.

I will now run some tests and let you know if everything is working fine.

[ClaudioNahmad]

Everything is working fine. Im closing this.