Skip to content

[BUG] Do not support CUDA 13.0+ #5016

@OutisLi

Description

@OutisLi

Bug summary

Compile error when using cuda 13.0

DeePMD-kit Version

latest of devel branch

Backend and its version

pytorch2.9.0+cu130

How did you download the software?

Built from source

Input Files, Running Commands, Error Log, etc.

 [4/11] Building CUDA object lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/prod_env_mat.cu.o
  FAILED: [code=1] lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/prod_env_mat.cu.o
  /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/home/outisli/miniforge3/envs/dpmd/bin/g++ -DGOOGLE_CUDA -D_GLIBCXX_USE_CXX11_ABI=1 -Ddeepmd_op_cuda_EXPORTS -I/home/outisli/Software/deepmd-kit/source/lib/src/gpu/../../include -I/usr/local/cuda/targets/x86_64-linux/include -I/usr/local/cuda/targets/x86_64-linux/include/cccl -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_120,code=sm_120 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -DCUB_IGNORE_DEPRECATED_CPP_DIALECT -DCUB_IGNORE_DEPRECATED_CPP_DIALECT -O3 -DNDEBUG -std=c++11 -Xcompiler=-fPIC -MD -MT lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/prod_env_mat.cu.o -MF lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/prod_env_mat.cu.o.d -x cu -c /home/outisli/Software/deepmd-kit/source/lib/src/gpu/prod_env_mat.cu -o lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/prod_env_mat.cu.o
  In file included from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_arch.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/config.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/block_load.cuh:34,
                   from /home/outisli/Software/deepmd-kit/source/lib/src/gpu/prod_env_mat.cu:2:
  /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_cpp_dialect.cuh:88:8: error: #error CUB requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
     88 | #      error CUB requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
        |        ^~~~~
  In file included from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/detail/libcxx/include/__config:24,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/detail/__config:43,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/version:14,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/version:14,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/version.cuh:49,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_namespace.cuh:40,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_macro.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_arch.cuh:47:
  /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/__internal/cpp_dialect.h:40:6: error: #error libcu++ requires at least C++ 17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
     40 | #    error libcu++ requires at least C++ 17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
        |      ^~~~~
  In file included from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/detail/config/config.h:37,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/detail/config.h:22,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/iterator/discard_iterator.h:23,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_type.cuh:49,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_ptx.cuh:47,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/block_exchange.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/block_load.cuh:44:
  /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/detail/config/cpp_dialect.h:78:6: error: #error Thrust requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
     78 | #    error Thrust requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
        |      ^~~~~
  [5/11] Building CUDA object lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/neighbor_list.cu.o
  FAILED: [code=1] lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/neighbor_list.cu.o
  /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/home/outisli/miniforge3/envs/dpmd/bin/g++ -DGOOGLE_CUDA -D_GLIBCXX_USE_CXX11_ABI=1 -Ddeepmd_op_cuda_EXPORTS -I/home/outisli/Software/deepmd-kit/source/lib/src/gpu/../../include -I/usr/local/cuda/targets/x86_64-linux/include -I/usr/local/cuda/targets/x86_64-linux/include/cccl -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_120,code=sm_120 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -DCUB_IGNORE_DEPRECATED_CPP_DIALECT -DCUB_IGNORE_DEPRECATED_CPP_DIALECT -O3 -DNDEBUG -std=c++11 -Xcompiler=-fPIC -MD -MT lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/neighbor_list.cu.o -MF lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/neighbor_list.cu.o.d -x cu -c /home/outisli/Software/deepmd-kit/source/lib/src/gpu/neighbor_list.cu -o lib/src/gpu/CMakeFiles/deepmd_op_cuda.dir/neighbor_list.cu.o
  In file included from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_arch.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/config.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/block_scan.cuh:35,
                   from /home/outisli/Software/deepmd-kit/source/lib/src/gpu/neighbor_list.cu:2:
  /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_cpp_dialect.cuh:88:8: error: #error CUB requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
     88 | #      error CUB requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
        |        ^~~~~
  In file included from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/detail/libcxx/include/__config:24,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/detail/__config:43,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/version:14,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/version:14,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/version.cuh:49,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_namespace.cuh:40,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_macro.cuh:46,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_arch.cuh:47:
  /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cuda/std/__internal/cpp_dialect.h:40:6: error: #error libcu++ requires at least C++ 17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
     40 | #    error libcu++ requires at least C++ 17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
        |      ^~~~~
  In file included from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/detail/config/config.h:37,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/detail/config.h:22,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/iterator/discard_iterator.h:23,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/util_type.cuh:49,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/block_raking_layout.cuh:47,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/specializations/block_scan_raking.cuh:47,
                   from /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/cub/block/block_scan.cuh:45:
  /usr/local/cuda-13.0/targets/x86_64-linux/include/cccl/thrust/detail/config/cpp_dialect.h:78:6: error: #error Thrust requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
     78 | #    error Thrust requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.
        |      ^~~~~
  [6/11] Linking CXX shared library lib/libdeepmd.so
  [7/11] Building CXX object op/pt/CMakeFiles/deepmd_op_pt.dir/print_summary.cc.o
  [8/11] Building CXX object op/pt/CMakeFiles/deepmd_op_pt.dir/comm.cc.o
  [9/11] Building CXX object op/pt/CMakeFiles/deepmd_op_pt.dir/tabulate_multi_device.cc.o
  ninja: build stopped: subcommand failed.

  *** CMake build failed
  error: subprocess-exited-with-error

  × Building editable for deepmd-kit (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/outisli/miniforge3/envs/dpmd/bin/python3.14 /home/outisli/miniforge3/envs/dpmd/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_editable /tmp/tmp_rt4ymxw
  cwd: /home/outisli/Software/deepmd-kit
  Building editable for deepmd-kit (pyproject.toml) ... error
  ERROR: Failed building editable for deepmd-kit
Failed to build deepmd-kit
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> deepmd-kit

Steps to Reproduce

conda update -n base -c conda-forge conda -y ; conda update -n base -c conda-forge mamba -y
conda deactivate && conda env remove -n dpmd -y ; rm -rf build ; git clean -xdf ; conda create -n dpmd gcc=15 gxx=15 cmake python=3.14 -c conda-forge -y && conda activate dpmd && pip install --upgrade pip && pip install uv

# 2.1 Install pytorch
uv pip install -U torch --index-url https://download.pytorch.org/whl/cu130

# 3. Install deepmd-kit
export CUDAToolkit_ROOT="/usr/local/cuda" CUDA_HOME="/usr/local/cuda" CUDA_PATH="/usr/local/cuda" CUDA_VERSION=13.0 && export DP_VARIANT="cuda" DP_ENABLE_PYTORCH=1 DP_ENABLE_TENSORFLOW=0 DP_ENABLE_PADDLE=0 DP_ENABLE_NATIVE_OPTIMIZATION=1 && pip install -e . -v

Further Information, Files, and Links

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions