QUESTION: How do I build with/for CUDA? #35

cement-head · 2021-02-16T18:27:50Z

If I want to build with CUDA capacity (on Ubuntu 18.04 LTS), would the command be?:

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=racon_enable_cuda .. && make

The text was updated successfully, but these errors were encountered:

rvaser · 2021-02-16T18:30:47Z

Hi,
the command should be:

cmake -DCMAKE_BUILD_TYPE=Release -Dracon_build_cuda=ON .. && make

Best regards,
Robert

cement-head · 2021-02-16T18:56:09Z

Any options recommendations for the CUDA specific commands? Or just start with the defaults?

only available when built with CUDA:
   -c, --cuda-poa-batches <int>
     default: 0
     number of batches for CUDA accelerated polishing
   -b, --cuda-banded-alignment
     use banding approximation for polishing on GPU
     (only applicable when -c is used)
   -a, --cuda-alignment-batches <int>
     default: 0
     number of batches for CUDA accelerated alignment

Something like:

$ raven -t 124 -c -b -a <input_file.fastq>

cement-head · 2021-02-16T19:02:05Z

Also, Ubuntu 18.04 LTS will need an updated PPA for cmake: https://blog.kitware.com/ubuntu-cmake-repository-now-available/

rvaser · 2021-02-17T01:19:42Z

You should first run with -c 1 -a 1, and then increase it until you hit maximum memory on your GPU. -b might decrease accuracy.

cement-head · 2021-02-17T14:47:29Z

Okay, I'm running dual (nVIDIA SLI) RTX TITANS (24 GB each).

Just to clarify; the GPUs are used for the polishing/alignment step(s) of RACON, correct?

rvaser · 2021-02-17T15:13:51Z

Yes, only polishing (POA) and alignment (pairwise).

cement-head · 2021-02-22T18:59:38Z

Nope. Build failed. Are you sure it's:

cmake -DCMAKE_BUILD_TYPE=Release -Dracon_build_cuda=ON .. && make

and not

cmake -DCMAKE_BUILD_TYPE=Release -Dracon_**enable**_cuda=ON .. && make

rvaser · 2021-02-22T19:08:26Z

My bad, it should be enable.

cement-head · 2021-02-22T20:18:57Z

CUDA Build fails; here's the terminal output:

$ cmake -DCMAKE_BUILD_TYPE=Release -Dracon_enable_cuda=ON .. && make
-- The CXX compiler identification is GNU 9.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found suitable version "1.2.11", minimum required is "1.2.8") 
CMake Deprecation Warning at build/_deps/cereal-src/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Warning (dev) at build/_deps/cereal-src/CMakeLists.txt:2 (project):
  Policy CMP0048 is not set: project() command manages VERSION variables.
  Run "cmake --help-policy CMP0048" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  The following variable(s) would be set to empty:

    PROJECT_VERSION
    PROJECT_VERSION_MAJOR
    PROJECT_VERSION_MINOR
    PROJECT_VERSION_PATCH
This warning is for project developers.  Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 9.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Boost: /usr/include (found version "1.65.1") found components: serialization 
-- boost_variant.cpp
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.13") found components: doxygen dot 
Setting warning flags
-- Performing Test WOLD_STYLE_CAST
-- Performing Test WOLD_STYLE_CAST - Success
-- Performing Test WSHADOW
-- Performing Test WSHADOW - Success
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Using CUDA 11.2 from /usr/local/cuda-11.2
-- Building GenomeWorks libraries as static objects
-- Found CUDA: /usr/local/cuda-11.2 (found suitable version "11.2", minimum required is "9.0") 
-- Using CUDA 11.2 from /usr/local/cuda-11.2
-- Build type: Release
-- Package generator - DEB
-- nvcc flags for gwbase:  -lineinfo -lineinfo -use_fast_math -Xcompiler -Wall,-Wno-pedantic -std=c++14
-- nvcc flags for cudapoa:  -lineinfo -lineinfo -use_fast_math -Xcompiler -Wall,-Wno-pedantic -std=c++14 --expt-relaxed-constexpr
-- Using CUDA 11.2 from /usr/local/cuda-11.2
-- Autodetected CUDA architecture(s):  7.5 7.5
-- nvcc flags for cudamapper:  -lineinfo -lineinfo -use_fast_math -Xcompiler -Wall,-Wno-pedantic -lineinfo --expt-extended-lambda -use_fast_math -Xcompiler -Wall,-Wno-pedantic -std=c++14 -gencode;arch=compute_75,code=sm_75
-- nvcc flags for cudaaligner:  -lineinfo -lineinfo -use_fast_math -Xcompiler -Wall,-Wno-pedantic -std=c++14 --expt-relaxed-constexpr
-- Enabling Doxygen documentation generation
CMake Deprecation Warning at build/_deps/googletest-src/CMakeLists.txt:4 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Deprecation Warning at build/_deps/googletest-src/googlemock/CMakeLists.txt:45 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Deprecation Warning at build/_deps/googletest-src/googletest/CMakeLists.txt:56 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Found PythonInterp: /usr/bin/python (found version "2.7.17") 
-- Configuring done
CMake Warning (dev) at build/_deps/edlib-src/CMakeLists.txt:54 (add_library):
  Policy CMP0063 is not set: Honor visibility properties for all target
  types.  Run "cmake --help-policy CMP0063" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  Target "edlib" of type "STATIC_LIBRARY" has the following visibility
  properties set for CXX:

    CXX_VISIBILITY_PRESET
    VISIBILITY_INLINES_HIDDEN

  For compatibility CMake is not honoring them for this target.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /home/cbfgws6/Programs/raven/build
Scanning dependencies of target gtest
[  0%] Building CXX object _deps/googletest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[  0%] Linking CXX static library ../../../lib/libgtest.a
[  0%] Built target gtest
Scanning dependencies of target gwbase
[  0%] Building CXX object _deps/genomeworks-build/common/base/CMakeFiles/gwbase.dir/src/cudautils.cpp.o
In file included from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/fmt.h:21,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/common.h:28,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/spdlog.h:12,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/common/base/include/claraparabricks/genomeworks/logging/logging.hpp:99,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/common/base/include/claraparabricks/genomeworks/utils/cudautils.hpp:22,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/common/base/src/cudautils.cpp:17:
/home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/bundled/format.h:3475:55: warning: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3475 | FMT_CONSTEXPR internal::udl_formatter<Char, CHARS...> operator""_format() {
      |                                                       ^~~~~~~~
[  0%] Building CXX object _deps/genomeworks-build/common/base/CMakeFiles/gwbase.dir/src/logging.cpp.o
In file included from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/fmt.h:21,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/common.h:28,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/spdlog.h:12,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/common/base/include/claraparabricks/genomeworks/logging/logging.hpp:99,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/common/base/src/logging.cpp:17:
/home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/bundled/format.h:3475:55: warning: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3475 | FMT_CONSTEXPR internal::udl_formatter<Char, CHARS...> operator""_format() {
      |                                                       ^~~~~~~~
[  0%] Building CXX object _deps/genomeworks-build/common/base/CMakeFiles/gwbase.dir/src/graph.cpp.o
[  0%] Linking CXX static library ../../../racon-build/lib/libgwbase.a
[  0%] Built target gwbase
[  0%] Building NVCC (Device) object _deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/src/cudaaligner_generated_ukkonen_gpu.cu.o
[ 12%] Building NVCC (Device) object _deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/src/cudaaligner_generated_hirschberg_myers_gpu.cu.o
[ 12%] Building NVCC (Device) object _deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/src/cudaaligner_generated_myers_gpu.cu.o
Scanning dependencies of target cudaaligner
[ 12%] Building CXX object _deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/src/cudaaligner.cpp.o
In file included from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/fmt.h:21,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/common.h:28,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/spdlog.h:12,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/common/base/include/claraparabricks/genomeworks/logging/logging.hpp:99,
                 from /home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/cudaaligner/src/cudaaligner.cpp:18:
/home/cbfgws6/Programs/raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/bundled/format.h:3475:55: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Werror=pedantic]
 3475 | FMT_CONSTEXPR internal::udl_formatter<Char, CHARS...> operator""_format() {
      |                                                       ^~~~~~~~
cc1plus: all warnings being treated as errors
_deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/build.make:102: recipe for target '_deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/src/cudaaligner.cpp.o' failed
make[2]: *** [_deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/src/cudaaligner.cpp.o] Error 1
CMakeFiles/Makefile2:5856: recipe for target '_deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/all' failed
make[1]: *** [_deps/genomeworks-build/cudaaligner/CMakeFiles/cudaaligner.dir/all] Error 2
Makefile:170: recipe for target 'all' failed
make: *** [all] Error 2

cement-head · 2021-02-22T20:23:17Z

CMakeError.log

Attached is the Build Error Log; looks like there's a pthreads issue.

Is it this bug: https://stackoverflow.com/questions/24813827/cmake-failing-to-detect-pthreads-due-to-warnings/25130590#25130590

cement-head · 2021-02-22T22:48:40Z

This file
CMakeLists.txt
partially fixes the issue, gets past the pthreads issue, but now we have a -pedantic issue (C vs C++) compatibility?

rvaser · 2021-02-23T04:54:07Z

Can you compile Raven without option racon_enable_cuda? What is your compiler version and OS?

cement-head · 2021-02-23T15:56:48Z

YES. Using this file in place of the gitclone repo's CMakeLists.txt makes the compile work; one has to specify <-pthread> at the beginning for the C and CXX compiler.

Using Ubuntu 18.04.5 LTS - the rest of the information is in the terminal output attached below.

RAVEN_Fixed_CMakeLists.txt

Here's the terminal output (there were/was no CMakeErrorLog.txt) generated.

raven_build_terminal_NO-CUDA.txt

I believe the CUDA compile error has to do with the <-pedantic> flag indicating to treat ALL warnings as errors, thus causing the CUDA compiler to barf (see above).

cement-head · 2021-02-23T22:01:22Z

https://forums.developer.nvidia.com/t/compile-error-pedantic-bug/169182

cement-head · 2021-02-23T22:20:34Z

FIXED. It was a bad set of quotes in </raven/build/_deps/genomeworks-src/3rdparty/spdlog/include/spdlog/fmt/bundled/format.h>

Pulled solution from here: https://www.gitmemory.com/issue/yuzu-emu/yuzu/2597/507715224

Changed Line 3475 from:
FMT_CONSTEXPR internal::udl_formatter<Char, CHARS...> operator""_format() {

To (remove quotes):
FMT_CONSTEXPR internal::udl_formatter<Char, CHARS...> operator_format() {

Re-ran compile command & ran sudo make install

cement-head · 2021-02-24T02:16:30Z

Okay, so now...getting back to the original question: CUDA parameters.

If I have 24 GB per card (two cards) gives me 24,000 MB; and each band is 200 MB, that works out to (24,000/200=120) 120 "bands". To be safe, should I be setting both -a and -c to 110?

Something like:

$ raven -t 124 -c 110 -a 110 <input_file.fastq>

rvaser · 2021-02-24T03:03:07Z

You can try with -c 100 -a 100. Do not forget to redirect the output from stdout to a file > raven_asm.fasta.

cement-head · 2021-02-24T11:33:24Z

So, the full command would then be:

$ raven -t 124 -c 110 -a 110 <input_file.fastq> > raven_asm.fasta

Correct? If so, can you please update the documentation <$ raven -h> file and the <ReadMe.md> file? TIA

rvaser · 2021-02-24T12:01:50Z

Yes. It is already written (see below) but I can emphasize this more.

usage: raven [options ...] <sequences>

  # default output is to stdout in FASTA format  <------------
  <sequences>
    input file in FASTA/FASTQ format (can be compressed with gzip)

cement-head · 2021-02-24T12:08:25Z

Yeah, okay, my bad (not a real geek) & I missed that!

So, I'm running:

$ raven -t 124 -c 110 -a 110 <input_file.fastq> | tee raven_asm.fasta

Which should give me both a terminal output AND a written file output <raven_asm.fasta>

rvaser · 2021-02-24T12:43:54Z

This will also put sequences to stdout, which you do not want if they are big. Just redirect them to a file as showed above, and on stderr (directly to terminal) you will get the log. If you want the log in a file, redirect stderr with 2> raven_asm.err.

cement-head · 2021-02-24T13:04:51Z

So:
$ raven -t 124 -c 110 -a 110 <input_file.fastq> > raven_asm.fasta 2> raven_asm.err
Would give the assembly and the log as separate files?
How would you have the stderr on both the terminal and in a file (might be useful if the stderr is large)?

cement-head · 2021-02-24T14:27:48Z

FYI: NVIDIA-Genomics-Research/GenomeWorks#637

GenomeWorks will be fixing this (CUDA compile) issue in the next release.

rvaser · 2021-02-25T03:59:44Z

Not sure how to both redirect to file and see it in terminal 😕.

cement-head closed this as completed Feb 16, 2021

cement-head reopened this Feb 16, 2021

cement-head closed this as completed Mar 2, 2021

jkbenotmane mentioned this issue Aug 17, 2021

CUDA automatically enabled when Build with -Dracon_enable_cuda=ON isovic/racon#196

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

QUESTION: How do I build with/for CUDA? #35

QUESTION: How do I build with/for CUDA? #35

cement-head commented Feb 16, 2021 •

edited

Loading

rvaser commented Feb 16, 2021

cement-head commented Feb 16, 2021

cement-head commented Feb 16, 2021

rvaser commented Feb 17, 2021

cement-head commented Feb 17, 2021 •

edited

Loading

rvaser commented Feb 17, 2021

cement-head commented Feb 22, 2021

rvaser commented Feb 22, 2021

cement-head commented Feb 22, 2021

cement-head commented Feb 22, 2021 •

edited

Loading

cement-head commented Feb 22, 2021

rvaser commented Feb 23, 2021 •

edited

Loading

cement-head commented Feb 23, 2021 •

edited

Loading

cement-head commented Feb 23, 2021

cement-head commented Feb 23, 2021

cement-head commented Feb 24, 2021

rvaser commented Feb 24, 2021

cement-head commented Feb 24, 2021 •

edited

Loading

rvaser commented Feb 24, 2021 •

edited

Loading

cement-head commented Feb 24, 2021 •

edited

Loading

rvaser commented Feb 24, 2021 •

edited

Loading

cement-head commented Feb 24, 2021 •

edited

Loading

cement-head commented Feb 24, 2021

rvaser commented Feb 25, 2021

QUESTION: How do I build with/for CUDA? #35

QUESTION: How do I build with/for CUDA? #35

Comments

cement-head commented Feb 16, 2021 • edited Loading

rvaser commented Feb 16, 2021

cement-head commented Feb 16, 2021

cement-head commented Feb 16, 2021

rvaser commented Feb 17, 2021

cement-head commented Feb 17, 2021 • edited Loading

rvaser commented Feb 17, 2021

cement-head commented Feb 22, 2021

rvaser commented Feb 22, 2021

cement-head commented Feb 22, 2021

cement-head commented Feb 22, 2021 • edited Loading

cement-head commented Feb 22, 2021

rvaser commented Feb 23, 2021 • edited Loading

cement-head commented Feb 23, 2021 • edited Loading

cement-head commented Feb 23, 2021

cement-head commented Feb 23, 2021

cement-head commented Feb 24, 2021

rvaser commented Feb 24, 2021

cement-head commented Feb 24, 2021 • edited Loading

rvaser commented Feb 24, 2021 • edited Loading

cement-head commented Feb 24, 2021 • edited Loading

rvaser commented Feb 24, 2021 • edited Loading

cement-head commented Feb 24, 2021 • edited Loading

cement-head commented Feb 24, 2021

rvaser commented Feb 25, 2021

cement-head commented Feb 16, 2021 •

edited

Loading

cement-head commented Feb 17, 2021 •

edited

Loading

cement-head commented Feb 22, 2021 •

edited

Loading

rvaser commented Feb 23, 2021 •

edited

Loading

cement-head commented Feb 23, 2021 •

edited

Loading

cement-head commented Feb 24, 2021 •

edited

Loading

rvaser commented Feb 24, 2021 •

edited

Loading

cement-head commented Feb 24, 2021 •

edited

Loading

rvaser commented Feb 24, 2021 •

edited

Loading

cement-head commented Feb 24, 2021 •

edited

Loading