add libraft wheels

[jameslamb]

Pulling just the RAFT-related changes off of #438

Complements rapidsai/raft#2531

Notes for Reviewers

Proposing:

1. see this pass CI pointed at my fork (branch from introduce libraft wheels raft#2531)
2. admin-merge introduce libraft wheels raft#2531
3. point this back at upstream rapidsai/raft and see it pass CI
4. merge

[bdice]

Pre-approving. All looks fine to me.

[jameslamb]

CUDA 12.0 builds segfaulted (!!!) compiling cuDF:

2025-01-16T16:41:35.3774797Z FAILED: CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o 
2025-01-16T16:41:35.3793351Z /usr/bin/sccache /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/usr/bin/g++ -DBS_THREAD_POOL_ENABLE_PAUSE=1 -DCUDF_CUFILE_FOUND -DCUDF_KVIKIO_REMOTE_IO -DCUDF_LOG_ACTIVE_LEVEL=CUDF_LOG_LEVEL_INFO -DFMT_HEADER_ONLY=1 -DJITIFY_PRINT_LOG=0 -DKVIKIO_CUDA_FOUND -DKVIKIO_CUFILE_BATCH_API_FOUND -DKVIKIO_CUFILE_FOUND -DKVIKIO_LIBCURL_FOUND -DLIBCUDACXX_ENABLE_EXPERIMENTAL_MEMORY_RESOURCE -DSPDLOG_FMT_EXTERNAL -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_CUDA -DTHRUST_DISABLE_ABI_NAMESPACE -DTHRUST_FORCE_32_BIT_OFFSET_TYPE=1 -DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_CPP -DTHRUST_IGNORE_ABI_NAMESPACE_ERROR -Dcudf_EXPORTS -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/dlpack-src/include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/jitify-src -I/home/coder/cudf/cpp/include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/include -I/home/coder/cudf/cpp/src -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/nanoarrow-src/src -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/flatbuffers-src/include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/cccl-src/thrust/thrust/cmake/../.. -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/cccl-src/libcudacxx/lib/cmake/libcudacxx/../../../include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/cccl-src/cub/cub/cmake/../.. -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/nvtx3-src/c/include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/cuco-src/include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/nanoarrow-build/src -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/spdlog-src/include -I/home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/fmt-src/include -isystem /home/coder/rmm/include -isystem /usr/local/cuda/targets/x86_64-linux/include -isystem /home/coder/rmm/build/pip/cuda-12.0/release/include -isystem /home/coder/kvikio/cpp/build/pip/cuda-12.0/release/_deps/bs_thread_pool-src/include -isystem /home/coder/cudf/cpp/build/pip/cuda-12.0/release/_deps/nvcomp_proprietary_binary-src/include -isystem /home/coder/kvikio/cpp/include -t=1 -O3 -DNDEBUG -std=c++17 "--generate-code=arch=compute_70,code=[sm_70]" "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_86,code=[sm_86]" "--generate-code=arch=compute_90,code=[compute_90,sm_90]" -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden --expt-extended-lambda --expt-relaxed-constexpr -Werror=all-warnings -Xcompiler=-Wall,-Werror,-Wno-error=deprecated-declarations -diag-suppress=1407 -Xfatbin=-compress-all -MD -MT CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o -MF CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o.d -x cu -c /home/coder/cudf/cpp/src/io/parquet/bloom_filter_reader.cu -o CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o
2025-01-16T16:41:35.3805407Z Segmentation fault (core dumped)

(build link)

CUDA 12.5 builds have already progressed beyond that point (successfully compiled cuDF): https://github.com/rapidsai/devcontainers/actions/runs/12813231633/job/35726830742?pr=438

I'm going to stop and restart these builds (since they're still early on), to see if that segfault was something temporary.

[bdice]

@jameslamb This is something I think we've seen elsewhere in Slack. @mhaseeb123 This appears to be related to rapidsai/cudf#17289.

  FAILED: CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o 
  /usr/bin/sccache /home/coder/.conda/envs/rapids/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/home/coder/.conda/envs/rapids/bin/x86_64-conda-linux-gnu-c++ -DBS_THREAD_POOL_ENABLE_PAUSE=1 -DCUDF_CUFILE_FOUND -DCUDF_KVIKIO_REMOTE_IO -DCUDF_LOG_ACTIVE_LEVEL=CUDF_LOG_LEVEL_INFO -DFMT_SHARED -DJITIFY_PRINT_LOG=0 -DKVIKIO_CUDA_FOUND -DKVIKIO_CUFILE_BATCH_API_FOUND -DKVIKIO_CUFILE_FOUND -DKVIKIO_LIBCURL_FOUND -DLIBCUDACXX_ENABLE_EXPERIMENTAL_MEMORY_RESOURCE -DSPDLOG_FMT_EXTERNAL -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_CUDA -DTHRUST_DISABLE_ABI_NAMESPACE -DTHRUST_FORCE_32_BIT_OFFSET_TYPE=1 -DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_CPP -DTHRUST_IGNORE_ABI_NAMESPACE_ERROR -Dcudf_EXPORTS -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/dlpack-src/include -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/jitify-src -I/home/coder/cudf/cpp/include -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/include -I/home/coder/cudf/cpp/src -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/nanoarrow-src/src -I/include -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/cccl-src/thrust/thrust/cmake/../.. -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/cccl-src/libcudacxx/lib/cmake/libcudacxx/../../../include -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/cccl-src/cub/cub/cmake/../.. -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/nvtx3-src/c/include -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/cuco-src/include -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/nanoarrow-build/src -I/home/coder/cudf/cpp/build/conda/cuda-12.0/release/_deps/fmt-src/include -isystem /home/coder/rmm/include -isystem /home/coder/.conda/envs/rapids/targets/x86_64-linux/include -isystem /home/coder/rmm/build/conda/cuda-12.0/release/include -isystem /home/coder/kvikio/cpp/build/conda/cuda-12.0/release/_deps/bs_thread_pool-src/include -isystem /home/coder/.conda/envs/rapids/include -isystem /home/coder/kvikio/cpp/include -t=1 -O3 -DNDEBUG -std=c++17 "--generate-code=arch=compute_70,code=[sm_70]" "--generate-code=arch=compute_75,code=[sm_75]" "--generate-code=arch=compute_80,code=[sm_80]" "--generate-code=arch=compute_86,code=[sm_86]" "--generate-code=arch=compute_90,code=[compute_90,sm_90]" -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden --expt-extended-lambda --expt-relaxed-constexpr -Werror=all-warnings -Xcompiler=-Wall,-Werror,-Wno-error=deprecated-declarations -diag-suppress=1407 -Xfatbin=-compress-all -Xcompiler -pthread -MD -MT CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o -MF CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o.d -x cu -c /home/coder/cudf/cpp/src/io/parquet/bloom_filter_reader.cu -o CMakeFiles/cudf.dir/src/io/parquet/bloom_filter_reader.cu.o
  Segmentation fault (core dumped)

[bdice]

CI on this will probably be blocked until we have a resolution for rapidsai/cudf#17753. We will have to temporarily disable any failing jobs -- seems like 12.0 is affected but perhaps 12.5 is not? Or we can admin-merge.

[jameslamb]

CI on this will probably be blocked until we have a resolution for rapidsai/cudf#17753.

If we see the CUDA 12.5 jobs pass, I'm in favor of skipping the CUDA 12.0 jobs until that cuDF issue is resolved.

The 12.5 job just made it to RAFT (still in progress), so I'll let it keep running for a bit to at least ensure we see the RAFT-related changes in this PR working.

[jameslamb]

🎉 the CUDA 12.5 CI jobs ran successfully!

https://github.com/rapidsai/devcontainers/actions/runs/12813231633/job/35729224938?pr=438

That's strong evidence that rapidsai/raft#2531 will work with RAPIDS devcontainers.

[bdice]

I opened #439 to disable CUDA 12.0 RAPIDS builds in devcontainers for now.

[bdice]

CI is blocked until rapidsai/cuvs#454 goes in.

xref: rapidsai/raft#2487 (comment)

I am going to merge this once the CUDA 12.5 pip devcontainer gets past RAFT. We expect it will fail on cuVS. (It is currently halfway done building cudf.)