Compile bug: b7551 build failed on CUDA 12.8.1

[codingl2k1]

Git commit

https://github.com/ggml-org/llama.cpp/tree/a52dc60ba3ae0ef1e941ce9a4585672cc335a175

Operating systems

Linux

GGML backends

CUDA

Problem description & steps to reproduce

Build with CUDA 12.4.1 is good, build b7376 with CUDA 12.8.1 is also good.

First Bad Commit

No response

Compile command

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0
update from llama.cpp main repo
Building for CUDA
Running CMake with arguments: -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_INSTALL_LIBDIR=lib -DLLAMA_CURL=OFF -DLLAMA_LLGUIDANCE=ON -DCMAKE_AR=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-ar -DCMAKE_CXX_COMPILER_AR=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_C_COMPILER_AR=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_RANLIB=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-ranlib -DCMAKE_CXX_COMPILER_RANLIB=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_C_COMPILER_RANLIB=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_LINKER=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-ld -DCMAKE_STRIP=/home/runner/miniconda3/envs/llamacpp/bin/x86_64-conda-linux-gnu-strip -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=OFF -DGGML_CUDA=ON -DGGML_CUDA_FORCE_MMQ=ON -DCMAKE_CUDA_ARCHITECTURES=all


cd /home/runner/work/xllamacpp/xllamacpp/thirdparty/llama.cpp/build/ggml/src/ggml-cuda && /home/runner/miniconda3/envs/llamacpp/bin/nvcc -forward-unknown-to-host-compiler -DGGML_CUDA_FORCE_MMQ -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128 -DGGML_CUDA_USE_GRAPHS -DGGML_SCHED_MAX_COPIES=4 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 --options-file CMakeFiles/ggml-cuda.dir/includes_CUDA.rsp -O3 -DNDEBUG -std=c++17 -arch=all -Xcompiler=-fPIC -use_fast_math -extended-lambda -compress-mode=size -Xcompiler "-Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wextra-semi -Wno-pedantic" -MD -MT ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/mmq-instance-mxfp4.cu.o -MF CMakeFiles/ggml-cuda.dir/template-instances/mmq-instance-mxfp4.cu.o.d -x cu -c /home/runner/work/xllamacpp/xllamacpp/thirdparty/llama.cpp/ggml/src/ggml-cuda/template-instances/mmq-instance-mxfp4.cu -o CMakeFiles/ggml-cuda.dir/template-instances/mmq-instance-mxfp4.cu.o

Relevant log output

nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
ptxas /tmp/tmpxft_00004b0d_00000000-6_mmq-instance-mxfp4.compute_120.ptx, line 1025; error   : Instruction 'mma with block scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00004b0d_00000000-6_mmq-instance-mxfp4.compute_120.ptx, line 1025; error   : Feature '.kind::mxf4' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00004b0d_00000000-6_mmq-instance-mxfp4.compute_120.ptx, line 1025; error   : Feature '.block_scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00004b0d_00000000-6_mmq-instance-mxfp4.compute_120.ptx, line 1025; error   : Feature '.scale_vec::2X' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00004b0d_00000000-6_mmq-instance-mxfp4.compute_120.ptx, line 1039; error   : Instruction 'mma with block scale' not supported on .target 'sm_120'

[am17an]

Just to check instead of -DCMAKE_CUDA_ARCHITECTURES=all, if you specify something like "70;86;89;120", it works?

[codingl2k1]

Just to check instead of -DCMAKE_CUDA_ARCHITECTURES=all, if you specify something like "70;86;89;120", it works?

I am running the build pipeline, will reply once I have results.

[codingl2k1]

Just to check instead of -DCMAKE_CUDA_ARCHITECTURES=all, if you specify something like "70;86;89;120", it works?

-- x86 detected
-- Adding CPU backend variant ggml-cpu: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C;__BMI2__;GGML_BMI2
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.8/include (found version "12.8.93")
-- CUDA Toolkit found
-- Using CUDA architectures: 70;86;89;120
-- The CUDA compiler identification is NVIDIA 12.8.93 with host compiler MSVC 19.44.35222.0
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.8/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
CMake Error at ggml/src/ggml-cuda/CMakeLists.txt:65 (message):
  Compute capability 120 used, use 120a or 120f for Blackwell specific
  optimizations


-- Configuring incomplete, errors occurred!

[am17an]

I think at the moment CMAKE_CUDA_ARCHITECTURES=all is not handled gracefully, so in your case you can use 120a instead of 120

[JohannesGaessler]

If you need only the ggml CUDA backend you should be compiling https://github.com/ggml-org/ggml instead of llama.cpp. But even then I refuse to provide support for people manually fiddling around with nvcc, I will only provide support for CMake.

[aheid]

FWIW I get the same on Windows with CUDA 12.8.2, NVIDIA 5090.

cmake -B build -DLLAMA_CURL=OFF -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_GENERATOR_TOOLSET='cuda="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8"' -DCMAKE_BUILD_TYPE=Release

cmake --build build --config Release

results in a bunch of these errors

ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121935; error : Feature '.kind::mxf4' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121935; error : Feature '.block_scale' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121935; error : Feature '.scale_vec::2X' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121949; error : Instruction 'mma with block scale' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]

Earlier commits work. I didn't bisect myself, but at least b7488 works.

edit: Using -DCMAKE_CUDA_ARCHITECTURES="120a" seems to work.

[codingl2k1]

FWIW I get the same on Windows with CUDA 12.8.2, NVIDIA 5090.

cmake -B build -DLLAMA_CURL=OFF -DGGML_CUDA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_GENERATOR_TOOLSET='cuda="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8"' -DCMAKE_BUILD_TYPE=Release

cmake --build build --config Release

results in a bunch of these errors

ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121935; error : Feature '.kind::mxf4' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121935; error : Feature '.block_scale' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121935; error : Feature '.scale_vec::2X' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]
ptxas C : /Users/username/AppData/Local/Temp/tmpxft_0000d2f0_00000000-8_mmq-instance-mxfp4.ptx, line 121949; error : Instruction 'mma with block scale' not supported on .target 'sm_120' [G:\Dev\llama.cpp\build\ggml\src\ggml-cuda\ggml-cuda.vcxproj]

Earlier commits work. I didn't bisect myself, but at least b7488 works.

edit: Using -DCMAKE_CUDA_ARCHITECTURES="120a" seems to work.

This config works. I tried -DCMAKE_CUDA_ARCHITECTURES=all with the latest CMake, but it does not work.