[RFC] Redesigned CMake build infrastructure for C++ API

[diptorupd]

Overview

This PR improves the CMake build system for the C++ API of flashinfer, making it more modular, easier to configure, and overall easier to integrate into downstream projects. The changes introduce a proper component-based installation for different usage scenarios for the C++ API, adds CMake-based dependency management, and generally offers a cleaner build organization.

Key Changes

Directory Structure Changes

The C++ sources, tests, and benchmarks are now reorganized inside a libflashinfer directory.

BEFORE

flashinfer/
├── CMakeLists.txt
├── include/
│ └── flashinfer/
└── src/

AFTER

flashinfer/
├── CMakeLists.txt
├── cmake/
│ └── utils/
│ │ └── ConfigureTargets.cmake
│ ├── Config.cmake.in
│ ├── Dependencies.cmake
│ ├── Options.cmake
├── libflashinfer/
│ ├── CMakeLists.txt
│ ├── include/
│ │ └── flashinfer/
│ │ ├── attention/
│ │ └── distributed/
│ │ └── gemm/
│ ├── tests/
│ │ ├── CMakeLists.txt # Consistent test configuration
│ │ └── fp8/ # FP8-specific tests
│ ├── utils/
│ │ └── fp8/ # FP8 utility files
│ └── benchmarks/
│ ├── CMakeLists.txt # Standardized benchmark setup
│ └── fp8/ # FP8-specific benchmarks

High-level CMake changes

• Separated flashinfer CMake option definitions into distinct files for better organization
• Created utility modules for configuring tests and benchmarks consistently
• Consolidated dependency management in a single location
• Added a flashinferconfig for downstream projects to be able to do find_package(flashinfer) in their CMake scripts.

CMake configuration and build improvements

• The C++ API is broken down into multiple components: Headers, Kernels, TVMBinding, Distributed.

• Components can be configured and built separately

  • Ex1. Configuring the C++ API to build AOT kernels, C++ unit tests and benchmarks

      cmake .. -GNinja -DFLASHINFER_CUDA_ARCHITECTURES=80 -DFLASHINFER_BUILD_KERNELS=ON -DFLASHINFER_UNITTESTS=ON -DFLASHINFER_CXX_BENCHMARKS=ON -DFLASHINFER_CUTLASS_DIR=../3rdparty/cutlass -DCMAKE_INSTALL_PREFIX=~/devel/install

  • Ex2. Build various components as needed

      # Build the test_single_decode unit tests
      cmake --build . --target test_single_decode 
      # Build all unit tests (broken right now)
      cmake --build . --target build_tests
      # Build bench_single_decode 
      cmake --build . --target bench_single_decode 
      # Build all benchmarks (broken right now)
      cmake --build . --target build_benchmarks

  • Ex3. Install components

        cmake --build . --target install

• Added CMake targets to generate sources based on the same logic as in the current setup.py. Generated sources are now placed under the CMAKE_CURRENT_BINARY_DIR

  • Ex4. Install components

        cmake --build . --target generate_kernels

These changes are primarily meant to streamline the changes to add new CMake config options for a proposed HIP back end. However, it is foreseeable to build upon these changes to convert the build system to a fully unified CMake-based using scikit-build and use a single configuration system for both C++ and Python APIs. In my experience, using a unified system makes it easier to package and do integration testing.

[diptorupd]

ping @yzh119 @abcdabcd987 @MasterJH5574 @happierpig @rtmadduri

[yzh119]

Hi @diptorupd thanks for the work, I'll review this PR.

[diptorupd]

@yzh119 Thank you. I will resolve the conflicts. I will also push some additional changes I have been doing to handle issues I discovered during local testing.

[diptorupd]

@yzh119 Thank you. I will resolve the conflicts. I will also push some additional changes I have been doing to handle issues I discovered during local testing.

I have rebased my changes onto main and also pushed few additional changes and fixes.