`cuda.bindings` build time on Linux becomes ~2x slower

[leofang]

This is a new situation and is different from #602 (and the timings there can be used as a reference). I suspect this has to do with Cython 3.1.0, based on my eyeball-bisecting the recent build times taken in the main branch:
https://github.com/NVIDIA/cuda-python/actions/workflows/ci.yml?query=branch%3Amain++

I suspect this starts after Cython 3.1.0 was released, the first slowness observation:

• https://github.com/NVIDIA/cuda-python/actions/runs/14920621978

in contrast with the previous run:

• https://github.com/NVIDIA/cuda-python/actions/runs/14889184430

What I don't understand is Windows seems to be unaffected (and continues to be on the order of 4-5 mins)...

[leofang]

Confirmed in #640 that the build time can be restored by pinning Cython<3.1.0

[leofang]

I can reproduce the 2x slowdown locally. But when I tried to chase after the root cause, I probably picked the wrong module to look at. cuda.bindings.cydriver compilation is slower with newer Cython, but its compilation/linking times are insignificant compared to the overall build time. Anyway I attach the .cpp file generated by Cython 3.0.12 (old) and 3.1.2 (new): cydriver.zip

Below is the commands generated by setuptools with pip install --no-build-isolation --no-deps -v .:

• compile

/local/home/leof/miniforge3/envs/py312_cu129/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include -fPIC -I/local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include -I/local/home/leof/miniforge3/envs/py312_cu129/include/python3.12 -c old_cydriver.cpp -o cydriver.o -std=c++14 -fpermissive -Wno-deprecated-declarations "-D _GLIBCXX_ASSERTIONS" -fno-var-tracking-assignments -O3

• link

/local/home/leof/miniforge3/envs/py312_cu129/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include -pthread -B /local/home/leof/miniforge3/envs/py312_cu129/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/local/home/leof/miniforge3/envs/py312_cu129/lib -Wl,-rpath-link,/local/home/leof/miniforge3/envs/py312_cu129/lib -L/local/home/leof/miniforge3/envs/py312_cu129/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/local/home/leof/miniforge3/envs/py312_cu129/lib -Wl,-rpath-link,/local/home/leof/miniforge3/envs/py312_cu129/lib -L/local/home/leof/miniforge3/envs/py312_cu129/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/local/home/leof/miniforge3/envs/py312_cu129/lib -Wl,-rpath-link,/local/home/leof/miniforge3/envs/py312_cu129/lib -L/local/home/leof/miniforge3/envs/py312_cu129/lib -L/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/lib -L/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/lib/stubs -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /local/home/leof/miniforge3/envs/py312_cu129/include -I/local/home/leof/miniforge3/envs/py312_cu129/targets/x86_64-linux/include build/temp.linux-x86_64-cpython-312/cuda/bindings/cydriver.o -L/local/home/leof/miniforge3/envs/py312_cu129/lib/python3.12/site-packages -L/local/home/leof/miniforge3/envs/py312_cu129/lib -o build/lib.linux-x86_64-cpython-312/cuda/bindings/cydriver.cpython-312-x86_64-linux-gnu.so -Wl,--strip-all