How to install CUDA & cuDNN

Setting up CUDA & cuDNN for Machine Learning can be an overwhelming process. In this guide, I will walk you through the steps to install CUDA and cuDNN on your system, ensuring your machine is correctly set up for deep learning tasks. 

System Configuration:

• Operating System: Ubuntu 22.4
• GPU: GeForce RTX 3090
• ML Framework: Pytorch

Install NVIDIA drivers

Update & Upgrade

sudo apt update && sudo apt upgrade

Remove previous NVIDIA installation

Uninstall the previous NVIDIA and CUDA installation to avoid messing up the new installation.

sudo apt-get remove --purge -y '*nvidia*' '*cuda*' 'libcudnn*' 'libnccl*' '*cudnn*' '*nccl*'

sudo apt-get autoremove --purge -y

sudo apt-get clean

Check if there are any remaining packages or files.

dpkg -l | grep -E 'nvidia|cuda|cudnn|nccl'

Detecting and Managing Drivers on Ubuntu

ubuntu-drivers devices

We will install the NVIDIA driver tagged recommended - Which indicates which drivers are recommended for each piece of hardware based on compatibility and performance.

Install Ubuntu drivers

sudo ubuntu-drivers autoinstall

Install NVIDIA drivers

My recommended version is 555, change "XYZ" in the following command to your recommended driver.

sudo apt install nvidia-driver-XZY

Reboot the system for these changes to take effect.

reboot

Check Installation

After reboot verify that the following command works:

nvidia-smi

Install CUDA drivers

Update & Upgrade

Again, check for updates on your OS.

sudo apt update && sudo apt upgrade

Install CUDA Toolkit

At the moment of writing this text, the newest CUDA version supported by Pytorch is 12.1.

You can find older versions in the CUDA Toolkit Archive. In my case, I will be continuing with CUDA Toolkit 12.1.1 (April 2023).

You will need to select your operating system (Linux in my case). Afterwards, you will be prompted to select the Architecture. If you are not sure what is the Architecture of your PC you can use the command below (in my case the Architecture is x86_64).

uname -m

Next, we need to select the distribution and version of our operating system, in my case Ubuntu 22.04. Lastly, I am using the deb (local) installer type.

These are the commands for installing CUDA Toolkit 12.1:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin

sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb

sudo dpkg -i cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb

sudo cp /var/cuda-repo-ubuntu2204-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update

sudo apt-get -y install cuda-12-1

⚠️ NOTE: The last command differs from the one on the CUDA installation page. I added “-12–1” to specify the CUDA version to install.

Check CUDA install

nvcc --version

---

If you are not getting the CUDA version as output, do the following:

• Ensure that CUDA 12.1 is installed in the correct directory, typically /usr/local/cuda-12.1.
• Create a symlink to the CUDA directory to make it easier to reference.

  sudo ln -s /usr/local/cuda-12.1 /usr/local/cuda

• Add the CUDA paths to your .bashrc file to ensure they are set up every time you open a terminal.

  echo 'export PATH=/usr/local/cuda-12.1/bin:$PATH' >> ~/.bashrc

• Apply the changes made to the .bashrc

  source ~/.bashrc

Install cuDNN

Download the cuDNN .deb file

Firstly, to match appropriate cuDNN with our CUDA and Driver versions we can use the table provided by NVIDIA. In my case, the newest 9.2 cuDNN version is appropriate.

You can download the cuDNN file here, for which you will need an Nvidia account. Same as for CUDA, you will have to select OS, architecture, distribution, version, and installer type.

In my case, to Download Installer for Linux Ubuntu 22.04 x86_64 instructions are as follows:

wget https://developer.download.nvidia.com/compute/cudnn/9.2.0/local_installers/cudnn-local-repo-ubuntu2204-9.2.0_1.0-1_amd64.deb

sudo dpkg -i cudnn-local-repo-ubuntu2204-9.2.0_1.0-1_amd64.deb

sudo cp /var/cudnn-local-repo-ubuntu2204-9.2.0/cudnn-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update

sudo apt-get -y install cudnn-cuda-12

Test CUDA with PyTorch

I assume that you already have Python installed on your machine, so this tutorial would not be covering that part.

⚠️ NOTE: The latest PyTorch requires Python 3.8 or later.

Create a Directory

mkdir test_cuda

cd test_cuda

Create and Activate a Virtual Environment

python -m venv .venv

source .venv/bin/activate

Install PyTorch

pip3 install torch torchvision torchaudio

Execute Python test script

import torch


# Check PyTorch version
print(f"PyTorch version: {torch.__version__}")
# Check if CUDA device is recognised
print(f"CUDA device is available: {torch.cuda.is_available()}")
assert torch.cuda.is_available(), ValueError("CUDA device is not recognised.")

device = "cuda"
a = torch.rand(3, 1).to(device)
b = torch.rand(1, 3).to(device)

c = a @ b

print(f"Variable shape: {c.shape}")
print(f"Variable devie: {c.device}")

Conclusion

If you carefully followed these instructions, you have successfully installed CUDA and cuDNN on your Ubuntu 22.4 system. Your NVIDIA GPU is now ready for deep learning tasks with PyTorch.

You can also view this guide on Medium. If this guide helped you give it a 👏, share it, and give it a ⭐️ on GitHub.