Installing CUDA components

Connect to a VM with a GPU, or create a new one if you haven't done so already.
Install the GPU driver if you haven't done so already.
Check whether CUDA Toolkit, CUDA Runtime, CUDA Libraries, and cuDNN are installed:
- CUDA Toolkit:
```
/usr/local/cuda/bin/nvcc --version 2>/dev/null || /usr/local/cuda-*/bin/nvcc --version
```
  A No such file or directory error usually means CUDA Toolkit is not installed.
- CUDA Runtime, CUDA Libraries, and cuDNN:
```
ldconfig -p | grep -E 'libcudart\.so|libcuda\.so|libcublas\.so|libcudnn\.so'
```
  - libcudart.so — CUDA Runtime.
  - libcublas.so — CUDA Libraries.
  - libcuda.so — the driver library. It should be listed if the driver is installed correctly. You can check whether the driver is installed or reinstall it in Installing the Linux GPU driver.
  - libcudnn.so — cuDNN.
  If a library is not listed, the corresponding component is not installed on the system.
If all required components are installed but their version is lower than required, go to Updating CUDA components.

If a component is not installed, choose one of the following scenarios.
- Install only the missing components, using the same version as the components already installed. Use this method if the installed versions suit your needs. You can update them later if needed.
- Reinstall all components from scratch. To do this, remove all components completely, and then return to the installation. Use this method if you need to downgrade the version or perform a clean install from scratch.

Install CUDA Toolkit, CUDA Runtime, CUDA Libraries, and cuDNN from the NVIDIA CUDA repository:

Ubuntu/Debian

Red Hat Enterprise Linux

Determine your distribution version:
```
cat /etc/os-release
```

Download and install the cuda-keyring_1.1-1_all.deb package from the NVIDIA CUDA repository for the corresponding version of your distribution:


wget https://developer.download.nvidia.com/compute/cuda/repos/<DISTRIBUTION><VERSION>/x86_64/cuda-keyring_1.1-1_all.deb && \sudo dpkg -i cuda-keyring_1.1-1_all.deb

Here:

<DISTRIBUTION> — the distribution name in Latin letters.
<VERSION> — the distribution version. Numbers only, without dots.

Examples: debian13, ubuntu2404.

Update the package manager indexes:
```
sudo apt update
```

Make sure the required versions of the CUDA components are available for installation:


apt-cache policy -<MAJOR_VERSION>-<MINOR_VERSION> \   cuda-libraries-<MAJOR_VERSION>-<MINOR_VERSION> \   cuda-runtime-<MAJOR_VERSION>-<MINOR_VERSION> \   cudnn9-cuda-<MAJOR_VERSION>-<MINOR_VERSION>

Here <MAJOR_VERSION> and <MINOR_VERSION> are the major and minor version numbers of the CUDA components. For example, for CUDA Toolkit 12.4, the package name is cuda-toolkit-12-4.

If the command succeeds, the output will include a Candidate field indicating the version to be installed for the specified package.

Install the CUDA components:


sudo apt -y install \   cuda-toolkit-<MAJOR_VERSION>-<MINOR_VERSION> \   cuda-libraries-<MAJOR_VERSION>-<MINOR_VERSION> \   cuda-runtime-<MAJOR_VERSION>-<MINOR_VERSION> \   cudnn9-cuda-<MAJOR_VERSION>-<MINOR_VERSION>

Check that CUDA Toolkit works:
```
nvcc --version
```

If running nvcc --version results in a command not found error, add CUDA Toolkit to the environment variables manually:


sudo tee /etc/profile.d/cuda-path.sh >/dev/null <<'EOF'# Add CUDA Toolkit to PATH (system-wide)CUDA_HOME=/usr/local/cudacase ":$PATH:" in*":$CUDA_HOME/bin:"*) ;;*) export PATH="$CUDA_HOME/bin:$PATH" ;;esacEOF

Check again that CUDA Toolkit works:
```
nvcc --version
```

(Optional) Check again that CUDA Runtime, CUDA Libraries, and cuDNN are installed:


ldconfig -p | grep -E 'libcudart\.so|libcuda\.so|libcublas\.so|libcudnn\.so'

Was this article helpful?