Installing NVIDIA Container Toolkit

Connect to a VM with a GPU, or create a new one if you haven't done so already.

Install Docker:


curl -fsSL https://get.docker.com | shsudo usermod -aG docker $USER && newgrp docker

Install NVIDIA Container Toolkit:

Ubuntu/Debian

Red Hat Enterprise Linux

Install the required dependencies:


sudo apt update &&sudo apt-get update && sudo apt-get install -y --no-install-recommends \ca-certificates \curl \gnupg2

Add the NVIDIA Container Toolkit repository:


curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Install NVIDIA Container Toolkit:


sudo apt update &&sudo apt install -y nvidia-container-toolkit

Configure Docker to work with NVIDIA Container Toolkit:
```
sudo nvidia-ctk runtime configure --runtime=docker
```
Running the command modifies the /etc/docker/daemon.json file to work with NVIDIA Container Toolkit.

Restart Docker:


sudo systemctl restart docker

(Optional) Check that the GPU works inside a container:
```
docker run --rm --gpus all <DOCKER_IMAGE> nvidia-smi
```
Here <DOCKER_IMAGE> is the exact name of the required image from Docker Hub in the format nvidia/cuda/<IMAGE_NAME>. For example, nvidia/cuda/12.4.0-devel-ubuntu22.04.

If the check succeeds, information about the GPU and driver on the system will be displayed.

Was this article helpful?