Table of Contents
Docker - GPU - Use NVIDIA CUDA Toolkit within a Docker container
Installation of NVIDIA Drivers
Ensure that the NVIDIA drivers are correctly installed on the host system.
- These drivers act as the bridge between the operating system and the NVIDIA GPU hardware, ensuring optimal communication and performance.
See: Setup.
Install the NVIDIA Container Toolkit
This toolkit extends Docker to leverage NVIDIA GPUs fully, ensuring that the GPU capabilities can be used within containers without any hitches.
Download the NVIDIA GPG key
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey -o /tmp/nvidia-gpgkey
Dearmor the GPG key and save it
gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg /tmp/nvidia-gpgkey
Download the NVIDIA container toolkit list file
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list -o /tmp/nvidia-list
Modify the list file to include the signature
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' /tmp/nvidia-list > /etc/apt/sources.list.d/nvidia-container-toolkit.list
Update the package database
apt update
Configuring Docker for NVIDIA Support
Having the NVIDIA Container Toolkit in place, the next essential task is configuring Docker to recognize and utilize NVIDIA GPUs.
Configure the Docker runtime to use NVIDIA Container Toolkit by using the nvidia-container-cli command.
- The Docker configuration file will be modified to use the NVIDIA runtime
nvidia-container-cli configure --runtime=docker
NOTE: Behind the scenes, this command makes alterations to the /etc/docker/daemon.json file.
- As a result, Docker becomes aware of the NVIDIA runtime and can access GPU features.
Restart the Docker daemon
systemctl restart docker
Running the NVIDIA CUDA Docker Image
With all the required setups in place, the exciting part begins: running a Docker container with NVIDIA GPU support.
NVIDIA maintains a series of CUDA images on Docker Hub.
Pull the specific NVIDIA CUDA image:
docker pull nvidia/cuda:12.2.0-base-ubuntu22.04
NOTE: Always check for the latest tags at NVIDIA CUDA Docker Hub to stay updated.
- Use that instead of the 12.2.0-base-ubuntu22.04 tag shown here.
Run the Docker container with GPU support
docker run --gpus all -it nvidia/cuda:12.2.0-base-ubuntu22.04 bash
NOTE: This command runs the Docker container with full GPU access (–gpus all) and provides an interactive shell inside the container.
- Once inside, use NVIDIA utilities like nvidia-smi to confirm GPU access.