infra-phytron/server/phy-z-srv-gpu01/nvidia-driver-install.md

Manual nvidia driver, cuda driver and container toolkit

NVIDIA driver

Check if GPUs are recognized by the base OS:

sudo lspci | grep -i nvidia

Which should some output if it finds nvidia deivces.

Search for required drivers for your GPUs:

sudo ubuntu-drivers devices

Automatically install all drivers:

sudo ubuntu-drivers autoinstall

Reboot the system for changes to take effect:

sudo reboot

Shot GPU stats with:

nvidia-smi

Cuda driver

Disable Secure Boot in BIOS

Install Cuda drivers:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb  
sudo apt update  
sudo apt -y install cuda-toolkit-12-8  
sudo apt install -y cuda-drivers

Container toolkit

Install the Nvidia Container toolkit:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
apt update  
apt install -y nvidia-container-toolkit

Test a simple cuda container and nvidia-smi command inside:

docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi