How to enable GPU for AI in Linux
What you require
1. OS: Ubuntu 22.04
2. Nvidia GPU
3. Driver Version: 550 (check if the driver is installed correctly using: lspci | grep -i nvidia)
4. Update docker-compose config
NVIDIA driver installation
Install CUDA Keyring and update the system
sudo apt-get update && apt-get upgrade -y
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
*If Secure boot is enabled, use signed drivers (550-open)
Install the CUDA Toolkit and required drivers
sudo apt-get -y install cuda-toolkit-12-4
sudo apt-get install -y nvidia-driver-550-open
sudo apt-get install -y cuda-drivers-550
Install NVIDIA Docker
sudo apt-get install -y nvidia-docker2
Edit Docker daemon.json to use NVIDIA runtime
nano /etc/docker/daemon.json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "3"
},
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
sudo systemctl restart docker
export PATH=/usr/local/cuda-12.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:$LD_LIBRARY_PATH
Prepare the VMS
Amend the ai part in docker-compose.yml
nano docker-compose.yml
ai:
image: vcloudaiorg/vcloudai-vms-ai:latest
container_name: ai
restart: always
privileged: true
volumes:
- ./static/customAIModels:/app/Models/customAIModels
- ./static/customAIstatic:/app/static
- ./static/customAIStorage:/app/Storage
- /etc/timezone:/etc/timezone
- /etc/localtime:/etc/localtime
network_mode: host
depends_on:
- appback
- appfont
environment:
MYSQL_HOST: ${MACHINE_HOST}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
MYSQL_DB: ${MYSQL_DB}
EXTERNAL_MYSQL_PORT: ${EXTERNAL_MYSQL_PORT}
MACHINE_HOST: ${MACHINE_HOST}
APP_HOST: ${APP_HOST}
SERVER_PORT: ${LOCAL_PORT}
SECRET_WORD: ${SECRET_WORD}
AI_PORT: ${AI_PORT}
NVIDIA_VISIBLE_DEVICES: all
NVIDIA_DRIVER_CAPABILITIES: all
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
Save and restart ai container
docker-compose down ai
docker-compose up ai -d