Run a docker container with GPU

Background: I was use TGI from HuggingFace to serve a local LLM and was getting an error that the docker container couldn't access the GPU, specifically a driver was missing with the gpu capability:

model=casperhansen/deepseek-r1-distill-qwen-14b-awq                                                                                  
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run                                   
docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data     ghcr.io/huggingface/text-generation-inference:3.0.1     --model-i
d $model                                                                                                                                                      
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].  

Searching this led me here but the instructions were a bit lacking, from looking here they should be:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt get update
sudo apt-get install -y nvidia-container-toolkit
sudo apt get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker
# docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data     ghcr.io/huggingface/text-generation-inference:3.0.1     --model-id $model

My command then worked and the GPU engaged