GeForce GTX1060 w/6GB RAM - not detected as gpu?

[keith721]

Intel i5 Socket 1700, Intel H610 chipset, GeForce GTX 1060 6GB - GPU not detected by docker process?

 => exporting to image                                                                                                                               0.0s
 => => exporting layers                                                                                                                              0.0s
 => => writing image sha256:0e1665e012f1b0893c2c1f6c5a0f37dae7c75f076dcab5881546e08777d64037                                                         0.0s
 => => naming to docker.io/library/willow-inference-server:latest                                                                                    0.0s
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].
willow@willow:~/willow-inference-server$

Detected by nvidia-smi:
willow@willow:~/willow-inference-server$ **nvidia-smi**
Thu Jul 20 00:37:34 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 1060 6GB    Off | 00000000:01:00.0 Off |                  N/A |
| 39%   36C    P0              27W / 120W |      0MiB /  6144MiB |      3%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

willow@willow:~/willow-inference-server$ **sudo lspci | grep VGA**
[sudo] password for willow:
01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] (rev a1)

What should I add to my .env file to get past this issue? Thanks in advance!!

[nikito]

When I had this problem I had to re-run the deps/ubuntu.sh script, which redownloaded the updated nvidia docker toolkit.

[keith721]

Nope, re-ran it, now it's complaining about

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: mount error: failed to add device rules: unable to find any existing device filters attached to the cgroup: bpf_prog_query(BPF_CGROUP_DEVICE) failed: operation not permitted: unknown.

I do understand, this is currently pre-release, and I only got this system assembled and loaded tonight. So I'm in no big rush, just trying to better understand the build process, especially from the minimal GTX 1060 perspective. Thanks for your quick response!

[nikito]

What OS are you using? Could also try a reboot for the heck of it. I only saw this with myself after upgrading my Nvidia drivers, but running the deps script resolved it for me. 🫤

[nikito]

Also assume you are running utils.sh install after running the ubuntu.sh script?

[keith721]

Ubuntu Server 20.04 LTS - rebooted a few times, no resolution. Yes, definitely running utils.sh install. Maybe wait a few more days, see what else may come up. Thanks again for posting. I can re-load the operating system to a non-server/desktop version, but really don't want a GUI desktop of any kind. Just a lightweight server os.

[nikito]

Yeah I am running headless Ubuntu server 22.04 as well, no gui and working fine here, same driver version as you as well I believe.

[keith721]

I'll probably clean the directory and re-download, then try it again, then.

--> NOPE! Still complaining. I'll just patiently await the 1.0 release, most likely.

[nikito]

Only other thing I can think to try is to uninstall the Nvidia container toolkit and reinstall, (or use ubuntu.sh to do it for you) and can also try a apt-get update and apt-get upgrade. Not sure if what else to try 😕

[kristiankielhofner]

Try testing your installation with the base Nvidia CUDA container:

docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu20.04 nvidia-smi

Also FYI: we've only done extensive testing with driver versions 525 and 530. Likely not related to this issue but wanted to mention it.

[keith721]

Tried that, still got the same 'shim' error attempting to start the image within the container.
Tried changing that command to cuda:12.2.0-base-ubuntu22.04, same error.

I'm going to reload Ubuntu Server 22.04 and start over. I was beginning to think it was the Docker install script, but Docker mostly works fine, from what I can tell. I've tried it in two different Docker configurations: (a) rootless and (b) with the willow user added to the Docker group; no difference, same error. Will also fall back to the recommended nVidia driver version of 530.

[keith721]

Well, hallelujah!! 💯 Apparently, starting fresh helped greatly. Currently loading the models. 👍

Suggested steps to add to Readme.md:
a) add your Willow user that will run the server to the Docker group:
sudo usermod -aG docker willow

b) Install the nvidia drivers:
sudo apt-get install nvidia-driver-535

Then run ./deps/ubuntu.sh

[hamishcunningham]

I think I've seen this type of error when I installed the nvid docker tools without using the ubuntu.sh script and maybe didn't delete other versions first or something...