Novice Linux user WIS Server Installation notes

[jazzmonger]

I started w/ installing a Virgin Debian 12 on my Dell Optiplex system.
DONT USE DEBIAN 12. IT IS NOT SUPPORTED... I JUST FOUND THAT OUT THE HARD WAY... Im now using Ubuntu 22.04.

A BIG thanks to @nikito Nick Bento for his massive dose of patience in walking me thru this.

I'm not a Linux expert by any stretch, but if I can do this, most anyone can if you have basic Linux skills and aren't afraid of tinkering.
Also, my "new" nVidia 1070 GPU needs a power cable adapter for my PSU so I'll be using the CPU w/out the GPU just to get things going until my power cable adapter gets here in a few days.

The WAS server is easy to install - just follow the guide exactly.
The WIS server is more complex.

Make sure you have the 'curl' command installed. None of my fresh Linux installs had it.

$ sudo apt update && sudo apt upgrade
$ sudo apt install curl

Clone the WIS repo:

$ git clone https://github.com/toverainc/willow-inference-server.git && cd willow-inference-server

if Docker is already installed, DONT run the dependencies script. If Docker is not installed, run

$ ./deps/ubuntu.sh

add your Linux username to the dialout group.

$ sudo adduser yourusername dialout

I install Portainer because its easy to manage Docker Containers. Im a big fan of not using the command line unless I can help it. Use these command to install Portainer:

$ sudo docker pull portainer/portainer-ce:latest
$ sudo systemctl restart docker
$ docker run -d -p 9000:9000 --restart always -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer-ce:latest

Then you MUST run all the post Docker cleanup commands documented here:
https://docs.docker.com/engine/install/linux-postinstall

make sure you can nw access Docker/Portainer via this link. If not, stop now and seek help:
http://yourhostname.local:9000

then run these commands:

$ ./utils.sh install

this install script takes a long time to download and install everything. It is several GB of data. at times during the install, my CPU was at 100% for extended periods of time. Just let it finish, it took about 2 hours on my system and I have Starlink (avg of 50mb download speeds)

Then run

./utils.sh gen-cert myhost.local
./utils.sh run

If all went well, the server should start.
Now you should see console messagess from WIS like this when talking to Box3:

willow-inference-server-wis-1    | [2023-11-30 18:02:17 +0000] [69] [DEBUG] FASTAPI: Got WILLOW request for model medium beam size 1 language detection False
willow-inference-server-wis-1    | [2023-11-30 18:02:17 +0000] [69] [DEBUG] WILLOW: Audio information: sample rate: 16000, bits: 16, channel(s): 1, codec: pcm
willow-inference-server-wis-1    | [2023-11-30 18:02:18 +0000] [69] [DEBUG] WILLOW: Source audio is raw PCM, creating WAV container
willow-inference-server-wis-1    | [2023-11-30 18:02:18 +0000] [69] [DEBUG] WHISPER: Loading audio took 1.1969999999999998 ms
willow-inference-server-wis-1    | [2023-11-30 18:02:18 +0000] [69] [DEBUG] WHISPER: Feature extraction took 45.132 ms
willow-inference-server-wis-1    | [2023-11-30 18:02:18 +0000] [69] [DEBUG] WHISPER: Using system default language en
willow-inference-server-wis-1    | [2023-11-30 18:02:18 +0000] [69] [DEBUG] WHISPER: Using model medium with beam size 1
willow-inference-server-wis-1    | [2023-11-30 18:02:18 +0000] [69] [DEBUG] Processing GPU batch 1 of expected 1
willow-inference-server-wis-1    | [2023-11-30 18:02:39 +0000] [69] [DEBUG] WHISPER: Model took 20263.959 ms
willow-inference-server-wis-1    | [2023-11-30 18:02:39 +0000] [69] [DEBUG] WHISPER: Decode took 0.12999999999999998 ms
willow-inference-server-wis-1    | [2023-11-30 18:02:39 +0000] [69] [DEBUG] WHISPER: ASR transcript:  Two plus two equals 5
willow-inference-server-wis-1    | [2023-11-30 18:02:39 +0000] [69] [DEBUG] WHISPER: Inference took 20311.447 ms
willow-inference-server-wis-1    | [2023-11-30 18:02:39 +0000] [69] [DEBUG] WHISPER: Inference speedup: 0x

Control-C exits the server. $ ./utils.sh run starts it again.

Once its all working, to run permanently instead of in a terminal window:

$ ./utils.sh run -d

then use the docker or Portainer logs to see the output.

Now, I just need a GPU! avg inference times are ~20sec w/ my CPU, an Intel Xeon (circa 2012) running at 2.8GHZ.

[hamishcunningham]

You might try 22.04 which also works well and is a bit more recent

[jazzmonger]

Yup, I got Ubuntu 22.04 installed last night and the Willow server is also installing. That takes quite a while to install so I left it going all night.