Skip to content

Latest commit

 

History

History
372 lines (279 loc) · 16.5 KB

setup-guide.md

File metadata and controls

372 lines (279 loc) · 16.5 KB
Raw

Setup Guide

This guide covers the setup process for onnx-web, including downloading the Windows bundle.

Contents

Cross-platform method

This works on both Linux and Windows, for both AMD and Nvidia, but requires some familiarity with the command line.

Install Git and Python

Install Git and Python 3.10 for your environment:

The latest version of git should be fine. Python should be 3.9 or 3.10, although 3.8 and 3.11 may work if the correct packages are available for your platform. If you already have Python installed for another form of Stable Diffusion, that should work, but make sure to verify the version in the next step.

Make sure you have Python 3.9 or 3.10:

> python --version
Python 3.10

If your system differentiates between Python 2 and 3 and uses the python3 and pip3 commands for the Python 3.x tools, make sure to adjust the commands shown here. They should otherwise be the same: python3 --version.

Once you have those basic packages installed, clone this git repository:

> git clone https://github.com/ssube/onnx-web.git

Note about setup paths

This project contains both Javascript and Python, for the client and server respectively. Make sure you are in the correct directory when working with each part.

Most of these setup commands should be run in the Python environment and the api/ directory:

> cd api
> pwd
/home/ssube/code/github/ssube/onnx-web/api

The Python virtual environment will be created within the api/ directory.

The Javascript client can be built and run within the gui/ directory.

Create a virtual environment

Change into the api/ directory, then create a virtual environment:

> pip install virtualenv
> python -m venv onnx_env

This will contain all of the pip libraries. If you update or reinstall Python, you will need to recreate the virtual environment.

If you receive an error like Error: name 'cmd' is not defined, there may be a bug in the venv module on certain Debian-based systems. You may need to install venv through apt instead:

> sudo apt install python3-venv   # only if you get an error

Every time you start using onnx-web, activate the virtual environment:

# on linux:
> source ./onnx_env/bin/activate

# on windows:
> .\onnx_env\Scripts\Activate.bat

Update pip itself:

> python -m pip install --upgrade pip

Install pip packages

You can install all of the necessary packages at once using the requirements/base.txt file and the requirements/ file for your platform. Install them in separate commands and make sure to install the platform-specific packages first:

> pip install -r requirements/amd-linux.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/amd-windows.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/cpu.txt
> pip install -r requirements/base.txt
# or
> pip install -r requirements/nvidia.txt
> pip install -r requirements/base.txt

Only install one of the platform-specific requirements files, otherwise you may end up with the wrong version of PyTorch or the ONNX runtime. The full list of available ONNX runtime packages can be found here .

If you have successfully installed both of the requirements files for your platform, you do not need to install any of the packages shown in the following sections and you should skip directly to testing the models.

The ONNX runtime nightly packages used by the requirements/*-nightly.txt files can be substantially faster than the last release, but may not always be stable. Many of the nightly packages are specific to one version of Python and some are only available for Python 3.8 and 3.9, so you may need to find the correct package for your environment. If you are using Python 3.10, download the cp310 package. For Python 3.9, download the cp39 package, and so on. Installing with pip will figure out the correct package for you.

For AMD on Linux: PyTorch ROCm and ONNX runtime ROCm

If you are running on Linux with an AMD GPU, install the ROCm versions of PyTorch and onnxruntime:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/rocm5.2
# and one of
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.14.1%2Brocm54-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
# or
> pip install https://download.onnxruntime.ai/onnxruntime_training-1.15.0.dev20230326001%2Brocm542-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Make sure you have installed ROCm 5.x (see their documentation for more details) and that the version of onnxruntime matches your ROCm drivers. The version of PyTorch does not need to match exactly, and they only have limited versions available.

Ubuntu 20.04 supports ROCm 5.2 and Ubuntu 22.04 supports ROCm 5.4, unless you want to build custom packages. The ROCm 5.x series supports many discrete AMD cards since the Vega 20 architecture, with a partial list of supported cards shown here.

For AMD on Windows: PyTorch CPU and ONNX runtime DirectML

If you are running on Windows with an AMD GPU, install the DirectML ONNX runtime as well:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
# and one of
> pip install onnxruntime-directml
# or
> pip install ort-nightly-directml --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall

If you DirectML package upgrades numpy to an incompatible version >= 1.24, downgrade it:

> pip install "numpy>=1.20,<1.24" --force-reinstall  # the DirectML package will upgrade numpy to 1.24, which will not work

You can optionally install the latest DirectML ORT nightly package, which may provide a substantial performance increase.

For CPU everywhere: PyTorch CPU and ONNX runtime CPU

If you are running with a CPU and no hardware acceleration, install onnxruntime and the CPU version of PyTorch:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cpu
# and
> pip install onnxruntime
# or
> pip install ort-nightly --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall

For Nvidia everywhere: Install PyTorch GPU and ONNX GPU

If you are running with an Nvidia GPU on any operating system, install onnxruntime-gpu and the CUDA version of PyTorch:

> pip install "torch==1.13.1" "torchvision==0.14.1" --extra-index-url https://download.pytorch.org/whl/cu117
# and
> pip install onnxruntime-gpu
# or
> pip install ort-nightly-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ --force-reinstall

Make sure you have installed CUDA 11.x and that the version of PyTorch matches the version of CUDA (see their documentation for more details).

Download and convert models

Before continuing, you will need to download or convert at least one Stable Diffusion model into the ONNX format.

Some pre-converted models are available in the models/preconverted-*.json files, including Stable Diffusion v1.5, SDXL, and SDXL Turbo.

You can also download and extract the models yourself. For example:

> wget -O ../models/.cache/stable-diffusion-v1-5-fp32.zip https://models.onnx-files.com/stable-diffusion-v1-5-fp32.zip

...
Saving to: ‘../models/.cache/stable-diffusion-v1-5-fp32.zip’
...

> unzip ../models/.cache/stable-diffusion-v1-5-fp32.zip -d ../models/stable-diffusion-onnx-v1-5

Archive:  ../models/.cache/stable-diffusion-v1-5-fp32.zip
   creating: ../models/stable-diffusion-onnx-v1-5/vae_encoder/
  inflating: ../models/stable-diffusion-onnx-v1-5/vae_encoder/model.onnx
  inflating: ../models/stable-diffusion-onnx-v1-5/LICENSE.txt
  inflating: ../models/stable-diffusion-onnx-v1-5/README.txt
  ...

> file ../models/stable-diffusion-onnx-v1-5/model_config.json

../models/stable-diffusion-onnx-v1-5/model_index.json: JSON data

Note that the included --base models and the pre-converted models have different folder names. This is intentional, so they don't conflict with each other during testing.

Test the models

You should verify that all of the steps up to this point have worked correctly by attempting to run the api/scripts/test-diffusers.py script, which is a slight variation on the original txt2img script.

If the script works, there will be an image of an astronaut in outputs/test.png.

If you get any errors, check the known errors section of the user guide.

Download the web UI bundle

Once the server environment is working, you will need the latest files for the web UI. This is a Javascript bundle and you can download a pre-built copy from Github or compile your own.

From the gh-pages branch, select the version matching your server and download all three files:

  • bundle/main.js
  • config.json
  • index.html

Copy them into your local api/gui folder. Make sure to keep the main.js bundle in the bundle subfolder and copy the files into the gui folder within the api folder, not the gui folder in the root of the repository.

For example, for a v0.12.0 server, copy the files from https://github.com/ssube/onnx-web/tree/gh-pages/v0.12.0 into your local copy of https://github.com/ssube/onnx-web/tree/main/api/gui and https://github.com/ssube/onnx-web/tree/main/api/gui/bundle.

Launch the server

After you have confirmed that you have a working environment, launch the server using one of the provided launch scripts in the api folder:

# on Linux
> ./launch.sh

# on Windows
> .\launch.ps1

This will download and/or convert any missing models, then launch an API server on port :5000. If the server starts up correctly, you should see an admin token in the logs:

[2023-12-31 13:46:16,451] INFO: MainProcess MainThread onnx_web.main: all plugins loaded successfully
[2023-12-31 13:46:16,466] INFO: MainProcess MainThread onnx_web.server.load: available acceleration platforms: any - CPUExecutionProvider (None), cpu - CPUExecutionProvider (None)
[2023-12-31 13:46:16,494] INFO: MainProcess MainThread onnx_web.main: starting v0.12.0 API server with admin token: RANDOM-TOKEN

If you see any errors about the port already being in use, make sure you are not running any other servers or programs that use port 5000, or change the --port argument in the launch script. If you change the port, make sure to use that new port in any other commands that you run.

Open the web UI

With the server running, open the web UI in your favorite web browser. If you are running the server locally, the UI will be available at http://localhost:5000/.

If you are running the server on a different computer, you will need to use that computer's IP address or local DNS name and provide that same address in the ?api argument. For example, with a server running on a remote computer at 10.2.2.100 and using port 5001, the URL would be http://10.2.2.100:5001?api=http://10.2.2.100:5001.

You can change the ?api argument to use multiple servers while keeping your state and results. Note that changing the server while an image is generating will cause it to fail in the web UI, since the new server will not be aware of that image.

Windows-specific methods

These methods are specific to Windows, tested on Windows 10, and still experimental. They should provide an easier setup experience.

Windows all-in-one bundle

  1. Install the latest Visual C++ 2019 redistributable
    1. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
    2. https://aka.ms/vs/17/release/vc_redist.x64.exe
  2. Download the latest ZIP file from the apextoaster Nexus server
  3. Find the ZIP file and Extract All to a memorable folder
  4. Open the folder where you extracted the files
    1. Your models will be converted into the models folder, and you can add your own models
    2. Your images will be in the outputs folder, along with files containing the parameters used to generate them
  5. Make sure the server is allowed to run
    1. Open the server folder
    2. Right-click the onnx-web.exe file and click Properties
    3. On the General tab, click Unblock next to the message This file came from another computer and might be blocked to help protect this computer.
    4. Go back to the folder where you extracted the files
    5. Repeat step 3 for the onnx-web-*.bat files
  6. Run the local server using one of the onnx-web-*.bat scripts
    1. Run onnx-web-half.bat if you are using a GPU and you have < 12GB of VRAM
      • -half mode is compatible with both AMD and Nvidia GPUs
      • -half mode is not compatible with CPU mode
    2. Run onnx-web-full.bat if you are using CPU mode or if you have >= 16GB of VRAM
      • Try the onnx-web-half.bat script if you encounter out-of-memory errors or generating images is very slow
  7. Wait for the models to be downloaded and converted
    1. Most models are distributed in PyTorch format and need to be converted into ONNX format
    2. This only happens once for each model and takes a few minutes
  8. Open one of the URLs shown in the logs in your browser
    1. This will typically be http://127.0.0.1:5000?api=http://127.0.0.1:5000
    2. If you running the server on a different PC and not accessing it from a browser on the same system, use that PC's IP address instead of 127.0.0.1
    3. Any modern browser should work, including Chrome, Edge, and Firefox
    4. Mobile browsers also work, but have stricter mixed-content policies

Windows Python installer

  1. Install the latest Visual C++ 2019 redistributable
    1. https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170
    2. https://aka.ms/vs/17/release/vc_redist.x64.exe
  2. Install Git
  3. Install Python 3.10
  4. Clone or download the onnx-web repository
  5. Open a command prompt window
  6. Run one of the setup-*.bat scripts
    1. Run setup-amd.bat if you are using an AMD GPU and DirectML
    2. Run setup-nvidia.bat if you are using an Nvidia GPU and CUDA
    3. Run setup-cpu.bat if you are planning on only using CPU mode
  7. After the first run, you can run launch.bat instead of the setup script
    1. You should only need to run the setup script once
    2. If you encounter any errors with Python imports, run the setup script again