What are the specs for training a ControlNet?

[CarstenHoyer]

Currently running out of memory running the tool_add_control.py on my localmachine, which is probably understandable, but I was wondering: What are the minimal requirements for running a training session like Fill50K?

My system can't handle it. I have a 16GB GPU and 16GB RAM.

[geroldmeisinger]

HuggingFace Diffusers ControlNet training script https://huggingface.co/docs/diffusers/training/controlnet uses different performance optimizations
min 8GB on Linux with DeepSpeed (which is only available on Linux right now), and 12GB on Windows

duplicate of #21