Slow convergence on training on toy dataset that you provided

[iLori-Jiang]

First of all, thanks for your amazing work! @lllyasviel

However, I met the problem that as I followed your steps to retrain the model to fill the circle with your toy dataset, the model converges really slow.

Below is the sampling result of 3900 steps with batch size of 4 (all the parameters remain unchanged as your tutorial_train.py).

G.T. [reconstruction_gs-003900_e-000000_b-003900]:

Output [samples_cfg_scale_9 00_gs-003900_e-000000_b-003900]:

The model seems to be able to understand the color, but cannot understand the position of the circle.

\

While continuing the training, the sampling result of 11875 steps:

G.T. [reconstruction_gs-011875_e-000001_b-003000]:

Output [samples_cfg_scale_9 00_gs-011875_e-000001_b-003000]:

The model finally learns the position of the circle, but seems no longer understand the color anymore.

Do you have any insights on this problem, or do you have any instructions on helping me solve this? Thank you in advance!

[Neltherion]

I'm also facing this problem. Did further training improve anything? or did you have to change the code somehow?🤔

[crapthings]

@iLori-Jiang

wait until 25000 steps

i've tested this dataset and 5w other dataset and 3w dataset.
they all work after > 18000

this use same circle dataset, just run the script and wait
https://github.com/huggingface/diffusers/tree/main/examples/controlnet