How are the TiTok VAE models trained?

[Jyouhou]

Great work!

Interested in the details of training the VAE variants.

(1) Are they also trained via the 2-stage procedure?
(2) what losses are used?

[cornettoyu]

Hi,

The new checkpoints (VAE or VQ) are trained with single-stage recipe. Losses include L2, perceptual loss (lpips + convnext), and gan loss. You can find more details in MaskBit and MaskGen tech report. We expect the codebase to be updated with the updated singe-stage training recipe in the near future and please stay tuned :)