Higher GPU memory usage when setting load_text_encoder_in_8bit to true

[bryanyzhu]

In this line of ltxv_2b_full.yaml , it shows

load_text_encoder_in_8bit: true # Load text encoder in 8-bit precision to save memory

However, if I set it to false, it actually reduces the memory. For example, for some small resolution video training samples, setting this false will only take 27GB memory during training, but setting this to true will actually increase to 31GB memory, which conflicts with the comment.

In addition, setting this to true or false impact the training pipeline,

• when setting to true: the pipeline first do validation, then training
• when setting to false: the pipeline directly go to training, then validation

Can anyone help to explain why?