XLNet cached memory/recurrence on segments for fine-tuning

[paultran47]

Hi there,

Apologies if this is a dumb question, but I was curious if the usage of XLNet for fine-tuning utilises the cached memory feature for long sequences.

My preliminary understanding is that XLNet is able to get around the issue of fixed segment lengths that BERT has, and that this discussion in this repo seems to show pretraining was completed with cached memory.

But when using XLNet through simpletransformers, is there a particular setting I need to set for cached memory to be used for fine-tuning? Or is it already done by default?

Thanks!