Qustions about dynamic NTK interpolation fine-tuning and non-linear interpolation methods

[Yiyi-philosophy]

Hello LongLora Team,

I have been following your work with great interest, particularly regarding the dynamic NTK. While reviewing the code and paper, I noticed that the current experimental results are primarily based on the PI (linear interpolation) method. I am curious to know if you have conducted experiments on interpolation fine-tuning with dynamic NTK. If so, I would be interested in learning more about the corresponding results and findings.

Additionally, I am intrigued by the possibility of experiments with non-linear interpolation methods such as YaRN and NTK. Have you explored similar approaches, and if there are any relevant experiments or findings, I would like to know more about them.

Thank you for your time and efforts!

[yukang2017]

Hi,

Many thanks for your question. Actually, we have not conducted these experiments yet.

Regards,
Yukang Chen