Can you support fast resume with streaming option?

[JonghwanMun]

Reminder

• I have read the README and searched the existing issues.

System Info

No response

Reproduction

The problem is too slow resume when using data with streaming option.

• This is mainly because streaming option makes the data as iterable dataset.
• Then, resume with iterable dataset requires the data skip process (until the saved last iterations) that takes a lot of times.
• Actually, this is an known issue and there are some efforts to address this issue:
  • discussion
  • One possible solution with Stream
  • Another solution of using StatefulDataLoader
  • Other solution: integration of this PR

Can you add this feature into LLaMA-Factory?

Expected behavior

Add a feature of supporting fast resume with streaming data.

Others

No response