Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

我有多个dataset.json文件在不同目录下，llamafactory是否可以加载多个目录 #6351

Closed
1 task done
lhx692135353 opened this issue Dec 16, 2024 · 1 comment
Closed
1 task done

我有多个dataset.json文件在不同目录下，llamafactory是否可以加载多个目录 #6351

lhx692135353 opened this issue Dec 16, 2024 · 1 comment
Labels
solved This problem has been already solved

Comments

@lhx692135353
Copy link

Reminder

  • I have read the README and searched the existing issues.

System Info

数据集的使用方式应该不需要这个

Reproduction

数据集的使用方式应该不需要这个

Expected behavior

我的数据集是存在多个不同目录下的，他们都有自己的dataset.json文件，如果我想同时使用，是否可以同时加载多个，我不想合并我的dataset文件

Others

No response
@github-actions github-actions bot added the pending This problem is yet to be addressed label Dec 16, 2024
@2792715512
Copy link

  1. 编写脚本新建一个tmp目录将dataset文件 cp 进来
  2. train在指定这个文件夹即可
  3. 注意要把描述文件dataset_info.json也同步生成即可

@hiyouga hiyouga added solved This problem has been already solved and removed pending This problem is yet to be addressed labels Dec 17, 2024
@hiyouga hiyouga closed this as completed Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
solved This problem has been already solved
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hiyouga @lhx692135353 @2792715512