Add entry in dataset card to help fine-tuning using TRL with the generated dataset #1079
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
MathShepherd
tasks.This PR adds a new method to the
_Step
class called_dataset_use
. By default doesn't contain anything, but can be used to enhance the dataset card of aDistiset
. The examples implemented here correspond to the classes that prepare the data for fine-tuning, it includes an example command usingTRL
.To include a new "use" for a dataset associated with a step, we have to override the
_dataset_use
method, take a look atFormatTextGenerationSFT
orFormatTextGenerationDPO
for a example.An example can be seen in this dummy dataset: https://huggingface.co/datasets/plaguss/test_dataset_use