Fine-tuned Model Quality

[nichellehouston]

I fine tuned gemma 2b and 9b to translate language with json dataset
{
"instruction": "Translation text from english into french",
"input": "english word or Sentence",
"output": "french word or Sentence"
},
after fine tune finished copy/paste one of word or Sentence used in dataset but translate wrong.I use different dataset size 1k-50k and Settings.
batch_size=1-2-4
gradient_accumulation_steps=2-4-8
num_train_epochs=3-4
lora_rank=8-16-32
lora_alpha=16-32

training_loss report:

[hiyouga]

see FAQs