issues about prediction on "Ludwig + DeepLearning.ai: Efficient Fine-Tuning for Llama2-7b on a Single GPU.ipynb"

[leeywin]

I tried the "Ludwig + DeepLearning.ai: Efficient Fine-Tuning for Llama2-7b on a Single GPU.ipynb" in Colab, (https://colab.research.google.com/drive/1Ly01S--kUwkKQalE-75skalp-ftwl0fE?usp=sharing ) and it seemed to work well including the training till the prediction step, which could not make a right prediction as below. What could be the issue?

[justinxzhao]

Hi @leeywin,

Thanks for giving the notebook a try! It looks like there's a ton of extra (and repetitive) logging statements.

You'll find the predictions if you scroll all the way to the bottom of the output cell.

Instruction: Create an array of length 5 which contains all even numbers between 1 and 10.
Input: 
Generated Output: arr = [2, 4, 6, 8, 10]



Instruction: Create an array of length 15 containing numbers divisible by 3 up to 45.
Input: 
Generated Output: arr = [3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45]



Instruction: Create a nested loop to print every combination of numbers between 0-9
Input: 
Generated Output: for i in range(10):
    for j in range(10):
        print(i, j)



Instruction: Generate a function that computes the sum of the numbers in a given list
Input: 
Generated Output: def sum_list(numbers):
    total = 0
    for num in numbers:
        total += num
    return total



Instruction: Create a class to store student names, ages and grades.
Input: 
Generated Output: class student(object):
    def __init__(self, name, age, grade):
        self.name = name
        self.age = age
        self.grade = grade



Instruction: Print out the values in the following dictionary.
Input: my_dict = {
  'name': 'John Doe',
  'age': 32,
  'city': 'New York'
}
Generated Output: john doe (32 years old from new york)

Let me know if you find the correct outputs. I will look into cleaning up these logs in the meantime.

[leeywin]

Hi, @justinxzhao ,

Thank you for getting back to me. I tried it many times and once again, but actually, no correct output on the bottom, which is weird.

Could you help to figure it out?

[justinxzhao]

Hi @leeywin,

It's possible that the model needs to train for longer. Can you double check that the fine-tuning configuration you are using is:

trainer:
  type: finetune
  epochs: 1
  batch_size: 1
  eval_batch_size: 2
  gradient_accumulation_steps: 16
  learning_rate: 0.0004
  learning_rate_scheduler:
    warmup_fraction: 0.03

You might have been using a version of the notebook with train_steps: 5 (if you refresh the notebook, you should get an updated configuration). This is a good setting for quick tests, but it also means that Ludwig will only fine-tune the model for 5 training steps, which explains why the outputs you are seeing in the notebook aren't good.

Fine-tuning for 1 epoch on a T4 takes about 15 minutes.

Let me know if using epochs: 1 in the config produces better predictions.

[leeywin]

Cool, man, it works! That is exactly the issue you talked about.

Thank you^