Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stepi vs lossi plot bug fixed in makemore2 mlp notebook #10

Open
wants to merge 23 commits into
base: master
Choose a base branch
from

Conversation

ibadrather
Copy link

To keep track of training steps and loss you initialised lossi and stepi lists. Then while running the training loop multiple times loss was appended to lossi correctly, but you were appending i to stepi and that restarting from 0 again. Hence when you ran the training loop multiple times in the video, the loss was appended but with duplicate steps.

####################################
To keep track of actual steps I did:
step = 0 # initialise

for i in range(50000):
step += 1
stepi.append(step)

instead of:
for i in range(50000):
stepi.append(i)

#####################################

I have uploaded the plots with incorrect and correct step tracking. I ran the training loop 2 times in both cases with 50000 training steps each time. But one plot is only showing 50000 steps instead of 100000 steps.

@ibadrather
Copy link
Author

These are the plots I am talking about. Both have been run for 100000 epochs

loss_with_correct_step
loss_with_wrong_step

Sorry for a messy pull request I am still learning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants