You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the notebook makemore_part4_backprop.ipynb, there is a small issue with the calculation of dbnmeani in the following line:
dbnmeani= (-dbndiff).sum(0)
This should be corrected to:
dbnmeani= (-dbndiff).sum(0, keepdim=True)
The current implementation without keepdim results in dbnmeani.shape being [64], whereas bnmeani.grad.shape is [1, 64]. While for cmp() torch.tensor([1, 2, 3]) with shape [3] is equivalent to torch.tensor([[1, 2, 3]]) with shape [1, 3], the lack of keepdim here can cause confusion due to the shape mismatch.
The text was updated successfully, but these errors were encountered:
conscell
added a commit
to conscell/nn-zero-to-hero
that referenced
this issue
Dec 27, 2024
In the notebook
makemore_part4_backprop.ipynb
, there is a small issue with the calculation ofdbnmeani
in the following line:This should be corrected to:
The current implementation without
keepdim
results indbnmeani.shape
being[64]
, whereasbnmeani.grad.shape
is[1, 64]
. While for cmp()torch.tensor([1, 2, 3])
with shape [3] is equivalent totorch.tensor([[1, 2, 3]])
with shape [1, 3], the lack ofkeepdim
here can cause confusion due to the shape mismatch.The text was updated successfully, but these errors were encountered: