-
-
Notifications
You must be signed in to change notification settings - Fork 844
-
-
Notifications
You must be signed in to change notification settings - Fork 844
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug in new wkv6state_cuda #241
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
RWKV-LM/RWKV-v5/cuda/wkv6state_cuda.cu
Line 15 in 666f645
This line is missing the batch offset and should read:
_s += b*H*_N_*_N_ + h*_N_*_N_ + i*_N_;
Probably why this code didn't work for BPTT when we tried it a while back!
The text was updated successfully, but these errors were encountered: