-
Notifications
You must be signed in to change notification settings - Fork 26.3k
-
Notifications
You must be signed in to change notification settings - Fork 26.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_pt_flax_equivalence and test_encoder_decoder_model_standalone fail running on device (cuda or xpu) #33517
Labels
Comments
dvrogozh
added a commit
to dvrogozh/transformers
that referenced
this issue
Sep 16, 2024
This commit fixes the following errors: * Fix "expected all tensors to be on the same device" error * Fix "can't convert device type tensor to numpy" According to pytorch documentation torch.Tensor.numpy(force=False) performs conversion only if tensor is on CPU (plus few other restrictions) which is not the case. For our case we need force=True since we just need a data and don't care about tensors coherency. Fixes: huggingface#33517 See: https://pytorch.org/docs/2.4/generated/torch.Tensor.numpy.html Signed-off-by: Dmitry Rogozhkin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
With:
Issue seen on NVidia A10 and Intel PVC.
test_pt_flax_equivalence
andtest_encoder_decoder_model_standalone
are failing across multiple models due to missing models or tensors placements on devices. Specifically, there are 3 types of issues:model.to(cuda)
is missing)input.to(cuda)
is missing)torch.Tensor.numpy()
called with tensor being on device (should first be moved to CPU according to https://pytorch.org/docs/2.4/generated/torch.Tensor.numpy.html)Proposed fix:
CC: @sanchit-gandhi, @amyeroberts
See the following log for repro cmdline and list of errors (log running on NVidia A10, for XPU log will be similar):
The text was updated successfully, but these errors were encountered: