Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_NoopCatFunc in transformer layer #1384

Open
robot-transformer opened this issue Dec 22, 2024 · 0 comments
Open

_NoopCatFunc in transformer layer #1384

robot-transformer opened this issue Dec 22, 2024 · 0 comments

Comments

@robot-transformer
Copy link

Hello!
I'm currently rewriting my LM-pipeline with TE. And I noticed that the following line doesn't work as expected. I.e for the first forward of my layer this cat noop function goes to this point (i.e. all checks of layout were passed), but out tensor instead of being concatenation of all weights looks like cat of tensor[0] and uninitialised memory.
After some debugs I found that tensors[i].data_ptr() + tensors[i].size(dim) * data_ptr_stride = tensors[I+1].data_ptr(), but tensors[i].untyped_storage().data_ptr() are different (but storages located sequentially). And as far as I know here is expected that tensors[i].untyped_storage().data_ptr() would be the same. Am I wrong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant