Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZDNN_MISALIGNED_TENSOR when invoking NNPA on zOS when using a constants binary file. #2993

Open
christopherlmunoz opened this issue Oct 30, 2024 · 4 comments

Comments

@christopherlmunoz
Copy link
Contributor

This is on zOS using the --store-constants-binary-file. This issue is specifically on a zdnn_conv2d call. I can provide a full output if needed.

bash-5.2$ client/bin/modelzoo --lib mnist-8-with-binary.so --file mnist-8.tests --validate -m VERBOSE --fc-parms 0.01,0.158752,9,10

ERROR: invoke_nnpa() (./zdnn.c:288): ZDNN_MISALIGNED_TENSOR: Tensor address is not on 4K-byte boundary

With ZDNN_ENABLE_PRECHECK enabled, the Kernel and Bias tensors are non 4K aligned addresses.

Contents of zdnn_ztensor: kernel
Pre-transformed Descriptor:
Outermost Innermost
Dimensions: 5 5 1 8
Layout: ZDNN_HWCK Format: ZDNN_FORMAT_4DFEATURE
Type: FP16
Transformed Descriptor:
Outermost Innermost
Dimensions: 5 5 1 8
Layout: ZDNN_HWCK Format: ZDNN_FORMAT_4DKERNEL
Type: ZDNN_DLFLOAT16
Buffer Addr: b4df2ffd363083d7 Size: 102400
Transformed: True

Thanks! FYI @tungld

@tungld
Copy link
Collaborator

tungld commented Nov 8, 2024

Created a PR #3002 to make debugging easier.

@tehbone
Copy link
Contributor

tehbone commented Nov 8, 2024

Can you please paste the mmap code that is used?

@tehbone
Copy link
Contributor

tehbone commented Nov 8, 2024

Now that I look at it, the buffer address looks to be out of sorts, leading to the following questions:

  1. Is __MAP_64 used? If not, the buffer address is incredibly wrong.
  2. I suspect that some offset is used to calculate the tensor's buffer address. Is that correct?

@tungld
Copy link
Collaborator

tungld commented Nov 13, 2024

The starting address of the buffer was wrong and it is fixed by this #3002.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants