-
-
Notifications
You must be signed in to change notification settings - Fork 156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Undocumented behaviour of bitshuffle when blocksize is not a multiple of 8*type_size. #312
Comments
Yes, that could be the case. The issue here is that we are always improving C-Blosc2 and sometimes it is difficult to track down which improvements can be backported to C-Blosc (although I can ensure you that we are proactively doing so in general). If that is something that is important for you, we would be happy to add a backport to C-Blosc (of course, a PR would help in getting this included as soon as possible). |
I'm not sure this can be fixed by just backporting bitshuffle and bitunshuffle from blosc2 because because it's a breaking change and the I can make a PR for a note in the documentation if there's no other way. |
Bitshuffling fallbacks to memcpy when
size % 8 !=0
. This is fixed in version 3, so I presume it's not fixed here for valid reasons. I would propose at least visibly documenting this behaviour as it is completely transparent to the users and results in much worse compression. It took me hours to narrow this down.The text was updated successfully, but these errors were encountered: