Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some dtypes not supported by numba #521

Open
aulemahal opened this issue Oct 13, 2021 · 2 comments
Open

Some dtypes not supported by numba #521

aulemahal opened this issue Oct 13, 2021 · 2 comments
Labels

Comments

@aulemahal
Copy link

Describe the bug
Big-endian floats (and maybe others) are not supported by numba, which leads to TypingError on some sparse operations.

To Reproduce

import numpy as np
import sparse as sp

little = np.array([1, 1, 1, 1, 1, 1], dtype='<f8')
big = np.array([1, 1, 1, 1, 1, 1], dtype='>f8')

Anp = np.random.randint(100, size=(4, 6)) / 100
A = sp.COO.from_numpy(Anp)

A.dot(little) # works
A.dot(big) # fails

The other way around is the same;

Abig = sp.COO.from_numpy(Anp.astype('>f8'))

Abig.dot(little) # fails
Abig.dot(big) # fails

Expected behavior
I expected the operation to work in all cases.

Could on-the-fly byte swap be implemented? May be with a warning?

In this specific case, on-the-fly byte swapping is what numpy does:

r = Anp.dot(big)  
r.dtype # <f8, little-endian

For the case with Abig, where the sparse array itself is in a unsupported dtype, if not on-the-fly conversion is done, maybe there could be an issue raised, or at least a warning, on creation? To signify that the input array doesn't support the full features of sparse.

System

  • OS and version: Ubuntu 18.04
  • sparse version : 0.13.0
  • NumPy version : 1.21.2
  • Numba version : 0.53.1

Additional context

@aulemahal aulemahal added the bug Indicates an unexpected problem or unintended behavior label Oct 13, 2021
@hameerabbasi hameerabbasi added upstream and removed bug Indicates an unexpected problem or unintended behavior labels Oct 14, 2021
@hameerabbasi
Copy link
Collaborator

Hmm, this is an upstream Numba bug, as you already said. It's hard to detect the full scope of these kinds of things without try-catch.

@hameerabbasi
Copy link
Collaborator

On-the-fly swapping could be done, in theory. That's something I would consider cleaner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants