For oe_matvec, potential speedup. We want to avoid too much conversions between sparse and dense matrices, so probably we want to switch to sparse matrices operations there.
For example, the replacement for np.inner:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.LinearOperator.html#scipy.sparse.linalg.LinearOperator
Ordinary dot product:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csc_matrix.dot.html