-
-
Notifications
You must be signed in to change notification settings - Fork 139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rm_corr error LinAlgError: SVD did not converge #440
Comments
Hi Sophie, Thanks for opening the issue. The Pingouin implementation is based on an ANCOVA model that is implemented with statsmodels. I am not able to reproduce the error on my machine: I am using Python 3.9 with statsmodels 0.14. What versions of Python, pingouin, and statsmodels are you using? Second, I noticed that your datasets includes many subjects with only 1 or 2 observations. Do you still get the error if you remove these participants from the dataset? Thanks, |
Hi Raphael, Thanks so much for your reply. I am using the following versions (on Linux): statsmodels 0.14.0 Which version of pingouin are you using when you don't reproduce the error? Thanks so much! |
Hi, I am using pingouin 0.5.5 (on Mac), pandas 2.2.2, numpy 1.26.4, statsmodels 0.14.0 |
Hi Raphael, |
Sure thing @Eric-Kobayashi:
|
Hi Raphael, Thanks for providing these information. It turns out not to be a version issue but it might be a deeper problem with the I was able to successfully run the rm_corr function after shuffling the dataframe. I've simulated reshuffling many times and found there is around 8.12% of failure to converge. On the other hand, would you mind testing the same and see if you replicate the issue?
|
Hmm, very strange behavior indeed. I can replicate the error: 5 out of 100 run of the function on resampled data failed (5% failure). |
Hello,
I am trying to run rm_corr on multiple columns of a dataframe (gene expression data), and while the function works well on many column pairs (and match the expected output from the R rmcorr function) one pair of columns throws an error
LinAlgError: SVD did not converge
. However, this pair of columns has no trouble running in the R rmcorr implementation. Therefore, I am curious what the difference between the two implementations is, and whether it is possible to get this to converge in python? I would prefer to continue to use your implementation as it seems to be much faster (I am in the midst of benchmarking how both implementations scale, so as a side note if you have any data on that I would very much appreciate it!).I am using Pingouin v.0.5.5. I have attached a minimal dataframe to recreate my error, along the with the code below:
Dataframe:
df_pingouin_fail.csv
Thanks so much for your help!
Best,
Sophie
The text was updated successfully, but these errors were encountered: