ENH add Lasso.jl julia solver #89

jolars · 2022-05-11T09:47:06Z

This PR adds support for the Lasso.jl Julia package. One problem with the solver is that it has no setting for maximum number of iterations, so when tolerance is low it may fail to converge. I've filed an issue at the repo about this and plan to file a PR eventually to get this fixed.

tomMoral

Overall, this looks super nice!! Thx @jolars

A few comments along the way.

solvers/lasso_jl.py

solvers/lasso_jl.jl

solvers/lasso_jl.py

tomMoral · 2022-05-11T11:39:28Z

solvers/lasso_jl.jl

+        intercept=fit_intercept,
+        randomize=false,
+        stopearly=false,
+        maxncoef=max(size(X, 1), size(X, 2)) * 100,


can you comment on what this is?

randomize shuffles indices before cd loops (sometimes or always, I don't know), stopearly is for stopping the path (change in deviance, saturation), maxncoef is also related to stopping the path early (maximum number of nonzero coefficients). I ran into a bug in the package when these were at their defaults, but I think only the maxncoef setting is actually necessary to work around it. I'll experiment a bit. randomize should really be true I think, but the others do not really matter.

tomMoral · 2022-05-11T11:41:02Z

For the broken test, we need to skip the julia solver for OSX as this causes a segfault in this case.

Co-authored-by: Thomas Moreau <[email protected]>

…to lassojl-solver

tomMoral

LGTM! thx @jolars

agramfort

@mathurinm merge if happy

…jl-solver

mathurinm · 2022-05-17T09:13:21Z

It's running fine on simulated on my machine.:

I only get a lot of

/home/mathurin/anaconda3/envs/benchopt_lasso/lib/python3.9/site-packages/julia/core.py:703: FutureWarning: Accessing `Julia().<name>` to obtain Julia objects is deprecated.  Use `from julia import Main; Main.<name>` or `jl = Julia(); jl.eval('<name>')`.
  warnings.warn(

can we fix it of silence it ? sorry if this has been discussed already

jt2gtwci · 2022-05-17T11:07:54Z

I only get a lot of

/home/mathurin/anaconda3/envs/benchopt_lasso/lib/python3.9/site-packages/julia/core.py:703: FutureWarning: Accessing `Julia().<name>` to obtain Julia objects is deprecated.  Use `from julia import Main; Main.<name>` or `jl = Julia(); jl.eval('<name>')`.
  warnings.warn(

can we fix it of silence it ? sorry if this has been discussed already

We talked about this a bit in benchopt/benchopt#372. I actually think the warning might be a bug: JuliaPy/pyjulia#497

mathurinm · 2022-05-17T11:18:38Z

So we silence the warning in the solver for now ?

…-solver

jolars · 2022-05-18T10:31:00Z

my setup is segfaulting for some reason right now, but I submitted a patch to ignore the warnings somewhat blindly. Can anybody please check that it works?

mathurinm · 2022-05-18T10:52:16Z

can you send me the command that causes the segfault ?

jolars · 2022-05-18T10:55:06Z

can you send me the command that causes the segfault ?

I just tried to call benchopt run -e . -s lasso_jl -d simulated, but I'm sure it's not related to benchopt in any way, just some annoying result of version/compilation problems from the pyenv, conda, and julia mix.

mathurinm · 2022-05-18T11:05:35Z

It works but:

it takes around 1 min for the solver to start
I still get the warnings
for some runs it outputs error

jolars · 2022-05-18T11:57:45Z

it takes around 1 min for the solver to start

that sounds like JIT compiliation

I still get the warnings

Hm, strange. I can silence the same warning in a small script just by doing the same thing:

from julia import Julia
import warnings

warnings.filterwarnings("ignore", category=FutureWarning)
jl = Julia(compiled_modules=False)
jl.eval("1 + 1")

for some runs it outputs error

What kind of errors? I used to receive errors previously due to convergence issues but I thought I had fixed them with a try catch statement.

Are you sure that you don't have results cached or something?

…-solver

mathurinm · 2022-05-19T06:22:07Z

There seems to be something wrong with sparse datasets:

(benchopt_lasso) ➜  lasso git:(bench) ✗ benchopt run . -s lasso_jl -d "libsvm[rcv1.binary]" -r1
WARNING: astropy not found, will default to scipy for convolution
Benchopt is running
libsvm[dataset=rcv1.binary]                                                                      
  |--Lasso Regression[fit_intercept=True,reg=0.5]                                                
/home/mathurin/anaconda3/envs/benchopt_lasso/lib/python3.9/site-packages/julia/core.py:703: FutureWarning: Accessing `Julia().<name>` to obtain Julia objects is deprecated.  Use `from julia import Main; Main.<name>` or `jl = Julia(); jl.eval('<name>')`.
  warnings.warn(
  |--Lasso Regression[fit_intercept=True,reg=0.1]                                                
  |--Lasso Regression[fit_intercept=True,reg=0.05]                                               
  |--Lasso Regression[fit_intercept=True,reg=0.01]                                               
  |--Lasso Regression[fit_intercept=True,reg=0.001]                                              
  |--Lasso Regression[fit_intercept=False,reg=0.5]                                               
  |--Lasso Regression[fit_intercept=False,reg=0.1]                                               
  |--Lasso Regression[fit_intercept=False,reg=0.05]                                              
  |--Lasso Regression[fit_intercept=False,reg=0.01]                                              
  |--Lasso Regression[fit_intercept=False,reg=0.001]                                             
No output produced.                                                                              
Saving result in: None

It's because sparse datasets are skipped by benchopt for julia, I remove this in
benchopt/benchopt#404

mathurinm · 2022-05-20T09:07:53Z

I don't get it to converge on rcv1.binary, for fit_intercept=False or True:

Same behavior for smaller reg (0.01, 0.001)

jolars · 2022-05-20T09:08:12Z

I am getting the following error when trying to run this now:

signal (11): Segmentation fault
in expression starting at /home/gerd-jln/.conda/envs/benchopt_benchmark_lasso/lib/python3.8/site-packages
/julia/install.jl:34
PyVectorcall_Function at /tmp/python-build.20220215103121.191136/Python-3.9.10/./Include/cpython/abstract
.h:73 [inlined]
_PyObject_Call at /tmp/python-build.20220215103121.191136/Python-3.9.10/Objects/call.c:265
unknown function (ip: 0x7ffcf9909daf)
Allocations: 5608687 (Pool: 5606149; Big: 2538); GC: 6

I am guessing that this has something to do with the mismatch in versions between conda and whatever this other python 3.9.10 version is doing there

jolars · 2022-05-20T09:11:21Z

I don't get it to converge on rcv1.binary, for fit_intercept=False or True:

Same behavior for smaller reg (0.01, 0.001)

Hm, could it be that the iteration limit is not sufficiently large?

…-solver

mathurinm · 2022-05-24T13:55:04Z

Ah, because it behaves like glmnet and returns a vector of 0s if it does not converge for the desired lambda ?

jolars and others added 20 commits November 30, 2021 16:17

Fix bug in glmnet solver

3232c37

add debug script

df67ce6

add computation of objective functions

6aa76de

test that for lambda just below lambda max both solutions are non zero

47ed265

Drop unnecessary computation of lambda_max

4d24648

Harmonize lambda scaling for glmnet solver

fed8fa4

Merge branch 'main' of github.com:jolars/benchmark_lasso

93f5a40

Switch benchopt method to use tolerance instead

790607a

tweaks to test_glmnet

8f947d8

change glmnet criterion to use a higher patience

1be208a

faster tol decrease for glmnet, try to have common initial point

46f6f04

increase patience again

68a379c

get common starting point, comment code

c57994d

CLN remove test_glmnet.py

43723de

more comments on glmnet behavior

5fa72e9

Update solvers/glmnet.py

96e5292

Merge branch 'main' of github.com:benchopt/benchmark_lasso

f182a0b

Merge branch 'main' of github.com:benchopt/benchmark_lasso

749dacf

Merge branch 'main' of github.com:benchopt/benchmark_lasso

e9da0b3

ENH add Lasso.jl Julia solver

ec88f65

tomMoral approved these changes May 11, 2022

View reviewed changes

jolars and others added 6 commits May 11, 2022 14:37

FIX remove unnecessary using calls

bf09186

Co-authored-by: Thomas Moreau <[email protected]>

FIX remove unnecessary dependencies

dc0d5be

Co-authored-by: Thomas Moreau <[email protected]>

fix: set randomize and stopearly to defaults

b60f905

Merge branch 'lassojl-solver' of github.com:jolars/benchmark_lasso in…

653db19

…to lassojl-solver

fix: run tol == INFINITY iteration on julia side

406b071

fix: decrease tolerance used for JIT compilation

b48933f

tomMoral approved these changes May 11, 2022

View reviewed changes

agramfort approved these changes May 11, 2022

View reviewed changes

jolars and others added 5 commits May 12, 2022 14:00

fix(ci): disable julia multi-threading on osx

b4f3c2a

fix(ci): move JULIA_NUM_THREADS export to Test run

1a2f1f6

fix(ci): stop testing on OSX

a5346f1

Merge branch 'main' of github.com:benchopt/benchmark_lasso into lasso…

1e97f76

…jl-solver

imports in import_ctx

4f09c92

jolars added 5 commits May 17, 2022 13:32

Merge branch 'main' of github.com:benchopt/benchmark_lasso

5670ac5

Merge branch 'main' of github.com:benchopt/benchmark_lasso

227a574

Merge branch 'main' of github.com:jolars/benchmark_lasso into lassojl…

7e4f8b2

…-solver

FIX ignore FutureWarnings from pyjulia

b50db11

FIX shorten line to appease flake8

9661be3

FIX remove trailing whitespace

fad0d0c

Merge branch 'main' of github.com:jolars/benchmark_lasso into lassojl…

3d5c70f

…-solver

Merge branch 'main' of github.com:benchopt/benchmark_lasso

336f607

jolars added 3 commits May 23, 2022 11:57

Merge branch 'main' of github.com:benchopt/benchmark_lasso

fc4e4ce

Merge branch 'main' of github.com:jolars/benchmark_lasso into lassojl…

2486c6b

…-solver

Merge branch 'main' of github.com:jolars/benchmark_lasso into lassojl…

acb003f

…-solver

refactor: increase the maximum number of iters

ed18ec9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH add Lasso.jl julia solver #89

ENH add Lasso.jl julia solver #89

jolars commented May 11, 2022

tomMoral left a comment

tomMoral May 11, 2022

jolars May 11, 2022

tomMoral commented May 11, 2022

tomMoral left a comment

agramfort left a comment

mathurinm commented May 17, 2022 •

edited

Loading

jt2gtwci commented May 17, 2022

mathurinm commented May 17, 2022

jolars commented May 18, 2022

mathurinm commented May 18, 2022

jolars commented May 18, 2022

mathurinm commented May 18, 2022

jolars commented May 18, 2022

mathurinm commented May 19, 2022 •

edited

Loading

mathurinm commented May 20, 2022

jolars commented May 20, 2022

jolars commented May 20, 2022

mathurinm commented May 24, 2022

ENH add Lasso.jl julia solver #89

Are you sure you want to change the base?

ENH add Lasso.jl julia solver #89

Conversation

jolars commented May 11, 2022

tomMoral left a comment

Choose a reason for hiding this comment

tomMoral May 11, 2022

Choose a reason for hiding this comment

jolars May 11, 2022

Choose a reason for hiding this comment

tomMoral commented May 11, 2022

tomMoral left a comment

Choose a reason for hiding this comment

agramfort left a comment

Choose a reason for hiding this comment

mathurinm commented May 17, 2022 • edited Loading

jt2gtwci commented May 17, 2022

mathurinm commented May 17, 2022

jolars commented May 18, 2022

mathurinm commented May 18, 2022

jolars commented May 18, 2022

mathurinm commented May 18, 2022

jolars commented May 18, 2022

mathurinm commented May 19, 2022 • edited Loading

mathurinm commented May 20, 2022

jolars commented May 20, 2022

jolars commented May 20, 2022

mathurinm commented May 24, 2022

mathurinm commented May 17, 2022 •

edited

Loading

mathurinm commented May 19, 2022 •

edited

Loading