Support AMDGPU.jl v1.0 in MadNLPGPU.jl

[amontoison]

No description provided.

[sshin23]

do we have a linear solver?

[amontoison]

Yes :)
I solve KKT systems related to crystal sensing problems (40M of variables) with Krylov solvers.
These problems can be preconditioned efficiently and solved in a matrix-free way.

[sshin23]

I see. Makes sense. Are you doing IPM or is it purely Newton?

[amontoison]

We are doing primal-dual IPM.
We can see the problem as a QP where we call two linear operators (ifft and then fft).
We wanted to use MadQP.jl but it doesn't support GPUs yet.

cc @frapac

[amontoison]

I thought that AMDGPU.jl was a dependency of MadNLPGPU.jl but it's AMD.jl.
We don't have anything for AMD GPUs.
I can quickly interface the LAPACK routines of rocSOLVER.

[KSepetanc]

Will Krylov approach be implemented for CUDA as well? How it compares to cuDSS?

[amontoison]

Krylov.jl works with all GPU backends. We just need to implement a custom KKT system with the API of MadNLP.jl.
We have this example:
https://github.com/exanauts/CompressedSensingIPM.jl/blob/main/kkt.jl

We have a paper that should be submitted very soon with excellent results.
cuDSS can't be used in our application because we have problems with 350 millions of variables and materializing the Hessian of the Lagrangian will be dense (dense Vandermonde matrix that represents fft / ifft operators).

In optimization solvers, direct methods (like cuDSS or HSL) are generally more relevant because of ill-conditioning but for specific applications Krylov methods works well and could be also the only solution.
Compressed sensing is one of them.