Move LOBPCG out

[antoine-levitt]

We should move LOBPCG to the broader ecosystem. Mohamed Tarek was open to moving it to IterativeSolvers if we can demonstrate it's superior to the existing implementation. We should benchmark it against the existing implementation for a range of (M,N). Ideally using a diagonal A, so that doesn't skew the timings.

[mfherbst]

Main difficulty in my opinion is GPU compatibility

[antoine-levitt]

Benchmarks:

using DFTK
using IterativeSolvers
using Random

function bench(N, density, tol)
    Random.seed!(0)
    M = ceil(Int, density*N)

    d = vcat(rand(M), 2 .+ rand(N-M))
    A = Diagonal(d)
    X = randn(N, M)
    t1 = @elapsed DFTK.LOBPCG(A, X, I, I, tol)
    t2 = @elapsed IterativeSolvers.lobpcg(A, I, false, X, tol=tol)
    (t1, t2)
end

for (N, density) in Iterators.product((100, 1000, 4000, 10000), (.01, .1))
    println("$N, $density")
    display(bench(N, density, 1e-3))
end

Results

100, 0.01
(0.000709827, 0.000113247)
1000, 0.01
(0.007919946, 0.006568147)
4000, 0.01
(0.102116052, 0.118252976)
10000, 0.01
(0.74366979, 0.906752279)
100, 0.1
(0.002315039, 0.00134439)
1000, 0.1
(0.098804512, 0.104305223)
4000, 0.1
(3.08999895, 4.546904686)
10000, 0.1
(46.322246197, 88.606136158)