JuliaFirstOrder
Documentation: expand sparse least squares example to other algorithms (Douglas-Rachford)
I managed to get most algorithms running on the Sparse regression example, but not the DRLS algorithm. I guess it should be applicable to sparse regression, but what needs to be done with: drls = ProximalAlgorithms.DRLS(tol=1e-4,verbose=true) solution_dr, iterations_dr = drls(x0=zeros(n_features+1),f=training_loss, g=reg,gamma=1e-7)
In that example, a function can be passed as f argument to the proximal-gradient-based algorithms, since gradients will be taken with respect to f via automatic differentiation.
For other algorithms, including DouglasRachford and DRLS, the proximal mapping of f is needed. Therefore a regular Julia function will not work. Instead, you can construct an equivalent function using eg ProximalOperators. For this specific example, something along the lines of this should work, for appropriate A and b and scaling coefficient: this is used a little below exactly with DRLS.
I think this is once again a documentation issue: feel free to leave this open, I’ll just adjust the title.
It seems as if ProximalAlgorithms.DRLS(tol = 10 * TOL, directions=acc) doesn't care about the values of the tol parameter.
It seems as if ProximalAlgorithms.DRLS(tol = 10 * TOL, directions=acc) doesn't care about the values of the tol parameter.
Do you have an example?
It definitely cares about the value of tol, but like other Newton-type methods it may suddenly converge fast to the solution, and therefore appear to be insensitive to the tolerance.
It's a rather large data set, can I send it by email? I have tested your test_small_lasso.jl and DRLS works fine on that small data, but not on my larger data. The interesting point is that all other methods from that file works fine on my data.
How big is it? Otherwise you could upload the data somewhere and share the link here, in case it’s fine to have the data publicly available
@patwa67 looks like there is a bug in the way DRLS displays progress: https://github.com/JuliaFirstOrder/ProximalAlgorithms.jl/blob/3de04a9b2878925f8307f756d66fbe528afb2e5a/src/algorithms/drls.jl#L188-L190
Fixing it, thanks!