Benchopt suite for machine learning

[gdalle]

It would be nice to support the Benchopt problem suite, which is also available in Python:

• [ ] Ordinary Least Squares
• [ ] Non-Negative Least Squares
• [ ] LASSO: L1-Regularized Least Squares
• [ ] LASSO Path
• [ ] Elastic Net
• [ ] MCP
• [ ] L2-Regularized Logistic Regression
• [ ] L1-Regularized Logistic Regression
• [ ] L2-regularized Huber regression
• [ ] L1-Regularized Quantile Regression
• [ ] Linear SVM for Binary Classification
• [ ] Linear ICA
• [ ] Approximate Joint Diagonalization
• [ ] 1D Total Variation Denoising
• [ ] 2D Total Variation Denoising
• [ ] ResNet Classification
• [ ] Bilevel Optimization

[dpo]

We don’t have any infrastructure for bilevel problems.

Problems with (smooth or nonsmooth) regularizers should go in RegularizedProblems.jl. That includes TV problems, LASSO, etc.

Least-squares problems should go in NLSProblems.jl.

[tmigot]

For NLS, we don't have real JuMP NLS here (they are in NLSProblems) but we do have ADNLSModel though, e.g. https://github.com/JuliaSmoothOptimizers/OptimizationProblems.jl/blob/main/src/ADNLPProblems/arglina.jl.