Sparse Regression

[storopoli]

New content for Sparse Regression:

• [ ] What is Sparsity?
• [ ] The frequentist case with Ridge, Lasso and Elastic Net (optimization with Lagrangian constraints)
• [ ] Bayesian with:
  • [ ] Original Horseshoe (Carvalho, Polson & Scott, 2009)
  • [ ] Houseshoe+ (Bhadra et al., 2015)
  • [ ] Finnish Horseshoe (Piironen & Vehtari, 2017)
  • [ ] R2-D2 (Zhang et al., 2020)
• [ ] Code
  • [ ] Stan
  • [ ] Turing: Use the code from Robust Regression - Julia Discourse

References:

• Carvalho, C. M., Polson, N. G., & Scott, J. G. (2009). Handling sparsity via the horseshoe. In International Conference on Artificial Intelligence and Statistics (pp. 73-80).
• Bhadra, A., Datta, J., Polson, N. G., & Willard, B. (2015). The Horseshoe+ Estimator of Ultra-Sparse Signals. ArXiv:1502.00560 [Math, Stat] . [1502.00560] The Horseshoe+ Estimator of Ultra-Sparse Signals
• Piironen, J., & Vehtari, A. (2017). Sparsity information and regularization in the horseshoe and other shrinkage priors. ArXiv:1707.01694 [Stat] . Sparsity information and regularization in the horseshoe and other shrinkage priors
• Zhang, Y. D., Naughton, B. P., Bondell, H. D., & Reich, B. J. (2020). Bayesian regression using a prior on the model fit: The R2-D2 shrinkage prior. Journal of the American Statistical Association.