dask
A new accelerated, parallel, proximal descent method
Probably not the most orthodox thing to put in a GitHub issue, but it seems like it could be helpful for this project.
In the latest SIAM Review a paper by Fercoq and Richtárik appears: Optimization in High Dimensions vis Accelerated, Parallel, Coordinate Descent. I've got a paper copy, and I know the second author, and can certainly get an electronic copy if interested. Here is a preprint.
I can vouch for these folks, they've been working for years to parallelize some of the very optimization problems we're aiming to tackle here.
This seems like a great thing to put in a github issue to me :)
I'll be playing a bit more with dask performance early this week, mostly around the current implementations in @moody-marlin 's no_api branch (which seems to have the most recent development). I suspect that those optimizations will apply to any such solution though.
cc @hussainsultan @moody-marlin
Oh, actually, it looks like they've paid to make the PDF's free! Here you go:
Interesting work; however, they are mainly focused on the situation with a "huge" number of features. I think in the GLM space it is uncommon to use more than, say, 300 features at a time, which makes me more inclined to focus on algorithms like ADMM / SGD that can distribute across training examples / groups and not across features.
