keystone icon indicating copy to clipboard operation
keystone copied to clipboard

WeightedBlockSolver for multiple lambdas

Open ericmjonas opened this issue 9 years ago • 1 comments

We often perform a sweep over lambdas to figure out the right regularization -- it matters a lot.

@shivaram mentioned that it's possible for the weighted solver to potentially solve for a large number of lambdas with minimal overhead. There was evidently some code in a repo associated with a paper that is an example of this. It might be possible (per Shiv's suggestion) that @tomerk could hack on this?

ericmjonas avatar Feb 13 '16 00:02 ericmjonas

I misremembered things a little bit - we've never done multi-lambda for the old BlockWeightedLeastSquares. But for the new PerClassWeightedLeastSquares it should be pretty simple to add it by just following wherever we have a loop around lambdas in the unweighted implementation [1].

FWIW If we want feature compatibility across the solvers, the other one should also be possible -- but my guess is that it'll not be that cheap in terms of sharing costs and need some new code.

[1] https://github.com/amplab/ml-matrix/blob/c0c0d7c754f3873c419af85bc298cdde2cb4cdfe/src/main/scala/edu/berkeley/cs/amplab/mlmatrix/BlockCoordinateDescent.scala#L88

shivaram avatar Feb 13 '16 00:02 shivaram