bpr-spark icon indicating copy to clipboard operation
bpr-spark copied to clipboard

model parameters updating procedure in your implementation seems wrong

Open wenmin-wu opened this issue 7 years ago • 2 comments

In BPR the object function is to maximize the posterior probability, so the learning algorithm should be stochastic gradient ascent, that is to say we need subtract the regularized term not add. The codes in your implementation:

prodMat(prodPos, ::) :+= ((userMat(userId, ::) :* scale) + (prodMat(prodPos, ::) :* lambdaReg)) :* alpha
prodMat(prodNeg, ::) :+= ((-userMat(userId, ::) :* scale) + (prodMat(prodNeg, ::) :* lambdaReg)) :* alpha
userMat(userId, ::) :+= (((prodMat(prodPos, ::) - prodMat(prodNeg, ::)) :* scale) +
          (userMat(userId, ::) :* lambdaReg)) :* alpha

should be changed as following:

prodMat(prodPos, ::) :+= ((userMat(userId, ::) :* scale) - (prodMat(prodPos, ::) :* lambdaReg)) :* alpha
prodMat(prodNeg, ::) :+= ((-userMat(userId, ::) :* scale) - (prodMat(prodNeg, ::) :* lambdaReg)) :* alpha
userMat(userId, ::) :+= (((prodMat(prodPos, ::) - prodMat(prodNeg, ::)) :* scale) -
          (userMat(userId, ::) :* lambdaReg)) :* alpha

wenmin-wu avatar May 09 '17 02:05 wenmin-wu

is it just the bug have to fix?

ghost avatar Mar 22 '18 14:03 ghost

In BPR the object function is to maximize the posterior probability, so the learning algorithm should be stochastic gradient ascent, that is to say we need subtract the regularized term not add. The codes in your implementation:

prodMat(prodPos, ::) :+= ((userMat(userId, ::) :* scale) + (prodMat(prodPos, ::) :* lambdaReg)) :* alpha
prodMat(prodNeg, ::) :+= ((-userMat(userId, ::) :* scale) + (prodMat(prodNeg, ::) :* lambdaReg)) :* alpha
userMat(userId, ::) :+= (((prodMat(prodPos, ::) - prodMat(prodNeg, ::)) :* scale) +
          (userMat(userId, ::) :* lambdaReg)) :* alpha

should be changed as following:

prodMat(prodPos, ::) :+= ((userMat(userId, ::) :* scale) - (prodMat(prodPos, ::) :* lambdaReg)) :* alpha
prodMat(prodNeg, ::) :+= ((-userMat(userId, ::) :* scale) - (prodMat(prodNeg, ::) :* lambdaReg)) :* alpha
userMat(userId, ::) :+= (((prodMat(prodPos, ::) - prodMat(prodNeg, ::)) :* scale) -
          (userMat(userId, ::) :* lambdaReg)) :* alpha

It was a MISLEADING derivative formula from the original paper. There is a simple way to prove it was wrong: the regulation item should be used to reduce the overall scale of the parameters. However, if we put i == j here, the parameter booms. If you want to fix it up, you're challenging the original paper

Heermosi avatar Sep 18 '19 07:09 Heermosi