Matrix-Capsules-EM-Tensorflow icon indicating copy to clipboard operation
Matrix-Capsules-EM-Tensorflow copied to clipboard

Published 20 hours ago • verified publisher icon www0wwwjs1

Back propagation withing em_routing

Open dipanjan06 opened this issue 7 years ago • 3 comments

Hello, Thanks for releasing great code base on Capsule Network. One quick query are not stopping gradient during em_routing?

dipanjan06 avatar Jan 02 '18 07:01 dipanjan06

We have tried stopping gradient during routing. It leads to failures during training.

www0wwwjs1 avatar Jan 02 '18 07:01 www0wwwjs1

Ok thank you for your quick response. So my understanding is that , latent variable "r" does update during back propagation , then at E step whatever value being updated for r will overwrite during back propagation. You said that you have tried to stopping the gradient . please let know which variable are you trying stop gradient . is It votes variable?

dipanjan06 avatar Jan 03 '18 11:01 dipanjan06

Votes variables are used during different iterations of routing. It means parameters in transformation matrix should be trained to adapt different iterations. We have tried to stop gradient in the 2nd and 3rd iterations, which means parameters in transformation matrix are only trained for the initialization of routing. However, it leads to failures in training. The latent variables of EM algorithm update during forward clustering, and thus have no relationship with training.

www0wwwjs1 avatar Jan 04 '18 03:01 www0wwwjs1