How to do the model parallelism ?

Open vvictor-lee opened this issue 5 years ago • 0 comments

Great Work ! I found that the code works well on multiple gpu training but only for the data parallelism. However, it is hard to train a model with the classes more than 10 million or even more. In that case, the model parallelism should solve the problem. I was doing the implementations but the diam softmax may seriously interfere the model parallelism. Could you give the solutions or any idea ?

Mar 11 '20 01:03 vvictor-lee