Deep-Learning-Projects
Deep-Learning-Projects copied to clipboard
reconstr_loss calculated in variational_model.py
I wonder if the reconstr_loss calculated in variational_model.py in line 174:
self.reconstr_loss = -tf.reduce_sum(tf.mul(logits, self.X), 1)
should modify to:
self.reconstr_loss = -tf.reduce_sum(tf.mul(logits, self.mask_xs), 1)
just as you said in blog(https://s4sarath.github.io/2016/11/23/variational_autoenocder_for_Natural_Language_Processing)
"To achieve this in fast matrix operation, after calculating the ‘'’softmax’’’, we multiply the resultant matrix with the mask_xs ( which has 1 at the index where words are present and 0 if words are absent), matrix and then do the summation"