Hi, @pesser, thanks for your work!
According to here, it seems that you don't use cn in the enc_up pass, which is different from the net in your paper.
cn
enc_up
Could you explain that? Do we need cn there? Thanks a lot!