PPLM Is it possible to apply PPLM to encoder-decoder models (such as BERT2GPT2)

Is it possible to apply PPLM to encoder-decoder models (such as BERT2GPT2)

Open nooralahzadeh opened this issue 4 years ago • 3 comments

Nov 24 '20 12:11 nooralahzadeh

Hi @nooralahzadeh,

yes, it is possible. Just need to get the activation, attentions matrix, of both enc-dec.

Andrea

Dec 28 '20 03:12 andreamad8

IF it is possible, what's the input of the discriminator? The encoder hidden state or the decoder hidden state or both?

May 06 '21 13:05 monkdou0

I think the decoder hidden state should work, but the perturbation can be for both the encoder and the decoder.

May 06 '21 13:05 andreamad8