Optimus icon indicating copy to clipboard operation
Optimus copied to clipboard

One question about the decoder of vae

Open Hsintien-Ng opened this issue 2 years ago • 0 comments

File: https://github.com/ChunyuanLI/Optimus/blob/master/code/examples/big_ae/modules/vae.py

code in line 188, 133, 143: outputs = self.decoder(input_ids=labels, past=latent_z, labels=labels, label_ignore=self.pad_token_id) this line takes labels as the input_ids of decoder. I wonder know if it is an error. Should it be input_ids=inputs? Since there exits -1 in labels, it occurs an error in line 460 (inputs_embeds = self.wte(input_ids)) in modeling_gpt2.py in pytorch_transformers.

Thanks.

Hsintien-Ng avatar Nov 01 '21 14:11 Hsintien-Ng