Optimus
Optimus copied to clipboard
ChunyuanLI
Optimus: the first large-scale pre-trained VAE language model
Hello, thank you very much for making the code available. I'm confused about the mutual information math, more specifically about the line E_{q(z|x)}log(q(z|x)) = -0.5*nz*log(2*\pi) - 0.5*(1+logvar).sum(-1) neg_entropy += (-0.5...
It seems that your demo webset can not be accessed. Can you fix it?
Hi! I'm trying to reproduce the reported result on SNLI, I followed the doc 'optimus_for_snli.md' and successfully downloaded the checkpoints, but when I run your examples, it turns out that...
hi ! Thanks for releasing the code and checkpoints, but i want to know have you released a model of pretrained with Chinese dataset? look forward to your reply!
Thank you for this great repo ! I was trying to use it for language modeling but I couldn't find, amongst the checkpoints you provide, any model that performed well...
Thank your for sharing this great work ! I have been trying some of your checkpoints but they don't even seem to perform reconstruction right (even at beta=0). Then I...
File: https://github.com/ChunyuanLI/Optimus/blob/master/code/examples/big_ae/modules/vae.py code in line 188, 133, 143: outputs = self.decoder(**input_ids=labels**, past=latent_z, labels=labels, label_ignore=self.pad_token_id) this line takes labels as the input_ids of decoder. I wonder know if it is an...
Hi! Great work. But one open question :D I am curious about the performance of using gpt2 as both encoder and decoder? I am not sure if the discrepancy from...
Hi, great work using VAE. I can't open your demo website.Could you rewrite the demo website link?
Where can I get the preprocessed dailydialog dataset used for spacefusion pretraining code? Any suggestion on how to preprocess the original dailydialog would be appreciated! Thanks
Metadata
Owner
Metadata
Optimus: the first large-scale pre-trained VAE language model