codertimo
MASKGAN: BETTER TEXT GENERATION VIA FILLING IN THE ___
어떤 내용의 논문인가요? 👋
GAN을 사용해서 문장의 일정 Masked 부분을 생성하는 모델인 것 같아요! Ian Goodfellow 가 2저자로 참여한 논문이네요 ㅎㄷㄷ
Abstract (요약) 🕵🏻♂️
Neural text generation models are often autoregressive language models or seq2seq models. These models generate text by sampling words sequentially, with each word conditioned on the previous word, and are state-of-the-art for several machine translation and summarization benchmarks. These benchmarks are often defined by validation perplexity even though this is not a direct measure of the quality of the generated text. Additionally, these models are typically trained via maximum likelihood and teacher forcing. These methods are well-suited to optimizing perplexity but can result in poor sample quality since generating text requires conditioning on sequences of words that may have never been observed at training time. We propose to improve sample quality using Generative Adversarial Networks (GANs), which explicitly train the generator to produce high quality samples and have shown a lot of success in image generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them. We claim that validation perplexity alone is not indicative of the quality of text generated by a model. We introduce an actor-critic conditional GAN that fills in missing text conditioned on the surrounding context. We show qualitatively and quantitatively, evidence that this produces more realistic conditional and unconditional text samples compared to a maximum likelihood trained model.
이 논문을 읽어서 무엇을 배울 수 있는지 알려주세요! 🤔
Text 에 GAN 을 사용했을 때 어떤 문제점이 있었는지, 해결은 가능한지 등을 알아볼 수 있을 것 같아요. 무엇보다 GAN을 사용했을 때 기존의 SEQ2SEQ 보다 어떤 점이 좋았는지 배울 수 있을 것 같아요!
같이 읽어보면 좋을 만한 글이나 이슈가 있을까요?
만약에 있다면 자유롭게 작성해 주세요!
레퍼런스의 URL을 알려주세요! 🔗
https://arxiv.org/pdf/1801.07736.pdf
