PaperNotes

PaperNotes copied to clipboard

Natural Language Processing (NLP)

• [x] CARTA: How Language Evolves (2015, Symposia) [media link] [notes]
• [x] Recent Trends in Deep Learning Based Natural Language Processing (Oct 2018) [arXiv] [notes]
• [x] What Level of Quality can Neural Machine Translation Attain on Literary Text? (Jan 2018) [arXiv] [notes]
• [x] Nematus: a Toolkit for Neural Machine Translation (Mar 2017) [arXiv] [notes]

Benchmark/Datasets

• [x] SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (Jul 2019) [arXiv] [Benchmark] [PyTorch] [notes]
• [x] GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (Sep 2018) [arXiv] [Benchmark] [notes]
• [x] Evaluating Natural Language Understanding Services for Conversational Question Answering Systems (Aug 2017, SIGDIAL 2017) [aclweb] [dataset] [notes]

Embeddings

• [x] My Notes on Autoencoders and Variants [notes]
• [x] Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder (Nov 2018, EMNLP 2018) [arXiv] [notes]
• [x] ELMo: Deep contextualized word representations (Mar 2018, NAACL 2018, Allen Institute) [arXiv] [notes]

Recurrent Neural Networks (RNN)

• [x] How to Construct Deep Recurrent Neural Networks (Dec 2013) [arXiv] [notes]
• [x] Read + Verify: Machine Reading Comprehension with Unanswerable Questions (Aug 2018, AAAI 2019, Microsoft) [arXiv] [notes]
• [x] Neural Speed Reading via Skim-RNN (Mar 2018) [arXiv] [code] [notes]
• [x] Look, Listen and Learn (May 2017) [arXiv] [TwoMinutePapers] [notes]
• [x] Get To The Point: Summarization with Pointer-Generator Networks (Apr 2017) [arXiv] [code] [notes]

Attention

• [x] My Notes on Attention and Self-Attention in NLP [notes]

• [x] My Notes on Visual-Linguistic BERT-based models [notes]

• [x] "NLP's ImageNet moment has arrived" (Jul 2018) [Blog] [notes]

• [x] Attention Is All You Need [Transformer] (Dec 2017, NIPS 2017, Google Brain) [arXiv] [Harvard Blog] [tensor2tensor] [ppt] [notes]

• [x] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Oct 2018, Google AI Language) [arXiv] [Tensorflow, PyTorch] [notes]

• [x] Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures (Nov 2018, EMNLP 2018) [arXiv] [notes]

• [x] MT-DNN: Multi-Task Deep Neural Networks for Natural Language Understanding (May 2019, Microsoft) [arXiv] [notes]

• [x] Reformer: The Efficient Transformer (ICLR 2020, Google AI) [OpenReview] [Tensorflow] [notes]

• [x] DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference (ACL 2020, Vector Institute) [paper] [slideslive] [PyTorch] [notes]

Less attention

• [ ] Pay Less Attention with Lightweight and Dynamic Convolutions (Sep 2018, ICLR 2019, Facebook AI Research) [OpenReview] [PyTorch] [notes]

Attention + RNN

• [x] A GRU-Gated Attention Model for Neural Machine Translation (Apr 2017) [arXiv] [code] [notes]
• [x] DialogueRNN: An Attentive RNN for Emotion Detection in Conversations (Nov 2018, AAAI) [arXiv] [code] [ppt] [notes]

Grammar Error Correction (GEC)

• [x] Neural Quality Estimation of Grammatical Error Correction (Nov 2018, EMNLP 2018) [aclweb] [PyTorch] [notes]

Intent Classification

• [x] Simultaneous Identification of Tweet Purpose and Position (AAAI 2020) [arXiv] [notes]
• [x] Fast Intent Classification for Spoken Language Understanding [BranchyNet Application] (arXiv, Dec 2019) [arXiv] [ppt] [notes]
• [x] Subword Semantic Hashing for Intent Classification on Small Datasets (Dec 2018) [arXiv] [PyTorch] [ppt] [notes]

GAN for text

• [x] Style Transfer Through Back-Translation (May 2018, ACL 2018) [arXiv] [PyTorch] [notes]

Imbalanced data / Data Augmentation

• [x] My Notes on Solutions for Imbalanced data [notes]
• [x] EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks (Jan 2019) [arXiv] [code] [notes]
• [x] SMOTE: Synthetic Minority Over-sampling Technique (Jun 2002, JAIR) [arXiv] [notes]
• [x] ARCID: A new approach to deal with imbalanced datasets classification (Jan 2018, SOFSEM 2018) [paper] [notes]

Noisy/Incomplete data

• [x] Don't Underestimate the Benefits of Being Misunderstood (2017) [WebMIT] [notes]
• [x] Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation (Oct 2018) [arXiv] [notes]

Convolutional Neural Networks (CNN)

• [x] Deep image reconstruction from human brain activity (Dec 2017) [bioRxiv] [notes]

Audio

• [x] The challenge of realistic music generation: modelling raw audio at scale (Jun 2018) [arXiv] [notes]
• [x] SampleRNN: An Unconditional End-to-End Neural Audio Generation Model (Feb 2017, ICLR) [arXiv] [code, pytorch] [notes]
• [x] WaveNet: A Generative Model for Raw Audio (Sep 2016) [arXiv] [code] [notes]
• [x] FFTNet: a Real-Time Speaker-Dependent Neural Vocoder (April 2018, ICASSP) [paper] [code] [notes]
• [x] WaveRNN: Efficient Neural Audio Synthesis (June 2018, ICML) [arXiv] [code] [notes]
• [x] Song From PI: A Musically Plausible Network for Pop Music Generation (Nov 2016) [arXiv] [website] [notes]
• [x] Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions (Dec 2017) [arXiv] [notes]
• [x] Tacotron: A Fully End-to-End Text-To-Speech Synthesis Model (March 2017) [arXiv] [code1, code2] [notes]
• [x] MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation (Mar 2017) [arXiv] [code] [notes]

Multimodality

• [x] Multimodal Machine Learning: A Survey and Taxonomy (Aug 2017, PAMI 2018) [arXiv] [notes]
• [x] Using sparse semantic embeddings learned from multimodal text and image data to model human conceptual knowledge (Nov 2018) [arXiv] [notes]
• [x] Char2Wav: End-to-End Speech Synthesis (ICLR Workshop 2017) [OpenReview] [code] [notes]

Generative Adversarial Networks (GAN)

• [x] Generative Adversarial Network (Jun 2014) [arXiv] [code] [notes]
• [x] Coupled Generative Adversarial Networks (Jun 2016) [arXiv] [code] [notes]
• [x] Adversarial Autoencoders [Blog]
• [x] Conditional Image Generation with PixelCNN Decoders (Jun 2016) [arXiv] [code] [notes]
• [x] C-RNN-GAN: Continuous recurrent neural networks with adversarial training (Nov 2016) [arXiv] [code] [notes]
• [x] Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (Nov 2015) [arXiv] [code1, code2] [notes]
• [x] A Neural Algorithm of Artistic Style [arXiv] [code1, code2, code3, code4] [notes]
• [x] Video Pixel Networks (Oct 2016) [arXiv] [notes]
• [x] SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient (Sep 2016) [arXiv] [code1, code2, code3] [notes]
• [x] SinGAN: Learning a Generative Model from a Single Natural Image (ICCV 2019, Best Paper) [paper] [video1, video2] [paper webpage] [PyTorch] [notes]

Blockchain

• [x] Bitcoin: A Peer-to-Peer Electronic Cash System (2008) [bitcoin] [notes]

Others

• [x] i-RevNet: Deep Invertible Networks (ICLR 2018) [OpenReview] [PyTorch] [notes]
• [x] The Reversible Residual Network: Backpropagation Without Storing Activations (NeurIPS 2017) [paper] [Tensorflow, PyTorch] [notes]
• [x] How AI can save our humanity by Kai-Fu Lee [TEDTalk] [notes]
• [x] Dynamic Routing Between Capsules (Oct 2017) [arXiv] [Siraj Raval] [notes]
• [x] GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (2018, Google Brain) [arXiv] [notes]

Blogs/Journals/Books

• The Learning Machine [blog] [codes]
• Browse SoTA
• DeepMind
• Distill Journal: Online journal that doesn't limit your work to a pdf, it allows the reader to have a more interactive experience
• AI: Kaggle and Numerai
• Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
• Towards Data Science: Learn Docker