Awesome Data Science

The below list is by no means an exhaustive list of data science, it is a set of my personal bookmarks of interesting reads, primarily related to neural networks. It also is heavily skewed towards resources that are useful for source{d}

Online books

• Deep Learning by Ian Goodfellow, Aaron Courville, and Yoshua Bengio
• Neural Networks and Deep Learning by Michael Nielsen

Online Resource Collections

• Tutorials for TensorFlow The best neural network tutorials I've come across
• Christopher Olah's Blog One of the best resources for truly understanding neural networks
• Deep Learning for NLP resources
• What NLP problems has deep learning or neural networks been applied to successfully?
• Speech and Natural Language Processing
• How to actually learn data science
• GitXiv — Collaborative Open Computer Science

Online courses

• CS224d: Deep Learning for Natural Language Processing
• Neural networks class - Université de Sherbrooke
• NVIDIA Deep Learning Courses
• YC HN thread with interesting courses

Research papers (incl. arXiv)

• Semantics, Representations and Grammars for Deep Learning; 2015 - David Balduzzi
• Towards Neural Network-based Reasoning; 2015 - Baolin Peng, Zhengdong Lu, Hang Li, Kam-Fai Wong
  • Notes on the paper
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank; 2013 - Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng and Christopher Potts
• Text Understanding from Scratch; 2015 - Xiang Zhang, Yann LeCun
• Dropout: A Simple Way to Prevent Neural Networks from Overfitting; 2013 - Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov
• Learning To Execute; 2014 - Wojciech Zaremba, Ilya Sutskever
• When Are Tree Structures Necessary for Deep Learning of Representations?; 2015 - Jiwei Li, Minh-Thang Luong, Dan Jurafsky, Eudard Hovy
• Distributed Representations of Sentences and Documents; 2014 - Quoc Le, Tomas Mikolov
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets; 2015 - Armand Joulin, Tomas Mikolov
• Multi-View Learning of Word Embeddings via CCA; 2011 - Paramveer S. Dhillon, Dean Foster, Lyle Ungar
• Is deep learning really necessary for word embeddings?; 2013 - Remi Lebret, Joel Legrand, Ronan Collobert
• Intriguing properties of neural networks; 2013 - Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus
• Building Program Vector Representations for Deep Learning; 2014 - Lili Mou, Ge Li, Yuxuan Liu, Hao Peng, Zhi Jin, Yan Xu, Lu Zhang
• Who Wrote This Code? Identifying the Authors of Program Binaries; 2011 - Nathan Rosenblum, Xiaojin Zhu, Barton P. Miller

Interesting articles

• The Unreasonable Effectiveness of Recurrent Neural Networks
• Wolfram Language Artificial Intelligence: The Image Identification Project
• Deep Learning vs Probabilistic Graphical Models vs Logic
• A Step by Step Backpropagation Example
• A Neural Network in 11 lines of Python (Part 1)
• Suddenly, a leopard print sofa appears
• Hacker's guide to Neural Networks
• Auto-Generating Clickbait With Recurrent Neural Networks
• What a Deep Neural Network thinks about your #selfie
• Neural Networks, Types, and Functional Programming
• Composing Music With Recurrent Neural Networks
• Image Scaling using Deep Convolutional Neural Networks
• The neural networks behind Google Voice transcription
• 26 Things I Learned In The Deep Learning Summer School
• Growing Pains for Deep Learning
• Understanding Natural Language Deep Neural Networks Using Torch
• Generating Magic cards using deep, recurrent neural networks
• Google voice search: faster and more accurate
• How Google Translate squeezes deep learning onto a phone
• Evolution of Deep learning models
• AMA Geoffrey Hinton
• Baidu explains how it’s mastering Mandarin with deep learning
• Calculus on Computational Graphs: Backpropagation
• Geoffrey Hinton on max pooling (reddit AMA)
• Algorithms of the Mind
• The elastic brain
• How does word2vec work?
• A conversation with Sussman on AI and asynchronous programming
• The Brain vs Deep Learning Part I: Computational Complexity — Or Why the Singularity Is Nowhere Near
• Learning from the best
• Data has a shape
• Petabyte-Scale Text Processing with Spark
• Document Clustering With Python
• A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).
• Faster deep learning with GPUs and Theano

Interesting news articles

• You're Using Neural Networks Every Day Online—Here's How They Work
• Five crazy abstractions my Deep Learning word2vec model just did
• Google DeepMind Teaches Artificial Intelligence Machines to Read
• Google's AI can learn to play video games
• How Google’s New Photos App Can Tell Cats From Dogs
• Neural network chip built using memristors
• Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning and Make Machines Smarter
• Biggest Neural Network Ever Pushes AI Deep Learning
• The Yahoo Behind Fresh Deep Learning Approaches at Flickr
• Neuroscience: The Man Who Saw Time Stand Still

Interesting videos

• Interview with Google's AI and Deep Learning 'Godfather' Geoffrey Hinton
• Why Google believes nothing will be lost in translation
• List of talks on Deep Learning

Opinions on ethics

• Machine intelligence, part 1/2
• The AI Revolution: The Road to Superintelligence

Other

• Git from the bottom up