annotated_deep_learning_paper_implementations icon indicating copy to clipboard operation
annotated_deep_learning_paper_implementations copied to clipboard

๐Ÿง‘โ€๐Ÿซ 60 Implementations/tutorials of deep learning papers with side-by-side notes ๐Ÿ“; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gan...

Twitter Sponsor Deep Learning Paper Implementations

This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,

The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.


We are actively maintaining this repo and adding new implementations almost weekly. Twitter for updates.

Paper Implementations

โœจ Transformers

โœจ Eleuther GPT-NeoX

โœจ Diffusion models

โœจ Generative Adversarial Networks

โœจ Recurrent Highway Networks

โœจ LSTM

โœจ HyperNetworks - HyperLSTM

โœจ ResNet

โœจ ConvMixer

โœจ Capsule Networks

โœจ U-Net

โœจ Sketch RNN

โœจ Graph Neural Networks

โœจ Counterfactual Regret Minimization (CFR)

Solving games with incomplete information such as poker with CFR.

โœจ Reinforcement Learning

โœจ Optimizers

โœจ Normalization Layers

โœจ Distillation

โœจ Adaptive Computation

โœจ Uncertainty

โœจ Activations

โœจ Langauge Model Sampling Techniques

โœจ Scalable Training/Inference

Highlighted Research Paper PDFs


pip install labml-nn


If you use this for academic research, please cite it using the following BibTeX entry.

 author = {Varuna Jayasiri, Nipun Wijerathne},
 title = { Annotated Paper Implementations},
 year = {2020},
 url = {},

Other Projects

๐Ÿš€ Trending Research Papers

This shows the most popular research papers on social media. It also aggregates links to useful resources like paper explanations videos and discussions.


This is a library that let's you monitor deep learning model training and hardware usage from your mobile phone. It also comes with a bunch of other tools to help write deep learning code efficiently.