d2l.ai

Material and references from the MLT Dive into Deep Learning coding sessions

About

These bi-weekly sessions aim to provide code-focused sessions by reimplementing selected models from the book Dive into Deep Learning. These sessions are meant for people interested in implementing models from scratch. We hope to help participants either get started in their Machine Learning journey or deepen their knowledge if they already have previous experience.

We will try to achieve this by:

• Helping participants to create their models end-to-end by reimplementing models from scratch and discussing what modules/elements need to be included (e.g., data preprocessing, dataset generation, data transformation, etc.) to train an ML model.

• Discussing and resolving coding questions participants might have during the sessions.

Prerequisites

Even though we welcome everybody to join the sessions, it is highly recommended to have at least intermediate Python skills as we will be using PyTorch to implement models. We also recommend participants have a foundational knowledge of calculus, linear algebra, and statistics/probability theory.

Session Structure

• 30 min – Introduction
• 60 min – Live Coding
• 30 min – Discussion

FULL CURRICULUM

Session 1:

• Coding env setup example and book presentation
• A quick review of ML domains (supervised/unsupervised/RL)
• General Architecture/Components of ML code
• Implementations:
  • simple MLP-model

Session 2:

• Basics of Convolution and related Layers
• Implementations:
  • LeNet
  • VGG
  • Inception Block
  • Resnet Block

Session 3:

• Basics of Recurrent Neural Networks
• Tokenization, Vocabulary
• Backpropagation through time
• Implementations:
  • RNN
  • GRU
  • LSTM

Session 4:

• Attention mechanism #1
• Sequence to Sequence Models
• Bahdanau attention
• Luong Attention

Session 5:

Attention mechanism (Transformer) implementation

Session 6:

Generative adversarial networks (DCGAN) implementation

CODE OF CONDUCT

MLT promotes an inclusive environment that values integrity, openness, and respect. https://github.com/Machine-Learning-Tokyo/MLT_starterkit