Add classical/new architectures

[omarsar]

Some ideas for figures to add to the PPT

• [ ] Linear regression, single-layer neural network
• [ ] Multilayer Perceptron with hidden layer
• [ ] Backpropagation
• [ ] Batch Normalization and alternatives
• [ ] Computational Graphs
• [ ] Dropout
• [ ] CNN - padding, stride, pooling,...
• [ ] LeNet
• [ ] AlexNet
• [ ] VGG
• [ ] GoogleNet
• [ ] ResNet
• [ ] DenseNet
• [ ] Memory Networks
• [ ] RNN
• [ ] Deep RNN
• [ ] Bidirectional RNN
• [ ] GRU
• [ ] LSTM
• [ ] Language RNN models
• [ ] Backpropagation through time
• [ ] Encoder-Decoder Architecture
• [ ] Seq2seq with RNN encoder-decoder
• [ ] Bearm search and other decoding strategies
• [ ] Attention
• [ ] Multi-head attention
• [ ] Self-attention
• [ ] Transformer
• [ ] Word2vec/GloVe/Skip-gram/CBOW/BERT/GPT....
• [ ] Common/Popular CV/NLP Tasks

List adopted from multiple resources including nlpoverview and d2l.ai which both contain a very solid syllabus.

Please feel free to make suggestions below. If you would like to help, also let me know.

[heany]

Maybe, the generative adversarial networks and graph neural networks should be included in the list.

[narutomst]

Deep Belief Networks

[nashid]

Can we Graph Neural Network models like GCN, GAT, GGNN, GraphSage, etc?

[nashid]

Including Contrastive Representation Learning models would be great!