CSC_deeplearning

Crash-course deep learning in 3 days.

Lectures and corresponding seminars are in the ./day* folders.

Useful links:

• feedback_form
• GPU instructions for CSC students

Syllabus

• day 0 morning Recap

  • [ ] Lecture: Linear models, stochastic optimization, regularization
  • [ ] Seminar: Adaptive optimization methods for subgradient SVM

• day 0 afternoon Going deeper

  • [ ] Lecture: Neural networks 101
  • [ ] Seminar: theano, symbolic graphs and basic neural networks

• day 0 evening Yandex/HSE introductory note

• day 1 morning Vision & convolutional networks

  • [ ] Lecture: Computer vision problems. Convolutional neural networks. Tips and tricks. Fine-tuning.
  • [ ] Seminar: lasagne and CIFAR

• day 1 afternoon Natural language processing & embeddings

  • [ ] Lecture: NLP problems and applications, bag of words, word embeddings, word2vec, text convolution.
  • [ ] Seminar: Embeddings and text convolutions for salary prediction.

• day 1 evening Generative adversarial networks

• day 2 morning Recurrent neural networks for sequences

  • [ ] Lecture: Sequence modelling. Simple RNN. BPTT. Gradient vanishing/explosion. LSTM/GRU. Grad clipping.
  • [ ] Seminar: Generating names & modecules with recurrent neural networks

• day 2 afternoon Connecting it all together: image captioning

  • [ ] Lecture: RNN as Encoder/Decoder/Seq2seq. Common semantic space. Image captioning.
  • [ ] Seminar: Image captioning

• day 2 evening Outro

Contributors & course staff

Course staff

• Fedor Ratnikov
• Maxim Borisyak
• Andrey Ustyuzhanin
• Mikhail Usvyatsov

Contributors

• Arseniy Ashukha - image captioning, sound processing, week7&9 lectures
• Oleg Vasilev - seminars here and there
• Vadim Lebedev - seminar for linear models.

The course is based on our HSE_deepelarning track.