Deep Learning for Audio (DLA)

• Lecture and seminar materials for each week are in ./week* folders, see README.md for materials and instructions
• Any technical issues, ideas, bugs in course materials, contribution ideas - add an issue
• The current version of the course is conducted in autumn 2021 at the CS Faculty of HSE

Syllabus

• week01 Introduction to Digital Signal Processing

  • Lecture: Introduction to Course
  • Seminar: Intro in pytorch

• week02 Introduction to Digital Signal Processing

  • Lecture: Signals, Fourier transform, Spectrograms, MelScale, MFCC and etc
  • Seminar: torchaudio, spoken digit classification

• week03 Automatic Speech Recognition (ASR) I

  • Lecture: Metrics, Attention, CTC, LAS, BeamSearch
  • Seminar: Audio augmentations, CTC decoding, CTC BeamSearch

• week04 Automatic Speech Recognition (ASR) II

  • Lecture: RNN-T, LM-fusion, BPE
  • Seminar: W&B tutorial, homework barebones overview

• week05 Speaker verification and identification

  • Lecture: Metric Learning: Cosine, Contrastive, Triplet Losses. Angular Softmax. ArcFace
  • Seminar: Q&A about homework

• week06 Key-word spottind (KWS)

  • Lecture: (DNN, CNN, RNN+Attention) based KWS, SVDF, Orthogonality Regularization and other Tricks
  • Seminar: Implementation of CNN+Attention+RNN KWS model

• week07 Text to Speech (TTS)

  • Lecture: Tacotron, DeepVoice, GST, FastSpeech, AdaSpeech, Attention Tricks
  • Seminar: TTS in torchaudio

• week08 Neural Vocoders

  • Lecture: WaveNet, Parallel WaveGAN
  • Seminar: Implementation of WaveNet

• week09 Advanced TTS and Vocoders

  • Lecture: Introduction into generative models. ParallelWaveNet, WaveGlow, WaveFlow, MelGAN, HiFiGAN, VITS

• week10 Voice Conversion

  • Lecture: AutoVC, ConVoice, CycleGAN-VC, StarGAN-VC, Blow, NVC, MOSNet
  • Seminar: Q&A about homework

• week11 Self-supervision in Audio and Speech

  • Lecture:
  • Seminar: Reading Group

Homeworks

• ASR Implementation of ASR model

• KWS Implementation of KWS model

• TTS Implementation of TTS model

• NV Implementation of Neural Vocoder Model

Contributors & course staff

Course materials and teaching performed by

• Alexander Markovich
• Daniil Ivanov