ESPnet Notebook Information

What's new

• 2023.06.01: new course materials used in speech processing course, spring 2023.

ESPnet2

Demo

ASR (Speech recognition)

• asr_realtime_demo.ipynb: ASR realtime inference with various pre-trained models.
• asr_transfer_learning_demo.ipynb: Demo on how to use pre-trained ASR models for fine-tuning.
• streaming_asr_demo.ipynb: Streaming ASR realtime inference with pre-trained models.

SE (Speech enhancement/separation)

• se_demo.ipynb: Speech enhancement/separation inference with various pre-trained models.
• se_demo_for_waspaa_2021.ipynb: WASPAA2021 version of ESPnet-SE demo.

SLU (Spoken language understanding)

• 2pass_slu_demo.ipynb: Two pass spoken language understanding pre-trained model examples.

TTS (Text-to-speech)

• tts_realtime_demo.ipynb: TTS realtime inference with various pre-trained models.

Others (Other utilities)

• onnx_conversion_demo.ipynb: How to convert ESPnet models into ONNX format.

Course

CMU SpeechProcessing Spring2023

• assignment0_data-prep.ipynb: Course assignment on how to prepare ESPnet-format data.
• assignment1_espnet-tutorial.ipynb: A simplified version of previous year's new task tutorial.
• assignemnt3_spk.ipynb: Examples of using ESPnet to extract speaker embeddings and conduct speaker recognition.
• assignment4_ssl.ipynb: Exploration on using self-supervised speech representation to ESPnet ASR training.
• assignment5_st.ipynb: Examples of state-of-the-art speech translation models in ESPnet.
• assignment6_slu.ipynb: Examples of state-of-the-art spoken language understanding models in ESPnet.
• assignment7_se.ipynb: Examples of state-of-the-art speech enhancement/separation in ESPnet.
• assignment8_tts.ipynb: A student version of espnet2-tts realtime demonstration.
• s2st_demo.ipynb: An example of existing speech-to-speech translation model for ESPnet.

CMU SpeechRecognition Fall2022

• recipe_tutorial.ipynb: A general tutorial of stage-by-stage explanation of ESPnet2 recipes (with new functions).
• new_task_tutorial.ipynb: A tutorial on how to add new models/tasks to ESPnet framework.

CMU SpeecRecognition Fall2021

• general_tutorial.ipynb: A general tutorial of stage-by-stage explanation of ESPnet2 recipes.

ESPnet1

• asr_library.ipynb: Speech recognition library explanation with network training.
• asr_recipe.ipynb: Speech recognition recipe explanation.
• pretrained.ipynb: Tutorial on how to use pre-trained models.
• st_demo.ipynb: Speech translation demonstration with a TTS model to achieve speech-to-speech translation.
• tts_realtime_demo.ipynb: TTS demonstration with different pre-trained TTS models.
• tts_recipe.ipynb: Stage explanation for TTS recipes.