coqui_tts_korea

Korean TTS using coqui TTS (glowtts and multiband melgan) - 한국어 TTS

• colab

pretrain with KSS data
finetune HalfLife scientist data

Result

• input text
  • "신은 우리의 수학 문제에는 관심이 없다. 신은 다만 경험적으로 통합할 뿐이다."
• output

https://user-images.githubusercontent.com/46513852/146552491-76df02ca-870d-4900-ab47-e956ede4cb84.mov

https://user-images.githubusercontent.com/46513852/150669352-d1c0aaf8-915b-498e-8441-32e99651fe1a.mov

Train detail

• glowtts

  • trained with kss data 190000 step
  • train ipynb file : coqui_train_glowtts.ipynb
  • google drive link : https://drive.google.com/drive/folders/1quLOabjkAmmw6mFbcCsMqmGxMC4bbbCW

• multiband-melgan

  • trained with korea concat data (KSS, Zeroth and Pansori-TEDxKR) 150000 step
  • train ipynb file : coqui_train_mbmelgan.ipynb
  • google drive link : https://drive.google.com/drive/folders/1FOlcOjx47j_ALNw28rZkr62iOWqHY6tE

• halfLife finetuned glowtts

  • trained with kss data 190000 step + halfLife 90000 step
  • train ipynb file : halfLife_finetune_glowtts.ipynb
  • google drive link : https://drive.google.com/drive/folders/1RubvJSDKZ_hNp3xj8mCocwtWG3KBmT4R?usp=sharing

• halfLife finetuned multiband-melgan

  • trained with korea concat data (KSS, Zeroth and Pansori-TEDxKR) 150000 step + halfLife 20000 step
  • train ipynb file : halfLife_finetune_mbmelgan.ipynb
  • google drive link : https://drive.google.com/drive/folders/15eAW8jTHSIOAisiPQa03VOMOH-pACguc?usp=sharing

Dataset

• KSS Dataset
• Zeroth Korean
• Pansori-TEDxKR
• half_life_dataset

Required Environment to run

!pip install TTS
!pip install jamo
!pip install torchaudio==0.9.0
!pip install gdown
!conda install -c conda-forge kaggle -y
!pip install librosa

Acknowledgement and References

• coqui tts
• TensorFlowTTS
• glow tts
• Multi-band MelGAN
• FastSpeech 2
• speech-japanese-korean-vietnamese
• openslr
• half_life_dataset
• KSS Dataset
• Zeroth Korean
• Pansori-TEDxKR
• Fine-Tuning with a small dataset
• Siri를 아이유 목소리로 바꾸기
• 인공지능 deep voice를 이용한 TTS(음성합성) 구현하기 _ 손석희 앵커
• SCE-TTS: 내 목소리로 TTS 만들기
• huggingface_fastspeech2_kss
• huggingface_TensorFlowTTS
• jamo
• korean.py
• freeconvert