SNAC : Speaker-normalized Affine Coupling Layer in Flow-based Architecture for Zero-Shot Multi-Speaker Text-to-Speech

This is the unofficial pytorch implementation of SNAC We built up our codes based on VITS

0. VCTK dataset is used.
1. LibriTTS dataset (train-clean-100 and train-clean-360) is also supported.
2. This is the implementation of Proposed + REF + FLOW in the paper.
3. Major modifications are applied in modules.py (SN/SDN transformation) and losses.py (Revised log determinant)
4. We followed the same flow setting with VITS, using volume-preserving transformation with the Jacobian determinant of one.

Prerequisites

0. Clone this repository.
1. Install python requirements. Please refer requirements.txt
   1. You may need to install espeak first: apt-get install espeak
2. Download datasets
   1. Download and extract the VCTK dataset, and downsample wav files to 22050 Hz. Then rename or create a link to the dataset folder: ln -s /path/to/VCTK-Corpus/downsampled_wavs DUMMY3
   2. For LibriTTS dataset, downsample wav files to 22050 Hz and link to the dataset folder: ln -s /path/to/LibriTTS DUMMY2
3. Build Monotonic Alignment Search and run preprocessing if you use your own datasets.

# Cython-version Monotonoic Alignment Search
cd monotonic_align
python setup.py build_ext --inplace

Training Exmaple

python train.py -c configs/vctk_base.json -m vctk_base

Inference Example

See inference.ipynb