WaveRNN
WaveRNN copied to clipboard
Can't train WaveRNN with voc_mode = 'RAW'
I had successfully trained this model with voc_mode = 'MOL' but its synthesized speed is not good.
After that, I changed voc_mode from MOL to RAW but it has a bug:
/pytorch/aten/src/THCUNN/SpatialClassNLLCriterion.cu:103: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T *, T *, T *, long *, T *, int, int, int, int, int, long) [with T = float, AccumT = float]: block: [11,0,0], thread: [671,0,0] Assertion t >= 0 && t < n_classes
failed.
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCCachingHostAllocator.cpp line=296 error=59 : device-side assert triggered
Traceback (most recent call last):
File "train_wavernn.py", line 162, in
Most likely cause of this error is data format mismatch. You need to reprocess raw data with preprocess.py again for RAW.
@tuong-olli May i ask the loss and steps you trained on wavernn ?
@tsungruihon The loss did not change in step 596k: | Epoch: 563/2276 (820/820) | Loss: 5.6460 | 0.7 steps/s | Step: 596k | I changed the learning rate to continue training. But it only can down to 5.4514. | Epoch: 571/1474 (820/820) | Loss: 5.4514 | 1.0 steps/s | Step: 1260k | And the synthesized audios had some noise.
@tuong-olli same as mine. If you have some free time, please visit this issue @ #177
After you try to change RAW, is LOSS better than before? look forward to your reply.
After you try to change RAW, is LOSS better than before? look forward to your reply.
What code did you change to use the RAW format? tks