tacotron
tacotron copied to clipboard
Kyubyong
Empty generated waves
Hi, Thank you so much for your great work!
To save training time, I took the wave files under Genesis folder, and updated the text.csv file accordingly. Train was done for these 1532 files successfully as shown below:
python train.py Training Graph loaded 2017-06-08 10:35:51.200597: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-06-08 10:35:51.200628: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-06-08 10:35:51.200633: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. Done
Then, I ran the evaluation script which generated a text file with 31 sentences and their waves. python eval.py Graph loaded 2017-06-08 13:22:30.452388: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-06-08 13:22:30.452540: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-06-08 13:22:30.452657: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. WARNING:tensorflow:Standard services need a 'logdir' passed to the SessionManager Restored! Done
The problem is that all the generated wave files are almost empty with some noise in the middle until the end.
This is one of the these generated files: (model_epoch_200_gs_1000_0.zip)
Any idea how to fix this issue? Many thanks, Hamdy
Thanks for you fast reply! Could you please tell me where I can find this file?
From: Kyubyong Park [mailto:[email protected]] Sent: Thursday, June 8, 2017 2:12 PM To: Kyubyong/tacotron [email protected] Cc: Hamdy S. Mubarak [email protected]; Author [email protected] Subject: Re: [Kyubyong/tacotron] Empty generated waves (#31)
Can you share your training loss graph?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Kyubyong/tacotron/issues/31#issuecomment-307073167, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGF-N7ivnmpP44SJsBEvu1E2HkNos9i8ks5sB9b5gaJpZM4Nz6cp.
CONFIDENTIALITY NOTICE: This email and any attachments transmitted with it are confidential and intended for the use of individual or entity to which it is addressed. If you have received this email in error, please delete it immediately and inform the sender. Unless you are the intended recipient, you may not use, disclose, copy or distribute this email or any attachments included. The contents of this email, including any attachments, may be subjected to copyright law. In such cases, the contents may not be copied, adapted, distributed or transmitted without the consent of the copyright owner.
From the TensorBoard. I guess 200 epochs, or 1000 global steps are far from enough. Note that the paper said they trained for 2 million global steps. And I think that's one of the biggest reasons why people including me have had difficulties getting a good result. (The other reason is, of course, some possible bugs in the implementation. But it's also true that mostly we don't have enough gpus or time or patience!) But the training curve keeps slowly decreasing, I think that's a good sign.
@hmubarak I have the same problem here with yours.
All of the generated voice is like this:
Loss
@candlewill As said before, we need to be more patient with the training. 800 global steps is very unlikely to yield good results.
I faced the same problem. Did you solve this problem by keeping training?(i am now over 26k steps)
@candlewill @Kyubyong @hmubarak How did you solve this issue? I trained for almost 50k steps. But still when I synthesize using synthesize.py, it is generating empty wavs. Can you please solve this issue?
