deep-voice-conversion icon indicating copy to clipboard operation
deep-voice-conversion copied to clipboard

How long does it take to run train1 ?

Open JiaYK opened this issue 7 years ago • 11 comments

Hi~ My server has four Intel(R) Xeon(R) CPU E5-2680 v4 and sixteen K80, but I run train1 for 72H. The program epoch 30 and step is 4320. In hparams.py, num_epoch is 1000. My dataset is TIMIT. It is too slow? I hope you can give me an idea what to expect. Thank you very much

JiaYK avatar Jan 30 '18 03:01 JiaYK

I'm using a GTX 1080 with TIMIT and I'm at epoch 30 after only 13 hours, so you probably only have one GPU utilized.

VictoriaBentell avatar Mar 12 '18 00:03 VictoriaBentell

Thanks for your reply. K80 is the same as GTX1060, Maybe I will buy a new GPU.

JiaYK avatar Mar 15 '18 08:03 JiaYK

@VictoriaBentell hi, I'm using a GTX 1080 too. And when i run train1.py with queue, its stuck.Is this normal?

ThomasXu18 avatar Mar 20 '18 12:03 ThomasXu18

Did you reach 70+% accuracy as said in README ?

Techmind avatar May 04 '18 09:05 Techmind

I trained the model in the Google Colab for 6 hours and got 93% accuracy...Here is the model for train1.py code. https://drive.google.com/file/d/1yC3G3V03X3s8mKJ1J6bMkOqDT8r-TBb8/view?usp=sharing Use Tensorboard to view the confusion matrix for this model output. sudo pip install tensorflow cd train1 tensorboard --logdir . open in chrome : localhost:6006

gouravsb17 avatar Nov 19 '18 20:11 gouravsb17

@gouravsb17 How to run in Google Colab ? Thanks!

hungknorton avatar Jan 07 '19 14:01 hungknorton

@hungknorton I changed a few codes. Here are the changed codes. Now u can substitute these codes with the original ones downloaded when u do in the git pull. I have also supplied the google colab page here. In the step where I do git clone, u can change the individual files that I supplied earlier and then execute the code.

If u face any issue then let me know.

gouravsb17 avatar Jan 09 '19 11:01 gouravsb17

Hi gouravsb17, I use your trained model to evaluate timit test dataset, but the acc is only 0.42. Meanwhile, I trained a model by myself with around 0.78 acc and got a 0.73 evaluated acc on timit test dataset. Could you figure out the reason?

MorganCZY avatar Jan 30 '19 06:01 MorganCZY

@gouravsb17 I encountered the following problem when I ran your model: 3333

When I run your code,I encountered the following problem: 444

iamxiaoyubei avatar Feb 18 '19 09:02 iamxiaoyubei

Hi iamxiaoyubei,

Did u try to run the model with the changed codes I supplied. Because the point at which u are getting stuck happens because of a slight error in the code params.

I am attaching the codes here again. Archive.zip

let me know if this works.

gouravsb17 avatar Feb 18 '19 11:02 gouravsb17

@gouravsb17 Thanks for your sharing!

I found out that my voice file(uppercase"WAV") is named differently from your voice file(lowercase "wav"). After modifying the parameters and a line of code in data_load.py, it works now! Thank you very much!

iamxiaoyubei avatar Feb 18 '19 15:02 iamxiaoyubei