deep-voice-conversion icon indicating copy to clipboard operation
deep-voice-conversion copied to clipboard

Question on train1.py's and train2.py's runtimes.

Open rolanvc opened this issue 7 years ago • 8 comments

Hi,

I'm trying this out, but I've only a gtx 960 gpu. I was wondering what kinds of training run times do you have for these 2? Will it take me weeks, days, hours? When I started python train1.py, A few lines of logging came out, but seems to have frozen already, with no sign of life nor error.

I hope you can give me an idea what to expect.

Thank you very much, =)

rolanvc avatar Dec 05 '17 09:12 rolanvc

I actually don't know how it compares to the GTX 960, but I was using a p2.xlarge (which is half of an nVidia K80 Tesla) on AWS, and train1 took about 72 hours to converge on 70% accuracy.

jswilson avatar Jan 09 '18 18:01 jswilson

@jswilson would you mind sharing how long train2 took you, if you tried it on the arctic dataset?

VictoriaBentell avatar Mar 13 '18 21:03 VictoriaBentell

I've been running Train2 for over a day now and my results are more erratic than a mountain goat on LSD, so I'm a tad concerned that it'll never converge.

VictoriaBentell avatar Mar 15 '18 05:03 VictoriaBentell

@VictoriaBentell any news? did it converged?

0i0 avatar Mar 29 '18 23:03 0i0

No, I assume that it's similar to this issue. It seems like the arctic dataset itself might be the problem, and so I'm going to try and find another dataset to replace that.

VictoriaBentell avatar Apr 01 '18 18:04 VictoriaBentell

@VictoriaBentell May I ask when you were using train2, did you use your own voice corpus? If so, how did you do that?

Super1ZC avatar Apr 20 '18 06:04 Super1ZC

Originally I was using the arctic dataset, but I haven't done anything with it since my last response, so I'll try making my own corpus and get back to you on the results. As far as I'm aware, it should be as straightforward as replacing each wav file in bdl and slt with any source and target respectively. As long as the source and target are saying the same things for 3 or 4 seconds each, the result should be fine... I hope.

VictoriaBentell avatar Apr 21 '18 01:04 VictoriaBentell

@VictoriaBentell May I ask if your model converges now? Did it work out using arctic dataset? If you have train2 result, could you please share it with me for a try?

iamxiaoyubei avatar Feb 21 '19 09:02 iamxiaoyubei