naturalspeech2-pytorch

naturalspeech2-pytorch copied to clipboard

the same Microsoft group has released code for natural speech 3 https://speechresearch.github.io/naturalspeech3/ Could this help?

jjsmcneil1113

two pitch?

3

the first pitch in the sample() as follow: https://github.com/lucidrains/naturalspeech2-pytorch/blob/659bec7f7543e7747e809e950cc2f84242fbeec7/naturalspeech2_pytorch/naturalspeech2_pytorch.py#L1478-L1479 the second pitch in the forward() of Naturalspeech2 as follow: https://github.com/lucidrains/naturalspeech2-pytorch/blob/659bec7f7543e7747e809e950cc2f84242fbeec7/naturalspeech2_pytorch/naturalspeech2_pytorch.py#L1543-L1556 1. Personally, I think the first pitch is from the...

a897456

unconditional version seems not to work correctly

5

Hi, thanks for the great job! I've follow the training process below and use pure audio without conditional input. I've tried dataset with 110000+ audios and also tried dataset with...

ethanyhzhang

 Hi, @lucidrains I found that when step=1000, the system could save the .pt file automatically and output two audio files at the same time, but at this time, a...

a897456

loss.backward()??

3

In Usage: `loss = diffusion(raw_audio)` `loss.backward()` Thank you for your work, very nice! And I'm sorry, as a newbie, I have to ask two stupid questions: 1. Where does this...

a897456

Question about generate speech

5

heya my lad, I have run you Usage code for generate speech, but nothing output. Could you please give a full example to accept a prompt and text, then generating...

text2speechClone

Can you provide the training code for that model? ___ import torch from naturalspeech2_pytorch import Trainer, EncodecWrapper, Model, NaturalSpeech2, SpeechPromptEncoder codec = EncodecWrapper() def main(): model = Model( dim =...

ksl103177

Can you also share the inference code? Thank you.

ksl103177

Please let me know the format of the data to train the model and a detailed guide to train the model. Thank you.

ksl103177

Duration pitch loss not used.

2

Hello, I was looking into your code and it seems like the code does not consider the duration_pitch_loss. https://github.com/lucidrains/naturalspeech2-pytorch/blob/659bec7f7543e7747e809e950cc2f84242fbeec7/naturalspeech2_pytorch/naturalspeech2_pytorch.py#L1522 Maybe, it might be related to the aux_loss you have made....

jaykim9870