Comprehensive-Transformer-TTS icon indicating copy to clipboard operation
Comprehensive-Transformer-TTS copied to clipboard

Gibberish synthesized speech from my own model

Open godspirit00 opened this issue 2 years ago • 0 comments

Hi, I am training a model on the ryanspeech dataset. Currently it is on 125k+ steps, and I tried to synthesize a speech with the checkpoint, but the result is rather hard to understand.

https://user-images.githubusercontent.com/16176151/210700350-e75c108b-1a86-464e-bfb5-e6b55c2dde85.mp4

I tried adding the --duration_control 1.3 to the command, but I got

Traceback (most recent call last):
  File "synthesize.py", line 231, in <module>
    synthesize(device, model, args, configs, vocoder, batchs, control_values)
  File "synthesize.py", line 95, in synthesize
    output = model(
  File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/Comprehensive-Transformer-TTS/model/CompTransTTS.py", line 112, in forward
    ) = self.variance_adaptor(
  File "/root/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/Comprehensive-Transformer-TTS/model/modules.py", line 1088, in forward
    pitch_prediction, pitch_embedding = self.get_pitch_embedding(
  File "/root/Comprehensive-Transformer-TTS/model/modules.py", line 933, in get_pitch_embedding
    f0_denorm = denorm_f0(f0, uv, self.preprocess_config["preprocessing"]["pitch"], pitch_padding=pitch_padding)
  File "/root/Comprehensive-Transformer-TTS/utils/pitch_tools.py", line 79, in denorm_f0
    f0[uv > 0] = 0
IndexError: The shape of the mask [1, 154] at index 1 does not match the shape of the indexed tensor [1, 173] at index 1

My config is

block_type: "transformer_fs2"

duration_modeling:
  learn_alignment: False
  aligner_temperature: 0.0005

prosody_modeling:
  model_type: "liu2021"

What am I missing? Thank you!

godspirit00 avatar Jan 05 '23 04:01 godspirit00