Attention doesn't work well for downsample_step=1 and outputs_per_step=1

[r9y9]

Noticed while working on https://github.com/r9y9/deepvoice3_pytorch/pull/21.

Trained 300k steps, but the model was not generalized well. Need to figure out how we can improve.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[vikrantsharma7]

Hello @r9y9 Any update on this? downsample_step=1 fails to learn, but works fine with 4.

[r9y9]

@vikrantsharma7 I'm sorry but I don't have time to work on that recently.

Meanwhile, you might want to have a look at https://github.com/espnet/espnet, which provides Tacotron 2 and Transformer-based TTS. And it works well with downsample_step=1 (i.e. no reduction factor). Espnet is initially designed for end-to-end ASR and is not that simple compared to my repo, but it looks promising and people in the community are very kind (and I am part of it).