Quality after 20 epoch training

[thanhnew2001]

Hello, I tried your script and the resulting model took about 10 hours to train on single 3060 but the quality is still not very good. How could I improve it?

[thanhnew2001]

[FredyRivera-dev]

Increase the batch size to 16 or 32, the epochs maybe to 25 or 30, increase the sq_len to 512 or 1024, the d_model to 768 or 1024. And make other options that you can modify in the config.py to improve the model. I hope I have helped you :b