New transformer architecture

[aletote]

I see that, like GANs in their time, a lot of new transformer architectures came out. Is there a better one that we could use now than gpt-2 to generate for example, in my case, ABC music?

[NeonWizard]

This project's name is "gpt-2-simple". It's a simplification wrapper around the GPT2 model. Changing to a different architecture or model is out of scope for this project.